Merge branch 'main' into develop

Author: mhucka

.flake8

@@ -36,3 +36,5 @@ ignore =
     # line breaks *should* come before a binary operator, but as of version 4,
     # Flake8 still flags the breaks as bad. So:
     W503
+    # For performance reasons, we use if __debug__: log(...) all over the place.
+    E701

.gitignore

@@ -63,6 +63,7 @@ __pycache__
 .cache
 .pytest_cache
 .coverage
+.eggs
 
 # Project-specific things to ignore:
 # .............................................................................
@@ -71,3 +72,4 @@ build
 dist
 commonpy.egg-info
 *.bak
+*.tmp

CHANGES.md

@@ -1,10 +1,32 @@
 # Change log for CommonPy
 
+## Version 1.12.0
+
+Additions in this release:
+* New function `network` in the `network_utils` module. It is a companion to `net` and takes the same arguments, but returns only one value (the response). If an error occurs, it raises the error as an exception. This makes it possible for callers to use `network(...)` in somewhat more Pythonic style than `net(...)`,  by wrapping the call to `network(...)` in `try`-`except`.
+
+Changes in this release:
+* Removed `slice` from `data_utils` module because it shadows a Python built-in.
+* Fixed `hostname`  in `network_utils` to be more general and not hardwire a test for `http`.
+* Fixed a bunch of `flake8` warnings.
+
+
+## Version 1.11.0
+
+Additions in this release:
+* New class `CaseFoldSet`, similar to `CaseFoldDict` but … a set.
+
+Changes in this release:
+* Fixed a bug in the class documentation in the `README.md` file.
+* Added missing dependency for [twine]() in requirements-dev
+* Now using lazy `import`s in more places, for faster load times.
+
+
 ## Version 1.10.0
 
 Changes in this release:
 
-* `data_utils.flattened` now would outputs `[]` as the value of dict or mapping keys whose original values are an empty sequence (e.g., when the value of a dict key is `[]`). Previously, it would output `None` as the value, which was an unexpected transformation of the input.
+* `data_utils.flattened` now outputs `[]` as the value of dict or mapping keys whose original values are an empty sequence (e.g., when the value of a dict key is `[]`). Previously, it would output `None` as the value, which was an unexpected transformation of the input.
 
 Bug fixes in this release:
 

CITATION.cff

@@ -10,8 +10,8 @@ cff-version: "1.1.0"
 message: "If you use this software, please cite it using these metadata."
 repository-code: "https://github.com/caltechlibrary/commonpy"
 title: "CommonPy: assortment of Python helper functions and utility classes"
-date-released: 2023-01-23
-version: "1.10.0"
+date-released: 2023-03-24
+version: "1.12.0"
 doi: 10.22002/20217
 keywords: 
   - Python

README.md

@@ -62,9 +62,10 @@ The following subsections describe the different modules available.
 
 The `data_structures` module provides miscellaneous data classes.
 
-| Function              | Purpose |
-|-----------------------|---------|
-| `CaseInsensitiveDict` | A version of `dict` that compares keys in a case-insensitive manner |
+| Class          | Purpose |
+|----------------|---------|
+| `CaseFoldDict` | A version of `dict` that compares keys in a case-insensitive manner |
+| `CaseFoldSet`  | A version of `set` that compares keys in a case-insensitive manner |
 
 
 ### Data utilities
@@ -73,14 +74,14 @@ The `data_utils` module provides a number of miscellaneous simple functions for
 
 | Function           | Purpose |
 |--------------------|---------|
+| `expanded_range(string)` | Given a string of the form "X-Y", returns the list of integers it represents |
 | `flattened(thing)` | Takes a list or dictionary and returns a recursively flattened version |
-| `unique(list)`     | Takes a list and return a version without duplicates |
 | `ordinal(integer)` | Returns a string with the number followed by "st", "nd, "rd", or "th" |
-| `slice(list, n)`   | Yields `n` number of slices from the `list` |
-| `timestamp()`      | Returns a string for an easily-readable form of the current time and date |
 | `parsed_datetime(string)` | Returns a date object representing the given date string |
 | `pluralized(word, n, include_num)`  | Returns a plural version of `word` if `n > 1` |
-| `expanded_range(string)` | Given a string of the form "X-Y", returns the list of integers it represents |
+| `sliced(list, n)`  | Yields `n` number of slices from the `list` |
+| `timestamp()`      | Returns a string for an easily-readable form of the current time and date |
+| `unique(list)`     | Takes a list and return a version without duplicates |
 
 
 ### File utilities
@@ -89,20 +90,20 @@ The `file_utils` module provides a number of miscellaneous simple functions for
 
 | Function           | Purpose |
 |--------------------|---------|
-| `readable(dest)`   | Returns `True` if file or directory `dest` is accessible and readable |
-| `writable(dest)`   | Returns `True` if file or directory `dest` can be written |
-| `nonempty(file)`   | Returns `True` if file `file` is not empty |
-| `relative(file)`   | Returns a path string for `file` relative to the current directory |
-| `filename_basename(file)` | Returns `file` without any extensions |
-| `filename_extension(file)` | Returns the extension of filename `file` |
 | `alt_extension(file, ext)` | Returns `file` with the extension replaced by `ext` |
-| `rename_existing(file)` | Renames `file` to `file.bak` |
-| `delete_existing(file)` | Deletes the given `file` |
 | `copy_file(src, dst)` | Copies file from `src` to `dst` |
+| `delete_existing(file)` | Deletes the given `file` |
+| `filename_basename(file)` | Returns `file` without any extensions |
+| `filename_extension(file)` | Returns the extension of filename `file` |
+| `files_in_directory(dir, ext, recursive)` | |
+| `filtered_by_extensions(list, endings)` | |
+| `nonempty(file)`   | Returns `True` if file `file` is not empty |
 | `open_file(file)` | Opens the `file` by calling the equivalent of "open" on this system |
 | `open_url(url)` | Opens the `url` in the user's default web browser |
-| `filtered_by_extensions(list, endings)` | |
-| `files_in_directory(dir, ext, recursive)` | |
+| `readable(dest)`   | Returns `True` if file or directory `dest` is accessible and readable |
+| `relative(file)`   | Returns a path string for `file` relative to the current directory |
+| `rename_existing(file)` | Renames `file` to `file.bak` |
+| `writable(dest)`   | Returns `True` if file or directory `dest` can be written |
 
 
 ### Interruptible wait and interruption handling utilities
@@ -112,11 +113,11 @@ The `interrupt` module includes `wait(...)`, a replacement for `sleep(...)` that
 | Function                 | Purpose |
 |--------------------------|---------|
 | `config_interrupt(callback, raise_ex, signal)` | Sets up a callback function |
-| `wait(duration)`         | Waits for `duration` in an interruptible fashion |
 | `interrupt()`            | Interrupts any `wait` in progress |
 | `interrupted() `         | Returns `True` if an interruption has been called |
 | `raise_for_interrupts()` | Raises an exception if `interrupt()` has been invoked |
 | `reset_interrupts()`     | Resets the interruption flag |
+| `wait(duration)`         | Waits for `duration` in an interruptible fashion |
 
 
 ### Module utilities
@@ -125,11 +126,11 @@ The `module_utils` collection of functions is useful for working with paths rela
 
 | Function           | Purpose |
 |--------------------|---------|
+| `config_path(module_name)` | Returns the path to local config data directory for the module |
+| `datadir_path(module_name)` | Returns the path to the `/data` subdirectory of the module |
 | `desktop_path()`   | Returns the path to the user's Desktop directory on this system |
-| `module_path(module_name)` | Returns the path to the installed module |
 | `installation_path(module_name)` | Returns the path to module's installation directory |
-| `datadir_path(module_name)` | Returns the path to the `/data` subdirectory of the module |
-| `config_path(module_name)` | Returns the path to local config data directory for the module |
+| `module_path(module_name)` | Returns the path to the installed module |
 
 Function `config_path(...)` is useful to use in conjunction with Python's [`configparser`](https://docs.python.org/3/library/configparser.html) module.  It returns `~/.config/modulename/` on Unix-like systems.
 
@@ -146,8 +147,8 @@ The `network_utils` module provides several functions that are useful when perfo
 | `net(...)`                       | See below                                                           |
 | `netlock(url)`                   | Returns the hostname, port number (if any), and login info (if any) |
 | `network_available()`            | Returns `True` if external hosts are reacheable over the network    |
-| `scheme(url)`                    | Returns the protocol portion of the url; e.g., "https"              |
 | `on_localhost(url)`              | Returns `True` if the address of `url` points to the local host     |
+| `scheme(url)`                    | Returns the protocol portion of the url; e.g., "https"              |
 
 
 #### _`net`_
@@ -259,6 +260,7 @@ CommonPy makes use of numerous open-source packages, without which it would have
 * [pywin32](https://github.com/mhammond/pywin32) – Windows APIs for Python
 * [sidetrack](https://github.com/caltechlibrary/sidetrack) – simple debug logging/tracing package
 * [tldextract](https://github.com/john-kurkowski/tldextract) – module to parse domains from URLs
+* [twine](https://twine.readthedocs.io) – package for publishing Python packages to PyPI
 * [validator-collection](https://pypi.org/project/validator-collection/) – collection of Python functions for validating data
 
 <div align="center">

codemeta.json

@@ -6,7 +6,7 @@
   "codeRepository": "https://github.com/caltechlibrary/commonpy",
   "issueTracker": "https://github.com/caltechlibrary/commonpy/issues",
   "license": "https://github.com/caltechlibrary/commonpy/blob/master/LICENSE",
-  "version": "1.10.0",
+  "version": "1.12.0",
   "author": [
     {
       "@type": "Person",

commonpy/__init__.py

@@ -21,7 +21,7 @@
 #  |    by the Makefile. Manual changes to these values will be lost.      |
 #  ╰────────────────────── Notice ── Notice ── Notice ─────────────────────╯
 
-__version__     = '1.10.0'
+__version__     = '1.12.0'
 __description__ = 'Assortment of Python helper functions and utility classes'
 __url__         = 'https://github.com/caltechlibrary/commonpy'
 __author__      = 'Michael Hucka'

commonpy/data_structures.py

@@ -15,6 +15,8 @@
 '''
 
 import collections
+from   collections.abc import MutableSet
+from   contextlib import suppress
 
 
 # Classes.
@@ -26,7 +28,10 @@
 # (version 3.0.0).
 
 class CaseFoldDict(collections.OrderedDict):
-    '''A subclass of OrderedDict that compares keys in a case-fold manner.'''
+    '''A subclass of OrderedDict that compares keys in a case-fold manner.
+
+    The case of stored values is preserved.
+    '''
 
     class Key(str):
         def __init__(self, key):
@@ -67,3 +72,60 @@ def __eq__(self, other):
         else:
             return NotImplemented
         return dict(self.items()) == dict(other.items())
+
+
+# The following code is based in part on a posting by user Martijn Pieters on
+# 2020-04-07 to Stack Overflow at https://stackoverflow.com/a/27531275/743730
+
+class CaseFoldSet(MutableSet):
+    '''A subclass of MutableSet tests for containment in a case-fold manner.
+
+    The case of values is preserved. Note that this class preserves the case
+    of the last-inserted variant; i.e., if the same value is added multiple
+    times, but with different cases, the value remembered is the last one.
+
+    Example: the following test will be True:
+    > 'Foo' in CaseFoldSet(['FOO'])
+    '''
+    def __init__(self, *args):
+        self._values = {}
+        if len(args) > 1:
+            raise TypeError(
+                f"{type(self).__name__} expected at most 1 argument, "
+                f"got {len(args)}"
+            )
+        values = args[0] if args else ()
+        self._fold = str.casefold
+        for v in values:
+            self.add(v)
+
+
+    def __repr__(self):
+        return '<{}{} at {:x}>'.format(
+            type(self).__name__, tuple(self._values.values()), id(self))
+
+
+    def __contains__(self, value):
+        return self._fold(value) in self._values
+
+
+    def __iter__(self):
+        return iter(self._values.values())
+
+
+    def __len__(self):
+        return len(self._values)
+
+
+    def add(self, value):
+        self._values[self._fold(value)] = value
+
+
+    def discard(self, value):
+        with suppress(KeyError):
+            del self._values[self._fold(value)]
+
+
+    def update(self, values):
+        for v in values:
+            self.add(v)

commonpy/data_utils.py

@@ -14,10 +14,8 @@
 file "LICENSE" for more information.
 '''
 
-from   boltons.iterutils import flatten_iter
 from   boltons.strutils import pluralize
 from   collections.abc import MutableMapping
-import datetime
 from   datetime import datetime as dt
 from   dateutil import tz
 from   typing import Sequence, Generator, Iterator, KeysView, ValuesView
@@ -34,15 +32,15 @@
 # Functions.
 # .............................................................................
 
-def slice(lst, n):
+def sliced(lst, n):
     '''Yield n number of slices from lst.'''
     # Original algorithm from Jurgen Strydom posted 2019-02-21 Stack Overflow
     # https://stackoverflow.com/a/54802737/743730
-    for i in range(0, n):
+    for i in range(n):
         yield lst[i::n]
 
 
-def flattened(original, parent_key = False, separator = '.'):
+def flattened(original, parent_key=False, separator='.'):
     '''Return a recursively flattened version of a nested list or dictionary.
 
     :param original: The original to flatten (a dict or a list)
@@ -63,7 +61,7 @@ def flattened(original, parent_key = False, separator = '.'):
                 else:
                     items.extend(flattened(value, new_key, separator).items())
             elif isinstance(value, Sequence) and not isinstance(value, (str, bytes)):
-                if len(value):
+                if len(value) > 0:
                     for k, v in enumerate(value):
                         items.extend(flattened({str(k): v}, new_key, separator).items())
                 else:
@@ -80,11 +78,11 @@ def flattened(original, parent_key = False, separator = '.'):
             if isinstance(el, (str, bytes)):
                 result.append(el)
             elif isinstance(el, Sequence):
-                result.extend(flattened(el, separator = separator))
+                result.extend(flattened(el, separator=separator))
             elif isinstance(el, (KeysView, ValuesView)):
                 result.extend(el)
             else:
-                result.append(flattened(el, separator = separator))
+                result.append(flattened(el, separator=separator))
         return result
 
     # Fallback if we don't know how to deal with this kind of thing.
@@ -102,7 +100,7 @@ def ordinal(n):
     '''Print a number followed by "st" or "nd" or "rd", as appropriate.'''
     # Spectacular algorithm by user "Gareth" at this posting:
     # http://codegolf.stackexchange.com/a/4712
-    return '{}{}'.format(n, 'tsnrhtdd'[(n/10%10!=1)*(n%10<4)*n%10::4])
+    return '{}{}'.format(n, 'tsnrhtdd'[(n/10 % 10 != 1)*(n % 10 < 4)*n % 10::4])
 
 
 def expanded_range(text):
@@ -116,15 +114,15 @@ def expanded_range(text):
         # Malformed cases of x-, where 2nd number missing.  Can't handle this.
         if not range_list[1].isdigit():
             raise ValueError(f'Malformed range expression: "{text}"')
-        range_list.sort(key = int)
+        range_list.sort(key=int)
         return [*map(str, range(int(range_list[0]), int(range_list[1]) + 1))]
     else:
         return text
 
 
 def timestamp():
     '''Return a string describing the date and time right now.'''
-    return dt.now(tz = tz.tzlocal()).strftime(DATE_FORMAT)
+    return dt.now(tz=tz.tzlocal()).strftime(DATE_FORMAT)
 
 
 def parsed_datetime(string):
@@ -133,10 +131,10 @@ def parsed_datetime(string):
     # Dateparser imports regex, a large package that takes a long time to load.
     # Delay loading it so that application startup times can be faster.
     import dateparser
-    return dateparser.parse(string, settings = {'RETURN_AS_TIMEZONE_AWARE': True})
+    return dateparser.parse(string, settings={'RETURN_AS_TIMEZONE_AWARE': True})
 
 
-def pluralized(word, count, include_number = False):
+def pluralized(word, count, include_number=False):
     '''Pluralize the "word" if "count" is > 1 or has length > 1.
 
     If "include_number" is true, then the value of "count" will be prepended