fix: fix issues for rare cases across deadlocking, caching, serialization, GC, and Python frontend

CHANGELOG.md

@@ -22,6 +22,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 
 - Define `Py_GIL_DISABLED` for free-threaded debug builds on Windows when building the C extension to work around an upstream CMake `FindPython` bug by [@XuehaiPan](https://github.com/XuehaiPan) in [#285](https://github.com/metaopt/optree/pull/285).
+- Fix a deadlock when registering or unregistering a pytree node concurrently with flattening, caused by releasing the GIL while holding the registry lock by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix a deadlock while flattening on free-threading builds, caused by acquiring the non-recursive dictionary insertion order lock twice by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix a spurious `SystemError` from `optree._C.get_registry_size()` when a concurrent registration slipped between its two reads by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix the type caches handing an interpreter a value owned by another one, by keying them on the interpreter in addition to the type address by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix the type caches retaining entries for a finalized interpreter, which leaked immortal keys and could be inherited by a fresh interpreter reusing the same ID by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix a deadlock in the `PyStructSequence` field cache, caused by re-acquiring the GIL while still holding the cache lock by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `optree.structseq_fields()` and the struct sequence accessors reporting the trailing hidden field names for unnamed sequence slots (e.g. `os.stat_result` slots 7-9 reported as `st_atime` / `st_mtime` / `st_ctime`), by mapping `tp_members` by offset instead of by position by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix the treespec repr printing the raw unnamed-field marker as if it were a keyword argument, now rendered as `<unnamed@N>` by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `PyTreeSpec.__setstate__()` accepting a malformed state, which could read out of bounds or abort the interpreter when the treespec was later used by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `PyTreeSpec.__getstate__()` returning the treespec's internal mutable containers, so mutating the pickled state corrupted an otherwise immutable treespec by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix pickling a `PyTreeSpec` with protocol 0 or 1 aborting the interpreter, by reducing through `copyreg.__newobj__` by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `PyTreeSpec.compose()`, `PyTreeSpec.broadcast_to_common_suffix()`, `treespec_transform()`, and `treespec_from_collection()` silently rebinding a custom node to a different registration when an empty namespace adopted a non-empty one by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `PyTreeSpec.broadcast_to_common_suffix()` sorting the argument treespec's dictionary keys in place while building its key-mismatch error message, corrupting a treespec the caller still holds by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `PyTreeSpec.broadcast_to_common_suffix()` dropping a custom node's entries and falling back to positional ones by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix the dictionary key order being lost when a treespec built under the global namespace is promoted to an insertion-ordered namespace by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `treespec_from_collection()` on a leaf reporting an escalated `UserWarning` as a confusing `SystemError`, by checking the return value of `PyErr_WarnEx()` by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `treespec_is_prefix()` and `treespec_is_suffix()` comparing against a stale subtree when a dictionary node's keys had been reordered by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix the tree iterator never reporting or clearing its `is_leaf` predicate to the garbage collector, leaking any reference cycle that passes through it by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix a reference cycle passing through a registered custom type not being collectable once the registry no longer holds the registration by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix a deeply nested treespec overflowing the native stack and crashing in `treespec_paths()`, `treespec_accessors()`, and `PyTreeSpec.broadcast_to_common_suffix()` instead of raising `RecursionError` by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `DataclassEntry` and `AttrsEntry` resolving an integer entry against every field rather than only the tree children, which returned the wrong attribute for a class holding a metadata or non-`init` field by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `optree.dataclasses.register_node()` and `optree.integrations.attrs.register_node()` marking a class as registered before the registration succeeded, leaving the class permanently unusable after a failure by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `optree.dataclasses.register_node()` silently dropping `InitVar` pseudo-fields, which are neither children nor metadata and cannot round-trip, now rejected with a pointer to the generic API by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
+- Fix `tree_broadcast_common()` and `broadcast_common()` applying the caller's `is_leaf` predicate to an internal sentinel tree, which could raise from the predicate or collapse a filled subtree and under-replicate by [@XuehaiPan](https://github.com/XuehaiPan) in [#290](https://github.com/metaopt/optree/pull/290).
 
 ### Removed
 

docs/source/spelling_wordlist.txt

@@ -2,6 +2,7 @@ abc
 ABCMeta
 accessor
 accessors
+antisymmetric
 arg
 args
 arity
@@ -70,7 +71,9 @@ optree
 OrderedDict
 param
 params
+picklable
 pragma
+preorder
 py
 pypy
 pytree

include/optree/hashing.h

@@ -19,7 +19,6 @@ limitations under the License.
 
 #include <cstddef>     // std::size_t
 #include <functional>  // std::hash, std::{not_,}equal_to
-#include <string>      // std::string
 #include <utility>     // std::pair
 
 #include <pybind11/pybind11.h>
@@ -89,6 +88,26 @@ struct std::not_equal_to<std::pair<std::string, py::handle>> {
                std::not_equal_to<py::handle>{}(lhs.second, rhs.second);
     }
 };
+template <>
+struct std::equal_to<std::pair<interpid_t, py::handle>> {
+    using is_transparent = void;
+    inline constexpr Py_ALWAYS_INLINE bool operator()(const std::pair<interpid_t, py::handle> &lhs,
+                                                      const std::pair<interpid_t, py::handle> &rhs)
+        const noexcept(noexcept(std::equal_to<interpid_t>{}(lhs.first, rhs.first))) {
+        return std::equal_to<interpid_t>{}(lhs.first, rhs.first) &&
+               std::equal_to<py::handle>{}(lhs.second, rhs.second);
+    }
+};
+template <>
+struct std::not_equal_to<std::pair<interpid_t, py::handle>> {
+    using is_transparent = void;
+    inline constexpr Py_ALWAYS_INLINE bool operator()(const std::pair<interpid_t, py::handle> &lhs,
+                                                      const std::pair<interpid_t, py::handle> &rhs)
+        const noexcept(noexcept(std::not_equal_to<interpid_t>{}(lhs.first, rhs.first))) {
+        return std::not_equal_to<interpid_t>{}(lhs.first, rhs.first) ||
+               std::not_equal_to<py::handle>{}(lhs.second, rhs.second);
+    }
+};
 template <class T, class U>
 struct std::hash<std::pair<T, U>> {
     using is_transparent = void;

include/optree/pymacros.h

@@ -71,6 +71,19 @@ inline constexpr Py_ALWAYS_INLINE bool Py_IsConstant(PyObject *x) noexcept {
 }
 #define Py_IsConstant(x) Py_IsConstant(x)
 
+// `PyStructSequence_UnnamedField` is declared `extern` with hidden visibility (so it is not an
+// exported dynamic symbol for extension modules) before Python 3.11.0a2, where it became
+// `PyAPI_DATA`. Referencing it directly leaves an undefined symbol that makes the module fail to
+// import on those versions. Its value is the stable marker "unnamed field", and callers only ever
+// use it by value (never by pointer identity), so fall back to that literal there.
+inline const char *PyStructSequenceUnnamedField() noexcept {
+#if PY_VERSION_HEX >= 0x030B00A2  // Python 3.11.0a2
+    return PyStructSequence_UnnamedField;
+#else
+    return "unnamed field";
+#endif
+}
+
 using interpid_t = decltype(PyInterpreterState_GetID(nullptr));
 
 #if defined(PYBIND11_HAS_SUBINTERPRETER_SUPPORT) &&                                                \