Make collection iterators forward_iterator* and beyond

doc/collections_as_container.md

@@ -88,11 +88,17 @@ In the following tables a convention from `Collection` is used: `iterator` stand
 | `std::input_or_output_iterator` | ✔️ yes | ✔️ yes |
 | `std::input_iterator` | ✔️ yes | ✔️ yes |
 | `std::output_iterator` | ❌ no | ❌ no |
-| `std::forward_iterator` | ❌ no | ❌ no |
-| `std::bidirectional_iterator` | ❌ no | ❌ no |
-| `std::random_access_iterator` | ❌ no | ❌ no |
+| `std::forward_iterator` | ✔️ yes (see note below) | ✔️ yes (see note below) |
+| `std::bidirectional_iterator` | ✔️ yes (see note below) | ✔️ yes (see note below) |
+| `std::random_access_iterator` | ✔️ yes (see note below) | ✔️ yes (see note below) |
 | `std::contiguous_iterator` | ❌ no | ❌ no |
 
+:::{note}
+The collections' iterators fulfil the `std::forward_iterator` except that the pointers obtained with `->` remain valid only as long as the iterator itself is valid instead of as long as the range remains valid. In practice, this means a `ptr` obtained with `auto* ptr = it.operator->();` is valid only as long as `it` is valid.
+The values obtained immediately through `->` (for instance, `auto& e = it->energy();`) behave as expected for `std::forward_iterator`, as their validity is tied to the collection's validity.
+Despite the exception, the collection iterators are made to still report themselves as fulfilling the forward iterator requirements since the problematic usage (`auto* ptr = it.operator->();`) is rare in normal scenarios.
+:::
+
 ### LegacyIterator
 
 | Requirement | Fulfilled by `iterator`/`const_iterator`? | Comment |
@@ -162,7 +168,7 @@ In addition to the *LegacyForwardIterator* the C++ standard specifies also the *
 
 | Adaptor | Compatible with Collection? | Comment |
 |---------|-----------------------------|---------|
-| `std::reverse_iterator` | ❌ no | `iterator` and `const_iterator` not *LegacyBidirectionalIterator* or `std::bidirectional_iterator` |
+| `std::reverse_iterator` | ❗ attention | `operator->` not defined as `iterator`'s and `const_iterator`'s `operator->` are non-`const` |
 | `std::back_insert_iterator` | ❗ attention | Compatible only with SubsetCollections, otherwise throws `std::invalid_argument` |
 | `std::front_insert_iterator` | ❌ no | `push_front` not defined |
 | `std::insert_iterator` | ❌ no | `insert` not defined |
@@ -180,9 +186,9 @@ In addition to the *LegacyForwardIterator* the C++ standard specifies also the *
 | `std::ranges::sized_range` | ✔️ yes |
 | `std::ranges::input_range` | ✔️ yes |
 | `std::ranges::output_range` | ❌ no |
-| `std::ranges::forward_range` | ❌ no |
-| `std::ranges::bidirectional_range` | ❌ no |
-| `std::ranges::random_access_range` | ❌ no |
+| `std::ranges::forward_range` | ✔️ yes |
+| `std::ranges::bidirectional_range` | ✔️ yes |
+| `std::ranges::random_access_range` | ✔️ yes |
 | `std::ranges::contiguous_range` | ❌ no |
 | `std::ranges::common_range` | ✔️ yes |
 | `std::ranges::viewable_range` | ✔️ yes |
@@ -203,20 +209,21 @@ std::fill(std::begin(collection), std::end(collection), value ); // requires For
 std::sort(std::begin(collection), std::end(collection)); // requires RandomAccessIterator and Swappable -> might compile, wrong result
 ```
 
-## Standard range algorithms
+## Collection and standard range algorithms
 
 The arguments of standard range algorithms are checked at compile time and must fulfil certain iterator concepts, such as `std::input_iterator` or `std::ranges::input_range`.
 
-The iterators of PODIO collections model the `std::input_iterator` concept, so range algorithms that require this iterator type will work correctly with PODIO iterators. If an algorithm compiles, it is guaranteed to work as expected.
+The iterators of PODIO collections model the `std::random_access_iterator` concept, so range algorithms that require this iterator type will work correctly with PODIO iterators. If an algorithm compiles, it is guaranteed to work as expected.
 
 In particular, the PODIO collections' iterators do not fulfil the `std::output_iterator` concept, and as a result, mutating algorithms relying on this iterator type will not compile.
 
-Similarly the collections themselves model the `std::input_range` concept and can be used in the range algorithms that require that concept. The algorithms requiring unsupported range concept, such as `std::output_range`, won't compile.
+Similarly the collections themselves model the `std::random_access_range` concept and can be used in the range algorithms that require that concept. The algorithms requiring unsupported range concept, such as `std::output_range`, won't compile.
 
 For example:
 ```c++
 std::ranges::find_if(collection, predicate ); // requires input_range -> OK
-std::ranges::adjacent_find(collection, predicate ); // requires forward_range -> won't compile
+std::ranges::adjacent_find(collection, predicate ); // requires forward_range -> OK
+std::ranges::views::reverse(collection); //requires bidirectional_range -> OK
 std::ranges::fill(collection, value ); // requires output_range -> won't compile
 std::ranges::sort(collection); // requires random_access_range and sortable -> won't compile
 ```

include/podio/UserDataCollection.h

@@ -81,6 +81,8 @@ class UserDataCollection : public CollectionBase {
   using iterator = typename std::vector<BasicType>::iterator;
   using difference_type = typename std::vector<BasicType>::difference_type;
   using size_type = typename std::vector<BasicType>::size_type;
+  using const_reverse_iterator = typename std::vector<BasicType>::const_reverse_iterator;
+  using reverse_iterator = typename std::vector<BasicType>::reverse_iterator;
 
   UserDataCollection() = default;
   /// Constructor from an existing vector (which will be moved from!)
@@ -222,6 +224,25 @@ class UserDataCollection : public CollectionBase {
   const_iterator cend() const {
     return _vec.cend();
   }
+  // reverse iterators
+  reverse_iterator rbegin() {
+    return _vec.rbegin();
+  }
+  const_reverse_iterator rbegin() const {
+    return _vec.rbegin();
+  }
+  const_reverse_iterator crbegin() const {
+    return _vec.crbegin();
+  }
+  reverse_iterator rend() {
+    return _vec.rend();
+  }
+  const_reverse_iterator rend() const {
+    return _vec.rend();
+  }
+  const_reverse_iterator crend() const {
+    return _vec.crend();
+  }
 
   typename std::vector<BasicType>::reference operator[](size_t idx) {
     return _vec[idx];

include/podio/detail/LinkCollectionImpl.h

@@ -16,6 +16,8 @@
 #include "podio/utilities/MaybeSharedPtr.h"
 #include "podio/utilities/TypeHelpers.h"
 
+#include <iterator>
+
 #ifdef PODIO_JSON_OUTPUT
   #include "nlohmann/json.hpp"
 #endif
@@ -48,6 +50,8 @@ class LinkCollection : public podio::CollectionBase {
   using iterator = LinkMutableCollectionIterator<FromT, ToT>;
   using difference_type = ptrdiff_t;
   using size_type = size_t;
+  using const_reverse_iterator = std::reverse_iterator<const_iterator>;
+  using reverse_iterator = std::reverse_iterator<iterator>;
 
   LinkCollection() = default;
 
@@ -172,6 +176,25 @@ class LinkCollection : public podio::CollectionBase {
   iterator end() {
     return iterator(m_storage.entries.size(), &m_storage.entries);
   }
+  // reverse iterators
+  reverse_iterator rbegin() {
+    return reverse_iterator(end());
+  }
+  const_reverse_iterator rbegin() const {
+    return const_reverse_iterator(end());
+  }
+  const_reverse_iterator crbegin() const {
+    return rbegin();
+  }
+  reverse_iterator rend() {
+    return reverse_iterator(begin());
+  }
+  const_reverse_iterator rend() const {
+    return const_reverse_iterator(begin());
+  }
+  const_reverse_iterator crend() const {
+    return rend();
+  }
 
   bool isValid() const override {
     return m_isValid;

include/podio/detail/LinkCollectionIterator.h

@@ -17,7 +17,9 @@ class LinkCollectionIteratorT {
   using reference = LinkType;
   using pointer = LinkType*;
   using iterator_category = std::input_iterator_tag;
-  using iterator_concept = std::input_iterator_tag;
+  // `std::forward_iterator` is supported except that the pointers obtained with `operator->()`
+  // remain valid as long as the iterator is valid, not as long as the range is valid.
+  using iterator_concept = std::random_access_iterator_tag;
 
   LinkCollectionIteratorT(size_t index, const LinkObjPointerContainer<FromT, ToT>* coll) :
       m_index(index), m_object(podio::utils::MaybeSharedPtr<LinkObjT>{nullptr}), m_collection(coll) {
@@ -29,8 +31,8 @@ class LinkCollectionIteratorT {
   LinkCollectionIteratorT& operator=(LinkCollectionIteratorT&&) = default;
   ~LinkCollectionIteratorT() = default;
 
-  bool operator!=(const LinkCollectionIteratorT& other) const {
-    return m_index != other.m_index;
+  auto operator<=>(const LinkCollectionIteratorT& other) const {
+    return m_index <=> other.m_index;
   }
 
   bool operator==(const LinkCollectionIteratorT& other) const {
@@ -57,6 +59,51 @@ class LinkCollectionIteratorT {
     return copy;
   }
 
+  LinkCollectionIteratorT& operator--() {
+    --m_index;
+    return *this;
+  }
+
+  LinkCollectionIteratorT operator--(int) {
+    auto copy = *this;
+    --m_index;
+    return copy;
+  }
+
+  LinkCollectionIteratorT& operator+=(difference_type n) {
+    m_index += n;
+    return *this;
+  }
+
+  LinkCollectionIteratorT operator+(difference_type n) const {
+    auto copy = *this;
+    copy += n;
+    return copy;
+  }
+
+  friend LinkCollectionIteratorT operator+(difference_type n, const LinkCollectionIteratorT& it) {
+    return it + n;
+  }
+
+  LinkCollectionIteratorT& operator-=(difference_type n) {
+    m_index -= n;
+    return *this;
+  }
+
+  LinkCollectionIteratorT operator-(difference_type n) const {
+    auto copy = *this;
+    copy -= n;
+    return copy;
+  }
+
+  LinkType operator[](difference_type n) const {
+    return LinkType{podio::utils::MaybeSharedPtr<LinkObjT>((*m_collection)[m_index + n])};
+  }
+
+  difference_type operator-(const LinkCollectionIteratorT& other) const {
+    return m_index - other.m_index;
+  }
+
 private:
   size_t m_index{0};
   LinkType m_object{podio::utils::MaybeSharedPtr<LinkObjT>{nullptr}};

python/templates/Collection.h.jinja2

@@ -50,6 +50,8 @@ public:
   using iterator = {{ class.bare_type }}MutableCollectionIterator;
   using difference_type = ptrdiff_t;
   using size_type = size_t;
+  using const_reverse_iterator = std::reverse_iterator<const_iterator>;
+  using reverse_iterator = std::reverse_iterator<iterator>;
 
   {{ class.bare_type }}Collection();
   {{ class.bare_type }}Collection({{ class.bare_type }}CollectionData&& data, bool isSubsetColl);
@@ -167,6 +169,26 @@ public:
   const_iterator cend() const {
     return end();
   }
+  // reverse iterators
+  reverse_iterator rbegin() {
+    return reverse_iterator(end());
+  }
+  const_reverse_iterator rbegin() const {
+    return const_reverse_iterator(end());
+  }
+  const_reverse_iterator crbegin() const {
+    return rbegin();
+  }
+  reverse_iterator rend() {
+    return reverse_iterator(begin());
+  }
+  const_reverse_iterator rend() const {
+    return const_reverse_iterator(begin());
+  }
+  const_reverse_iterator crend() const {
+    return rend();
+  }
+
 
 {% for member in Members %}
   std::vector<{{ member.full_type }}> {{ member.name }}(const size_t nElem = 0) const;

python/templates/macros/iterator.jinja2

@@ -8,7 +8,9 @@ public:
   using reference = {{ prefix }}{{ class.bare_type }};
   using pointer = {{ prefix }}{{ class.bare_type }}*;
   using iterator_category = std::input_iterator_tag;
-  using iterator_concept = std::input_iterator_tag;
+  // `std::forward_iterator` is supported except that the pointers obtained with `operator->()`
+  // remain valid as long as the iterator is valid, not as long as the range is valid.
+  using iterator_concept = std::random_access_iterator_tag;
 
   {{ iterator_type }}(size_t index, const {{ class.bare_type }}ObjPointerContainer* collection) : m_index(index), m_object({{ ptr_init }}), m_collection(collection) {}
   {{ iterator_type }}() = default;
@@ -19,8 +21,8 @@ public:
   {{ iterator_type }}& operator=({{iterator_type}}&&) = default;
   ~{{ iterator_type }}() = default;
 
-  bool operator!=(const {{ iterator_type}}& x) const {
-    return m_index != x.m_index;
+  auto operator<=>(const {{ iterator_type}}& other) const {
+    return m_index <=> other.m_index;
   }
 
   bool operator==(const {{ iterator_type }}& x) const {
@@ -31,6 +33,15 @@ public:
   pointer operator->();
   {{ iterator_type }}& operator++();
   {{ iterator_type }} operator++(int);
+  {{ iterator_type }}& operator--();
+  {{ iterator_type }} operator--(int);
+  {{ iterator_type }}& operator+=(difference_type n);
+  {{ iterator_type }} operator+(difference_type n) const;
+  friend {{ iterator_type }} operator+(difference_type n, const {{ iterator_type }}& it);
+  {{ iterator_type }}& operator-=(difference_type n);
+  {{ iterator_type }} operator-(difference_type n) const;
+  reference operator[](difference_type n) const;
+  difference_type operator-(const {{ iterator_type }}& other) const;
 
 private:
   size_t m_index{0};
@@ -64,5 +75,50 @@ private:
   return copy;
 }
 
+{{ iterator_type }}& {{ iterator_type }}::operator--() {
+  --m_index;
+  return *this;
+}
+
+{{ iterator_type }} {{ iterator_type }}::operator--(int) {
+  auto copy = *this;
+  --m_index;
+  return copy;
+}
+
+{{ iterator_type }}& {{ iterator_type }}::operator+=(difference_type n) {
+  m_index += n;
+  return *this;
+}
+
+{{ iterator_type }} {{ iterator_type }}::operator+(difference_type n) const {
+  auto copy = *this;
+  copy += n;
+  return copy;
+}
+
+{{ iterator_type }} operator+({{ iterator_type }}::difference_type n, const {{ iterator_type }}& it) {
+    return it + n;
+}
+
+{{ iterator_type }}& {{ iterator_type }}::operator-=(difference_type n) {
+  m_index -= n;
+  return *this;
+}
+
+{{ iterator_type }} {{ iterator_type }}::operator-(difference_type n) const {
+  auto copy = *this;
+  copy -= n;
+  return copy;
+}
+
+{{ iterator_type }}::reference {{ iterator_type }}::operator[](difference_type n) const {
+  return reference{ {{ ptr_type }}((*m_collection)[m_index + n]) };
+}
+
+{{ iterator_type }}::difference_type {{ iterator_type }}::operator-(const {{ iterator_type }}& other) const {
+    return m_index - other.m_index;
+}
+
 {% endwith %}
 {% endmacro %}

tests/unittests/links.cpp

@@ -477,6 +477,19 @@ TEST_CASE("Links with interfaces", "[links][interface-types]") {
   REQUIRE(rLink.get<ExampleCluster>() == cluster);
 }
 
+TEST_CASE("Links reverse iterators", "[links][iterator]") {
+  const auto [linkColl, hitColl, clusterColl] = createLinkCollections();
+  REQUIRE(linkColl.size() > 1);
+  auto it = --linkColl.rend();
+  REQUIRE(*it == *linkColl.begin());
+  it = linkColl.rbegin();
+  REQUIRE(*it == *--linkColl.end());
+  auto cit = --linkColl.rend();
+  REQUIRE(*cit == *linkColl.cbegin());
+  cit = linkColl.crbegin();
+  REQUIRE(*cit == *--linkColl.cend());
+}
+
 #ifdef PODIO_JSON_OUTPUT
 TEST_CASE("Link JSON conversion", "[links][json]") {
   const auto& [links, hits, clusters] = createLinkCollections();