Feed endpoints for Open Terms Archive collections

[clementbiron]

Feed endpoints for Open Terms Archive collections

Context

Open Terms Archive collections currently rely on GitHub-generated Atom feeds to expose version changes to subscribers.

Problem statement

GitHub-generated feeds are outside OTA's control: their format, content, and filtering capabilities are dictated by GitHub. Collections cannot expose custom metadata, filter entries by service or terms type, or guarantee continuity if GitHub changes or discontinues this feature.

Proposed solution

The collection API exposes dedicated feed endpoints that generate and serve Atom feeds directly from the versions repository, without relying on any external platform.

Endpoints:

• GET /feed Global feed (all services, all terms types), newest-first, limited to last N items (configurable, default 100)
• GET /feed/:serviceId Filtered to a specific service
• GET /feed/:serviceId/:termsType Filtered to a specific service and a specific terms type

Feed-level metadata:

• <title> Collection name
• <subtitle> Collection tagline
• <id> urn:{namespace}:{collectionId}:{languages}:{jurisdictions}
• <updated> Fetch date of most recent version
• <link rel="self" href={feedBasePath} />
• <author><name>OTA-Bot</name></author>
• <logo> Collection logo URL

Entry-level metadata:

• <id> {storageType}:{identifier} Storage type (Git or MongoDB) and unique identifier (SHA or ObjectId)
• <link rel="alternate" type="text/html"> {apiBasePath}/version/{serviceId}/{termsType}/{fetchDate} Link to collection API route
• <title> Mirrors Git commit message
• <updated> Fetch date
• <category term="{serviceId}" scheme="urn:ota:service" />
• <category term="{termsType}" scheme="urn:ota:terms-type" />
• <category term="{recordType}" scheme="urn:ota:record-type" />

Notes

• The three endpoints allow subscribers to choose their level of granularity: the global feed covers all activity across the collection, while the service and terms type filters let subscribers focus on what matters to them. For subscribers using the global feed, the elements on each entry expose the same filtering dimensions directly in the feed, allowing feed readers to apply client-side rules without requiring a separate subscription.
• If no versions match the request, an empty feed is returned, allowing subscribers to keep their subscription active and receive future entries.
• The Atom format is proposed because it benefits from clear, mandatory fields and unambiguous semantics as defined by the IETF standard, but also because it is flexible and extensible, supports custom elements and attributes, and is widely supported by feed readers.

Example: global feed response

<?xml version="1.0" encoding="utf-8"?>
  <feed xmlns="http://www.w3.org/2005/Atom">
    <title>Demo</title>
    <subtitle>Services used by Open Terms Archive</subtitle>
    <id>urn:ota:demo:en:eu</id>
    <updated>2024-03-15T10:30:00Z</updated>
    <link rel="self" href="http://162.19.74.224/collection-api/v1/feed"/>
    <author>
      <name>OTA-Bot</name>
    </author>
    <logo>https://opentermsarchive.org/images/collections/demo.png</logo>
    <entry>
      <id>git:2bfa15dd66f89e8612f33875a5be799a45f4b50a</id>
      <link rel="alternate" type="text/html">http://162.19.74.224/collection-api/v1/version/npm%20Public%20Registry/Copyright%20Claims%20Policy/2026-02-12T00%3A30%3A15Z</link>
      <title>Record new changes of npm Public Registry Copyright Claims Policy</title>
      <updated>2026-02-12T00:30:15Z</updated>
      <category term="npm Public Registry" scheme="urn:ota:service"/>
      <category term="Copyright Claims Policy" scheme="urn:ota:terms-type"/>
      <category term="Change" scheme="urn:ota:record-type"/>
    </entry>
    <entry>
      <id>git:b7c967be56caa71d700282880bca901565ec544d</id>
      <link rel="alternate" type="text/html">http://162.19.74.224/collection-api/v1/version/Buffer/Terms%20of%20Service/2026-02-04T00%3A30%3A03Z</link>
      <title>Record new changes of Buffer Terms of Service</title>
      <updated>2026-02-04T00:30:03Z</updated>
      <category term="Buffer" scheme="urn:ota:service"/>
      <category term="Terms of Service" scheme="urn:ota:terms-type"/>
      <category term="Change" scheme="urn:ota:record-type"/>
    </entry>
    <entry>
      <id>git:3af22e88474e02fe99a69b92e8fdd0fa43f6756f</id>
      <link rel="alternate" type="text/html">http://162.19.74.224/collection-api/v1/version/UptimeRobot/Terms%20of%20Service/2025-09-22T15%3A54%3A35Z</link>
      <title>Apply technical or declaration upgrade on UptimeRobot Terms of Service</title>
      <updated>2025-09-22T15:54:35Z</updated>
      <category term="UptimeRobot" scheme="urn:ota:service"/>
      <category term="Terms of Service" scheme="urn:ota:terms-type"/>
      <category term="Technical upgrade" scheme="urn:ota:record-type"/>
    </entry>
    <entry>
      <id>git:4990c3bed01498668c93a7cabdacbe60478dccae</id>
      <link rel="alternate" type="text/html">http://162.19.74.224/collection-api/v1/version/Brevo/Privacy%20Policy/2023-09-26T12%3A30%3A00.000Z</link>
      <title>First record of Brevo Privacy Policy</title>
      <updated>2023-09-26T12:30:00.000Z</updated>
      <category term="Brevo" scheme="urn:ota:service"/>
      <category term="Privacy Policy" scheme="urn:ota:terms-type"/>
      <category term="First record" scheme="urn:ota:record-type"/>
    </entry>
    <!-- ... -->
  </feed>

Feedback expectations

We invite you to provide feedback to:

• Point out any limitations or edge cases you see in the proposed implementation.
• Suggest improvements or refinements to the current proposal.
• Share any alternative approaches you believe could address the problem more effectively.

If you support the proposal as it stands, please react with a 👍 or leave a positive comment.

Please provide your feedback by April 14th.

[clementbiron]

I would like to take this opportunity to notify those who might be interested in this RFC. This list is not exhaustive, and all feedback is welcome.

@margelacool @LVerneyPEReN @zenobioarn @mvidonne @Cli4d @glebbt @shlomihod @Kissaki

[shlomihod]

@clementbiron Thank you for the tag — very cool feature!

One question: could you please describe the use cases you have in mind? The proposed feed returns only minimal information. I can imagine a scenario where someone simply wants to know that a change occurred and then checks it manually (e.g., via GitHub). But for any more complex use case I can think of (including Terms Watch), I'd imagine developers would also want to receive the actual diff. It might be worth considering an "extended" version of the feed for that purpose.

[Ndpnt]

The current feed <id> is urn:ota:{collectionId}:{languages}:{jurisdictions}. I think the collectionId alone would be sufficient for uniqueness. Languages and jurisdictions are collection metadata, not identity components, if a collection adds a language, its feed identifier should not change.
So I suggest to simplify feed id to urn:ota:{collectionId}.

[Ndpnt]

The RFC uses urn:ota as a namespace for identifiers and category schemes. Since this namespace is not registered with IANA, there is no formal mechanism preventing another party from using the same identifier. With a three-letter namespace like ota, the collision risk is notable.

I'd suggest using one of the two following alternatives:

Option 1: urn:opentermsarchive

This would make the collision risk negligible in practice, as the namespace is specific enough that an accidental collision is very unlikely, even without IANA registration.

Feed ID: urn:ota:demo → urn:opentermsarchive:feed:demo
Entry ID: git:2bfa15… → urn:opentermsarchive:version:demo:git:2bfa15…
Category scheme: urn:ota:service → urn:opentermsarchive:scheme:service

Option 2: Tag URIs

They guarantee uniqueness through domain ownership without any registration, and are the format GitHub uses for its own Atom feeds (tag:github.com,2008:…), which happens to be what current OTA subscribers already consume. The date component (2026 in the examples below) is chosen once and never changes, it can be any date at which opentermsarchive.org was owned by OTA.

Feed ID: tag:opentermsarchive.org,2026:feed:demo
Entry ID: tag:opentermsarchive.org,2026:version:demo:git:2bfa15…
Category scheme: tag:opentermsarchive.org,2026:scheme:service

Which one would be preferable?

[Ndpnt]

The global feed is described as "limited to last N items (configurable, default 100)". Could you clarify how this would be configured? For example, is it a query parameter (/feed?limit=50) or a server-side configuration option?

[clementbiron]

Hi @shlomihod

One question: could you please describe the use cases you have in mind?

Some users are already using the Atom feeds provided by GitHub to keep up to date with changes to a repository and we have secured a grant from Nlnet aimed at replacing hard-coded connections to proprietary software with standardised APIs.

I'd imagine developers would also want to receive the actual diff. It might be worth considering an "extended" version of the feed for that purpose.

That’s a very good point, thank you. I deliberately chose not to include the diff because:

• it would significantly increase the size of the Atom feed
• including it requires it to be interpreted correctly by the reader
• not all subscribers need it
• it can be viewed with a single click

Feel free to share your thoughts if you have a different opinion ;)

[clementbiron]

Hi @Ndpnt

Thank you for your feedback.

I think that the option 2 you suggested is the most robust, i would go with that.

The global feed is described as "limited to last N items (configurable, default 100)". Could you clarify how this would be configured? For example, is it a query parameter (/feed?limit=50) or a server-side configuration option?

I don' see any scenario where you'd need to specify the number of items in a query param. So i would go for server side configuration.

[margelacool]

Hi @clementbiron , Thanks for the update and tag. The ATOM format and filtering a specific term type are super relevant! That would make the feed way more readable. Here is my only suggestions: populate the <summary> or <content> field of each entry with a line change count (e.g. +12 lines / -3 lines). Let me know if that makes sense

[glebbt]

Hi @clementbiron thank you for making the update and keeping us informed! We at the PGA team don't really use the feed technology to get notified about the changes. We thought, however, it might be useful, if there was some kind of a filter which could distinguish between insignificant changes (noise) and actual alterations in the documents. Is something like that potentially possible?

[clementbiron]

Hi @margelacool

Here is my only suggestions: populate the <summary> or <content> field of each entry with a line change count (e.g. +12 lines / -3 lines). Let me know if that makes sense

That's an interesting suggestion, thank you! The number of lines changed is indeed useful for conveying the scale of a change at a glance. That said, I have one reservation: presented on its own, this metric risks being mistaken for an indicator of significance. A single-line change can have major implications for users' rights and obligations, while a complete rewrite may introduce no meaningful difference at all. I'd hate for subscribers to deprioritise something important just because the diff looks small, or vice versa.

For now, I'd prefer to leave it out, but if you feel strongly about it, I'm happy to keep the discussion.

[clementbiron]

Hi @glebbt

We thought, however, it might be useful, if there was some kind of a filter which could distinguish between insignificant changes (noise) and actual alterations in the documents. Is something like that potentially possible?

At present, categorisation has to be done manually, but yes, it would be possible to automatically detect some changes that resemble noise; we have a few ideas on how to do this, but there is currently no funding for it and it is not on the agenda.