[NO-TICKET] Add new changelog/add.rb nano tool by ivoanjo · Pull Request #6119 · DataDog/dd-trace-rb

ivoanjo · 2026-07-29T01:21:30Z

What does this PR do?

This PR adds the a new changelog/add.rb nano tool to help us maintain changelog entries for the next release in-repo, rather than in PR descriptions as we've been doing so far.

The envisioned workflow is:

Adding an entry: ruby changelog/add.rb --category Added --label Profiling --entry "Add Matz conference talks profiling" (or interactively: run ruby changelog/add.rb and follow along)

This will create a changelog/unreleased-branchname.md file with the change
Updating an entry: Just open the markdown file and change!
At release time: Fast castle will flush all unreleased entries in the changelog/ directory into the release draft. Then the release_prep rake task will delete them (as part of the PR that updates CHANGELOG.md)

Motivation:

For a while we've wanted a in-repo mechanism to keep changelogs, so that we can comment on them, do PRs to update them, etc.

This nano tool is my proposal to do such a thing.

Additional Notes:

This PR includes:

The nano tool
An update to CLAUDE.md and to PULL_REQUEST_TEMPLATE.md to mention use of the tool
An update to the ensure-changelog-entry.yml workflow to now check for a file in-repo, rather than the PR description + a lightweight exclusion list for branches which aren't expected to have changelog entries
An update to release_prep to delete the entries as part of the release process

We're still missing the https://github.com/DataDog/fast_castle changes. Once we're happy with this approach, I plan to open a PR on the fast_castle side to handle the new format.

I propose that, if we like this change, we merge it as the first PR after the next/a future release: this way we simplify the migration. I'll then go around merging master to recent existing PRs so the new "you're missing the changelog" check runs for them.

How to test the change?

Run the tool and check it out!

**What does this PR do?** This PR adds the a new `changelog/add.rb` nano tool to help us maintain changelog entries for the next release in-repo, rather than in PR descriptions as we've been doing so far. The envisioned workflow is: * _Adding an entry_: `ruby changelog/add.rb --category Added --label Profiling --entry "Add Matz conference talks profiling"` (or interactively: run `ruby changelog/add.rb` and follow along) This will create a `changelog/unreleased-branchname.md` file with the change * _Updating an entry_: Just open the markdown file and change! * _At release time_: Fast castle will flush all unreleased entries in the `changelog/` directory into the release draft. Then the `release_prep` rake task will delete them (as part of the PR that updates `CHANGELOG.md`) **Motivation:** For a while we've wanted a in-repo mechanism to keep changelogs, so that we can comment on them, do PRs to update them, etc. This nano tool is my proposal to do such a thing. **Additional Notes:** This PR includes: * The nano tool * An update to `CLAUDE.md` and to `PULL_REQUEST_TEMPLATE.md` to mention use of the tool * An update to the `ensure-changelog-entry.yml` workflow to now check for a file in-repo, rather than the PR description + a lightweight exclusion list for branches which aren't expected to have changelog entries * An update to `release_prep` to delete the entries as part of the release process We're still missing the https://github.com/DataDog/fast_castle changes. Once we're happy with this approach, I plan to open a PR on the `fast_castle` side to handle the new format. **How to test the change?** Run the tool and check it out!

datadog-prod-us1-3 · 2026-07-29T01:22:25Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 90.02% (-0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 4488d1d | Docs | Datadog PR Page | Give us feedback!}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 920e834924

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-29T01:32:46Z

+            })
+            const files = await github.paginate(options)
+
+            const hasEntry = files.some((file) => file.filename === expectedPath && file.status !== "removed")


Keep the old changelog gate until fragments are imported

Until the release-draft generator is deployed to read changelog/unreleased-*.md, this path-only check lets a customer-facing PR satisfy the workflow by adding a fragment while release_prep:prepare still inserts only the GitHub draft body and then deletes all fragments. In that rollout window the entry is no longer present in the PR body and can be removed by the release PR without ever reaching CHANGELOG.md; either keep requiring the old PR-body entry or make release prep/draft generation consume fragments before this workflow passes them.

Useful? React with 👍 / 👎.

This is a reasonable call-out. I'll update my plan above; I think the simplest option is to introduce the new tool right after a release -- this way we'll have the whole baking period of a release to experiment with it + no need to backfill.

Backfilling in this very PR would be nice tho, it means there's no weird "in between" state, where the tool is there but not really used, and also allows to see what the tree looks like in practice.

chatgpt-codex-connector · 2026-07-29T01:32:46Z

+            })
+            const files = await github.paginate(options)
+
+            const hasEntry = files.some((file) => file.filename === expectedPath && file.status !== "removed")


Validate the fragment before accepting it

When a contributor creates or edits the fragment manually, this check passes any non-removed file at the expected path, including an empty file or one with a malformed category/label. That silently disables the changelog reminder even though the release tooling may have no usable customer-facing entry to import; please also read the file contents and require either the None opt-out line or a valid non-empty changelog line.

Useful? React with 👍 / 👎.

I think this is fine for now, that's what PR review is for. We can adjust as needed as we get more comfortable with the mechanism.

Maybe an opportunity for a linting tool down the road?

The more we save our eyeballs for important stuff the better, plus it coerces off-track-veering LLMs into doing the correct thing.

lloeki

No blockers, only a Christmas wishlist.

lloeki · 2026-07-30T13:37:59Z

 - **Tests.** Every behavior change ships with test coverage in the same PR. A PR that changes code without a corresponding test change is incomplete, not a follow-up to file later.
 - **Size.** If the resulting diff would exceed roughly 1000 lines of additions, stop and propose a split into smaller, stackable PRs before generating any code.
 - **Description.** Use `.github/PULL_REQUEST_TEMPLATE.md` as the structure. At most three sentences per section — high-level intent, not a file-by-file list. Answer *what* and *why*, and call out any non-obvious trade-off; the reviewer reads the diff for the rest. Before opening the PR, count the sentences in each section; if a section runs longer, cut it or move the detail to Additional Notes.
- **Change log entry.** Start with `Yes.` plus a one-sentence customer-facing summary if the change affects observable provider behavior (flag evaluation results, hooks, the public `OpenFeature` API); otherwise `None.` Refactors, tests, and tooling changes internal are always `None.`


Should we keep the instruction here in some form to have the thing add a changelog entry via the add tool?

lloeki · 2026-07-30T13:39:47Z

Out of scope for this PR: CLAUDE.md should probably become AGENTS.md, and have CLAUDE.md import @AGENTS.md instead.

lloeki · 2026-07-30T13:48:36Z

-You can skip this section and it will be filled or deleted during PR review.
-Please do not remove this section from the PR though.
-->
+<!-- (Changelogs are now kept in-tree using the `changelog/add.rb` nano tool) -->


Nit: This comment now looks like it belongs to the Motivation section.

Suggested change





But really, I also find it a bit awkwardly noisy to add this callout here. The section not being there anymore looks like signal enough to me to not try and write it again, especially with the bot calling out the lack of changelog content and hopefully directing how to add one (which it is going to do, right? riiight????)

lloeki · 2026-07-30T13:51:04Z

+            })
+            const files = await github.paginate(options)
+
+            const hasEntry = files.some((file) => file.filename === expectedPath && file.status !== "removed")


Backfilling in this very PR would be nice tho, it means there's no weird "in between" state, where the tool is there but not really used, and also allows to see what the tree looks like in practice.

lloeki · 2026-07-30T13:54:37Z

+            })
+            const files = await github.paginate(options)
+
+            const hasEntry = files.some((file) => file.filename === expectedPath && file.status !== "removed")


Maybe an opportunity for a linting tool down the road?

The more we save our eyeballs for important stuff the better, plus it coerces off-track-veering LLMs into doing the correct thing.

ivoanjo requested review from a team as code owners July 29, 2026 01:21

ivoanjo requested review from sameerank and vjfridge July 29, 2026 01:21

dd-octo-sts Bot added the openfeature A new component that provider an ability to configure feature flags label Jul 29, 2026

Add changelog folder to excluded folders list in release_gem_spec.rb

4488d1d

dd-octo-sts Bot added the openfeature A new component that provider an ability to configure feature flags label Jul 29, 2026

chatgpt-codex-connector Bot reviewed Jul 29, 2026

View reviewed changes

lloeki approved these changes Jul 30, 2026

View reviewed changes

Uh oh!

Conversation

ivoanjo commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-prod-us1-3 Bot commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lloeki left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ivoanjo commented Jul 29, 2026 •

edited

Loading

datadog-prod-us1-3 Bot commented Jul 29, 2026 •

edited

Loading