perf(origdatablocks): add datasetId index to OrigDatablock and Datablock

[alubbock]

Description

Adds a MongoDB index on datasetId to the OrigDatablock and Datablock collections.

Motivation

All $lookup pipelines that join datasets to their blocks filter on datasetId, but neither collection had an index on that field. Without it, every lookup requires a full collection scan, O(n) per parent document. At scale this is the dominant cost of the dataset detail view (which eagerly joins origdatablocks and datablocks by default) and of any archival workflow that loads blocks by dataset.

Changes:

• src/origdatablocks/schemas/origdatablock.schema.ts -- OrigDatablockSchema.index({ datasetId: 1 })
• src/datablocks/schemas/datablock.schema.ts -- DatablockSchema.index({ datasetId: 1 })

Tests included

• Included for each change/fix?
• Passing?

Two new schema regression tests (origdatablock.schema.spec.ts, datablock.schema.spec.ts) verify that the index definition is present on the compiled Mongoose schema, guarding against accidental removal.

Documentation

• swagger documentation updated (required for API changes) -- n/a, no API change
• official documentation updated -- n/a, internal index definition only

Summary by Sourcery

Add a MongoDB index on datasetId to datablock-related collections and guard it with schema regression tests.

New Features:

• Introduce a datasetId index on the Datablock collection.
• Introduce a datasetId index on the OrigDatablock collection.

Tests:

• Add schema regression tests to verify the datasetId index exists on Datablock and OrigDatablock Mongoose schemas.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[Junjiequan]

lgtm