Skip to content

feat: sync proposals, experiments and samples to SciCat from UOS#680

Merged
Junjiequan merged 16 commits into
developfrom
new-proposal-and-expeirment-data-for-scicat
May 22, 2026
Merged

feat: sync proposals, experiments and samples to SciCat from UOS#680
Junjiequan merged 16 commits into
developfrom
new-proposal-and-expeirment-data-for-scicat

Conversation

@Junjiequan
Copy link
Copy Markdown
Contributor

@Junjiequan Junjiequan commented May 18, 2026

Description

The connector now fetches proposals, experiments and samples from UOS via GraphQL using primary keys received from RabbitMQ, and syncs them to SciCat.

Changes

  • Added UOS GraphQL API class (uosApi.ts) to fetch proposals and experiments by primary key
  • Added mappers to transform UOS data into SciCat DTOs (uoToScicatProposal.mapper.ts, uoToScicatExperiment.mapper.ts)
  • Added ScicatApi class to handle all SciCat HTTP operations (create, update, find)
  • Proposal and experiment upsert logic based on existence check in SciCat
  • Sample upsert with hash-based change detection to skip unchanged samples
  • Added validators for proposal and experiment RabbitMQ messages
  • Added unit tests for mappers and UOS API client

Depends on

Tests included/Docs Updated?

  • I have added tests to cover my changes.
  • All relevant doc has been updated

@Junjiequan Junjiequan marked this pull request as draft May 18, 2026 15:07
2. map uoProposal to scicatProposal
3. created experiment queue
@janosbabik janosbabik self-requested a review May 21, 2026 07:49
@Junjiequan Junjiequan marked this pull request as ready for review May 21, 2026 07:52
Comment thread src/queue/consumers/scicat/scicatProposal/mappers/uoToScicatExperiment.mapper.ts Dismissed
Copy link
Copy Markdown
Contributor

@janosbabik janosbabik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Junjiequan, it looks amazing! I left some comments.

type: 'Sample Information',
accessGroups: [experiment.experimentId],
proposalId: experiment.experimentId,
description: sample.sample.title, //TODO: this should be replaced with sampleName before merge
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TODO is blocked by Scicat Backend SciCatProject/backend#2747. I will address it as soon as BE PR is merged

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 960349a

I will hold this PR until Scicat PR gets merged first

ownerGroup: scicatApi.username || '',
accessGroups: [experiment.experimentId],
proposalId: experiment.experimentId,
description: sample.sample.title, //TODO: this should be replaced with sampleName before merge
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

Comment thread src/queue/consumers/scicat/scicatProposal/mappers/uoToScicatProposal.mapper.ts Outdated
Comment thread src/queue/consumers/scicat/scicatProposal/utils/scicatApi.ts Outdated
Comment thread src/queue/consumers/scicat/scicatProposal/utils/scicatApi.ts Outdated
@janosbabik janosbabik self-requested a review May 22, 2026 09:36
@Junjiequan Junjiequan merged commit 5daf495 into develop May 22, 2026
7 checks passed
@Junjiequan Junjiequan deleted the new-proposal-and-expeirment-data-for-scicat branch May 22, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants