XRD appdef draft by domna · Pull Request #35 · FAIRmat-NFDI/data-modeling

domna · 2022-07-19T13:00:22Z

This is the cleaned XRD appdef yaml file. Please leave comments directly in the file so we can discuss (in Files changed tab). You may also comment in this thread for general discussions.

@aalbino2 @cmmngr @mkuehbach @tomio13 @rettigl

domna · 2022-07-19T13:11:49Z

Here is the powder diffraction appdef we derive from:
https://manual.nexusformat.org/classes/applications/NXmonopd.html#nxmonopd

This is an interesting class to represent 2D detector scattering experiments:
https://manual.nexusformat.org/classes/applications/NXsas.html#nxsas

Some other interesting links:

XRD analysis packages: https://github.com/dkriegner/xrayutilities, https://github.com/silx-kit/pyFAI
Here is a nice python file from Carsten Richter (@carichte) to read brml and rasx files: https://github.com/carichte/IKZ/blob/master/IKZ/xray/io.py We could use this to investigate the metadata inside these files.
Nexus constructor to view NXtransformations: https://github.com/ess-dmsc/nexus-constructor/blob/main/getting_started.md

I have added a bunch of recommendations / comments, hoping to improve the document.

domna · 2022-07-21T13:39:49Z

There is also an eulerian cradle definition in nexus which encodes angular settings on a sphere:
https://manual.nexusformat.org/classes/applications/NXxeuler.html#nxxeuler

rettigl

Good start, I added some comments

rettigl · 2022-07-26T08:02:19Z

XRD/xrd.yaml

+symbols:
+# I would recommend using a bit more verbose symbol names, e.g.
+# N_wavelengths, N_detectors
+  wlens: Number of wavlengths in the monochromated beam


This comment seems ill-defined to me. Wavelength is a continouts parameter. One could define a number of wavelength points or something. Also not clear to me why this should be part of the AppDef.

rettigl · 2022-07-26T08:03:14Z

XRD/xrd.yaml

+(NXxrd):
+  (NXentry):
+    # to me title makes no sense... acn we drop it?
+    title(NX_CHAR):


Is there a good reason this "extended title" field should be mandatory?

rettigl · 2022-07-26T08:04:07Z

XRD/xrd.yaml

+    # move the time after the definition part
+    # because that part describes this document, all after it the experiment
+    start_time(NX_DATE_TIME):
+      doc: "Datetime of the start of the measurement"


Is this (qualitatively) different from the one in NXentry? Otherwise it's redundant, remove.

rettigl · 2022-07-26T08:10:16Z

XRD/xrd.yaml

+      enumeration: ["NXxrd"]
+    (NXinstrument):
+      (NXsource):
+        type(NX_CHAR):


We can also add an enum here. there are only so many possible x-ray sources.

rettigl · 2022-07-26T08:13:14Z

XRD/xrd.yaml

+        # if you expect to have wlens or N_wavelengts number of wavelength in the output,
+        # how can this be a single number?


This corresponds to the center wavelength. there is also wavelength_spread, that corresponds to a width.
the one in crystal corresponds probably to a spectrum.

For any practical purposes, a single number, and potentially, the energy spread should typically be sufficient. The spectrum you can anyways store within the bae class fields if you believe it important.

These parameters can also be multidemensional. You can always extend the dimensions in Nexus, as far as I know. I am thinking here e.g. about energy-dependent diffraction data at sychrotrons, where one scans the energy across an absorption edge and simultaneously moves all diffraction angles.

rettigl · 2022-07-26T08:36:47Z

XRD/xrd.yaml

+      rotation_angle(NX_FLOAT):
+        doc: |
+          Optional rotation angle for the case when the powder 
+          diagram has been obtained through an omega-2theta scan 
+          like from a traditional single detector powder 
+          diffractometer
+        unit: NX_ANGLE


Well, this is something special. I would also make this part of an NXtransformation, but we also need a field to specify that this kind of operation mode has been used, and which angle corresponds to the roation being used for this.

Missing is an NXtransformation for the sample, and a depends_on

rettigl · 2022-07-26T08:38:11Z

XRD/xrd.yaml

+          like from a traditional single detector powder 
+          diffractometer
+        unit: NX_ANGLE
+    # should not this (monitor) more belong to the instrument / detector part?


I think Nxmonitor is somehow trated specially like NXdata in nexus. It can be a link to something in NXintrument, like NXdata typically is.

rettigl · 2022-07-26T08:42:10Z

XRD/xrd.yaml

+      mode(NX_CHAR):
+        enumeration: ["monitor", "timer"]
+        doc: |
+          Count to a preset value based on either clock time (timer) 
+          or received monitor counts (monitor).
+      preset(NX_FLOAT):
+        doc: Preset value for time or monitor
+      integral(NX_FLOAT):
+        doc: Total integral monitor counts
+        unit: NX_ANY


NXmonitor should be optional.
All the fields listed here should also be optional, or even not appear here, to my opinion. However, the one that really is important, data, is missing.

rettigl · 2022-07-26T08:44:07Z

XRD/xrd.yaml

+      polar_angle(link):
+        doc: Should contain a link to polar angle in /NXentry/NXinstrument/NXdetector


this is too restrictive. Leave away, and let the people decide based on their NXtransformations and use case.

rettigl · 2022-07-26T08:45:18Z

XRD/xrd.yaml

+      polar_angle(link):
+        doc: Should contain a link to polar angle in /NXentry/NXinstrument/NXdetector
+      data(link):
+        doc: Should contain a link to data in /Nxentry/Nxinstrument/Nxdetector


I would also not restrict it in this way. If people want to process their data (flat field correction, whatnot), they should do so in NXprocess entries, and then link NXdata to whatever endpoint of this chain they want to choose.

domna · 2022-08-01T10:13:50Z

Hi @mkuehbach @tomio13 @rettigl @sunyue-xfel @cmmngr @sanbrock

I added a part to account for different experimental angles with NXtransformations. I introduced a NXgoniometer to supply all of the angles (did not yet add the base class, though). I wrote all of the angles explicitly, even if we maybe should keep it a bit more open. But for now I thought it would be nice to have something more specific for discussion.

Maybe we can discuss this shortly tomorrow in TF B meeting if we have time, @sanbrock @mkuehbach ?

XRD/NXgoniometer.yaml

XRD/NXxrd.yaml

lukaspie · 2025-01-08T11:33:57Z

@RubelMozumder I noticed there are some ideas here that are not part of our current NXxrd. Are these ever to be used again? If not, let's close this PR.

RubelMozumder · 2025-01-08T13:51:44Z

@RubelMozumder I noticed there are some ideas here that are not part of our current NXxrd. Are these ever to be used again? If not, let's close this PR.

All of the core concepts have been available in the xrd_pan app def regarding the scan data, but it is not elaborated for all the metadata described here. From the safe side we can keep it.

Cleans xrd appdef

b5b08bc

domna requested a review from tomio13 July 19, 2022 13:00

Update xrd.yaml

37b1e87

I have added a bunch of recommendations / comments, hoping to improve the document.

rettigl reviewed Jul 26, 2022

View reviewed changes

domna added 3 commits July 26, 2022 15:24

Adds changes from TF meeting

3788c7d

Renames nDet to N_detectors

e0a88c9

Draft for XRD goniometer positions w/ NXtransformations

a9c9d55

Deleted binarily redundant RSM(1).brml, added suggestions to xrd.yaml

354dfe6

domna mentioned this pull request Aug 4, 2022

3d XRD data modeling #32

Closed

domna added 2 commits August 4, 2022 16:24

NXxrd: Refinements from TF meeting

3066761

Deletes xrd.yaml

ec7b30f

rettigl reviewed Aug 5, 2022

View reviewed changes

XRD/NXgoniometer.yaml Outdated Show resolved Hide resolved

rettigl reviewed Aug 5, 2022

View reviewed changes

XRD/NXgoniometer.yaml Outdated Show resolved Hide resolved

rettigl reviewed Aug 5, 2022

View reviewed changes

XRD/NXxrd.yaml Outdated Show resolved Hide resolved

domna added 6 commits August 9, 2022 17:25

Adds analysis files for rasx parser

58f78cc

Use NXmanipulator and move transformations inside group

8f48d80

Deletes NXgoniometer base class

19e89db

Corrects NXxrd for parsing into nxdl

b826812

Adds nb to readout data from brml and rasx files

3c02848

Adds width and height to graphs

fe213ad

lukaspie added the invalid This doesn't seem right label Jan 8, 2025

		# if you expect to have wlens or N_wavelengts number of wavelength in the output,
		# how can this be a single number?

		polar_angle(link):
		doc: Should contain a link to polar angle in /NXentry/NXinstrument/NXdetector

Conversation

domna commented Jul 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

domna commented Jul 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

domna commented Jul 21, 2022

Uh oh!

rettigl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

domna commented Aug 1, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lukaspie commented Jan 8, 2025

Uh oh!

RubelMozumder commented Jan 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

domna commented Jul 19, 2022 •

edited

Loading

domna commented Jul 19, 2022 •

edited

Loading