Merge from asdf feature branch

[0Navin0]

This PR avails an option to save the output file in ASDF format that satisfies roman_datamodels. This is a step towards migration from FITS to ASDF format. The changes include file format and metadata propagation to the output file. Further improvements can include assignment of specific values to the currently populated default-values (from create_fake_data) for some keys in the metadata, and an implementation of the output file name consistent with WFI File Naming Conventions.

[0Navin0]

On commit 50b40ab50b40ab
Thanks for the cleanup work. Some imports are now missing in output.py because wcs_from_fits_header() method was moved from sca.py to output.py without its dependencies. I didn't copy that function to output.py because I couldn't make a decision as to which file should have this method, sca.py or output.py. I thought, as long as we are still documenting the use of this function in sca.py, everything related to this function is already imported in sca.py, so I just called it from RomanSCAImageBuilder instead of repeating so many lines in output.py. So I guess either you can reimport the dependencies again in output.py or just keep using RomanSCAImageBuilder for now.

Including these dependencies in output.py fixes the problem (Note that all these imports are already there in sca.py)

import astropy.coordinates
import astropy.units as u
import gwcs
from astropy import wcs as fits_wcs
from astropy.modeling.models import Mapping, Pix2Sky_TAN, Polynomial2D, RotateNative2Celestial, Shift
from gwcs import coordinate_frames as cf

[arunkannawadi]

I'll move these imports too, thanks for the headsup. As to where this method should live, output.py makes the most sense now since that's where it is being used. If this is more useful, I can see this being moved to utils.py but that can happen later.

[arunkannawadi]

FYI, this PR won't change sca.py. All relevant changes will be moved to output.py.

[arunkannawadi]

I ran the hack.yaml in the save_asdf branch of roman_imsim_testdata and confirmed that it produces an ASDF file that can be read with roman_datamodels. Kudos on this great work Boyan, Navin and Raul 🎉

[arunkannawadi]

Requesting Axel and Sid to take a quick look as well - this could use more clean up, but I need to produce images in ASDF soon, so willing to cut corners as long as it works now.

[sidneymau]

I ran hack.yaml and things seem to look ok --

Python 3.14.3 | packaged by conda-forge | (main, Feb  9 2026, 21:56:02) [GCC 14.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asdf
>>> import roman_datamodels as rdm
>>> dm = rdm.open("output/RomanWAS_new/images/truth/Roman_WAS_truth_J129_12909_4.asdf")
>>> dm.validate()

And the corresponding image looks reasonable

A few changes/comments throughout. IMO, the use of create_fake_data is probably a blocker

[sidneymau]

Is this still uncertain? (also below)

[aguinot]

Maybe something to check, usually "target" refer to where the telescope is pointing which can be the center of the mosaic but necessarily. image.wcs.center can refer to the center of the SCA or mosaic. It most cases they are not the same. The "target" coordinates should be provided by the observation sequence file.
Probably not a big if not set correctly for simulations though..

[0Navin0]

meaning of meta.pointing.target_aperture:

title: Aperture Name Used for Pointing
description: Name of the aperture used to align the instrument to a position on the sky.

meaining of target_ra:

title: Right Ascension of the Target Aperture (deg)
description: Right ascension in units of degrees at the location of the target aperture.

I wasn't sure if one of these should go to pointing.target_ra: wcs_header["RA_TARG"] or image.wcs.header.header['CRVAL1'] or base['world_center'].ra.deg.

Also, please note that the origin differs between image and WCS pixel coordinates. And that creates some discrepancy between base['world_center'].ra.deg and image.wcs.header.header['CRVAL1'].

Please note the following -

# ==> But why is this value different? Is it expected?
ipdb> base['world_center'].ra.deg
10.832447108980707
# Versus
ipdb> image.wcs.header.header['CRVAL1']
np.float64(10.83242929473425)

----------------------------
ipdb> base['world_center'].dec.deg
-44.477474568888496
# Versus
ipdb> image.wcs.header.header['CRVAL2']
np.float64(-44.47749107477652)
----------------------------
# >>>>>>>>>>>> This is the issue
ipdb> for k in ['image_xsize', 'image_ysize', 'image_origin', 'image_center', 'image_bounds', 'wcs', 'world_center']:
    print(k, "==>", base[k])

image_xsize ==> 4088
image_ysize ==> 4088
image_origin ==> galsim.PositionI(1,1)
image_center ==> galsim.PositionD(2044.5,2044.5)
image_bounds ==> galsim.BoundsI(1,4088,1,4088)
# Versus
ipdb> base['wcs'].origin
galsim.PositionD(x=0.0, y=0.0)

# Versus # comment = right ascension of the target (deg) (J2000)
ipdb> wcs_header["RA_TARG"]
10.566 

[aguinot]

From what I see, I would recommend:

tree.meta.pointing.target_ra = wcs_header['RA_TARG']

Now the difference between base['world_center'] and image.wcs.header.header['CRVAL1'] is 0.5 pixels. This is because the pixel center in the WCS is defined as: n_pix/2 which is 2044 while the base['world_center'] is defined with respect to the image center which is 2044.5

[arunkannawadi]

I've been told that target_ra has to point to WFI_CEN which is the detector plane center (SCA0) not the individual SCA's center. RA_TARG and DEC_TARG apepar to be the appropriate candidates for this.

[arunkannawadi]

The unresolved comments are for @0Navin0 , @BoyanYin and @rbgdet to respond

[aguinot]

I did a comparison between fits and asdf and I found some differences.

• First of all, the pixels are not the same and I don't understand why.. I used the same config file in the same environment I just saved to fits instead of asdf and I am not using gz compression. I found that 16% of the pixels are different with a mean difference of 1.4e-10 and a maximum difference of 7e-4.
• The WCS are also different. I think (hope) that it comes from the +1 in the NAXIS. right now fits_wcs.pixel_to_world(100, 100) != asdf_wcs.pixel_to_world(100, 100) BUT fits_wcs.pixel_to_world(100, 100) == asdf_wcs.pixel_to_world(101, 101)
• I really don't understand all the assignement of tree.meta.SOMETHING = tree.meta.SOMETHING. Why are most of the keys assigned to themselves?

Edit: sorry for requesting changes afterward, I didn't found a way to update my review.. The +1 in NAXIS is a problem that need to be resolved before merge otherwise the wcs are mostlikely wrong.

[schlafly]

This function fills out default values for all of the fields that are adequate to make a valid file. You should only need to fill out new data in the tree that you have actually computed and where you care about the result. All of the lines that follow the pattern

tree.meta.product_type = tree.meta.product_type

are not doing anything and can be deleted---most of the mass of this function, lines 208 - 376 or so.

Re when using this is appropriate vs. not---the issue is that this is inventing a large amount of stuff---a large number of times, locations in orbits, investigator names, etc.. Conceptually the validation structure exists to make sure that this was all filled out correctly, and using create_fake_data(...) prevents that validation functionality from verifying that an attempt has been made to fill out all of the needed metadata. But for this code you likely do not want to be warned that you didn't enter a value for the status of the relative calibration system during this exposure, and indeed want some default garbage value and do not want to think more about this. That's what create_fake_data is for.

[sidneymau]

Thanks for this context, Eddie -- this makes sense, I guess our use case isn't quite addressed by the cautions against using create_fake_data from the documentation (and or I misinterpreted)

[arunkannawadi]

Thanks, Eddie. I'll remove all tautological statements from here. But more importantly, are you saying that using create_fake_data somehow skips some or all of the validation that happens in roman_datamodels?

[schlafly]

No, create_fake_data fills out empty / zero values for all of the required metadata, so that model.create_fake_data(...) will pass validation. It doesn't skip validation; it fills out values so that the result will pass validation. All the values are made up and wrong, whence the warnings about its usage, but when you want to fill a bunch of required columns where you don't know / don't have the right values, it's a good way to get those values.

[arunkannawadi]

Ah, thanks Eddie! We will iterate with the broader Roman community for the appropriate values for various use cases.

[arunkannawadi]

Thanks, Eddie. I'll remove all tautological statements from here. But more importantly, are you saying that using create_fake_data somehow skips some or all of the validation that happens in roman_datamodels?

[arunkannawadi]

I've been told that target_ra has to point to WFI_CEN which is the detector plane center (SCA0) not the individual SCA's center. RA_TARG and DEC_TARG apepar to be the appropriate candidates for this.

[arunkannawadi]

The ASDF/Roman datamodels-related changes don't affect the outputs, since we are not yet quite ready to write outputs directly in ASDF. The header changes in sca.py seem appropriate, and not covered by the end-to-end tests either. So with the acknowledgement that the ASDF outputs will likely have a couple of bugs to fix, it is best to merge these changes to main and patch bugfixes later on as we encounter them.