Skip to content

tempSpecificDepthLakes_level1_grp_reshape does not appear to handle sensor swaps properly #1205

@covesturtevant

Description

@covesturtevant

Error in tempSpecificDepthLakes_level1_grp_reshape at datum: /2026/05/05/temp-specific-depths-lakes_FLNT103100

Logs:

INFO  [2026-05-08 00:27:48.360] Processing path to datum: /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885
ERROR [2026-05-08 00:27:48.362] In def.validate.json::::: Error, the file does not exist, /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51220_locations.json
ERROR [2026-05-08 00:27:48.362] In def.validate.json::::: Error, the file does not exist, /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51655_locations.json
ERROR [2026-05-08 00:27:48.363]
Error resulted from call to:
NEONprocIS.base::def.loc.geo.hist(locFile, log = NULL)
Call stack:
[1] "tryCatch(withCallingHandlers(wrap.file.comb.tsdl.splt(InfoDirIn = InfoDirIn"
[2] "tryCatchList(expr, classes, parentenv, handlers)"
[3] "tryCatchOne(expr, names, parentenv, handlers[[1]])"
[4] "doTryCatch(return(expr), name, parentenv, handler)"
[5] "withCallingHandlers(wrap.file.comb.tsdl.splt(InfoDirIn = InfoDirIn, DirOut "
[6] "wrap.file.comb.tsdl.splt(InfoDirIn = InfoDirIn, DirOut = Para$DirOut, fileP"
[7] "NEONprocIS.base::def.loc.geo.hist(locFile, log = NULL)"
[8] "stop()"
[9] ".handleSimpleError(function (err) \n{\n call.stack <- base::sys.calls()\n N"
[10] "h(simpleError(msg, call))"
INFO [2026-05-08 00:27:48.365] Re-routing failed datum path /tmp/pfs/filter_joined/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885 to /pfs/out/errored_datums
INFO [2026-05-08 00:27:48.367] Removed partial output for errored datum: /pfs/out/2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885

There was a sensor swap this day. Repo structure of tempSpecificDepthLakes_group_path at this datum:

$ pc glob file tempSpecificDepthLakes_group_path@master:/2026/05/05/*FLNT*/**
SIZE    NAME
750B    /2026/05/05/temp-specific-depths-lakes_FLNT103100/group/
750B    /2026/05/05/temp-specific-depths-lakes_FLNT103100/group/CFGLOC113885.json
54.12KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/
54.12KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/
9.722KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/data/
9.722KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/data/tchain_parsed_CFGLOC113885_2026-05-05.parquet
14.14KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/flags/
14.14KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/flags/tchain_parsed_CFGLOC113885_2026-05-05_flagsCal.parquet
4.419KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/
655B    /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/CFGLOC113885.json
1.89KiB /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51220_locations.json
1.89KiB /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/location/tchain_51655_locations.json
9.419KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_coef/
9.419KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_coef/tchain_parsed_CFGLOC113885_2026-05-05_uncertaintyCoef.json
16.42KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_data/
16.42KiB        /2026/05/05/temp-specific-depths-lakes_FLNT103100/tchain/CFGLOC113885/uncertainty_data/tchain_parsed_CFGLOC113885_2026-05-05_uncertaintyData.parquet

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions