Availability WS shows restricted stations as open

[luvec]

Dear all,

I don't know if the includerestricted switch in the FDSN Availability WS works well. See this case:

All stations from the WB network are restricted:
https://geofon.gfz-potsdam.de/waveform/archive/network.php?ncode=WB

And yes, I cannot download the data:
https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/query?network=WB&station=BUBD&start=2024-12-01T00:00:00&end=2024-12-01T00:01:00&nodata=404
Result:

Error 403: Forbidden
access denied
Usage details are available from /fdsnws/dataselect/1/
Request:
/fdsnws/dataselect/1/query?network=WB&station=BUBD&start=2024-12-01T00:00:00&end=2024-12-01T00:01:00&nodata=404
Request Submitted:
2025-01-09T13:41:34.98561
Service Version:
1.1.1

But the Availability Service is showing that the data is open:
https://geofon.gfz-potsdam.de/fdsnws/availability/1/query?network=WB&station=BUBD&channel=?HZ&includerestricted=false&start=2024-12-01T00:00:00&end=2024-12-01T00:01:00&nodata=404
Result:

#Network Station Location Channel Quality SampleRate Earliest Latest
WB BUBD CHZ D 250.0 2024-12-01T00:00:00.000000Z 2024-12-01T00:01:00.000000Z

The includerestricted switch is described in FDSN Web Service Specification as follows:

If true, all data are reported. If false, only data that can be openly accessed.

Regards,
Ludek

[jschaeff]

Thank you for spotting this issue. We will look into this.

[jschaeff]

@jbienkowski and @javiquinte I'm not sure if the problem is in the availability ws implementation or in the metadata used by availability ?

[javiquinte]

I'm taking a look at that and it is still not clear to me where the error comes from. The metadata is OK. I've been going through the code and cache construction looks OK. (I still haven't checked the cache itself).
I'll let you know as soon as I see something.

[petrrr]

From what I understood when looking into the database, it seems that the flag is always set to open, at least in the documents of the collection. I do not think it is possible to get the correct information from the wfcatalog documents and it cannot be determined from harvesting.

Not sure if the web application should determine the correct status?

[jschaeff]

@jbienkowski developped a mechanism that imports this metadata from the station webservice.

[andres-h]

@jbienkowski developped a mechanism that imports this metadata from the station webservice.

Indeed, this requires the availability-cacher cronjob to be run daily and it works correctly -- the data is marked as restricted:

https://geofon.gfz-potsdam.de/fdsnws/availability/1/extent?network=WB&station=BUBD&channel=?HZ&includerestricted=false&start=2024-12-01T00:00:00&end=2024-12-01T00:01:00&nodata=404

It seems that the "includerestricted" parameter is ignored, though.

[petrrr]

This looks more like a software issue and we may want to move it here https://github.com/EIDA/ws-availability