Skip to content

Home

Jump to bottom
Alex Hutko edited this page Apr 20, 2023 · 18 revisions

SSA 2023 oral presentation slidedeck

SQUAC Seismic Quality Assessment Console is built from:

squac the web app with dashboards and alarms. squac.pnsn.org
squacapi an API for interacting with the backend database of billions of measurements from dozens of metrics on thousands of channels. squacapi.pnsn.org
squacapi_client the user friendly python client that makes reading from/writing to the squac database easy. Great for data mining.

Request an account & questions: squac-help@uw.edu

squacapi (squacapi.pnsn.org)

Database/storage details

The backend squacapi database is a high performance AWS postgresDB that extends from 2020/10/20 to the present. Any requests to get or attempts to post measurements earlier than Oct 2020 will result in an ungraceful error. We may extend this further in the past. As of May 2021, the database is ~200 GB with roughly 800 million measurements (rows) with ~7 million new measurements from ~7000 channels being added every day. The database of raw measurements at their original granularity is currently limited to approximately the past 180 days or so in order to maintain reasonable performance. This will likely get shorter as our daily measurement count grows. Older measurements are available using their daily or monthly archives. All measurements at their original granularity are stored in (very cheap) AWS S3 buckets.

Archives of measurements, aka how to make performance snappy

Very large requests, e.g. "latency every 10 minutes for every station at PNSN for a month", can be prohibitively slow. Also, we can't grow the database indefinitely. Enter table partitioning aka "archives". Archives are the daily and monthly aggregates of raw measurements calculated using the raw data in their original granularity. min, max, mean, median, stdev, number of samples, 5th percentile, 10th percentile, 90th percentile, 95th percentile, min(abs(values)), and max(abs(values)) are available.

Metrics

Descriptions of metrics going into the database (does not include externally contributed metrics): https://github.com/pnsn/station_metrics/blob/master/station_metrics.md.
Database details (metric_id, createtime etc.) of metrics (requires account): http://squacapi.pnsn.org/v1.0/measurement/metrics

Channels

Channels in SQUAC are updated daily via IRIS using load_from_fdsn.py which has details on Lat/Lon boundaries, networks and channel types. Email squac@uw.edu if there are channels you'd like for us to consider adding.

Measurements: calculated where?

As of May 2021, roughly 7 million rows (measurements) are being added to the database daily. The measurements come basically from five sources and use squacapi_client to upload unless otherwise noted:

  • latency and gap info sniffed from the four ShakeAlert centers with a 10-minute granularity for all ShakeAlert and all PNSN channels. 10 minutes behind real-time via via sniffwave_tally.
  • power at 10, 5, 1, 0.2, 0.025 from IRIS noise-psd with hourly granularity. The PSD calculations are too heavy for PNSN to calculate under current resources. IRIS does not archive data/calculate PSDs for all ShakeAlert channels, e.g. some NCSN SMAs. Usually about 3-4 days behind real-time.
  • 1- daily FinDer and EPIC trigger counts for all ShakeAlert channels and 2- SCSN AQMS trigger counts from Caltech via a service set up by Igor S. Uploaded by Alex. A few days behind real-time.
  • dozens of waveform based (mean, various spike counts, noise floor etc.) metrics with hourly granularity calculated at PNSN on ~14 CPUs and about 2-3 hours behind real-time.
  • Hourly associated and unassociated EPIC trigger counts. Uploaded by UCB ShakeAlert team.
  • Various experimental metrics from external folks test driving SQUAC.

Clone this wiki locally