Error message and sanity checking improvements #59
Description
@alyssaobester 1:15 PM
the package is working for me on some gages and not others. any ideas as to why?
Nick Santos 2:47
can you send me some of the working and some of the not working gage ids?
Alyssa Obester 2:49 PM
okay, thanks
2:51
this seem to be working:
USGS gage: 11445500, SF American River near Lotus
COMID: 14982092
and if I manually provide flow data for a COMID on the Middle Fork American River (COMID 14994421) it runs. (edited)
2:52
but this gage on San Antonio Creek doesn't work (11117500) and I get the following error:
Error in if (median >= predictions[["p10"]] && median <= predictions[["p90"]]) { :
missing value where TRUE/FALSE needed
2:53
And when I download that USGS gage data and try and feed it into the package, this is the error I get:
2:53
(Error in process_data(flows_df, params, flow_field = flow_field, date_field = date_field, : Bad Request (HTTP 400).
2:53
This actually was something Bronwen ran into, and I also tried it and got the same error
2:56
we also found a gage on USGS that doesn't seem to exist to the package, but that's a separate and less important issue right now
Nick Santos 3:15 PM
is the gage new, or possibly lacking daily flow data?
3:16
the package keeps a list of info about the daily flow gages in CA - Ryan and I discussed missing gages as a possible hazard when he wrote that part of the gage code, but decided it was probably better than the alternative for now, I think
3:17
it'd be good for us to maybe provide a warning message if someone plugs in a nonexistent gage though
Alyssa Obester 3:17 PM
I don't think so, this is the gage: https://waterdata.usgs.gov/ca/nwis/inventory/?site_no=11118501
3:17
I think a warning would be helpful
Nick Santos 3:17 PM
that says something like "can't find gage - if you're sure it's in California and provides daily flow data, then this package may need an update - please file an issue at ____"
Alyssa Obester 3:17 PM
yes yes
Nick Santos 3:18 PM
we may be able to code around it and use the package internal data as a starting point and then try the internet services if it can't find it internally, but most of that probably isn't high on the todo list - and adds complexity - might just be worth always going to the internet, even if it's a bit slower
3:19
other failures will also need error messages, such as the san antonio creek gage - I think that's a data availability problem - probably need to do some sanity checks on the percentile data and make sure to only run appropriate alteration comparisons
3:19
huh, yeah, that is definitively not a new gage
Alyssa Obester 3:20 PM
oh interesting
Nick Santos 3:21 PM
we've had issues with gages with low or intermittent data - if the data was spotty, our comparisons ran into issues - we fixed at least one issue like that, which caused processing to fail, but probably need to figure out what's required to make san antonio creek work, and then have a test for it
3:21
OK, I'm going to file an issue with this conversation as the body of it
Alyssa Obester 3:21 PM
there's also another gage that's right on top of where that gage should be (11118501)
3:21
but apparently it's not the same one
Nick Santos 3:21 PM
huh - that's odd
3:22
did the one on top of it replace it?
Alyssa Obester 3:22 PM
oops I meant to paste 11118500
Nick Santos 3:23 PM
huh, it's older and lasts longer for daily discharge
Alyssa Obester 3:23 PM
I don't think it replaced it