add wclouser_fxa_db_counts job. Fixes DSRE-1779#295
add wclouser_fxa_db_counts job. Fixes DSRE-1779#295clouserw wants to merge 1 commit intomozilla:mainfrom
Conversation
clouserw
force-pushed
the
wclouser-fxa-db-counts
branch
from
d533d5c to
cba4b95
Compare
|
@akkomar for review please |
akkomar
left a comment
akkomar
left a comment
There was a problem hiding this comment.
There is a test failure that might have something to do with pytest setup. Since we don't really have meaningful tests here, I'd remove the skeleton files (suggested this inline).
More comments/suggestions inline.
| ] | ||
|
|
||
| # Insert rows into the new table | ||
| errors = client.insert_rows_json("mozdata.analysis.wclouser_fxa_health_db_counts", rows_to_insert) |
There was a problem hiding this comment.
This is fine in an MVP, but we can also parameterize this to put this data in accounts_backend_derived.
Note that tables created in analysis dataset now have default expiration of 6 months which can be changed manually.
There was a problem hiding this comment.
Thanks for the note on expiration. I'll update that on my table. I don't have strong feelings on where this goes, fwiw.
clouserw
force-pushed
the
wclouser-fxa-db-counts
branch
from
cba4b95 to
857818c
Compare
|
|
||
|
|
||
| @click.command() | ||
| @click.option("--bq_project_id", help="GCP BigQuery project id", show_default=True, default="moz-fx-fxa-prod") |
There was a problem hiding this comment.
Recognizing this is probably going to be replaced by something in bqetl: if this was planned to be running via airflow, moz-fx-data-shared-prod is a more appropriate project id to default to since airflow doesn't have roles/bigquery.jobUser access to the fxa projects.
|
We'll move this to bigquery-etl to make this simpler. I filed a draft PR: mozilla/bigquery-etl#6511 that I'll work on. |
3 participants
Checklist for reviewer:
referenced, the pull request should include the bug number in the title)
.circleci/config.yml) will cause environment variables (particularlycredentials) to be exposed in test logs
telemetry-airflow
responsibly.