Terra should support the push of workspace data to BigQuery
For a lot of the pipelines that our team runs regularly, it would be advantageous if there was an easy way to get the data out of the Terra / Firecloud data model, and into a remote storage location, such as a BigQuery database, to track any changes or fluctuations in metrics over time or across projects. At the moment is a little tedious, involving either a manual download and push to BigQuery. It would be nice if there was an interface built in to Terra to allow users to select specific metrics (columns) from the data table, and push them to a BigQuery database.
Comments
1 comment
Hey Carrie! I was curious, were you hoping to get metrics on the Data Delivery workspaces? If that's the case, it might be the type of metrics you can extract from Data Repo.
Please sign in to leave a comment.