Workflow Cost Estimator - getting 404s from API

Post author
Chris Miller

I grabbed the Workflow Cost Estimator (from the BioData Catalyst collection) and copied it into my workspace (under leylab_terra1/ch_fusion_topmed). When I try to run it, using a default cloud profile, I'm getting multiple errors. The first is from the pip installation, which says

Successfully built terra-notebook-utils bgzip cli-builder . . .

But also dumps a bunch of (possibly ignorable?) errors:

ERROR: pip's dependency resolver does not currently take into account all 
the packages that are installed. This behaviour is the source of the
following dependency conflicts.
tensorflow-io 0.18.0 requires tensorflow-io-gcs-filesystem==0.18.0, which is
not installed.
explainable-ai-sdk 1.3.2 requires xai-image-widget, which is not installed.
tfx-bsl 1.3.0 requires absl-py<0.13,>=0.9, but you have absl-py 0.15.0 which
is incompatible.
. . .

If I ignore those and try to continue, when I run this block:

for s in list_submissions_chronological():
print(s['submissionId'], s['submissionDate'], s['status'])

I get a lengthy error that culminates in this 404:

HTTPError: 404 Client Error: Not Found for url: https://api.firecloud.org/api/workspaces/terra-61b00e16/ch_fusion_topmed/submissions

The same thing occurs if I try to manually drop a submission ID from my job list into the subsequent block:

HTTPError: 404 Client Error: Not Found for url: https://api.firecloud.org/api/workspaces/terra-61b00e16/ch_fusion_topmed/submissions/542ee6a6-444d-4d09-825f-bdc8c8f9667b

Is the API down? Am I doing something wrong? Any advice you can give would be appreciated!

Comments

5 comments

  • Comment author
    Chris Miller

    Update - by mucking around with the swagger API site and editing the notebook, I was able to add the following lines in the second block to manually define my project and workspace, and then I could go below and get some reporting to work as expected. 

    WORKSPACE_GOOGLE_PROJECT="leylab_terra1"
    WORKSPACE_NAME="ch_fusion_topmed"

    If that's necessary, seems like it should be explicitly defined on this page where the instructions are outlined: https://support.terra.bio/hc/en-us/articles/360037862771-How-much-does-my-workflow-cost-#h_01F9F5M9M83QTN1R341P3XY5GT

    Or maybe something is amiss, because it wasn't able to populate those correctly.  Either way, might be worth investigating!

    0
  • Comment author
    Jason Cerrato

    Hey Chris,

    Thanks for writing in about this issue. I'm going to try and replicate this in a clone of my own. I'll get back to you as soon as I can!

    Kind regards,

    Jason

    0
  • Comment author
    Jason Cerrato

    Hi Chris,

    I was able to replicate your issue. I'm going to reach out to the authors of this workspace to let them know so they can add a cell that allows you to define your project and workspace names just as you've done.

    Thanks again for reporting this!

    Kind regards,

    Jason

    (P.S. I'm going to move this post to our Featured Workspaces section since it's related to an issue with one of our featured workspaces!)

    1
  • Comment author
    Jason Cerrato
    • Edited

    Hey Chris,

    I tried running this notebook in a different, older workspace and it actually seemed to work as intended. It looks like this issue crops up when using the newer style of workspace where each workspace is associated with its own project.

    You can fix this by changing WORKSPACE_GOOGLE_PROJECT to WORKSPACE_NAMESPACE.

    I'll let the authors know about this so they can update the notebook in the featured workspace.

    Kind regards,

    Jason

    0
  • Comment author
    Chris Miller

    Yep, that seems to do the trick. Thanks for the speedy response, Jason!

    0

Please sign in to leave a comment.