Need Help?

Search our documentation and community forum

Terra is a cloud-native platform for biomedical researchers to access data, run analysis tools, and collaborate.
Terra powers important scientific projects like FireCloud, AnVIL, and BioData Catalyst. Learn more.

very, very long lags in retrieving cached results

Comments

11 comments

  • Avatar
    Haley J Abel

    For example, this entire workflow (up to and beyond the current point) should have been cached but for some reason the 'Sort_Index_VCF_DEL' step (which has already been run and cached) has taken over 5 hours. 

     

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Haley,

    Thank you for writing in about this. It looks like Cromwell started experiencing some slowness around this time yesterday. I'm going to chat with the engineers to get a better sense of what's going on and get back to you ASAP.

    Can you share the submission and workflow IDs for this affected job (and any others you notice)?

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Haley J Abel

    I can find some workflow ids if needed, but the problems seems improved now.

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Haley,

    If things are looking better, no need to share. Many thanks for letting me know!

    If things start looking weird again please feel free to flag it up.

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Edyta Malolepsza

    Hi Jason,

    My jobs experience similar problem since yesterday evening. Here is an example of a job that was finished at 4:03 pm and still "is running":

    Thanks for your help.

    Best regards,

    Edyta

    0
    Comment actions Permalink
  • Avatar
    Edyta Malolepsza

    An update: my jobs that finished around 4 pm are still shown as running and the data model was not yet populated with these results.

    0
    Comment actions Permalink
  • Avatar
    Edyta Malolepsza

    My jobs are in the same spot as yesterday, in running mode.

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Edyta,

    Thank you for writing in about this. We are still investigating the delays in submission and results retrieval. You can follow this article for the latest updates: https://support.terra.bio/hc/en-us/articles/360042960371-Service-Incident-April-30-2020

    This is the same article that links from the banner in the Terra interface. I will also update this thread when I receive word on resolution.

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Edyta Malolepsza

    Hi Jason,

    Thank you and good luck with solving the problem.

    Best,

    Edyta

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Edyta,

    We are seeing queues having returned to normal. I've sent you a message directly to follow up on your issue in case it persists.

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Edyta Malolepsza

    Hi Jason,

    I shared my workspace right after receiving your direct message. My jobs are still in the running mode. I will abort them.

    Best,

    Edyta

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk