Comments

5 comments

  • Sushma Chaluvadi

    Thanks Elena, will keep you posted on status of the request.

    0
    Comment actions Permalink
  • Sushma Chaluvadi

    Elena,

    Would you be able to provide a bit more context on how you would like to use the ability to have multiple runtime environments per billing-project in Terra? We are trying to understand what you find is lacking in current functionality that would be resolved with this Feature to see if there are better methods or modifications to current methods.

     

    Sushma

    0
    Comment actions Permalink
  • Benjamin Doran

    Not sure if this is the reason Elena wants multiple run times.

    But for me a big use case, and part of our reason for moving into the cloud is that our analyses can take a long time to run. It is really common to set a script/notebook running and then in another kernel work on another aspect that needs more interaction.

    Having everything (installed packages, data, scripts) in the same environment is functional but not optimal. I often have to use larger environments than are usually necessary for the analysis because I need bandwidth to run the larger analysis, along with the smaller ones. If we had multiple environments we could use the resources necessary for that particular analysis rather than needing to over estimate. 

    These longer jobs don't really work as workflows (wdl) because for the most part you are just trying something out that you might never run again, and writing the wdl has too much boilerplate to easily test new things and drastically refactor every day. I also have tried dsub which seemed like a nice middle ground between writing a workflow and testing smaller things in jupyter notebooks, but at the time I tried it, it was not well integrated to the terra ecosystem
    1
    Comment actions Permalink
  • Sushma Chaluvadi

    Thanks Benjamin, I have passed your input along to the team for further investigation.

    0
    Comment actions Permalink
  • Elena Torlai Triglia

    I agree with what Benjamin wrote. The case use for me as well would be to run scripts/ notebook on parallel kernels  - with a single environment, I need to estimate the 'total' resources needed rather than being able to individually allocate them.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk