I have two Jupyter notebooks which I've tried running simultaneously from the same Workspace environment by launching two separate tmux sessions. The notebooks are essentially identical, just use different input file lists, and are both using hail to pull GWAS vcf files from a remote repo, process them and upload into an s3 bucket.
When I launch one notebook instance from the terminal I use: jupyter nbconvert --stdout --to markdown --execute notebook_1_name.ipynb &
and it works fine. However, when I detach the current tmux session, launch a new one and launch the second notebook, both runs grind to a halt.
When I look at the hail log files I see this:
2021-06-16 15:56:15 YarnScheduler: WARN: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
Please sign in to leave a comment.