Download sample set metadata and then cannot upload it again
Hello Terra Team,
I am trying to slightly modify metadata for sample_set. However, after downloading it and trying to upload back sample_set_entity.tsv file (even if that file was unmodified). I always get this error:
Here are how sample_sets look:
So I suspect this is because there are 100 long gs:// paths strings. So I am not sure why it can hold 100 samples when it is generated by method but not when I am trying to upload it as metadata.
Anyway, my issue is I am trying to merge metadata for those 2 sample sets (Breast_100 & Breast_26). We have to run workflows in chunks as DRS cannot handle a large number of requests.
Thank you,
Luda
Comments
9 comments
Hi Luda,
Thanks for reporting this. Would you be able to share the workspace in question where you are receiving this error with GROUP_FireCloud-Support@firecloud.org? Please provide us a link of the workspace after sharing if so.
Kind regards,
Jason
Hello Jason,
Unfortunately, I cannot share that particular workspace. But I shared another workspace:
So if you try to download sample_set metadata and then upload file sample_set_entity.tsv you will get this error:
Hello Jason,
I was wondering if you had any success replicating this error and if there is a solution for it we could use.
Thank you,
Luda
Hi Luda,
I have not yet had a chance to take a look. Apologies for the delay here, we are experiencing a very high volume of support requests at present. I will take a look as soon as I am possibly able.
If this is particularly urgent, please let me know and I will try to get to it sooner!
Kind regards,
Jason
Hi Luda,
Just so I'm clear, do you also see this issue when you try uploading through the Terra UI?
Kind regards,
Jason
Yes, I do.
Hi Luda,
Thank you for confirming. I am discussing this issue with one of our engineers now.
Kind regards,
Jason
Hi Luda,
The reason why this table gets updated without issue when writing from workflows but does have issue when uploading a TSV is because updating it through workflows doesn't utilize the TSV parser, which is where the restriction is at the moment. Our engineers are currently brainstorming a solution for this restriction and I will get back to you as soon as I hear more.
Kind regards,
Jason
Hi Luda,
At this time, the only way to get around the .tsv limit would be to make your changes using an API call: https://api.firecloud.org/#!/Entities/update_entity
You mentioned you are trying to make a slight modification. What kind of change are you looking to make? We may be able to help you in figuring out how to construct the right call.
Kind regards,
Jason
Please sign in to leave a comment.