Is there a way to incorporate "gsutil -m" for the localization of the input files for a Terra method?

Post author
Seunghun Han

Hi, I was reviewing a log from a method I ran which takes in array file input, and I saw a following message in the log

- NOTE: You are performing a sequence of gsutil operations that may run significantly faster if you instead use gsutil -m cp ... Please see the -m section under "gsutil help options" for further information about when gsutil -m can be advantageous

So I was wondering if there is a way to enable the use of gsutil -m feature during the localization of the input files when running Terra method. I think it would greatly save time and cost especially in case when dealing with hundreds of files in large size







  • Comment author
    Samantha (she/her)

    Hi Seunghun Han,

    Thank you for writing in about this issue. Can you share the workspace where you are seeing this issue with by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.

    1. Add to the User email field and press Enter or click the Add button
    2. Click Save.

    Please provide us with

    1. A link to your workspace
    2. The relevant submission ID
    3. The relevant workflow ID

    Please provide us with a link to your workspace. We’ll be happy to take a closer look as soon as we can!

    Kind regards,


  • Comment author
    Ray Jones

    I see this in my WDL workflows, as well, any time I have an Array[File] with several large files.  

    The -m flag to gsutil just allows parallel operation of copies ( and I think could be made the default for all localization/delocalization calls.


Please sign in to leave a comment.