Best way to specify a directory input?

Post author
Bryan Cosca

Hi!

I was wondering what's the best way of specifying a directory input of files in a task. I see a pull request with it being implemented but I don't see it in the spec documentation. I can think of a couple ways, using structs or tarring the directory and unzipping it at runtime. Is there another way?

Thanks,

Bryan

Comments

1 comment

  • Comment author
    Tiffany Miller

    Hi Bryan,

    Yes, those are definitely ways. Structs are used for bundling groups of variables into a single variable so that it's easier to pass them around as task inputs. Another option is creating a list of file paths and passing it to the task to be localized by gsutil as described here and in the spec here.
    Keep in mind that using these options will require the whole files to be localized instead of being streamed to the task. If your tool(s) are able to stream the files, that would be a good option as well. 

    Many thanks,
    Tiffany

    0

Please sign in to leave a comment.