You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To ultimately drop the reliance on a shared filesystem, we need to be able to use Pegasus File objects for our inputs and outputs. However, Pegasus by default is set up expecting the input and output files to be specified as job arguments. We would need to have a way of interpolating the local path to a file transferred by Pegasus on the execution machine into the parameter file ultimately passed to our executing script. That's it's own issue. For this issue, one of the Pegasus developers suggested that as a baby step we make the ___ckpt files themselves Pegasus Files. A job would depend on the Files corresponding the the checkpoint files of its "parent" jobs and its output would always be the File corresponding to its own checkpoint.
To ultimately drop the reliance on a shared filesystem, we need to be able to use Pegasus File objects for our inputs and outputs. However, Pegasus by default is set up expecting the input and output files to be specified as job arguments. We would need to have a way of interpolating the local path to a file transferred by Pegasus on the execution machine into the parameter file ultimately passed to our executing script. That's it's own issue. For this issue, one of the Pegasus developers suggested that as a baby step we make the ___ckpt files themselves Pegasus Files. A job would depend on the Files corresponding the the checkpoint files of its "parent" jobs and its output would always be the File corresponding to its own checkpoint.
See #23 for context.
The text was updated successfully, but these errors were encountered: