Skip to content
/ arx Public

Bundles code and a job to run for local or remote execution.

License

Notifications You must be signed in to change notification settings

solidsnack/arx

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SYNOPSIS

          arx (-h | -[?] | --help)
          arx (-v | --version)
          arx shdat (-b <size>)? (-o <output file>)? < input
          arx shdat (-b <size>)? (-o <output file>)? <input file>+
          arx tmpx <option,archive>* (//+ <command> (//+ <option,archive>*)?)?

DESCRIPTION
       A UNIX executable is a simple thing -- a file the kernel can execute,
       one way or another, via an interpreter or directly as object code.
       Every executable induces a family of executions -- instances of
       execution with different command line arguments, with different files
       in the working directory and with different environment variables
       present.

       The arx tool captures the parameters of an execution and encodes them
       as an executable, making for easy, consistent transfer and repetition
       of a particular run. The generated executable ensures that each run
       occurs in a freshly allocated temporary directory, with only the
       desired files in scope; it uses traps to ensure the cleanup of this
       directory; and its format is a simple POSIX shell script, relying on
       just a few shell tools.

DEPENDENCIES
       The arx tool relies on the presence of sed, tr, date, head, tar,
       hexdump and sh. When unpacking tar archives, it may use the -j or -z
       (bzip2 and gzip, respectively) options of tar. Scripts have been tested
       with dash and the GNU tools as well as the sh implementation and
       userland tools that are part of busybox.

APPLICATION
       The tmpx subcommand of arx offers a variety of options for bundling
       code and a task to run. The shdat subcommand exposes the lower-level
       functionality of encoding binary data in a shell script that outputs
       that binary data, using HERE documents and some odd replacement rules
       for nulls.

       Scripts generated by tmpx and shdat may be fed to sh over STDIN to
       execute them. This can be helpful when using ssh and sudo to set up an
       execution context; for example:

          arx tmpx ... | ssh user@host.com sudo sh

       Scripts generated by tmpx will pass their arguments to the contained
       script or command. To pass arguments when piping to sh, use -s:

          arx tmpx ... | ssh user@host.com sudo sh -s a b c

       Some arguments to the generated script will be treated specially,
       namely, --extract, --no-rm and --no-run. Please see the section on
       Passing Arguments, below, for more information about these options.

ARX COMMANDLINE PROCESSING
       For all subcommands, when options overlap in their effect -- for
       example, setting the output with -o -- the rightmost option takes
       precedence.  Whenever -h, -? or --help is present on the command line,
       help is displayed and the program exits.

       When paths are specified on an arx command line, they must be
       qualified, starting with /, ./ or ../. This simplifies the command line
       syntax, overall, without introducing troublesome ambiguities.

TMPX
       The tmpx subcommand bundles together archives, environment settings and
       an executable or shell command in to a Bourne-compatible script that
       runs the command or executable in a temporary directory, after having
       unpacked the archives and set the environment.

       Any number of file path arguments may be specified; they will be
       interpreted as tar archives to include in bundled script. If - is
       given, then STDIN will be included as an archive stream. If no
       arguments are given, it is assumed that no archives are desired and
       only the command and environment are bundled.

       The temporary directory created by the script is different for each
       invocation, with a name of the form /tmp/tmpx-<timestamp>-<randomhex>.
       The timestamp format is %Y.%m.%dT%H.%M.%SZ, in UTC. One happy
       consequence of this is that earlier jobs sort ASCIIbetically before
       later jobs. After execution, the temporary directory is removed (or
       not, depending on the -rm[10!_] family of options).

          -rm0, -rm1, -rm_, -rm!
                 By default, the temporary directory created by the script
                 will be deleted no matter the exit status status of the task.
                 These options cause a script to be generated that deletes the
                 temporary directory only on success, only on failure, always
                 (the default) or never.

          --shared
                 Causes the temporary directory to be identified by a hash of
                 the ARX archive, instead of by date and time. Different runs
                 of the same archive will share the same directory. Note that
                 this implies shared state and every disadvantage thereof.

          -b <size>
                 Please see the documentation for this option, shared with
                 shdat, below.

          -o <path>
                 By default, the generated script is sent to STDOUT. With -o,
                 output is redirected to the given path.

          -e <path>
                 Causes the file specified to be packaged as the task to be
                 run. A binary executable, a Ruby script or a longish shell
                 script all fit here.

       In addition to these options, arguments of the form VAR=VALUE are
       recognized as environment mappings and stored away in the script, to be
       sourced on execution.

       Without -e, the tmpx subcommand tries to find the task to be run as a
       sequence of arguments delimited by a run of slashes. The following
       forms are all recognized:

          arx tmpx  ...some args... // ...command...
          arx tmpx  ...some args... // ...command... // ...more args...
          arx tmpx // ...command... // ...some args...

       The slash runs must have the same number of slashes and must be the
       longest continuous runs of slashes on the command line. The command
       will be included as-is in a Bourne shell script.

SHDAT
       The shdat subcommand translates binary data in to a shell script which
       outputs the binary data. The data is encoded in HERE documents in such
       a way that data without NULs is not changed and that data with NULs is
       minimally expanded: about 1% for randomish data like compressed
       tarballs and about 10% in pathological cases.

       The shdat subcommand can be given any number of paths, which will be
       concatenated in the order given. If no path is given, or if - is given,
       then STDIN will be read.

          -b <size>
                 The size of data chunks to place in each HERE document. The
                 argument is a positive integer followed by suffixes like B,
                 K, KiB, M and MiB, in the manner of dd, head and many other
                 tools. The default is 4MiB.  This is unlikely to make a
                 difference for you unless the generated script is intended to
                 be run on a memory-constrained system.

          -o <path>
                 By default, the generated script is sent to STDOUT. With -o,
                 output is redirected to the given path.

EXAMPLES

          # Installer script that preserves failed builds.
          git archive HEAD | bzip2 | arx tmpx -rm0 - // make install > go.sh
          # Now install as root; but don't log in as root.
          cat ./go.sh | ssh joey@hostname sudo /bin/sh

          # Variation of the above.
          git archive HEAD | bzip2 | arx tmpx -rm0 - -e ./build-script.py > go.sh

          # Bundle an instance of an application with DB credentials and run it.
          arx tmpx -rm! ./app.tbz ./stage-info.tgz // rake start | ssh ...

          # Get dump of linking info for build that works here but not there.
          arx tmpx ./server-build.tgz LD_DEBUG=files // ./bin/start | ssh ...

          # Test out Cabal source distribution of this package:
          arx tmpx // 'cd arx-* && cabal configure && cabal build' // \
                   -rm0 ./dist/arx-0.0.0.tar.gz | sh

PASSING ARGUMENTS TO GENERATED SCRIPTS
       The scripts generated by tmpx treat some arguments as special, internal
       options, to allow for inspecting them should there be a need to
       determine their contents.

          --extract
                 Unpack the data in the present directory and do nothing else.

          --no-rm
                 Run the script as normal but do not delete the generated
                 temporary directory.

          --no-run
                 Unpack into a temporary directory as normal but do not run
                 the user's command.

       To prevent arguments from being specially treated, use // in the
       argument list:

          a-tmpx-script.sh --no-rm // a b c --extract

       In the above example, --extract will be passed to the inner command, in
       the same way as a, b, c. The following example causes ab, c and --no-rm
       to be printed one after another, each on their own line.

          arx tmpx // printf "'%s\n'" '"$@"' | sh -s // ab c --no-rm

NOTES
       The timestamp is not the common ISO 8601 format, %Y-%m-%dT%H:%M:%SZ,
       because of software and build processes that attach special meaning to
       colons in pathnames.

BUGS
       The command line parser offers no hints or help of any kind; it fails
       with the simple message "argument error". The two most common mistakes
       I make are:

       • Not qualifying paths with /, ./ or ../.

       • Not specifying a subcommand (tmpx or shdat).

About

Bundles code and a job to run for local or remote execution.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •