Subcommands

[thekid]

Scope of Change

This RFC intends to change the XP runners to a single entry point with multiple subcommands. E.g., instead of writing unittest src/test/php, we'd write xp test src/test/php.

Rationale

There are a couple of problems with the current implementation, especially in conjunction with the splitting of the framework:

• Running unittest will not work if you don't have the unittest library in your class path once Remove unittests from core core#95 is merged. You can already see this with xcc, for example.
• Even if you have unittest in your class path, the command will not work because the unittest runner classes are loaded too late on.
• We do not integrate well with Composer's scripts as our runners really are platform-specific executables and need an installer to work well.
• Creating new runners, e.g. for the measure library, means introducing a new shell and a C# implementation, plus putting it in a rather proprietary place.
• We might run out of good names for command line utilities as we seek to find memorizeable names which don't conflict with existing tools.

Functionality

General idea:

Basic functionality

$ xp run Test
$ xp ar cvf app.xar src/main/php=.
$ xp ar xvf app.xar
$ xp eval {code}
$ xp version
$ xp write {code}
$ xp dump {code}

From other packages

$ xp reflect {ref}           # = xp lang.mirrors.Inspect {ref}
$ xp test unittest.ini       # = unittest unittest.ini
$ xp serve .                 # = xpws -r .
$ xp measure src/prof/php    # = xp xp.measure.Runner src/prof/php

Providing subcommands

Libraries should be able to provide subcommands in an easy way. Inside a project, we usually state our dependencies by using Composer, therefore being able to provide subcommands via its infrastructure would be awesome.

These kinds of subcommands exist:

1. The entry point class. This is the easiest of all subcommands. Running "xp test ...", which invokes the subcommand named xp-test, for example, is the same as running "xp unittest.Runner ..."
2. The foreground process. For example, "xp serve" (the new "xpws") needs to keep the process running until a certain event, in this case a key press by the user.
3. The daemon. Something like the service system; see XPWS Service Mode xp-runners#28.

Security considerations

Speed impact

Dependencies

The form xp Test which runs Test::main($argv) must continue to work, as an alias for xp run Test.

Related documents

• https://golang.org/doc/articles/go_command.html and https://golang.org/cmd/go/
• XP Runners #166 - the original XP Runners RFC

[thekid]

Can vendor binaries solve the "easy to provide" requirement? Here's how it works for PHPUnit:

Definition by PHPUnit

composer.json

{
   "name": "phpunit/phpunit",
    ...
    "bin": [
        "phpunit"
    ],
    "config": {
        "bin-dir": "bin"
    }
}

phpunit

#!/usr/bin/env php
// ...
foreach (array(__DIR__ . '/../../autoload.php', __DIR__ . '/vendor/autoload.php') as $file) {
    if (file_exists($file)) {
        define('PHPUNIT_COMPOSER_INSTALL', $file);
        break;
    }
}

if (!defined('PHPUNIT_COMPOSER_INSTALL')) {
    echo 'You need to set up the project dependencies using the following commands:' . PHP_EOL .
        'wget http://getcomposer.org/composer.phar' . PHP_EOL .
        'php composer.phar install' . PHP_EOL;
    die(1);
}

require PHPUNIT_COMPOSER_INSTALL;

PHPUnit_TextUI_Command::main();

Generated code

vendor/bin/phpunit

#!/usr/bin/env sh
SRC_DIR="`pwd`"
cd "`dirname "$0"`"
cd "../phpunit/phpunit"
BIN_TARGET="`pwd`/phpunit"
cd "$SRC_DIR"
"$BIN_TARGET" "$@"

vendor/bin/phpunit.bat

@ECHO OFF
SET BIN_TARGET=%~dp0/../phpunit/phpunit/phpunit
php "%BIN_TARGET%" %*

👎 Won't work directly for two reasons:

1. We need to influence the command line options to PHP itself, we're out of luck (except if we fork PHP from inside PHP:/)
2. PHP is hardcoded, this doesn't work if PHP isn't in $ENV{PATH} or if we want to use another binary, e.g. "/home/thekid/devel/php-src/sapi/cli/php" or "/usr/bin/php7"

[thekid]

Inner workings

This is how an XP runner works in pseudo-code.

##
# Returns the command line via xp.ini / environment
# E.g. /usr/bin/php5.5 -dinclude_path=src/main/php -ddate.timezone=Europe/Berlin
cmdline(args):
    cmd = find_exe($config.rt | $ENV{XP_RT} | "php")
    cmd += include_path(from: $config.use_xp | $ENV{USE_XP} | ".")
    cmd += ini_settings(from: $config)
    <- cmd

## 
# Returns the runner based on ??? - currently always "!default", xpws
# uses "!wait" (and maybe "!daemon" in the future).
runner():
    if (???):
        <- new Runner.default(class, run(proc) = {
            proc.start()
            proc.wait()
            <- proc.exit
        })
    else if (???)
        <- new Runner.wait(class, run(proc) = {
            proc.start()
            Console.readln()
            proc.kill($SIG{TERM})
            <- proc.exit
        })
    else if (???)
        <- new Runner.daemon(class, run(proc) = {
            switch (subcommand):
                start: proc.pid >>> file($name.pid)
                status: Console.writeln(file($name.pid) | "Not running")
                stop: Process.get(file($name.pid)).kill()
            <- 0
        })
    <- "Unknown runner type"

##
# Entry point -> command line
# E.g. "class-main.php xp.reflect.Command"
entry(runner):
    cmd = ""
    cmd += runner.entry_point         # Either "class-main.php" or "web-main.php"
    cmd += runner.entry_class         # E.g. xp.scriptlet.Runner
    <- cmd

##
# Execute command / args, e.g. "xp test src/test/php" = "test", ["src/test/php"]
# Returns exitcode
execute(subcommand, options, arguments)
    runner = runner(subcommand)

    <- runner(new Process(cmdline(options) + entry(runner) + arguments)

Basic subcommand parsing

Before invoking execute(), the command line is parsed:

$ xp -cp lib run Test 1 2 --verbose
# subcommand := "run"
# options    := [ "-cp" => lib ]
# arguments  := ["Test", "1", "2", "--verbose"]

Next, locate "xp-run" subcommand, invoke it w/ options and arguments. If no subcommand can be found, default it to "xp-run" (BC case, see above!)

Possible implementation in C#

Challenges

We need to get this right

• Where should it look for subcommands?
• When it finds them, which execution model to use?
• Work across platforms
• Be as fast as possible!!!

[thekid]

💡

So maybe the ??? could be determined by parsing the line in the .bat file:

SET BIN_TARGET=%~dp0/../xp-framework/unittest/xp-test
               ^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^
               |     |                        |      
               |     |                        subcommand
               |     combined path
               vendor dir reference

The file "xp-test" itself would contain the necessary configuration, which the runner would parse. If invoked directly by PHP, it could display an error message: "please invoke by using xp $command".

⚠️ We're relying on a specific way Composer generates the .bat files...

[thekid]

💡

The other idea is to compile the runner on first use, so using xp test src/test/php would check through the class path, see where it can find a xp-test file, and create a subcommand on the fly in vendor/bin.

xp-test.ini:

runner=default

[default]
class=unittest.cli.Runner

From this we could generate code, compile it (saving it to the filesystem for the next run) and finally include it:

C# implementation

var declaration = new Net.XpFramework.Runner.Ini(subcommand + ".ini");
var unit = new CodeSnippetCompileUnit(@"
    using System;
    using System.Collections.Generic;

    class {{name}} : Default
    {
        public {{name}}(string name): base(name) { }

        override public string Class() { return ""{{class}}""; }
        override public string Entry() { return ""class""; }
    }
    "
    .Replace("{{name}}", subcommand)
    .Replace("{{class}}", declaration.Get("default", "class"))
);

var parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("System.dll");
parameters.ReferencedAssemblies.Add("Runner.dll");
parameters.GenerateExecutable = false;
parameters.OutputAssembly = subcommand + ".dll";

var provider = new CSharpCodeProvider();
var results = provider.CompileAssemblyFromDom(parameters, unit);

if (results.Errors.Count > 0)
{
    Console.Error.WriteLine("*** Cannot compile `{0}'", subcommand);
    foreach (var error in results.Errors)
    {
        Console.WriteLine("    {0}", error.ToString());
    }
    return 0xff;
}

Bash implementation

TODO

[thekid]

_There is now a reference implementation available at https://github.com/xp-runners/reference_.

Re the above ideas: Its plugin architecture use Composer scripts' filenames to determine the necessary information.

• vendor/bin/xp.xp-framework.unittest := xp.unittest.Runner from xp-framework/unittest
• vendor/bin/xp.xp-framework.unittest.test := xp.unittest.TestRunner from xp-framework/unittest

[thekid]

_The first official plugin subcommand is now xp test- via https://github.com/xp-framework/unittest/releases/tag/v6.8.0_

Installation:

Timm@slate ~
$ composer global require xp-framework/unittest
Changed current directory to C:/Users/Timm/AppData/Roaming/Composer
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Removing xp-framework/unittest (dev-master 7d2659b)
  - Installing xp-framework/unittest (v6.8.0)
    Downloading: 100%

Writing lock file
Generating autoload files

Invoking: