Simplified Command Calling Convention

[kevsmith]

Overview

After living with our execution model for some time, it's become apparent that it is much more complex than it needs to be. It requires a matrix to reason about how command execution, variable binding, and permission enforcement works. Worse, it requires specific knowledge about whether the command is once or multi for users to know how their command lines are going to work.

We are going to get rid of calling_convention, execution, and enforcing in favor of more consistent behavior.

Multi all the things!

All commands will become multi commands. That is, they will be run once for each result from the previous stage of the pipeline. In order to support commands that need to be able to consume the aggregate output of the previous stage of the pipeline, we'll do a few things:

• Add additional context to command execution to let commands know whether they are the first or last step in the pipeline stage. I called this COG_STAGE_POS in the examples below for illustration purposes, but this name is subject to change.
• Provide a way for commands to accumulate data from each execution to allow the use cases that were previously handled by once commands.
• Allow command executions to return no results.

First? Last? Why would commands care?

The first execution may have some state to initialize, and the last may want to accumulate data that was stored by previous stages to process in aggregate. The table command is a good example of a command which would want to accumulate its inputs and process them in a single batch.

Details

Calling Convention: Yes

Deprecate calling_convention. Bindings will be evaluated for every command. Additionally, every command will receive the unfiltered JSON response context from the previous stage of the pipeline. For foreign commands, this will be delivered via STDIN.

When do we enforce? Always.

All commands will require rules. We will adjust the rule language to support world executable commands. We may still have an enforcing column in the database as an optimization hint so that we know that we don't have to evaluate permissions, but this is purely an internal implementation detail and not exposed to users.

Proposed syntax: when command is bundle:command allow

Beyond simplification, this change allows Cog users to require permissions for commands, regardless of the bundle configuration that was defined by the bundle author.

Indicate first and last inputs

See First? Last? Why would commands care? for motivation.

Note: Environment below is just the relevant subset, we're still passing the other standard environment variables like COG_PIPELINE_ID, COG_CHAT_HANDLE, and so on.

mycommand (1 of 3):

COG_ARGC=1
COG_ARGV_0="myarg"
COG_OPTS="key"
COG_OPT_KEY="one"
COG_STAGE_POS="first"

STDIN:
{"key": "one"}

mycommand (2 of 3):

COG_ARGC=1
COG_ARGV_0="myarg"
COG_OPTS="key"
COG_OPT_KEY="two"

STDIN:
{"key": "two"}

mycommand (3 of 3):

COG_ARGC=1
COG_ARGV_0="myarg"
COG_OPTS="key"
COG_OPT_KEY="three"
COG_STAGE_POS="last"

STDIN:
{"key": "three"}

Depends on: #396