Core Communication Modules
The language and action modules are bridged by shared action protocols we call n-tuples. Both the Problem Solver and Specializer have access to n-tuple templates, which are JSON structures oriented towards particular tasks, action, and information retrieval. The Specializer uses information from the SemSpec to fill in a template; the filled-in template is called an n-tuple. This is then transported over a network to the Problem Solver.
See the Transport documentation for more information.
The Core Agent defines a class that has the ability to send and receive n-tuples. Both the Core UI-Agent and the Core Problem Solver subclass the Core Agent.
A Core Agent is designed to be easy to initialize. The only required argument is the desired name of the channel, e.g.:
python core_agent.py Agent1
The above command would create a CoreAgent with a channel name of "Agent1". All CoreAgents have a Transport object as a field attribute, as well as:
- Name ("Agent1")
- Federation (by default, this is set to "FED1")
- Address (the concatenation of a federation and a name, such as "FED1_Agent1")
The federation is an important attribute, since it helps avoid potential name collisions, in the event of running multiple same-named agents on the same network. The federation can be set with an environment variable, e.g.:
export $ECG_FED=FED1
Optionally, the user can also designate a log-file and a log-level, in which to log errors or messages.
read_templates(self, filename)
This method reads in a JSON file from the given filename, and returns an OrderedDict of templates. Since a user can specify an inheritance hierarchy for templates, this method also updates a "child" with the key/value pairs of its "parent", unless a particular key/value pair is overridden. If a specified parent does not exist, the method raises a TemplateException, which should cause the program to quit (as this is almost certainly an error).
Whenever a CoreAgent subscribes to a particular channel, it passes in a callback method, which is called whenever the CoreAgent receives an n-tuple from that channel name. The CoreAgent defines a simple callback method, which just prints out the received n-tuple. Both the UI-Agent and Problem Solver override this callback method as part of their default behavior. Additional callbacks can be written for different purposes, and to subscribe to different channels.
N-tuple templates serve two primary functions in our system:
- Declarative instructions: While building an n-tuple, the templates act as declarative specifications for which aspects of the Semantic Specification the Core Specializer should extract, and give instructions on how the output should be formatted.
- Shared semantics: Templates are oriented towards particular tasks, and are shared between the Specializer and Problem Solver, which establishes a shared vocabulary and semantics between a human user and a robot agent (or other task).
These are high level templates (file), which map the “mood” of a SemSpec onto a given template. The mood corresponds to a given predicate_type.
For example, a Yes/No question like "is the green box near the blue box?" will match to the "yn_question" template.
These are also high level templates (file), which correspond roughly to the Event-Descriptor schemas in ECG. They include event-features (such as modality information like “can”, temporal information like duration and telicity, and whether or not an event is negated), as well as other embedded templates, which are either Parameter Templates, Descriptor Templates, or more event templates. For example, a conditional command like “Robot1, if the box near the green box is red, push it south, otherwise push it north!” would correspond to the conditional EventDescriptor template
The portions of the sentence would map roughly as follows:
condition: {the box near the green box is red}
conclusion: {push it south}
alternative: {push it north}
complexKind: conditional
conditionalValue: bounded
Of course, the contents of the condition/conclusion/alternative roles will also be encoded via n-tuples, using the parameter templates (see below).
These templates (file) correspond roughly to Process schemas in ECG, though importantly, non-Process schemas can be described here, such as schemas/frames for concepts like Possession or Spatial-Relations. For each entry in a parameter template, such as the Core Specializer calls fill_value.
A simple example is the "Perception" template, which is used for sentences like "John saw the house." Here, we see that it inherits all the roles from "Process", and adds a role for content (e.g., the object of perception).
These templates are ways of describing referents, locations, etc., so that the Problem Solver can identify the actual referent. Language is always under-specified, and the “meaning” of a SemSpec is only realized through context and simulation. While the Specializer can identify that the human was referring to “the box”, it has no way of knowing which box the human is referring to; this is the Problem Solver’s job.
For example, an object-descriptor includes information from the RD schema (such as ontological-category, gender, number, etc.), but also includes information encoded elsewhere in the SemSpec, such as in a Modification schema. The [template](https://github.com/icsi-berkeley/framework_code/blob/master/src/main/nluas/descriptors.json# 3-23) defines the list of RD entries to include, as well as a list of acceptable pointers or modifiers of that RD to incorporate into the descriptor.
See the documentation on the get_objectDescriptor method for more information.
The Core Specializer uses the fill_value method to fill a particular entry in a template. By default, there are several types of declarative instructions that the Core Specializer can already handle. These template types should be general enough so that fill_value doesn't need to be overridden, though new templates themselves will certainly be needed in new applications.
Note: A template can subclass another template using the formalism:
parents: ['PARENT1', 'PARENT2', ...]
All of the key/value pairs from each parent are added, in order, to the child template, unless that key already exists in the child. If a named parent does not exist, an exception is thrown.
state: {method: get_state}
This calls the get_state method on the contents of the state role. More generally, this is expressed as:
{ROLE}: {method: {METHOD_NAME}}
protagonist: {descriptor: objectDescriptor}
This calls the get_objectDescriptor method on the contents of the protagonist role. More generally, this is expressed as:
{ROLE}: {descriptor: {DESCRIPTOR_TYPE}}
Note: One nice thing about the descriptor procedure is that new descriptors can be defined (e.g., "robotDescriptor"), without changing the fill_value method; the user would just have to define a new method, called get_robotDescriptor.
Note 2: Additionally, "default" values can be added to a descriptor, e.g.:
protagonist: {descriptor: objectDescriptor, default: {type: robot}}
If no information is found from the SemSpec, the default value is used.
eventProcess: {parameters: eventProcess}
This calls the fill_parameters method on the "eventProcess" role. This allows the Core-Specializer to represent the compositionality and nesting that is present in the SemSpec. More generally:
ROLE: {parameters: {ROLE}}
condition: {eventDescription: condition}
This calls the specialize_event method on the "condition" role. Again, this allows the Core-Specializer to represent the compositionality and nesting that is present in the SemSpec. More generally:
ROLE: {eventDescription: {ROLE}}
actionary: actionary
Finally, the Core Specializer can simply extract the type or value of a role. This is usually the ontology value (e.g., @run), but it can also include Strings ("Declarative") or scalar-values (.5).
More generally, this is expressed as:
ROLE: ROLE