Merge pull request #549 from OpenSimulationInterface/migration

Migration of documentation to AsciiDoc

Author: Stefan Cyliax

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "asciidoc-resources"]
+	path = asciidoc-resources
+	url = https://code.asam.net/simulation/asciidoc-resources.git

asciidoc-resources

@@ -0,0 +1,41 @@
+= Overview of OSI architecture
+
+OSI contains an object-based environment description that uses the message format of the https://github.com/protocolbuffers/protobuf/wiki[Protocol Buffer] library.
+The Protocol Buffer library was developed and is maintained by Google.
+OSI defines top-level messages that are used to exchange data between separate models.
+Top-level messages define the `GroundTruth` interface, the `SensorData` interface, and – since OSI version 3.0.0 – the interfaces `SensorView`, `SensorViewConfiguration`, and `FeatureData`.
+
+The following figure shows the interfaces and models involved in modeling a sensor.
+
+.Open Simulation Interface overview
+image::{images_open_simulation_interface}/osi-context.png[1100]
+
+
+OSI also defines interfaces for traffic participant models.
+The `TrafficCommand` interface makes it possible to send commands to traffic participant models.
+The `TrafficUpdate` interface makes it possible to receive the updated state from traffic participant models.
+The following figure shows the interfaces of a generic traffic participant.
+
+.Interface of a traffic participant
+image::{images_open_simulation_interface}/osi-traffic-participant-principle.png[1100]
+
+Traffic participant models may use other OSI interfaces internally, for example, to model autonomous vehicles.
+The following figure shows a more advanced use case for traffic participants.
+
+.Traffic participant with sensor models, AD function, and dynamic model
+image::{images_open_simulation_interface}/osi-traffic-participant-advanced.png[1100]
+
+The `HostVehicleData` interface describes the measured internal states of a traffic participant.
+OSI currently provides only limited support for data structures that describe measured internal states of traffic participants.
+Actuator intentions are currently not covered by OSI and must be handled with a different data description format.
+
+All fields in an interface are set to `optional`.
+`required` is not used.
+This has been done to allow backward-compatible changes in the field.
+Additionally, this is the default behavior in Protocol Buffer version 3 that does no longer have the `required` type.
+Setting all fields to `optional` thus ensures update compatibility.
+However, this does not mean that filling the field is optional.
+For the purpose of providing a complete interface, all existing fields should be set, unless not setting a field carries a specific meaning as indicated in the accompanying comment.
+
+All field numbers equal to or greater than 10000 are available for user-specific extensions via custom fields.
+Therefore, no future evolution of OSI will use field numbers equal to or greater than 10000.

doc/architecture/architecture_overview.adoc

@@ -0,0 +1,14 @@
+= Data layer
+
+The data layer of OSI is defined in the message specifications using the ProtoBuf IDL.
+It defines the data that can be transmitted using OSI, including the structure and the semantics of the data.
+
+Additionally, it specifies the encoding to be used when OSI data is transmitted.
+Currently, ProtoBuf encoding is used, but other encodings are possible with the ProtoBuf IDL.
+FlatBuffer encoding has been implemented as an experimental feature.
+
+The data layer does not directly define components and transmission routes.
+These are defined in the packaging layer of OSI.
+There may be different packaging layer implementations using the shared data layer definitions.
+The data that is exchanged remains compatible regardless of the packaging layer implementation.
+The use of a shared data layer ensures easy bridging between different packaging layer implementations.

doc/architecture/data_layer.adoc

@@ -0,0 +1,9 @@
+= Environmental effect model
+
+Environmental effect models consume sensor-view messages and produce sensor-view messages.
+Environmental effect models may, for example, alter sensor-view messages to include effects and phenomena caused by:
+
+* Shadows and occlusions
+* Weather effects
+* Physics of a sensor
+* Pre-processing of raw sensor data

doc/architecture/environmental_effect_model.adoc

@@ -0,0 +1,5 @@
+= Feature data
+
+Feature-data messages contain detected features in the reference frame of a sensor.
+Feature-data messages are generated from ground-truth messages.
+They serve, for example, as input to sensor models simulating object detection or feature fusion models.

doc/architecture/feature_data.adoc

@@ -0,0 +1,52 @@
+= Trace-file formatting scripts
+
+The OSI repository contains Python scripts for converting trace files from one format to the other.
+The formatting scripts are stored in `open-simulation-interface/format/`
+
+**txt2osi.py**
+
+`txt2osi.py` converts plain-text trace files to binary `.osi` trace files.
+This script takes the following parameters:
+
+`--data`, `-d`::
+String containing the path to the file with serialized data.
+
+`--type`, `-t`::
+Optional string describing the message type used to serialize data.
+Allowed values are `'SensorView'`, `'GroundTruth'`, or `'SensorData'`.
+The default value is `'SensorView'`.
+
+`--output`, `-o`::
+Optional string containing the name of the output file.
+The default value is `'converted.osi'`.
+
+`--compress`, `-c`::
+Optional boolean controlling whether to compress the output to a lzma file.
+Allowed values are `True`, or `False`.
+The default value is `False`.
+
+**osi2read.py**
+
+`osi2read.py` converts trace files to human-readable `.txth` trace files.
+This script takes the following parameters:
+
+`--data`, `-d`::
+String containing the path to the file with serialized data.
+
+`--type`, `-t`::
+Optional string describing the message type used to serialize data.
+Allowed values are `'SensorView'`, `'GroundTruth'`, or `'SensorData'`.
+The default value is `'SensorView'`.
+
+`--output`, `-o`::
+Optional string containing the name of the output file.
+The default value is `'converted.txth'`.
+
+`--format`, `-f`::
+Optional string containing the format type of the trace file.
+Allowed values are `'separated'`, or `None`.
+The default value is `None`.
+
+**Related topics**
+
+* <<#top-1a2f4b0c-195c-4f18-89ad-d48a123bd8c1>>[OSI trace file formats]

doc/architecture/formatting_scripts.adoc

@@ -0,0 +1,5 @@
+= Ground truth
+
+Ground-truth messages describe the simulated environment containing all simulated objects in the global coordinate system at consecutive time instances.
+It is based on data available to the simulation environment.
+Typically, ground-truth messages are contained in sensor view messages.

doc/architecture/ground_truth.adoc

@@ -0,0 +1,6 @@
+= Logical model
+
+Logical models consume sensor-data messages and produce sensor-data messages.
+
+An example of a logical model is a sensor-fusion model, which combines the output of multiple sensor models to produce data with less uncertainty.
+Another use case is the fault-injection model which, contrary to a sensor-fusion model, may be used to increase uncertainties.

doc/architecture/logical_model.adoc

@@ -0,0 +1,13 @@
+= Packaging layer
+
+The packaging layer of OSI specifies how components that use the OSI data layer, for example, sensor models, are packaged for exchange.
+
+It specifies model types and their mandatory and optional OSI inputs, OSI outputs, and parameter interfaces.
+A model type may be, for example, a sensor model or a traffic participant model.
+The packaging layer also specifies component technology standards.
+This makes it possible to encapsulate model types in easily exchangeable component packages that can be used across platforms and implementations.
+
+Multiple packaging layer implementations are possible within the OSI framework.
+The shared data layer ensures easy bridging between the different implementations.
+The currently defined central packaging layer is the OSI Sensor Model Packaging (OSMP) specification.
+It is based on FMI 2.0 and uses certain additional conventions to allow packaging of OSI using models as FMUs.

doc/architecture/packaging_layer.adoc

@@ -0,0 +1,78 @@
+= Protobuffer files
+
+TODO: Add general description.
+
+osi_common.proto::
+TODO: Add description.
+
+osi_datarecording.proto::
+TODO: Add description.
+
+osi_detectedlane.proto::
+TODO: Add description.
+
+osi_detectedobject.proto::
+TODO: Add description.
+
+osi_detectedoccupant.proto::
+TODO: Add description.
+
+osi_detectedroadmarking.proto::
+TODO: Add description.
+
+osi_detectedtrafficlight.proto::
+TODO: Add description.
+
+osi_detectedtrafficsign.proto::
+TODO: Add description.
+
+osi_environment.proto::
+TODO: Add description.
+
+osi_featuredata.proto::
+TODO: Add description.
+
+osi_groundtruth.proto::
+TODO: Add description.
+
+osi_hostvehicledata.proto::
+TODO: Add description.
+
+osi_lane.proto::
+TODO: Add description.
+
+osi_logicaldetectiondata.proto::
+TODO: Add description.
+
+osi_object.proto::
+TODO: Add description.
+
+osi_occupant.proto::
+TODO: Add description.
+
+osi_roadmarking.proto::
+TODO: Add description.
+
+osi_sensordata.proto::
+TODO: Add description.
+
+osi_sensorspecific.proto::
+TODO: Add description.
+
+osi_sensorview.proto::
+TODO: Add description.
+
+osi_sensorviewconfiguration.proto::
+TODO: Add description.
+
+osi_trafficcommand.proto::
+TODO: Add description.
+
+osi_trafficlight.proto::
+TODO: Add description.
+
+osi_trafficsign.proto::
+TODO: Add description.
+
+osi_trafficupdate.proto::
+TODO: Add description.

doc/architecture/proto-files.adoc

@@ -0,0 +1,20 @@
+= Coordinate systems and reference points
+
+OSI uses DIN ISO 8855:2013-11 for coordinate systems and transformations between coordinate systems.
+OSI uses three coordinate systems:
+
+World coordinate system::
+Coordinate system for all entities that are part of ground truth.
+The world coordinate system is an inertial x/y/z-coordinate system.
+The origin is the global reference point that is determined by the environment simulation.
+This reference point may be derived from map data or other considerations.
+World coordinates can be mapped to a geographic coordinate system via `osi3::GroundTruth::proj_string`.
+
+Sensor coordinate system::
+Coordinate system for all entities that are part of sensor data.
+The origin is the mounting position of the physical sensor or a virtual mounting position, depending on the OSI message.
+
+Object coordinate system::
+Local object coordinate system.
+The origin of the corresponding coordinate system is not necessarily identical to the center of the object's bounding box.
+If the origin of the corresponding coordinate system is not identical to the center of the object's bounding box, the object documentation will provide the actual definition.

doc/architecture/reference_points_coordinate_systems.adoc

@@ -0,0 +1,7 @@
+= Sensor data
+
+Sensor-data messages imitate the output of real sensors.
+It can be generated from ground-truth messages, sensor-view messages, feature-data messages or from sensor-data messages.
+Except feature data, all information regarding the environment is given with respect to the virtual sensor coordinate system.
+Feature data is given with respect to the physical sensor coordinate system.
+Sensor data can be used as input for an automated driving function, a sensor model simulating limited perception, or a sensor fusion model.

doc/architecture/sensor_data.adoc

@@ -0,0 +1,5 @@
+= Sensor model
+
+Sensor models consume sensor-view messages and produce sensor-data messages.
+
+Sensor-model output does not represent raw data but detected features or classified objects.

doc/architecture/sensor_model.adoc

@@ -0,0 +1,9 @@
+= Sensor view
+
+The sensor view provides the input to OSI sensor models.
+Sensor-view messages are derived from ground-truth messages.
+All information regarding the environment is given with respect to the virtual sensor coordinate system, with two exceptions:
+
+* Physical technology-specific data, given with respect to the physical sensor coordinate system specified in the corresponding physical sensor's mounting position.
+  Example of technology-specific data: https://opensimulationinterface.github.io/open-simulation-interface/structosi3_1_1CameraSensorView.html#ac58456a34babf78792ea2608eb963f36[`image_data` of `osi3::CameraSensorView`]
+* Ground truth, given in the global coordinate system.

doc/architecture/sensor_view.adoc

@@ -0,0 +1,34 @@
+= Sensor view configuration
+
+The sensor view is flexibly defined to provide different kinds of sensor models with appropriate input.
+The sensor view configuration defines the configuration of a particular sensor view.
+
+The sensor-view-configuration message is used in the initialization phase of a simulation to negotiate the sensor view configuration for a particular sensor view input.
+It is also included as a sub-message in sensor view messages to indicate that the sensor view configuration is valid for a particular sensor view message.
+
+Sensor-view-configuration data has two main applications:
+
+- It enables the environment simulation to provide the necessary input to a sensor model.
+- It enables a sensor model to check whether the input matches its requirements.
+If the input does not match the requirements, the sensor model may terminate the simulation.
+
+NOTE: Sensor-view-configuration data is intended for the automatic configuration of the sensor view interface between environment simulation and sensor model.
+The data is not intended as a mechanism to parametrize a generic sensor model.
+
+During the initialization phase, there are two sources for sensor-view-configuration data:
+
+1. Sensor-view-configuration data may be provided by the sensor model to the environment simulation.
+In this case, the data describes the input configuration that is requested by the sensor model.
+If no such data is provided by the sensor model, then the environment simulation will fall back to manual configuration of the sensor view.
+
+2. Sensor-view-configuration data may be provided by the environment simulation.
+In response to the request by the sensor model, or based on manual configuration, the environment simulation configures the input and provides a new message that describes the actual configuration.
+
+The configuration requested by the sensor model may differ from the configuration provided by the environment simulation.
+This happens when the environment simulation does not support a requested configuration or when the requested configuration is ambiguous.
+
+In response to this difference, the sensor model can either accept this difference and adapt to it, or it can terminate the simulation to indicate that it is not able to accept the difference.
+
+The packaging layer defines the specifics of this auto-negotiation mechanism.
+
+After the initialization phase, the environment simulation provides the actual sensor view configuration as part of each sensor view message.

doc/architecture/sensor_view_configuration.adoc

@@ -0,0 +1,45 @@
+= Test scripts
+
+TODO: Add general description.
+
+__init__.py::
+TODO: Add description.
+
+test_comment_type.py::
+TODO: Add description.
+
+test_doxygen_output.py::
+TODO: Add description.
+
+test_invalid_comment.py::
+TODO: Add description.
+
+test_invalid_enum.py::
+TODO: Add description.
+
+test_invalid_html.py::
+TODO: Add description.
+
+test_invalid_message.py::
+TODO: Add description.
+
+test_invalid_punctuation.py::
+TODO: Add description.
+
+test_invalid_tabs.py::
+TODO: Add description.
+
+test_newline.py::
+TODO: Add description.
+
+test_non_ascii.py::
+TODO: Add description.
+
+test_osi_trace.py::
+TODO: Add description.
+
+test_rules.py::
+TODO: Add description.
+
+test_units.py::
+TODO: Add description.

doc/architecture/test_scripts.adoc

@@ -0,0 +1,19 @@
+[#top-1a2f4b0c-195c-4f18-89ad-d48a123bd8c1]
+= OSI trace file formats
+
+There are multiple formats for storing multiple serialized OSI messages into one trace file.
+
+*.osi::
+Binary trace file.
+Messages are separated by a length specification before each message.
+The length is represented by a four-byte, little-endian, unsigned integer.
+The length does not include the integer itself.
+
+*.txt::
+Plain-text trace file.
+Messages are separated by `$$__$$`.
+
+*.txth::
+Human-readable plain-text trace file.
+Messages are separated by newlines.
+Such a file may be used for manual checks.