Change: add component binary config loader

[arsibo]

Change Request Type

Component Request

Description of the Change Request

Abstract

This feature introduces FlatBuffers binary configuration format to replace JSON-based module configuration.
FlatBuffers provides zero-copy access, schema validation, and access code generation for C++, Rust and further languages.
The approach eliminates the need for runtime parsing and accelerates module startup times.
Safety certification covers flatc tool qualification, runtime library verification and module-level testing of generated code.

Estimates for realization

tbd

Affects work products

• Requirements
• Architecture
• Safety/Security Analysis
• Detailed Design

Impact analysis

See component request template .
Missing artifacts will be contributed after acceptance

Safety or Security relevance

• none
• Safety relevant
• Security relevant

ASIL classification

ASIL_B

Expected Implementation Version

1.0

[arsibo]

discussed in https://github.com/orgs/eclipse-score/discussions/1239?sort=new#discussioncomment-14750150

[FScholPer]

1. Do we then still want nlohmann json to pilot(and tsf)?
2. Can you please first write a feature modification request like defined in our handbook
3. What version should that be pitched(v0.5,v1.0)?

[arsibo]

1. Do we then still want nlohmann json to pilot(and tsf)?
   Has to be discussed in the F2F Arc Meeting next 4-6.11.2025
2. Can you please first write a feature modification request like defined in our handbook
   Add module request binary config loader for baselibs. #1937
3. What version should that be pitched(v0.5,v1.0)?
   tbd. probably v1.0

[OliverHeilwagen]

Note: In order to get a quick look & feel for FlatBuffers the following branch was created in etas-contrib, to demo the functionality. https://github.com/etas-contrib/score_module_template/tree/flatbuffer_config_example/tests/config_example

[chheis]

Preliminary Meeting Minutes:

@OliverHeilwagen please upload your slides here

• Motivation is performance of loading big configurations
• comparing JSON (baselibs), JSON (nlohman), Flatbuffers
• Test results with GoogleTest is also shown
• JSON flatbuffers is a flatbuffers internal implemenation
• Flatbuffer parser is available in a bigger range
• The dev implementation it is available in less languages
• C++ is used in the performance analysis
• Overall observation: the runtime diff between flatbuffer and JSON baselibs implementation is about 3x faster
• Loading time of big config files is much more beneficial (see presentation) up to infinity ;)
• 2 examples in the test:
  • Code as it was developed
  • Schema evolution (accessible with old code as well)
  • Implemented in Rust, C++, python
  • C++ full blown test
  • Rust basic implementation
  • python read in and use age over C++ test
• Q: How is the deprecated keywords are propagated?
  • A: Fields which are marked as deprecated are removed from generated code (Reader, Verify, Builder).
  • A: Usage of deprecated elements in config JSON do not produce warnings/errors when generating a binary buffer file.
• Q: What happens in case of errors? If the binary is corrupted?
  • A: C++: Get<> will return an invalid pointer usage will cause undefined behavior
  • A: Rust:
    • root::() will return error, verifier is automatically performed
    • root_unchecked::() returns an invalid element

Main reasons:

• Performance
• Speed up development

Q&A:

• Q (Nico): How does it map to ABI?
• A: Configuration is not solving the ABI discussion.
  • This is a proper "ABI type".
• Q (Nico): How does this map to the config management?
• A: The configuration management can use the flatbuffers e.g. the configuration deamon
• Statement: Configurations "just" started with JSON, so it if we change the configuration we should change it for all.
• This is the reason why this is here discussed.
• Q: Is the configuration solution a solution for all configuration? Also Application configuration
• Hint (Nico): There shall is only on solution for one problem.
• Hint (Nico): A lot of problems are solved with the flatbuffers.
• It is not only a
• Q: What does it do with strings?
• A: They are flexible. => A ABI must be defined also.
• Hint: Local configuration files are needed, but can it be combined.
• It is not solving all problems
• Q: Is the padding between the different architectures?
• A: Solved by flatbuffer as it is the main goal. No matter which languages.
• Q: Concurn: Complexity within the build system. The developers can change JSON files directly on the fly.
  • A: A Configuration API is always to be used.
• Next step:
  • How is local filebased config can be solved?
  • What are the usecases?
• Discuss this in the baselibs Feature team and push the decision to the Architecture Community meeting.
• @4og with support of @arsibo

[OliverHeilwagen]

Presented slides:
FlatBuffers_on-board_configuration.pdf

[OliverHeilwagen]

Update for the following questions:

• Q: How is the deprecated keywords are propagated?

  • A: Fields which are marked as deprecated are removed from generated code (Reader, Verify, Builder).
  • A: Usage of deprecated elements in config JSON do not produce warnings/errors when generating a binary buffer file.

• [UPDATE] Q: What happens in case of errors? If the binary is corrupted?
  (I accidentally mixed up the behavior of a wrapper we used and the flatc generated code, correct answer is the following)

  • A: C++: Get<> will return an invalid pointer usage will cause undefined behavior
  • A: Rust:
    • root::() will return error, verifier is automatically performed
    • root_unchecked::() returns an invalid element

I updated the playground branch accordingly:
https://github.com/etas-contrib/score_module_template/tree/flatbuffer_config_example/tests/config_example

As mentioned during the talk for safety certified usage it is expected that the buffer integrity is ensured via external mechanisms (e.g. CRC checks). Additionally the buffer identifier and semantic version shall be checked for compatibility. When these checks are implemented, the flatc generated verifier can be omitted, as it only guarantees structural validity, not payload correctness.

[dcalavrezo-qorix]

Hi @arsibo,

Is this issue still relevant? If not, please add the trash label so it can be marked for deletion.

[arsibo]

Hi @arsibo,

Is this issue still relevant? If not, please add the trash label so it can be marked for deletion.

Hi @dcalavrezo-qorix ,

yes it is. We will make an arc decision on Monday about that

[OliverHeilwagen]

Here is the proposal for the flatbuffer decision overview:

Module configuration

The module configuration is used by a module (library or daemon) to configure its functionality, e.g. communication, logging.

Criteria Overview

Criteria	JSON	FlatBuffers	Comments
Build & Tooling	+	+	Balanced
Tool dependency	0	0	Schema validator (e.g. ajv) VS flatc (code generation, validation)
Build integration	++	+	Schema validation, link of library VS code generation, link library, buffer generation
Multi-language support	+	++	C++ (Rust: planned, Python: to be selected) VS Support for multiple languages
Config & Schema IDE support	++	+	Very good VS varying on IDE
Runtime Characteristics	-	++	FlatBuffers
Memory allocation	-	++	Heap allocation required for parsing VS zero-copy access, no heap
Performance	-	++	Parsing and navigation overhead VS no parsing, offset based access
Memory footprint	-	++	Requires DOM representation VS Minimal buffer info overhead
Schema & Validation	0	++	FlatBuffers
Schema creation	-	++	Code independent schema file VS single truth for code and validation
Type safety	+	++	Runtime checks VS compile-time type safety
Default values	-	+	Manual handling VS Built-in support for scalar types
Evolution	0	++	Manual handling VS Built-in support for forward / backward compatibility including tool check
Development Experience	+	+	Tradeoff: Debug ease vs Maintainability
Config readability	++	-	Human readable off and on target VS Binary format on target (JSON in dev mode only)
Maintainability	-	++	Manual sync. of data, schema and code VS flatc generated code and validation
Testing effort	0	0	Validation scope is identical (access code and schema)

Safety Certification

Not compared, as both solutions are expected to co-exist for different purposes. Required header library of FlatBuffers introduces ~3250 LOC (13 headers ~250 LOC for readers and verifiers) requiring safety qualification. Flatc buffer generation is expected to be in scope for tool qualification. Schema specific buffer access and verification is done on module-level, which contributes to flatc tool qualification. Builders and advanced FlatBuffer features are expected to be out of scope.

Info Performance Comparison

FlatBuffers on-board configuration - Presented in Architecture Community F2F Workshop 06.11.2025