Skip to content

Conversation

@darkwisebear
Copy link
Contributor

This PR contains the draft API that will be used as the candidate API that gets implemented in terms of the s-core/mw/com C++ API that is to be found at https://github.com/eclipse-score/communication.

The PR divides the code into several crates and should mimic a real world example so that we can also detect violations of the orphan rule or similar problems like dependency problems.

* Add Subscriber trait with all agreed receive variants
* Move demo interface code to a separate module
Split crates so that they match a real use case, consisting of
* Generated interface declaration(s)
* Generated binding code
* Upper level API
* Binding specific common code
* Application

See the .puml for a overview
n Subscriber, we need to make sure that the returned sample pointers
as well as the types they wrap never outlive the lifetime of the
subscriber. Constrain the methods accordingly.
* Introduce service discovery on consumer side
* Introduce subscribe/unsubscribe
* Introduce offer/unoffer on producer side
* Revise receive calls on subscription side
Copy link

@LittleHuba LittleHuba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I focused on the tests of the example and only glanced over the rest. I'll leave the in-depth review for the Rust-Pros here.

Cargo.toml Outdated
@@ -1,3 +1,3 @@
[workspace]
members = ["com-api", "com-api-example"]
members = ["com-api", "com-api-example", "com-api-sample-instance", "com-api-sample-interface", "com-api-sample-runtime"]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing to underscores would be good. I do't know where we noted it down, but at some point we discussed using snake_case for paths in S-CORE.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, no objections.

.into_iter()
.find(|desc| desc.get_instance_id() == 42)
.unwrap();
let consumer_builder = descriptor.into_builder();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it viable to skip this step and make "descriptor" directly the builder?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the descriptor is something really lightweight, so into_builder might incur some costs.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the builder not also be very lightweight? Maybe that is a naive assumption on my side?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, this depends on the implementation. In mw::com terms a descriptor would be an interface-specific HandleType. The builder needs to take this and allow for additional configuration before the creation of the actual consumer. So the diff would be small; actually, the diff should only consist of the runtime configuration parameters, which could be as small as one additional pointer to the parameter list; therefore, the diff is rather negligible.

If no one objects, I'd merge those two.

let runtime = runtime_builder.build().unwrap();

// Create service discovery
let consumer_discovery = com_api_sample_instance::RuntimeBuilderImpl::find_service::<

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should find_service not be in the runtime? Or was this this dependency issue?
Also, having this static is bad for testing or?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should. But due to the typing in Rust, the orphan rules for trait impls prevent this. We're still struggling here and a nicer solution would be highly welcome, but so far, I was unable to come up with one.

// Factory
let runtime_builder = com_api_sample_instance::RuntimeBuilderImpl::new();
let runtime = runtime_builder.build().unwrap();
let producer_builder = com_api_sample_instance::RuntimeBuilderImpl::create_provided_service::<

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here I would have assumed this is part of the runtime. But this is the dependency issue again, or?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

println!("Hello, world!");
}

#[cfg(test)]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, showing more error-handling in the examples would show better whether our API is nice to use. Not something we have to focus immediately on, but we should do so soon. After all, our users can't just unwrap everything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we need to add several test kinds still:

  • Usage in a multi threaded async environment
  • How to mock on different levels
  • Error handling
  • ...more?

@MarkSchmitt MarkSchmitt changed the title Mege Rust API draft into incubator Merge Rust API draft into incubator Jun 5, 2025
Copy link

@MarkSchmitt MarkSchmitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed during the workshop, I created and linked issues in the communication (not inc_mw_com as that one will be closed down soon probably!) repository. I commented with the links to it.

type ServiceEnumerator: IntoIterator<Item = Self::ConsumerDescriptor>;

fn get_available_instances(&self) -> Result<Self::ServiceEnumerator>;
// TODO: Provide an async stream for newly available services / ServiceDescriptors

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

//! # API Design principles
//!
//! - We stick to the builder pattern down to a single service
//! - We stick to the builder pattern down to a single service (TODO: Should this be introduced to the C++ API?)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

//! - We stick to the builder pattern down to a single service (TODO: Should this be introduced to the C++ API?)
//! - We make all elements mockable. This means we provide traits for the building blocks.
//! We strive for enabling trait objects for mockable entities.
//! We strive for enabling trait objects for mockable entities. (TODO: Inspect all consuming methods for adherence to this rule)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type Output;
/// Open point: Should this be &mut self so that this can be turned into a trait object?
fn build(self) -> Self::Output;
/// TODO: Should this be &mut self so that this can be turned into a trait object?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

@vinodreddy-g vinodreddy-g Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a trait object here for dependency injection of builder type for testing ?By losing the compile type check of not reusing it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Builder is very generic, so it's thinkable that there are use cases where you'd want to call it through &mut dyn reference. We need to figure that out.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general we have to consider the potential advantage to have a fundamental builder trait. As we always have to know what exactly we build we do not require a generic trait here. Even if we would pass the builder around we would have to know what it actually is. That statement would only change if being a Builder would imply other traits, e.g. "every builder is Clone + Send" (which has something to it).

pub trait Sample<T>: Deref<Target = T> + Send
///
/// The ordering of SamplePtrs is total over the reception order
// TODO: C++ doesn't yet support this. Expose API to compare SamplePtr ages.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +257 to +258
/// TODO: How to make sure that the provided container contains samples from the same
/// TODO: `Subscription` instance?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +273 to +274
/// TODO: C++ cannot fully support this yet since there is no way to retain potentially-reusable
/// TODO: samples.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// The replacement semantics as well as the post conditions of the resolved future are equal
/// to `try_receive`.
///
/// TODO: See above for C++ limitations.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as try_receive, eclipse-score/communication#16

Comment on lines 1 to 52
@startuml
'https://plantuml.com/component-diagram
package "com-api" {
[Com API]
}

package "com-api-sample-runtime" {
[Com API] <-- [RuntimeImpl]
}

package "com-api" {
[Com API]
}

package "com-api-sample-interface" #Yellow {
[AutoInterface]
}

package "com-api-sample-instance" #Yellow {
[AutoInstance]
[AutoInstance] --> [RuntimeImpl]
[AutoInstance] --> [AutoInterface]
[RuntimeBuilderImpl]
[RuntimeBuilderImpl] --> [AutoInstance]

note right of [AutoInstance]
Code that implements AutoInterface
in terms of RuntimeImpl.
end note
}

note top of "com-api-sample-instance"
This crate knows all supported interfaces
and provides instances for them that are
implemented in terms of
com-api-sample-runtime::RuntimeImpl
end note

package "com-api-sample" {
[Factory]
[Business Logic] --> [AutoInterface]
[Factory] --> [RuntimeBuilderImpl]
[Factory] --> [Com API]
[Factory] ..> [Business Logic] : builds
}

package "Package" #Yellow
note bottom of "Package"
Yellow = generated
end note

@enduml No newline at end of file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@startuml
'https://plantuml.com/component-diagram
package "com-api" {
[Com API]
}
package "com-api-sample-runtime" {
[Com API] <-- [RuntimeImpl]
}
package "com-api" {
[Com API]
}
package "com-api-sample-interface" #Yellow {
[AutoInterface]
}
package "com-api-sample-instance" #Yellow {
[AutoInstance]
[AutoInstance] --> [RuntimeImpl]
[AutoInstance] --> [AutoInterface]
[RuntimeBuilderImpl]
[RuntimeBuilderImpl] --> [AutoInstance]
note right of [AutoInstance]
Code that implements AutoInterface
in terms of RuntimeImpl.
end note
}
note top of "com-api-sample-instance"
This crate knows all supported interfaces
and provides instances for them that are
implemented in terms of
com-api-sample-runtime::RuntimeImpl
end note
package "com-api-sample" {
[Factory]
[Business Logic] --> [AutoInterface]
[Factory] --> [RuntimeBuilderImpl]
[Factory] --> [Com API]
[Factory] ..> [Business Logic] : builds
}
package "Package" #Yellow
note bottom of "Package"
Yellow = generated
end note
@enduml
@startuml
'https://plantuml.com/component-diagram
package "com-api" {
[Com API]
}
package "com-api-sample-runtime" {
[Com API] <-- [RuntimeImpl]
}
package "com-api" {
[Com API]
}
package "com-api-sample-interface" #Yellow {
[VehicleInterface]
}
package "com-api-sample-instance" #Yellow {
[AutoInstance]
[AutoInstance] --> [RuntimeImpl]
[AutoInstance] --> [VehicleInterface]
[RuntimeBuilderImpl]
[RuntimeBuilderImpl] --> [AutoInstance]
note right of [AutoInstance]
Code that implements VehicleInterface
in terms of RuntimeImpl.
end note
}
note top of "com-api-sample-instance"
This crate knows all supported interfaces
and provides instances for them that are
implemented in terms of
com-api-sample-runtime::RuntimeImpl
end note
package "com-api-sample" {
[Factory]
[Business Logic] --> [VehicleInterface]
[Factory] --> [RuntimeBuilderImpl]
[Factory] --> [Com API]
[Factory] ..> [Business Logic] : builds
}
package "Package" #Yellow
note bottom of "Package"
Yellow = generated
end note
@enduml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VehicleInterface is still not perfect .. it should express something that I want to subscribe to, something that is clear that it subsumes a number of event topics that I want to subscribe to ... yes a Car, a Vehilce etc has that sort of information, but that's wayyyy to general, as normally you would split that. Perhaps something like DistanceMeasurements with topics like Midrange, CornerRadarFrontLeft, CornerRadarFrontRight as topics might be clearer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, it was just meant to be an example and could have been anything. Please comment on the IDL-like example with usggestions and I can update the example accordingly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the diagram and fixed naming there as well.


fn is_sync<T: Sync>(_val: T) {}
/// Will remove the first element from the container (if any) and return it to the user.
fn pop_front(&mut self) -> Option<S>;
Copy link

@pawelrutkaq pawelrutkaq Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when this is needed in COM internals ? I see that this trait is given from user into internal to get data so I bet user will anyway use API from container to get data and not from trait (FIFO order may be too limiting for user itself)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This trait is intended to be implemented on user-defined container types so that they can be used during data reception. The user is free to use other means to modify the container, but the API needs a thing that it can operate on, with the given methods.

///
/// TODO: C++ cannot fully support this yet since there is no way to retain potentially-reusable
/// TODO: samples.
fn try_receive<'a, C>(&self, scratch: C, max_samples: usize) -> (C, Result<usize>)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alias here for Result manifest itself highly confusing. Just naming Result as ComResult or sth like that will make code better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating a typedef like this is a regular Rust pattern, see for example https://doc.rust-lang.org/std/io/type.Result.html. If one wants to be more specific about which result type is referred, they're free to use the full path still.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still consfusing ;) especially during review .

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fully qualified type of Result<T> defined as alias in a module will usually inferred correctly, so it can and will be omitted in type declarations.
Also, I would make a convention that E in Result is always a module specific xxxError type, implementing a global Error trait.

where
Self: 'a,
T: 'a,
C: SampleContainer<Self::Sample<'a>> + 'a;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future::Output missing Send

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output of the future doesn't necessarily need to be send. The Future itself should definitely be Send. I think with additional example code that is closer to real world use cases, we will be able to better determine the bounds, so I'd be conservative until we see the compiler complain. We certainly need to stress the API more by using the "usual" async executors to see that.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IF the future is Send, the output has to be send because Future can change thread execute there, create result and back. However in this case, due to 'a maybe even Future shall not be marked Send. This would bascially mean you can await in current task, but you cannot spawn it (which potentially sounds okey as you keep this with subscription).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the real use case is not to spawn a task with the already-received sample onto an executor but instead to spawn a task that will repeatedly query the subscription. Since the subscription is then part of the spawned task (and the subscription is 'static), we shouldn't run into any lifetime issues. But as already stated, we really need to write tests that mimic this pattern to see whether we got all bounds correct, including lifetime bounds.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes i agree, for that, Future does not need to be send here. Anyway lets do as you said, just using it in some sample app.

scratch: C,
new_samples: usize,
max_samples: usize,
) -> impl Future<Output = (C, Result<usize>)> + Send

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: usage of 'a will prevent to spawn such future (Fut has to be 'static to be able to spawn it in mt). The plain await in current async context may work. This mean that Subscription has to be Send to let it be usable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most likely. However, we also need to make sure that no sample outlives the subscription, so I guess you cannot use the normal MT executors anyway which expect the Future to live forever. It most likely needs to be a scoped executor which lives shorter than the subscriptions to ensure that.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might add that from my point of view the statement a Sample cannot outlive a Subscription it origininated from is a little bit artificial. Technically, we could let the Sample live as long as the underlying memory lives. However, to prevent blockage of that memory (and is communication memory after all) we settled on the enforcement of Samples not outliving their creators.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, however that implies that the memory really belongs to Sample or that Sample has some kind of runtime refcount on the memory so that it can outlive the subscription. Imo still owning data from a past subscription is an odd state so that we can (imo should) put a constraint on Sample not to outlive the subscription, thus "promising" the underlying implementation to not refer to the memory anymore when the subscription is about to end so that we can allocate / deallocate the buffer memory in one go each.

fn load_config(&mut self, config: &Path) -> &mut Self;
}

pub struct InstanceSpecifier {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What it will be ? "String" or InstanceSpecifier::from(String) ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's to be seen. For the mw::com case, its certainly something that can be created from a &str.

Timeout,
}

pub type Result<T> = std::result::Result<T, Error>;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Highly confusing alias, think over to name it *Result to know its not from std

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

fn builder_is_sync() {
is_sync(crate::sample_impl::RuntimeBuilderImpl::new());
/// Will add a sample to the end of the container.
fn push_back(&mut self, new: S) -> Result<()>;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When push back fails (is user has some Bounded container with static size), S is lost now. Is it okey and we will be able to recover this sample somewhere internally in implementation (did not saw Sample being Copy/Clone, still it can be only wrapped ptr) ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be ok since the internal algo will first remove one sample by means of pop_front before pushing. If the container can still return an error in case we removed a sample before, it would be lost indeed. But we can only do so much...

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but does that mean you never append more then SINGLE sample internally ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to remove the first (as in FIFO) sample in case there is a newer sample available, and do this repeatedly until there are no samples anymore in the input queue or you collected the number of samples the user requested. Therefore, you can have multiple samples removed from the buffer.

where
T: Reloc + Send,
{
type Subscriber = SubscribableImpl<T>;
Copy link

@vinodreddy-g vinodreddy-g Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is like a typestate pattern applied for offeredproducer and producer , could be also for consumer and subscriber (like active and inactive subscription states)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well no, on consumer side there is no "offered". However, for data communication, we need to support the SOME/IP subscription feature, which is why there is a similar pattern applied on a per-data granularity, as opposed to per-interface.

By merging the generated interface and the runtime specific code and
turning Builder into a parameterized trait, we can avoid violating
the orphan rule. This enables moving a lot of stuff to the
non-generated part of the code, only keeping what really needs to be
generated.
This is actually the right place. Unfortunately, we cannot add treit
methods to the Runtime trait as they would be too generic, requiring
_any_ implementor of the Interface trait to be instantiable, not just
the ones generated.
@darkwisebear
Copy link
Contributor Author

darkwisebear commented Jun 10, 2025

I pushed changes that merge both generated crates into one, resolving issues caued by the orphan rules. They enable the complete separation of (to-be) generated code and base code that gets used by the generated code.

Now the only remaining issue is the missing generic trait that enables a user to mock the creation of producers and consumers. We need to add sample test code that resembles test code that needs this feature to find out how serious this is.

Edit: I will also push a commit that updates the puml.

Copy link

@LittleHuba LittleHuba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll put some lightweight naming suggestions. Apply them if you want or leave things for a later renaming round. I'm fine with either. Just wanted to record my thoughts here.

let runtime_builder = com_api_sample_runtime::RuntimeBuilderImpl::new();
let runtime = runtime_builder.build().unwrap();
let producer_builder =
runtime.create_provided_service::<VehicleInterface>(InstanceSpecifier {});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runtime.create_provided_service::<VehicleInterface>(InstanceSpecifier {});
runtime.producer_builder::<VehicleInterface>(InstanceSpecifier {});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorporated.

let runtime = runtime_builder.build().unwrap();

// Create service discovery
let consumer_discovery = runtime.find_instance::<VehicleInterface>(InstanceSpecifier {});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's align this with the C++ API. Also conceptually your are searching for a service here with restrictions on the instances of that service. With that change the get_available_instances() call is more logical.

Suggested change
let consumer_discovery = runtime.find_instance::<VehicleInterface>(InstanceSpecifier {});
let consumer_discovery = runtime.find_service::<VehicleInterface>(InstanceSpecifier {});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorporated


// Create service discovery
let consumer_discovery = runtime.find_instance::<VehicleInterface>(InstanceSpecifier {});
let available_services = consumer_discovery.get_available_instances().unwrap();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let available_services = consumer_discovery.get_available_instances().unwrap();
let available_service_instances = consumer_discovery.get_available_instances().unwrap();

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorporated.

Instead of having to create a builder from a descriptor, make the
descriptor trait a supertrait of the ConsumerBuilder, and make
ServiceDiscovery return a list of builders immediately.
* create_provided_service -> producer_builder
* find_instance -> find_service
* available_services -> available_service_instances
@darkwisebear
Copy link
Contributor Author

@MarkSchmitt @LittleHuba Can you please check the PR whether it is now good to go? If not, please state what you're missing still so that I can add this. Remember, though, that this is only the start, the API will still see heavy work. I will also add some issues to communication for things that I see as missing.

@darkwisebear
Copy link
Contributor Author

@MarkSchmitt can we proceed with the merge or do you need something still?

@LittleHuba LittleHuba merged commit 215dcce into eclipse-score:main_api_rs Jul 1, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants