Survey: Crates / libraries / tools you need / want

[japaric]

Let's try to find out what libraries the embedded ecosystem lacks with a survey!

Please request one library / tool per comment. You can comment several times though. I suggest sticking to the guidelines below but feel free to ignore some of them if you think it makes sense to do so:

• Microcontroller and development board specific crates are out of scope for this survey. The requested crate must be general enough so that it can be useful to a non-trivial fraction of the embedded Rust community.

"I want a crate to program the e.g. STM32F4DISCOVERY development board" is out scope, otherwise we'd get a very long list of similar requests.

• Check the issue Survey: Crates you use for embedded development #12 first.

You may be asking for something that already exists! If a crate an crates.io or GitHub doesn't work for you, please tell us why that's the case.

• Please be as specific as possible and elaborate on your use case.

Don't just say "an Arduino-like framework that supports my microcontroller", which BTW would be out of scope because of point 1. That's too vague. Instead say "I need a low latency mechanism to send data from an interrupt handler to the main loop without disabling interrupts because I do $HARD_REALTIME_STUFF".

• If someone has already posted here a library that you also need then upvote their answer so we can get an idea of how much demand there is for such library / functionality.

• Bonus points if you can point to an existing C library that solves your problem. That way we can inform ourselves about prior art.

[genbattle]

It would be good to see statically allocated data structures like those implemented by ETL for C++. The biggest blocker for the creation of data structures with a similar API in Rust at the moment is type-level integers for Generics. There may be a way around this using a different approach in Rust, but there's also various hacks that can provide something like type-level integers.

[japaric]

I want a no-std binary serialization crate that doesn't require memory allocations and that uses fixed size buffers in its API to eliminate the possibility of runtime errors. I'm thinking of some API like this:

#[derive(BinSerialize, BinDeserialize)]
struct Foo {
    x: i32,
    y: i32, 
    z: i32,
}

// then I get methods like these
impl Foo {
    fn serialize(&self, buffer: &mut [u8; 12]) { .. }
    fn deserialize(buffer: &[u8; 12]) -> Self { .. }
}

Currently I'm using the byteorder for this task and it works wondefully but I'd like to avoid having to write the serialization / deserialization code myself.

[japaric]

statically allocated data structures

I got some (2) in the heapless crate. You can workaround the lack of integer generics by including the type of the array buffer, instead of just its size, in the data structure. So for example an std::Vec<T> backed by an array looks like Vec<T, [T; 8]> where [T; 8] is the statically allocated storage (capacity = 8).

[m4b]

Yea afaics I've literally already done this some time ago:

https://GitHub.com/m4b/scroll for the base reading traits of various flavors. Basically a pread style version where you specify an offset, and another version which takes a mutable offset and when it successfully reads it increments the offset for you.

Each trait has different methods which have all proved extremely useful for different kinds of binary reading, that I've come across in various contexts. Please see documentation.

It implements various basic types, as well as leb128 integers and allows safe, zero copy reading of strings. I've even considered adding safe marshalling from &[u8] to arrays of fixed size, e.g. [u8; N] for some N. All this is possible from the design.

For custom types, just as you have above, I have
https://GitHub.com/m4b/scroll_derive for the proc macros

You can also manually implement with whatever parsing context you want, in case endianness isn't sufficient. Again see documentation for examples (there should be a lot of it)

Nothing allocates. Errors also don't allocate.

I've used it for various tasks and some projects not online and I just find it superior to every alternative. Refactoring byteorder methods is literally horrible if something changes, and was the source of numerous manual errors in my experience.

Also I wanted to read without the bytes being mutable which I've used for great effect in parallel operations with sharded byte slices, but that isn't necessarily relevant here.

Lastly, if there's interest I can add a result less version for only a small class (e.g. numbers) when that is desired. There's also the unsafe versions which don't return a result either.

[fluffysquirrels]

@japaric wrote:

I want a no-std binary serialization crate that doesn't require memory allocations and that uses fixed size buffers in its API to eliminate the possibility of runtime errors. I'm thinking of some API like this:

For those that find their way to this thread, ssmarshal seems to this job nicely, as mentioned elsewhere.

[fluffysquirrels]

I want no_std-friendly std::io::Read and std::io::Write traits. It should be easy to write a crate that use std::io::{Read, Write} when using std and has some no_std alternative when no_std.

Repository https://github.com/QuiltOS/core-io has some nice ideas. It's not on crates.io. It has std::io-inspired Read and Write traits, but with a generic associated type as an error type. In practice I found the error type made it difficult to create an implementation of a module that worked with std and with no_std.

The core_io crate doesn't compile out of the box. It's a set of scripts that fetches the source to std::io and applies some patches to remove the requirement on the standard library.

Crate embedded-serial has similar ideas, providing traits with generic error type parameters designed for embedded peripherals to implement. It doesn't aim to be drop-in replacement for std::io::{Read, Write}, so it's hard to use in projects that must support both no_std and std.

I just did a quick search in the rust-lang/rust and rust-lang/rfc repositories and can't actually find any issues around moving std::io::{Read, Write} into core for no_std use. I'll do that shortly.

I'm working on ergonomics for framed now, and if no good options present themselves I will probably make my own crate (no_std-io or similar) to supply Read and Write traits, with the hope that several no_std crates end up supporting them.

[whitequark]

Please do something about core::io! It's been bothering me for over a year at this point, with no resolution. I've tried to do something with the core_io crate but the model seems a dead end to me.

[fluffysquirrels]

More searching for core Read, Write traits found these links:

• Error is crippled jethrogb/rust-core_io#3
• Move ::std::error to ::core::error rust-lang/rust#33149
• Port libstd to "no operating system" target rust-lang/rust#37133
• https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301
• Read Write for core BurntSushi/byteorder#51
• https://github.com/rust-lang/rfcs/blob/master/text/0517-io-os-reform.md#coreio
• RFC: io and os reform: initial skeleton rust-lang/rfcs#517
• no_std support (via an enabled-by-default "std" feature) rust-lang-deprecated/error-chain#157
• https://internals.rust-lang.org/t/crate-evaluation-for-2017-06-27-error-chain/5362/25

[fluffysquirrels]

@whitequark wrote:

Please do something about core::io! It's been bothering me for over a year at this point, with no resolution. I've tried to do something with the core_io crate but the model seems a dead end to me.

I was thinking it should be possible to have some core_io2 crate that supports compiling with and without std with some feature flags, std itself re-exports the traits from that, and no_std crates can use it directly. The problem with core_io was it keeps breaking and needing more patches whenever std::io changes, whereas with core_io2 the upstream project would have that responsibility.

I'm happy to work on a proof of concept or RFC for this if it stands a reasonable chance of getting into rust-lang/rust, but I'm not sure if it does. rust-lang/rust maintainers may not be interested in the approach or the maintenance burden. I'll post a quick issue on rust-lang/rfcs.

[pftbest]

There is also genio crate by @Kixunil. I think it has some compatibility with std::io traits, but it's using associated types for errors.

[fluffysquirrels]

I added this issue: rust-lang/rfcs#2262