Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to serialize to a Writer #202

Closed
Boscop opened this issue Feb 26, 2020 · 0 comments · Fixed by #206
Closed

Add function to serialize to a Writer #202

Boscop opened this issue Feb 26, 2020 · 0 comments · Fixed by #206
Labels

Comments

@Boscop
Copy link

Boscop commented Feb 26, 2020

Bincode has serialize_into which supports an arbitrary Writer, this would be very useful for RON as well, especially with BufWriter, because the data gets very large.

I'm seeing huge RAM spikes because of RON serialization: From ~90 MB baseline RAM usage to over 700 MB when it's serializing. Btw, the resulting ron file is 20 MB large.

image


In the above application, I'm serializing app state to disk every 10s.
Btw, the CPU usage is also quite high. This is the CPU & RAM usage when serializing to bincode instead. With ROM, the CPU usage is almost always at ~27% (probably because it takes so long), with bincode, it only shows short spikes (to 24%) when serializing:

image


Anyway, with a function like serialize_into+BufWriter we could at least reduce the RAM spikes.
This would also allow chaining it with e.g. a Gzip encoder for streaming compression, e.g. like with bincode:

	bincode::serialize_into(
		GzBuilder::new()
			.write(BufWriter::new(File::create(path)?), Compression::best()),
		&data,
	)?;
@kvark kvark added enhancement good first issue Perfect for new contributors help wanted labels Mar 3, 2020
bors bot added a commit that referenced this issue Apr 27, 2020
206: feat: port `ser::Serializer` to `io::Write` r=kvark a=Plecra

Makes the `Serializer` struct generic over `io::Write`, allowing it to be used with more types (and less allocation). As a side-effect, the trailing commas are also removed from the output.

This was a quick implementation, written in response to compiler messages, so it could probably do with some cleaning up. It might be a good time to approach #175 

Closes #202 

## TODO

- [ ] Update docs


Co-authored-by: Marli Frost <marli@frost.red>
@bors bors bot closed this as completed in 23649eb May 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants