Skip to content

Fix potential crash if using UTF8 directory names. #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 16, 2022
Merged

Fix potential crash if using UTF8 directory names. #114

merged 2 commits into from
Dec 16, 2022

Conversation

@kjnilsson
Copy link
Contributor

@kjnilsson kjnilsson commented Dec 15, 2022

No description provided.

@kjnilsson kjnilsson marked this pull request as ready for review December 15, 2022 11:27
michaelklishin
michaelklishin reviewed Dec 15, 2022
View reviewed changes
@kjnilsson kjnilsson requested a review from ansd December 15, 2022 16:32
ansd
ansd reviewed Dec 16, 2022
View reviewed changes
src/osiris_log.erl
% convert to binary for faster operations later
% mostly in segment_from_index_file/1
evaluate_retention(list_to_binary(Dir), Specs);
evaluate_retention(unicode:characters_to_binary(Dir), Specs);
Copy link
Member

@ansd ansd Dec 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You always convert to unicode = utf8 even if the VM is running in latin1 mode. I think it's probably fine here. Just want to add a note that
https://github.com/erlang/otp/blob/a1f235fc05653fa524fc7c4a352398af6ce0b364/lib/stdlib/src/filename.erl#L845-L851 uses file:native_name_encoding() as OutEncoding.

@ansd
Copy link
Member

ansd commented Dec 16, 2022

osiris/src/osiris_util.erl

Line 271 in 5ff6513

list_to_binary(Name).

I assume we don't support Unicode in the stream name, correct?

Changes make sense to me.
With this branch I also cannot see anymore any formatter crashes that I used to see with the stream test included in rabbitmq/rabbitmq-server#6684

@kjnilsson
Copy link
Contributor Author

kjnilsson commented Dec 16, 2022

osiris/src/osiris_util.erl

Line 271 in 5ff6513

list_to_binary(Name).

I assume we don't support Unicode in the stream name, correct?
Changes make sense to me. With this branch I also cannot see anymore any formatter crashes that I used to see with the stream test included in rabbitmq/rabbitmq-server#6684

all stream names only contain base64uri compatible characters so this should be fine here.

@kjnilsson kjnilsson merged commit 399b470 into main Dec 16, 2022
@ansd ansd deleted the utf8-filename-fix branch December 16, 2022 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@michaelklishin michaelklishin michaelklishin left review comments

@ansd ansd ansd approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kjnilsson @ansd @michaelklishin