fix: make line_column_to_offset character-based

[VolodymyrBg]

This change updates SourceContent::line_column_to_offset to interpret columns as Unicode character positions (Rust char) rather than raw bytes. The previous implementation treated columns as byte offsets and could panic when a column landed in the middle of a UTF-8 code point. It also contradicted location(), which already reports columns based on character counts.
With this change:

• line_column_to_offset now computes a safe byte offset via char_indices(), aligning semantics with location() and editor/LSP expectations.
• select() and update() become correct for non-ASCII input since they build on line_column_to_offset.
• A new unit test verifies round-trip consistency on a line containing multi-byte characters (aβ中💖).

These fixes prevent panics on valid UTF-8, ensure consistent column semantics across APIs, and better support LSP-style operations. If UTF-16 positions are needed by a client, conversion should occur at the integration boundary rather than within these core types.

[bitwalker]

This is also redundant - char_indices().nth() already handles this case, and removing this conditional avoids the need for us to compute num_chars at all (and thus iterate the characters of the line twice).

[VolodymyrBg]

This is also redundant - char_indices().nth() already handles this case, and removing this conditional avoids the need for us to compute num_chars at all (and thus iterate the characters of the line twice).

all done

[bitwalker]

One last change, and then we should be able to merge this. Could you also add an entry to CHANGELOG.md for this fix?

[bitwalker]

All of this can be replaced with just:

let byte_in_line = line_src.char_indices().nth(column_index).map(|(byte_index, _)| byte_index)?;

[bitwalker]

It looks like this exposes an issue with SourceContent::update, namely that having a Selection whose end specifies a line that doesn't exist/is empty, will fail, presumably because the attempt to resolve a ColumnIndex of 0 on that line will fail (char_indices() on that line will be an empty iterator, so None is returned, which causes SourceContent::update to fail).

We want the semantics of Selection::from(LineIndex(0)..LineIndex(1)) on a source file that has a single line of text with a trailing newline, to produce a Selection whose end specifies the byte offset of the trailing newline. I think we probably need to handle that case in line_column_to_offset, by checking that when char_indices().nth(column_index) produces None, if column_index == 0, then we should return the byte index of the start of the line.

[VolodymyrBg]

It looks like this exposes an issue with SourceContent::update, namely that having a Selection whose end specifies a line that doesn't exist/is empty, will fail, presumably because the attempt to resolve a ColumnIndex of 0 on that line will fail (char_indices() on that line will be an empty iterator, so None is returned, which causes SourceContent::update to fail).

We want the semantics of Selection::from(LineIndex(0)..LineIndex(1)) on a source file that has a single line of text with a trailing newline, to produce a Selection whose end specifies the byte offset of the trailing newline. I think we probably need to handle that case in line_column_to_offset, by checking that when char_indices().nth(column_index) produces None, if column_index == 0, then we should return the byte index of the start of the line.

seems like failling tests are unrelated

[bitwalker]

Can you rebase on next? That should clear up the CI failures

[VolodymyrBg]

Can you rebase on next? That should clear up the CI failures

Yes, CI feels good now