[ty] Refactor `Range` to/from `TextRange` conversion as prep for notebook support #21230

MichaReiser · 2025-11-03T01:58:45Z

Summary

This PR changes the signature of the TextSize <-> Position and TextRange <-> Range conversion methods in preparation for notebook support.

According to the LSP specification, each Notebook consists of many TextDocuments. However, ty internally concatenates all cells to a single text document. That means, offsets coming from ty need to be mapped back to their corresponding cell, and then to the offset within the cell. The same applies for LSP offsets: The cell-relative offset needs to be converted to an absolute offset into the concatenated notebook document.

This PR doesn't implement the logic itself for mapping between the different offsets but it refactors the call-sites and the methods used for those conversions to add the required logic in a follow-up PR.

The most important change is that we now pass db and File to the conversion methods. This will allow us to query the notebook to resolve the cell metadata (offset per cell, etc).

Test Plan

Existing tests. I tested different LSP features and they seem to work fine

github-actions · 2025-11-03T02:00:45Z

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

github-actions · 2025-11-03T02:02:20Z

`mypy_primer` results

No ecosystem changes detected ✅
No memory usage changes detected ✅

MichaReiser · 2025-11-03T02:08:16Z

crates/ty_server/src/server/api/requests/completion.rs

-            &line_index,
+            db,
+            file,
+            snapshot.url(),


It's a bit unfortunate that we need the url here but the file isn't unique enough (the file maps to the entire notebook). We need the url to lookup the corresponding cell within the notebook.

MichaReiser · 2025-11-03T02:10:32Z

crates/ty_server/src/server/api/requests/completion.rs

+                    let range = edit
+                        .range()
+                        .as_lsp_range(db, file, snapshot.encoding())
+                        .to_local_range();


Using to_local_range here is technically incorrect. In fact, it's very likely that the importer adds the import to the very first cell within the notebook, in which case the edit is not local to a single cell.

However, fixing this requires changes to the importer and they're substantial enough to justify a separate PR

MichaReiser · 2025-11-03T02:27:29Z

crates/ty_server/src/server/api/requests/doc_highlights.rs

                    .range()
-                    .to_lsp_range(&source, &line_index, snapshot.encoding());
+                    .as_lsp_range(db, file, snapshot.encoding())
+                    .to_local_range();


We'll need to limit the document_highlights to the current cell. This seems very unfortunate as the highlights are very likely to be cross cell but the LSP protocol doesn't allow us to specify a URI. Again, I think this is best done in a separate PR

This isn't something that requires to be investigated right now but I tested Pylance and it seems to be able to highlight a symbol across multiple cells in a notebook.

Screen.Recording.2025-11-04.at.15.48.55.mov

I'm not sure this is something Pylance does (although it could because it isn't an LSP).

But I don't see a highlight request for cells at all and the highlighting is also incorrect, suggesting it's just a simple name matching:

Screen.Recording.2025-11-04.at.15.58.25.mov

It could mean that we should return Option from to_location but I've to revisit this when being further along with the notebook integration

That makes sense.

MichaReiser · 2025-11-03T02:29:31Z

crates/ty_server/src/server/api/requests/document_symbols.rs

+        range: symbol
+            .full_range
+            .as_lsp_range(db, file, encoding)
+            .to_local_range(),


Same here: We need to limit the document symbol request to the current cell. But I'd prefer doing that in a separate PR

MichaReiser · 2025-11-03T02:31:42Z

crates/ty_server/src/server/api/semantic_tokens.rs


    for token in &*semantic_token_data {
-        let lsp_range = token.range().to_lsp_range(&source, &line_index, encoding);
+        let lsp_range = token


We'll need to limit the semantic tokens to a single range but I'll do this in a separate PR

sharkdp · 2025-11-03T15:35:07Z

Seems like something that maybe @dhruvmanila would be a good candidate to review?

dhruvmanila

Looks good!

Introduce new infrastructure to support converting positions and ranges for documents that may map to multiple URIs (like notebook cells): - Add Db trait with document() and notebook_document() queries - Provides access to document metadata needed for conversion - notebook_document() returns notebook info for a file if it exists - Add LspRange and LspPosition types for deferred LSP conversion - Stores file, offset/range, db reference, and encoding - Allows callers to choose conversion strategy: * to_local_range()/to_local_position(): For use within same document/cell * to_location(): For cross-document references with URI - Critical for notebooks where one file maps to multiple cell URIs - Update conversion traits (RangeExt, PositionExt, etc.) - Add methods to create LspRange/LspPosition from internal types - Support both notebook cells and regular files This infrastructure prepares the codebase for notebook support where position/range conversions must account for cell boundaries and URIs.

* main: (188 commits) [ty] Discover site-packages from the environment that ty is installed in (astral-sh#21286) [ty] Make special cases for `UnionType` slightly narrower (astral-sh#21276) Require ignore 0.4.24 in `Cargo.toml` (astral-sh#21292) [ty] Favour imported symbols over builtin symbols (astral-sh#21285) docs: revise Ruff setup instructions for Zed editor (astral-sh#20935) [ty] Update salsa (astral-sh#21281) [syntax-error]: no binding for nonlocal PLE0117 as a semantic syntax error (astral-sh#21032) [ty] Constraining a typevar with itself (possibly via union or intersection) (astral-sh#21273) [`ruff`] Fix false positives on starred arguments (`RUF057`) (astral-sh#21256) [ty] Simplify unions containing multiple type variables during inference (astral-sh#21275) [ty] Add `ty_server::Db` trait (astral-sh#21241) [ty] Refactor `Range` to/from `TextRange` conversion as prep for notebook support (astral-sh#21230) [ty] Fix playground crash when file name includes path separator (astral-sh#21151) [`refurb`] Fix false negative for underscores before sign in `Decimal` constructor (`FURB157`) (astral-sh#21190) [ty] Allow values of type `None` in type expressions (astral-sh#21263) Run codspeed benchmarks with `profiling` profile (astral-sh#21261) [ty] Update expected diagnostic count in benchmarks (astral-sh#21269) Avoid extra parentheses for long `match` patterns with `as` captures (astral-sh#21176) [ty] Update salsa (astral-sh#21265) [ty] `dict` is not assignable to `TypedDict` (astral-sh#21238) ...

MichaReiser added internal An internal refactor or improvement ty Multi-file analysis & type inference labels Nov 3, 2025

MichaReiser commented Nov 3, 2025

View reviewed changes

MichaReiser force-pushed the micha/lsp-range branch 3 times, most recently from 4b69dd8 to 1114c07 Compare November 3, 2025 02:21

MichaReiser commented Nov 3, 2025

View reviewed changes

MichaReiser force-pushed the micha/lsp-range branch 2 times, most recently from 61afb54 to a62875f Compare November 3, 2025 02:42

MichaReiser marked this pull request as ready for review November 3, 2025 02:50

MichaReiser requested review from carljm, dcreager and sharkdp as code owners November 3, 2025 02:50

MichaReiser added the server Related to the LSP server label Nov 3, 2025

dhruvmanila approved these changes Nov 4, 2025

View reviewed changes

MichaReiser added 2 commits November 5, 2025 08:12

Clippy

44ad219

MichaReiser force-pushed the micha/lsp-range branch from 5662955 to 44ad219 Compare November 5, 2025 13:13

MichaReiser changed the title ~~[ty] Refactor Range<->TextRange conversion as prep for notebook support~~ [ty] Refactor Range to/from TextRange conversion as prep for notebook support Nov 5, 2025

MichaReiser removed request for carljm, dcreager and sharkdp November 5, 2025 13:21

MichaReiser merged commit 7009d60 into main Nov 5, 2025
40 checks passed

MichaReiser deleted the micha/lsp-range branch November 5, 2025 13:24

[ty] Refactor Range to/from TextRange conversion as prep for notebook support #21230

[ty] Refactor Range to/from TextRange conversion as prep for notebook support #21230

Uh oh!

Conversation

MichaReiser commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

github-actions bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

github-actions bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dhruvmanila Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sharkdp commented Nov 3, 2025

Uh oh!

dhruvmanila left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[ty] Refactor `Range` to/from `TextRange` conversion as prep for notebook support #21230

[ty] Refactor `Range` to/from `TextRange` conversion as prep for notebook support #21230

MichaReiser commented Nov 3, 2025 •

edited

Loading

github-actions bot commented Nov 3, 2025 •

edited

Loading

github-actions bot commented Nov 3, 2025 •

edited

Loading

`mypy_primer` results

dhruvmanila Nov 4, 2025 •

edited

Loading