Skip to content

Conversation

@MaxAtoms
Copy link
Collaborator

@MaxAtoms MaxAtoms commented Oct 31, 2025

Adds a REPL line parser to all queries that accept one or multiple slice criteria 🎉

Dataframe Shape
Single criterion (optional)

R> :query @df-shape (18@df3) file://test/testfiles/dataframe.R
Query: df-shape (3 ms)
   ╰ 18@df3: {"colnames":["id","score","level","category"],"cols":[3,4],"rows":[0,4]}
All queries together required ≈11 ms (1ms accuracy, total 12 ms)

Lineage
Single criterion

R> :query @lineage (11@sum) file://test/testfiles/example.R
Query: lineage (11 ms)
   ╰ 11@sum: {40, 0, 23, built-in:sum, ... (see JSON)}
All queries together required ≈11 ms (1ms accuracy, total 12 ms)

Location-Map
Multiple criteria (optional)

R> :query @location-map (11@sum;12@product) file://test/testfiles/example.R
Query: location-map (3 ms)
   ╰ File List:
      ╰ 0: `test/testfiles/example.R`
   ╰ Id List: {40, 48}
All queries together required ≈6 ms (1ms accuracy, total 8 ms)

Origin
Single criterion

R> :query @origin (11@sum) file://test/testfiles/example.R
Query: origin (11 ms)
   ╰ Origins for {11@sum}
        ╰ {"type":0,"id":0}, {"type":0,"id":23}

Resolve Value
Multiple criteria

R> :query @resolve-value (1@sum;4@N) file://test/testfiles/example.R
Query: resolve-value (7 ms)
   ╰ Values for {1@sum, 4@N}
        ╰ [0L, 0L], [10L, 10L]
All queries together required ≈7 ms (1ms accuracy, total 9 ms)

Static Slice
Multiple criteria, backward and forward slice

R> :query @static-slice (11@sum) file://test/testfiles/example.R
Query: static-slice (9 ms)
   ╰ Slice for {11@sum}
     ╰ Code (newline as <code>&#92;n</code>): <code>sum <- 0\nw <- 7\nN <- 10\nfor(i in 1:(N-1)) sum <- sum + i + w\nsum</code>
All queries together required ≈9 ms (1ms accuracy, total 10 ms)
R> :query @static-slice (11@sum)f file://test/testfiles/example.R
Query: static-slice (7 ms)
   ╰ Slice for {11@sum}
     ╰ Code (newline as <code>&#92;n</code>): <code>cat("Sum:", sum, "\n")</code>
All queries together required ≈7 ms (1ms accuracy, total 8 ms)

Example Code

test/testfiles/dataframe.R:

library(dplyr)

df1 <- data.frame( 
	id = 1:4, 
	age = c(25, 32, 35, 40),
	score = c(90, 85, 88, 92)
)
df2 <- data.frame( 
	id = c(1, 2, 4),
	category = c("A", "B", "A") 
)
df3 <- df1 %>% 
	filter(age > 30) %>% 
	mutate(level = score^2) %>% 
	left_join(df2, by = "id") %>%
	select(-age)
	
print(df3$level)

@MaxAtoms MaxAtoms self-assigned this Oct 31, 2025
default: current += c;
}
} else if(c === split && !inQuotes && current !== '') {
} else if(inputString.slice(i, i + split.length) === split && !inQuotes && current !== '') {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We look for a character sequence here, but only skip the first character of the sequence...any ideas on how to write this more clearly, @EagleoutIce? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean with skipping the first character here? we could also try a peek ahead that pattern matches against a regex for that matter, but in genreal i dont think this looks that bad

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We match on the split string (e.g. ;:) and if it is present, we forward over the semicolon and start the next argument with the :. That behavior is correct for our single specific use case currently, but it might be surprising in general.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Todo: Use regex to allow arbitrary number of whitespace chars between the semicolon and colon

@MaxAtoms MaxAtoms requested a review from EagleoutIce November 4, 2025 10:04
@MaxAtoms MaxAtoms marked this pull request as ready for review November 4, 2025 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants