Convert julia-repl blocks to jldoctest format #58594
Conversation
|
This has been mostly written by various AI agents - it's been a bit of my go to test case. All of them struggle with this quite a bit. However, I do think the changes are useful and I'm mostly out of patience with the AI agents, so this is the rollup of the useful changes. |
Convert appropriate julia-repl code blocks to jldoctest format to enable automatic testing. In addition, this introduces a new `nodoctest = "reason"` pattern to annotate code blocks that are deliberate not doctested, so future readers will know not to try. Many code blocks are converted, in particular: - Manual pages: arrays.md, asynchronous-programming.md, functions.md, integers-and-floating-point-numbers.md, metaprogramming.md, multi-threading.md, performance-tips.md, variables.md, variables-and-scoping.md - Base documentation: abstractarray.jl, bitarray.jl, expr.jl, file.jl, float.jl, iddict.jl, path.jl, scopedvalues.md, sort.md - Standard library: Dates/conversions.jl, Random/RNGs.jl, Sockets/addrinfo.jl Key changes: - Add filters for non-deterministic output (timing, paths, memory addresses) - Add setup/teardown for filesystem operations - Fix parentmodule(M) usage in expr.jl for doctest compatibility - Document double escaping requirement for regex filters in docstrings - Update AGENTS.md with test running instructions Note: Some julia-repl blocks were intentionally left unchanged when they demonstrate language internals subject to change or contain non-deterministic output that cannot be properly filtered. Refs #56921
| @@ -76,7 +76,14 @@ The statement `x[1] = 42` *mutates* the object `x`, and hence this change *will* | |||
| by the caller for this argument. On the other hand, the assignment `y = 7 + y` changes the *binding* ("name") | |||
| `y` to refer to a new value `7 + y`, rather than mutating the *original* object referred to by `y`, | |||
| and hence does *not* change the corresponding argument passed by the caller. This can be seen if we call `f(x, y)`: | |||
| ```julia-repl | |||
| ```jldoctest | |||
There was a problem hiding this comment.
Should have carried state from the definition above
There was a problem hiding this comment.
so... we should give this block a label, and change the preceding block defining f to also use that label (and presumably turn it into a jldoctest block for that as well... ?
I.e. like this (I cannot post this as a code change comment here because the block above is "out of range")
For example, in the function
```jldoctest argpassing; output = false
function f(x, y)
x[1] = 42 # mutates x
y = 7 + y # new binding for y, no mutation
return y
end
# output
(generic function with 1 method)
```
The statement .... blah ...
```jldoctest argpassing
julia> a = [4, 5, 6]
...
```
| @@ -318,7 +318,6 @@ if use_revise | |||
| end | |||
| function maybe_revise(ex) | |||
| use_revise || return ex | |||
| STDLIB_DIR = Sys.STDLIB | |||
| STDLIBS = filter!(x -> isfile(joinpath(STDLIB_DIR, x, "src", "$(x).jl")), readdir(STDLIB_DIR)) | |||
There was a problem hiding this comment.
I am very surprised that this still works. I think the removed lines should be kept here
There was a problem hiding this comment.
I think this is OK because in lines 18 and 20 of this file, we define global const STDLIB_DIR = ...
| julia> map(tuple, 1/(i+j) for i=1:2, j=1:2, [1:4;]) | ||
| ERROR: syntax: invalid iteration specification | ||
| ERROR: ParseError: | ||
| # Error @ none:1:44 |
There was a problem hiding this comment.
is this line number none:1:44 stable?
There was a problem hiding this comment.
I think it is stable because 1 is just "the current line" and 44 is the character offset where the error is, which also is fixed here as long as the input stays fixed.
| @@ -1006,7 +1006,10 @@ julia> r"\x" | |||
| r"\x" | |||
|
|
|||
| julia> "\x" | |||
| ERROR: syntax: invalid escape sequence | |||
| ERROR: ParseError: | |||
| # Error @ none:1:2 | |||
There was a problem hiding this comment.
same answer: it should be fine :-)
| julia> try = "No" | ||
| ERROR: syntax: unexpected "=" | ||
| ERROR: ParseError: | ||
| # Error @ none:1:1 |
There was a problem hiding this comment.
and twice in this code block
| ```jldoctest mysum_example | ||
| julia> mysum(x::Real, y::Real) = x + y | ||
| mysum (generic function with 1 method) |
There was a problem hiding this comment.
If we want to preserve the style of the original text, this and similar snippets could be turned into
| ```jldoctest mysum_example | |
| julia> mysum(x::Real, y::Real) = x + y | |
| mysum (generic function with 1 method) | |
| ```jldoctest mysum_example ; output = false | |
| mysum(x::Real, y::Real) = x + y | |
| # output | |
| mysum (generic function with 1 method) |
| - The output contains arrays with undefined memory (e.g. from `undef` or `similar`) | ||
| - The output contains random numbers | ||
| - The output contains timing information | ||
| - The output contains file system pathts |
There was a problem hiding this comment.
| - The output contains file system pathts | |
| - The output contains file system paths |
| ### Teardown code | ||
|
|
||
| If you need teardown code (e.g. to delete created temporary files or to reset | ||
| the current directory), you can use the `teardown =` option: |
There was a problem hiding this comment.
Yay for seeing this feature arrive here 🥳
Convert appropriate julia-repl code blocks to jldoctest format to enable
automatic testing. In addition, this introduces a new
nodoctest = "reason"pattern to annotate code blocks that are deliberate not doctested, so future
readers will know not to try.
Many code blocks are converted, in particular:
integers-and-floating-point-numbers.md, metaprogramming.md,
multi-threading.md, performance-tips.md, variables.md,
variables-and-scoping.md
float.jl, iddict.jl, path.jl, scopedvalues.md, sort.md
Sockets/addrinfo.jl
Key changes:
Note: Some julia-repl blocks were intentionally left unchanged when they
demonstrate language internals subject to change or contain
non-deterministic output that cannot be properly filtered.
Refs #56921