Skip to content

bindings/python: +fix Preserve parallel edge symbols, tighten the Symbol width, and stop mutating noodlify arguments - #880

Draft
Adda0 wants to merge 1 commit into
develfrom
issue-768-python-fixes-pr
Draft

Adda0 wants to merge 1 commit into
develfrom
issue-768-python-fixes-pr

Conversation

@Adda0

@Adda0 Adda0 commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Nfa.to_networkx_graph() kept only the last symbol per (source, target)
pair; all symbols now land in a sorted 'symbols' edge attribute, with a
scalar 'symbol' kept for single-symbol edges. The Cython Symbol typedef
was 64-bit while mata::Symbol is 32-bit, silently wrapping large
symbols at the boundary; it is now 'unsigned int', so out-of-range
values raise OverflowError instead. unify_final() returned None instead
of self, str(nfa) showed the b'' bytes prefix on symbol names, several
.pxd/.pyi declarations were stale (removed initial/final mutator
overloads, make_complete returning void), and noodlify_for_equation()
with params={"reduce": ...} unified the initial and final states of the
caller's left-side automata in place. The C++ side now gets copies of
them, owned for the whole call so the raw pointers stay valid.

Regression tests cover every item: parallel symbols survive the
networkx conversion, 2**32-size symbols are rejected, unify_final()
chains like unify_initial(), str() decodes symbol names, and the
left-side automata after noodlify_for_equation(..., reduce="forward")
have the same states and initial states as before the call.

Fixes #768.

Adda0 added a commit that referenced this pull request Oct 16, 2026
…mata_str (#957)

`copy.copy(nfa)` and `copy.deepcopy(nfa)` raised "no default __reduce__
due to non-trivial __cinit__"; only the custom `deepcopy()` method
existed, and it does not copy the label. `Nfa.__copy__` copies the
automaton with the C++ copy constructor and shares the label;
`Nfa.__deepcopy__` also deep-copies the label and registers the copy in
the memo, so an automaton referenced twice in a deep-copied container
stays a single object. The alphabet is shared in both cases, as the C++
copy constructor does.

`Nfa.to_mata_str(alphabet=None)` binds `Nfa::print_to_mata`, as
`Nft.to_mata_str()` already does for transducers. Symbols are named
from the given alphabet, else from the automaton's own alphabet, else
printed as numbers.

Part of #755.

### Tests

`test_nfa.py`: `to_mata_str()` with numeric symbols, with an explicit
alphabet and with the automaton's own alphabet;
`copy.copy`/`copy.deepcopy` keep the states, the initial and final
states and the delta, are independent of the original, share/copy the
label, and a label that refers back to the automaton resolves to the
copy. Dropping the memo registration fails that last check.

No performance claim: these are missing features of the issue (items c).

### Overlap

No overlap with #880 or #895; rebased onto #892 (merged) without
conflicts. Independent of the other #755 PRs; all of them merge together
without conflicts.
Adda0 added a commit that referenced this pull request Oct 16, 2026
…luctant replace without dead states (#943)

Segmentation kept every ε-transition twice: in an unordered_map of
vectors keyed by depth and in a nested unordered_map keyed by depth and
source. Each state is expanded once and all its ε-moves are handled in
a row, so its ε-transitions are contiguous at a single depth. They now
live once in a vector indexed by depth, with a per-state span into it.
get_epsilon_transitions_by_depth() and get_epsilon_transitions(state)
expose them without copies; split_aut_into_segments(), noodlify() and
noodlify_mult_eps() use them. get_epsilon_depths() keeps its type for
the Python binding and is built on its first call, safely for
concurrent const calls; get_epsilon_depth_trans_map() is built the
same way and deprecated. Segmentation stays copy- and
move-constructible: a copy carries the ε-transition index and builds
its own views on demand.

noodlify_mult_eps() probed segments_one_initial_final with
find({target, final}) for every final state of the next segment after
every ε-transition. The ordered map keeps the pairs of one initial
state contiguous, so one lower_bound scan finds exactly the connected
ones. The hits are visited in the order of the segment's final states,
so the noodle order is unchanged. This also answers the "should we use
unordered_map?" TODO: an unordered map would lose that contiguity.

reluctant_nfa_with_marker() added a marker self-loop to every state,
intersected the result with a two-state guard NFA and reduced it. The
copy of a final state behind a marker was a dead lasso that ended up
in the NFT. In the reduced product, the after-marker copy of a
non-final state merges with its letter copy and the one behind a final
state is dead. The function now builds this result directly: the part
of the NFA reachable over alphabet letters, with final states without
successors and a marker self-loop on every non-final state. Symbols
outside the alphabet are still dropped.

reluctant_leftmost_nft() ended the replacement with three or four
ε-transitions: insert_word_by_levels() created a fresh level-1 target,
then an ε to a level-0 state that was inserted into and cleared from
the final states, and for ReplaceMode::All two more ε-steps back to the
initial state. The last replacement letter (or the single ε for an
empty replacement) now goes straight to the initial state, or to the
final state for ReplaceMode::Single.

For the example from #568 (replace 'a' by 'f' over {a, f}),
reluctant_leftmost_nft() drops from 15 to 9 states and has no isolated
states, and replace_reluctant_regex() drops from 20 to 12 states.

begin_marker_nfa() reverted the marker DFA and then swapped its initial
and final states back, with a TODO asking for a flag on revert(). An
internal revert_delta() helper now names that step; the public revert()
is unchanged, as no other caller needs the flag.

Segmentation::get_segments() trims each raw segment with nfa::trim()
instead of trimming a copy in place (#745, item 1).

The tests of reluctant_leftmost_nft() compare against hard-coded NFTs
with nft::are_equivalent(), which reads ε as a letter. Their expected
NFTs are regenerated.

Fixes #795.
Fixes #568.
Part of #745.

## Measurements

Throwaway harness linked against `build/release/g++/src/libmata.a` of
this commit and of `devel` (b6f7fd1), g++ `-O3 -DNDEBUG`. The machine
was loaded (load average ~23 on 16 cores), so user-space instruction
counts (`perf stat -e instructions:u`, deterministic) are the primary
figure; wall clock is the median of 9 alternating base/new pairs on one
pinned core.

Workloads:
- `replace`: `replace_reluctant_regex()` for 11 regexes (`a`, `ab`,
`a+b`, `(a|b)c*`, `a*`, `(ab|ba)+c?`, `a(b|c)*a`, `(a|b)*c(a|b)`,
`a+b+c`, `cb+a+`, `(a|b|c)*ab`) × replacements `''`, `d`, `ddd` × both
modes.
- `seg`: `Segmentation` + `get_segments()`.
- `noodle`: segment automaton with 4 segments of 40 states, 12 ε-sources
per depth (~23k noodles).
- `wide`: 3 segments of 800 states, 400 ε-sources per depth, letters
only inside blocks of 4 states, so most pair probes miss.

| workload | devel instr. | this PR instr. | change |
|---|---:|---:|---:|
| replace ×20 | 554,189,026 | 435,911,013 | −21.3 % |
| noodle seg ×200 | 458,281,680 | 332,013,904 | −27.6 % |
| noodle `noodlify_mult_eps` ×4 | 508,649,431 | 509,477,078 | +0.2 % |
| noodle `noodlify` ×4 | 331,395,567 | 325,482,021 | −1.8 % |
| wide seg ×20 | 415,027,966 | 298,211,326 | −28.1 % |
| wide `noodlify_mult_eps` ×1 | 336,822,446 | 311,681,463 | −7.5 % |
| wide `noodlify` ×1 | 585,575,482 | 535,628,445 | −8.5 % |

| workload (wall clock) | devel median | this PR median | median ratio |
|---|---:|---:|---:|
| replace ×40 | 144.1 ms | 112.2 ms | 0.80 |
| noodle seg ×400 | 107.1 ms | 71.2 ms | 0.67 |
| wide `noodlify_mult_eps` ×2 | 82.1 ms | 70.0 ms | 0.89 |

The range scan in `noodlify_mult_eps()` gains only when the next segment
has many final states that a given ε-target does not reach (`wide`);
with few final states (`noodle`) it is flat. Most of the time in
noodlification remains in `segs_one_initial_final()`, which this PR does
not touch.

Equivalence checks with the same harness, base vs this PR:
- noodles of `noodlify_mult_eps()` and `noodlify()` and the segments of
`get_segments()` on both generated automata: byte-identical
order-sensitive digests (segment state/transition counts and ε counters
per noodle, in output order; segments printed in full).
- `reluctant_leftmost_nft()` for all regex × replacement × mode
combinations: identical images (ε removed) of every input word up to
length 5 over `{a, b, c, BEGIN_MARKER}`; `replace_reluctant_regex()`:
identical images of every word up to length 6 over `{a, b, c}`;
`reluctant_nfa_with_marker()`: identical word sets up to length 6. 66
NFTs, 6,762 distinct output words.

State counts of `replace_reluctant_regex()` drop on every case, e.g. `a`
→ `d` (Single) 24 → 20, (All) 21 → 13; `ab` → `ddd` (All) 43 → 31.

## Tests

- `reluctant_nfa_with_marker()`: no dead marker state (state counts,
trim-stable, deterministic, marker accepted inside but not after a
match); symbols outside the alphabet are still dropped. Fails on devel
(`3 == 2`, `4 == 3`, `5 == 4`; trim shrinks the result).
- `reluctant_leftmost_nft()`: the #568 example has 9 states and is
trim-stable (devel: 15, trim 13). Expected NFTs of the three existing
sections regenerated (`nft::are_equivalent()` reads ε as a letter);
semantic equality with the old NFTs is checked above.
- `Segmentation::get_epsilon_transitions()`: per-depth vector, per-state
views over several ε symbols, states without ε-transitions and out of
range. Fails when the view is truncated.
- `Segmentation` copies and moves: copy- and move-constructible
(static_assert), and a copy or move made before or after the lazy
getters ran, queried after the source is destroyed, returns the same
`get_epsilon_depths()`, `get_epsilon_transitions()` and segments. Fails
to compile when the lazy views are not copyable.
- The existing index-compared noodle tests catch a changed noodle order
(checked by reversing the sort).

`just tests-cpp::check` on devel ba4a338 (debug and release): `test
cases: 330 | 324 passed | 2 skipped | 4 failed as expected` both times.
`just bindings::python::check`: `107 passed, 3 skipped, 2 xfailed`.
(Measurements above were taken on devel b6f7fd1; the commits since
then do not touch the strings code.)

## Overlap with open PRs

- #878 (draft) edits `end_marker_dfa()` / `marker_nft()` in `replace.cc`
and appends tests at the end of `tests/applications/strings/replace.cc`.
This PR does not touch those functions and adds its tests next to the
`reluctant_*` test cases, so no textual conflict is expected.
- #939 changes the `unused_state` line in
`noodlify()`/`noodlify_mult_eps()` and the `curr_state` line in
`reluctant_leftmost_nft()`; both are adjacent to lines changed here, so
whichever lands second needs a trivial rebase.
- #910 changes `determinize()` calls in `replace.cc` elsewhere; #872
changes `noodlify_for_equation()`; no overlap.
- #880 (draft) touches the Python `strings.pyx`; `get_epsilon_depths()`
keeps its type here, so the binding is unaffected.
Adda0 added a commit that referenced this pull request Oct 16, 2026
… once (#958)

`Delta.state_post()`, `Delta.get_transitions_from()` and
`Nfa.get_transitions_from_state()` copied the whole `StatePost` and
then its `to_vector()` before converting it to Python objects. They now
read the post through a const pointer into the delta; the two
`SymbolPost` getters share one helper.

`Segmentation.get_segments()` and `get_untrimmed_segments()` copied the
segment vector by value, then each segment, then assigned its initial
and final states and its delta into a fresh `Nfa`, which also dropped
the segment's alphabet. The getters are now declared to return the
const reference the C++ API returns, and each segment is copied once
into the wrapping `Nfa`, alphabet included.

Part of #755.

### Measurements

Release bindings, `taskset -c 13`, each value a median of 7 calls, 3
alternating runs, median shown. Posts: one state with 2000 symbol posts
of 8 targets each. Segments: a segment automaton of 4 segments with 5000
states each.

| | devel | this branch |
|---|---|---|
| `Delta.state_post()` | 0.94 ms | 0.68 ms |
| `Nfa.get_transitions_from_state()` | 0.96 ms | 0.59 ms |
| `Delta.get_transitions_from()` | 5.18 ms | 3.17 ms |
| `Segmentation.get_segments()` | 7.60 ms | 1.33 ms |
| `Segmentation.get_untrimmed_segments()` | 33.06 ms | 10.12 ms |

Measured on devel `ba4a338a78`; the commits since (#948, #910, #890,
#883, #892, #944) do not change these code paths.

### Tests

`test_strings.py`: segments (trimmed and untrimmed) keep the alphabet of
the segmented automaton and are independent copies; this fails on
`devel` (`segment.alphabet` is `None`). `test_trans.py`: state posts and
transitions from a state with several targets per symbol, from a state
without transitions, and from a state past the delta.

### Overlap

- #892 (merged) already reads posts in place in
`get_trans_from_state_as_sequence()` and `iter_transitions_from()`; this
PR does the same for the three getters #892 left copying.
- No overlap with #880 or #895. Independent of the other #755 PRs; all
of them merge together without conflicts.
Adda0 added a commit that referenced this pull request Oct 16, 2026
No call of the bindings released the GIL, so Python threads running
automata operations serialised on it. The heavy entry points are now
declared `except + nogil` and called under `with nogil:`: determinize
(with and without the subset map), union, intersection (with and
without the product map), concatenate (with and without the state
maps), complement, revert, remove_epsilon, minimize, reduce (with and
without the state map), reduce_residual_after/with, compute_relation,
is_included (with and without a counterexample), equivalence_check,
and noodlify, noodlify_mult_eps, noodlify_for_equation_sides and
noodlify_for_transducer. The `params` dictionaries are converted to a
`ParameterMap` before the GIL is dropped, through a shared
`encode_params()` that strings.pyx cimports. The C++ library does not
synchronise anything, so the declarations document that the automata
and alphabets passed in must not be mutated by another thread meanwhile.

Declaring them `except +` also turns the C++ exceptions of the externs
that lacked it (determinize, union, intersection, concatenate, revert,
minimize, reduce_residual_*, the 4-argument is_included, both
are_equivalent overloads, compute_relation) into Python exceptions
instead of std::terminate.

`compute_relation` is declared to return by value, as the C++ function
does, and the result is assigned straight into the returned
`BinaryRelation` instead of being copied cell by cell through untyped
`get`/`set` calls. `BinaryRelation.to_matrix()` reads the relation
through typed C++ calls.

Concurrent calls on *different* automata still meet the function-local
static buffers of `StatePost::find` and `OrdVector::insert(const
OrdVector&)` (#731); #868 removes them and should be merged first.
noodlify_for_equation is left as it is; #880 rewrites it.

Part of #755.
Adda0 added a commit that referenced this pull request Oct 16, 2026
…rom the export (#956)

Exporting the transitions built one Python `Transition` object per
transition. `Nfa.transitions_as_arrays()` instead fills three parallel
int64 `array('q')` buffers (sources, symbols, targets) in one C++ pass
over the delta, in the order of `iterate()`. The pass holds the GIL: a
thread mutating the delta mid-pass would invalidate the iterators and
overrun the counted buffers.

`to_dataframe()` builds its columns from these buffers, wrapped with
`numpy.frombuffer` (numpy comes with pandas; pandas copies an
`array.array` element by element but an ndarray in one go). The columns
keep their int64 dtype.

`Nfa` could not be pickled ("no default __reduce__ due to non-trivial
__cinit__"), so it could not be passed to `multiprocessing` workers.
`Nfa.__reduce__` now pickles the number of states, the initial and the
final states, the columnar export and the alphabet; the rebuild imports
the transitions with `Delta.add_transitions` (#892). The label is the
pickle state, restored by `Nfa.__setstate__` after the automaton is
memoized, so a label referring back to the automaton round-trips.
Pickling an automaton whose alphabet cannot be pickled raises
`TypeError`, as before, instead of silently dropping the alphabet.

Part of #755.

### Measurements

Release bindings, 1 000 000 transitions (50 000 states, 64 symbols),
`taskset -c 12`, each value a median of 5 calls, 3 alternating runs,
median shown:

| | devel | this branch |
|---|---|---|
| `to_dataframe()` | 1236 ms | 37 ms |
| `transitions_as_arrays()` | — | 35.1 ms |
| `list(iterate())`, per-transition objects, for comparison (code
unchanged; the gap is run-to-run noise) | 286 ms | 163 ms |

Column dtypes are `int64` before and after. Measured on devel
`ba4a338a78`; the commits since do not change these code paths (#957
adds `copy` support and `to_mata_str` next to them). The fill now holds
the GIL instead of releasing it, one release/acquire pair less per call,
so the single-threaded figures stand.

### Tests

`test_nfa.py`: the columnar export equals `iterate()` element by element
and has typecode `q` (also on an automaton with no transitions);
`to_dataframe()` keeps three int64 columns; a pickle round trip keeps
the number of states (including an isolated state), initial and final
states, the delta and the label, and the restored automaton is
independent; a label that is the automaton itself round-trips to the
restored automaton (`RecursionError` before the label moved to the
pickle state); an unpicklable alphabet raises `TypeError`; an `Nfa`
passes to a `spawn` multiprocessing worker. Replacing the pickled state
count with 0 fails the round-trip and the multiprocessing test.

### Overlap

- Builds on #892 (merged): the unpickling rebuild uses its
`Delta.add_transitions`.
- #880 changes `to_networkx_graph`, so this PR leaves that function on
`iterate()`.
- Rebased onto #957 (merged), which adds `__copy__`/`__deepcopy__` next
to `__reduce__`; independent of the other #755 PRs.
Adda0 added a commit that referenced this pull request Oct 16, 2026
No call of the bindings released the GIL, so Python threads running
automata operations serialised on it. The heavy entry points are now
declared `except + nogil` and called under `with nogil:`: determinize
(with and without the subset map), union, intersection (with and
without the product map), concatenate (with and without the state
maps), complement, revert, remove_epsilon, minimize, reduce (with and
without the state map), reduce_residual_after/with, compute_relation,
is_included (with and without a counterexample), equivalence_check,
and noodlify, noodlify_mult_eps, noodlify_for_equation_sides and
noodlify_for_transducer. The `params` dictionaries are converted to a
`ParameterMap` before the GIL is dropped, through a shared
`encode_params()` that strings.pyx cimports. The C++ library does not
synchronise anything, so the declarations document that the automata
and alphabets passed in must not be mutated by another thread meanwhile.

Declaring them `except +` also turns the C++ exceptions of the externs
that lacked it (determinize, union, intersection, concatenate, revert,
minimize, reduce_residual_*, the 4-argument is_included, both
are_equivalent overloads, compute_relation) into Python exceptions
instead of std::terminate.

`compute_relation` is declared to return by value, as the C++ function
does, and the result is assigned straight into the returned
`BinaryRelation` instead of being copied cell by cell through untyped
`get`/`set` calls. `BinaryRelation.to_matrix()` reads the relation
through typed C++ calls.

Concurrent calls on *different* automata still meet the function-local
static buffers of `StatePost::find` and `OrdVector::insert(const
OrdVector&)` (#731); #868 removes them and should be merged first.
noodlify_for_equation is left as it is; #880 rewrites it.

Part of #755.
Adda0 added a commit that referenced this pull request Oct 16, 2026
No call of the bindings released the GIL, so Python threads running
automata operations serialised on it. The heavy entry points are now
declared `except + nogil` and called under `with nogil:`: determinize
(with and without the subset map), union, intersection (with and
without the product map), concatenate (with and without the state
maps), complement, revert, remove_epsilon, minimize, reduce (with and
without the state map), reduce_residual_after/with, compute_relation,
is_included (with and without a counterexample), equivalence_check,
and noodlify, noodlify_mult_eps, noodlify_for_equation_sides and
noodlify_for_transducer. The `params` dictionaries are converted to a
`ParameterMap` before the GIL is dropped, through a shared
`encode_params()` that strings.pyx cimports. The C++ library does not
synchronise anything, so the declarations document that the automata
and alphabets passed in must not be mutated by another thread meanwhile.

noodlify_for_equation_sides and noodlify_for_transducer unify the
initial and final states of their input automata in place. Each input
with more than one initial or final state is now copied before the
call, so the call never writes an automaton shared with the caller or
with a concurrent call, and the caller's automata are no longer
modified. `SparseSet::size()` is declared as the method it is; the
declaration as a field could not be called.

The result of `compute_relation` is assigned straight into the
returned `BinaryRelation` instead of being copied cell by cell through
untyped `get`/`set` calls. `BinaryRelation.to_matrix()` reads the
relation through typed C++ calls.

noodlify_for_equation is left as it is; #880 rewrites it.

Part of #755.
…bol width, and stop mutating noodlify arguments

Nfa.to_networkx_graph() kept only the last symbol per (source, target)
pair; all symbols now land in a sorted 'symbols' edge attribute, with a
scalar 'symbol' kept for single-symbol edges. The Cython Symbol typedef
was 64-bit while mata::Symbol is 32-bit, silently wrapping large
symbols at the boundary; it is now 'unsigned int', so out-of-range
values raise OverflowError instead. unify_final() returned None instead
of self, str(nfa) showed the b'' bytes prefix on symbol names, several
.pxd/.pyi declarations were stale (removed initial/final mutator
overloads, make_complete returning void), and noodlify_for_equation()
with params={"reduce": ...} unified the initial and final states of the
caller's left-side automata in place. The C++ side now gets copies of
them, owned for the whole call so the raw pointers stay valid.

Regression tests cover every item: parallel symbols survive the
networkx conversion, 2**32-size symbols are rejected, unify_final()
chains like unify_initial(), str() decodes symbol names, and the
left-side automata after noodlify_for_equation(..., reduce="forward")
have the same states and initial states as before the call.

Fixes #768.
@Adda0
Adda0 force-pushed the issue-768-python-fixes-pr branch from a6141c8 to 5557f8a Compare October 16, 2026 11:39
Adda0 added a commit that referenced this pull request Oct 16, 2026
…#955)

No call of the bindings released the GIL, so Python threads running
automata operations serialised on it. The heavy entry points are now
declared `except + nogil` and called under `with nogil:`: determinize
(with and without the subset map), union, intersection (with and
without the product map), concatenate (with and without the state
maps), complement, revert, remove_epsilon, minimize, reduce (with and
without the state map), reduce_residual_after/with, compute_relation,
is_included (with and without a counterexample), equivalence_check,
and noodlify, noodlify_mult_eps, noodlify_for_equation_sides and
noodlify_for_transducer. The `params` dictionaries are converted to a
`ParameterMap` before the GIL is dropped, through a shared
`encode_params()` that strings.pyx cimports. The C++ library does not
synchronise anything, so the declarations document that the automata
and alphabets passed in must not be mutated by another thread meanwhile.

noodlify_for_equation_sides and noodlify_for_transducer unify the
initial and final states of their input automata in place. Each input
with more than one initial or final state is now copied before the
call, so the call never writes an automaton shared with the caller or
with a concurrent call, and the caller's automata are no longer
modified. `SparseSet::size()` is declared as the method it is; the
declaration as a field could not be called.

The result of `compute_relation` is assigned straight into the
returned `BinaryRelation` instead of being copied cell by cell through
untyped `get`/`set` calls. `BinaryRelation.to_matrix()` reads the
relation through typed C++ calls.

noodlify_for_equation is left as it is; #880 rewrites it.

Part of #755.

### Measurements

Release bindings (CPython 3.14, Cython 3.3), this branch vs `devel`,
alternating runs, throwaway scripts (not committed). Machine shared with
other builds, so absolute times are noisy; the contrast is what matters.

GIL release, `determinize()` of a random NFA (300 states, 1200
transitions, 2 symbols), `taskset -c 4,5,6`, 5 alternating runs:

| | devel | this branch |
|---|---|---|
| 3 threads at once / 1 call, wall-time ratio (median) | 5.69 | 1.66 |
| 3 threads at once, wall time (median) | 1.73 s | 0.869 s |

Main thread counting in a pure-Python loop while a second thread runs
one `determinize()` (`taskset -c 4,5`, 3 alternating runs, median):
devel 0.47 M iterations/s (the main thread barely runs), this branch
5.86 M iterations/s (idle baseline 9.5–14.5 M/s).

The GIL measurements were taken on devel `ba4a338a78`; the rebase onto
the current `devel` did not change the binding code they measure.

`compute_relation()` and `to_matrix()` on a random NFA with N=1500
states (6000 transitions, 2 symbols, 50 final states), `taskset -c 6`,
each value a median of 5 calls, 5 alternating runs, median shown,
measured on devel `1ff4da094c` (which already declares
`compute_relation` to return by value, #939; the commits since, #871,
#947 and #872, do not touch the bindings):

| | devel | this branch |
|---|---|---|
| `compute_relation()` | 733 ms | 450 ms |
| `BinaryRelation.to_matrix()` | 139 ms | 15 ms |

### Tests

`test_nfa.py`: determinize (with subset map), intersection, minimize and
reduce run in three threads at once and must match the single-threaded
results; `test_heavy_operations_let_other_threads_run` checks, for
determinize (with and without the subset map), minimize, complement,
reduce, compute_relation and intersection, that the main thread runs
Python code while the call is still running in another thread
(`runs_without_the_gil` fixture in `conftest.py`: with
`sys.setswitchinterval(60)` the main thread can only run before the
worker finishes if the call releases the GIL). Replacing `with nogil:`
by `if True:` in `determinize` or `reduce` makes the matching case fail.

`test_strings.py`: the four noodlify variants run in three threads at
once on shared inputs with two initial and two final states, and every
returned automaton, epsilon counter and input/output index must match
the single-threaded results; `noodlify_for_equation_sides` and
`noodlify_for_transducer` must leave their inputs unchanged and must let
the main thread run during the call. Dropping the input copy makes the
unchanged-inputs test fail, and replacing `with nogil:` by `if True:` in
`noodlify_for_equation_sides` makes its thread test fail. Existing
`compute_relation`/`to_matrix` tests cover the relation copy.

On devel `8b4c09bbc6`: Python bindings 153 passed, 3 skipped, 2 xfailed;
C++ suite (debug and release) 363 passed, 2 skipped, 4 failed as
expected.

### Merge order and overlap

- #868 (static buffers of `StatePost::find` and `OrdVector::insert`,
#731), on which this PR was stacked, is merged; this PR now targets
`devel` directly.
- #880 rewrites `noodlify_for_equation`; this PR leaves that function
alone.
- #939 (32-bit `State`) already added `except +` to the externs this PR
redeclares, and declares `compute_relation` to return by value; this PR
only adds `nogil` to them.
- Independent of the other #755 PRs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant