Repository navigation
Expand file tree
/
Copy pathcheck_native_c_api_surface.py
More file actions
579 lines (526 loc) · 31.3 KB
/
Copy pathcheck_native_c_api_surface.py
File metadata and controls
579 lines (526 loc) · 31.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
#!/usr/bin/env python3
"""Every NativeStrategyHost member has a C spelling or a stated exclusion.
R5 lane N8. The audit's G1-b finding was that fourteen C++ capabilities had no
C spelling and that the header's own exclusion list named three unrelated ones,
so a reader could not tell "not exposed" from "nobody looked". This guard
removes that possibility mechanically:
* the public surface of `class NativeStrategyHost`
(include/pineforge/native_host.hpp) is the authority for WHAT exists;
* the COVERAGE block in include/pineforge/native_c_api.h is the authority for
what each member's C spelling is, or why it has none;
* the two must agree EXACTLY -- no member missing from the block, no row in
the block naming a member that no longer exists.
A `[C]` row must also name a symbol or field that really is declared in the C
headers, so a spelling cannot be claimed for something that was never added.
Exclusion rows carry their reason in the same line and are free text.
R5 lane E15 adds the members that census cannot see: a protected member of
the base, `class BacktestEngine` (include/pineforge/engine.hpp), that a host
is documented to call or override from its callbacks. Those are opt-in: a
`// @host-seam` line marks each one's declaration, and the block's
BASE-CLASS SEAMS list must name exactly the marked set, under the same row
rules. A marker outside the class, or one that does not precede a member
function, fails too, so a marker cannot quietly mark nothing.
R5 lane F4 adds the enumerations (audit P4-b). Every `typedef enum` in the two
C headers is classified here, once: either it is the C twin of a kernel
`enum class` (ENUM_TWINS names the header and the enumeration), or it has no
enumeration to mirror (C_ONLY says why -- a std::variant's alternative index, a
bit mask, a bool, a C-only protocol). For a twin, the kernel enumeration is
parsed out of its header and compared by VALUE: every kernel enumerator needs
a C enumerator of the same value unless it is excluded by name with a reason,
an excluded one must stay unspelled, and every C enumerator must name a kernel
value. So a kernel enumerator appended without a C name fails here -- the
static_asserts in src/native_c_host.cpp pin the names they list, and cannot see
one that nobody listed -- and so does a C enumeration nobody classified.
R5 lane H-DOCGATES adds the 1.0 C boundary (supervisor ruling, 2026-09-26:
C-SURFACE-2 moves to 1.1.0, so 1.0 lists every C++ capability the C surface
does not expose, with its reason and the row that pins it). The censuses above
are existence checks: a `[C]` row proves a C spelling exists, not that the C
record carries every C++ fact, and ENUM_TWINS starts from the C side. Each
C_V1_EXCLUSIONS row names one capability the C surface lacks: the C++
declaration that must still exist, and the C spellings that must stay absent
from a C record or from every C declaration. A pattern ends in `\w*`, so a
suffixed or prefixed spelling (`strategy_..._entry_comment_v1`,
`request_origin`) closes the gap as visibly as the plain one. The boundary
table of docs/pages/native-engine.md cites every row exactly once, and cites no
row this checker does not have.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
INCLUDE = ROOT / "include" / "pineforge"
HOST = INCLUDE / "native_host.hpp"
ENGINE = INCLUDE / "engine.hpp"
C_API = INCLUDE / "native_c_api.h"
PUBLIC_C = INCLUDE / "pineforge.h"
DOC_C_BOUNDARY = ROOT / "docs" / "pages" / "native-engine.md"
# Members that are not part of the host's own surface: the copy/move deletions
# the class spells out, and the friend declaration.
_SKIP = frozenset({"NativeStrategyHost", "operator="})
_ROW = re.compile(r"^\s*\*\s+\[(C|--)\]\s+(\w+)\s+(\S.*?)\s*$")
# Every C enumeration that mirrors a kernel `enum class`: C tag -> (the header
# under include/pineforge that declares the kernel enumeration, its name, the
# kernel enumerators deliberately left without a C name -> why).
ENUM_TWINS: dict[str, tuple[str, str, dict[str, str]]] = {
"pf_magnifier_distribution_e": ("magnifier.hpp", "MagnifierDistribution", {}),
"pf_close_cause_e": ("execution.hpp", "CloseCause", {}),
"pf_native_scope_claim_e": ("native_order.hpp", "ScopeClaim", {}),
"pf_native_side_e": ("native_order.hpp", "Side", {}),
"pf_native_size_time_e": ("native_order.hpp", "SizeTime", {}),
"pf_native_grid_policy_e": ("native_order.hpp", "ExecutionGridPolicy", {}),
"pf_native_anchor_rounding_e": ("native_order.hpp", "NativeAnchorRounding", {}),
"pf_native_arm_visibility_e": ("native_order.hpp", "NativeArmVisibility", {}),
"pf_native_arm_first_match_e": ("native_order.hpp", "NativeArmFirstMatch", {}),
"pf_native_arm_scope_e": ("native_order.hpp", "NativeArmScope", {}),
"pf_native_group_effect_e": ("native_order.hpp", "GroupEffect", {}),
"pf_native_request_field_e": ("native_host.hpp", "NativeRequestField", {}),
"pf_native_price_rule_e": ("native_host.hpp", "NativeCurrentPriceRule", {}),
"pf_native_refusal_e": ("native_host.hpp", "NativeCurrentRefusal", {}),
"pf_native_lifecycle_e": ("native_host.hpp", "NativeLifecycleKind", {}),
"pf_native_risk_limit_e": ("native_order.hpp", "RiskLimitKind", {}),
"pf_native_risk_day_e": ("native_run_spec.hpp", "NativeRiskDay", {}),
"pf_native_risk_action_e": ("native_run_spec.hpp", "NativeRiskAction", {}),
"pf_native_fee_kind_e": ("native_run_spec.hpp", "NativeFeeKind", {}),
"pf_native_close_execution_e": ("native_run_spec.hpp", "NativeCloseExecution", {}),
"pf_native_open_directions_e": ("native_run_spec.hpp", "NativeOpenDirections", {}),
"pf_native_report_policy_e": ("native_run_spec.hpp", "NativeReportPolicy", {
"KernelRecordedAtHostMarks":
"under it the host marks each report point, and the kernel's mark is "
"reached only by the Pine source host (not installed API); a C host has "
"no call that marks one and could only declare a series nobody records "
"(lane E22; the 1.0 C boundary table)",
}),
"pf_native_price_grid_e": ("native_run_spec.hpp", "NativePriceGrid", {}),
"pf_native_grid_rounding_e": ("native_run_spec.hpp", "NativeGridRounding", {}),
"pf_native_calc_trigger_e": ("native_run_spec.hpp", "NativeCalculationTrigger", {}),
"pf_native_open_bar_view_e": ("native_run_spec.hpp", "NativeOpenBarView", {}),
"pf_native_liquidation_sizing_e": ("native_run_spec.hpp", "NativeLiquidationSizing", {}),
"pf_native_event_retention_e": ("native_run_spec.hpp", "NativeEventRetention", {}),
"pf_native_series_source_e": ("native_run_spec.hpp", "NativeSeriesSource", {
"InstrumentFeed":
"a C run spec installs no instrument feed and a C subscription row carries no "
"instrument key, so a C series cannot read one (lane XSYM-D; the 1.0 C boundary "
"table)",
}),
"pf_native_size_price_e": ("native_order.hpp", "SizePrice", {}),
"pf_native_scope_basis_e": ("native_order.hpp", "ScopeBasis", {}),
"pf_native_liquidation_check_e": ("native_run_spec.hpp", "NativeLiquidationCheck", {}),
"pf_native_margin_equity_basis_e": ("native_run_spec.hpp", "NativeMarginEquityBasis", {}),
"pf_native_margin_level_base_e": ("native_run_spec.hpp", "NativeLiquidationLevelBase", {}),
"pf_native_sample_eligibility_e": ("native_run_spec.hpp", "SampleEligibility", {}),
"pf_native_slot_label_e": ("native_run_spec.hpp", "NativeSlotLabelPolicy", {}),
"pf_native_feed_tolerance_e": ("native_run_spec.hpp", "NativeFeedTolerance", {}),
"pf_native_path_order_e": ("native_run_spec.hpp", "NativePathOrder", {}),
"pf_native_abort_reporting_e": ("native_run_spec.hpp", "NativeAbortReporting", {}),
"pf_native_calc_reason_e": ("native_host.hpp", "NativeCalculationReason", {}),
"pf_native_margin_check_kind_e": ("native_host.hpp", "NativeMarginCheckKind", {}),
"pf_native_opened_lot_fill_point_e": ("engine.hpp", "OpenedLotFillPoint", {}),
"pf_native_price_provenance_e": ("market_driver.hpp", "NativePriceProvenance", {}),
"pf_native_path_phase_e": ("market_driver.hpp", "NativePathPhase", {}),
"pf_native_completion_kind_e": ("market_driver.hpp", "NativeCompletionKind", {}),
"pf_native_quote_kind_e": ("native_host.hpp", "NativeCurrentQuoteKind", {}),
"pf_native_terminal_reason_e": ("native_order.hpp", "AppliedTerminalReason", {}),
"pf_native_reject_reason_e": ("native_order.hpp", "RequestRejectReason", {}),
"pf_native_cancel_reason_e": ("native_order.hpp", "CancelReason", {}),
"pf_native_match_reject_e": ("native_order.hpp", "MatchRejectReason", {}),
"pf_native_activation_e": ("native_order.hpp", "ActivationKind", {}),
"pf_native_origin_e": ("native_order.hpp", "RequestOrigin", {}),
"pf_native_failure_code_e": ("native_host.hpp", "NativeFailureCode", {}),
"pf_native_failure_operation_e": ("native_host.hpp", "NativeFailureOperation", {}),
"pf_native_run_phase_e": ("native_host.hpp", "NativeRunPhase", {}),
# The phase of a recorded event (pineforge.h, group pf_outputs).
"pf_output_phase_e": ("native_host.hpp", "NativeRunPhase", {}),
"pf_native_completion_e": ("native_host.hpp", "NativeCompletion", {}),
"pf_native_spec_error_e": ("native_run_spec.hpp", "NativeRunSpecError", {}),
"pf_native_spec_field_e": ("native_run_spec.hpp", "NativeRunSpecField", {}),
"pf_native_append_error_e": ("native_host.hpp", "NativeAuxiliaryAppendError", {}),
"pf_native_opening_shape_e": ("native_order.hpp", "OpeningShape", {}),
"pf_native_candidate_price_e": ("native_order.hpp", "NativeCandidatePriceKind", {}),
"pf_native_driver_class_e": ("native_order.hpp", "DriverEligibilityClass", {}),
"pf_native_precommit_verdict_e": ("native_host.hpp", "NativePrecommitVerdict", {}),
"pf_native_anchored_trigger_e": ("native_host.hpp", "NativeAnchoredTrigger", {}),
"pf_native_fx_curve_error_e": ("native_fx_curve.hpp", "NativeFxCurveError", {}),
}
# Every C enumeration with no kernel `enum class` to mirror, and why. A
# variant's alternative index is pinned in src/native_c_host.cpp by
# std::variant_size and translated by a std::visit that names every
# alternative, so it cannot drift silently either.
C_ONLY: dict[str, str] = {
"pf_settings_status_t": "the generated-strategy checked settings C protocol; "
"no kernel enumeration exists",
"pf_execution_contract_e": "BacktestEngine::execution_contract() answers an int",
"pf_native_spec_optional_e": "bit mask of optional_mask; each bit gates one std::optional "
"field of NativeRunSpec",
"pf_fill_qty_partition_e": "the Pine source host's pending-order probe writes an int; "
"no kernel enumeration exists",
"pf_native_intent_e": "alternative index of the std::variant native_order::OrderIntent",
"pf_native_reduction_e": "alternative index of the std::variant native_order::ReductionSize",
"pf_native_size_basis_e": "alternative index of the std::variant native_order::SizeBasis",
"pf_native_trigger_e": "alternative index of the std::variant native_order::Trigger",
"pf_native_anchor_e": "alternative index of the std::variant native_order::TriggerAnchor",
"pf_native_capacity_e": "alternative index of the std::variant native_order::Capacity",
"pf_native_owner_e": "alternative index of the std::variant native_order::Owner",
"pf_native_group_e": "alternative index of the std::variant native_order::Group",
"pf_native_execute_outcome_e": "the non-refusal alternatives of the std::variant "
"NativeCurrentExecutionResult",
"pf_native_event_kind_e": "the std::variant native_order::CommandEvent's alternatives plus "
"the driver and account rows",
"pf_native_remaining_e": "alternative index of the std::variant "
"native_order::RemainingProjection",
"pf_native_trigger_state_e": "alternative index of the std::variant native_order::TriggerState",
"pf_native_spec_ext_mask_e": "bit mask of the present blocks of pf_native_run_spec_ext_v1",
"pf_native_lookahead_e": "NativeTimeframeSubscription::lookahead is a bool",
"pf_native_gaps_e": "NativeTimeframeSubscription::gaps is a bool",
"pf_native_intrabar_kind_e": "alternative index of the std::variant IntrabarPath::value",
"pf_native_answer_e": "the C answering-hook protocol; a C++ hook answers a std::optional",
"pf_native_plan_e": "alternative index of the std::variant native_order::ExecutionPlan",
}
# The 1.0 C boundary: key -> (the header under include/pineforge that declares
# the C++ capability, a spelling of it that must still be there, and the pairs
# (C region, pattern) whose pattern must stay absent from that region). A region
# is a C struct's body, or "@declarations": every declaration of the two C
# headers, comments cut out. docs/pages/native-engine.md's boundary table gives
# each row's reason and C route.
C_V1_EXCLUSIONS: dict[str, tuple[str, str, tuple[tuple[str, str], ...]]] = {
"replace_options": ("native_order.hpp", "struct ReplaceOptions", (
("@declarations", r"\b\w*(?:keep_handle|keep_binding|retain_trigger_state|replace_opt)\w*"),)),
"applied_event_tail": ("native_order.hpp", "struct ExecutionAppliedEvent", (
("pf_native_applied_v1", r"\b\w*(?:origin|label|comment|definition|first_trade"
r"|trade_count|allowance|scope|cursor)\w*"),
("@declarations", r"\bpf_native_applied_(?!v1\b)\w+"))),
"closed_entry_comment": ("engine.hpp", "std::string entry_comment;", (
("@declarations", r"\bstrategy_\w*entry_comment\w*"),
("pf_trade_s", r"\b\w*comment\w*"))),
"closed_rows_in_place": ("engine.hpp", "const Trade& closed_trade(std::size_t i) const", (
("@declarations", r"\bstrategy_(?:native_)?(?:closed_trade_count|closed_trade_get"
r"|closed_trade_row|report_trade)\w*"),)),
"working_definition": ("native_order.hpp", "std::optional<double> best_seed", (
("pf_native_working_v1", r"\b(?:reduce_\w+|size_\w+|side|grid_policy|reserve_percent_fee"
r"|fill_through|\w*best_seed\w*|predecessor\w*|root\w*|priority\w*"
r"|kept_binding\w*|host_sized\w*)\b"),)),
"open_ended_session_day": ("market_driver.hpp", "bool closes_session_day_open_ended", (
("pf_native_decision_v1", r"\b\w*open_ended\w*"),)),
"callback_contexts": ("market_driver.hpp", "native_calendar::NativeInterval input_interval", (
("pf_native_decision_v1", r"\b\w*(?:input_interval|source_price_time|driver_statistics"
r"|sub_bars_processed|quote_origin)\w*"),)),
"host_sized_opening": ("native_order.hpp", "enum class HostSizedKind", (
("pf_native_request_v1", r"\b\w*(?:host_sized|sized_kind)\w*"),)),
"terms_units": ("native_order.hpp", "struct ExecutionTerms", (
("pf_native_terms_v1", r"\b\w*units\w*"),)),
"excursion_fx": ("engine.hpp", "double account_fx = 1.0;", (
("pf_native_lot_excursion_v1", r"\b\w*fx\w*"),)),
"hash_sink": ("engine.hpp", "class BrokerStateHashSink", (
("@declarations", r"\b\w*hash_sink\w*|\bpf_native_hash\w*"),)),
"entry_bar_mask_phase": ("engine.hpp", "void declare_opened_lot_entry_bar_mask(", (
("@declarations", r"\bstrategy_native_declare_opened_lot_entry_bar_mask_(?!v1\b)\w+"),)),
"timeframe_undetected": ("native_run_spec.hpp", "bool timeframe_undetected", (
("@declarations", r"\b\w*undetected\w*"),)),
"instrument_feeds": ("native_run_spec.hpp", "struct NativeInstrumentFeed", (
("@declarations", r"\bpf_native_instrument\w*|\b\w*instrument_feed\w*"),
("pf_native_subscription_v1", r"\b\w*instrument\w*"))),
"inherited_statistics": ("engine.hpp", "double max_contracts_held_all() const", (
("@declarations", r"\bstrategy_\w*(?:max_contracts_held|capital_held|open_profit"
r"|gross_profit|gross_loss|net_profit|runup|drawdown)\w*"),)),
"hook_views": ("native_host.hpp", "struct NativeExecutionTermsFacts", (
("pf_native_terms_view_v1", r"\b\w*(?:definition|allowance)\w*"),
("pf_native_precommit_view_v1", r"\b\w*(?:definition|readiness)\w*"))),
"event_payloads": ("native_order.hpp", "enum class CommandSurface", (
("pf_native_event_v1", r"\b\w*(?:surface|authority|definition|attempted|requested_delta"
r"|kept_handle|allowance|projection)\w*"),)),
"command_results": ("native_order.hpp", "struct SubmitResult", (
("@declarations", r"\bpf_native_(?:submit|replace|cancel|execute)\w*result\w*"
r"|\b\w*event_ordinal\w*"),)),
"state_view": ("native_host.hpp", "struct NativeStateView", (
("pf_native_state_v1", r"\b\w*(?:spec|context|recipient|cause)\w*"),)),
"host_minor": ("engine.hpp", "void trace(const std::string& name, double value);", (
("@declarations", r"\bstrategy_(?:native_)?(?:trace|declare_\w*hook)\w*"),)),
"library_facilities": ("native_run_spec.hpp", "std::uint64_t native_run_spec_digest(", (
("@declarations", r"\b\w*(?:spec_digest|fx_curve_digest|validate_native|preflight_native"
r"|parse_timeframe|parse_session|quantity_on_grid|civil_date|civil_days"
r"|pairing_aggregates)\w*"),)),
"toolkit": ("native_toolkit.hpp", "inline BracketReceipt submit_bracket", (
("@declarations", r"\b\w*(?:bracket|order_book)\w*"),)),
}
_CITATION = re.compile(r'C_V1_EXCLUSIONS\["(\w+)"\]')
_C_ENUM = re.compile(r"typedef\s+enum\s+(\w+)\s*\{(.*?)\}\s*\w+\s*;", re.DOTALL)
_ENUMERATOR = re.compile(r"^\s*([A-Za-z_]\w*)\s*(?:=\s*(.+?))?\s*$", re.DOTALL)
def _strip_comments(text: str) -> str:
text = re.sub(r"/\*.*?\*/", " ", text, flags=re.DOTALL)
return re.sub(r"//[^\n]*", " ", text)
def _squeeze(text: str) -> str:
return re.sub(r"\s+", " ", text)
def c_v1_exclusion_failures(c_api_text: str, public_text: str) -> list[str]:
"""Hold the 1.0 C boundary table to the declarations: each row's C++
capability still exists, its C spelling is still absent, and the page
cites every row once and no row that is not here."""
failures: list[str] = []
declarations = _strip_comments(c_api_text) + "\n" + _strip_comments(public_text)
records = {name: body for name, body in re.findall(
r"\btypedef\s+struct\s+(\w+)\s*\{(.*?)\}\s*\w+\s*;", declarations, re.DOTALL)}
page = DOC_C_BOUNDARY.read_text(encoding="utf-8")
cited = _CITATION.findall(page)
for key in sorted(set(cited) - set(C_V1_EXCLUSIONS)):
failures.append(f'C_V1_EXCLUSIONS["{key}"]: the boundary table cites a row this '
"checker does not have")
for key, (header, cpp_spelling, pins) in C_V1_EXCLUSIONS.items():
cpp = _squeeze(_strip_comments((INCLUDE / header).read_text(encoding="utf-8")))
spelling = _squeeze(cpp_spelling)
whole = ((r"(?<!\w)" if re.match(r"\w", spelling) else "") + re.escape(spelling)
+ (r"(?!\w)" if re.search(r"\w$", spelling) else ""))
if not re.search(whole, cpp):
failures.append(f'C_V1_EXCLUSIONS["{key}"]: the C++ declaration `{cpp_spelling}` '
f"is gone from {header}: retire the row, or re-pin it")
if cited.count(key) != 1:
failures.append(f'C_V1_EXCLUSIONS["{key}"]: the boundary table in '
f"docs/pages/native-engine.md cites it {cited.count(key)} times, "
"not once")
for region, forbidden in pins:
if region == "@declarations":
shape = declarations
elif region in records:
shape = records[region]
else:
failures.append(f'C_V1_EXCLUSIONS["{key}"]: the C record {region} is gone')
continue
found = re.search(forbidden, shape)
if found:
failures.append(f'C_V1_EXCLUSIONS["{key}"]: `{found.group(0)}` is now declared '
f"in {region}: the gap has closed, so retire or narrow the row "
"and its boundary-table entry")
return failures
def _value(expr: str, known: dict[str, int]) -> int:
"""An enumerator's value: an integer literal (decimal or hex, any u/l
suffix, optionally negated), a `1u << n` shift, or an earlier
enumerator's name."""
expr = expr.strip()
shift = re.fullmatch(r"(\d+)[uUlL]*\s*<<\s*(\d+)[uUlL]*", expr)
if shift:
return int(shift.group(1)) << int(shift.group(2))
number = re.fullmatch(r"(-?)\s*(0[xX][0-9a-fA-F]+|\d+)[uUlL]*", expr)
if number:
magnitude = int(number.group(2), 0)
return -magnitude if number.group(1) else magnitude
if expr in known:
return known[expr]
raise ValueError(f"cannot evaluate enumerator value {expr!r}")
def _enumerators(body: str) -> list[tuple[str, int]]:
known: dict[str, int] = {}
out: list[tuple[str, int]] = []
following = 0
for item in _strip_comments(body).split(","):
if not item.strip():
continue
match = _ENUMERATOR.match(item)
if not match:
raise ValueError(f"cannot parse enumerator {item.strip()!r}")
name, expr = match.group(1), match.group(2)
value = _value(expr, known) if expr is not None else following
known[name] = value
out.append((name, value))
following = value + 1
return out
def c_enumerations(*texts: str) -> dict[str, list[tuple[str, int]]]:
"""Every `typedef enum <tag> { ... } <name>;` of the C headers, by tag."""
out: dict[str, list[tuple[str, int]]] = {}
for text in texts:
for match in _C_ENUM.finditer(_strip_comments(text)):
out[match.group(1)] = _enumerators(match.group(2))
return out
def kernel_enumeration(text: str, name: str) -> list[tuple[str, int]] | None:
"""The enumerators of `enum class <name>` in one header, or None."""
code = _strip_comments(text)
match = re.search(r"\benum\s+class\s+" + re.escape(name) + r"\b[^{;]*\{(.*?)\}\s*;",
code, re.DOTALL)
return _enumerators(match.group(1)) if match else None
def enum_twin_failures(c_texts: tuple[str, ...]) -> tuple[list[str], int, int]:
"""(failures, twinned enumerations, kernel enumerators covered)."""
failures: list[str] = []
try:
c_enums = c_enumerations(*c_texts)
except ValueError as error:
return [f"a C enumeration does not parse: {error}"], 0, 0
for tag in sorted(set(c_enums) - set(ENUM_TWINS) - set(C_ONLY)):
failures.append(f"{tag} is neither twinned to a kernel enumeration nor ruled C-only")
for tag in sorted((set(ENUM_TWINS) | set(C_ONLY)) - set(c_enums)):
failures.append(f"{tag} is classified but no C header declares it")
for tag in sorted(set(ENUM_TWINS) & set(C_ONLY)):
failures.append(f"{tag} is classified twice")
covered = 0
for tag, (header, name, excluded) in sorted(ENUM_TWINS.items()):
if tag not in c_enums:
continue
path = INCLUDE / header
try:
kernel = kernel_enumeration(path.read_text(encoding="utf-8"), name)
except (OSError, ValueError) as error:
failures.append(f"{tag}: cannot read `enum class {name}` in {header}: {error}")
continue
if kernel is None:
failures.append(f"{tag}: no `enum class {name}` in {header}")
continue
c_values = {value: c_name for c_name, value in c_enums[tag]}
kernel_values = {value for _, value in kernel}
kernel_names = {kernel_name for kernel_name, _ in kernel}
for stale in sorted(set(excluded) - kernel_names):
failures.append(f"{tag}: the exclusion {name}::{stale} names no enumerator")
for kernel_name, value in kernel:
if kernel_name in excluded:
if value in c_values:
failures.append(
f"{name}::{kernel_name} is excluded ({excluded[kernel_name]}) but "
f"{c_values[value]} spells its value {value}")
continue
covered += 1
if value not in c_values:
failures.append(f"{name}::{kernel_name} ({value}) has no C name in {tag}")
for c_name, value in c_enums[tag]:
if value not in kernel_values:
failures.append(f"{c_name} ({value}) names no enumerator of {name}")
return failures, len(set(ENUM_TWINS) & set(c_enums)), covered
# The opt-in marker for a base-class seam, and the heading of its list.
_SEAM_MARK = re.compile(r"^\s*//\s*@host-seam\b")
_SEAMS_HEADING = "BASE-CLASS SEAMS"
def host_members(text: str) -> set[str]:
start = text.index("class NativeStrategyHost : public BacktestEngine {")
end = text.index("\n};", start)
body = text[start:end]
# Only the public section: the class has one, ending at `friend class`.
body = body.split("friend class", 1)[0]
names: set[str] = set()
for line in body.splitlines():
stripped = line.strip()
if stripped.startswith(("//", "/*", "*", "#")) or not stripped:
continue
# A declaration is `<stuff> name(` at paren depth 0 for this line.
match = re.search(r"(~?\w+)\s*\(", stripped)
if not match:
continue
name = match.group(1)
if name in _SKIP or name.startswith("~"):
continue
# Skip default-argument and initializer noise inside a body.
if stripped.startswith(("return", "if", "for", "while", "}")):
continue
names.add(name)
return names
def base_seams(text: str) -> tuple[set[str], list[str]]:
"""The BacktestEngine members engine.hpp marks `@host-seam`, and misuses."""
start = text.index("class BacktestEngine {")
end = text.index("\n};", start)
first = text.count("\n", 0, start)
last = text.count("\n", 0, end)
lines = text.splitlines()
names: set[str] = set()
errors: list[str] = []
for number, line in enumerate(lines):
if not _SEAM_MARK.match(line):
continue
if not first < number <= last:
errors.append(f"engine.hpp:{number + 1}: a @host-seam marker outside class "
"BacktestEngine")
continue
name = None
for follow in lines[number + 1:last + 1]:
stripped = follow.strip()
if not stripped or stripped.startswith(("//", "/*", "*")):
continue
match = re.search(r"(~?\w+)\s*\(", stripped)
name = match.group(1) if match else None
break
if name is None:
errors.append(f"engine.hpp:{number + 1}: a @host-seam marker is not followed by a "
"member function declaration")
continue
names.add(name)
return names, errors
def coverage_bounds(text: str) -> tuple[int, int]:
try:
start = text.index("COVERAGE")
return start, text.index("HARDENING RULES", start)
except ValueError: # pragma: no cover - the block is required
print("check_native_c_api_surface: no COVERAGE block in native_c_api.h",
file=sys.stderr)
raise SystemExit(2)
def _rows(block: str, seen: set[str]) -> tuple[dict[str, str], dict[str, str]]:
spelled: dict[str, str] = {}
excluded: dict[str, str] = {}
for line in block.splitlines():
match = _ROW.match(line)
if not match:
continue
kind, name, detail = match.groups()
if name in seen:
print(f"check_native_c_api_surface: {name} listed twice", file=sys.stderr)
raise SystemExit(1)
seen.add(name)
(spelled if kind == "C" else excluded)[name] = detail
return spelled, excluded
def coverage_rows(text: str) -> tuple[tuple[dict[str, str], dict[str, str]],
tuple[dict[str, str], dict[str, str]]]:
"""(spelled, excluded) of the host rows, then of the BASE-CLASS SEAMS rows."""
start, end = coverage_bounds(text)
block = text[start:end]
split = block.find(_SEAMS_HEADING)
if split < 0:
print(f"check_native_c_api_surface: no {_SEAMS_HEADING} list in the COVERAGE block",
file=sys.stderr)
raise SystemExit(2)
seen: set[str] = set()
return _rows(block[:split], seen), _rows(block[split:], seen)
def main() -> int:
host_text = HOST.read_text(encoding="utf-8")
c_api_text = C_API.read_text(encoding="utf-8")
public_text = PUBLIC_C.read_text(encoding="utf-8")
engine_text = ENGINE.read_text(encoding="utf-8")
members = host_members(host_text)
seams, marker_errors = base_seams(engine_text)
(spelled, excluded), (seam_spelled, seam_excluded) = coverage_rows(c_api_text)
listed = set(spelled) | set(excluded)
seams_listed = set(seam_spelled) | set(seam_excluded)
# A claimed spelling is resolved against the DECLARATIONS only: the
# COVERAGE block itself is cut out first, so a row can never be the sole
# evidence for the symbol it names.
block_start, block_end = coverage_bounds(c_api_text)
declarations = c_api_text[:block_start] + c_api_text[block_end:] + public_text
failures: list[str] = []
missing = sorted(members - listed)
if missing:
failures.append(
"members with neither a C spelling nor an exclusion: " + ", ".join(missing))
stale = sorted(listed - members)
if stale:
failures.append(
"COVERAGE rows naming members that no longer exist: " + ", ".join(stale))
failures.extend(marker_errors)
missing_seams = sorted(seams - seams_listed)
if missing_seams:
failures.append("marked BacktestEngine seams with neither a C spelling nor an "
"exclusion: " + ", ".join(missing_seams))
stale_seams = sorted(seams_listed - seams)
if stale_seams:
failures.append(f"{_SEAMS_HEADING} rows naming members engine.hpp does not mark "
"@host-seam: " + ", ".join(stale_seams))
enum_failures, twinned, covered = enum_twin_failures((c_api_text, public_text))
failures.extend(enum_failures)
failures.extend(c_v1_exclusion_failures(c_api_text, public_text))
# Every claimed spelling must name something the C headers declare.
for name, detail in sorted({**spelled, **seam_spelled}.items()):
tokens = re.findall(r"[A-Za-z_][A-Za-z_0-9]*", detail)
if not any(token in declarations for token in tokens
if token.startswith(("strategy_", "pf_", "PF_"))):
failures.append(f"{name}: the C spelling '{detail}' names no declared symbol")
if failures:
for failure in failures:
print("check_native_c_api_surface: " + failure, file=sys.stderr)
return 1
print(f"check_native_c_api_surface: {len(members)} NativeStrategyHost members, "
f"{len(spelled)} with a C spelling, {len(excluded)} excluded with a reason; "
f"{len(seams)} marked BacktestEngine seams, {len(seam_spelled)} with a C spelling, "
f"{len(seam_excluded)} excluded with a reason; {twinned + len(C_ONLY)} C enumerations, "
f"{twinned} twinned to a kernel enumeration ({covered} enumerators, each with a C "
f"name), {len(C_ONLY)} C-only with a reason; {len(C_V1_EXCLUSIONS)} C++ capabilities "
"outside the 1.0 C surface, each pinned and in the boundary table")
return 0
if __name__ == "__main__":
raise SystemExit(main())