An executable research program for safe, data-efficient, learning-assisted PID autotuning and adaptive control.
The project is not trying to prove that reinforcement learning belongs in a PID loop. It is trying to discover, under controlled experiments and explicit mathematical assumptions, which forms of intelligence actually improve the Pareto frontier of industrial feedback control.
The governing rule is:
Do not manufacture a breakthrough. Build an apparatus capable of discovering one, and equally capable of proving that an exciting idea is unnecessary.
Learning, Bayesian optimization, active experiment design, robust control, direct data-driven tuning, disturbance observers, set-membership methods, classical autotuning, and adaptive control are treated as competing mechanisms inside one common scientific framework.
The long-term target is an autonomous control system that can answer, online:
- What changed? Plant dynamics, disturbance, operating regime, sensor/actuator health, or merely the reference?
- What do the available data actually prove?
- Is the current PID structure adequate for the requested behavior?
- Would changing controller parameters help?
- If evidence is insufficient, what is the lowest-risk experiment that resolves the decision?
- Which controller is Pareto-optimal for the current preference and uncertainty set?
- Can the proposed change be certified or robustly bounded before execution?
- When should the system stop learning and leave a good controller alone?
The desired controller is therefore not simply
RL -> Kp, Ki, Kd
and certainly not
RL -> actuator
The current architecture hypothesis is closer to an uncertainty-aware autonomous control scientist whose deterministic executable primitive is PID.
measured history
|
+--------------+---------------+
| |
v v
working/context model bounded long-term memory
RLS / set membership episodic + semantic regimes
FRF / neural research lane counterexample coreset
| |
+--------------+---------------+
|
v
+----------------------------+
| causal / regime inference |
| disturbance? dynamics? |
| fault? operating point? |
+-------------+--------------+
|
+----------------+----------------+
| |
v v
uncertainty / model set disturbance lane
set membership / Bayesian DOB / ESO / residual
frequency response / direct data compensation
|
v
+--------------------------------+
| proof + decision informativity |
| SAFE / UNSAFE / UNKNOWN |
| memory transfer calibrated? |
+---------------+----------------+
|
unknown | -> active experiment design
| relay / multisine / pulse /
| natural data / NOOP
v
candidate-controller experts
+-----------------------------------------+
| robust/model-based PID |
| SIMC / relay / VRFT / IFT |
| set-membership adaptation |
| extremum seeking / safe BO / MOBO |
| MMAC / unfalsified control |
| memory / learned proposal priors |
+--------------------+--------------------+
|
v
Pareto frontier/archive
|
v
deterministic governor
proof obligations / constraints
|
v
industrial PID loop
The learner is allowed to propose. It does not automatically receive authority.
The inner loop must continue running correctly if every learning component disappears.
Current/future responsibilities:
- P / PI / PID / 2-DOF PID;
- derivative filtering;
- anti-windup;
- output and slew constraints;
- bumpless parameter/mode transfer;
- deterministic timing and fallback;
- no neural dependency in the hard real-time path.
- update uncertainty/model representation;
- detect drift, disturbance, regime change, and faults;
- evaluate controller candidates;
- maintain a nondominated controller archive;
- project/reject unsafe gain changes;
- roll back degradation.
- decide whether more information is valuable;
- select an excitation family, amplitude, and bandwidth;
- learn reusable plant/regime structure;
- compare competing tuner families;
- update offline/meta-learning priors;
- distill repeated experience into bounded semantic/controller sufficient statistics;
- retain a bounded severity/diversity counterexample coreset;
- maintain compressed replay for slow parametric-model consolidation;
- reject continual-learning updates that regress old validated regimes.
The project does not assume that a point-estimated model is the truth.
Given data (\mathcal D), define the compatible-system set
{G: G\text{ remains compatible with the observations and uncertainty assumptions}}. ]
For a proposed controller (C_\theta) and property (P), the important question is whether every compatible plant agrees:
[ P(G,C_\theta),\qquad \forall G\in\Sigma(\mathcal D). ]
The decision gate therefore returns three states:
PROVED / INFORMATIVE-SAFE
REJECTED / INFORMATIVE-UNSAFE
NON-INFORMATIVE
NON-INFORMATIVE is not failure. It means the controller knows what it does not know and can either remain conservative or acquire targeted information.
The project is intentionally moving from
identify plant -> tune controller
toward
data -> compatible dynamics -> pending decision -> acquire only useful information
One central research thesis is:
Knowledge buys control authority.
If the compatible uncertainty set contracts,
[ \Sigma_{t+1}\subseteq\Sigma_t, ]
then the set of controllers that can be certified across every compatible model may expand.
The executable scalar set-membership experiment already demonstrates this exactly:
initial feasible parameter-set area: 0.32
initial common stabilizing K interval: EMPTY
after informative data:
feasible set contracts by orders of magnitude
common stabilizing interval emerges
later robust interval: approximately (-0.323, 6.296)
This is not a statistical confidence metaphor; for that bounded-noise fixture it is a direct set-theoretic result.
These are regression-tested research observations, not universal claims.
Two positive-gain PI controllers were constructed such that each frozen closed loop is Hurwitz stable, yet periodic switching produces a monodromy matrix with
[ \rho(\Phi)\approx3.32059>1. ]
The state therefore grows exponentially under that switching law.
Furthermore, sweeping scheduling time produced multiple alternating stable and unstable windows rather than one monotone update-rate threshold.
Consequence: online gain adaptation is a switched/hybrid-control problem. Gain clipping alone is not a scheduler stability proof.
The project contains two plants with identical frequency-response magnitude at every frequency and nearly identical low-frequency phase, yet an aggressive PI controller stabilizes one and destabilizes the other.
Consequence: identification quality must be judged relative to the pending control decision/bandwidth, not merely prediction MSE.
For a tractable PID-controlled plant family, two robustly stable PID tunings have an unstable arithmetic midpoint.
For that specific family, a plant-informed logarithmic coordinate transformation converts the Routh-Hurwitz stability condition into a half-space; the transformed interpolation remained stable across all 1001 sampled points.
Consequence: the geometry in which a learner moves matters. Controller coordinates should be informed by certificate geometry where possible.
Using identical PI gains and identical steady-state initial conditions on a temporary load-disturbance fixture:
plain PI pulse IAE = 0.67856
PI + ESO pulse IAE = 0.08332
plain PI recovery IAE = 0.23682
PI + ESO recovery IAE = 0.07222
Consequence: sustained error is not sufficient evidence that PID gains should change. The supervisor needs causal routing between disturbance compensation and plant retuning.
The first minimax set-membership experiment planner reduced a compatible parameter-set area from
[ 0.32\rightarrow7.55\times10^{-5} ]
while resolving a previously non-informative feedback-gain decision. Once the decision was resolved, the planner ranked zero excitation as best because further probing had no decision value and only added process cost.
Consequence: NOOP must be a legitimate experiment action.
The relay lane now performs coherent Fourier extraction over an integer number of observed limit cycles. On the analytic FOPDT fixture
[ G(s)=\frac{e^{-0.5s}}{s+1}, ]
the refined critical-point errors are roughly
Ku error: +1.69%
Tu error: -1.79%
and the same relay trajectory recovers complex FRF points at the first, third, and fifth harmonics with low-percent or sub-percent error in the canonical fixture.
Consequence: relay feedback is better treated as a frequency-domain evidence primitive than as a Ziegler-Nichols oracle.
On the first dense PI FOPDT scout, a deterministic scrambled-Sobol search evaluated 256 gain candidates plus SIMC.
evaluated candidates: 257
feasible candidates: 237
nondominated: 53
SIMC on frontier: yes
The lowest sampled IAE improved only slightly over SIMC but paid materially in actuator total variation, sensitivity peak, and overshoot.
Consequence: classical methods remain serious competitors. A learned tuner must beat strong baselines rather than weak hand-picked gains.
The nonlinear physical-process gym now includes a gravity tank, heated tank, DC motor, and exothermic CSTR. Before tuning, the engine can numerically map safe steady-state equilibria over the declared actuator range. If a requested operating point lies outside that envelope, the correct result is specification/authority infeasible, not a search for more aggressive gains.
The gym also found that commanded saturation and delivered actuator authority must be tracked separately: hardware derating can leave the PID pinned at its command limit while the actual actuator delivers only a fraction of that request.
Consequence: an intelligent autotuner must distinguish bad gains, poor robustness, hardware/authority loss, and an impossible control objective.
On the current physical candidate banks, many controllers are legitimately nondominated. Counterexample-guided exact robust-Pareto certification therefore required the complete controller/case matrix on several process families. The engine now also implements finite-pool epsilon-Pareto coverage, where application-declared tolerances can certify a practical controller core without resolving every microscopic tradeoff.
Consequence: exact Pareto membership is not automatically the right decision object; practical epsilon/regret specifications can be both more meaningful and more data efficient.
Persistent memory now compresses repeated experience into bounded regime sufficient statistics, controller coresets, episodic exemplars, and counterexamples. A 6,000-observation stress stream remained at the configured capacity of 32 semantic prototypes, 768 controller aggregates, 96 episodes, and at most 32 counterexamples. Lossless gzip reduced the current persisted snapshot from roughly 727 KB pretty JSON to 87.6 KB; compressed neural replay used about 3.7x less in-memory storage than the retained float64 trajectory equivalent.
More importantly, memory is not trusted because a context merely looks familiar. Leave-one-regime-out CSTR experiments initially produced catastrophic false transfer. The current system splits control-relevant representation collisions, asks whether retrieved regimes agree on the controller decision, and calibrates that transfer against stored leave-one-regime-out errors. When that evidence is insufficient it keeps the full search space rather than pruning.
The continual-memory program distinguishes recurrence from transfer. A regime that has been directly verified before can earn stronger proposal priority than a merely similar unseen regime, but its remembered controller is still re-verified before use.
In the first cross-process (A\rightarrow B\rightarrow C\rightarrow A) campaign under a common teacher safety shield, physical episodes to a 5%-regret controller changed as:
family first encounter return
gravity tank 16 4
heated tank 10 10
DC motor 10 4
CSTR 4 4
The median fell from 10 to 4 episodes on this four-family fixture. This is promising but not universal, and the teacher shield uses declared simulator knowledge; it is an upper bound on what a deployable safety governor might permit, not a real-plant certificate.
A causal Transformer and GRU class model were trained across heterogeneous normalized first-order, second-order, and nonlinear synthetic systems. Online ARX/RLS remains much stronger on the present benchmark:
IID NRMSE parameter-OOD NRMSE
RLS / ARX 0.093 0.171
GRU class model 0.859 0.785
Transformer 0.973 0.996
A retrieval-initialized RLS variant also currently underperforms cold RLS. These are negative results, not failures of the research program: neural complexity has not earned control authority yet.
The physical gym now contains an evidence-directed supervisor that keeps compatible explanations separate from actions. HAC/Newey-West covariance corrected severe overconfidence in closed-loop ARX drift statistics, and the router distinguishes actuator delivery loss, sensor/disturbance ambiguity, actual parameter drift, and infeasible specifications rather than treating all tracking degradation as a gain-tuning request.
An incumbent-centered PI stability shield is also being validated. It writes a candidate as (C_1=C_0(1+W)) and uses the exact factorization (1+PC_1=(1+PC_0)(1+T_0W)). Given a validated complete-frequency envelope for the known-good loop's complementary sensitivity, (|T_0W|_\infty<1) provides a conservative small-gain certificate for physical candidate verification.
The newest slow-loop layer maintains causal ambiguity sets instead of reflexively retuning whenever error rises. For the simple measured first-order process
[ \dot y=-ay+bu+d,\qquad y_m=y+\beta, ]
a constant sensor bias is exactly equivalent in measured coordinates to shifting the additive disturbance by (a\beta). No neural network can infer information that is not in the available channels. The supervisor therefore requests discriminating evidence when sensor bias, load disturbance, and/or dynamics change remain observationally compatible.
Physical DC-motor stress testing also exposed severe overconfidence from IID least-squares covariance in closed-loop ARX identification. A Newey--West/HAC covariance inflated nominal uncertainty by roughly 90x on the current fixture and largely removed false "dynamics changed" evidence for actuator loss and sensor bias while retaining strong evidence for the deliberately changed motor parameters. Statistical residual whiteness, predictive adequacy, and regressor conditioning are now tracked as separate questions.
When several causal/proof hypotheses survive, the engine can plan a short-horizon diagnostic decision tree rather than always injecting another plant perturbation. Candidate actions can include redundant-sensor checks, actuator feedback, bounded process probes, or natural excitation. The finite planner minimizes worst-case remaining ambiguity first and then evidence cost/risk; a regression fixture proves that the best one-step split can be inferior over a two-step plan.
There is no universally optimal PID.
The research engine preserves objective vectors such as
[ \mathbf J(\theta)= [IAE,\ ITAE,\ TV(u),\ E_u,\ M_s,\ overshoot,\ settling,\ldots]. ]
Hard constraints are evaluated before preference tradeoffs.
hard feasibility / safety
-> objective vector
-> nondominated archive
-> preference selection only after frontier discovery
Future optimization algorithms will be compared by more than final reward:
- hypervolume versus number of plant experiments;
- Pareto regret;
- unsafe/failed evaluations;
- controller robustness;
- total excitation energy/process cost;
- wall-clock and compute cost;
- OOD degradation;
- reproducibility across seeds/plants.
The supervisor additionally reports diagnosis/evidence metrics: false retuning events, time-to-cause-resolution, evidence/probe cost, recurrence speedup, transfer abstention, and the fraction of proposed controller experiments rejected by the deterministic shield.
This makes it substantially harder to claim progress by hiding a poor tradeoff behind arbitrary reward weights.
src/adaptive_pid_lab/ currently contains:
| Module | Role |
|---|---|
pid.py |
deterministic PID core and controller state |
simulation.py |
deterministic LTI simulation, RK4 and exact-ZOH paths, delays/disturbances |
metrics.py |
tracking/control objective metrics |
benchmarks.py |
canonical adversarial plant bank |
tuners.py |
classical tuning baselines such as SIMC |
relay.py |
relay feedback, stationarity diagnostics, coherent multi-harmonic FRF extraction |
control_relevance.py |
control-relevant model-gap / robustness calculations |
set_membership.py |
bounded-noise compatible parameter sets and robust gain intervals |
informativity.py |
three-way decision-informativity logic |
experiment_design.py |
minimax decision-aware experiment selection |
observers.py |
disturbance-estimation / compensation research lane |
pareto.py |
hard-feasibility-aware nondominance and Pareto utilities |
decision_engine.py |
explicit proof-obligation / decision-resolution routing |
delay_stability.py |
exact FOPDT PI and lower-evidence general delay stability screens |
robustness.py |
sensitivity-peak/global robustness calculations |
sampled_data.py |
exact implemented digital-PI / sampling / jitter analysis |
safety_envelope.py |
state/actuator-authority viability primitives |
physical_processes.py |
first-principles nonlinear tank, thermal, motor, and CSTR models |
real_world_gym.py |
nonlinear physical scenario simulation and robust PID search |
control_memory.py |
bounded semantic/episodic/counterexample continual control memory |
neural_context.py |
optional causal neural class model and compressed replay research lane |
context_tournament.py |
RLS/retrieval/GRU/Transformer identification tournament |
adaptive_search.py |
finite-controller random/ranked/GP-BO sample-efficiency baselines |
evidence_supervisor.py |
causal slow-loop routing over compatible fault/disturbance/dynamics hypotheses |
diagnostics.py |
HAC-aware identification, innovation, residual-dependence, and model-adequacy diagnostics |
diagnostic_planning.py |
finite-horizon minimax planning of evidence-acquisition actions |
incumbent_stability.py |
incumbent-centered PI small-gain stability shield from validated closed-loop frequency envelopes |
certified_paths.py |
Pareto path planning over verified controller-transition graphs |
certified_homotopy.py |
certificate-governed log-gain PI waypoints toward remote controller targets |
closed_loop_probe.py |
bounded closed-loop FRF probing and pointwise complementary-sensitivity uncertainty disks |
adaptation_transaction.py |
transactional controller deployment, verification, commit, and rollback semantics |
certificate_portfolio.py |
composition and selection of heterogeneous controller-safety certificates |
drift_detection.py |
task-conditioned sequential innovation drift detection |
evidence_store.py |
bounded content-addressed storage for reusable control evidence |
lifetime_evaluation.py |
never-ending deployment and recurrence evaluation primitives |
memory_allocation.py |
bounded-memory budget allocation across control-memory channels |
memory_rate_distortion.py |
control-relevant memory compression and rate-distortion evaluation |
process_interventions.py |
explicit process intervention and evidence-acquisition semantics |
representation_bridge.py |
versioned context-representation compatibility and migration research |
similarity_metric.py |
control-relevant retrieval geometry and similarity diagnostics |
unsafe_cover.py |
bounded geometric cover for persistent unsafe-controller witnesses |
The benchmark bank deliberately includes friendly and adversarial structures:
- moderate-delay FOPDT;
- delay-dominated FOPDT;
- overdamped SOPDT;
- integrating dynamics;
- open-loop unstable dynamics;
- lightly damped resonance;
- nonminimum-phase/RHP-zero dynamics;
- hidden high-order poles;
- resonance plus delay.
Structural holdouts are treated differently from ordinary parameter variation.
Requirements: Python 3.12+.
python -m pip install -e .[dev]
python -m pytestRepresentative experiments:
python experiments/scheduler_instability.py
python experiments/scheduler_rate_scan.py
python experiments/identification_ambiguity.py
python experiments/pid_safe_region_geometry.py
python experiments/set_membership_informativity.py
python experiments/active_set_membership_design.py
python experiments/disturbance_vs_retuning.py
python experiments/relay_fopdt_identification.py
python experiments/relay_adversarial_scout.py
python experiments/pareto_pi_fopdt.py
python experiments/stress_gymnasium.py
python experiments/real_world_process_gymnasium.py
python experiments/memory_compression_stress.py
python experiments/continual_memory_holdout.py
python experiments/context_model_tournament.py
python experiments/causal_supervisor_physical_gym.py
python experiments/lifelong_recurrence_all_processes.py
python experiments/incumbent_pi_shield_fopdt.py
python experiments/memory_bo_physical_holdout.py
python experiments/lifelong_regime_recurrence.py
python experiments/incumbent_digital_pi_shield.py
python experiments/incumbent_digital_global_norm_shield.py
python experiments/incumbent_pi_certified_homotopy.py
python experiments/closed_loop_multisine_probe.pyThe suite is intentionally growing with every falsified assumption. Run the complete
pytest command above rather than relying on a stale test-count claim in documentation.
Calling an existing method “agentic,” “AI-native,” or “autonomous” is not a contribution.
Combining PID + RL + BO + CBF is not automatically novel. A paper-worthy mechanism must demonstrate a non-obvious principle, algorithm, theorem, or empirical result that survives strong ablation.
Every learned method must face appropriate competitors, including where relevant:
- SIMC / robust fixed PID;
- relay/frequency-response methods;
- VRFT / IFT;
- extremum seeking;
- safe/constrained BO;
- set-membership methods;
- DOB / ADRC;
- MMAC / unfalsified control;
- robust MPC / SLS teacher bounds;
- simple contextual routing.
Tail risk, worst-case behavior, structural OOD cases, constraint violations, and failure recovery are mandatory outputs.
Every probabilistic guarantee must state its calibration distribution, horizon, dependence assumptions, and behavior under detected shift.
No hidden oracle channels
Generic PID-facing features may use only signals actually available to the deployed loop unless an optional observer/redundant sensor/actuator-feedback channel is explicitly declared. Simulator-only true-state/output information may be used for scoring, not for retrieval or causal diagnosis.
Plant evaluations, simulator evaluations, training transitions, wall time, and compute should be reported separately.
The project will not predeclare one. Candidate breakthrough classes include:
- A new decision-directed experiment criterion that materially reduces the number or cost of physical experiments required to certify/improve a controller compared with identification-first, Fisher-information, safe-BO, and set-membership baselines.
- A new controller-space representation or projection geometry that makes meaningful PID robustness/scheduler constraints substantially easier to optimize while retaining useful controller expressivity.
- A provably safe Pareto-learning mechanism whose certified admissible region grows with evidence and whose sample efficiency dominates existing safe controller-tuning approaches.
- A causal routing architecture that demonstrably distinguishes disturbance, plant drift, and structural inadequacy well enough to beat universal gain adaptation across a broad benchmark family.
- A meta-learned proposal prior that earns its complexity by reducing online experiment count while reliably surrendering authority on structural OOD plants.
- A negative result of sufficient generality showing that a fashionable learning architecture is fundamentally dominated by a simpler robust/adaptive method under realistic control constraints.
Any claimed breakthrough must survive literature red-teaming, independent baselines, ablation, adversarial fixtures, and full-budget confirmation.
The immediate program is:
- Sample-efficiency tournament — Sobol/random versus model-based search versus BO/MOBO versus later RL/meta-RL, measured by Pareto hypervolume/regret per plant experiment.
- Safe frontier expansion — combine deterministic/model-based admissible controller regions with black-box multiobjective optimization instead of asking a GP to learn safety from scratch.
- Decision-directed experiment design — optimize experiments for pending controller/Pareto decisions rather than generic model accuracy alone.
- Multi-step active design / dual control — move beyond one-step set contraction while preserving explicit process-risk accounting.
- Control-relevant uncertainty geometry — connect compatible model sets to (\nu)-gap, robustness margins, and controller-class adequacy.
- Structural inadequacy detection — explicitly return “PID cannot satisfy this specification” rather than generating pathological gains.
- Scheduler certification — quantify when a sequence/path of individually valid controllers is safe to traverse.
- Causal adaptation tournament — compare evidence-directed routing against always-retune, disturbance-observer-first, and learned routing policies on nonstationary physical sequences where superficially similar error has different causes.
- Certified recurrence and safe transfer — replace the current teacher safety shield with compatible-model/set-membership viability certificates and measure whether lifelong memory still reduces adaptation episodes.
- Active proof-obligation planning — jointly choose between process excitation, passive waiting, redundant sensing, and controller verification according to which unresolved fact blocks useful authority.
The project should become more sophisticated only when the evidence requires it.
docs/
architecture/ runtime and simulation architecture
experiments/ campaign plans and falsification reports
math/ guarantee and informativity machinery
red-team/ assumptions, failure modes, kill criteria
research/ literature synthesis and active hypotheses
experiments/ executable scientific probes
src/ reusable research engine
tests/ regression and mathematical property tests
references/ curated source ledger
High-value entry points:
docs/research/frontier-map.mddocs/research/information-centric-control.mddocs/research/active-experiment-design.mddocs/research/pareto-tuning.mddocs/research/alternative-architectures.mddocs/research/research-hypotheses.mddocs/math/decision-informativity-gate.mddocs/math/guarantee-stack.mddocs/red-team/adversarial-audit.mddocs/experiments/research-program.mdreferences/source-ledger.md
The intended end state is not “the smartest-looking controller.”
It is a controller that can explain, with evidence:
what it knows, what it does not know, what action is worth trying next, which tradeoffs are unavoidable, and why the action it executes is allowed.
If the best final system turns out to contain less machine learning than expected, that is a successful research outcome.
This repository is released under the MIT License.