For AI agents: This document provides a deep-dive analysis of Eagle's
[sql]command internals, including ADO.NET integration, the-variableoptions for automatic resource cleanup via DbTraceCallback, the script bundle database system, and the query execution pipeline. For basic command syntax, seecore_language.md. For usage examples, seecore_examples.md. For database utility procedures, seecore_script_library.md.
Eagle's [sql] command provides complete ADO.NET database access
from script code — connecting to databases, executing queries with
parameterized inputs, iterating over result sets, managing transactions,
and profiling query performance. It works with any .NET data provider
(SQLite, SQL Server, Oracle, ODBC, OLE DB) through the standard
IDbConnection / IDbCommand / IDbTransaction interfaces.
The command has two capabilities that go well beyond basic database access:
-
Automatic resource cleanup via
-variableand DbTraceCallback — When connections or transactions are stored in variables using the-variableoption, Eagle attaches a variable trace callback that automatically closes connections and commits/rolls back transactions when the variable goes out of scope. This prevents resource leaks even in the presence of exceptions or early returns. -
Script bundle databases — Eagle can mount SQLite databases containing digitally signed Eagle scripts as virtual file systems. Each script in the bundle is individually signed with an RSA key pair, and the bundle database follows a specific schema with security metadata, evaluation ordering, isolation levels, and rule-set filtering. This enables secure distribution of script packages as single encrypted, signed database files.
The command is not part of standard Tcl — it has no Tcl equivalent.
It carries CommandFlags.Unsafe | CommandFlags.NonStandard and is
unavailable in safe interpreters.
Key source files:
| File | Lines | Role |
|---|---|---|
Eagle/Library/Commands/Sql.cs |
1,802 | Main command implementation (9 sub-commands) |
Eagle/Library/Components/Private/DataOps.cs |
4,214 | Database operations: connection creation, query execution, result formatting, bundle scripts |
Eagle/Library/Components/Private/ObjectOps.cs |
large | SQL execute option definitions and processing |
Eagle/Library/Components/Public/Interpreter.cs |
124,855 | Connection/transaction storage, DbTraceCallback, SetDbVariableValue |
Eagle/Library/Components/Public/DatabaseVariable.cs |
~1,700 | Database-backed variable infrastructure |
Eagle/Library/Components/Private/BundleData.cs |
264 | Script bundle data representation |
Eagle/Library/Components/Private/BundleManager.cs |
394 | Bundle mounting, unmounting, and data retrieval |
Eagle/Library/Components/Private/Vars.cs |
~500 | Result set variable name constants |
Scripting languages frequently need to interact with databases. In the
Tcl world, database access requires loading extension packages (e.g.,
tdbc, sqlite3, mysqltcl) that provide Tcl commands for database
operations. These extensions are platform-specific native libraries
that must be compiled, distributed, and loaded separately.
Eagle operates in the .NET/CLR environment where ADO.NET provides a
rich, provider-independent database abstraction. The [sql] command
exposes this abstraction directly to script code, eliminating the need
for external database extensions.
The command follows ADO.NET's layered architecture:
Script Layer: [sql] command → sub-commands
↓
Provider Abstraction: IDbConnection → IDbCommand → IDataReader
↓
Concrete Providers: SQLiteConnection, SqlConnection, OdbcConnection, ...Each database operation maps directly to ADO.NET concepts:
| Script operation | ADO.NET equivalent |
|---|---|
[sql open] |
new XxxConnection(connStr); connection.Open() |
[sql execute] |
connection.CreateCommand(); command.ExecuteReader() |
[sql foreach] |
while (reader.Read()) { ... } |
sql transaction begin |
connection.BeginTransaction(isolation) |
sql transaction commit |
transaction.Commit() |
[sql close] |
connection.Close() |
Native Tcl has no built-in database command. Eagle's [sql] is entirely
Eagle-specific (CommandFlags.NonStandard). The closest Tcl comparison
would be package require tdbc plus tdbc::connection create, but
Eagle's approach is more deeply integrated with the runtime.
The [sql] command supports 9 sub-commands:
| Sub-command | Purpose |
|---|---|
[open] |
Create and open a database connection |
[close] |
Close and release a database connection |
isopen |
Check if a connection exists and is open |
connection |
Query connection metadata |
execute |
Execute SQL and return results |
[foreach] |
Execute SQL and iterate over each row |
transaction |
Manage transactions (begin/commit/rollback) |
hasbegun |
Check if a transaction is active |
types |
List available database provider types |
sql open ?options? connectionStringOpens a new database connection using the specified connection string and returns a connection handle (name) for use with other sub-commands.
Options:
| Option | Type | Description |
|---|---|---|
-type <DbConnectionType> |
enum | Primary database type (SQLite, Sql, Odbc, OleDb, Oracle, SqlCe) |
-type1 <DbConnectionType> |
enum | Override primary type |
-type2 <DbConnectionType> |
enum | Fallback type (tried if type1 fails) |
-variable <varName> |
string | Store connection name in variable with DbTraceCallback for auto-cleanup |
-assemblyfilename <path> |
string | Path to the provider assembly to load |
-typename <name> |
string | Short type name of the connection class |
-typefullname <name> |
string | Full (namespace-qualified) type name |
-valueflags <ValueFlags> |
enum | Type resolution flags |
-trustedonly |
flag | Only accept assemblies with trusted signatures |
-maybetrustedonly |
flag | Like -trustedonly but allowed in safe interpreters |
-publickeytoken1 <hex> |
string | Public key token for primary type assembly |
-publickeytoken2 <hex> |
string | Public key token for fallback type assembly |
-nocase |
flag | Case-insensitive type name matching |
-stricttype |
flag | Prevent automatic type name resolution |
-verbose |
flag | Return full type resolution error information |
SQLite auto-fallback:
When -type SQLite is specified without an explicit -type2, the
command automatically tries SQLiteEnterprise first, then falls back
to standard SQLite. This transparent upgrade behavior ensures that
the enterprise edition (with encryption support, etc.) is used when
available.
Provider verification:
The interpreter can enforce DataFlags.TrustedOnly and
DataFlags.VerifiedOnly globally, requiring all database provider
assemblies to pass trust and public key token checks. The default
public key tokens for SQLite Enterprise and standard SQLite are used
when -publickeytoken1 and -publickeytoken2 are not explicitly
specified but verification is required.
sql close connectionCloses the database connection and removes it from the interpreter's
connection registry. Fires NotifyType.Connection /
NotifyFlags.Removed notification.
sql isopen connectionReturns 1 if the named connection exists in the interpreter's
registry, 0 otherwise.
sql connection connectionReturns a list of key-value pairs describing the connection:
type SQLiteConnection state Open database main timeout 15 string "Data Source=test.db"Fields: type (provider class name), state (ConnectionState),
database (current database), timeout (connection timeout),
[string] (connection string).
sql execute ?options? connection string ?{paramName ?paramType? ?paramValue? ?paramSize? ?paramValueFlags?} ...?Executes a SQL statement and returns results. The execution type and result format are controlled by options.
Execution pipeline:
- Parse options and validate connection
- Attach
-changedcallback (if specified) - Create
IDbCommandfrom connection - Set command text, timeout, type, and transaction
- Bind parameters via
DataOps.GetParameters() - Call
command.Prepare() - Execute via
DataOps.ExecuteCommandAndGetResults() - Store timing data (if
-timeis enabled) - Dispose command
sql foreach ?options? connection string ?{params} ...? bodyLike execute, but evaluates body for each result row. The current
row data is available in the row variable (default: row, an array
keyed by 1-based row number). Supports [break], [continue], and [return] from
the body script.
The key difference from execute: the default rows variable name is
Vars.ResultSet.Row (singular) instead of Vars.ResultSet.Rows
(plural), reflecting the one-row-at-a-time processing model.
sql transaction ?options? action connection/transactionActions:
| Action | Syntax | Description |
|---|---|---|
begin |
sql transaction ?options? begin connection |
Start a new transaction; returns transaction handle |
commit |
sql transaction ?options? commit transaction |
Commit the transaction |
rollback |
sql transaction ?options? rollback transaction |
Roll back the transaction |
Options:
| Option | Type | Description |
|---|---|---|
-isolation <IsolationLevel> |
enum | Isolation level: Unspecified (default), ReadUncommitted, ReadCommitted, RepeatableRead, Serializable |
-variable <varName> |
string | Store transaction name in variable with DbTraceCallback for auto-cleanup |
sql hasbegun transaction ?connection?- With one argument: returns
1if the transaction exists,0otherwise - With two arguments: returns
1only if the transaction exists AND belongs to the specified connection (checksObject.ReferenceEquals(transaction.Connection, connection))
sql types ?pattern?Returns a list of available database connection type names, optionally filtered by a glob pattern. The list combines:
- Primary (built-in) connection types:
Odbc,OleDb,Sql - Trusted "other" connection types (e.g.,
SQLiteEnterprise) - Public/verified connection types (e.g.,
SQLite,Oracle,SqlCe)
This is one of Eagle's most distinctive database features: automatic resource cleanup tied to variable lifetime.
Traditional database code requires explicit cleanup:
set conn [sql open -type SQLite "Data Source=test.db"]
# If an error occurs here, the connection leaks
sql execute $conn "CREATE TABLE t1(x)"
sql close $connIf an error occurs between [open] and [close], the connection leaks.
While [try]/finally can help, it's error-prone and verbose.
sql open -variable conn -type SQLite "Data Source=test.db"
# $conn now holds the connection name
# When $conn is unset (scope exit, explicit unset, etc.),
# the connection is automatically closed
sql execute $conn "CREATE TABLE t1(x)"
# No explicit close needed — cleanup happens automaticallyWhen -variable is used with [sql open] or sql transaction begin,
the command calls interpreter.SetDbVariableValue() instead of simply
returning the handle:
// In sql open (Sql.cs, line 1348):
code = interpreter.SetDbVariableValue(varName, connectionName, ref result);
// In sql transaction begin (Sql.cs, line 1502):
code = interpreter.SetDbVariableValue(varName, transactionName, ref result);SetDbVariableValue sets the variable's value AND attaches the
dbTraceList — a TraceList containing the DbTraceCallback — as a
variable trace:
// Interpreter.cs, line 30491:
internal ReturnCode SetDbVariableValue(
string name, string value, ref Result error)
{
lock (syncRoot)
{
return SetVariableValue(
VariableFlags.None, name, value, dbTraceList, ref error);
}
}The dbTraceList is initialized during interpreter construction:
// Interpreter.cs, line 81420:
dbTraceList = new TraceList(
clientData, TraceFlags.None, plugin,
new TraceCallback[] { DbTraceCallback });The DbTraceCallback is a static method that fires when a
traced variable is unset:
[MethodFlags(MethodFlags.VariableTrace | MethodFlags.System | MethodFlags.NoAdd)]
private static ReturnCode DbTraceCallback(
BreakpointType breakpointType,
Interpreter interpreter,
ITraceInfo traceInfo,
ref Result result)Trigger condition: BreakpointType.BeforeVariableUnset only.
Cleanup flow:
- Gather old values — Extract the variable's current value(s), which contain connection/transaction names
- Collect transactions — Call
GetDbTransactionsForTrace(names, ref transactions)to find matching active transactions - Commit or roll back — For each transaction, attempt
commit or rollback based on the variable's
VariableFlags.Successflag:bool commit = FlagOps.HasFlags( traceInfo.Flags, VariableFlags.Success, true); CommitOrRollback(transaction, ref commit, ref errors);
- Fail-safe retry — If the initial operation fails,
CommitOrRollbackretries with the opposite operation (commit failure triggers rollback attempt) - Remove transaction — On success, calls
interpreter.RemoveDbTransaction(name)to unregister it - Collect connections — Call
GetDbConnectionsForTrace(names, ref connections)to find matching connections - Close connections — For each connection, calls
connection.Close()andinterpreter.RemoveDbConnection(name) - Report errors — Errors are reported via
DebugOps.Complain()but the callback always returnsReturnCode.Ok(cleanup errors do not block variable unset)
This method implements the fail-safe transaction finalization:
private static bool CommitOrRollback(
IDbTransaction transaction, ref bool commit, ref ResultList errors)
{
try
{
if (commit)
transaction.Commit();
else
transaction.Rollback();
return true;
}
catch (Exception e)
{
if (commit)
commit = false; // Force rollback on commit failure
if (errors == null)
errors = new ResultList();
errors.Add(e);
return false;
}
}If commit fails, the commit flag is set to false, and the caller
retries — this time performing a rollback. This ensures transactions
are always finalized, even if the database connection is in a bad state.
When -variable is used, the ordering of operations differs from the
non-variable path:
With -variable (sql open):
- Generate connection name
- Set variable (attaches DbTraceCallback) ← trace is live before Open
connection.Open()
Without -variable (sql open):
connection.Open()← connection opens first- Generate connection name
This ordering ensures the trace is in place before the connection
opens, so if Open() throws, the cleanup still fires when the
variable goes out of scope (though there may be nothing to clean up).
With -variable (sql transaction begin):
- Generate transaction name
- Set variable (attaches DbTraceCallback) ← trace is live before Begin
connection.BeginTransaction(isolationLevel)
Without -variable (sql transaction begin):
connection.BeginTransaction(isolationLevel)← transaction starts first- Generate transaction name
sql open -variable conn "connStr"
│
├── SetDbVariableValue("conn", connectionName)
│ ├── Sets $conn = connectionName
│ └── Attaches DbTraceCallback as BeforeVariableUnset trace
│
├── connection.Open()
└── AddDbConnection(connectionName, connection)
... script uses $conn ...
$conn goes out of scope (proc returns, unset, etc.)
│
├── DbTraceCallback fires (BeforeVariableUnset)
│ ├── GetDbTransactionsForTrace() → (none found)
│ └── GetDbConnectionsForTrace() → finds connection
│ ├── connection.Close()
│ └── RemoveDbConnection(name)
└── Variable is unsetThe execute and [foreach] sub-commands share a large set of options
defined by ObjectOps.GetSqlExecuteOptions().
Controls how the SQL statement is executed:
| Value | ADO.NET method | Returns |
|---|---|---|
NonQuery |
command.ExecuteNonQuery() |
Number of rows affected |
Scalar |
command.ExecuteScalar() |
Single value from first row, first column |
Reader |
command.ExecuteReader() |
Full result set |
ReaderAndCount |
command.ExecuteReader() + count |
Result set plus row count |
Controls the structure of returned results:
| Value | Description |
|---|---|
None |
No result formatting |
Invalid |
Invalid format (error signaling) |
Reserved |
Reserved for future use |
RawArray |
Raw array format |
RawList |
Raw list format |
Array |
Tcl array variable (column names as keys) |
List |
Flat list of values |
Dictionary |
Tcl dictionary |
NestedList |
List of lists (one per row) |
NestedDictionary |
Nested dictionary (one dictionary per row) |
DataRecord |
IDataRecord-based access |
DataReader |
IDataReader-based streaming access |
DataTable |
Materialized DataTable with conversion methods |
| Option | Type | Description |
|---|---|---|
-allownull |
bool | Allow null values in results |
-nullvalue <string> |
string | String representation for null values |
-dbnullvalue <string> |
string | String representation for DBNull.Value |
-errorvalue <string> |
string | String representation for error values |
-datetimebehavior <enum> |
DateTimeBehavior | How to handle DateTime values (Ticks, ToString, etc.) |
-datetimeformat <string> |
string | Format string for DateTime conversion |
-datetimekind <enum> |
DateTimeKind | DateTimeKind for parsed DateTime values (Local, Utc, Unspecified) |
-datetimestyles <enum> |
DateTimeStyles | Styles for DateTime parsing |
-blobbehavior <enum> |
BlobBehavior | How to handle BLOB/binary data |
-numberformat <string> |
string | Format string for numeric values |
-valueformat <string> |
string | General value format string |
-valueflags <enum> |
ValueFlags | Value conversion behavior flags |
-verbatim |
flag | Skip value conversion; use raw values |
-culture <CultureInfo> |
CultureInfo | Culture for parsing and formatting |
Null value fallback chain: If -dbnullvalue is not set, it defaults
to -nullvalue. If -errorvalue is not set, it also defaults to
-nullvalue. This ensures consistent null representation across all
value types.
| Option | Type | Default | Description |
|---|---|---|---|
-rowsvar <varName> |
string | rows |
Variable for result rows (execute) |
-rowvar <varName> |
string | row |
Variable for current row (foreach) |
-timevar <varName> |
string | time |
Variable for timing data |
-pairs |
flag | false | Return results as name-value pairs |
-names |
flag | false | Include column names in results |
-nested |
flag | false | Allow nested result sets |
-nofixup |
flag | false | Skip result object fixup/conversion |
-limit <int> |
int | unlimited | Maximum number of rows to return |
-nocreate |
flag | false | Don't create opaque object handles |
| Option | Type | Description |
|---|---|---|
-transaction <name> |
string | Transaction to use with this query |
-timeout <seconds> |
int | Command execution timeout |
-commandtype <enum> |
CommandType | Text (default), StoredProcedure, TableDirect |
-behavior <enum> |
CommandBehavior | Command behavior flags |
-time |
flag | Enable performance profiling |
| Option | Type | Description |
|---|---|---|
-objectname <name> |
string | Name for created opaque object handle |
-returntype <type> |
Type | Expected return type |
-objecttype <type> |
Type | Expected object type |
-create |
flag | Create opaque object handle for result |
-nodispose |
flag | Don't dispose returned objects |
-alias |
flag | Create command alias for result object |
-aliasraw |
flag | Create raw (unprocessed) alias |
-aliasall |
flag | Alias all result objects |
-aliasreference |
flag | Alias by reference (unsafe) |
-tostring |
flag | Convert result objects to string |
-objectflags <enum> |
ObjectFlags | Flags controlling object behavior |
-noforcedelete |
flag | Don't force object deletion |
| Option | Type | Description |
|---|---|---|
-changed <callback> |
ICallback | Hook for the database connection's "Changed" event |
The -changed option uses reflection to find the connection type's
Changed event (e.g., SQLiteConnection.Changed) and attaches the
callback delegate as an event handler. This enables monitoring of
connection state changes, command execution events, and other
provider-specific notifications.
SQL parameters prevent injection attacks and enable type-safe value passing.
Parameters are specified as lists after the SQL text:
sql execute $conn "SELECT * FROM users WHERE id = @id AND name = @name" \
{id Int32 42} {name String "Alice"}Each parameter is a list with the format:
{paramName ?paramType? ?paramValue? ?paramSize? ?paramValueFlags?}| Field | Required | Description |
|---|---|---|
paramName |
Yes | Parameter name (e.g., @id, @name) |
paramType |
No | DbType enum value (Int32, String, DateTime, Binary, etc.) |
paramValue |
No | The parameter value |
paramSize |
No | Size hint for string/binary parameters |
paramValueFlags |
No | ValueFlags for conversion control |
Parameters are processed by DataOps.GetParameters(), which:
- Parses the parameter list from the argument array
- Creates
IDbDataParameterobjects on the command - Converts values using the specified culture, DateTime settings, and value format
- Supports object references (opaque handles) as parameter values
Always use parameterized queries to prevent SQL injection:
# DANGEROUS: Direct string interpolation
sql execute $conn "SELECT * FROM users WHERE name = '$userInput'"
# SAFE: Parameterized query
sql execute $conn "SELECT * FROM users WHERE name = @name" \
{name String $userInput}When -time is specified, the command measures two phases:
- Prepare phase — time spent in
command.Prepare() - Execute phase — time spent in result retrieval
Timing data is stored in the time variable (default:
sql(ResultSet.Time)) as an array:
sql execute -time $conn "SELECT * FROM large_table"
puts "Prepare: $sql(ResultSet.Time)(prepare)"
puts "Execute: $sql(ResultSet.Time)(execute)"The profiler uses IProfilerState / ProfilerState.Create():
- On Windows with
NATIVEcompilation: uses high-resolution native performance counters (henceCommandFlags.NativeCode) - Otherwise: uses managed timing mechanisms
The profiler is created before Prepare(), started, stopped after
prepare, timing stored, restarted before execute, and stopped after
execute. Both measurements are stored in the time variable.
Script bundles are SQLite databases that contain digitally signed Eagle scripts. They serve as secure, portable containers for distributing script packages.
Eagle is frequently embedded inside larger applications — often in security-sensitive roles — where the scripts it runs are themselves trust-bearing: they may drive licensing decisions, configuration, automation, or privileged operations. In that setting, loose script files on disk are a liability: anyone (or any process, or any supply-chain step) can read, modify, substitute, or inject them, and the host has no built-in way to know whether the script it is about to evaluate is authentic and unmodified.
The bundle subsystem solves this by giving scripts the guarantees that code signing gives compiled binaries — and several that it does not:
- Authenticity & integrity (tamper-evidence). Every row is RSA-signed over all of its columns — the script text and its metadata — and that signature is verified against the interpreter's trusted script key rings immediately before the script is evaluated. A modified, unsigned, or untrusted script is refused, not silently run.
- Confidentiality. The entire database can be encrypted with the SQLite Encryption Extension (SEE), so the script text is not even readable without the mount password.
- Controlled execution (defense in depth). Each script carries its
own
IsolationLevel,SecurityLevel,SecurityFlags, andRuleSet, so a bundle declares how and where each script may run — in a safe interpreter, in an isolated child interpreter or AppDomain, with only an explicitly allowed set of commands and policies. Trust is not all-or-nothing. - Provenance & auditability. The
Vendor,PublicKeyToken,HashAlgorithm, andTimeStampcolumns record who produced and signed each script, with what key, and when. - Portability & convenience. An entire library of scripts — plus their package-index files — ships as a single file that is trivial to copy, embed, version, and deploy, instead of a sprawling tree of loose files.
In short, a bundle is the script-level analogue of a signed, encrypted,
sandboxed software package: it lets a host execute third-party or vendor
scripts only when it can prove they are authentic, unmodified, and
confined to a declared security envelope. This is the same trust chain
Eagle uses for its licensing/signing infrastructure; see
safe.md for the surrounding security model.
Operational lifecycle (high level). A bundle is produced by writing
each script as a row in the Scripts table and signing that row (over
all columns) with an RSA key pair from a trusted key ring; the database
may then be encrypted with SEE and distributed as a single file. At run
time the database is mounted read-only (optionally copied into memory
first), and its fully-qualified path becomes a virtual file system
whose members are the embedded scripts, addressed by their POSIX
FullName. Scripts with a positive Sequence are evaluated in order
by EvaluateBundleFile; scripts with a negative Sequence are
package-index / package files reached through
[package scan -normal -host -bundle --] or
[interp readorgetscriptfile]. Every script's signature is re-verified
against the trusted key rings just before evaluation, and its
IsolationLevel / SecurityLevel / SecurityFlags / RuleSet
constrain how and where it runs.
The bundle database schema is defined in scratch/eagle/sql/scripts.sql:
CREATE TABLE IF NOT EXISTS Scripts(
Id BLOB(16) NOT NULL, -- Unique script identifier (GUID)
Language TEXT NOT NULL, -- Always "Eagle"
Sequence INTEGER NOT NULL, -- Evaluation order (non-zero)
IsolationLevel TEXT NULL, -- None, Interpreter, AppDomain,
-- AppDomainOrInterpreter, Process,
-- Session, or Machine
SecurityLevel TEXT NULL, -- None, Safe, or Sdk
SecurityFlags TEXT NULL, -- ScriptSecurityFlags
RuleSet TEXT NULL, -- Command/policy filtering rules
BlockType TEXT NULL, -- XmlBlockType: normally Automatic,
-- Text, Base64, or Uri; reserved for
-- future use, currently must be None
FullName TEXT NOT NULL, -- POSIX path, e.g. /some/script.eagle
"Group" TEXT NOT NULL, -- Logical group name
Description TEXT NOT NULL, -- Human-readable description
TimeStamp DATETIME NOT NULL, -- Creation/modification timestamp
PublicKeyToken BLOB(8) NOT NULL, -- SNK public key token
Text TEXT NOT NULL, -- Script source code
Vendor TEXT NOT NULL, -- e.g. "Mistachkin Systems"
HashAlgorithm TEXT NOT NULL, -- e.g. "SHA512"
Signature BLOB NOT NULL, -- RSA signature (typically 2048 bytes
-- for 16384-bit keys)
UNIQUE (Id),
UNIQUE (FullName),
UNIQUE (Language, Sequence)
);Sequence — Controls evaluation order and package association:
- Positive values: Scripts evaluated in this order by
EvaluateBundleFile. The sequence is relative to other scripts in the same bundle. - Negative values: Package index files and associated package
scripts. These are NOT directly evaluated by
EvaluateBundleFile; instead, they are accessed via[package scan]or[interp readorgetscriptfile]. Package index files require[package scan -normal -host -bundle --].
IsolationLevel — Controls the isolation boundary for evaluating the bundled script:
| Value | Meaning |
|---|---|
None |
No isolation; evaluated in the current interpreter |
Interpreter |
Evaluated in a new child interpreter |
AppDomain |
Evaluated in a new AppDomain |
AppDomainOrInterpreter |
AppDomain if available, otherwise interpreter |
Process |
Evaluated in a new process |
Session |
Session-level isolation |
Machine |
Machine-level isolation |
SecurityLevel — Security context for evaluation:
| Value | Meaning |
|---|---|
None |
Standard execution |
Safe |
Evaluated in a safe interpreter |
Sdk |
SDK-level security context |
RuleSet — Optional list of rules for command/policy filtering in interpreters created to evaluate the bundle. Each rule is a dictionary:
rule {
type Include
kind Command
mode {Include Exact}
patterns after
}Rule fields:
type—IncludeorExcludekind—CommandorPolicymode— Match modes:Exact,Glob,RegExp,SubString, etc.patterns— List of identifier name patterns
Signature — RSA digital signature of ALL data in the row. Signatures are typically 2048 bytes (corresponding to 16384-bit RSA keys) and use SHA512 hashing. Before any script is evaluated, its signature is verified against the interpreter's trusted script key rings.
The BundleData class (Eagle._Components.Private.BundleData)
represents a single script entry from a bundle database:
IBundleData
├── Language (string)
├── Sequence (long)
├── Vendor (string)
├── Path (string) — the database file path
├── FullName (string) — the script's POSIX path within the bundle
├── HashAlgorithmName (string) — e.g., "SHA512"
├── FileBytes (byte[]) — the raw script bytes
├── IsolationLevel (IsolationLevel)
├── SecurityLevel (SecurityLevel)
├── SecurityFlags (ScriptSecurityFlags)
└── RuleSet (IRuleSet) — command/policy filtering rulesSecurity flags include ReadOnly, Immutable, NoVendor,
NoHashAlgorithm, NoEntityType, NoEntityName, NoEntityValue,
NoBlockType, and TreatAsFile. The typical value is BundleMask.
The MakeImmutable() method sets the Immutable flag permanently —
once called, it cannot be undone. This is used for security when
passing script objects to the policy engine.
The BundleManager class manages mounted script bundle databases as
virtual file systems:
Mounting and unmounting:
BundleManager.Mount(interpreter, fileName, password, errorOnMounted)
BundleManager.Unmount(interpreter, fileName, errorOnNotMounted)
BundleManager.ListMounts(interpreter, pattern, noCase)- Databases are mounted read-only — and deliberately so. If SQLite opened the file read-write it might rewrite an internal header field (e.g. the "last opened with version"), which would alter the file's bytes and invalidate the per-row signatures; read-only mounting preserves them.
- Databases may optionally be copied into memory before mounting (useful for read-only media, or to avoid leaving the plaintext on disk once decrypted).
- The mounted database's fully-qualified path becomes its own virtual
file system, within which embedded scripts are located by their
FullName. - Databases may be encrypted using the SQLite Encryption Extension (SEE); the mount password is required to open them.
- Passwords are stored as
byte[]in the mount dictionary - File names use platform-appropriate path comparison
(
PathOps.Comparer)
Data retrieval:
BundleManager.GetData(interpreter, cultureInfo, encoding, path, ref data)The path format for bundle scripts is {databaseFile}:{scriptPath}:
/path/to/scripts.db:/some/script.eagleThe retrieval process:
- Validates the path against security regex patterns
(
DataOps.VerifyBundlePath) - Splits into database file name and script full name
- Looks up the database password in the mount dictionary
- Calls
DataOps.GatherBundleScripts()to query the database - Calls
DataOps.VerifyOneBundleScript()to validate that exactly one matching script was found and returns its bytes
Evaluation tracking:
BundleManager.BeginEvaluation(interpreter, fileName, out savedFileName)
BundleManager.EndEvaluation(interpreter, ref savedFileName)These methods track which bundle is currently being evaluated, using save/restore semantics to support nested bundle evaluation.
Scripts with negative sequence numbers are treated as package components:
- They are NOT evaluated by the main bundle evaluation loop
- They can be accessed via
[package scan -normal -host -bundle --] - Package index paths follow the pattern:
databaseFile:/path/to/pkg/dir/pkgIndex.eagle - The full name is matched via regular expression patterns, e.g.:
^\/path\/to\/pkg\/dir\/pkgIndex(?:_[0-9a-f]{16})?$ - Associated package script files can be evaluated using their
database-qualified paths, e.g.,
eagle.db:/script.eagle
Script bundles implement defense-in-depth security:
- Database integrity — Verified via
PRAGMA integrity_checkbefore processing - Per-script signatures — Each script row is individually signed with RSA, covering ALL columns including metadata
- Key ring verification — Signatures are verified against the interpreter's trusted script key rings
- Hash algorithm enforcement — Configurable but typically SHA512
- Public key token matching — SNK public key tokens must match trusted publishers
- Vendor identification — Each script identifies its vendor
- Encryption — Databases can be encrypted using the SQLite Encryption Extension, requiring a password to mount
- Immutability — Script data can be made immutable once loaded, preventing modification
- Isolation levels — Scripts can require specific isolation boundaries for execution
- Rule-set filtering — Scripts can restrict which commands and policies are available during execution
Eagle supports these built-in connection types:
| Type | .NET Provider | Assembly |
|---|---|---|
Odbc |
System.Data.Odbc.OdbcConnection |
System.Data |
OleDb |
System.Data.OleDb.OleDbConnection |
System.Data |
Sql |
System.Data.SqlClient.SqlConnection |
System.Data |
Oracle |
System.Data.OracleClient.OracleConnection |
System.Data.OracleClient |
SqlCe |
System.Data.SqlServerCe.SqlCeConnection |
System.Data.SqlServerCe |
SQLite |
System.Data.SQLite.SQLiteConnection |
System.Data.SQLite |
SQLiteEnterprise |
System.Data.SQLite.SQLiteConnection |
System.Data.SQLite (Enterprise) |
The connection creation process (DataOps.CreateDbConnection())
follows a fallback chain:
- Try
type1(primary type) - If that fails, try
type2(fallback type) - If custom type names are specified (
-typename/-typefullname), resolve via reflection - If
-assemblyfilenameis specified, load the assembly first - Apply public key token verification if required
- Apply trust verification if
ValueFlags.TrustedOnlyis set
When the user specifies -type SQLite:
if ((dbConnectionType1 == DbConnectionType.SQLite) &&
(dbConnectionType2 == DbConnectionType.None))
{
dbConnectionType1 = DbConnectionType.SQLiteEnterprise;
dbConnectionType2 = DbConnectionType.SQLite;
}This transparent upgrade ensures the enterprise edition (with encryption support, enhanced features, etc.) is used when available, with automatic fallback to the standard edition.
Eagle provides a separate but related feature: database-backed
array variables. A DatabaseVariable transparently maps Eagle array
variable operations to SQL DML statements:
| Variable operation | SQL operation |
|---|---|
| Get element | SELECT by name |
| Set element | INSERT or UPDATE |
| Unset element | DELETE by name |
| List elements | SELECT all names |
This is implemented via its own TraceCallback (separate from
DbTraceCallback) that intercepts variable get/set/unset operations
and translates them to SQL queries. The implementation supports
database-specific syntax for Oracle (ROWID), SQL Server ($IDENTITY),
and SQLite (rowid, CAST).
The [sql] command fires interpreter notifications at key lifecycle
points:
| Operation | NotifyType | NotifyFlags |
|---|---|---|
| Connection opened | Connection |
Added |
| Connection closed | Connection |
Removed |
| Transaction begun | Transaction |
Added |
| Transaction committed | Transaction |
Removed |
| Transaction rolled back | Transaction |
Removed |
These notifications are gated by the NOTIFY compilation flag and
enable plugins and other subsystems to monitor database lifecycle
events.
# Open a SQLite connection
set conn [sql open -type SQLite "Data Source=mydb.db"]
# Execute a non-query (returns rows affected)
sql execute -execute NonQuery $conn "CREATE TABLE users(id INTEGER, name TEXT)"
sql execute -execute NonQuery $conn "INSERT INTO users VALUES(1, 'Alice')"
# Execute a scalar query (returns single value)
set count [sql execute -execute Scalar $conn "SELECT COUNT(*) FROM users"]
# Execute a reader query (returns result set)
sql execute -execute Reader -format NestedList $conn "SELECT * FROM users"
# Close
sql close $connproc queryDatabase {dbFile query} {
# Connection is automatically closed when $conn goes out of scope
sql open -variable conn -type SQLite \
"Data Source=$dbFile;Read Only=True"
return [sql execute -execute Reader -format NestedList \
$conn $query]
# No explicit close needed — DbTraceCallback handles it
}proc transferFunds {dbFile fromId toId amount} {
sql open -variable conn -type SQLite "Data Source=$dbFile"
# Transaction auto-commits on clean exit, auto-rolls-back on error
sql transaction -variable trans begin $conn
sql execute -transaction $trans $conn \
"UPDATE accounts SET balance = balance - @amt WHERE id = @id" \
{amt Int32 $amount} {id Int32 $fromId}
sql execute -transaction $trans $conn \
"UPDATE accounts SET balance = balance + @amt WHERE id = @id" \
{amt Int32 $amount} {id Int32 $toId}
sql transaction commit $trans
# If an error occurs before commit, DbTraceCallback rolls back
}set conn [sql open -type SQLite "Data Source=app.db"]
# The row variable is an array keyed by the (1-based) row number; with
# -format Dictionary each element is a column-name/value dictionary.
sql foreach -execute Reader -format Dictionary $conn \
"SELECT id, name, email FROM users" {
set n [lindex [array names row] 0]
puts "User [dict get $row($n) id]: \
[dict get $row($n) name] <[dict get $row($n) email]>"
}
sql close $conn# ALWAYS use parameters for user input
sql execute -execute Reader $conn \
"SELECT * FROM products WHERE category = @cat AND price < @max" \
{cat String "Electronics"} {max Double 99.99}
# With explicit parameter size
sql execute -execute NonQuery $conn \
"INSERT INTO logs(message) VALUES(@msg)" \
{msg String "Event occurred" 1000}sql execute -time -execute Reader $conn "SELECT * FROM large_table"
puts "Prepare time: $sql(ResultSet.Time)(prepare)"
puts "Execute time: $sql(ResultSet.Time)(execute)"# SQL Server
set sqlConn [sql open -type Sql \
"Server=localhost;Database=mydb;Integrated Security=True"]
# ODBC
set odbcConn [sql open -type Odbc \
"Driver={SQL Server};Server=localhost;Database=mydb"]
# Custom provider with explicit assembly
set customConn [sql open -assemblyfilename "/path/to/provider.dll" \
-typefullname "Custom.Data.Connection" \
"CustomConnectionString"]set conn [sql open -type SQLite "Data Source=test.db"]
set trans [sql transaction begin $conn]
# Is the transaction active?
puts [sql hasbegun $trans] ;# 1
# Does it belong to this connection?
puts [sql hasbegun $trans $conn] ;# 1
sql transaction commit $trans
puts [sql hasbegun $trans] ;# 0
sql close $conn| Aspect | Eagle [sql] |
Python DB-API |
|---|---|---|
| Provider abstraction | ADO.NET interfaces | DB-API 2.0 (PEP 249) |
| Connection | sql open -type SQLite connStr |
sqlite3.connect("db.sqlite3") |
| Parameters | {name Type value} lists |
? or :name placeholders |
| Auto-cleanup | DbTraceCallback on variable unset | with statement (context manager) |
| Result iteration | [sql foreach] |
cursor.fetchall() / for row in cursor |
| Transaction | sql transaction begin/commit |
connection.commit() |
| Script bundles | SQLite-based signed script databases | No equivalent |
| Aspect | Eagle [sql] |
Tcl tdbc |
|---|---|---|
| Installation | Built-in | package require tdbc + driver |
| Connection | sql open -type SQLite connStr |
tdbc::sqlite3::connection create db "file.db" |
| Query | sql execute $conn "SELECT ..." |
$db allrows "SELECT ..." |
| Parameters | {name Type value} |
:name with variable binding |
| Transactions | sql transaction begin/commit/rollback |
$db begintransaction / $db commit |
| Auto-cleanup | DbTraceCallback | Object command deletion |
| Script bundles | Signed SQLite databases | No equivalent |
| Provider types | 7+ built-in types | One package per driver |
Eagle's [sql] maps almost 1:1 to ADO.NET, but adds:
- Automatic type discovery for connection providers
- Integrated result formatting (lists, dictionaries, arrays)
- DbTraceCallback for scope-based resource management
- Script-level parameter binding without compile-time types
- Built-in performance profiling
- Script bundle database system
Always use parameterized queries for user-supplied input:
# VULNERABLE:
sql execute $conn "SELECT * FROM t WHERE x = '$userInput'"
# SAFE:
sql execute $conn "SELECT * FROM t WHERE x = @val" {val String $userInput}[sql] is marked Unsafe — it is hidden and inaccessible in safe
interpreters. This prevents untrusted code from:
- Opening arbitrary database connections
- Reading or modifying database contents
- Exhausting database resources
- Accessing the file system via database file paths
The -trustedonly option and DataFlags.TrustedOnly /
DataFlags.VerifiedOnly interpreter flags ensure that only signed
and trusted provider assemblies can be loaded. Public key token
verification (-publickeytoken1, -publickeytoken2) prevents loading
of unauthorized provider implementations.
Script bundles implement a multi-layer security model (see Section 8.6) that ensures scripts are authentic, unmodified, and authorized for execution.
| Command | Relationship |
|---|---|
[object] |
The [sql] command uses [object]-style opaque handles for connection and transaction management. Result objects can be aliased via -alias, -create, etc. |
[load] |
Database provider assemblies may need to be loaded via [load] or -assemblyfilename before use. See load.md. |
[source] |
Script bundle databases can be evaluated via [source -bundle]. |
[package] |
Package index files in bundles are accessed via [package scan -normal -host -bundle --]. |
[interp] |
Bundle scripts with IsolationLevel of Interpreter or Safe SecurityLevel create child interpreters for evaluation. See interp.md. |
[library] |
Native FFI. Complementary: [sql] accesses managed databases; [library] calls native C functions. See library.md. |
The canonical example of production-grade Eagle database scripting is the
System.Data.SQLite test suite (sqlite/dotnet/Tests/ — 113 .eagle
test files, backed by the sqlite/dotnet/lib/System.Data.SQLite/common.eagle
library at ~7,000 lines). These patterns have been refined over many years
of real-world use.
Production code never calls [sql open] directly. Instead, a wrapper
procedure builds the connection string from multiple sources, opens the
connection, configures PRAGMAs, and runs setup SQL:
proc setupDb {fileName {mode ""} {dateTimeFormat ""} {dateTimeKind ""}
{flags ""} {extra ""} {qualify true} {delete true}
{uri false} {temporary true} {varName db} {quiet false}} {
upvar 1 $varName db
# Build connection string from multiple sources
set connection "Data Source=${fileName};ToFullPath=${qualify}"
if {$mode ne ""} { append connection ";Journal Mode=${mode}" }
if {$dateTimeFormat ne ""} { append connection ";DateTimeFormat=${dateTimeFormat}" }
append connection [getTestProperties $flags $extra]
# Open connection
set db [sql open -type SQLite $connection]
# Configure temporary directory
sql execute $db "PRAGMA temp_store_directory = \"${tempDir}\";"
# Run per-connection setup SQL (configurable)
set setupSql [getExecuteOnSetup]
if {$setupSql ne ""} { sql execute $db $setupSql }
}Cleanup is equally thorough:
proc cleanupDb {fileName {varName db} {collect true} {qualify true}
{delete true} {pool true} {quiet false}} {
upvar 1 $varName db
# Clear connection pools
catch {object invoke System.Data.SQLite.SQLiteConnection ClearAllPools}
# Force garbage collection to release file handles
if {$collect} { collectGarbage $::test_channel }
# Close connection
sql close $db
# Delete associated files (WAL, SHM, main database)
if {$delete} {
catch {file delete "${fileName}-wal"}
catch {file delete "${fileName}-shm"}
catch {file delete $fileName}
}
unset db
}Connection strings are assembled from multiple sources with priority merging:
# Local flags + global overrides + shared flags
set flags [combineFlags $localFlags ""]
if {[info exists ::connection_flags]} {
set flags [combineFlags $flags $::connection_flags]
}
# Extra connection properties
set extra [combineExtra $::connection_extra $localExtra]This allows per-user configuration files (settings.before.username.eagle)
to override defaults without modifying test code.
Parameters use ? placeholders with typed list bindings:
# Check if a table exists (parameterized)
set sql {SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?;}
set exists [expr {[sql execute -execute scalar $db $sql \
[list param1 String $tableName]] > 0}]
# Insert with typed parameters
sql execute $db {INSERT INTO t1 (x, y) VALUES(?, ?);} \
[list param1 Int64 42] [list param2 String "value"]Each parameter is a list: {paramName DbType Value}. The paramName is
a logical name (the actual binding is positional for ? placeholders, or
named for @param syntax).
Scalar results (single value):
set count [sql execute -execute scalar $db "SELECT COUNT(*) FROM t1;"]Reader with implicit $rows array:
sql execute -execute reader $db "SELECT x, y, z FROM t1;"
# $rows(count) = number of rows
# $rows(names) = column name list
# $rows(1) = first row as value list
# $rows(2) = second rowDataReader for streaming large result sets:
set reader [sql execute -execute reader -format datareader \
-alias $db "SELECT * FROM large_table;"]
while {[$reader Read]} {
set id [$reader GetValue [$reader GetOrdinal "id"]]
set name [$reader GetValue [$reader GetOrdinal "name"]]
# Process one row at a time -- constant memory usage
}
unset readerDataTable for materialized results with named column access:
set table [sql execute -execute reader -format datatable -alias \
$db "SELECT id, name, age FROM users;"]
# Built-in conversion methods (replaces getRowsFromDataTable)
set rows [$table ToList] ;# {{1 Alice 30} {2 Bob 25}}
set dicts [$table ToDictionary] ;# {{id 1 name Alice age 30} ...}
set cols [$table GetColumnNames] ;# {id name age}
# Named column access on individual rows
object foreach -alias row [$table Rows] {
puts "[$row Item name]: [$row Item age]"
}
# In-memory filtering (no new query needed)
set filtered [$table Select "age >= 30"]
unset tableThe DataTable format returns a custom DataOps._DataTable object (derived
from System.Data.DataTable) that captures the value formatting parameters
(DateTimeBehavior, BlobBehavior, etc.) so that ToList and ToDictionary
apply the same conversion pipeline (MarshalOps.FixupDataValue) as other
[sql execute] formats. This replaces the manual getRowsFromDataTable
pattern from the SQLite test suite library.
set transaction [sql transaction begin $db]
if {[catch {
sql execute $db "INSERT INTO t1 VALUES(1, 'a');"
sql execute $db "INSERT INTO t1 VALUES(2, 'b');"
sql transaction commit $transaction
} error]} {
catch {sql transaction rollback $transaction}
error $error
}Nested transactions:
set outer [sql transaction begin $db]
set inner [sql transaction begin $db] ;# savepoint
catch {sql execute $db "INSERT ..."}
sql transaction rollback $inner ;# rollback to savepoint
sql transaction commit $outer ;# commit the restThe test suite demonstrates deep .NET interop for advanced database features:
Event handlers (authorization callbacks):
proc onAuthorize {sender e} {
if {[$e ActionCode] eq "CreateTable"} {
$e ReturnCode Deny
}
}
set connection [getDbConnection]
object invoke $connection add_Authorize onAuthorize
# ... use connection ...
# Cleanup ritual: remove handler, remove callback, delete proc
catch {object invoke $connection remove_Authorize onAuthorize}
catch {object removecallback onAuthorize}
rename onAuthorize ""Type callbacks (custom marshaling):
set callback {-callbackflags +Default readValueCallback}
set typeCallbacks [object invoke -marshalflags +DynamicCallback \
System.Data.SQLite.SQLiteTypeCallbacks Create \
null $callback null null]
$connection SetTypeCallbacks "TYPENAME" $typeCallbacksConnection pooling:
# Enable pooling in connection string
setupDb $fileName "" "" "" "" "Pooling=True;"
# Explicit pool management
catch {
object invoke -flags +NonPublic \
System.Data.SQLite.SQLiteConnectionPool ClearAllPools
}# Handle leak detection
proc getSQLiteHandleCounts {channel} {
set counts [list]
foreach name {connectionCount statementCount backupCount blobCount} {
lappend counts $name [object invoke -flags +NonPublic \
System.Data.SQLite.UnsafeNativeMethods \
sqlite3_changes_interop ...]
}
return $counts
}
# Full shutdown with leak detection
proc shutdownSQLite {channel} {
# Roll back leaked transactions
foreach transaction [info transactions] {
catch {sql transaction rollback $transaction}
}
# Close leaked connections
foreach connection [info connections] {
catch {sql close $connection}
}
}Every test follows a consistent lifecycle:
runTest {test data-1.1 "basic CRUD operations" \
-setup {
setupDb [set fileName data-1.1.db]
} \
-body {
sql execute $db "CREATE TABLE t1(x INTEGER, y TEXT);"
sql execute $db "INSERT INTO t1 VALUES(1, 'hello');"
set result [sql execute -execute scalar $db \
"SELECT y FROM t1 WHERE x = ?;" \
[list param1 Int64 1]]
} \
-cleanup {
cleanupDb $fileName
unset -nocomplain result
} \
-constraints {eagle command.sql compile.DATA SQLite \
System.Data.SQLite} \
-result {hello}
}- Source code:
Eagle/Library/Commands/Sql.cs—[sql]command implementation (9 sub-commands) - Source code:
Eagle/Library/Components/Private/DataOps.cs— database operations, connection creation, result formatting, bundle scripts - Source code:
Eagle/Library/Components/Private/ObjectOps.cs— SQL execute option definitions - Source code:
Eagle/Library/Components/Public/Interpreter.cs— DbTraceCallback, SetDbVariableValue, connection/transaction storage - Source code:
Eagle/Library/Components/Public/DatabaseVariable.cs— database-backed variable system - Source code:
Eagle/Library/Components/Private/BundleData.cs— script bundle data representation - Source code:
Eagle/Library/Components/Private/BundleManager.cs— bundle mounting and data retrieval - Schema:
scratch/eagle/sql/scripts.sql— script bundle database schema - Related documentation:
core_language.md— basic sql command reference - Related documentation:
core_examples.md— sql examples - Related documentation:
tips_and_tricks.md— SQL best practices - Related documentation:
core_script_library.md— database utility procedures (haveColumnValue, getColumnValue, etc.) - Related documentation:
load.md— plugin loading (for database provider assemblies) - Related documentation:
interp.md— interpreter security model (for bundle isolation) - Production reference:
sqlite/dotnet/Tests/*.eagle— 113 production test files demonstrating real-world database access patterns - Production reference:
sqlite/dotnet/lib/System.Data.SQLite/common.eagle— ~7,000-line test support library withsetupDb,cleanupDb, connection string building, DataTable conversion, and resource management