Streaming module for Teradata profiler extracts

[dey-abhishek]

Changes

What does this PR do?

Streams Teradata profiler extracts instead of buffering the whole result set in memory, and bumps the core DBQL look-back window from 7 to 30 days.

TeradataConnector now implements the streaming interface (supports_streaming() → True), so the pipeline pulls each SQL step's results in bounded fetchmany() batches (4096 rows) rather than materializing the entire set. This keeps peak memory flat for large DBQL extracts. Snowflake already streamed; this brings Teradata onto the same path (the pipeline's _stream_sql_step routing and the base interface already exist on main).

Relevant implementation details

• Fixed per-column Arrow schema. teradatasql reports each column's type_code as the Python type it returns; the schema is derived once from cursor.description and reused for every batch, so batches stay schema-identical even when a column is all-NULL in a given batch (the pipeline appends batches positionally into the DDL-pre-created DuckDB table).
• Exactness/normalization. DECIMAL columns are carried as exact text (not float64) so a counter past 2**53 doesn't round; DuckDB casts the string back. tz-aware datetimes (from TIMESTAMP WITH TIME ZONE) are normalized to UTC-naive so they don't get rejected against the tz-naive batch schema.
• Batch size is _STREAM_BATCH_ROWS = 4096.
• The core DBQL extract's window is hardcoded to CURRENT_DATE - INTERVAL '30' DAY (was '7'); the TOP 100000 row cap is unchanged.

Docs

• Corrects the Teradata profiler docs: Teradata has no --variant (it was never registered in SOURCE_SYSTEM_VARIANTS), so the stale flag is dropped from the run command and the "Choosing a Profiler Variant" section is rewritten as "Workload Extraction" — a single run extracts the core DBQL workload and additionally attempts PDCR, with the optional PDCR steps skipped automatically when PDCRINFO is absent.

Functionality

• updated user documentation (Teradata profiler docs)
• added new CLI command
• modified existing command

Tests

• added unit tests — Arrow schema stability (incl. all-NULL batches), large-DECIMAL exactness, tz-aware normalization, unknown-type-code fallback
• make fmt clean (black/ruff/mypy, pylint 10.00/10) and make test passing
• validated end-to-end against a live Teradata box: all steps complete, PDCR steps skipped automatically (ABSENT) where PDCRINFO is absent, and the streamed DuckDB extract is schema-identical to prior buffered runs (release 0.14.2/0.13.0). Row-level comparison against a buffered run confirms streaming loses/alters nothing: the static/inventory tables (td_db_object_types, td_dwh_udf, td_sys_info, td_sys_nodes_info, td_user_databases) are byte-identical row-for-row; the streamed td_dbql_core_info_extract is a strict superset of the earlier run (every earlier row present, extras are only newer live-log queries — 0 rows dropped); and the only remaining deltas are same-key value drift in the inherently time-varying tables (td_sys_disk_utilization point-in-time usage, td_sys_usage_agg re-aggregated ResUsageSpma buckets) — no rows lost, added, or retyped by streaming

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 61.11111% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.19%. Comparing base (e71dffa) to head (cf208dd).

Files with missing lines	Patch %	Lines
...ks/labs/lakebridge/connections/database_manager.py	61.11%	20 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2649      +/-   ##
==========================================
- Coverage   71.25%   71.19%   -0.06%     
==========================================
  Files         112      112              
  Lines       10119    10173      +54     
  Branches     1111     1122      +11     
==========================================
+ Hits         7210     7243      +33     
- Misses       2695     2715      +20     
- Partials      214      215       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

✅ 181/181 passed, 8 flaky, 2 skipped, 1h21m38s total

Flaky tests:

• 🤪 test_recon_for_report_type_is_data (1m6.27s)
• 🤪 test_recon_sql_server_job_succeeds (3m42.885s)
• 🤪 test_recon_databricks_job_succeeds (4m11.673s)
• 🤪 test_recon_redshift_job_succeeds (4m58.149s)
• 🤪 test_recon_teradata_job_succeeds (5m16.334s)
• 🤪 test_recon_snowflake_job_succeeds (4m53.662s)
• 🤪 test_auto_configure_tables_writes_table_recon_config (4m43.84s)
• 🤪 test_recon_oracle_job_succeeds (1m56.86s)

_{Running from acceptance #5419}

[dey-abhishek]

@asnare - Could you review the PR and share your feedback/approval