Add Schema Registry to Provide Table Metadata

[suddendust]

Description

This PR introduces a way to get table schema information using the SchemaRegistry interface. This interface provides methods to retrieve a table's metadata from the underlying DB. For PG, this is the query that is used:

postgres=# SELECT "column_name", "udt_name", "is_nullable" FROM information_schema.columns WHERE table_schema = 'public' AND "table_name" = 'myTestFlat';
 column_name  |  udt_name   | is_nullable 
--------------+-------------+-------------
 _id          | int4        | NO
 item         | text        | YES
 price        | int4        | YES
 quantity     | int4        | YES
 date         | timestamptz | YES
 in_stock     | bool        | YES
 tags         | _text       | YES
 categoryTags | _text       | YES
 props        | jsonb       | YES
 sales        | jsonb       | YES
 numbers      | _int4       | YES
 scores       | _float8     | YES
 flags        | _bool       | YES

Cache Refresh and Expiry

This registry fetches and caches column metadata from PostgreSQL's information_schema on demand. It provides:

1. Lazy loading: Schema metadata is fetched only when first requested for the particular table.
2. TTL-based caching: Cached schemas expire after a configurable duration (default: 24 hours).
3. Circuit breaker: Prevents excessive database calls by enforcing a cooldown period between refresh attempts for missing columns (default: 15 minutes).

Testing

[x] Added UTs.
[x] Added integration tests.

Checklist:

• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• Any dependent changes have been merged and published in downstream modules

[codecov]

Codecov Report

❌ Patch coverage is 94.17476% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.51%. Comparing base (a7acef1) to head (1d14adf).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../postgres/PostgresLazyilyLoadedSchemaRegistry.java	84.37%	4 Missing and 1 partial ⚠️
...a/org/hypertrace/core/documentstore/Datastore.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #263      +/-   ##
============================================
+ Coverage     80.28%   80.51%   +0.22%     
- Complexity     1341     1385      +44     
============================================
  Files           231      234       +3     
  Lines          6093     6194     +101     
  Branches        548      554       +6     
============================================
+ Hits           4892     4987      +95     
- Misses          826      831       +5     
- Partials        375      376       +1     

Flag	Coverage Δ
integration	80.51% <94.17%> (+0.22%)	⬆️
unit	58.65% <93.20%> (+0.57%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[suddendust]

@suresh-prakash

[suresh-prakash]

Prefer Optional instead of nulls. That way, we could avoid errors/bugs where the clients forget NULL handling. By returning Optional, we are forcing the clients to do something with the null.

[suddendust]

Agreed. Changed to optional now.

[suresh-prakash]

This change might not be really needed (since all the services have moved to the new way of configuring).

[suddendust]

As discussed, changed this to use customParameters.