Adding changes for supporting RLS

[9aman]

Adding Row-Level Security (RLS) Support to Apache Pinot

Overview

This PR introduces comprehensive Row-Level Security (RLS) support to Apache Pinot, enabling fine-grained access control at the row level.

Key Features

🔐 Row-Level Security Framework

• Extends the existing AccessControl interface with getRowColFilters() method
• Supports configurable RLS filters per user principal and table
• Integrates with both single-stage and multi-stage query engines

🔄 Query Rewriting Engine

• Implements RlsFiltersRewriter for automatic query transformation
• Injects RLS filters as WHERE clause conditions

⚙️ Configuration Support

• Extends BasicAuthPrincipal with RLS filter configuration
• Supports multiple RLS policies per user per table

Architecture Changes

Core Components Modified

1. AccessControl Interface (AccessControl.java)

   • Added getRowColFilters(RequesterIdentity, String table) method
   • Returns TableRowColAuthResult containing RLS and CLS filters

2. BasicAuthPrincipal (BasicAuthPrincipal.java)

   • Added _rlsFilters field to store table-specific RLS filters.

3. Query Rewriters (RlsFiltersRewriter.java)

   • New rewriter that applies RLS filters to SQL queries
   • Extracts filters from query options and integrates them into WHERE clauses

4. Broker Request Handlers

   • SingleStageBrokerRequestHandler: Enhanced to retrieve and apply RLS filters
   • MultiStageBrokerRequestHandler: Integrated RLS filter processing

Query Rewriting Flow

graph TD
    A[User Query] --> B[Authentication & Authorization]
    B --> C[Extract User Principal]
    C --> D[Retrieve RLS Filters for Table]
    D --> E{RLS Filters Exist?}
    E -->|Yes| F[Add Filters to Query Options]
    E -->|No| I[Execute Original Query]
    F --> G[RlsFiltersRewriter]
    G --> H[Inject Filters into WHERE Clause]
    H --> I[Execute Modified Query]
    
    style F fill:#e1f5fe
    style G fill:#f3e5f5
    style H fill:#e8f5e8

Loading

Example:

RLS Filter Configuration

# User principal with RLS filters
pinot.broker.access.control.principals="admin, alice"
pinot.broker.access.control.principals.alice.password="alice123"
pinot.broker.access.control.principals.alice.tables="sales_table,user_table"
pinot.broker.access.control.principals.alice.rls.sales_table="department='marketing'"
pinot.broker.access.control.principals.alice.rls.user_table=user_id='alice'

Query Transformation for above config

The user being alice:

Original Query:

SELECT * FROM sales_table WHERE region = 'US'

With RLS Filter Applied:

SELECT * FROM sales_table 
WHERE region = 'US' 
  AND department = 'marketing'

Configuration Format

Implementation Details

Filter Application Strategy

• AND Logic: Multiple RLS filters for the same table are combined using AND operator

Testing Strategy

The implementation includes:

• Unit tests for RLS filter extraction and application
• Integration tests for end-to-end RLS functionality

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 37.96296% with 67 lines in your changes missing coverage. Please review.

Project coverage is 63.19%. Comparing base (1a476de) to head (d10e3f2).
Report is 351 commits behind head on master.

Files with missing lines	Patch %	Lines
...sthandler/BaseSingleStageBrokerRequestHandler.java	14.28%	11 Missing and 1 partial ⚠️
...t/broker/broker/BasicAuthAccessControlFactory.java	0.00%	10 Missing ⚠️
...requesthandler/MultiStageBrokerRequestHandler.java	0.00%	10 Missing ⚠️
...pinot/sql/parsers/rewriter/RlsFiltersRewriter.java	44.44%	9 Missing and 1 partial ⚠️
...he/pinot/spi/auth/TableRowColAccessResultImpl.java	0.00%	10 Missing ⚠️
...org/apache/pinot/sql/parsers/CalciteSqlParser.java	0.00%	7 Missing ⚠️
...ava/org/apache/pinot/core/auth/BasicAuthUtils.java	78.57%	2 Missing and 1 partial ⚠️
...ry/runtime/plan/server/ServerPlanRequestUtils.java	50.00%	2 Missing and 1 partial ⚠️
.../java/org/apache/pinot/query/QueryEnvironment.java	0.00%	1 Missing ⚠️
...va/org/apache/pinot/query/runtime/QueryRunner.java	66.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #16043      +/-   ##
============================================
+ Coverage     62.90%   63.19%   +0.29%     
+ Complexity     1386     1365      -21     
============================================
  Files          2867     2956      +89     
  Lines        163354   170462    +7108     
  Branches      24952    26072    +1120     
============================================
+ Hits         102755   107726    +4971     
- Misses        52847    54573    +1726     
- Partials       7752     8163     +411     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-11	63.14% <37.96%> (+0.27%)	⬆️
java-21	63.17% <37.96%> (+0.34%)	⬆️
skip-bytebuffers-false	?
skip-bytebuffers-true	?
temurin	63.19% <37.96%> (+0.29%)	⬆️
unittests	63.19% <37.96%> (+0.29%)	⬆️
unittests1	64.73% <52.05%> (+8.91%)	⬆️
unittests2	33.39% <13.88%> (-0.19%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[praveenc7]

@9aman Looks like a good change, given it rewrites the query. Did we see any performance overhead?