[Feature][PublicService] Optimize OOM issue in ps-public service after enabling Kerberos authentication #5312

Linkis Component

linkis-public-enhancements/linkis-ps-public-service

What happened

English:

After enabling Kerberos authentication, the ps-public service frequently encounters Out Of Memory (OOM) issues due to excessive memory consumption in authentication-related operations.

Problem Description:
When Kerberos authentication is enabled, the ps-public service experiences significant memory pressure and eventually runs out of memory. This appears to be related to Kerberos ticket caching, principal object creation, or authentication state management.

中文：

开启Kerberos认证后，ps-public服务由于认证相关操作的内存消耗过大，频繁出现内存溢出(OOM)问题。

问题描述：
启用Kerberos认证时，ps-public服务会遇到严重的内存压力并最终耗尽内存。这似乎与Kerberos票据缓存、主体对象创建或认证状态管理有关。

What you expected to happen

English:

After enabling Kerberos authentication, the ps-public service should:

1. Maintain stable memory usage without OOM
2. Properly cache and reuse Kerberos tickets
3. Release authentication objects after use
4. Implement memory-efficient Kerberos authentication

中文:

开启Kerberos认证后，ps-public服务应该：

1. 保持稳定的内存使用而不会OOM
2. 正确缓存和重用Kerberos票据
3. 使用后释放认证对象
4. 实现内存高效的Kerberos认证

How to reproduce

English:

1. Enable Kerberos authentication in Linkis configuration
2. Configure ps-public service with Kerberos principals
3. Submit multiple tasks or requests through ps-public
4. Monitor service memory usage over time
5. Observe gradual memory increase leading to OOM

中文:

1. 在Linkis配置中启用Kerberos认证
2. 使用Kerberos主体配置ps-public服务
3. 通过ps-public提交多个任务或请求
4. 监控服务内存使用随时间的变化
5. 观察逐渐增加的内存导致OOM

Anything else

English:

Potential Root Causes:

1. Kerberos ticket cache not properly managed: Tickets accumulate without cleanup
2. Principal objects not released: UserGroupInformation or LoginContext objects retained
3. Memory leak in authentication interceptor: Objects created per request not garbage collected
4. Excessive ticket renewal: Too frequent renewal operations consuming memory

Suggested Solutions:

1. Implement ticket cache management: Add LRU cache with size limits for Kerberos tickets
2. Use object pooling: Reuse LoginContext and UGI objects instead of creating new ones
3. Add cleanup hooks: Ensure proper cleanup of Kerberos objects after authentication
4. Optimize renewal strategy: Reduce ticket renewal frequency, implement lazy renewal
5. Memory profiling: Use heap dump analysis to identify specific leak sources

Configuration Recommendations:

# JVM settings for Kerberos-enabled service
-Xms4g -Xmx8g
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-Djava.security.krb5.conf=/etc/krb5.conf
-Dsun.security.krb5.debug=false  # Disable Kerberos debug logging in production

中文:

可能的根本原因:

1. Kerberos票据缓存未正确管理：票据累积而没有清理
2. 主体对象未释放：UserGroupInformation或LoginContext对象被保留
3. 认证拦截器中的内存泄漏：每个请求创建的对象未被垃圾回收
4. 票据更新过于频繁：过于频繁的更新操作消耗内存

建议解决方案:

1. 实现票据缓存管理：为Kerberos票据添加具有大小限制的LRU缓存
2. 使用对象池：重用LoginContext和UGI对象而不是创建新对象
3. 添加清理钩子：确保认证后正确清理Kerberos对象
4. 优化更新策略：减少票据更新频率，实现延迟更新
5. 内存分析：使用堆转储分析识别具体的泄漏源

配置建议:

# 启用Kerberos的服务的JVM设置
-Xms4g -Xmx8g
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-Djava.security.krb5.conf=/etc/krb5.conf
-Dsun.security.krb5.debug=false  # 生产环境禁用Kerberos调试日志

Are you willing to submit a PR?

• Yes I am willing to submit a PR!