[rllib] Debug test_offline_prelearner as test is flaky

[pseudo-rnd-thoughts]

Description

test_offline_prelearner is a flaky test from timing out

[2026-01-29T08:48:20Z] (00:48:19) INFO: From Testing //rllib:test_offline_prelearner:
--
[2026-01-29T08:48:20Z] ==================== Test output for //rllib:test_offline_prelearner:
[2026-01-29T08:48:20Z] /opt/miniforge/lib/python3.10/site-packages/glfw/__init__.py:916: GLFWError: (65544) b'X11: The DISPLAY environment variable is missing'
[2026-01-29T08:48:20Z]   warnings.warn(message, GLFWError)
[2026-01-29T08:48:20Z] /opt/miniforge/lib/python3.10/site-packages/paramiko/pkey.py:82: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
[2026-01-29T08:48:20Z]   "cipher": algorithms.TripleDES,
[2026-01-29T08:48:20Z] /opt/miniforge/lib/python3.10/site-packages/paramiko/transport.py:253: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
[2026-01-29T08:48:20Z]   "class": algorithms.TripleDES,
[2026-01-29T08:48:20Z] ============================= test session starts ==============================
[2026-01-29T08:48:20Z] platform linux -- Python 3.10.19, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniforge/bin/python3
[2026-01-29T08:48:20Z] cachedir: .pytest_cache
[2026-01-29T08:48:20Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/io_ray/bazel-out/k8-opt/bin/rllib/test_offline_prelearner.runfiles/io_ray
[2026-01-29T08:48:20Z] plugins: anyio-4.12.0, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, pytest_httpserver-1.1.3, lazy-fixtures-1.1.2, mock-3.14.0, remotedata-0.3.2, rerunfailures-11.1.2, shutil-1.8.1, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, virtualenv-1.8.1, typeguard-2.13.3
[2026-01-29T08:48:20Z] asyncio: mode=auto
[2026-01-29T08:48:20Z] collecting ... collected 7 items
[2026-01-29T08:48:20Z]
[2026-01-29T08:48:20Z] rllib/offline/tests/test_offline_prelearner.py::TestOfflinePreLearner::test_offline_prelearner_buffer_class PASSED [ 14%]
[2026-01-29T08:48:20Z] rllib/offline/tests/test_offline_prelearner.py::TestOfflinePreLearner::test_offline_prelearner_convert_from_old_sample_batch_to_episodes PASSED [ 28%]
[2026-01-29T08:48:20Z] rllib/offline/tests/test_offline_prelearner.py::TestOfflinePreLearner::test_offline_prelearner_convert_to_episodes PASSED [ 42%]
[2026-01-29T08:48:20Z] rllib/offline/tests/test_offline_prelearner.py::TestOfflinePreLearner::test_offline_prelearner_ignore_final_observation PASSED [ 57%]
[2026-01-29T08:48:20Z] rllib/offline/tests/test_offline_prelearner.py::TestOfflinePreLearner::test_offline_prelearner_in_map_batches PASSED [ 71%]
[2026-01-29T08:48:20Z] rllib/offline/tests/test_offline_prelearner.py::TestOfflinePreLearner::test_offline_prelearner_sample_from_episode_data -- Test timed out at 2026-01-29 08:38:12 UTC --
[2026-01-29T08:48:20Z] ================================================================================

As it is timing out then we don't see the logs for where this is happening.
This PR adds "-s" to the pytest for this test only and added debug print statements everywhere + changes the timeout from medium to large

[gemini-code-assist]

Code Review

This pull request aims to debug a flaky test, test_offline_prelearner, by increasing its timeout and adding extensive debug logging. While these changes are useful for diagnostics, the debug print statements should be removed before this PR is merged, or in a follow-up PR, to maintain code clarity. The addition of algo.stop() is a good permanent improvement for resource management.