[doc][rllib] the rest of missing api references + lint checker (ray-p…

…roject#47114)

Add the rest of missing API references for rllib. We can also now enable
the API policy lint checker for rllib, now that all missing references
are documented

Test:
- CI

<img width="1351" alt="Screenshot 2024-08-13 at 12 15 08 PM"
src="https://github.com/user-attachments/assets/cc1d1c8e-763e-4d2e-a7d1-28243a7fdbab">

Signed-off-by: can <can@anyscale.com>

.buildkite/lint.rayci.yml

@@ -35,6 +35,7 @@ steps:
       - api_policy_check core
       - api_policy_check serve
       - api_policy_check data
+      - api_policy_check rllib
 
   - label: ":lint-roller: lint: linkcheck"
     instance_type: medium

ci/ray_ci/doc/cmd_check_api_discrepancy.py

@@ -42,6 +42,11 @@
             "ray.remote_function.RemoteFunction",
         },
     },
+    "rllib": {
+        "head_modules": {"ray.rllib"},
+        "head_doc_file": "doc/source/rllib/package_ref/index.rst",
+        "white_list_apis": {},
+    },
 }
 
 

doc/source/rllib/package_ref/policy.rst

@@ -26,8 +26,7 @@ which maps agent IDs to a policy ID.
     by sub-classing either of the available, built-in classes, depending on your
     needs.
 
-.. include::
-    policy/custom_policies.rst
+.. include:: policy/custom_policies.rst
 
 .. currentmodule:: ray.rllib
 

doc/source/rllib/package_ref/utils.rst

@@ -248,10 +248,15 @@ Policy utilities
 Other utilities
 ~~~~~~~~~~~~~~~
 
-.. currentmodule:: ray.rllib.utils
+.. currentmodule:: ray.rllib
 
 .. autosummary::
    :nosignatures:
    :toctree: doc/
 
-   tensor_dtype.get_np_dtype
+   utils.tensor_dtype.get_np_dtype
+   common.CLIArguments
+   common.FrameworkEnum
+   common.SupportedFileType
+   core.rl_module.validate_module_id
+   train.load_experiments_from_file