[Offload] Improve liboffload documentation

[callumfare]

• Update the main README to reflect the current project status
• Rework the main API generation documentation. General fixes/tidying, but also spell out explicitly how to make API changes at the top of the document since this is what most people will care about.

[llvmbot]

@llvm/pr-subscribers-offload

Author: Callum Fare (callumfare)

Changes

• Update the main README to reflect the current project status
• Rework the main API generation documentation. General fixes/tidying, but also spell out explicitly how to make API changes at the top of the document since this is what most people will care about.

---

Full diff: https://github.com/llvm/llvm-project/pull/142403.diff

4 Files Affected:

• (modified) offload/include/Shared/OffloadErrcodes.inc (+1-2)
• (modified) offload/liboffload/API/README.md (+55-37)
• (modified) offload/liboffload/README.md (+32-6)
• (modified) offload/tools/offload-tblgen/MiscGen.cpp (+1-1)

diff --git a/offload/include/Shared/OffloadErrcodes.inc b/offload/include/Shared/OffloadErrcodes.inc
index 130e553aa70a4..f89429a0384c9 100644
--- a/offload/include/Shared/OffloadErrcodes.inc
+++ b/offload/include/Shared/OffloadErrcodes.inc
@@ -11,8 +11,7 @@
 #endif
 
 // Error codes are shared between PluginInterface and liboffload.
-// To add new error codes, add them to offload/liboffload/API/Common.td and run
-// the GenerateOffload target.
+// To add new error codes, add them to offload/liboffload/API/Common.td.
 
 OFFLOAD_ERRC(SUCCESS, "success", 0)
 OFFLOAD_ERRC(UNKNOWN, "unknown or internal error", 1)
diff --git a/offload/liboffload/API/README.md b/offload/liboffload/API/README.md
index fda1ad39fa937..8f34a91e12e2f 100644
--- a/offload/liboffload/API/README.md
+++ b/offload/liboffload/API/README.md
@@ -1,8 +1,5 @@
 # Offload API definitions
 
-**Note**: This is a work-in-progress. It is loosely based on equivalent
-tooling in Unified Runtime.
-
 The Tablegen files in this directory are used to define the Offload API. They
 are used with the `offload-tblgen` tool to generate API headers, print headers,
 and other implementation details.
@@ -10,15 +7,42 @@ and other implementation details.
 The root file is `OffloadAPI.td` - additional `.td` files can be included in
 this file to add them to the API.
 
+## Modifying the API
+API modifications, including additions, can be made by modifying the existing
+`.td` files. It is also possible to add a new tablegen file to the API by adding
+it to the includes in `OffloadAPI.td`. When Offload is rebuilt the new
+definition will be included in the generated files.
+
+Most API changes and additions do not require any additional work beyond this,
+other than new functions which are described below.
+
+### Adding a new function to the API
+
+When a new function  is added (e.g. `offloadDeviceFoo`), the actual entry
+point is automatically generated, which contains validation and tracing code.
+It expects an implementation function (`offloadDeviceFoo_impl`) to be defined,
+which it will call into. The definition of this implementation function should
+be added to `liboffload/src/OffloadImpl.cpp`
+
+In short, the steps to add a new function are:
+* Add the new function definition to the `.td` files.
+* Build the `LLVMOffload` target. The relevant files will be regenerated, but
+  the library will fail to link because it is missing the implementation
+  function.
+* Add the new implementation function to `liboffload/src/OffloadImpl.cpp`. You
+  can copy the new function declaration from the generated
+  `OffloadImplFuncDecls.inc` file.
+* Rebuild `LLVMOffload`
+
 ## API Objects
 The API consists of a number of objects, which always have a *name* field and
 *description* field, and are one of the following types:
 
 ### Function
 Represents an API entry point function. Has a list of returns and parameters.
-Also has fields for details (representing a bullet-point list of
-information about the function that would otherwise be too detailed for the
-description), and analogues (equivalent functions in other APIs).
+Also has fields for details (representing a bullet-point list of information
+about the function that would otherwise be too detailed for the description),
+and analogues (equivalent functions in other APIs).
 
 #### Parameter
 Represents a parameter to a function, has *type*, *name*, and *desc* fields.
@@ -30,17 +54,23 @@ A *handle* type is a pointer to an opaque struct, used to abstract over
 plugin-specific implementation details.
 
 There are two special variants of a *parameter*:
-* **RangedParameter** - Represents a parameter that has a range described by other parameters. Generally these are pointers to an arbitrary number of objects. The range is used for generating validation and printing code. E.g, a range might be between `(0, NumDevices)`
-* **TypeTaggedParameter** - Represents a parameter (usually of `void*` type) that has the type and size of its pointee data described by other function parameters. The type is usually described by a type-tagged enum. This allows functions (e.g. `olGetDeviceInfo`) to return data of an arbitrary type.
+* **RangedParameter** - Represents a parameter that has a range described by
+  other parameters. Generally these are pointers to an arbitrary number of
+  objects. The range is used for generating validation and printing code. E.g,
+  a range might be between `(0, NumDevices)`
+* **TypeTaggedParameter** - Represents a parameter (usually of `void*` type)
+  that has the type and size of its pointee data described by other function
+  parameters. The type is usually described by a type-tagged enum. This allows
+  functions (e.g. `olGetDeviceInfo`) to return data of an arbitrary type.
 
 #### Return
 A return represents a possible return code from the function, and optionally a
 list of conditions in which this value may be returned. The conditions list is
-not expected to be exhaustive. A condition is considered free-form text, but
-if it is wrapped in \`backticks\` then it is treated as literal code
-representing an error condition (e.g. `someParam < 1`). These conditions are
-used to automatically create validation checks by the `offload-tblgen`
-validation generator.
+not expected to be exhaustive. A condition is considered free-form text, but if
+it is wrapped in \`backticks\` then it is treated as literal code representing
+an error condition (e.g. `someParam < 1`). These conditions are used to
+automatically create validation checks by the `offload-tblgen` validation
+generator.
 
 Returns are automatically generated for functions with pointer or handle
 parameters, so API authors do not need to exhaustively add null checks for
@@ -62,9 +92,9 @@ which preserves ABI compatibility with C.
 Represents a C-style enum. Contains a list of `etor` values, which have a name
 and description.
 
-A `TaggedEtor` record type also exists which additionally takes a type. This type
-is used when the enum is used as a parameter to a function with a type-tagged
-function parameter (e.g. `olGetDeviceInfo`).
+A `TaggedEtor` record type also exists which additionally takes a type. This
+type is used when the enum is used as a parameter to a function with a
+type-tagged function parameter (e.g. `olGetDeviceInfo`).
 
 All enums automatically get a `<enum_name>_FORCE_UINT32 = 0x7fffffff` value,
 which forces the underlying type to be uint32.
@@ -96,7 +126,10 @@ files, rather than requiring a mix of C source and tablegen.
 The comments in the generated header are in Doxygen format, although
 generating documentation from them hasn't been implemented yet.
 
-The entirety of this header is generated by Tablegen, rather than having a predefined header file that includes one or more `.inc` files. This is because this header is expected to be part of the installation and distributed to end-users, so should be self-contained.
+The entirety of this header is generated by Tablegen, rather than having a
+predefined header file that includes one or more `.inc` files. This is because
+this header is expected to be part of the installation and distributed to
+end-users, so should be self-contained.
 
 ### Entry Points
 ```
@@ -121,8 +154,8 @@ in the API, e.g. `offloadDeviceFoo_impl` for `offloadDeviceFoo`.
 ```
 ./offload-tblgen -I <path-to-llvm>/offload/API  <path-to-llvm>/offload/API/OffloadAPI.td --gen-print-header
 ```
-This header contains `std::ostream &operator<<(std::ostream&)` definitions for
-various API objects, including function parameters.
+This header contains `llvm::raw_ostream &operator<<(llvm::raw_ostream &)`
+definitions for various API objects, including function parameters.
 
 As with the API header, it is expected that this header is part of the installed
 package, so it is entirely generated by Tablegen.
@@ -130,21 +163,6 @@ package, so it is entirely generated by Tablegen.
 For ease of implementation, and since it is not strictly part of the API, this
 is a C++ header file. If a C version is desirable it could be added.
 
-### Future Tablegen backends
-`RecordTypes.hpp` contains wrappers for all of the API object types, which will
-allow more backends to be easily added in future.
-
-## Adding to the API
-
-A new object can be added to the API by adding to one of the existing `.td`
-files. It is also possible to add a new tablegen file to the API by adding it
-to the includes in `OffloadAPI.td`. When the `OffloadGenerate` target is
-rebuilt, the new definition will be included in the generated files.
-
-### Adding a new entry point
-
-When a new entry point is added (e.g. `offloadDeviceFoo`), the actual entry
-point is automatically generated, which contains validation and tracing code.
-It expects an implementation function (`offloadDeviceFoo_impl`) to be defined,
-which it will call into. The definition of this implementation function should
-be added to `src/OffloadImpl.cpp`
+### Additional Tablegen backends
+`RecordTypes.hpp` contains wrappers for all of the API object types, which
+allows new backends to be easily added if needed.
diff --git a/offload/liboffload/README.md b/offload/liboffload/README.md
index 95c9bf54d7bad..267d27172828c 100644
--- a/offload/liboffload/README.md
+++ b/offload/liboffload/README.md
@@ -1,8 +1,34 @@
-# Offload New API
+# Liboffload
 
-This directory contains the implementation of the experimental work-in-progress
-new API for Offload. It builds on top of the existing plugin implementations but
-provides a single level of abstraction suitable for runtimes for languages other
-than OpenMP to be built on top of.
+This directory contains the implementation of the work-in-progress new API for
+Offload. It builds on top of the existing plugin implementations but provides a
+single level of abstraction suitable for implementation of many offloading
+language runtimes, rather than just OpenMP.
 
-See the [API definition readme](API/README.md) for implementation details.
\ No newline at end of file
+## Testing liboffload
+
+The main test suite for liboffload can be run with the `check-offload-unit`
+target, which runs the `offload.unittests` executable. The test suite will
+automatically run on every available device, but can be restricted to a single
+platform (CUDA, AMDGPU) with a command line argument:
+
+```sh
+$ ./offload.unittests --platform=CUDA
+```
+
+Tracing of Offload API calls can be enabled by setting the `OFFLOAD_TRACE`
+environment variable. This works with any program that uses liboffload.
+
+```sh
+$ OFFLOAD_TRACE=1 ./offload.unittests
+---> olInit()-> OL_SUCCESS
+# etc
+```
+
+The host plugin is not currently supported.
+
+## Modifying liboffload
+
+The main header (`OffloadAPI.h`) and some implementation details are
+autogenerated with tablegen. See the [API definition README](API/README.md)
+for implementation details.
diff --git a/offload/tools/offload-tblgen/MiscGen.cpp b/offload/tools/offload-tblgen/MiscGen.cpp
index bc3d204f78d89..14787729503e9 100644
--- a/offload/tools/offload-tblgen/MiscGen.cpp
+++ b/offload/tools/offload-tblgen/MiscGen.cpp
@@ -82,7 +82,7 @@ void EmitOffloadErrcodes(const RecordKeeper &Records, raw_ostream &OS) {
 #endif
 
 // Error codes are shared between PluginInterface and liboffload.
-// To add new error codes, add them to offload/liboffload/API/Common.td and run the GenerateOffload target.
+// To add new error codes, add them to offload/liboffload/API/Common.td.
 
 )";