The OPA Website and Documentation

The content and tooling is separated into a few places:

devel/ - Developer documentation for OPA (not part of the website)

website/ - This directory contains all of the Markdown, HTML, Sass/CSS, and other assets needed to build the openpolicyagent.org website. See the section below for steps to build the site and test documentation changes locally. This content is not versioned for each release, it is common scaffolding for the website.

content/ - The raw OPA documentation can be found under the directory. This content is versioned for each release and should have all images and code snippets alongside the markdown content files.

content/integrations/ - Source for the OPA Ecosystem page. See OPA Ecosystem below for more details.

Website Components

The website (openpolicyagent.org) and doc's hosted there (openpolicyagent.org/docs) have a few components involved with the buildings and hosting.

The static site is generated with Hugo which uses the markdown in content/ as its content for pages under /docs/*. There is a script to generate the previous supported versions, automated via make generate, and the latest (current working tree) documentations is under /docs/edge/*.

The static site content is then hosted by Netlify. To support backwards compatible URLs (from pre-netlify days) and to have the latest version of the docs URLs work openpolicyagent.org/docs/latest the site relies on Netlify URL redirects and rewrites which are defined in website/layouts/index.redirects and are build into a _redirects file when the Hugo build happens via make production-build or make preview-build.

Site updates

The OPA site is automatically published using Netlify. Whenever changes in this directory are pushed to main, the site will be re-built and re-deployed.

Note: The site is built for many versions of the docs, this introduces some complexities to be aware of when making changes to the site's layout:

• Updates to the site's templates or styles/ are applied to all versions (edge, latest and all versions) when merged to main.
• Site data treated in the same way, so updates to data files also apply to all versions as soon as they are merged.
• Docs content/, when merged to main, is only shown on edge until the next release.
• Other, unversioned content/ is shown immediately after merging. This includes pages in the OPA Ecosystem as well as Security, Support, and Community pages.

How to Edit and Test

Preview Markdown content (*.md)

The majority of this can be done with any markdown renderer (typically built into or via a plug-in for IDEs and editors). The rendered output will be very similar to what Hugo will generate.

This excludes the Hugo shortcodes (places with {{< SHORT_CODE >}} in the markdown. To see the output of these you'll need to involve Hugo. Additionally, to validate and view live code blocks, a full site preview is required.

Full Site Preview

To build and serve the site install the following dependencies on your system:

• The Hugo static site generator (See details below)
• NodeJS (and NPM)
• The Netlify dev CLI

Installing Hugo

Running the website locally requires installing the Hugo static site generator. The required version of Hugo is listed in the netlify.toml configuration file (see the HUGO_VERSION variable).

Installation instructions for Hugo can be found in the official documentation.

Please note that you need to install the "extended" version of Hugo (with built-in support) to run the site locally. If you get errors like this, it means that you're using the non-extended version:

error: failed to transform resource: TOCSS: failed to transform "sass/style.sass" (text/x-sass): this feature is not available in your current Hugo version

Please also note that the current version of Hugo (e.g. installed with brew install hugo) is not compatible with the docs. If you get errors like this, it means that you're using the wrong version:

ERROR ... execute of template failed: template: partials/docs/sidenav.html:11:18: executing "partials/docs/sidenav.html" at <.URL>: can't evaluate field URL in type *hugolib.pageState

Remote Preview on Netlify

This option provides the best preview of the site content, using the exact same infrastructure as the production website.

1. Go to Netlify and create an account/log-in.

2. (Optional) Create a new "site" in Netlify, linking to your fork of OPA on GitHub. This step makes for an easy way to see the "real" site deployed from branches you are working on, but is not required for dev previews.

3. Log in with the netlify CLI tool https://docs.netlify.com/cli/get-started/#authentication

4. Deploy the site on Netlify using local content via the make docs-serve-remote. Follow any prompts the netlify tool asks. If you have not already linked the site select Create & configure a new site and specify your personal user account (which should have been configured in the previous step).

5. Netlify will then upload the built content and serve it via their CDN. A URL to the preview will be given in the CLI console output. Be sure to select the edge version within the Documentation pages to see your reflected changes in the remote site.

Local Preview via netlify dev

Similar to the "remote" option this will run a full preview of the website, however all Netlify components are run locally using Netlify Dev. This will load a preview significantly faster than deploying remotely as the content does not need to be uploaded to the Netlify CDN first.

1. Run make docs-serve-local target. A URL to the preview will be given in the CLI console output.

WARNING: This option will render the content and works as a good option for local development. However, there are some differences between what would be run locally and what gets deployed in "prod" on Netlify. This depends on the version of the CLI tool and how in-sync that is with the actual Netlify infrastructure. A common issue is with the redirects not working as expected locally, but working correctly on the "live" site in production. When in doubt use the "remote" Netlify preview to verify behavior.

Checking links

To check the site's links, first start the full site preview locally (see Serving the full site instructions))

Then run:

docker run --rm -it --net=host linkchecker/linkchecker $URL

Note: You may need to adjust the URL (host and/or port) depending on the environment. For OSX and Windows the host might need to be host.docker.internal instead of localhost.

This link checker will work best with Netlify previews! Just point it at the preview URL instead of the local server. The "pretty url" feature seems to work best when deployed, running locally may result in erroneous links.

Live Code Blocks

Live blocks enable readers to interact with and change Rego snippets from within the docs. They are written as standard markdown code blocks with a special syntax in the language label and are enabled used a postprocessing step during full site builds.

For them to render correctly you must build the site using make serve or Netlify. If the live blocks build starts failing, try running make live-blocks-clear-caches from the docs/ folder. For the page to be able to live update code blocks, you may also want to disable your browser's CORS checks when developing. For chrome you can use the command-line flags --disable-web-security --user-data-dir=/tmp/$RANDOM; this will open a new window that isn't tied to your existing user data.

Here's what they look like:

In this module:

```live:rule_body:module
package example

import rego.v1

u if {
	"foo" == "foo"
}
```

The rule `u` evaluates to true:

```live:rule_body:output
```

Groups

Each live block group within a page has a unique name, for example rule_body above. Each group can be composed of up to 1 of each type of live block:

• module - A complete or partial rego module.
• query - Rego expressions for which to show output instead of the module.
• input - JSON input with which to evaluate the module/ query.
• output - A block to contain the output for the group, contents will be inserted automatically.

Groups can also be structured hierarchically using slashes in their names. When evaluated, the module blocks will be concatenated and any other missing blocks will be inherited from ancestors. Here's what a more complex set of blocks could look like:

```live:eg:module:hidden
package example

import rego.v1
```

We can define a scalar rule:

```live:eg/string:module
string := "hello"
```

Which generates the document you'd expect:

```live:eg/string:output
```

We can then define a rule that uses the value of `string`:

```live:eg/string/rule:module
r if { input.value == string }
```

And query it with some input:

```live:eg/string/rule:input
{
  "value": "world"
}
```
```live:eg/string/rule:query:hidden
r
```

with which it will be undefined:

```live:eg/string/rule:output:expect_undefined
```

In that example, the output for eg/string is evaluated with only the module:

package example

string := "hello"

Whereas the eg/string/rule output is evaluated with the module:

package example

import rego.v1

string := "hello"

r if input.value == string

as well the given query and input.

If any of the blocks that impact the output are edited by a reader, the output will update accordingly

Tags

You'll notice in the previous example that some blocks have an additional section after the type, these are tags.

To apply multiple tags to a block, separate them with commas. E.g. live:group_name:block_type:tag1,tag2,tag2

Some tags can be applied to any block:

• hidden - Hide the code block.
• read_only - Prevent editing of the block.
• merge_down - Visually merge this code block with the one below it (remove bottom margin).
• openable - Add a button to the block that opens its group in the Rego Playground. This should typically only be used on complete module blocks.
• line_numbers - Show line numbers in the block. This should be used sparingly; the code will visually shift when they appear.

Outputs can also be tagged as expecting various errors. If one is tagged as expecting errors that do not occur or errors occur that it is not tagged as expecting, the build will fail noisily.

More specific errors appear before less specific ones in this list, try to use them. If you think a more specific tag could be added for your case, please create an issue.

• expect_undefined - The query result is undefined. If all the rules in a module are undefined, the output is simply {}.
• expect_assigned_above - A variable is already := assigned.
• expect_referenced_above - A variable is used before it is := assigned.
• expect_compile_error
• expect_rego_type_error - A compile-time type error.
• expect_unsafe_var
• expect_recursion
• expect_rego_error - Any parse/compile-time error.
• expect_conflict
• expect_eval_type_error - An evaluation-time type error.
• expect_builtin_error
• expect_with_merge_error
• expect_eval_error - Any evaluation-time error.
• expect_error - Any OPA error. This should generally not be used.

At this time, due to the way OPA loads modules on the CLI, expecting parse errors is not possible.

Finally, outputs can also be tagged one or more times with include(<group name>), which will include another group's module when evaluating (e.g. so that they can be imported).

If a query isn't specified for the output's group, when other modules are included the default becomes data instead of data.<package name>.

OPA Ecosystem

The OPA Ecosystem makes it easy to find either a specific integration with OPA or to browse the integrations with OPA within a particular category. It pulls information about different integrations (e.g. blogs, videos, tutorials, code) into a single place while allowing integration authors to update the docs content as needed.

Schema

Source information for the OPA Ecosystem is stored in the following places:

• content/integrations/ - each file creates a page in the OPA Ecosystem for a particular integration.
• content/organizations/ - each file is a page for organizations and companies associated with integrations.
• content/softwares/ - each file is for software categories related to integrations.

Integrations should have a file in /docs/website/content/integrations/ with the following schema:

---
title: <integration name>
software:
- <related software>
- <related software>
inventors:
- <inventor name>
- <inventor name>
tutorials: # optional, links to tutorials for the integration
- https://example.com/tutorial
code: # optional, links to code for the integration
- https://github.com/...
blogs: # optional, links to blog posts for the integration
- https://example.com/blog/1
videos: # optional, links to videos for the integration
- title: <video title>
  speakers:
  - name: <speaker name>
    organization: <speaker organization>
    venue: <venue>
    link: <link>
---
Description of the integration (required)

Any inventor that is not already in content/organizations/ will need to be added too in content/organizations/. Organizations have the following format:

---
link: https://example.com
title: <organization name>
---

Any software that is not already in content/softwares/ will need to be added too in content/softwares/. Software categories have the following format:

---
link: https://example.com
title: <software name>
---

Logos

For each file in under content/integrations/ a png or svg logo with the same name must be placed in ./website/static/img/logos/integrations.

For example:

# content/integrations/my-cool-integration.md

Would need a file called my-cool-integration.(png|svg) at ./website/static/img/logos/integrations/my-cool-integration.(png|svg).

Google Analytics

With the addition of the feedback button, we are now able to see how many users found a particular page of the docs useful.

To view the metrics you will need access to Google Analytics.

Feedback responses can be found in the right hand tree under Behavior -> Events -> Top Events.

From here you can set the desired time frame you wish to monitor. Then drill down into the helpful category to see the specific pages and how many clicks they received.