All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.
- Add manifest metadata to bundle data (#1079) (#1062)
- Add profile command to REPL (#838)
- Add decision ID note in API docs (#1061)
- Fix formatting of trailing comments in composites (#1060)
- Fix panic caused by input being set incorrectly (#1083)
- Fix partial eval to apply saved terms (#1074)
- Add Stringer implementation for expr values
- Add Stringer implementation on metrics object
- Add helper function to compile strings
- Add note to configuration reference about -c flag
- Add support for configuration discovery
- Add support for multiple tracers
- Add trace helper to rego package
- Add code coverage percentage
- Fix REPL to check number of assignment operands
- Fix bug in test runner rule name dedup
- Fix security link in REST API reference
- Fix formatting of empty sets
- Fix incorrect reporting of module parse time
- Fix out of range errors for eq/assign in compiler
- Fix parser to limit size of exponents
- Update compiler to iterate over modules in sort order
- Update OPA front page
- Mark diagnostics feature as deprecated
- Add show debug command to REPL (#750)
- Add
glob
built-ins for easier path matching (thanks @aeneasr) - Add support for specifying services as object
-
Wasm compiler. This release adds initial/experimental support for compiling Rego policies into Wasm executables. Wasm executables can be loaded and executed in compatible Wasm runtimes like V8 (nodejs). You can try this out by running
opa build
. -
Data mocking. This release adds support for replacing/mocking the
data
document using thewith
keyword. In the past,with
only supported theinput
document. This made it tricky to test context-dependent policies. With the newwith
keyword support, it's easier to write tests against contextual policies. -
Negation Optimization. This release includes an optimization in partial evaluation for dealing with negated statements (
not
keyword). In the past, OPA would generate a support rule for negated statements. This is harder for clients to consume and not readily optimized. The optimization computes the necessary cross-product of the negated query and inlines it into the caller. This leads to simpler partial evaluation results that are readily optimized, translated into other query languages (e.g., SQL and Elasticsearch), or compiled into Wasm.
- Add builtin to verify and decode JWT (#884)
- Add GoDoc sample for using rego.Tracer (#1002)
- Add built-in function to get runtime info (#420)
- Add support for YAML encoded input values (#290)
- Add support for client certificates (#684)
- Add support for non-zero exit code in eval subcommand (#981)
- Fix == rewriting on embedded terms (#995)
- Fix copy propagation panic in comprehensions (#1012)
- Implement regex.find_n (#1001) (#747)
- Improve with modifier target error (#343)
- Iterate over smaller set when intersecting (#531)
- Only write one trailing newline at end of file (#1032)
- Redirect HTTP requests with trailing slashes (#972)
- Update bundle reader to allow relative data.json (#1019)
- Expose version information via REST API (#277)
- Add default decision configuration
- Add extra helpers to loader result
- Add indentation to trace in failure output
- Add router option to the HTTP server
- Add support for headers in http.send (thanks @repenno)
- Deprecating --insecure-addr flag (thanks @repenno)
- Add POST v1/query API for large inputs (thanks @rite2nikhil)
- Remove heap allocations from AST set with open addressing
- Replace siphash with xxhash in AST
- Output traces on failures in verbose mode (thanks @srenatus)
- Rewrite duplicate test rule names (thanks @srenatus)
- Add option to enable http redirects (#921)
- Add copy propagation to support rules (#911)
- Add support for inlining negated expressions in partial evaluation
- Add deps subcommand to analyze base and virtual document dependencies
- Add partial evaluation support to eval subcommand
- Add
net.cidr_overlap
built-in function (thanks @aeneasr) - Add
regex.template_match
built-in function (thanks @aeneasr) - Add external security audit information (thanks @caniszczyk)
- Add initial support for plugin loading (thanks @vrnmthr)
- Fix copy propagator type assertion panic (#912)
- Fix panic in parser error detail construction (#948)
- Fix with value rewriting for call terms (#916)
- Fix coverage flag for test command (thanks @johscheuer)
- Fix compile operation timing in REPL
- Fix to indent 4 spaces instead of a tab (thanks @superbrothers)
- Fix REPL output in policy guide (thanks @ttripp)
- Multiple fixes in the Kubernetes admission controller tutorial (thanks @johscheuer)
- Improve formatting of empty ast.Body (#909)
- Improve Kubernetes admission control policy loading explanation (thanks @rite2nihkil)
- Update http.send test to work without internet access (#945)
- Update test runner to set Fail to true (#954)
- Improve token authentication docs and handler (#901)
- Link to security docs in tutorials (#917)
- Update bundle reader to cap buffer size (#920)
- Validate queries by checking unsafe builtins (#919)
- Fix XSS in debug page (#918)
- Add io.jwt.verify_es256 and io.jwt.verify_ps256 built-in functions (@optnfast)
- Add array.concat built-in function (#851)
- Add support for command line bundle loading (#870)
- Add regex split built-in function
- Fix incorrect AST node in Index events (#859)
- Fix terraform tutorial type check errors (#888)
- Fix CONTRIBUTING.md to include sign-off step (@optnfast)
- Improve save set performance (#860)
This release adds two major features to OPA itself.
-
Query Profiler: the
opa eval
subcommand now supports a--profiler
option to help policy authors understand the performance profile of their policies. Give it a shot and let us know if you find it helpful or if you find cases that could be improved! -
Compile API: OPA now exposes Partial Evaluation with first-class interfaces. In prior releases, Partial Evaluation was only used for optimizations purposes. As of v0.9, callers can use Partial Evaluation via HTTP or Golang to obtain conditional decisions that can be evaluated on the client-side.
- Add ADOPTERS.md file (#691)
- Add time.weekday builtin (#789)
- Fix REPL output for multiple bool exprs (#850)
- Remove support rule if default value is not needed (#820)
Here is a short list of notable miscellaneous improvements.
- Add any/all built-in functions (thanks @vrnmthr)
- Add built-in function to parse Rego modules
- Add copy propagation optimization to partial evaluation output
- Add docs for exercising policies with test framework
- Add extra output formats to eval subcommand
- Add support for providing input to eval via stdin
- Improve parser error readability
- Improve rule index to support unknown values
- Rewrite == with = in compiler
- Update build to enable CGO
...along with 30+ other fixes and improvements.
- Fix virtual document cache invalidation (#736)
- Fix partial cache invalidation for data changes (#589)
- Fix query to path conversion in decision logger (#783)
- Fix handling of pointers to structs (#722, thanks @srenatus)
- Improve sprintf number handling (#748)
- Reduce memory overhead of decision logs (#705)
- Set bundle status in case of HTTP 304 (#794)
- Add docs on best practices around identity
- Add built-in function to verify JWTs signed with HS246 (thanks @hbouvier)
- Add built-in function to URL encode objects (thanks @vrnmthr)
- Add query parameters to authorization policy input (#786)
- Add support for listening on a UNIX domain socket (#692, thanks @JAORMX)
- Add trace event for rule index lookups (#716)
- Add support for multiple listeners in server (thanks @JAORMX)
- Remove decision log buffer size limit by default
- Update codebase with various go-fmt/ineffassign/mispell fixes (thanks @srenatus)
- Update REPL command to set unknowns
- Update subcommands to support loader filter (#782)
- Update evaluator to cache storage reads
- Update object to keep track of groundness
- Add Kafka authorization tutorial
- Add URL query encoding built-ins
- Add runtime API to register plugins
- Update eval subcommand to support multiple files or directories (thanks @devenney)
- Update Terraform tutorial for OPA v0.8
- Fix bug in topdown query ID generation
This release includes a few major features that improve OPA's management capabilities.
-
Bundles: OPA can be configured to download bundles of policy and data from remote HTTP servers. This allows administrators to configure OPA to pull down all of the policy and data required at the enforcement point. When OPA boots it will download the bundle and active it. OPA will periodically check in with the server to download new revisions of the bundle.
-
Status: OPA can be configured to report its status to remote HTTP servers. The status includes a description of the active bundle. This allows administrators to monitor the status of OPA in a central place.
-
Decision Logs: OPA can be configured to report decision logs to remote HTTP servers. This allows administrators to audit and debug decisions in a central place.
The command line file loading convention has been changed slightly. If you were
previously loading files with opa run *
you should use opa run .
now. OPA
will not namespace data under top-level directory names anymore. The problem
with the old approach was that data layout was dependent on the root directory
name. For example opa run /some/path1
and opa run /some/path2
would yield
different results even if both paths contained identical data.
Thanks to @jyoverna for adding a trace
built-in function that allows policy
authors to include notes in the trace. For example, authors can now embed
trace
calls in their policies. When OPA encounters a trace
call it will
include a "note" in the trace. Callers can filter the trace results to show only
notes. This helps diagnose incorrect decisions in large policies. For example:
package example
allow {
input.method = allows_methods[_]
trace(sprintf("input method is %v", [input.method]))
}
allowed_methods = ["GET", "HEAD"]
- Add RS256 JWT signature verification built-in function (#421)
- Add X.509 certificate parsing built-in function (#635)
- Fix substring built-in bounds checking (#465)
- Generate support rules for negated expressions (#623)
- Ignore some built-in calls during partial eval (#622)
- Plug comprehensions in partial eval results (#656)
- Report safety errors for generated vars (#661)
- Update partial eval to check call args recursively (#621)
- Add base64 encoding built-in functions
- Add JSON format to test and check subcommands
- Add coverage package and update test subcommand to report coverage
- Add eval subcommand to run queries from the command line (deprecates opa run --eval)
- Add parse subcommand to parse Rego modules and print AST
- Add reminder/reminder (%) operator
- Update rule index to support ==
- Update to Go 1.10
- Various fixes to fmt subcommand and format package
- Fix input and data loading to roundtrip values. Allows loading of []string, []int, etc.
As well as many other smaller improvements, refactoring, and fixes.
- Use rego.ParsedInput to provide input from form (#571)
- Add omitempty tag for ad-hoc query result field
- Fix rego package to check capture vars
- Fix root document assignment in REPL
- Update query compiler to deep copy parsed query
-
Nested expressions: now you can write expressions like
(temp_f - 32)*5/9
! -
Assignment/comparison operators: now you can write
x := <expression>
to declare local variables andx == y
when you strictly want to compare two values (and not bind any variables like with=
). -
Prometheus support: now you can hook up Prometheus to OPA and collect performance metrics on the different APIs. (thanks @rlguarino)
This release adds and improves a bunch of new built-in functions. See the Language Reference for details.
- Add globs_match built-in function (thanks @yashtewari)
- Add HTTP request built-in function
- Add time.clock and time.date built-in functions
- Add n-way set union and intersection built-in functions
- Improve walk built-in function performance for partially ground paths
- Fix REPL assignment support (#615)
- Fix panic due to nil term value (#601)
- Fix safety check bug for call args (#625)
- Update Kubernetes Admission Control tutorial (#567)
- Update release script to build for Windows (#573)
- Add support for DELETE method in Data API (#609) (thanks @repenno)
- Add basic query performance instrumentation
- Add documentation covering how OPA compares to other systems
- Remove use of unsafe.Pointer for string hashing
This release adds initial support for partial evaluation. Partial evaluation allows callers to mark certain inputs as unknown and then evaluate queries to produce new queries which can be evaluated once inputs become known.
- Add initial implementation of partial evaluation
- Add sort built-in function (#465)
- Add built-in function to check value types
- Fix rule arg type inferencing (#542)
- Fix documentation on "else" keyword (#475)
- Fix REPL to deduplicate auto-complete paths (#432
- Improve getting started example (#532)
- Improve handling of forbidden methods in HTTP server (#445)
- Refactor sets and objects for constant time lookup
- Fix eval of objects/sets containing vars (#505)
- Fix REPL printing of generated vars
- Fix bug in expression formatting
- Fix dynamic rewriting to copy with modifiers
- Fix off-by-one bug in array helper
- Fix index usage for virtual docs (#490)
- Fix match error panic (#494)
- Fix wildcard mangling in rule head (#480)
- Add parse_duration_ns to generate nanos based on duration string
- Add product to calculate the product of array or set
- Fix docs example of set union
- Fix file watch bug causing panic in server mode
- Modify AST to represent function names as refs
- Refactor runtime to separate init and start
- Refactor test runner to accept Store argument
- Substitute comprehension terms requring eval (#453)
- Add alpine-based Docker image
- Add stdin mode to opa fmt
- Fix syntax error in comprehension example
- Improve input parsing performance in V0 API
- Refactor loader to read inputs once (allows use of process substitution)
- Remove backup creation from fmt subcommand
- Remove use of sprintf in formatter
This release adds a new test
subcommand to OPA. The test
subcommand enables policy unit testing. The unit tests are expressed as rules containing assertions over test data. The test
subcommand provides a test runner that automatically discovers and executes these test rules. See opa test --help
for examples.
- Fix type error marshalling bug (#391)
- Fix type inference bug (#381)
- Fix unification bug (#436)
- Fix type inferecen bug for partial objects with non-string keys (#440)
- Suppress match errors if closures contained errors (#438)
As part of this release, logrus was revendored to deal with the naming issue. If you use logrus, or one of your other dependencies does (such as Docker), be sure to check out sirupsen/logrus#570 (comment).
- Fix incorrect REPL interpretation of some exprs (#433)
- Fix inaccurate location information in some parser errors (#214)
- Add Terraform Testing tutorial to documentation
- Add shorthand for defining partial documents (e.g.,
p[1]
instead ofp[1] { true }
) - Add walk built-in function to recursively process nested documents
- Refactor Policy API response representations based on usage
This release adds Diagnostics support to the server. This greatly improves OPA's debuggability when deployed as a daemon.
- Fix data race in the parser extensions
- Fix index.html GET requests returning error on empty input
- Fix race condition in watch test
- Fix image version in HTTP API tutorial
- Add metrics command to the REPL
- Limit length of pretty printed values in the REPL
- Simplify input query parameters in data GET requests
- Update server to support pretty explanations
- Properly remove temporary files when running
opa fmt -d
- Add support for refs with composite operands (e.g,.
p[[x,y]]
)
- Add support for raw strings (#265)
- Add support to cancel compilation after some number of errors (#249)
- Add Kubernetes admission control tutorial
- Add tracing support to rego package
- Add watch package for watching changes to queries
- Add dependencies package to perform dependency analysis on ASTs
- Fix mobile view navigation bug
- Fix panic in compiler from concurrent map writes (#379
- Fix ambiguous syntax around body and set comprehensions (#377)
- Add support for set and object comprehensions
- Add support for system.main policy in server
- Add transaction support in rego package
- Improve type checking error messages
- Format REPL modules before printing them
- Correct
opa fmt
panic on missing files - Fix minor site issues
- Add rego examples with input and compiler
- Add support for query cancellation
OPA now supports user-defined functions that have the same semantics as built-in functions. This allows policy authors to quickly define reusable pieces of logic in Rego without overloading the input document or thinking about variable safety.
The storage layer has been improved to support single-writer/multiple-reader concurrency. The storage interfaces have been simplified in the process. Users can rely on https://godoc.org/github.com/open-policy-agent/opa/storage/inmem in place of the old storage package.
The website has been redesigned and the documentation has been ported over to GitBook.
OPA supports two new commands that check and format policies. Check out opa help
for more information.
- Add YAML serialization built-ins
- Add time built-ins
- Fixed incorrect source locations on refs and manually constructed terms. All term locations should be set correctly now.
- Fixed evaluation bug that caused partial sets and partial objects to be undefined in some cases.
This release includes a bunch of new built-in functions to help with string manipulation, JWTs, and more.
The JSON marshalling built-ins have been renamed. Policies that used json_unmarshal
and json_marshal
before will need to be updated to use json.marshal
and json.unmarshal
respectively.
- Add
else
keyword - Improved undefined built-in error message
- Fixed error message in
-
built-in - Fixed exit instructions in REPL tutorial
- Relax safety check for built-in outputs
This release includes a bunch of cool stuff!
- Basic type checking for queries and virtual docs (#312)
- Optimizations for HTTP API authorization policies (#319)
- New /v0 API to support webhook integrations (docs)
- Add support for namespaced built-ins (#314)
- Improve logging to include request/response bodies (#328)
- Add basic performance metrics (#320)
- Add built-ins to un/marshal JSON
- Add input form to diagnostic page
- Fix top-level navigation links
- Improve file loader error handling
- Fix recursive binding by short-circuiting ref eval (#298)
- Fix reordering for unsafe ref heads (#297)
- Fix rewriting of single term exprs (#299)
This release changes the run
command options:
- Removed glog in favour of Sirupsen/logrus. This means the command line arguments to control logging have changed. See
run --help
for details. - Removed
--policy-dir
option. For now, if policy persistence is required, users can treat policies as config files and manage them outside of OPA. Once OPA supports persistence of data (e.g., with file-based storage) then policy persistence will be added back in.
- Add support for additional HTTP listener (#289)
- Allow slash in policy id/path (#292)
- Improve request logging (#281)
- Add deployment documentation
- Remove erroneous flag.Parse() call
- Remove persist/--policy-dir option
- Replace glog with logrus
Also, updated Docker tagging so that latest points to most recent release (instead of most recent development build). The most recent development build can still be obtained with the {version}-dev tag.
This release adds support for TLS, token-based authentication, and authorization in the OPA APIs!
For details on how to secure the OPA API, go to http://openpolicyagent.org/documentation/references/security.
- Fix stray built-in error messages (#275)
- Update error codes and messages throughout (#237)
- Fix evaluation bug with nested value refs
- Fix rego.Eval to close transactions
- Fix buggy usage of errors.Cause
- Updated to support Go 1.8
- Fix issue in high-level Go API (#261)
- Fix parsing of inline comments (#258)
- Fix unset of input/data in REPL (#259)
- Handle non-string/var values in rule tree lookup (#257)
This release changes the Rego syntax. The two main changes are:
- Expressions are now separated by semicolons (instead of commas). When writing rules, the semicolons are optional.
- Rules are no longer written in the form:
head :- body
. Instead they are written ashead { body }
.
Also:
- Set, array, and object literals now support trailing commas.
- To declare a set literal with one element, you must include a trailing comma, e.g.,
{ foo, }
. - Arithmetic and set operations can now be performed with infix notation, e.g.,
x = 2 + 1
. - Sets can be referred to like objects and arrays now (#243). E.g.,
p[_].foo = 1 # check if element in has attr foo equal to 1
.
This release changes the evaluation behaviour for packages. Previously, if a package contained no rules OR all of the rules were undefined when evaluated, a query against the package path would return undefined. As of v0.4.2 the query will return an empty object.
This release changes the Data API to return an HTTP 200 response if the document is undefined. The message body will contain an object without the result
property.
- Allow sets to be treated like objects/arrays
- Added high level API for Go users. See
github.com/open-policy-agent/opa/rego
package. - Improved expression String() function to handle infix operators better.
- Added support for set intersection and union built-ins. See language docs.
For more details on these changes see sections in How Do I Write Policies.
- Added new default keyword. The default keyword can be used to provide a default value for rules with complete definitions.
- Added new with keyword. The with keyword can be used to programmatically set the value of the input document inside policies.
- Add basic REST API authorization benchmark
This release contains a few non-backwards compatible changes to the REST API:
-
The
request
document has been renamed toinput
. If you were calling the GET /data[/path]?request=value you should update to use POST requests (see below). -
The API responses have been updated to return results embedded inside a wrapper object:
{"result": value}
. This will allow OPA to return unambiguous metadata in future (e.g., pagination, analysis, etc.) If you were previously consuming Data API GET responses, you should update your code to access the value under the"result"
key of the response object. -
The API models have been updated to use snake_case (#222). This would only affect you if you were previously consuming error responses or policy ASTs.
The Data API has been updated to support the POST requests. This is the recommended way of supplying query inputs.
The built-in framework has been extended to support simplified built-in implementations:
- Refactor topdown built-in functions (#205)
- Add cURL note to REST API docs (#211)
- Fix empty request parameter parsing (#212)
- Fix handling of missing input document (#227)
- Improve floating point literal support (#215)
- Improve module parsing errors (#213)
- Fix ast.Number hash and equality
- Fix parsing of escaped strings
- Improve evaluation error messages
- Fixed unsafe vars with built-in operator names bug (#206)
- Fixed body to rule conversion bug (#202)
- Improved request parameter handling (#201)
- Improved release infrastructure
The last major/minor release of 2016! Woohoo! This release contains a few non-backwards compatible changes to the APIs.
These changes simplify and clean up the storage.Store interface. This should make it easier to implement custom stores in the future.
- Update storage to support context.Context (#155)
- Update underlying number representation (#154)
- Updates to use new storage.Path type (#159)
These changes update the language to align query arguments with state stored in OPA. With these changes, OPA can readily analyze policies and determine references that refer to state stored in OPA versus query arguments versus local variables.
These changes also update how query arguments are provided via the REST API.
- Updates to how query arguments are handled #197
- topdown.Context has been renamed to topdown.Topdown to avoid confusion with Golang's context.
- Add help topics to REPL (#172)
- Fix error handling bug in Query API (#183)
- Fix handling of prefixed paths with -w flag (#193)
- Improve exit handling in REPL (#175)
- Update parser support for = rules (#192)
- Add Visual Studio and Atom plugins
- Add lazy loading of modules during compilation
- Fix bug in serialization of empty objects/arrays
- Add YAML loading and refactor into separate file (#135)
- Add command line flag to eval, print, and exit (#152)
- Add compiler check for consistent rule types (#147)
- Add set_diff built-in (#133)
- Add simple 'show' command to print current module (#108)
- Added examples to 'help' output in REPL (#151)
- Check package declarations for conflicts (#137)
- Deep copy modules in compiler (#158)
- Fix evaluation of refs to set literals (#149)
- Fix indexing usage for refs with intersecting vars (#153)
- Fix output for references iterating sets (#148)
- Fix parser handling of keywords in variable names (#178)
- Improve file loading support (#163)
- Remove conflict error for same key/value pairs (#165)
- Support "data" query in REPL (#150)
- Add new compiler harness for ad-hoc queries
- Add tab completion of imports
- Added support for non-ground global values
- Added several new string manipulation built-ins
- Added TextMate grammar file
- Setup Docker image build and push to Docker Hub as part of CI
- Set literals
- Composite and reference values in Rule head
- Complete Rule definitions containing variables
- Builtins for regular expressions, string concatenation, casting to numbers
- Improved error reporting in parser and compiler
- Iteration over base and virtual documents (in the same reference)
- Query tracing and explanation support
- Pluggable data storage support
- GoDoc strings with examples
- REST API specification
- Concise language reference
- Per-query cache of virtual documents in topdown
And many other small improvements and fixes.
- Basic value types: null, boolean, number, string, object, and array
- Reference and variables types
- Array comprehensions
- Built-in functions for basic arithmetic and aggregation
- Incremental and complete rule definitions
- Negation and disjunction
- Module system
- Reference resolver to support packages
- Safety check to prevent recursive rules
- Safety checks to ensure successful evaluation will bind all variables in head and body of rules
- Initial top down query evaluation algorithm
- Basic in-memory storage that exposes JSON Patch style API
- REPL that can be run to experiment with ad-hoc queries
- Server mode supports HTTP APIs to manage policies, push and query documents, and execute ad-hoc queries.
- Basic build infrastructure to produce cross-platform builds, run style/lint/format checks, execute tests, static HTML site
- Introductions to policy, policy-enabling, and how OPA works
- Language reference that serves as guide for new users
- Tutorial that introduces users to the REPL
- Tutorial that introduces users to policy-enabling with a Docker Authorization plugin