Continue with compilation errors

[simar7]

Does OPA support a way to "offload" or "unload" failed modules when a compilation process fails.

The idea is something along the lines:

1. Call compiler.Compile(policies) where
2. Check compiler.Failed(), if true, check if the number of errors is above the ast.CompileErrorLimitDefault limit.
3. If greater, fail altogether.
4. If less, retry the compilation without the modules that failed.

The use case for this is that today if we load a number of policies at once, even if one of them fails (user error, typo, etc.) the entire evaluation fails. The idea is to be able to run the good set of policies and maybe keep all the errors on a log to be processed later so that a single bad/busted policy doesn't fail the entire compilation/evaluation process.

I've seen the implementation/comments around the ast codebase here https://github.com/open-policy-agent/opa/blob/main/ast/compile.go#L36-L39 – and today it seems that if compilation fails, there are no guarantees on the state of the modules.

[simar7]

The reason I wonder if there's support for this is because I see there's a constant for error limit https://github.com/open-policy-agent/opa/blob/main/ast/compile.go#L22-L24 - I was hoping to use this to see if the number of errors is under a certain limit (e.g. the default, 10) then remove the error'ed out modules and continue with the rest of the evaluation phase.

[simar7]

One way to tackle this would be to just manually remove the policies that error'd out and retry the compilation. But I was wondering if something of such sorts already exists within the compiler itself.

[johanfylling]

This is not a feature supported by the OPA compiler today.
One alternative solution for you might be to use the REST API to upload/update modules incrementally. If a pushed module fail to compile, OPA will keep its previous state, and evaluate incoming queries using the previously successful modules.

[simar7]

Unfortunately the API solution does not work in our case as we use OPA as a library within a self contained CLI tool. Any issue we can track to keep an eye out on the support for this in the compiler?

[johanfylling]

What you describe sounds like a very niche use case that is unlikely to have the broader appeal to warrant support in the compiler. Whenever you have compilation errors, it is very difficult to guarantee that the parts of your code that isn't the root cause of the errors will still work as intended even if the modules causing the errors are removed. It's sometimes even difficult to determine exactly what module is the culprit to an error; e.g. say that your error comes from two rules in different modules being in conflict: how do you know which one is the "wrong" one?
That being said, you're of course welcome to make a feature request.

As an alternative, would it be possible for you to introduce a validation step when new policies are submitted; where you run e.g. opa check, or even opa test on the new policy in isolation (or in combination with existing policies)? That way, you'd also get the benefit of more immediate feedback for invalid policies (depending on your setup, of course).