Skip to content

Commit

Permalink
Use MVEL for rule evaluation
Browse files Browse the repository at this point in the history
Co-Authored-By: Prakhar Sapre <[email protected]>
  • Loading branch information
willmostly and prakhar10 committed Nov 13, 2024
1 parent 2add137 commit 18d7e42
Show file tree
Hide file tree
Showing 10 changed files with 293 additions and 267 deletions.
97 changes: 5 additions & 92 deletions docs/routing-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,8 +296,8 @@ actions:
```

This can difficult to maintain with more rules. To have better control over the
execution of rules, we can use rule priorities and composite rules. Overall,
priorities, composite rules, and other constructs that MVEL support allows
execution of rules, we can use rule priorities. Overall,
priorities and other constructs that MVEL support allows
you to express your routing logic.

#### Rule priority
Expand Down Expand Up @@ -328,99 +328,12 @@ that the first rule (priority 0) is fired before the second rule (priority 1).
Thus `routingGroup` is set to `etl` and then to `etl-special`, so the
`routingGroup` is always `etl-special` in the end.

More specific rules must be set to a lesser priority so they are evaluated last
to set a `routingGroup`. To further control the execution of rules, for example
to have only one rule fire, you can use composite rules.

##### Composite rules

First, please refer to the [easy-rule composite rules documentation](https://github.com/j-easy/easy-rules/wiki/defining-rules#composite-rules).

The preceding section covers how to control the order of rule execution using
priorities. In addition, you can configure evaluation so that only the first
rule matched fires (the highest priority one) and the rest is ignored. You can
use `ActivationRuleGroup` to achieve this:

```yaml
---
name: "airflow rule group"
description: "routing rules for query from airflow"
compositeRuleType: "ActivationRuleGroup"
composingRules:
- name: "airflow special"
description: "if query from airflow with special label, route to etl-special group"
priority: 0
condition: 'request.getHeader("X-Trino-Source") == "airflow" && request.getHeader("X-Trino-Client-Tags") contains "label=special"'
actions:
- 'result.put("routingGroup", "etl-special")'
- name: "airflow"
description: "if query from airflow, route to etl group"
priority: 1
condition: 'request.getHeader("X-Trino-Source") == "airflow"'
actions:
- 'result.put("routingGroup", "etl")'
```

Note that the priorities have switched. The more specific rule has a higher
priority, since it should fire first. A query coming from airflow with special
label is matched to the "airflow special" rule first, since it's higher
priority, and the second rule is ignored. A query coming from airflow with no
labels does not match the first rule, and is then tested and matched to the
second rule.

You can also use `ConditionalRuleGroup` and `ActivationRuleGroup` to implement
an if/else workflow. The following logic in pseudocode:

```text
if source == "airflow":
if clientTags["label"] == "foo":
return "etl-foo"
else if clientTags["label"] = "bar":
return "etl-bar"
else
return "etl"
```

This logic can be implemented with the following rules:

```yaml
name: "airflow rule group"
description: "routing rules for query from airflow"
compositeRuleType: "ConditionalRuleGroup"
composingRules:
- name: "main condition"
description: "source is airflow"
priority: 0 # rule with the highest priority acts as main condition
condition: 'request.getHeader("X-Trino-Source") == "airflow"'
actions:
- ""
- name: "airflow subrules"
compositeRuleType: "ActivationRuleGroup" # use ActivationRuleGroup to simulate if/else
composingRules:
- name: "label foo"
description: "label client tag is foo"
priority: 0
condition: 'request.getHeader("X-Trino-Client-Tags") contains "label=foo"'
actions:
- 'result.put("routingGroup", "etl-foo")'
- name: "label bar"
description: "label client tag is bar"
priority: 0
condition: 'request.getHeader("X-Trino-Client-Tags") contains "label=bar"'
actions:
- 'result.put("routingGroup", "etl-bar")'
- name: "airflow default"
description: "airflow queries default to etl"
condition: "true"
actions:
- 'result.put("routingGroup", "etl")'
```
More specific rules must be set to a higher priority so they are evaluated last
to set a `routingGroup`.

##### If statements (MVEL Flow Control)

In the preceding section you see how `ConditionalRuleGroup` and
`ActivationRuleGroup` are used to implement an `if/else` workflow. You can
use MVEL support for `if` statements and other flow control. The following logic
You can use MVEL support for `if` statements and other flow control. The following logic
in pseudocode:

```text
Expand Down
26 changes: 3 additions & 23 deletions gateway-ha/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
<frontend.pnpmRegistryURL>https://registry.npmmirror.com</frontend.pnpmRegistryURL>

<!-- dependency versions -->
<dep.jeasy.version>4.1.0</dep.jeasy.version>
<dep.mockito.version>5.14.2</dep.mockito.version>
<dep.okhttp3.version>4.12.0</dep.okhttp3.version>
<dep.trino.version>462</dep.trino.version>
Expand Down Expand Up @@ -253,21 +252,9 @@
</dependency>

<dependency>
<groupId>org.jeasy</groupId>
<artifactId>easy-rules-core</artifactId>
<version>${dep.jeasy.version}</version>
</dependency>

<dependency>
<groupId>org.jeasy</groupId>
<artifactId>easy-rules-mvel</artifactId>
<version>${dep.jeasy.version}</version>
</dependency>

<dependency>
<groupId>org.jeasy</groupId>
<artifactId>easy-rules-support</artifactId>
<version>${dep.jeasy.version}</version>
<groupId>org.mvel</groupId>
<artifactId>mvel2</artifactId>
<version>2.5.2.Final</version>
</dependency>

<dependency>
Expand All @@ -290,13 +277,6 @@
<scope>runtime</scope>
</dependency>

<dependency>
<groupId>org.mvel</groupId>
<artifactId>mvel2</artifactId>
<version>2.5.2.Final</version>
<scope>runtime</scope>
</dependency>

<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.trino.gateway.ha.router;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
import com.fasterxml.jackson.dataformat.yaml.YAMLParser;
import io.trino.gateway.ha.config.RequestAnalyzerConfig;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ArrayList;
import java.util.List;

import static java.nio.charset.StandardCharsets.UTF_8;

public class MVELFileRoutingGroupSelector
extends RulesRoutingGroupSelector<MVELRoutingRule>
{
Path rulesPath;

MVELFileRoutingGroupSelector(String rulesPath, RequestAnalyzerConfig requestAnalyzerConfig)
{
super(requestAnalyzerConfig);
this.rulesPath = Paths.get(rulesPath);

setRules(readRulesFromPath(this.rulesPath));
}

@Override
void reloadRules(long lastUpdatedTimeMillis)
{
try {
BasicFileAttributes attr = Files.readAttributes(this.rulesPath, BasicFileAttributes.class);
if (attr.lastModifiedTime().toMillis() > lastUpdatedTimeMillis) {
synchronized (this) {
if (attr.lastModifiedTime().toMillis() > lastUpdatedTimeMillis) {
List<MVELRoutingRule> ruleList = readRulesFromPath(this.rulesPath);
setRules(ruleList);
}
}
}
}
catch (IOException e) {
throw new RuntimeException("Could not access rules file", e);
}
}

public List<MVELRoutingRule> readRulesFromPath(Path rulesPath)
{
ObjectMapper yamlReader = new ObjectMapper(new YAMLFactory());
try {
String content = Files.readString(rulesPath, UTF_8);
YAMLParser parser = new YAMLFactory().createParser(content);
List<MVELRoutingRule> routingRulesList = new ArrayList<>();
while (parser.nextToken() != null) {
MVELRoutingRule routingRules = yamlReader.readValue(parser, MVELRoutingRule.class);
routingRulesList.add(routingRules);
}
return routingRulesList;
}
catch (IOException e) {
throw new RuntimeException("Failed to read or parse routing rules configuration from path : " + rulesPath, e);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.trino.gateway.ha.router;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;

import java.io.Serializable;
import java.util.List;
import java.util.Map;

import static com.google.common.collect.ImmutableList.toImmutableList;
import static org.mvel2.MVEL.compileExpression;
import static org.mvel2.MVEL.executeExpression;

public class MVELRoutingRule
extends RoutingRule<Serializable>
{
@JsonCreator
public MVELRoutingRule(
@JsonProperty("name") String name,
@JsonProperty("description") String description,
@JsonProperty("priority") Integer priority,
@JsonProperty("condition") Serializable condition,
@JsonProperty("actions") List<Serializable> actions)
{
super(
name,
description,
priority,
condition instanceof String stringCondition ? compileExpression(stringCondition) : condition,
actions.stream().map(action -> {
if (action instanceof String stringAction) {
return compileExpression(stringAction);
}
else {
return action;
}
}).collect(toImmutableList()));
}

@Override
public void evaluate(Map<String, Object> variables)
{
if ((boolean) executeExpression(condition, variables)) {
actions.forEach(action -> executeExpression(action, variables));
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ static RoutingGroupSelector byRoutingGroupHeader()
*/
static RoutingGroupSelector byRoutingRulesEngine(String rulesConfigPath, RequestAnalyzerConfig requestAnalyzerConfig)
{
return new RuleReloadingRoutingGroupSelector(rulesConfigPath, requestAnalyzerConfig);
return new MVELFileRoutingGroupSelector(rulesConfigPath, requestAnalyzerConfig);
}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.trino.gateway.ha.router;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;

import java.util.List;
import java.util.Map;

import static java.util.Objects.requireNonNull;
import static java.util.Objects.requireNonNullElse;

public abstract class RoutingRule<T>
implements Comparable<RoutingRule<T>>
{
String name;
String description;
Integer priority;
T condition;
List<T> actions;

@JsonCreator
public RoutingRule(
@JsonProperty("name") String name,
@JsonProperty("description") String description,
@JsonProperty("priority") Integer priority,
@JsonProperty("condition") T condition,
@JsonProperty("actions") List<T> actions)
{
this.name = requireNonNull(name, "name is null");
this.description = requireNonNullElse(description, "");
this.priority = requireNonNullElse(priority, 0);
this.condition = requireNonNull(condition, "condition is null");
this.actions = requireNonNull(actions, "actions is null");
}

public abstract void evaluate(Map<String, Object> variables);

@Override
public int compareTo(RoutingRule o)
{
if (o == null) {
return 1;
}
return priority.compareTo(o.priority);
}
}
Loading

0 comments on commit 18d7e42

Please sign in to comment.