Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EQL: Add integration tests harness to test EQL feature parity with original implementation #52248

Merged
merged 12 commits into from
Feb 22, 2020

Conversation

aleksmaus
Copy link
Member

@aleksmaus aleksmaus commented Feb 12, 2020

The tests use the original test queries from
https://github.com/endgameinc/eql/blob/master/eql/etc/test_queries.toml
for EQL implementation correctness validation.
The file test_queries_unsupported.toml serves as a "blacklist" for the
queries that we do not support. Currently all of the queries are
blacklisted. Over the time the expectation is to eventually have an
empty "blacklist" when all of the queries are fully supported.

The tests use the original test vector from
https://raw.githubusercontent.com/endgameinc/eql/master/eql/etc/test_data.json
that was translated to ECS format that matches the latest mapping being
used for Endgame platform event streaming and is loaded from endgame.dat
file. The endgame.json file contains the matching index
mappings/setting.

Only one EQL spec response is stubbed for now to match the expected
output for that spec query. This part would need some tweaking after EQL is
fully wired.

The input .toml file is parsed by hand for now, which is sufficient for
our purposes and avoids introducing another dependency just for this
particular test case.

Related to #49581

@imotov could you take a look first before I promote this from draft PR?
Thanks!

@aleksmaus aleksmaus added the :Analytics/EQL EQL querying label Feb 12, 2020
@aleksmaus aleksmaus requested a review from imotov February 12, 2020 02:55
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/EQL)

@aleksmaus
Copy link
Member Author

@elasticmachine run elasticsearch-ci/packaging-sample-matrix-unix

@aleksmaus aleksmaus requested a review from costin February 12, 2020 14:25
@aleksmaus aleksmaus force-pushed the feature/eql_validation_tests branch from f944369 to 032b699 Compare February 12, 2020 14:44
…iginal implementation

The tests use the original test queries from
https://github.com/endgameinc/eql/blob/master/eql/etc/test_queries.toml
for EQL implementation correctness validation.
The file test_queries_unsupported.toml serves as a "blacklist" for the
queries that we do not support. Currently all of the queries are
blacklisted. Over the time the expectation is to eventually have an
empty "blacklist" when all of the queries are fully supported.

The tests use the original test vector from
https://raw.githubusercontent.com/endgameinc/eql/master/eql/etc/test_data.json
that was translated to ECS format that matches the latest mapping being
used for Endgame platform event streaming and is loaded from endgame.dat
file. The endgame.json file contains the matching index
mappings/setting.

Only one EQL and the response is stubbed for now to match the expected
output from that query. This part would need some tweaking after EQL is
fully wired.

The input .toml file is parsed by hand for now, which is sufficient for
our purposes and avoids introducing another dependency just for this
particular test case.

Related to elastic#49581
@aleksmaus aleksmaus force-pushed the feature/eql_validation_tests branch from 032b699 to 47aaef7 Compare February 12, 2020 15:08
@@ -0,0 +1,102 @@
{"user":{"group":{}},"host":{"os":{"platform":"windows","name":"Windows"},"ip":"127.0.0.1","hostname":"localhost","name":"localhost"},"event":{"module":"endgame","dataset":"esensor","action":"already_running","category":"process","kind":"event"},"labels":{"account_id":"f6c123dc-b6d9-4849-937b-08c444ce7d96","endpoint_id":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66"},"agent":{"type":"endgame","id":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66","version":"3.54.0"},"ecs":{"version":"1.1.0"},"@timestamp":0,"process":{"thread":{},"parent":{},"hash":{},"name":"System Idle Process"},"endgame":{"name":"localhost","eid":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66","MID":"","ip":"127.0.0.1","sver":"3.54.0","os_type":"windows","serial_event_id":1,"opcode":3,"event_type_full":"process_event","event_subtype_full":"already_running","timestamp":116444736000000000,"process_name":"System Idle Process","unique_pid":1,"xml_message":"","provider_name":"","provider_guid":"","channel_name":"","subject_user_sid":"","subject_user_name":"","subject_domain_name":"","subject_logon_id":"","target_domain_name":"","target_user_name":"","target_logon_id":"","ip_address":"","computer_name":"","privilege_list":"","logon_type":null,"system_thread_id":null,"system_pid":null,"system_process_name":"","query_name":"","query_type":null,"query_results":null,"metadata":{"collection_time":0}}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two q's

  • why the .dat extension?
  • I don't think this needs to be in ECS, other than the event.category field. Otherwise all of the queries need to be rewritten

Copy link
Member Author

@aleksmaus aleksmaus Feb 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. it's not quite a valid json, it's a collection of jsons. open for suggestions on the file extension.
  2. wanted to use the exact same ECS data format that is currently established for events streaming from SMP, so we could potentially flesh out the details applicable to our data format first.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically use *.json for that, even though *.ndjson is probably a better choice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to .ndjson

# official EQL implementation

[[queries]]
query = 'process where serial_event_id < 4'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one is supported

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, but not hooked up to the query engine yet. Once it is, we can validate the results and update the .toml files.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expecting the test_queries_unsupported.toml file to become empty and eventually dropped when we reach the feature parity

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I didn't realize that. in that case, this file is missing the first query from test_queries.toml. i just looked at the top and saw the discrepancy and reached the wrong conclusion

[[queries]]
query = 'process where serial_event_id = 1'
expected_event_ids  = [1]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the comment into the file per @rw-access recommendation.

@costin costin requested a review from astefan February 12, 2020 16:38
@aleksmaus aleksmaus marked this pull request as ready for review February 13, 2020 14:21
@aleksmaus aleksmaus force-pushed the feature/eql_validation_tests branch from 9b14a36 to 7edf6c7 Compare February 13, 2020 14:51
Copy link
Contributor

@imotov imotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @costin and @nik9000 might be better reviews for this since they had more experience with sql-spec tests.

@@ -0,0 +1,102 @@
{"user":{"group":{}},"host":{"os":{"platform":"windows","name":"Windows"},"ip":"127.0.0.1","hostname":"localhost","name":"localhost"},"event":{"module":"endgame","dataset":"esensor","action":"already_running","category":"process","kind":"event"},"labels":{"account_id":"f6c123dc-b6d9-4849-937b-08c444ce7d96","endpoint_id":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66"},"agent":{"type":"endgame","id":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66","version":"3.54.0"},"ecs":{"version":"1.1.0"},"@timestamp":0,"process":{"thread":{},"parent":{},"hash":{},"name":"System Idle Process"},"endgame":{"name":"localhost","eid":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66","MID":"","ip":"127.0.0.1","sver":"3.54.0","os_type":"windows","serial_event_id":1,"opcode":3,"event_type_full":"process_event","event_subtype_full":"already_running","timestamp":116444736000000000,"process_name":"System Idle Process","unique_pid":1,"xml_message":"","provider_name":"","provider_guid":"","channel_name":"","subject_user_sid":"","subject_user_name":"","subject_domain_name":"","subject_logon_id":"","target_domain_name":"","target_user_name":"","target_logon_id":"","ip_address":"","computer_name":"","privilege_list":"","logon_type":null,"system_thread_id":null,"system_pid":null,"system_process_name":"","query_name":"","query_type":null,"query_results":null,"metadata":{"collection_time":0}}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically use *.json for that, even though *.ndjson is probably a better choice.


@BeforeClass
public static void checkForSnapshot() {
assumeTrue("Only works on snapshot builds for now", Build.CURRENT.isSnapshot());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this anymore.

@aleksmaus
Copy link
Member Author

@imotov, I talked to @costin before and got few suggestions. Basically, wanted to keep the original test spec from EQL unchanged, so we can easily follow any original EQL implementation changes by copying the toml file over into our test harness. Also tried to avoid adding any toml library dependency, since the format of the file is pretty simple to parse as is without the need for a full toml parser implementation yet, especially for testing.

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm certainly ok reusing the tests from other spots so the toml is fine by me. I think it'd be better to pull in an apache licensed toml parser as a test dependency for this project rather than build it by hand. Test dependencies are generally not so big a deal, especially when we isolate them into a specific sub-project.

The fancy CSV tests that SQL has have the nice property of being able to include them in the docs because they look just like output from the command line sql util. toml doesn't have that property, but that seems like a small price to pay for being able to reuse tests from a mature project.

Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments.

I'm on the fence with the TOML reader. On one hand the parser is already written and we can customize it appropriately - on the other hand using an existing library would reduce (theoretically) the maintenance burdon.

I don't know enough about TOML to tell how much standard/non-standard it can get.
Regardless, reusing the test queries and results in the docs is something that's valuable to pursue in the future.

public Collection<Object> createComponents(Client client, ClusterService clusterService, ThreadPool threadPool,
ResourceWatcherService resourceWatcherService, ScriptService scriptService, NamedXContentRegistry xContentRegistry,
Environment environment, NodeEnvironment nodeEnvironment, NamedWriteableRegistry namedWriteableRegistry) {
public Collection<Object> createComponents(Client client, ClusterService clusterService,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why format the above?
In general, PRs should touch only lines that were modified not format the entire file. Both IntelliJ and Eclipse have this option.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just making it easier to read, also because it just was just brought to my attention that some of the code that I had some formatting issues.

}

@Override
public List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> getActions() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused about why the method was moved...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consistent with other plugins. can undo

);
}
return List.of(
new ActionHandler<>(XPackUsageFeatureAction.EQL, EqlUsageTransportAction.class),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If EQL is disabled, why register an action for it?
Also if these two actions are returned always, I would put them in a list and only add the extra 2 if enabled:

List<> actions = Arrays.asList(1,2)
if (enabled) {
   actions.add(3);
   actions.add(4);
}
return list

If you really want it immutable, it can be wrapped inside another collection but considering its for internal consumption it's no need imo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@costin see #52328 for why those two action have to be registered. At least, for the moment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not sweat about this particular code, it will be cleaner once we remove the feature flag.

@@ -103,9 +102,24 @@
public List<Setting<?>> getSettings() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either we use enabled or this method. On one hand enabled checks the property presence statically but this method tries to do it dynamic when a class is instantiated.
There's there are multiple pieces of isSnapshot in this class that need simplification.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried to follow the autoscaling plugin pattern, that I was told should be an example of handling the feature flag.

@@ -41,7 +40,7 @@

@Inject
public TransportEqlSearchAction(Settings settings, ClusterService clusterService, TransportService transportService,
ThreadPool threadPool, ActionFilters actionFilters, PlanExecutor planExecutor) {
ThreadPool threadPool, ActionFilters actionFilters, PlanExecutor planExecutor) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, formatting on non-modified lines is noise.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the formatting misalignment that I missed in my previous PR, now trying to fix it


private static String[] readArray(String line) throws Exception {
line = readValueLine(line);
if (!line.startsWith("[") && !line.endsWith("]")) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the preference in general is not to negate but instead use == false for clarity.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will update

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is no longer relevant, replacing with toml library

@@ -26,6 +26,10 @@ configurations {

archivesBaseName = 'x-pack-sql'

test {
testLogging.showStandardStreams = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will remove

}
}

if (supported) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this test should use @ParametersFactory so each example becomes a test that we can address individually.

Copy link
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff. Left some comments.

randomAlphaOfLength(16));
}

public static Tuple<String, String> pathAndName(String string) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this method used anywhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer used indeed, will remove

@Override
protected Settings nodeSettings(int nodeOrdinal) {
Settings.Builder settings = Settings.builder().put(super.nodeSettings(nodeOrdinal));
settings.put(XPackSettings.SECURITY_ENABLED.getKey(), false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At one point we'll have to test EQL in a security enabled context as well, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably. Was following AbstractSqlIntegTestCase as example in this case.

);
}
return List.of(
new ActionHandler<>(XPackUsageFeatureAction.EQL, EqlUsageTransportAction.class),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@costin see #52328 for why those two action have to be registered. At least, for the moment.

InputStream is = EqlActionIT.class.getResourceAsStream("/endgame.dat");
BulkRequestBuilder bulkBuilder = client().prepareBulk();
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(is, StandardCharsets.UTF_8))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this line fit on the try ( one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer relevant, replaced with the toml parser library

while ((line = reader.readLine()) != null) {
bulkBuilder.add(new IndexRequest(testIndexName).source(line.trim(), XContentType.JSON));
}
BulkResponse response = bulkBuilder.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE).get();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import static for IMMEDIATE?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this is consistent with 100+ other places
Screen Shot 2020-02-18 at 2 00 58 PM


private static String readString(String line, BufferedReader reader) throws Exception {
line = readValueLine(line);
String delim = "";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this initialization is necessary. A simple String delim; should be enough.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDE was complaining on unitialized var delim otherwise

List<EqlSpec> testSpecs = new ArrayList<>();

EqlSpec spec = null;
try (BufferedReader reader = new BufferedReader(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single try() line.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer relevant, replaced with toml parser.

}
}

if (StringUtil.isNullOrEmpty(delim)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a class org.elasticsearch.common.Strings that has the same method and I think we prefer that one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer relevant, replaced with toml parser.

}

private static void validateAndAddSpec(List<EqlSpec> specs, EqlSpec spec, boolean supported) throws Exception {
if (StringUtil.isNullOrEmpty(spec.query)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here about Strings class.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@@ -71,6 +70,7 @@
* EqlSearchResponse.Hits hits = new EqlSearchResponse.Hits(null, null, counts, totals);
* EqlSearchResponse response = new EqlSearchResponse(hits, 5, false);
*/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove this, please?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. It was initially useful to have while we were iterating on the response shape.

@aleksmaus
Copy link
Member Author

@nik9000 replacing the manual toml parsing with io.ous:jtoml:2.0.0.
The other couple of libraries that I looked at depend on antlr4 4.7.1 which breaks compatibility with existing 4.5.3.

@@ -0,0 +1,2531 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: a more descriptive filename. i.e. endgame_mappings.json

@@ -0,0 +1,102 @@
{"user":{"group":{}},"host":{"os":{"platform":"windows","name":"Windows"},"ip":"127.0.0.1","hostname":"localhost","name":"localhost"},"event":{"module":"endgame","dataset":"esensor","action":"already_running","category":"process","kind":"event"},"labels":{"account_id":"f6c123dc-b6d9-4849-937b-08c444ce7d96","endpoint_id":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66"},"agent":{"type":"endgame","id":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66","version":"3.54.0"},"ecs":{"version":"1.1.0"},"@timestamp":0,"process":{"thread":{},"parent":{},"hash":{},"name":"System Idle Process"},"endgame":{"name":"localhost","eid":"6a19472c-f0ab-4f3a-b83d-dc3698a5ff66","MID":"","ip":"127.0.0.1","sver":"3.54.0","os_type":"windows","serial_event_id":1,"opcode":3,"event_type_full":"process_event","event_subtype_full":"already_running","timestamp":116444736000000000,"process_name":"System Idle Process","unique_pid":1,"xml_message":"","provider_name":"","provider_guid":"","channel_name":"","subject_user_sid":"","subject_user_name":"","subject_domain_name":"","subject_logon_id":"","target_domain_name":"","target_user_name":"","target_logon_id":"","ip_address":"","computer_name":"","privilege_list":"","logon_type":null,"system_thread_id":null,"system_pid":null,"system_process_name":"","query_name":"","query_type":null,"query_results":null,"metadata":{"collection_time":0}}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the mappings are consistent with the data, but the queries are not. EQL is independent of ECS, so I think it's okay to leave the data in the same schema that it was in

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using this format mostly in order to validate and work through any possible issues with covering our particular use case

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that means we have to manually rewrite all of the queries though, which is likely more error prone.

Copy link
Member Author

@aleksmaus aleksmaus Feb 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with @rw-access on Slack.
Originally I was under impression that we will keep the queries/rules the same and use some internal mapping to adopt them for ECS. It looks like in the first iteration this will not be the case and the queries/rules will be rewritten for ECS.
I will update this PR to use the same flat json data from the original EQL tests and remove the ECS mapping template creation for now.

@aleksmaus
Copy link
Member Author

Updated PR with the latest recommendations from @rw-access, also including @nik9000 suggestion to use @ParametersFactory for the test harness.

The fake response change was done in order to test the EQL validation harness imlementation.
This code will be replaced once we have the actualy EQL implementation wired.
@aleksmaus
Copy link
Member Author

Added a small adjustment after merging the latest master changes that included @imotov HL client test against the currently hardcoded EQL response. I changed the shape of the response for the first of the EQL validation harness test queries expected response, while Igor was working on HL client.

@aleksmaus aleksmaus force-pushed the feature/eql_validation_tests branch from efb551b to 384cfd8 Compare February 20, 2020 15:04
Copy link
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aleksmaus
Copy link
Member Author

Had to merge master again and resolve conflicts. Will merge once it clears CI build.

@aleksmaus aleksmaus merged commit e06f2af into elastic:master Feb 22, 2020
aleksmaus added a commit to aleksmaus/elasticsearch that referenced this pull request Feb 22, 2020
…iginal implementation (elastic#52248)

The tests use the original test queries from
https://github.com/endgameinc/eql/blob/master/eql/etc/test_queries.toml
for EQL implementation correctness validation.
The file test_queries_unsupported.toml serves as a "blacklist" for the
queries that we do not support. Currently all of the queries are
blacklisted. Over the time the expectation is to eventually have an
empty "blacklist" when all of the queries are fully supported.

The tests use the original test vector from
https://raw.githubusercontent.com/endgameinc/eql/master/eql/etc/test_data.json.

Only one EQL and the response is stubbed for now to match the expected
output from that query. This part would need some tweaking after EQL is
fully wired.

Related to elastic#49581
aleksmaus added a commit that referenced this pull request Feb 24, 2020
…iginal implementation (#52248) (#52675)

The tests use the original test queries from
https://github.com/endgameinc/eql/blob/master/eql/etc/test_queries.toml
for EQL implementation correctness validation.
The file test_queries_unsupported.toml serves as a "blacklist" for the
queries that we do not support. Currently all of the queries are
blacklisted. Over the time the expectation is to eventually have an
empty "blacklist" when all of the queries are fully supported.

The tests use the original test vector from
https://raw.githubusercontent.com/endgameinc/eql/master/eql/etc/test_data.json.

Only one EQL and the response is stubbed for now to match the expected
output from that query. This part would need some tweaking after EQL is
fully wired.

Related to #49581
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants