fix(watcher): use exclude options to optimize file watching

[edvald]

What this PR does / why we need it:

This basically relies on optimizations already implemented in chokidar,
the library we use for file watching, in order to reduce overhead when using file watching on linux. Hopefully this can solve the
excessive CPU/RAM usage some users have reported.

Which issue(s) this PR fixes:

Closes #1269

Special notes for your reviewer:

In a large project folder, try adding exclusions via modules.exclude in the project config, or using the exclude field in individual module configs.

There are some potential alternative approaches, but I wanted to see if this works reasonably before doing something more elaborate.

[edvald]

@matuszeman could you try this and see if it solves your issue?

[eysi09]

It's a little confusing that exclude changes the watch behaviour whereas include does not. Perhaps we can mention this in the reference docs as well. For both the include and the exclude directive for that matter, so it's clear that they behave in subtly different ways.

[edvald]

I did put in clauses in the docs mentioning that these specific fields can be used to limit file watching, but didn't explain why the include filters aren't helpful.

The reason is that only rarely/incidentally can include statements help us optimize anything. Basically the only scenario where we could in theory use them to limit the scope, would be when all of the include filters are directory filters (e.g. my/directory/**/* or my/directory/*). Otherwise we need to match anything matching the filter, which can then be in any directory, and chokidar has no optimizations for that scenario.

I'm not sure where/how to explain this though. What do you think?

[eysi09]

Yeah I agree with the implementation. I suggested mentioning this in the reference docs for the include and exclude directives.

I figured users would expect include to also affect the watch behaviour since that's what exclude does. So explicitly mentioning that it doesn't in the include reference docs might prevent confusion.

[edvald]

Sure, makes sense! I'll add notes in the relevant schema field descriptions.

[edvald]

I've added a commit, and done some more testing. I believe this is good to go now.

Note that I've removed the optimization based on module-level exclude fields for now. Turns out using module exclude fields would need more elaborate logic, in the case of overlapping module sources.

[eysi09]

LGTM!