Supporting mutiple main files (entry points) for a single project

[n-pn]

Currently the extension only support a single entry point with the Main File option. But with a large project with many files and multiple entry points (for example microservice webapps) it it not a ideal situation.

For example I have a project with this structure:

/src
├── service_1
│   ├── service_1_foo.cr
│   ├── service_1_bar.cr
│   ├── ... a dozen more
├── service_1.cr # includes all file in service_1 folder
├── service_2
│   ├── service_2_foo.cr
│   ├── service_2_bar.cr
│   ├── ... a dozen more
├── service_2.cr # includes all file in service_2 folder
├── service_3.cr # ...
├── program_4.cr # ...
└── utilities # the other services require some file here

The services might have conflicting dependencies, so creating another file and require all the file (and point the Main File to it) is not possible as it can cause both compilation time error and runtime error.

So in order to make the extension to work properly I need to change the Main File setting every time I switch from working with service_a to service_b, or just give up making it work again.

Also, as Crystal/Crystalline struggle with larger projects, ideally we need to keep the files required count as little as possible. For example in the example folder structure, the working file service_1_foo.cr at most will only need the files from service_1 folder, the files from utilities folder and the files from lib. The files from service_2 and other services should not be included in the checking.

Another scenario is you implement the code in file src/service_x/foo.cr and write the test in spec/service_x/foo_test.cr. Ideally the main file of current working file src/service_x/foo.cr is spec/service_x/foo_test.cr, not src/service_x.cr so it can be even faster to check the error.

So I hope the extension can support multiple entry points (Main File) mapping for a single project. The structure can be like this:

endpoints:
  - main: spec/service_1/foo_spec.cr
    include: [src/service_1/foo.cr]
    
  - main: src/service_1.cr
    include: [src/service_1/**/*.cr, src/utilities/**/*.cr]
    exclude: [src/utilities/blacklist_this_file_so_it_can_fallback_to_other_endpoint.cr]
  
  - main: src/service_2.cr
 
  - main: spec/spec_helper.cr # capture-all endpoint as fallback
    include: [src/**/*]

For the current working file it will run through the include/exclude filters and return the first endpoint that matches. You can switch the main file of the current file by moving the rule up or down.

Also, since the file content can change a lot, it might be better to save it as .vscode setting instead of putting it in shard.yml

[nobodywasishere]

I think this could be really useful, especially for Crystalline as well. I think this would be best in shard.yml due to the limitations of .vscode/settings.json with multi-root workspaces (trying to limit configurations there as much as possible currently). I don't think shard.yml complains about extra information existing in it as well.

It would be good to get this (somewhat) standardized before implementing it though, for instance, do the rules parse top to bottom for a file until it's caught, or what happens when a file is caught by multiple endpoints? What happens with an endpoint that doesn't have any includes/excludes?

My other thoughts are how this will translate to looking at files in other folders outside of the current project (like shards in lib/). Should they use their own endpoints or should they be captured by the endpoints of the main repo? It could get tedious to update this every time you add new files to the project, but hopefully the rules would be simple enough for most people they only need to set it up once.

[nobodywasishere]

Using the crystal tool dependencies CLI tool, this should be able to work. In the v0.9.0-alpha.1 release, if a main file isn't set, it checks each of the targets to determine which target should be used for the given current file. This should be able to handle this case perfectly without needing extra setup, only a short additional delay for running the dependencies tool.

The only issue I forsee is if a file can be used with multiple targets, but I'm not sure that's a big issue to worry about right now.

[n-pn]

nice, too bad shard.overwrite.yml does not support targets override. changing targets' orders in shard.overwrite.yml should solve some of the problems choosing the right main file.

or we can just support reading targets from shard.overwrite.yml with this library?

[nobodywasishere]

Having targets in the shards override file would work for me, can implement soon