Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: isolate hardcoded language support #2796

Merged
merged 43 commits into from
Oct 1, 2024
Merged

Conversation

matt2e
Copy link
Collaborator

@matt2e matt2e commented Sep 24, 2024

Part of #2452
In preparation for language plugins, this PR defines a protocol for each language support to conform to. Later we will define the external plugin gRPC service and then move each language to being an external plugin one by one.

Breakdown of components:

Build engine

  • Watches for:
    • added and removed modules
    • updates to ftl.toml
  • Asks a module plugin to build when:
    • explicitly asked (eg: ftl build)
    • dependencies are updated
    • ftl.toml is updated
  • Listens to each module plugin's Updates topic to react to automatic builds

Module plugin

Currently these plugins arent actual plugins, but they will be soon!

  • Can be used to scaffold a new module (ftl new ...)
  • Gathers dependencies for a module
  • Builds when asked by the build engine
  • Watches for file changes within the module (language dependant. excludes ftl.toml)
    • When a change is detected, publishes automatic build events to it's Updates topic

Not included in this PR (coming in a later one):

  • allow ftl new to have language specific arguments (currently go and jvm ones are hardcoded into cmd_new.go)
  • currently no logic for collapsing multiple build requests into one or detecting if we have already built since a file change event was fired. This logic needs to change for external plugins anyway.

@ftl-robot ftl-robot mentioned this pull request Sep 24, 2024
@matt2e matt2e changed the title feat: isolate hard coded language support feat: isolate hardcoded language support Sep 24, 2024
@matt2e matt2e force-pushed the matt2e/language-isolation branch 2 times, most recently from bd0ab9b to 97737ab Compare September 25, 2024 22:59
@matt2e matt2e added the run-all A PR with this label will run the full set of CI jobs in the PR rather than in the merge queue label Sep 26, 2024
@matt2e matt2e force-pushed the matt2e/language-isolation branch 4 times, most recently from 322d7e1 to 045abee Compare September 30, 2024 02:58
@matt2e matt2e mentioned this pull request Sep 30, 2024
38 tasks
@matt2e matt2e force-pushed the matt2e/language-isolation branch 4 times, most recently from 7dc7580 to 35639ff Compare September 30, 2024 06:26
@@ -333,3 +324,11 @@ func parseImports(filePath string) ([]string, error) {
}
return imports, nil
}

func (c ModuleConfig) Schema() string {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema is going to be passed back from plugins over gRPC, so modules themselves shouldn't be configuring the location of the schema file.

@matt2e matt2e force-pushed the matt2e/language-isolation branch from 2aab2c7 to 03c6ad8 Compare September 30, 2024 06:39
@matt2e matt2e marked this pull request as ready for review September 30, 2024 06:39
@matt2e matt2e requested a review from alecthomas as a code owner September 30, 2024 06:39
@matt2e matt2e requested review from a team, safeer, wesbillman and alecthomas and removed request for a team, alecthomas and safeer September 30, 2024 06:39
Copy link
Collaborator

@alecthomas alecthomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks Matt.

internal/buildengine/engine.go Outdated Show resolved Hide resolved
internal/buildengine/engine.go Outdated Show resolved Hide resolved
internal/buildengine/engine.go Outdated Show resolved Hide resolved
for {
select {
case event := <-e.pluginEvents:
logger := log.FromContext(originalCtx).Module(event.ModuleName()).Scope("build")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe create the logger once at the top?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger needs to be scoped to the module, but the loop is for all modules. So it needs to be done per event

internal/buildengine/filehash.go Outdated Show resolved Hide resolved
internal/buildengine/plugin.go Outdated Show resolved Hide resolved
internal/buildengine/plugin_go.go Outdated Show resolved Hide resolved
internal/buildengine/plugin_java.go Outdated Show resolved Hide resolved
internal/buildengine/plugin_java.go Outdated Show resolved Hide resolved
internal/buildengine/plugin_java.go Outdated Show resolved Hide resolved
@matt2e matt2e requested review from a team as code owners October 1, 2024 00:56
@matt2e matt2e requested review from worstell and removed request for a team October 1, 2024 00:56
@matt2e matt2e force-pushed the matt2e/language-isolation branch from ffd4b38 to 5f5cbfe Compare October 1, 2024 01:07
@matt2e matt2e enabled auto-merge (squash) October 1, 2024 01:08
@matt2e matt2e merged commit ad9c942 into main Oct 1, 2024
93 checks passed
@matt2e matt2e deleted the matt2e/language-isolation branch October 1, 2024 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-all A PR with this label will run the full set of CI jobs in the PR rather than in the merge queue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants