Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin loader #4654

Merged
merged 2 commits into from
Sep 11, 2018
Merged

Plugin loader #4654

merged 2 commits into from
Sep 11, 2018

Conversation

dadgar
Copy link
Contributor

@dadgar dadgar commented Sep 9, 2018

This PR provides a plugin loader that scans a plugin directory for plugins and provides a unified interface for launching and reattaching to plugins whether they are internal or external.

The loader will configure the plugin before handing the instance back to the caller. The unit tests are slightly complicated, but the way it works is that there is a plugin main which allows switching to running the binary as a mock plugin. The plugin currently only can be a device plugin but can be extended later.

The test harness creates a plugin directory and then copies itself to the temp folder. This provides binaries that can be fingerprinted and run as plugins.

Copy link
Member

@schmichael schmichael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Most comments are ignorable nits, but I think the executable check and logging improvement are worth doing.

for k, config := range plugins {
// Create an instance
raw := config.Factory(l.logger)
base, ok := raw.(base.BasePlugin)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't Factory return a BasePlugin and avoid this assertion?

info.baseInfo = i

// Parse and set the plugin version
if v, err := version.NewVersion(i.PluginVersion); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] if info.version, err = version.New...

if v, err := version.NewVersion(i.PluginVersion); err != nil {
multierror.Append(&mErr, fmt.Errorf("failed to parse version %q for internal plugin %s: %v", i.PluginVersion, k, err))
continue
} else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for an explicit else


var plugins []os.FileInfo
for _, f := range files {
if f.IsDir() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we filter out files without the execute bit set? Thinking people might want to drop a README.txt or something else in the dir.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to test if f is a symlink and if so resolve it and test if its a directory.

(Or use Readdirnames and Stat which follows symlinks for you.)

info, err := l.fingerprintPlugin(p, c)
if err != nil {
l.logger.Error("failed to fingerprint plugin", "plugin", name)
multierror.Append(&mErr, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So will Nomad fail to start if a plugin fails to fingerprint?

I think this is the correct behavior as it seems really bad to allow Nomad to run without a driver plugin that may be necessary to communicate with running tasks. An operator can always fix these errors by removing the offending file (or removing the execute bit if you make that change).

Just wanted to verify as this seems like an important decision.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the New method to retrieve the factory which will happen in agent.go will return an error.

if v, err := version.NewVersion(i.PluginVersion); err != nil {
return nil, fmt.Errorf("failed to parse plugin %q (%v) version %q: %v",
i.Name, info.exePath, i.PluginVersion, err)
} else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for explicit else

if extPlugin.version.LessThan(internal.version) {
l.logger.Info("preferring external version of plugin",
"plugin", extPlugin.baseInfo.Name, "internal_version", internal.version.String(),
"external_version", extPlugin.version.String())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to explicitly call String(). Anything that implements String() will have it called automatically.

return nil, fmt.Errorf("invalid plugin loader configuration passed: %v", err)
}

logger := config.Logger.Named("plugin-loader").With("plugin-dir", config.PluginDir)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I've been using underscores as a separator in log names/keys so it looks like an identifier and matches hcl params (when applicable): https://gist.github.com/schmichael/da63914230b5127fb26486b01446f5a5

I don't mind switching, but we should probably standardize.

}

// newHarness returns a harness and copies our test binary to the temp directory
// with teh passed plugin names.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the*

logger := log.Default()
logger.SetLevel(log.Trace)
lconfig := &PluginLoaderConfig{
Logger: logger, // XXX Use testlog package
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 my branch has this

@dadgar dadgar merged commit 3af3f8a into master Sep 11, 2018
@dadgar dadgar deleted the f-plugin-factory branch September 11, 2018 00:33
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants