Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to skip loading unused plugins in Docker container #12486

Conversation

nineinchnick
Copy link
Member

Description

Note This is only a suggestion. I'm fine with closing this PR if this is not the best idea for any reason.

Sometimes users would like to completely avoid loading unused plugins. The best recommendation would be to rebuild the Docker image, but some might struggle with this. This PR introduces two new environmental variables recognized only by the Docker container: TRINO_ENABLE_PLUGINS and TRINO_DISABLE_PLUGINS, which are inclusive and exclusive comma-separated lists of plugin names. Only one of them can be non-empty.

Removing plugins doesn't affect the startup time much (there's about 1s of variance):

Baseline: 8.345482000
With tpch,tpcds only: 7.005982000
Without tpch,tpcds: 9.227644000

For curiosity, I checked for arm64 (with emulation):

Baseline: 115.821223000
With tpch,tpcds only: 96.419130000
Without tpch,tpcds: 116.562944000

If adding environmental variables like this is not a good idea, maybe it's better to add some support at build time, by defining build args that would do the same. That would also be a chance to document how to build custom images based on the official Trino images.

Is this change a fix, improvement, new feature, refactoring, or other?

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

How would you describe this change to a non-technical end user or system administrator?

Related issues, pull requests, and links

Documentation

( ) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

IFS=, read -ra names <<< "$TRINO_ENABLE_PLUGINS"
for name in "${names[@]}"; do
mv "$base/plugin.disabled/$name" "$base/plugin/$name"
if [ -f "/etc/trino/catalog.disabled/$name.properties" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plugin name != catalog name

did you mean TRINO_ENABLED_CATALOGS env var?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, you're right. I added this because there are default catalogs that will cause the server to fail at startup if the connectors/plugins they use are disabled. I want to keep this simple so I'd like to avoid adding even more env vars. Would grepping for connector.name=$name be ok instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's another reason to only do this at build time - to avoid adding new logic to parse catalog configs outside of Java.

test_trino_starts "${CONTAINER_NAME}" "${PLATFORM}"
test_trino_starts -e TRINO_ENABLE_PLUGINS=jmx,memory --platform "${PLATFORM}" "${CONTAINER_NAME}"
test_trino_starts -e TRINO_DISABLE_PLUGINS=tpch,tpcds --platform "${PLATFORM}" "${CONTAINER_NAME}"
test_trino_fails -e TRINO_ENABLE_PLUGINS=jmx,memory -e TRINO_DISABLE_PLUGINS=tpch,tpcds "${CONTAINER_NAME}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_trino_fails should assert on the "exception message"

Wait for the following message log line:
When Trino finishes starting up, the container will report its status as healthy.
You can check it in the output of `docker ps`, or wait for the following message
log line:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated

@hashhar
Copy link
Member

hashhar commented May 20, 2022

I don't have strong opinions about this but IMO I don't see a lot of benefits here - the startup time isn't affected much. The only thing it might help with is satisfying security scanners which we should do the proper way.

This does allow people to break things if they don't know what they are doing by for example excluding the geospatial plugin which is a dependency of Hive connector, or shared plugins like resource-group-managers/password-authenticators/teradata-functions.

@nineinchnick nineinchnick deleted the docker-enable-plugins-with-env-var branch May 22, 2022 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants