Proxy upstream support #60

lieut-data · 2020-04-23T02:51:04Z

Summary

Support proxying to an upstream marketplace via --upstream or the BUILD_UPSTREAM_URL environment variable that is baked into a binary.

This is useful for customers that want to supply their own plugins, but simultaneously stay up-to-date to an upstream marketplace like api.integrations.mattermost.com. This is immediately useful for launching a "beta listing" of the marketplace in which to list upcoming efforts, such as the incident response plugin, but without exposing to all users of the marketplace.

Ticket Link

N/A

Makefile

alifarooq0

Nicely done sir! Great feature 🎉

internal/store/proxy_test.go

hanzei

One questions and one comment about the code flow.

Great work Jesse 🎉

Makefile

cmd/lambda/main.go

DHaussermann · 2020-05-25T18:42:28Z

@lieut-data can you please provide some guidance for testing this change? It's not quite clear to me how to test this.
If start my server withgo run ./cmd/marketplace server --upstream https://api.integrations.mattermost.com and then point the Mattermost server to my local instance, should it also show me additional plgins coming from https://api.integrations.mattermost.com?

lieut-data · 2020-05-25T18:45:20Z

@DHaussermann, yes, that's basically the idea. You'll have to edit your local plugins.json with something to "overlay" the upstream. Note that this is live on https://api.private-beta.integrations.mattermost.com/ as well.

DHaussermann · 2020-06-09T16:05:51Z

@hanzei if you have time, is this something you would be able to peer test using the peer testing process documented here?https://mattermost.atlassian.net/wiki/spaces/INT/pages/619937892/Peer+Testing+Process

hanzei · 2020-06-10T09:50:19Z

Sure @DHaussermann 👍

hanzei

Scope

Allow upstream marketplace

Testing notes

Modified plugins.json to only contain a Matterpoll release
Run go run ./cmd/marketplace server --upstream https://api.integrations.mattermost.com
Confirmed that Marketplace Modal shows all plugins from https://api.integrations.mattermost.com and the Matterpoll release

Follow-up concerns

None

hanzei · 2020-06-10T10:35:25Z

@DHaussermann ☝️

lieut-data · 2020-06-10T12:06:08Z

Thanks, @hanzei!

DHaussermann · 2020-06-11T13:25:05Z

Thanks for testing @hanzei! Much appreciated.

* [MM-21749] [MM-22214] Add "Beta" and "Community" label (#37) * Add beta and community label * wording * Feedback * Make use of model.IsValid() (#44) * Pin github.com/rakyll/statik version (#47) * document automatic deployment (#46) * document automatic deployment * tweak merging language Co-authored-by: Ben Schumacher <[email protected]> * Add issue templates for community plugins (#45) Add two issue templates: one for adding a new community plugin and one for updating an existing community plugin. * [MM-23184] Omit empty labels list (#48) * Drop iconPaths workaround (#62) * Use GITHUB_TOKEN env variable instead of having the user define it (#54) * Proxy upstream support (#60) * s/store/staticStore/ * prefer later plugins if version exactly matches * introduce merged and proxy stores * support --upstream * generate on run-server * update to go1.14 * tidy up interface/comments * golangci-lint local-only complaints * Update internal/store/proxy_test.go * revert generate on run-server * avoid merged store unless needed Co-authored-by: Ben Schumacher <[email protected]> Co-authored-by: mattermod <[email protected]> * Ensure manually added plugins stay in the database (#68) * Send Announcements for new plugins (#76) * Limit enterprise plugins to E20 installations (#79) * Limit enterprise plugins to E20 installations * Apply suggestions from code review Co-authored-by: Aaron Rothschild <[email protected]> * Make getPlugins signature more clear * Don't omit enterprise in json * make generate * Include workaround for https://mattermost.atlassian.net/browse/MM-26507 * Fix typo * Changes as requested * Drop else * Move minVersionSupportingEnterpriseFlags into global scope Co-authored-by: Aaron Rothschild <[email protected]> * Add Enterprise label to MS Calendar * make generate Co-authored-by: Jesse Hallam <[email protected]> Co-authored-by: mattermod <[email protected]> Co-authored-by: Aaron Rothschild <[email protected]>

* [MM-21749] [MM-22214] Add "Beta" and "Community" label (#37) * Add beta and community label * wording * Feedback * Make use of model.IsValid() (#44) * Pin github.com/rakyll/statik version (#47) * document automatic deployment (#46) * document automatic deployment * tweak merging language Co-authored-by: Ben Schumacher <[email protected]> * Add issue templates for community plugins (#45) Add two issue templates: one for adding a new community plugin and one for updating an existing community plugin. * [MM-23184] Omit empty labels list (#48) * Drop iconPaths workaround (#62) * Use GITHUB_TOKEN env variable instead of having the user define it (#54) * Proxy upstream support (#60) * s/store/staticStore/ * prefer later plugins if version exactly matches * introduce merged and proxy stores * support --upstream * generate on run-server * update to go1.14 * tidy up interface/comments * golangci-lint local-only complaints * Update internal/store/proxy_test.go * revert generate on run-server * avoid merged store unless needed Co-authored-by: Ben Schumacher <[email protected]> Co-authored-by: mattermod <[email protected]> * Ensure manually added plugins stay in the database (#68) * Send Announcements for new plugins (#76) * Limit enterprise plugins to E20 installations (#79) * Limit enterprise plugins to E20 installations * Apply suggestions from code review Co-authored-by: Aaron Rothschild <[email protected]> * Make getPlugins signature more clear * Don't omit enterprise in json * make generate * Include workaround for https://mattermost.atlassian.net/browse/MM-26507 * Fix typo * Changes as requested * Drop else * Move minVersionSupportingEnterpriseFlags into global scope Co-authored-by: Aaron Rothschild <[email protected]> * Merge production into master (#88) * Merge master into production (#49) * [MM-21749] [MM-22214] Add "Beta" and "Community" label (#37) * Add beta and community label * wording * Feedback * Make use of model.IsValid() (#44) * Pin github.com/rakyll/statik version (#47) * document automatic deployment (#46) * document automatic deployment * tweak merging language Co-authored-by: Ben Schumacher <[email protected]> * Add issue templates for community plugins (#45) Add two issue templates: one for adding a new community plugin and one for updating an existing community plugin. * [MM-23184] Omit empty labels list (#48) Co-authored-by: Jesse Hallam <[email protected]> * Add v0.14.0 of mattermost-plugin-github to the Marketplace (#51) * Add v1.3.0 of mattermost-plugin-zoom to the Marketplace (#50) * Add v1.2.0 of mattermost-plugin-custom-attributes to the Marketpl… (#53) * Add mattermost plugin autolink v1.2.0 (#56) * Add v1.2.0 of mattermost-plugin-autolink to the Marketplace * Remove custom-attributes from this PR * add the statik.go file after manually removed autolink from plugins.json and reran 'make generate' Co-authored-by: Ben Schumacher <[email protected]> * Add v1.3.1 of mattermost-plugin-zoom to the Marketplace (#59) * Add mattermost plugin todo v0.2.0 (#57) * Add v2.4.0 of mattermost-plugin-jira to the Marketplace (#66) * Add v1.0.4 of mattermost-plugin-nps to the Marketplace (#63) * Add v0.2.0 of mattermost-plugin-skype4business to the Marketplace (#70) * Add v1.0.0 of mattermost-plugin-github to the Marketplace (#69) * Add v1.2.0 of mattermost-plugin-confluence to the Marketplace (#73) * Add mattermost plugin welcomebot v1.2.0 (#71) * Add v1.0.0 of mattermost-plugin-mscalendar to the Marketplace (#78) * Add v1.1.0 of mattermost-plugin-jenkins to the Marketplace (#80) * Add v2.0.0 of mattermost-plugin-jitsi to the Marketplace (#82) * Add Enterprise label to MS Calendar * make generate * Add v1.3.0 of mattermost-plugin-giphy-moussetc to the Marketplace (#85) Co-authored-by: Jesse Hallam <[email protected]> Co-authored-by: Jason Frerich <[email protected]> * s/sls/serverless Co-authored-by: Ben Schumacher <[email protected]> Co-authored-by: mattermod <[email protected]> Co-authored-by: Aaron Rothschild <[email protected]> Co-authored-by: Jason Frerich <[email protected]>

lieut-data added 5 commits April 22, 2020 22:49

s/store/staticStore/

982a583

prefer later plugins if version exactly matches

281639d

introduce merged and proxy stores

b9ce19a

support --upstream

7f5e04c

generate on run-server

2f60182

lieut-data added the 2: Dev Review Requires review by a core committer label Apr 23, 2020

lieut-data requested review from hanzei and alifarooq0 April 23, 2020 02:51

update to go1.14

a54a1db

lieut-data added the 3: QA Review Requires review by a QA tester label Apr 23, 2020

lieut-data commented Apr 23, 2020

View reviewed changes

Makefile Outdated Show resolved Hide resolved

lieut-data added 2 commits April 23, 2020 10:51

tidy up interface/comments

5087090

golangci-lint local-only complaints

2c93291

alifarooq0 approved these changes Apr 25, 2020

View reviewed changes

internal/store/proxy_test.go Outdated Show resolved Hide resolved

Update internal/store/proxy_test.go

bc9c8c7

hanzei requested changes Apr 29, 2020

View reviewed changes

Makefile Outdated Show resolved Hide resolved

cmd/lambda/main.go Outdated Show resolved Hide resolved

lieut-data added 2 commits April 29, 2020 14:39

revert generate on run-server

bc09c08

avoid merged store unless needed

fd73eea

lieut-data requested a review from hanzei April 29, 2020 17:48

hanzei approved these changes Apr 29, 2020

View reviewed changes

hanzei removed the 2: Dev Review Requires review by a core committer label Apr 29, 2020

lieut-data requested a review from DHaussermann April 30, 2020 15:01

hanzei and others added 2 commits May 14, 2020 11:06

Merge branch 'master' into proxy-upstream

e682de7

Merge branch 'master' into proxy-upstream

ed44183

hanzei self-requested a review June 10, 2020 09:50

hanzei approved these changes Jun 10, 2020

View reviewed changes

lieut-data added 4: Reviews Complete All reviewers have approved the pull request and removed 3: QA Review Requires review by a QA tester labels Jun 10, 2020

lieut-data removed the request for review from DHaussermann June 10, 2020 12:05

lieut-data merged commit 9b5b455 into master Jun 11, 2020

lieut-data deleted the proxy-upstream branch June 11, 2020 13:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Proxy upstream support #60

Proxy upstream support #60

lieut-data commented Apr 23, 2020

alifarooq0 left a comment

hanzei left a comment

DHaussermann commented May 25, 2020

lieut-data commented May 25, 2020

DHaussermann commented Jun 9, 2020

hanzei commented Jun 10, 2020

hanzei left a comment

hanzei commented Jun 10, 2020

lieut-data commented Jun 10, 2020

DHaussermann commented Jun 11, 2020

Proxy upstream support #60

Proxy upstream support #60

Conversation

lieut-data commented Apr 23, 2020

Summary

Ticket Link

alifarooq0 left a comment

Choose a reason for hiding this comment

hanzei left a comment

Choose a reason for hiding this comment

DHaussermann commented May 25, 2020

lieut-data commented May 25, 2020

DHaussermann commented Jun 9, 2020

hanzei commented Jun 10, 2020

hanzei left a comment

Choose a reason for hiding this comment

Scope

Testing notes

Follow-up concerns

hanzei commented Jun 10, 2020

lieut-data commented Jun 10, 2020

DHaussermann commented Jun 11, 2020