Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugins #1828

Closed
wants to merge 136 commits into from
Closed

Plugins #1828

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
136 commits
Select commit Hold shift + click to select a range
ffb974c
push up ABC class
fuziontech Oct 7, 2020
5db7fc8
Merge branches 'plugins' and 'master' of github.com:PostHog/posthog i…
fuziontech Oct 7, 2020
9cd8318
plugin progress
fuziontech Oct 7, 2020
41076d7
blah
fuziontech Oct 7, 2020
66fed09
add posthog config for plugins
fuziontech Oct 7, 2020
862a7d8
test gitignore
fuziontech Oct 7, 2020
bbfc091
new functionality for plugins
fuziontech Oct 7, 2020
632956e
support local plugin paths
mariusandra Oct 7, 2020
1506306
also ignore symlinks
mariusandra Oct 7, 2020
eceac05
add positional argument
mariusandra Oct 7, 2020
a7a10d3
fixes
mariusandra Oct 7, 2020
59736dd
small fixes
mariusandra Oct 7, 2020
28fa6f5
config polish
fuziontech Oct 8, 2020
1bc6a7a
config passed to posthog plugin
fuziontech Oct 8, 2020
6381322
ooooooops
fuziontech Oct 8, 2020
a4c211e
symlink fix
fuziontech Oct 8, 2020
a10a607
cleanse dir before loading
fuziontech Oct 8, 2020
fc40fe3
add cache to plugins
fuziontech Oct 8, 2020
2e29a72
pickle the goods
fuziontech Oct 8, 2020
b5f6552
unlink symlink
mariusandra Oct 8, 2020
ace5375
pass full config
fuziontech Oct 8, 2020
caf1e86
Merge branch 'plugins' of github.com:PostHog/posthog into plugins
fuziontech Oct 8, 2020
4d33754
unlink even if link points to nothing
mariusandra Oct 8, 2020
d7d24d4
Merge branch 'plugins' of github.com:PostHog/posthog into plugins
mariusandra Oct 8, 2020
a1f7050
fix fix
mariusandra Oct 8, 2020
41e880d
return none if value is empty
fuziontech Oct 8, 2020
3fe8fa6
Merge branch 'plugins' of github.com:PostHog/posthog into plugins
fuziontech Oct 8, 2020
9eb7142
plugin model
mariusandra Oct 8, 2020
6760803
plugins scene
mariusandra Oct 8, 2020
caac3d1
add config schema to plugins
mariusandra Oct 8, 2020
74aa457
install plugins
mariusandra Oct 8, 2020
ec250cd
save descriptions
mariusandra Oct 9, 2020
61d8245
show descriptions
mariusandra Oct 9, 2020
025860f
edit plugin
mariusandra Oct 9, 2020
87f88d1
save plugin config
mariusandra Oct 9, 2020
347cfc8
plugin modal
mariusandra Oct 9, 2020
b0ad187
uninstall plugins
mariusandra Oct 9, 2020
ebedf5b
UX cleanup
mariusandra Oct 9, 2020
7e10e0b
add "required" to plugin config
mariusandra Oct 9, 2020
0a30500
open plugin modal after install
mariusandra Oct 9, 2020
f5f96d5
split to subcomponents
mariusandra Oct 9, 2020
2d7c226
install custom plugins
mariusandra Oct 9, 2020
25dc309
rework backend for model plugins
fuziontech Oct 9, 2020
f069cd7
Plugins on models
fuziontech Oct 9, 2020
b8f5f11
Merge branch 'plugin-setup' of github.com:PostHog/posthog into plugin…
fuziontech Oct 9, 2020
e3bc924
simple reload pubsub
mariusandra Oct 9, 2020
7e36de4
Merge branch 'plugin-setup' of github.com:PostHog/posthog into plugin…
mariusandra Oct 9, 2020
5c6300d
fix apps not installed
mariusandra Oct 10, 2020
5c5caf4
fix master/main issue
mariusandra Oct 10, 2020
a4caa01
fix reload command
mariusandra Oct 10, 2020
1227d8b
use the github api to get the default branch
mariusandra Oct 10, 2020
a018abf
init plugins only if not running migrate/makemigrations
mariusandra Oct 10, 2020
c53f0b6
store plugins zip archives in postgres
mariusandra Oct 10, 2020
8463924
tag plugins to specific versions
mariusandra Oct 10, 2020
ffd9246
save plugins in pluginConfig
mariusandra Oct 10, 2020
a9e216c
update pluginConfigs instead of adding new rows, remove from redux on…
mariusandra Oct 10, 2020
a86e269
remove debug
mariusandra Oct 10, 2020
cc9c25d
run plugins from db by team
mariusandra Oct 10, 2020
21c0c14
reload when deleting
mariusandra Oct 10, 2020
622d324
remove debug
mariusandra Oct 10, 2020
d4dd514
smarter handling of dynamic plugins, support local plugins again
mariusandra Oct 13, 2020
5fea1fa
improve typings, add some nicer warnings
mariusandra Oct 13, 2020
15a0704
Merge branch 'master' into plugin-setup
mariusandra Oct 14, 2020
b80fb68
yarn lock file after merge
mariusandra Oct 14, 2020
51c1f96
squash migrations and add "locked" field to plugins
mariusandra Oct 14, 2020
9256cfb
error if folder not found in zip
mariusandra Oct 14, 2020
6c83b1c
unregister plugins
mariusandra Oct 14, 2020
cd38b1c
skip plugin init in test mode
mariusandra Oct 14, 2020
fbc987e
basic plugin test
mariusandra Oct 14, 2020
d5f2f26
avoid mutating the same prop hash
mariusandra Oct 14, 2020
925db29
add pip tools to requirements.txt
mariusandra Oct 14, 2020
249b0fe
fix mypy, fix manage.py script error
mariusandra Oct 14, 2020
de61cd3
avoid plugins with mypy
mariusandra Oct 14, 2020
a9a1485
mypy fix
mariusandra Oct 14, 2020
ceaff45
abstract redis into plugin and add team_id to reload
mariusandra Oct 14, 2020
22b6566
Merge branch 'master' into plugin-setup
mariusandra Oct 14, 2020
52b89f8
refactor and start work on syncing with posthog.json
mariusandra Oct 14, 2020
1fad2b0
start testing plugin loading from json
mariusandra Oct 14, 2020
3a74b65
test plugin deletion
mariusandra Oct 15, 2020
9651e9e
test for syncing plugins from config
mariusandra Oct 15, 2020
1a4690b
complete and then test local json plugin sync
mariusandra Oct 15, 2020
8f9cc74
test converting back and forward between an local and http path
mariusandra Oct 15, 2020
866fea5
remove global plugin config from plugins array in posthog.json
mariusandra Oct 15, 2020
2e77823
rename configSchema --> config_schema
mariusandra Oct 15, 2020
5f95bf4
Merge branch 'master' into plugin-setup
mariusandra Oct 15, 2020
d76e134
fix migration after merge
mariusandra Oct 15, 2020
47901fa
rename from_cli to from_json
mariusandra Oct 15, 2020
21c752c
mypy
mariusandra Oct 15, 2020
598b7b6
import pip after plugin loaded
mariusandra Oct 15, 2020
5e96698
show error details
mariusandra Oct 15, 2020
318d740
raise exceptions visible to the frontend
mariusandra Oct 15, 2020
fb25a47
sync plugins on load
mariusandra Oct 15, 2020
4075c2b
access control to updating plugins from the web
mariusandra Oct 15, 2020
a3da59c
access control
mariusandra Oct 15, 2020
355fc84
remove posthog.json from git
mariusandra Oct 15, 2020
c210068
test config schema from json
mariusandra Oct 15, 2020
5dbcf64
if you can install via the web, you can also configure
mariusandra Oct 15, 2020
95116ed
remove separate view access
mariusandra Oct 15, 2020
d2cda53
Merge branch 'master' into plugin-setup
mariusandra Oct 15, 2020
4ee561b
title as "Plugins" instead of "Installed Plugins" if we can't install…
mariusandra Oct 15, 2020
74f1b29
add self.team to plugin base class
mariusandra Oct 16, 2020
4002fd4
add instance_init method
mariusandra Oct 16, 2020
5329efd
refactor into files
mariusandra Oct 16, 2020
9880904
Merge branch 'master' into plugin-setup
mariusandra Oct 16, 2020
bd3ea3c
sync global plugin config from json
mariusandra Oct 16, 2020
5277078
make global plugins work, add test
mariusandra Oct 16, 2020
90c175b
global plugins in interface, make them take precedence over local plu…
mariusandra Oct 16, 2020
68bc12e
add comments to plugin base class
mariusandra Oct 16, 2020
0fea379
reload/reset plugins before each test
mariusandra Oct 16, 2020
612d515
add error field to plugins
mariusandra Oct 16, 2020
9c6baac
add many plugin zips
mariusandra Oct 16, 2020
c57613a
add many plugin zips, fix imports
mariusandra Oct 16, 2020
034a03c
store errors on plugin object and test them
mariusandra Oct 17, 2020
4e503b0
fix types
mariusandra Oct 17, 2020
2c821ed
add null to error
mariusandra Oct 17, 2020
bebd013
can be with any team ID in the test
mariusandra Oct 17, 2020
d2639d3
save problems running plugins in the plugin_config model
mariusandra Oct 17, 2020
90f62ae
Merge branch 'master' into plugin-setup
mariusandra Oct 17, 2020
b0762e6
try to create redis connection pool only once
mariusandra Oct 17, 2020
89940a6
throw if no redis
mariusandra Oct 17, 2020
63ea29e
mypy
mariusandra Oct 17, 2020
4136976
get instance inside heartbeat and not top level
mariusandra Oct 17, 2020
e5149e7
try caching pubsub
mariusandra Oct 17, 2020
6c01dfb
try pip install with -q
mariusandra Oct 19, 2020
16ef544
install pip externally
mariusandra Oct 19, 2020
717313f
remove uuid and typing, now in stdlib
mariusandra Oct 19, 2020
2890e51
more verbosity
mariusandra Oct 19, 2020
41d216c
add pip back
mariusandra Oct 19, 2020
441f2a2
catch exceptions
mariusandra Oct 19, 2020
aee8f48
quiet and no input for pip
mariusandra Oct 19, 2020
3f31fe0
check plugin reload every 10sec on new task
mariusandra Oct 19, 2020
51a4241
fix type errors
mariusandra Oct 19, 2020
d3486b4
fix requirements error message
mariusandra Oct 19, 2020
5729fd4
use repository.json
mariusandra Oct 19, 2020
a0d1b95
only load and reload plugins on workers
mariusandra Oct 19, 2020
47a5036
Merge branch 'master' into plugin-setup
mariusandra Oct 20, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
!manage.py
!gunicorn.config.py
!babel.config.js
!posthog.json
!package.json
!yarn.lock
!webpack.config.js
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,4 @@ yarn-error.log
yalc.lock
cypress/screenshots/*
docker-compose.prod.yml
posthog.json
1 change: 1 addition & 0 deletions dev.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ RUN mkdir /code/requirements/
COPY requirements/dev.txt /code/requirements/
RUN pip install -r requirements/dev.txt --compile

COPY posthog.json /code/
COPY package.json /code/
COPY yarn.lock /code/
COPY webpack.config.js /code/
Expand Down
2 changes: 1 addition & 1 deletion frontend/src/initKea.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export function initKea(): void {
<div>
<h1>Error loading "{reducerKey}".</h1>
<p className="info">Action "{actionKey}" responded with</p>
<p className="error-message">"{error.message}"</p>
<p className="error-message">"{error.message || error.detail}"</p>
</div>
)
window['Sentry'] ? window['Sentry'].captureException(error) : console.error(error)
Expand Down
10 changes: 10 additions & 0 deletions frontend/src/layout/Sidebar.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
TeamOutlined,
LockOutlined,
WalletOutlined,
ApiOutlined,
DatabaseOutlined,
} from '@ant-design/icons'
import { useActions, useValues } from 'kea'
Expand Down Expand Up @@ -64,6 +65,7 @@ const submenuOverride = {
annotations: 'settings',
billing: 'settings',
licenses: 'settings',
plugins: 'settings',
systemStatus: 'settings',
}

Expand Down Expand Up @@ -279,6 +281,14 @@ export function Sidebar({ user, sidebarCollapsed, setSidebarCollapsed }) {
<Link to={'/setup/licenses'} onClick={collapseSidebar} />
</Menu.Item>
)}

{user.plugin_access?.configure && (
<Menu.Item key="plugins" style={itemStyle} data-attr="menu-item-plugins">
<ApiOutlined />
<span className="sidebar-label">Plugins</span>
<Link to="/setup/plugins" onClick={collapseSidebar} />
</Menu.Item>
)}
</Menu.SubMenu>

<Menu.Item key="team" style={itemStyle} data-attr="menu-item-team">
Expand Down
40 changes: 40 additions & 0 deletions frontend/src/scenes/plugins/CustomPlugin.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import React from 'react'
import { Button, Col, Input, Row } from 'antd'
import { useActions, useValues } from 'kea'
import { pluginsLogic } from 'scenes/plugins/pluginsLogic'

export function CustomPlugin(): JSX.Element {
const { customPluginUrl, pluginError, loading } = useValues(pluginsLogic)
const { setCustomPluginUrl, installPlugin } = useActions(pluginsLogic)

return (
<div>
<h1 className="page-header">Install Custom Plugin</h1>
<p>
Paste the URL of the Plugin's <strong>Github Repository</strong> to install it
</p>

<Row style={{ maxWidth: 600, width: '100%' }}>
<Col style={{ flex: 1 }}>
<Input
value={customPluginUrl}
disabled={loading}
onChange={(e) => setCustomPluginUrl(e.target.value)}
placeholder="https://github.com/user/repo"
/>
</Col>
<Col>
<Button
disabled={loading}
loading={loading}
type="primary"
onClick={() => installPlugin(customPluginUrl, true)}
>
Install
</Button>
</Col>
</Row>
{pluginError ? <p style={{ color: 'var(--red)', marginTop: 10 }}>{pluginError}</p> : null}
</div>
)
}
105 changes: 105 additions & 0 deletions frontend/src/scenes/plugins/InstalledPlugins.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
import React from 'react'
import { Button, Col, Row, Table, Tooltip } from 'antd'
import { useActions, useValues } from 'kea'
import { pluginsLogic } from 'scenes/plugins/pluginsLogic'
import { GithubOutlined, CheckOutlined, ToolOutlined, PauseOutlined } from '@ant-design/icons'
import { PluginTypeWithConfig } from 'scenes/plugins/types'
import { userLogic } from 'scenes/userLogic'

function trimTag(tag: string): string {
if (tag.match(/^[a-f0-9]{40}$/)) {
return tag.substring(0, 7)
}
if (tag.length >= 20) {
return tag.substring(0, 17) + '...'
}
return tag
}

export function InstalledPlugins(): JSX.Element {
const { user } = useValues(userLogic)
const { installedPlugins, loading } = useValues(pluginsLogic)
const { editPlugin } = useActions(pluginsLogic)

const canInstall = user?.plugin_access?.install

return (
<div>
<h1 className="page-header">{canInstall ? 'Installed Plugins' : 'Plugins'}</h1>
<Table
data-attr="plugins-table"
size="small"
rowKey={(plugin) => plugin.name}
pagination={{ pageSize: 99999, hideOnSinglePage: true }}
dataSource={installedPlugins}
columns={[
{
title: 'Plugin',
key: 'name',
render: function RenderPlugin(plugin: PluginTypeWithConfig): JSX.Element {
return (
<>
<Row>
<Col>
<strong>{plugin.name}</strong>
</Col>
</Row>
<Row gutter={16}>
<Col>
{plugin.pluginConfig?.enabled ? (
<div style={{ color: 'var(--green)' }}>
<CheckOutlined />{' '}
{plugin.pluginConfig?.global ? 'Globally Enabled' : 'Enabled'}
</div>
) : (
<div style={{ color: 'var(--orange)' }}>
<PauseOutlined /> Disabled
</div>
)}
</Col>
<Col>
{!plugin.url?.startsWith('file:') && (
<a
href={`${plugin.url}/tree/${plugin.tag}`}
target="_blank"
rel="noreferrer noopener"
>
<GithubOutlined /> {trimTag(plugin.tag)}
</a>
)}
</Col>
</Row>
</>
)
},
},
{
title: 'Description',
key: 'description',
render: function RenderDescription(plugin: PluginTypeWithConfig): JSX.Element {
return <div>{plugin.description}</div>
},
},
{
title: '',
key: 'config',
align: 'right',
render: function RenderConfig(plugin: PluginTypeWithConfig): JSX.Element | null {
return !plugin.pluginConfig?.global ? (
<Tooltip title="Configure">
<Button
type="primary"
icon={<ToolOutlined />}
onClick={() => editPlugin(plugin.id)}
/>
</Tooltip>
) : null
},
},
]}
loading={loading}
locale={{ emptyText: 'No Plugins Installed!' }}
/>
</div>
)
}
87 changes: 87 additions & 0 deletions frontend/src/scenes/plugins/PluginModal.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
import React, { useEffect } from 'react'
import { useActions, useValues } from 'kea'
import { pluginsLogic } from 'scenes/plugins/pluginsLogic'
import { Button, Form, Input, Modal, Popconfirm, Switch } from 'antd'
import { DeleteOutlined } from '@ant-design/icons'
import { userLogic } from 'scenes/userLogic'

export function PluginModal(): JSX.Element {
const { user } = useValues(userLogic)
const { editingPlugin, pluginsLoading } = useValues(pluginsLogic)
const { editPlugin, savePluginConfig, uninstallPlugin } = useActions(pluginsLogic)
const [form] = Form.useForm()

const canDelete = user?.plugin_access?.install && !editingPlugin?.from_json

useEffect(() => {
if (editingPlugin) {
form.setFieldsValue({
...(editingPlugin.pluginConfig.config || {}),
__enabled: editingPlugin.pluginConfig.enabled,
})
} else {
form.resetFields()
}
}, [editingPlugin?.name])

return (
<Modal
forceRender={true}
visible={!!editingPlugin}
okText="Save"
onOk={() => form.submit()}
onCancel={() => editPlugin(null)}
confirmLoading={pluginsLoading}
footer={
<>
{canDelete && (
<Popconfirm
placement="topLeft"
title="Are you sure you wish to uninstall this plugin?"
onConfirm={editingPlugin ? () => uninstallPlugin(editingPlugin.name) : () => {}}
okText="Yes"
cancelText="No"
>
<Button style={{ color: 'var(--red)', float: 'left' }} type="link">
<DeleteOutlined /> Uninstall
</Button>
</Popconfirm>
)}
<Button onClick={() => editPlugin(null)}>Cancel</Button>
<Button type="primary" loading={pluginsLoading} onClick={() => form.submit()}>
Save
</Button>
</>
}
>
<Form form={form} layout="vertical" name="basic" onFinish={savePluginConfig}>
{editingPlugin ? (
<div>
<h2>{editingPlugin.name}</h2>
<p>{editingPlugin.description}</p>

<Form.Item label="Enabled?" fieldKey="__enabled" name="__enabled" valuePropName="checked">
<Switch />
</Form.Item>
{Object.keys(editingPlugin.config_schema).map((configKey) => (
<Form.Item
key={configKey}
label={editingPlugin.config_schema[configKey].name || configKey}
name={configKey}
required={editingPlugin.config_schema[configKey].required}
rules={[
{
required: editingPlugin.config_schema[configKey].required,
message: 'Please enter a value!',
},
]}
>
<Input />
</Form.Item>
))}
</div>
) : null}
</Form>
</Modal>
)
}
43 changes: 43 additions & 0 deletions frontend/src/scenes/plugins/Plugins.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import React, { useEffect } from 'react'
import { hot } from 'react-hot-loader/root'
import { PluginModal } from 'scenes/plugins/PluginModal'
import { CustomPlugin } from 'scenes/plugins/CustomPlugin'
import { Repository } from 'scenes/plugins/Repository'
import { InstalledPlugins } from 'scenes/plugins/InstalledPlugins'
import { useValues } from 'kea'
import { userLogic } from 'scenes/userLogic'

export const Plugins = hot(_Plugins)
function _Plugins(): JSX.Element {
const { user } = useValues(userLogic)

if (!user) {
return <div />
}

if (!user?.plugin_access?.configure) {
useEffect(() => {
window.location.href = '/'
}, [])
return <div />
}

return (
<div>
<InstalledPlugins />

{user.plugin_access?.install ? (
<>
<br />
<br />
<Repository />
<br />
<br />
<CustomPlugin />
</>
) : null}

<PluginModal />
</div>
)
}
Loading