Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Notifications): new notifications FE plugin, API and backend #933

Merged
merged 102 commits into from
Jan 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
5135a83
draft
ydayagi Sep 20, 2023
aa92561
feat(parodos-notifications): parodos plugin for notifications
mareklibra Sep 22, 2023
d3a0ef4
chore(parodos-notifications): add plugin metadata
mareklibra Sep 22, 2023
310dace
Add Parodos to the left-side menu
mareklibra Sep 25, 2023
79eb172
Add ParodosNotificationsTable
mareklibra Sep 25, 2023
b7d345d
Remove page header
mareklibra Sep 25, 2023
dcf37bd
Rename plugin to @backstage/plugin-parodos-notifications
mareklibra Sep 27, 2023
21f8da4
Fix types in myplugin-backend
mareklibra Sep 27, 2023
be31b23
Hotfix Knex type
mareklibra Sep 27, 2023
ca393a7
chore(parodos-notifications): Add tsconfig and turbo to the backend p…
mareklibra Oct 2, 2023
8676342
Fix typo in constants
mareklibra Oct 2, 2023
c3f6dd2
Add DB mock for backend tests
mareklibra Oct 2, 2023
92b922c
Read notifications from backend
mareklibra Oct 2, 2023
b0f4227
Add rxjs dependency
mareklibra Oct 4, 2023
b0e8e4a
First Notifaction API with Observable
mareklibra Oct 7, 2023
3fe07c0
Rename backend plugin to notifications-backend
mareklibra Oct 9, 2023
5dde218
Reimplement the Notifications to use tabs
mareklibra Oct 8, 2023
6289a8e
Fix endless "loading" state for an empty notifications list
mareklibra Oct 9, 2023
6c6a0ca
Rename front-end plugin to notifications-frontend
mareklibra Oct 9, 2023
abc2e25
Add coverage and passWithNoTests to notifications-frontend
mareklibra Nov 1, 2023
f1213ec
filter paging users
ydayagi Oct 24, 2023
0d89d4b
fix flpath-659
ydayagi Oct 30, 2023
1e88bd6
Add patternfly/react-core among dependencies
mareklibra Oct 11, 2023
e4ec935
Remove rxjs dependency
mareklibra Oct 13, 2023
c255312
Add new fields to the UI
mareklibra Oct 30, 2023
f3360d4
Rename System notifications tab
mareklibra Oct 17, 2023
2e81e86
Use Item instead of Group for the left-side menu
mareklibra Oct 17, 2023
c4bf49b
Mark "unread" notifications in the left-side menu
mareklibra Oct 17, 2023
44fb5fa
Update README to match recent endpoints
mareklibra Oct 30, 2023
5deadd4
Add default sorting
mareklibra Oct 30, 2023
94ebe5b
Add pagination
mareklibra Oct 31, 2023
20c7d2d
Remove TODO
mareklibra Nov 2, 2023
1cac6ae
FLPATH-661: Add notification actions
mareklibra Nov 2, 2023
8df95d8
Make the actions.title mandatory
mareklibra Nov 2, 2023
20a00d7
BE- implement posting to users and groups
ydayagi Nov 6, 2023
ed6c91e
FLPATH-670: Add filtering by text
mareklibra Nov 2, 2023
4a4843e
Remove unused code - FE notificationsService.ts
mareklibra Nov 7, 2023
dd94a30
Add @material-table/core among dependencies
mareklibra Nov 5, 2023
d41b7f1
FLPATH-671: Add Created After filter
mareklibra Nov 5, 2023
eb93b95
FLPATH-671: Add Created After selection
mareklibra Nov 5, 2023
c7f5d72
FLPATH-668: Add sorting
mareklibra Nov 6, 2023
7454fb3
Add stub for passing users
mareklibra Nov 7, 2023
b9a698d
Move sorting params validation to handler
mareklibra Nov 7, 2023
7621931
Polling in NotificationsSidebarItem can be configured
mareklibra Nov 7, 2023
1843ae1
FLPATH-675: Show system-wide notifications in the Updates tab
mareklibra Nov 7, 2023
5e9ccfa
FLPATH-675: Show system-wide alerts
mareklibra Nov 7, 2023
a3f89c3
update README
ydayagi Nov 7, 2023
925af87
Simplify BE cehck for orderBy
mareklibra Nov 8, 2023
49e725d
Avoid using Patternfly in the Notifications
mareklibra Nov 8, 2023
cc11068
Add react-router-dom to notifications-frontend
mareklibra Nov 8, 2023
cdbf284
Implement creating of notifications
mareklibra Nov 8, 2023
af6e3c1
Add a page to create notifications
mareklibra Nov 8, 2023
b6a58db
BE - implement mark as read
ydayagi Nov 8, 2023
9a6d9bc
FLPATH-664: Implement Mark as Read on FE
mareklibra Nov 9, 2023
b21161d
FLPATH-663: Pass logged-in user name to the backend
mareklibra Nov 10, 2023
746ccb9
Add defaut/guest user to the Catalog
mareklibra Nov 10, 2023
f3cd213
Update README for authentication section
mareklibra Nov 10, 2023
be6a6de
FLPATH-667: Add notifications-common library
mareklibra Nov 12, 2023
4b760ef
Add more descriptions
mareklibra Nov 13, 2023
a7b76be
Fix package.json after rebase
mareklibra Nov 13, 2023
8974f9e
Sync dependencies with main
mareklibra Nov 13, 2023
e804cbe
add defaults for GET requests
ydayagi Nov 13, 2023
68ba2b8
add indexes to db columns
ydayagi Nov 20, 2023
316da2b
FLPATH-738: render actions horizontally
mareklibra Nov 23, 2023
cb080ad
FLPATH-738: use a dropdown for the Unread filter
mareklibra Nov 23, 2023
b6ebe64
FLPATH-738: Change "mark as read" icons
mareklibra Nov 23, 2023
7692af4
openapi yaml for rest api
ydayagi Nov 26, 2023
56b1a89
create DB client in BE that supports all Knex configuration
ydayagi Nov 20, 2023
66128e0
add cascade to foreign key in actions table
ydayagi Nov 20, 2023
1c7e3ce
Add debouncing to the search filter
mareklibra Nov 25, 2023
6bbb154
add date-time format to time fields
ydayagi Nov 27, 2023
3a3abde
Make 'count' required in response
mareklibra Nov 27, 2023
b61120c
Fix required action fields in the OpenApi spec
mareklibra Nov 27, 2023
fb4eb1e
Add openapi client generator to the notifications-frontend
mareklibra Nov 27, 2023
65885eb
Regenerate openapi client in the frontend
mareklibra Nov 27, 2023
6d3cca7
Remove notifications-common from FE dependencies
mareklibra Nov 27, 2023
1700a09
Update frontend to use generated openapi calls
mareklibra Nov 27, 2023
6d3a876
Add ts-nocheck to generated OpenApi code
mareklibra Nov 27, 2023
e522111
Regenerate openapi
mareklibra Nov 27, 2023
c5aadd4
remove notifications-common
ydayagi Nov 28, 2023
1c3a953
Notifications-frontend install instructions and deregister
mareklibra Nov 28, 2023
f83bb49
Notifications-backend install instructions and deregister
mareklibra Nov 28, 2023
f30baf8
create router in BE based on openapi spec
ydayagi Dec 11, 2023
2f1bcbd
fix getNotificationsCount to return type number
ydayagi Dec 14, 2023
bd8a14c
support sqllite and other DBs
ydayagi Dec 21, 2023
5e1cb73
FLPATH-809: get the user via Identity API
mareklibra Dec 18, 2023
75b49cf
chore(notifications-backend): add permissions related deps
mareklibra Dec 18, 2023
3091502
chore: update README for authentication
mareklibra Dec 19, 2023
e0509ba
chore: make service-to-service checks optional
mareklibra Dec 19, 2023
7cbd6f1
chore: reduce notifications "read" permissions count
mareklibra Dec 19, 2023
357abb6
chore: separate getLoggedInUser and checkPermission funcs
mareklibra Dec 20, 2023
08d0a4e
Introduce auth.ts and shared secret for external callers
mareklibra Dec 20, 2023
eddd23b
chore: update README for SQLite
mareklibra Jan 4, 2024
48bb8e4
add unit test to BE
ydayagi Dec 25, 2023
f873e7a
chore: docs linter
mareklibra Jan 4, 2024
9a397b1
chore: use default/guest for external calls
mareklibra Jan 10, 2024
c345e82
chore: fix handling of promises in tests
mareklibra Jan 11, 2024
2bdac10
chore: remove forgotten router-test-ts
mareklibra Jan 11, 2024
56c05f7
chore: update yarn.lock after rebase
mareklibra Jan 11, 2024
dd61eb9
chore: add --coverage to tests to pass the CI
mareklibra Jan 11, 2024
8923d90
chore: hotfix for types mismatch for KNex MockClient
mareklibra Jan 11, 2024
9f477ff
chore: unify dep versions in notifications-backend
mareklibra Jan 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ yarn.lock @janus-idp/maintainers-plugins
/plugins/tekton/ @janus-idp/maintainers-plugins @debsmita1 @divyanshiGupta
/plugins/rbac-backend/ @janus-idp/maintainers-plugins @gorkem @AndrienkoAleksandr @PatAKnight
/plugins/rbac-common/ @janus-idp/maintainers-plugins @gorkem @AndrienkoAleksandr @PatAKnight

/plugins/notifications-frontend @janus-idp/maintainers-plugins @mareklibra
1 change: 0 additions & 1 deletion app-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ backend:
# This is for local development only, it is not recommended to use this in production
# The production database configuration is stored in app-config.production.yaml
database:
client: better-sqlite3
connection: ':memory:'
cache:
store: memory
Expand Down
1 change: 1 addition & 0 deletions packages/app/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
"@backstage/plugin-catalog-import": "^0.10.1",
"@backstage/plugin-catalog-react": "^1.8.5",
"@backstage/plugin-github-actions": "^0.6.6",
"@backstage/plugin-notifications-frontend": "0.1.0",
"@backstage/plugin-org": "^0.6.15",
"@backstage/plugin-permission-react": "^0.4.16",
"@backstage/plugin-scaffolder": "^1.15.1",
Expand Down
1 change: 1 addition & 0 deletions packages/app/src/components/Root/Root.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ export const Root = ({ children }: PropsWithChildren<{}>) => (
<SidebarItem icon={MapIcon} to="tech-radar" text="Tech Radar" />
</SidebarScrollWrapper>
</SidebarGroup>
<SidebarDivider />
<SidebarSpace />
<SidebarDivider />
<SidebarGroup
Expand Down
1 change: 1 addition & 0 deletions packages/backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
"@backstage/plugin-auth-backend": "^0.19.3",
"@backstage/plugin-auth-node": "^0.4.0",
"@backstage/plugin-catalog-backend": "^1.14.0",
"@backstage/plugin-notifications-backend": "^0.1.0",
"@backstage/plugin-permission-backend": "^0.5.29",
"@backstage/plugin-permission-common": "^0.7.9",
"@backstage/plugin-permission-node": "^0.7.17",
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"build": "backstage-cli package build",
"lint": "backstage-cli package lint",
"tsc": "tsc",
"test": "backstage-cli package test",
"test": "backstage-cli package test --passWithNoTests --coverage",
"clean": "backstage-cli package clean",
"start": "nodemon --"
},
Expand Down
2 changes: 1 addition & 1 deletion plugins/matomo/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"postversion": "yarn run export-dynamic",
"prepack": "backstage-cli package prepack",
"start": "backstage-cli package start",
"test": "backstage-cli package test",
"test": "backstage-cli package test --passWithNoTests --coverage",
"tsc": "tsc"
},
"dependencies": {
Expand Down
1 change: 1 addition & 0 deletions plugins/notifications-backend/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);
304 changes: 304 additions & 0 deletions plugins/notifications-backend/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,304 @@
# Notifications

This Backstage backend plugin provides REST API endpoint for the notifications.

It's backed by a relational database, so far tested with PostgreSQL.

## Getting started

The plugin uses a relational database to persist messages, it has been tested with the SQLite and PostgreSQL.

Upon backend's plugin start, the `backstage_plugin_notifications` database and its tables are created automatically.

### Optional: PostgreSQL

**To use the Backstage's default SQLite, no specific configuration is needed.**

Following steps describe requirements for PostgreSQL:

- Install [PostgresSQL DB](https://www.postgresql.org/download/)
- Configure Postgres for tcp/ip
Open Postgres conf file for editing:

```bash
sudo vi /var/lib/pgsql/data/pg_hba.conf
```

Add this line:

```bash
host all postgres 127.0.0.1/32 password
```

- Start Postgres server:

```bash
sudo systemctl enable --now postgresql.service
```

#### Backstage's configuration for PostgreSQL

If PostgreSQL is used, additional configuration in the in the `app-config.yaml` or `app-config.local.yaml` is needed. Example:

```
database:
client: pg
connection:
host: 127.0.0.1
port: 5432
user: postgres
password: your_secret
knexConfig:
pool:
min: 3
max: 12
acquireTimeoutMillis: 60000
idleTimeoutMillis: 60000
cache:
store: memory
```

### Add NPM dependency

```
cd packages/backend
yarn add @backstage/plugin-notifications-backend
```

### Add backend-plugin

Create `packages/backend/src/plugins/notifications.ts` with following content:

```
import { CatalogClient } from '@backstage/catalog-client';
import { createRouter } from '@backstage/plugin-notifications-backend';

import { Router } from 'express';

import { PluginEnvironment } from '../types';

export default async function createPlugin(
env: PluginEnvironment,
): Promise<Router> {
const catalogClient = new CatalogClient({ discoveryApi: env.discovery });
const dbConfig = env.config.getConfig('backend.database');
// Following is optional
const externalCallerSecret = env.config.getOptionalString(
'notifications.externalCallerSecret',
);

return await createRouter({
identity: env.identity,
logger: env.logger,
permissions: env.permissions,
tokenManager: env.tokenManager,
dbConfig,
catalogClient,
externalCallerSecret,
});
}
```

### Add to router

In the `packages/backend/src/index.ts`:

```
import notifications from './plugins/notifications';
...
{/* Existing code for reference: */}
const apiRouter = Router();
...
{/* New code: */}
const notificationsEnv = useHotMemoize(module, () =>
createEnv('notifications'),
);
apiRouter.use('/notifications', await notifications(notificationsEnv));
```

### Configure

#### Optional: Plugin's configuration

If you have issues to create valid JWT tokens by an external caller, use following option to bypass the service-to-service configuration for them:

```
notifications:
# Workaround for issues with external caller JWT token creation.
# When following config option is not provided and the request "authentication" header is missing, the request is ALLOWED by default
# When following option is present, the request must contain either a valid JWT token or that provided shared secret in the "notifications-secret" header
externalCallerSecret: your-secret-token-shared-with-external-services
```

Mind using HTTPS to help preventing leaking the shared secret.

Example of the request then:

```
curl -X POST http://localhost:7007/api/notifications/notifications -H "Content-Type: application/json" -H "notifications-secret: your-secret-token-shared-with-external-services" -d '{"title":"my-title","origin":"my-origin","message":"message one","topic":"my-topic"}'

```

Notes:

- The `externalCallerSecret` is an workaround, exclusive use of JWT tokens will probably be required in the future.
- Sharing the same shared secret with the "auth.secret" option is not recommended.

#### Authentication

Please refer https://backstage.io/docs/auth/ to set-up authentication.

The Notifications flows are based on the identity of the user.

All `targetUsers`, `targetGroups`` or signed-in users receiving notifications must have corresponding entities created in the Catalog.
Refer https://backstage.io/docs/auth/identity-resolver for details.

For the purpose of development, there is `users.yaml` listing example data created.

#### Authorization

Every service endpoint is guarded by a permission check, enabled by default.

It is up to particular deployment to provide corresponding permission policies based on https://backstage.io/docs/permissions/writing-a-policy. To register your permission policies, refer https://backstage.io/docs/permissions/getting-started#integrating-the-permission-framework-with-your-backstage-instance.

#### Service-to-service and External Calls

The notification-backend is expected to be called by FE plugins (including the notifications-frontend), other backend plugins or external services.

To configure those two flows, refer

- https://backstage.io/docs/auth/service-to-service-auth.
- https://backstage.io/docs/auth/service-to-service-auth#usage-in-external-callers

#### Catalog

The notifications require target users or groups (as receivers) to be listed in the Catalog.

As an example how to do it, add following to the config:

```
catalog:
import:
entityFilename: catalog-info.yaml
pullRequestBranchName: backstage-integration
rules:
# *** Here is new change:
- allow: [Component, System, API, Resource, Location, User, Group]
locations:
# Local example data, file locations are relative to the backend process, typically `packages/backend`
- type: file
# *** Here is new change, referes to a file stored in the root of the Backstage:
target: ../../users.yaml
```

The example list of users is stored in the `plugins/notifications-backend/users.yaml` and can be copied to the root of the Backstage for development purposes.

## REST API

See `src/openapi.yaml` for full OpenAPI spec.

### Posting a notification

A notification without target users or groups is considered a system notification. That means it is intended for all users (listed among Updates in the UI).

Request (User message and then system message):

```bash
curl -X POST http://localhost:7007/api/notifications/notifications -H "Content-Type: application/json" -d '{"title": "My message title", "message": "I have nothing to say", "origin": "my-origin", "targetUsers": ["jdoe"], "targetGroups": ["jdoe"], "actions": [{"title": "my-title", "url": "http://foo.bar"}, {"title": "another action", "url": "https://foo.foo.bar"}]}'
```

```bash
curl -X POST http://localhost:7007/api/notifications/notifications -H "Content-Type: application/json" -d '{"title": "My message title", "message": "I have nothing to say", "origin": "my-origin", "actions": [{"title": "my-title", "url": "http://foo.bar"}, {"title": "another action", "url": "https://foo.foo.bar"}]}'
```

Optionally add `-H "Authorization: Bearer eyJh.....` with a valid JWT token if the service-to-service authorization is enabled (see above).

Response:

```json
{ "msgid": "2daac6ff-3aaa-420d-b755-d94e54248310" }
```

### Get notifications

Page number starts at '1'. Page number '0' along with page size '0' means no paging.
User parameter is mandatory because it is needed for message status and filtering (read/unread).

Query parameters:

- `pageSize`. 0 means no paging.
- `pageNumber`. first page is 1. 0 means no paging.
- `orderBy`.
- `orderByDirec`. asc/desc
- `containsText`. filter title and message containing this text (case insensitive)
- `createdAfter`. fetch notifications created after this point in time
- `messageScope`. all/user/system. fetch notifications intended for specific user or system notifications or both
- `read`. true/false (read/unread)

Request:

```bash
curl 'http://localhost:7007/api/notifications/notifications?read=false&pageNumber=0&pageSize=0'
```

Response:

```json
[
{
"id": "2daac6ff-3aaa-420d-b755-d94e54248310",
"created": "2023-10-30T13:48:34.931Z",
"isSystem": false,
"readByUser": false,
"origin": "my-origin",
"title": "My title",
"message": "I have nothing to tell",
"topic": "my-topic",
"actions": []
}
]
```

### Get count of notifications

User parameter is mandatory because it is needed for filtering (read/unread).

**Important: Logged-in user:**

The query requires a signed-in user whose entity is listed in the Catalog.
With this condition is met, the HTTP `Authorization` header contains a JWT token with the user's identity.

Optionally add `-H "Authorization: Bearer eyJh.....` with a valid JWT token to the `curl` commands bellow.

Query parameters:

- `containsText`. filter title and message containing this text (case insensitive)
- `createdAfter`. fetch notifications created after this point in time
- `messageScope`. all/user/system. fetch notifications intended for specific user or system notifications or both
- `read`. true/false (read/unread)

Request:

```bash
curl http://localhost:7007/api/notifications/notifications/count
```

Response:

```json
{ "count": "1" }
```

### Set notification as read/unread

Request:

```bash
curl -X PUT 'http://localhost:7007/api/notifications/notifications/read?messageID=48bbf896-4b7c-4b68-a446-246b6a801000&read=true'
```

Response: just HTTP status

## Building a client for the API

We supply an Open API spec YAML file: openapi.yaml.
Loading