Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RAM] Rule type params OAS verification and POC #179212

Draft
wants to merge 51 commits into
base: main
Choose a base branch
from
Draft

[RAM] Rule type params OAS verification and POC #179212

wants to merge 51 commits into from

Conversation

JiaweiWu
Copy link
Contributor

@JiaweiWu JiaweiWu commented Mar 22, 2024 •
edited by lcawl
Loading

Summary

Depends on: #156357
Relates to #180056

POC to verify that we are able to generate Open API documentation for rule type schemas. This PR demonstrates this with the following rule types:

  • Custom threshold
  • Metric threshold
  • ES query

To test:

  • Run yarn start
  • Once ES and kibana has started, navigate to http://localhost:5601/<3 letter sequence>/api/oas?pathStartsWith=/internal/alerting or http://localhost:5601/<<3 letter sequence>/api/oas?pathStartsWith=/api/alerting
  • Take the output and copy it to https://editor.swagger.io/
  • Open API should generate documentation based on the JSON

jloleysens and others added 30 commits May 2, 2023 13:42
@jloleysens
PoC: Generate OAS spec from versioned router declarations (elastic#15…
02ef114 
…5518)

## Summary

PoC to explore whether we can generate good/high quality OAS spec from
our new versioned router. The general idea is to bring the versioned
router interface as inline as possible with OAS 3 so that we can map
from router declarations as easily as possible.

## How to test

* See the `README.md` for local testing instructions.
* See the `run.js` file for how we can declare the route and generate
the spec.

## What this PoC does


<details>
<summary>Given router declaration like</summary>

```ts
router
  .get({
    access: 'public',
    path: '/api/cool/endpoint/{id}',
  })
  .addVersion(
    {
      version: '2023-01-01',
      description: 'My cool route',
      validate: {
        request: {
          params: z.object({ id: z.string() }),
        },
        response: {
          200: {
            body: z.object({ foo: z.string().max(5) }),
          },
          400: {
            body: z.object({ error: z.string() }),
          },
        },
      },
    },
    (ctx, req, res) => res.ok()
  )
```

</details>

<details>

<summary>Generate OAS 3 like</summary>

```json
{
  "openapi": "3.0.0",
  "info": {
    "title": "PoC",
    "version": "1.0.0"
  },
  "servers": [
    {
      "url": "http://localhost:5601"
    }
  ],
  "paths": {
    "/api/cool/endpoint/{id}": {
      "get": {
        "responses": {
          "200": {
            "description": "No description",
            "content": {
              "application/json; Elastic-Api-Version=2023-01-01": {
                "schema": {
                  "type": "object",
                  "properties": {
                    "foo": {
                      "type": "string",
                      "maxLength": 5
                    }
                  },
                  "required": [
                    "foo"
                  ],
                  "additionalProperties": false
                }
              }
            }
          },
          "400": {
            "description": "No description",
            "content": {
              "application/json; Elastic-Api-Version=2023-01-01": {
                "schema": {
                  "type": "object",
                  "properties": {
                    "error": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "error"
                  ],
                  "additionalProperties": false
                }
              }
            }
          }
        },
        "parameters": [
          {
            "name": "id",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ]
      }
    }
  }
}
```

</details>

The script for doing this will cover

- [x] Request body
- [x] Responses
- [x] URL parameters
- [ ] Query parameters (still need to test)
- [ ] What else...?

Also makes adjustments to the versioned router interface as needed.

### Potential benefits

* We can have a single schema declaration for our request, response,
parameter, query types that will stay insync across: runtime
validations, OAS spec and TypeScript
* We need to get all of our public HTTP APIs onto a Elastic version
which requires a refactor, if we plan the refactor carefully we can
extract even more value from that effort

### Potential drawbacks

* It is considered more standard to write spec and generate code, not
the other way around. The risk is that our spec is ultimately not that
useful or lacking in certain aspects. The mitigation to this concern
should be that our router interface will closely match OAS 3 and so we
should be able to get the full benefits of OAS for consumers of the
spec.

### Out of scope

* This PoC assumes that the problem of generating JSONSchema from
runtime validation is "solved" using `zod-to-json-schema` and so `zod`,
not `config-schema` which uses `joi`, was used for schema declarations.
To migrate `config-schema` to `zod` was considered out of scope and
requires a different PoC to derisk (incoming).
* Not all OAS 3 capabilities are covered. We excluded examples....
* Covering all edge cases: there are many edge cases that exist that
this version of the script does not cover but it is assumed that we will
be able to address them if the core concerns are met.

### Screenshots

<img width="1359" alt="Screenshot 2023-04-21 at 16 29 45"
src="https://user-images.githubusercontent.com/8155004/233662129-0757d04a-76d5-4e0b-9238-26cc645f0412.png">
@jloleysens @kibanamachine
@kbn/schema part of OAS PoC (elastic#156344)
9e4b520 
## Summary
Part of PoC in elastic#156357 this PR
adds a new validation lib that is a port of `@kbn/config-schema` in many
ways but also a lot simpler.

This contribution is incomplete but preserves `@kbn/config-schema` API
where possible and updates some error message reporting. The intention
is to have a minimal set of runtime types such that we can experiment
with converting existing public APIs to the versioned router using them.

---------

Co-authored-by: kibanamachine <[email protected]>
@jloleysens
[OAS PoC] Incorporate feedback round I (elastic#156367)
89f1320 
## Summary

Address
elastic#155518 (comment).
Added unique operation ID.

Addresses part of
elastic#155518 (comment).
Added placeholder security.
@jloleysens
[OAS PoC] Restructure code (elastic#156395)
d0ca55d 
## Summary

A lot of changes to get the code in a slightly more mergable state.
@jloleysens
[OAS PoC] Just use zod directly (elastic#156447)
86905e6 
## Summary

This PR pivots to exposing `zod` API directly via a new package
`@kbn/zod`.
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
d4f22b2 
…fix'
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/generate codeowners'
d4ec086
@jloleysens
Slight refactor of zod exports and added a test for union special for…
5f035a2 
…matting and error message extraction
@jloleysens
updated the oas extraction algo to take latest params and query for now
661dd85
@jloleysens
updated contributors note
f405fd9
@jloleysens
Merge branch 'main' into versioned-router-and-oas
630343b 
* main: (153 commits)
  [Security Solution] {{state.signals_count}} Object not working (elastic#156472) (elastic#156707)
  [Synthetics] refresh data on visualization scrubbing (elastic#156777)
  [RAM] Docs for slack improvements (elastic#153885)
  [RAM] Alert search bar only KQL (elastic#155947)
  [ML] Functional tests - stabilize export job tests (elastic#156586)
  [Saved Search] Update saved search schema to allow empty `sort` arrays (elastic#156769)
  [ML] Rename `curated` model type to `elastic` (elastic#156684)
  [Discover] Enable sharing for text based languages (elastic#156652)
  [api-docs] 2023-05-05 Daily api_docs build (elastic#156781)
  Upgrade EUI to v77.2.2 (elastic#155208)
  [RAM][Maintenance Window][8.8]Fix window maintenance workflow (elastic#156427)
  [DOCS] Case file attachments (elastic#156459)
  [D4C] additional error handling for 'block' action added + policy editor UI fixes (elastic#156629)
  [Enterprise Search] refactor(SearchApplications): rename telemetry ids (elastic#156733)
  [Enterprise Search] Add telemetry to ELSER deployment buttons + error (elastic#156545)
  [Security Solution] fixes Data Quality dashboard errors when a `basePath` is configured (elastic#156233)
  [Logs onboarding] StepsFooter outside of main panel (elastic#156686)
  [Security Solution] Add a migration to unmute custom Security Solution rules (elastic#156593)
  [Enterprise Search][Behavioral Analytics] Update formulas (elastic#156704)
  Add API Events to Endpoint Security Advanced Policy (elastic#156718)
  ...
@jloleysens
fix types
4cab295
@jloleysens
Merge branch 'main' into versioned-router-and-oas
22c1ca4
@jloleysens
actually pass through the body, params and query types
8d1ec9b
@jloleysens
use ZodEsque to extract _output, rather than _input
aec8677
@jloleysens
use ZodEsque to extract _output, rather than _input
4a2cc6c
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
8c40322 
…fix'
@jloleysens
make generic optional, fix type narrowing
62d4534
@jloleysens
added mock
aaff092
@jloleysens
Merge branch 'main' into versioned-router-and-oas
edc4c0e
@jloleysens
remove zod from dev deps
5235e86
@jloleysens
Merge branch 'main' into versioned-router-and-oas
e582640 
* main: (3152 commits)
  [Security Solution][Detection Engine] fixes 410 error on index legacy template call (elastic#164682)
  [SavedObjects] Create serverless roots for jest integration tests (elastic#164157)
  Create upselling package and implement EntityAnalytics serverless upselling (elastic#164136)
  [Fleet] Change 'Out-of-date' to 'Outdated policy' in agent list table (elastic#164673)
  [IndexManagement] Use internal base path for API (elastic#164665)
  [Profiling] removing ~ symbol (elastic#164595)
  [Telemetry] Fetch snapshot: allow specifying the version via querystring (elastic#164670)
  [Cases] Show warning when all cases table reaches 10k cases message (elastic#164323)
  [ML] Removing token list from text expansion model testing (elastic#164560)
  [Fleet] Add secrets package API integration test (elastic#164583)
  [Fleet] Fix security solution tag id (elastic#164582)
  [Security Solution] Modal says "duplicating 0 rules" when you duplicate an individual rule (elastic#163908)
  [api-docs] 2023-08-24 Daily api_docs build (elastic#164658)
  [APM] Cleanup alerting api tests (elastic#164438)
  Upgrade EUI to 87.2.0 (elastic#164385)
  [ML] Fix query bar autocompletion for ML and AIOps embeddables (elastic#164485)
  [Fleet] Fix flaky unit test for the details page (elastic#164641)
  [Security Solution] update codeowner for serverless security subdir (elastic#164640)
  [Security Solution] Fixes Assistant Connector and Actions RBAC Flow (elastic#164382)
  [Discover] Removing large string truncation from doc viewer (elastic#164236)
  ...
@jloleysens @kibanamachine
[OAS PoC]OAS endpoint and lazily pass schema (elastic#164710)
2f0ea25 
## Summary

Diverges a little bit from the [original
PoC](elastic#156357) but largely remains
the same. The idea is to support lazily providing HTTP route schemas and
also provide the ability to serve up OAS from an HTTP endpoint.

* expose the generate OAS function so that it can be used from an
endpoint created in core's HTTP server
* create the ability to pass schema lazily via versioned router
interface so that it can be constructed after setup/start
* use the new lazy schema option in alerting, see
`x-pack/plugins/alerting/server/routes/update_rule.ts`
* also removes the CLI `node script/generate_oas` in favor of the new
endpoint for now

Much of this can also be done for existing, non-versioned and versioned
non-zod API routes if we add support for Joi.

## TODO

- [x] Needs to be tested end-to-end, currently the lazily loaded schemas
will not be used during actual route validation

## How to test

1. Start ES & Kibana
2. Issue a request like: `curl -uelastic:changeme
http://localhost:5601/<base-path-thingy>/api/oas` (optionally format
with `jq` for legibility)
3. Should see a fairly incomplete OAS doc bc zod is not being used in
any endpoints, but you should see one of the lazily added schemas:

```json
    "/api/alerting/rule/{id}": {
      "put": {
        "requestBody": {
          "content": {
            "application/json; Elastic-Api-Version=2023-10-31": {
              "schema": {
                "type": "object",
                "properties": {
                  "data": {
                    "type": "string",
                    "description": "I arrived lazily"
                  }
                },
                "required": [
                  "data"
                ],
                "additionalProperties": false
              }
            }
          }
        },
        "responses": {},
        "parameters": [
          {
            "name": "id",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "operationId": "/api/alerting/rule/{id}#0"
      }
    }
```

---------

Co-authored-by: kibanamachine <[email protected]>
@jloleysens
zod-to-json-schema a project dep
ea5bf93
@jloleysens
Revert "lazily pass schemas from alerting"
9e8d18f 
This reverts commit f230548.
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
14a9f1d 
…fix'
@jloleysens
Merge branch 'main' into versioned-router-and-oas
912d763 
* main: (4129 commits)
  [Logs Explorer] Change the default link for "Discover" in the serverless nav (elastic#173420)
  [Fleet] fix unhandled error in agent details when components are missing (elastic#174152)
  [Obs UX] Unskip transaction duration alerts test (elastic#174069)
  [Fleet] Fix keep policies up to date after package install (elastic#174093)
  [Profiling] Stack traces embeddable (elastic#173905)
  [main] Sync bundled packages with Package Storage (elastic#174115)
  [SLO Form] Refactor to use kibana data view component (elastic#173513)
  [Obs UX] Unskip APM Service Inventory Journey (elastic#173510)
  [Obs UX] Unskip preview_chart_error_count test (elastic#173624)
  [api-docs] 2024-01-03 Daily api_docs build (elastic#174142)
  Update babel runtime helpers (elastic#171745)
  Handle content stream errors in report pre-deletion (elastic#173792)
  [Cloud Posture] [Quick wins] Enable edit DataView on the Misconfigurations Findings Table (elastic#173870)
  [ftr] abort retry on invalid webdriver session (elastic#174092)
  Upgrade openai to 4.24.1 (elastic#173934)
  chore(NA): bump node into v20 (elastic#173461)
  [Security Solution][Endpoint] Fix index name pattern in SentinelOne dev. script (elastic#174105)
  fix versions.json
  [Obs AI Assistant] Add guardrails (elastic#174060)
  [ML] Transforms: Refactor validators and add unit tests. (elastic#173736)
  ...
@jloleysens
reformatting of package, tsconfig and lock files
c8899b7
@jloleysens
refactor to support other runtime validation libs
1a291eb
@jloleysens
minor cleanup
ed56ee8
jloleysens and others added 21 commits January 3, 2024 14:22
@jloleysens
fix type
d7a505e
@jloleysens
support joi AND zod schemas
3aa1c81
@jloleysens
add support for descriptions
3007b45
@jloleysens
Merge branch 'main' into versioned-router-and-oas
d52eff6
@jloleysens
support non-versioned router
a8bfeeb
@jloleysens
added support for path filtering and adding response schemas to `Rout…
a453f97 
…er` routes
@jloleysens
fixed OAS shape for Router output
1de1e12
@jloleysens
remove noise from resulting schema
3356e47
@jloleysens
also traverse schema array containers
6a086c7
@jloleysens
populate required field and some other fixes
03574f2
@jloleysens
fix return
b57f994
@jloleysens
fix determining optional path params logic
281d946
@jloleysens
Merge branch 'main' into versioned-router-and-oas
f5151f6 
* main: (520 commits)
  Update Kibana code editor dependencies (elastic#171720)
  [SLOs] Hide view in app in slo alerts table in slo details page (elastic#175441)
  [api-docs] 2024-01-25 Daily api_docs build (elastic#175502)
  [DOCS] Add buildkite links to doc preview comments (elastic#175463)
  skip flaky suite (elastic#175443)
  [Security Solution][Timeline] refactor timeline modal save timeline button (elastic#175343)
  [RAM] Stack Management::Rules loses user selections when navigating back (elastic#174954)
  [Security Solution][Timeline] refactor timeline modal attach to case button (elastic#175163)
  Upgrade EUI to v92.1.1 (elastic#174955)
  [Fleet]: Beta label is shown inconsistently while selecting proxy under Fleet settings. (elastic#170634)
  [Cloud Security] Rules Combo Box filters Custom component (elastic#175175)
  skip flaky suite (elastic#175407)
  [Security Solution][Timeline] refactor timeline modal open timeline button (elastic#175335)
  [Embedded Console] Introduce kbnSolutionNavOffset CSS variable (elastic#175348)
  [Console] disable access to embedded console without dev tools capability (elastic#175321)
  fix(x-pack/reporting): use FIPS-compliant ID generator `uuidv4` in Reporting plugin (elastic#174809)
  [Security Solution] Data quality dashboard persistence (elastic#173185)
  [RAM][Observability] Add alert fields table to Observability flyout (elastic#174685)
  test: add missing await for connector table disappearance (elastic#175430)
  [RAM][Maintenance Window] Fix maintenance window FE types and transforms  (elastic#173888)
  ...
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
411c8e6 
…fix'
@jloleysens
Merge branch 'main' into versioned-router-and-oas
87d38f7
@jloleysens
Merge branch 'main' into versioned-router-and-oas
369df9b
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/generate codeowners'
af4b0a0
@jloleysens
Merge branch 'main' into versioned-router-and-oas
32ba707 
* main: (224 commits)
  [Http] Replace `buildNr` with `buildSha` in static asset paths (elastic#175898)
  [Ops] Fix GCS bucket access for future buildkite agents (elastic#174756)
  [api-docs] 2024-02-07 Daily api_docs build (elastic#176362)
  skip flaky suite (elastic#176002)
  skip failing es promotion suite (elastic#176359)
  [Cloud Security] [Grouping] Add URL Params support to the grouping components (elastic#175749)
  chore(NA): update versions after v8.12.2 bump (elastic#176309)
  chore(NA): update versions after v7.17.19 bump (elastic#176313)
  skip failing test suite (elastic#176352)
  [SLO] Enable burn rate alert by default during creation via UI (elastic#176317)
  [Fleet] Add the uptime capability to observability projects (elastic#176285)
  [Security Solution][Endpoint] Fix Manifest Manger so that it works with large (>10k) (elastic#174411)
  [ResponseOps] Alert creation delay based on user definition (elastic#175851)
  [data views] Default field formatters based on field meta values (elastic#174973)
  [Cloud Security]Detection Rules counter on Rules Flyout (elastic#176041)
  [Security Solution] Data Quality Dashboard persistence (elastic#175673)
  [Ent Search] Connector client copy cleanup (elastic#176290)
  [ML] Anomaly Detection: Adds actions menu to anomaly markers in Single Metric Viewer chart. (elastic#175556)
  [ML] Anomaly Detection: Fix `values-dots` colors (elastic#176303)
  [Fleet] Logstash Output - being compliant to RFC-952 (elastic#176298)
  ...
@JiaweiWu
Merge branch 'main' into versioned-router-and-oas-clone
0a6260a
@JiaweiWu
Merge branch 'main' into issue-178014-oas-verification
d021d35
@JiaweiWu
Draft and WIP
ab6a25b
@JiaweiWu JiaweiWu mentioned this pull request Mar 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JiaweiWu @jloleysens @kibanamachine