Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency usage CLI #198920

Merged
merged 15 commits into from
Nov 25, 2024
Merged

Conversation

elena-shostak
Copy link
Contributor

@elena-shostak elena-shostak commented Nov 5, 2024

Summary

dependency-cruiser is used for building dependency graph.

Show all dependencies for a specific package/plugin or directory

Run for all plugins

bash scripts/dependency_usage.sh -p x-pack/plugins -o ./tmp/deps-result-all.json

Run for single plugin

bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -o ./tmp/deps-result-single.json

Run for multiple plugins

bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution x-pack/plugins/security -o ./tmp/deps-result-multiple.json

Run for x-pack/packages

bash scripts/dependency_usage.sh -p x-pack/packages -o ./tmp/deps-packages-1.json

Run for packages

bash scripts/dependency_usage.sh -p packages -o ./tmp/deps-packages-2.json

Benchmark

Analysis Real Time User Time Sys Time
All plugins 7m 21.126s 7m 53.099s 20.581s
Single plugin 31.360s 45.352s 2.208s
Multiple plugins 36.403s 50.563s 2.814s
x-pack/packages 6.638s 12.646s 0.654s
packages 25.744s 39.073s 2.191s

Show all packages/plugins within a directory that use a specific dependency

bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins/security_solution

Show all packages/plugins within a directory grouped by code owner

bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins -g owner

Group by code owner with adjustable collapse depth for fine-grained grouping

Fine-grained grouping:

bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 4

Collapsed grouping: groups the results under a higher-level owner (e.g., security_solution as a single group).

bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 1

Show all dependencies matching a pattern (e.g., react-*) within a package

bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -d 'react-*' -o ./tmp/result.json

Checklist

Related: #196767

@legrego legrego self-requested a review November 5, 2024 13:39
Copy link
Member

@legrego legrego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job on this so far. I like your structure and formatting of the CLI options, and I found it particularly useful to switch between the two implementations when reviewing. I left feedback based on my initial testing.

scripts/dependency-usage.sh Outdated Show resolved Hide resolved
packages/kbn-dependency-usage/kibana.jsonc Show resolved Hide resolved
packages/kbn-dependency-usage/.eslintrc.js Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
scripts/dependency-usage.sh Outdated Show resolved Hide resolved
packages/kbn-dependency-usage/src/cli.ts Outdated Show resolved Hide resolved
packages/kbn-dependency-usage/src/cli.ts Outdated Show resolved Hide resolved
packages/kbn-dependency-usage/src/cli.ts Outdated Show resolved Hide resolved
packages/kbn-dependency-usage/README.md Show resolved Hide resolved
@elena-shostak elena-shostak force-pushed the 196767-dependency-usage branch from ea06915 to e7140e0 Compare November 7, 2024 14:45
@elena-shostak
Copy link
Contributor Author

/ci

3 similar comments
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak
Copy link
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

merge conflict between base and head

@elena-shostak elena-shostak force-pushed the 196767-dependency-usage branch from a5d8bfa to 5d82424 Compare November 7, 2024 18:14
@elena-shostak
Copy link
Contributor Author

/ci

1 similar comment
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak elena-shostak force-pushed the 196767-dependency-usage branch 2 times, most recently from ad02e21 to f825ee3 Compare November 7, 2024 19:55
@elena-shostak
Copy link
Contributor Author

/ci

1 similar comment
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak elena-shostak force-pushed the 196767-dependency-usage branch from d1dc9c5 to 60103aa Compare November 8, 2024 13:08
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak elena-shostak force-pushed the 196767-dependency-usage branch from 60103aa to 17ee705 Compare November 8, 2024 14:33
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak
Copy link
Contributor Author

@elasticmachine merge upstream

@elena-shostak elena-shostak added Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! Feature:Dependencies and removed Feature:Dependencies labels Nov 12, 2024
@elena-shostak
Copy link
Contributor Author

/ci

1 similar comment
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak elena-shostak added enhancement New value added to drive a business result release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Nov 12, 2024
@elena-shostak
Copy link
Contributor Author

/ci

@elena-shostak elena-shostak force-pushed the 196767-dependency-usage branch from 59285f1 to dd8fade Compare November 20, 2024 12:47
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

API count

id before after diff
@kbn/ts-projects 39 40 +1

ESLint disabled in files

id before after diff
@kbn/dependency-usage - 1 +1

ESLint disabled line counts

id before after diff
@kbn/dependency-usage - 1 +1

Total ESLint disabled count

id before after diff
@kbn/dependency-usage - 2 +2

History

Copy link
Contributor

@jeramysoucy jeramysoucy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the additional feature! This is awesome.

@elena-shostak
Copy link
Contributor Author

@elasticmachine merge upstream

export const excludePaths = [
'(^|/)target($|/)',
'^kbn',
'^@kbn',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are these excluded from searches? don't we use kbn-* packages often as wrappers around dependencies?

Copy link
Contributor Author

@elena-shostak elena-shostak Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just excludes the outbound vertex of the dependency graph while aggregating results, since we are not interested in listing internal packages for the main output.

If we want to know which external packages the kbn-* package has we can still run

bash scripts/dependency_usage.sh -p packages/kbn-some-package

don't we use kbn-* packages often as wrappers around dependencies

Not every package is a wrapper, sometimes it's just types or development only package, like kbn-dev-utils, kbn-eslint-* etc

@elena-shostak elena-shostak merged commit 34bf83b into elastic:main Nov 25, 2024
41 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12010686533

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 26, 2024
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 198920 locally

elena-shostak added a commit to elena-shostak/kibana that referenced this pull request Nov 26, 2024
## Summary

[dependency-cruiser](https://github.com/sverweij/dependency-cruiser/tree/main)
is used for building dependency graph.

### Show all dependencies for a specific package/plugin or directory

#### Run for all plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins -o ./tmp/deps-result-all.json
```

#### Run for single plugin
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -o ./tmp/deps-result-single.json
```

#### Run for multiple plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution x-pack/plugins/security -o ./tmp/deps-result-multiple.json
```

#### Run for `x-pack/packages`
```bash
bash scripts/dependency_usage.sh -p x-pack/packages -o ./tmp/deps-packages-1.json
```

#### Run for `packages`
```bash
bash scripts/dependency_usage.sh -p packages -o ./tmp/deps-packages-2.json
```

#### Benchmark

| Analysis              | Real Time   | User Time   | Sys Time   |
|-----------------------|-------------|-------------|------------|
| All plugins           | 7m 21.126s  | 7m 53.099s  | 20.581s    |
| Single plugin         | 31.360s     | 45.352s     | 2.208s     |
| Multiple plugins      | 36.403s     | 50.563s     | 2.814s     |
| x-pack/packages       | 6.638s      | 12.646s     | 0.654s     |
| packages              | 25.744s     | 39.073s     | 2.191s     |

#### Show all packages/plugins within a directory that use a specific
dependency

```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins/security_solution
```
---
#### Show all packages/plugins within a directory grouped by code owner
```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins -g owner
```
---

#### Group by code owner with adjustable collapse depth for fine-grained
grouping
**Fine-grained grouping**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 4
```
**Collapsed grouping**: groups the results under a higher-level owner
(e.g., `security_solution` as a single group).
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 1
```
---

#### Show all dependencies matching a pattern (e.g., `react-*`) within a
package
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -d 'react-*' -o ./tmp/result.json
```

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

__Related: https://github.com/elastic/kibana/issues/196767__

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 34bf83b)

# Conflicts:
#	.github/CODEOWNERS
elena-shostak added a commit that referenced this pull request Nov 26, 2024
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Nov 26, 2024
## Summary


[dependency-cruiser](https://github.com/sverweij/dependency-cruiser/tree/main)
is used for building dependency graph.

### Show all dependencies for a specific package/plugin or directory

#### Run for all plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins -o ./tmp/deps-result-all.json
```

#### Run for single plugin
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -o ./tmp/deps-result-single.json
```

#### Run for multiple plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution x-pack/plugins/security -o ./tmp/deps-result-multiple.json
```

#### Run for `x-pack/packages`
```bash
bash scripts/dependency_usage.sh -p x-pack/packages -o ./tmp/deps-packages-1.json
```

#### Run for `packages`
```bash
bash scripts/dependency_usage.sh -p packages -o ./tmp/deps-packages-2.json
```

#### Benchmark

| Analysis              | Real Time   | User Time   | Sys Time   |
|-----------------------|-------------|-------------|------------|
| All plugins           | 7m 21.126s  | 7m 53.099s  | 20.581s    |
| Single plugin         | 31.360s     | 45.352s     | 2.208s     |
| Multiple plugins      | 36.403s     | 50.563s     | 2.814s     |
| x-pack/packages       | 6.638s      | 12.646s     | 0.654s     |
| packages              | 25.744s     | 39.073s     | 2.191s     |


#### Show all packages/plugins within a directory that use a specific
dependency

```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins/security_solution
```
---
#### Show all packages/plugins within a directory grouped by code owner
```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins -g owner
```
---

#### Group by code owner with adjustable collapse depth for fine-grained
grouping
**Fine-grained grouping**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 4
```
**Collapsed grouping**: groups the results under a higher-level owner
(e.g., `security_solution` as a single group).
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 1
```
---

#### Show all dependencies matching a pattern (e.g., `react-*`) within a
package
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -d 'react-*' -o ./tmp/result.json
```

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

__Related: https://github.com/elastic/kibana/issues/196767__

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
@elena-shostak elena-shostak removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 27, 2024
@elena-shostak
Copy link
Contributor Author

Backported in #201815

CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
## Summary


[dependency-cruiser](https://github.com/sverweij/dependency-cruiser/tree/main)
is used for building dependency graph.

### Show all dependencies for a specific package/plugin or directory

#### Run for all plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins -o ./tmp/deps-result-all.json
```

#### Run for single plugin
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -o ./tmp/deps-result-single.json
```

#### Run for multiple plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution x-pack/plugins/security -o ./tmp/deps-result-multiple.json
```

#### Run for `x-pack/packages`
```bash
bash scripts/dependency_usage.sh -p x-pack/packages -o ./tmp/deps-packages-1.json
```

#### Run for `packages`
```bash
bash scripts/dependency_usage.sh -p packages -o ./tmp/deps-packages-2.json
```

#### Benchmark

| Analysis              | Real Time   | User Time   | Sys Time   |
|-----------------------|-------------|-------------|------------|
| All plugins           | 7m 21.126s  | 7m 53.099s  | 20.581s    |
| Single plugin         | 31.360s     | 45.352s     | 2.208s     |
| Multiple plugins      | 36.403s     | 50.563s     | 2.814s     |
| x-pack/packages       | 6.638s      | 12.646s     | 0.654s     |
| packages              | 25.744s     | 39.073s     | 2.191s     |


#### Show all packages/plugins within a directory that use a specific
dependency

```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins/security_solution
```
---
#### Show all packages/plugins within a directory grouped by code owner
```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins -g owner
```
---

#### Group by code owner with adjustable collapse depth for fine-grained
grouping
**Fine-grained grouping**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 4
```
**Collapsed grouping**: groups the results under a higher-level owner
(e.g., `security_solution` as a single group).
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 1
```
---

#### Show all dependencies matching a pattern (e.g., `react-*`) within a
package
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -d 'react-*' -o ./tmp/result.json
```

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

__Related: https://github.com/elastic/kibana/issues/196767__

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) enhancement New value added to drive a business result release_note:skip Skip the PR/issue when compiling release notes Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants