Skip to content
This repository has been archived by the owner on Jul 28, 2023. It is now read-only.

Add table for workspace pipelines. #31

Merged
merged 36 commits into from
Feb 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
bd4ada0
Update SDK to v0.4.0.
sidr0cker Jan 11, 2023
6cc55d2
Add table for workspace pipelines.
sidr0cker Jan 11, 2023
41b7587
Add last_process_id and last_process to the pipeline table.
sidr0cker Jan 31, 2023
4f696b4
Add process and workspace process table.
sidr0cker Jan 31, 2023
3dfde6b
Remove `data_state` column from identity and workspace process.
sidr0cker Feb 6, 2023
7be8ac3
Update process table descriptions.
sidr0cker Feb 6, 2023
f3cb6d7
Add a fromcamel transform for identity_id in process for it to be pop…
sidr0cker Feb 6, 2023
3f4671e
Add ability to filter org workspace snapshot records by query quals.
sidr0cker Feb 6, 2023
50dd256
Add filter quals and query_where for pipelines and processes.
sidr0cker Feb 6, 2023
54acc31
Various doc updates.
sidr0cker Feb 6, 2023
df60a0a
Add documentation for process tables.
sidr0cker Feb 6, 2023
cf083cf
Upgrade SDK to v0.5.0.
sidr0cker Feb 6, 2023
6b833d7
Add links to look up supported APIs and columns for query filter.
sidr0cker Feb 6, 2023
758449e
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
3efa29f
Update steampipecloud/table_steampipecloud_workspace_pipeline.go
sidr0cker Feb 21, 2023
2064d70
Update steampipecloud/table_steampipecloud_workspace_pipeline.go
sidr0cker Feb 21, 2023
eaa4612
Update steampipecloud/table_steampipecloud_workspace_pipeline.go
sidr0cker Feb 21, 2023
6909084
Update steampipecloud/table_steampipecloud_workspace_pipeline.go
sidr0cker Feb 21, 2023
ba13198
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
de3b188
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
3dfeca2
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
daad29f
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
1ee851a
Update faulty comment.
sidr0cker Feb 21, 2023
9d3af27
Remove logger entries.
sidr0cker Feb 21, 2023
82a7c0e
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
3631604
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
26e5c97
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
880b02c
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
cc0e60b
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
d98aacb
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
ce5d37a
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
5d53bda
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
b73be52
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
4aaf394
Update steampipecloud/table_steampipecloud_process.go
sidr0cker Feb 21, 2023
ac4c905
Update steampipecloud/table_steampipecloud_workspace_pipeline.go
sidr0cker Feb 21, 2023
dad885d
Change limit logic.
sidr0cker Feb 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions docs/tables/steampipecloud_process.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Table: steampipecloud_process

Allows to track various activities performed on an identity in Steampipe Cloud.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we dont support optional quals in this table yet. The same has been added to the steampipecloud_workspace_process since that table supports it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also for the second point - I have tested the retryhydrate stuff and it works as expected.


## Examples

### Basic info

```sql
select
id,
identity_handle,
identity_type,
pipeline_id,
type,
state,
created_at
from
steampipecloud_process;
```

### List processes that are being run by an identity pipeline

```sql
select
id,
identity_handle,
identity_type,
pipeline_id,
type,
state,
created_at
from
steampipecloud_process
where
pipeline_id is not null;
```

### List user processes

```sql
select
id,
identity_handle,
identity_type,
pipeline_id,
type,
state,
created_at
from
steampipecloud_process
where
identity_type = 'user';
```

### List running processes

```sql
select
id,
identity_handle,
identity_type,
pipeline_id,
type,
state,
created_at
from
steampipecloud_process
where
state = 'running';
```
117 changes: 117 additions & 0 deletions docs/tables/steampipecloud_workspace_pipeline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Table: steampipecloud_workspace_pipeline

Pipelines allow users to run different kinds of activities in Steampipe Cloud on a schedule.

**Important notes:**

This table supports optional quals. Queries with optional quals in the `where` clause are optimised to use Steampipe Cloud filters.

Optional quals are supported for the following columns:

- `created_at`
- `id`
- `identity_handle`
- `identity_id`
- `pipeline`
- `query_where` - Allows use of [query filters](https://steampipe.io/docs/cloud/reference/query-filter). For a list of supported columns for pipelines, please see [Supported APIs and Columns](https://steampipe.io/docs/cloud/reference/query-filter#supported-apis--columns). Please note that any query filter passed into the `query_where` qual will be combined with other optional quals.
- `title`
- `updated_at`
- `workspace_handle`
- `workspace_id`

## Examples

### Basic info

```sql
select
id,
identity_handle,
workspace_handle,
title,
frequency,
pipeline,
args,
tags,
last_process_id
from
steampipecloud_workspace_pipeline;
```

### List pipelines for a specific workspace

```sql
select
id,
identity_handle,
workspace_handle,
title,
frequency,
pipeline,
args,
tags,
last_process_id
from
steampipecloud_workspace_pipeline
where
workspace_handle = 'dev';
```

### List pipelines of frequency type `interval` for a specific workspace

```sql
select
id,
identity_handle,
workspace_handle,
title,
frequency,
pipeline,
args,
tags,
last_process_id
from
steampipecloud_workspace_pipeline
where
workspace_handle = 'dev'
and frequency->>'type' = 'interval';
```

### List pipelines for the `AWS Compliance CIS v1.4.0` dashboard created in the last 7 days

```sql
select
id,
identity_handle,
workspace_handle,
title,
frequency,
pipeline,
args,
tags,
last_process_id
from
steampipecloud_workspace_pipeline
where
args->>'resource' = 'aws_compliance.benchmark.cis_v140'
and created_at >= now() - interval '7 days';
```

### List pipelines for the `AWS Compliance CIS v1.4.0` dashboard created in the last 7 days using [query filter](https://steampipe.io/docs/cloud/reference/query-filter)

```sql
select
id,
identity_handle,
workspace_handle,
title,
frequency,
pipeline,
args,
tags,
last_process_id
from
steampipecloud_workspace_pipeline
where
query_where = 'title = ''Scheduled snapshot: CIS v1.4.0'' and created_at >= now() - interval ''7 days''';
```
90 changes: 90 additions & 0 deletions docs/tables/steampipecloud_workspace_process.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Table: steampipecloud_workspace_process

Allows to track various activities performed on a workspace of an identity in Steampipe Cloud.

**Important notes:**

This table supports optional quals. Queries with optional quals in the `where` clause are optimised to use Steampipe Cloud filters.

Optional quals are supported for the following columns:

- `created_at`
- `id`
- `identity_handle`
- `identity_id`
- `pipeline_id`
- `query_where` - Allows use of [query filters](https://steampipe.io/docs/cloud/reference/query-filter). For a list of supported columns for workspace proceses, please see [Supported APIs and Columns](https://steampipe.io/docs/cloud/reference/query-filter#supported-apis--columns). Please note that any query filter passed into the `query_where` qual will be combined with other optional quals.
- `state`
- `type`
- `updated_at`
- `workspace_handle`
- `workspace_id`

## Examples

### Basic info

```sql
select
id,
identity_handle,
workspace_handle,
pipeline_id,
type,
state,
created_at
from
steampipecloud_workspace_process;
```

### List processes for a pipeline

```sql
select
id,
identity_handle,
workspace_handle,
pipeline_id,
type,
state,
created_at
from
steampipecloud_workspace_process
where
pipeline_id = 'pipe_cfcgiefm1tumv1dis7lg';
```

### List running processes for a pipeline

```sql
select
id,
identity_handle,
workspace_handle,
pipeline_id,
type,
state,
created_at
from
steampipecloud_workspace_process
where
pipeline_id = 'pipe_cfcgiefm1tumv1dis7lg'
and state = 'running';
```

### List running processes for a pipeline using [query filter](https://steampipe.io/docs/cloud/reference/query-filter)

```sql
select
id,
identity_handle,
workspace_handle,
pipeline_id,
type,
state,
created_at
from
steampipecloud_workspace_process
where
query_where = 'pipeline_id = ''pipe_cfcgiefm1tumv1dis7lg'' and state = ''running''';
```
2 changes: 0 additions & 2 deletions docs/tables/steampipecloud_workspace_snapshot.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,7 @@ Optional quals are supported for the following columns:
- `dashboard_name`
- `dashboard_title`
- `id`
- `inputs`
- `query_where` - Allows use of [query filters](https://steampipe.io/docs/cloud/reference/query-filter). For a list of supported columns for snapshots, please see [Supported APIs and Columns](https://steampipe.io/docs/cloud/reference/query-filter#supported-apis--columns). Please note that any query filter passed into the `query_where` qual will be combined with other optional quals.
- `tags`
- `visibility`

## Examples
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module steampipe-plugin-steampipecloud
go 1.19

require (
github.com/turbot/steampipe-cloud-sdk-go v0.3.0
github.com/turbot/steampipe-cloud-sdk-go v0.5.0
github.com/turbot/steampipe-plugin-sdk/v4 v4.1.8
)

Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -418,8 +418,8 @@ github.com/tkrajina/go-reflector v0.5.4 h1:dS9aJEa/eYNQU/fwsb5CSiATOxcNyA/gG/A7a
github.com/tkrajina/go-reflector v0.5.4/go.mod h1:9PyLgEOzc78ey/JmQQHbW8cQJ1oucLlNQsg8yFvkVk8=
github.com/turbot/go-kit v0.4.0 h1:EdD7Bf2EGAjvHRGQxRiWpDawzZSk3T+eghqbj74qiSc=
github.com/turbot/go-kit v0.4.0/go.mod h1:SBdPRngbEfYubiR81iAVtO43oPkg1+ASr+XxvgbH7/k=
github.com/turbot/steampipe-cloud-sdk-go v0.3.0 h1:2x0VlzAYoRZBWfYxZ3Z0kkMmERtZF8alfY3uctqWhGA=
github.com/turbot/steampipe-cloud-sdk-go v0.3.0/go.mod h1:8M2CspUHgCGqDCJV+FNn+boBPyLRHyzDinYnoZ/kZYw=
github.com/turbot/steampipe-cloud-sdk-go v0.5.0 h1:sgMpUL/gLnT5/9v6LaUDITo40npvSn+RQD5maT03wRQ=
github.com/turbot/steampipe-cloud-sdk-go v0.5.0/go.mod h1:8M2CspUHgCGqDCJV+FNn+boBPyLRHyzDinYnoZ/kZYw=
github.com/turbot/steampipe-plugin-sdk/v4 v4.1.8 h1:4/fEpsrWNSKA/p2otsibzijBCf4f35Y8jIgUcJgd0Ng=
github.com/turbot/steampipe-plugin-sdk/v4 v4.1.8/go.mod h1:t1uwq6KylUr2CzIinxeTafoktJvX8yWmhaoWCJJc4YI=
github.com/vmihailenco/msgpack v3.3.3+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
Expand Down
3 changes: 3 additions & 0 deletions steampipecloud/plugin.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ func Plugin(ctx context.Context) *plugin.Plugin {
"steampipecloud_connection": tableSteampipeCloudConnection(ctx),
"steampipecloud_organization_member": tableSteampipeCloudOrganizationMember(ctx),
"steampipecloud_organization": tableSteampipeCloudOrganization(ctx),
"steampipecloud_process": tableSteampipeCloudProcess(ctx),
"steampipecloud_organization_workspace_member": tableSteampipeCloudOrganizationWorkspaceMember(ctx),
"steampipecloud_token": tableSteampipeCloudToken(ctx),
"steampipecloud_user": tableSteampipeCloudUser(ctx),
Expand All @@ -33,6 +34,8 @@ func Plugin(ctx context.Context) *plugin.Plugin {
"steampipecloud_workspace_mod": tableSteampipeCloudWorkspaceMod(ctx),
"steampipecloud_workspace_mod_variable": tableSteampipeCloudWorkspaceModVariable(ctx),
"steampipecloud_workspace_db_log": tableSteampipeCloudWorkspaceDBLog(ctx),
"steampipecloud_workspace_pipeline": tableSteampipeCloudWorkspacePipeline(ctx),
"steampipecloud_workspace_process": tableSteampipeCloudWorkspaceProcess(ctx),
"steampipecloud_workspace_snapshot": tableSteampipeCloudWorkspaceSnapshot(ctx),
},
}
Expand Down
Loading