Migrate packages dashboards to by value visualizations

[ruflin]

Kibana visualizations can be added in a dashboard by value or by reference. As historically by value did not exist, most dashboards are still using by reference. Switching to value has the advantage that the dashboards are fully self contained and only need a single request to be installed. Because of this existing dashboards should be moved over to use by value.

The biggest have the most used packages and the ones with many dashboards and visualisations. This meta issue is to track the work on migrating these packages.

• Kubernetes Add OOTB Kubernetes dashboard to Kubernetes integration & implement best dashboard creation practices #3115
• System Migrate system dashboards to by value #3449
• AWS (multiple) Migrate AWS dashboards to by value #3454 and Migrate awsfargate dashboards to by value #3455
• [GCP] Migrate package dashboards to by value visualization #3719
• Security external integrations [security-external-integrations packages] Migrate dashboards to by_value #3463
• Azure [Azure] Convert dashboards to use by-value visualizations #4922

As soon as this list is migrated, we should expand it to all integrations.

Migration script

A migration script was created to help with this effort: https://github.com/flash1293/legacy_vis_analyzer

Documentation

• Update development guidelines to prescript using by-value visualizations
  • See dashboards guidelines

[andrewkroh]

Meta issue for security-external-integrations: #3463

[bradenlpreston]

@jamiehynds - can you take a look at this?

[jamiehynds]

@bradenlpreston - we've made a list of integrations that require conversion here. Assuming there isn't a hard deadline, I've added this work as an 8.5 item as part of our team's planning session later this week.

[bradenlpreston]

Great! - thanks Jamie.

[tommyers-elastic]

Hey all -For some dashboards that have been migrated to by-value, I still see a bunch of referenced objects for searches, tags, index patterns etc. Are these types of saved objects ok to be still referenced like this, or should these also be inlined? see this dashboard for an example

integrations/packages/gcp/kibana/dashboard/gcp-48e12760-cbe4-11ec-b519-85ccf621cbbf.json

Line 593 in 5f5ede9

"references": [

it seems like unless we do this this can't be true dashboards are fully self contained and only need a single request to be installed.

but at the same time, aren't saved searches sometimes top-level package assets too? in which case it wouldn't make sense to duplicate them.

thanks

[ruflin]

As you described, we likely not end up having all assets by value (and shouldn't). But we should have clear guidelines for which these is the case.

[endorama]

I opened a PR to update the dashboard guidelines documentation: #5021

[milanparmar-crest]

Team, Is anyone working on Migrating System and Azure dashboards to "by value" visualizations? If not we can take it up and start working on it.

[botelastic]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[ruflin]

@lalit-satapathy Where did we land here?

[lalit-satapathy]

@lalit-satapathy Where did we land here?

We are addressing this as part of the fixing of the skipped validations of the package migration work. For the pass-by-value pending the skip was SVR00004.

Here is one PR being reviewed, for the initial set of packages. There was also a fix done in the in-liner script. The remaining set of packages will be taken up next here.

[ruflin]

@lalit-satapathy Should we keep this issue open or consider it "completed" and all follow up is tracked separately?

[lalit-satapathy]

Keeping it open and assigned to me. Will close once all SVR00004 fixes here are done.