Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index lifecycle management wizard #21925

Merged
merged 82 commits into from
Sep 6, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
1b421cc
Index lifecycle management wizard
chrisronline Apr 27, 2018
eb7fc0f
Adding index lifecycle management files
chrisronline Apr 27, 2018
318964f
Updates
chrisronline Apr 27, 2018
ecdfeab
Fix errors and add flyout for node details
chrisronline Apr 30, 2018
7af8076
Merge in master
chrisronline Apr 30, 2018
0eface2
New diff tool
chrisronline May 4, 2018
9e5e2c4
Scroll to change for review diff
chrisronline May 7, 2018
b2251b6
Some feedback on copy
chrisronline May 7, 2018
6eba53f
Updating copy, moving components around and fixing bugs with the diff…
chrisronline May 11, 2018
4851068
Add logic to auto enable a phase when something is set
chrisronline May 14, 2018
5ca7b4f
redesign ilm
snide May 16, 2018
c8f7720
Merge pull request #1 from snide/design/ilm
chrisronline May 17, 2018
aa3abdd
Adding server api tests
chrisronline May 17, 2018
8df605c
Merge branch 'ilm' of github.com:chrisronline/kibana into ilm
chrisronline May 17, 2018
70caea4
Removing debug and some tweaks from dave's work
chrisronline May 17, 2018
bb15a26
Conditionally show this message
chrisronline May 17, 2018
3687b6d
Policy selection cleanup
chrisronline May 18, 2018
cae2abd
Merge in master
chrisronline May 18, 2018
2ad2735
Updates for better UX
chrisronline May 18, 2018
877d849
[COPYEDIT | ILM] Copyedit of text in index lifecycle management UI
gchaps May 21, 2018
5ac5215
Merge pull request #2 from gchaps/copyedit/ilm
chrisronline May 22, 2018
fab23a2
Merge in master
chrisronline May 25, 2018
374c6c4
Use better default text
chrisronline May 29, 2018
19acb8d
Merge remote-tracking branch 'elastic/master' into ilm
chrisronline Jun 1, 2018
d19072d
Merge remote-tracking branch 'elastic/master' into ilm
chrisronline Jun 5, 2018
f817990
Remove debug
chrisronline Jun 6, 2018
cc9c8b6
Adding readme and comments
chrisronline Jun 11, 2018
475540a
Update readme
chrisronline Jun 11, 2018
36e001d
Do not need this anymore
chrisronline Jun 11, 2018
934f4fb
Remove debug or commented out code
chrisronline Jun 11, 2018
910f3eb
Remove these - they are in the tests PR
chrisronline Jun 12, 2018
07d7cf6
Merge remote-tracking branch 'elastic/master' into ilm
chrisronline Jun 15, 2018
99aef82
Toggle system indices
chrisronline Jun 15, 2018
21cace8
Merge remote-tracking branch 'elastic/master' into ilm
chrisronline Jun 18, 2018
a3d8f24
Aliases are not defined here anymore
chrisronline Jun 18, 2018
db39544
Handle rollover better in warm phase and remove from cold,delete
chrisronline Jun 18, 2018
68cfaa9
Merge branch 'master' of github.com:elastic/kibana into ilm
bmcconaghy Aug 1, 2018
ec2dc4e
adding learn more link component and switching over to using that
bmcconaghy Aug 1, 2018
77f65e4
fixing UI issue when no policies exist
bmcconaghy Aug 1, 2018
bc52ab1
various fixes
bmcconaghy Aug 3, 2018
fdd5178
Merge branch 'master' of github.com:elastic/kibana into ilm
bmcconaghy Aug 8, 2018
69cd684
Merge branch 'master' of github.com:elastic/kibana into ilm
bmcconaghy Aug 9, 2018
e192153
some cleanup
bmcconaghy Aug 9, 2018
41f6aaf
Merge branch 'master' of github.com:elastic/kibana into ilm
bmcconaghy Aug 13, 2018
eca34f9
moving number_of_replicas due to API change
bmcconaghy Aug 13, 2018
87f8455
Merge branch 'master' of github.com:elastic/kibana into index-lifecycle
bmcconaghy Aug 14, 2018
bc52b19
modifying some messaging
bmcconaghy Aug 15, 2018
f8bbc30
Merge branch 'master' of github.com:elastic/kibana into index-lifecycle
bmcconaghy Aug 20, 2018
472213b
fixing typo
bmcconaghy Aug 20, 2018
e3ba85f
fixing some diff issues and not adding sattr_name if none chosen
bmcconaghy Aug 20, 2018
e97639e
making write alias required for template step and making necessary AP…
bmcconaghy Aug 22, 2018
61a3f95
removing alias definition from template patching as it needs to be pe…
bmcconaghy Aug 22, 2018
c507a35
some copy edits for clarity
bmcconaghy Aug 27, 2018
f74e602
fixing issue with editing existing policy when rollover starts the wa…
bmcconaghy Aug 27, 2018
1f3da34
addressing PR feedback on server side code
bmcconaghy Aug 27, 2018
effd909
addressing PR feedback
bmcconaghy Aug 27, 2018
8d644d0
removing additional spaces from findMatchingNodes call
bmcconaghy Aug 27, 2018
32a7243
changing template to index template in one more place
bmcconaghy Aug 27, 2018
448e661
Merge branch 'master' of github.com:elastic/kibana into index-lifecycle
bmcconaghy Aug 29, 2018
852abb9
Merge branch 'master' of github.com:elastic/kibana into index-lifecycle
bmcconaghy Aug 30, 2018
8592496
fixing issue with error message showing when bootstrap is successful
bmcconaghy Aug 30, 2018
6201e60
fixing node options for warm and cold phase
bmcconaghy Aug 30, 2018
1282054
adding seconds to duration fields to match what ES supports
bmcconaghy Aug 30, 2018
de40de4
changing icon for enabled steps so it does not look like an error ind…
bmcconaghy Aug 31, 2018
121c196
adjusting icon color for enabled lifecycle steps
bmcconaghy Aug 31, 2018
280d44c
fixing issue with editing an existing policy with warm phase on rollo…
bmcconaghy Aug 31, 2018
ea06e41
fixing issue with default unit for age dropdowns
bmcconaghy Aug 31, 2018
7dcefbc
fixing issues with shrink action serialization and deserialization
bmcconaghy Aug 31, 2018
7fb7509
fixing issue with deserialization of ES policy for shrink
bmcconaghy Aug 31, 2018
797675d
removing shrink option from UI when primary shard count is not greate…
bmcconaghy Aug 31, 2018
02dc6cd
going straight to create policy when no policies exist
bmcconaghy Sep 4, 2018
6f3e339
Merge branch 'master' of github.com:elastic/kibana into index-lifecycle
bmcconaghy Sep 4, 2018
d2a6763
improving lifecycle policy selection
bmcconaghy Sep 4, 2018
a327cb8
adding active badge instead of checkmark for active lifecycle policy …
bmcconaghy Sep 4, 2018
f508177
some cleanup of unneeded properties and only showing save as new when…
bmcconaghy Sep 4, 2018
f5316bb
removing stray fullWidth attribute
bmcconaghy Sep 4, 2018
979cd62
adding missing minimum for replica count for warm phase
bmcconaghy Sep 5, 2018
873d65b
adding scroll to top for review step
bmcconaghy Sep 5, 2018
8d658a5
fixing issue with start warm phase after rollover introduced by time …
bmcconaghy Sep 6, 2018
dcba513
making shrink options not show for primary shard count of 1 as you ca…
bmcconaghy Sep 6, 2018
1ecbf20
Merge branch 'master' of github.com:elastic/kibana into index-lifecycle
bmcconaghy Sep 6, 2018
33df733
fixing issue with editing existing policy and saveAsNew
bmcconaghy Sep 6, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions x-pack/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import { apm } from './plugins/apm';
import { licenseManagement } from './plugins/license_management';
import { cloud } from './plugins/cloud';
import { indexManagement } from './plugins/index_management';
import { indexLifecycleManagement } from './plugins/index_lifecycle_management';
import { consoleExtensions } from './plugins/console_extensions';
import { notifications } from './plugins/notifications';
import { kueryAutocomplete } from './plugins/kuery_autocomplete';
Expand All @@ -44,6 +45,7 @@ module.exports = function (kibana) {
indexManagement(kibana),
consoleExtensions(kibana),
notifications(kibana),
indexLifecycleManagement(kibana),
kueryAutocomplete(kibana)
];
};
72 changes: 72 additions & 0 deletions x-pack/plugins/index_lifecycle_management/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Index lifecyle management

## What is it
-- TODO --

## UI

The UI currently consists of a single wizard, broken into three steps.

### Step 1
The first step involves choosing the index template in which the created/selected policy will be applied.
Then, it lets the user tweak configuration options on this template including shard and replica count as well as allocation rules.

### Step 2
The second step lets the user choose which policy they want to apply to the selected index template. They can choose a new one or select an existing one. Either way, after selection, they will see configuration options for the policy itself. This includes configuration for the hot, warm, cold, and delete phase.

### Step 3
The third and last step lets the user name their policy and also see the affected indices and index templates. These indices and index templates are what will be affected once the user saves the work done in the wizard (This includes changes to the index template itself which will change indices created from the template and also changes to a policy that is attached to another index template). The user can also see a visual diff of what will change in the index template. Then, the user clicks the Save button and blamo!

## UI Architecture

The UI is built on React and Redux.

### Redux

The redux store consists of a few top level attributes:
```
indexTemplate
nodes
policies
general
```

The logic behind the store is separate into four main concerns:
1) reducers/
2) actions/
3) selectors/
4) middleware/

The reducers and actions are pretty standard redux, so no need to discuss much there.

### Selectors

The selectors showcase how we access any stateful data. All access comes through selectors so if there are any changes required to the state tree, we only need to update the reducers and selectors.

#### Middleware

The middleware folder contains specific pieces of state logic we need to handle side effects of certain state changing.

One example is the `auto_enable_phase.js` middleware. By default, the warm, cold and delete phases are disabled. However, the user can expand the section in the UI and edit configuration without needing to enable/disable the phase. Ideally, once the user edits any configuration piece within a phase, we _assume_ they want that phase enabled so this middleware will detect a change in a phase and enable if it is not enabled already.

#### Generic phase data

Each of our four phases have some similar and some unique configuration options. Instead of making each individual phase a specific action for that phase, the code is written more generically to capture any data change within a phase to a single action. Therefore, each phase component's configuration inputs will look similar, like: `setPhaseData(PHASE_ROLLOVER_AFTER_UNITS, e.target.value)`. The top level container for each phase will handle automatically prefixing the `setPhaseData` prop with the right phase: ` setPhaseData: (key, value) => setPhaseData(PHASE_COLD, key, value),`.

To complement this generic logic, there is a list of constants that are used to ensure the right pieces of data are changed. These are contained within `store/constants.js`

### Diff View

The third step of the wizard features a visual diff UI component which is custom to this feature. It is based off Ace/Brace and the custom code used to power is it lives in `lib/diff_ace_addons.js` and `lib/diff_tools.js`. The UI parts are in `sections/wizard/components/review/diff_view.js`. See those individual files for more detailed comments/explanations.

### Validation

Every step in the wizard features validation and will show error states after the user attempts to move to the next step assuming there are errors on the current page.

This works by constantly revalidating the entire wizard state after each state change. This is technically optional as the method to trigger validation is manually called in each UI component that triggers a state change.

It's important to note that the validation logic does not apply to a single step in the wizard, but will always validate the entire state tree. This helps prevent scenarios where a change in a step might invalidate a change in another step and we lose that validation state.

Once a step change is initiated (like clicking Next Step), the current step is marked as able to see errors and will reject the change if there are errors. It will show a toast to the user that there are errors and make each error visible on the relevant UI control.

As a way to consolidate showing these errors, there is a custom UI component called `ErrableFormRow` that wraps a `EuiFormRow` and it's child with the appropriate error states when appropriate.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export const BASE_PATH = '/management/elasticsearch/index_lifecycle_management/';
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export { PLUGIN } from './plugin';
export { BASE_PATH } from './base_path';
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export const PLUGIN = {
ID: 'index_lifecycle_management'
};
38 changes: 38 additions & 0 deletions x-pack/plugins/index_lifecycle_management/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/




import { resolve } from 'path';
import { registerTemplatesRoutes } from './server/routes/api/templates';
import { registerNodesRoutes } from './server/routes/api/nodes';
import { registerPoliciesRoutes } from './server/routes/api/policies';
import { registerLifecycleRoutes } from './server/routes/api/lifecycle';
import { registerIndicesRoutes } from './server/routes/api/indices';
import { registerLicenseChecker } from './server/lib/register_license_checker';
import { PLUGIN } from './common/constants';

export function indexLifecycleManagement(kibana) {
return new kibana.Plugin({
id: PLUGIN.ID,
publicDir: resolve(__dirname, 'public'),
require: ['kibana', 'elasticsearch', 'xpack_main'],
uiExports: {
managementSections: [
'plugins/index_lifecycle_management',
]
},
init: function (server) {
registerLicenseChecker(server);
registerTemplatesRoutes(server);
registerNodesRoutes(server);
registerPoliciesRoutes(server);
registerLifecycleRoutes(server);
registerIndicesRoutes(server);
}
});
}
58 changes: 58 additions & 0 deletions x-pack/plugins/index_lifecycle_management/public/api/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import chrome from 'ui/chrome';
let httpClient;
export const setHttpClient = (client) => {
httpClient = client;
};
const apiPrefix = chrome.addBasePath('/api/index_lifecycle_management');

export async function loadNodes() {
const response = await httpClient.get(`${apiPrefix}/nodes/list`);
return response.data;
}

export async function loadNodeDetails(selectedNodeAttrs) {
const response = await httpClient.get(`${apiPrefix}/nodes/${selectedNodeAttrs}/details`);
return response.data;
}

export async function loadIndexTemplates() {
const response = await httpClient.get(`${apiPrefix}/templates`);
return response.data;
}

export async function loadIndexTemplate(templateName) {
if (!templateName) {
return {};
}
const response = await httpClient.get(`${apiPrefix}/template/${templateName}`);
return response.data;
}

export async function loadPolicies() {
const response = await httpClient.get(`${apiPrefix}/policies`);
return response.data;
}

export async function saveLifecycle(lifecycle, indexTemplatePatch) {
const response = await httpClient.post(`${apiPrefix}/lifecycle`, { lifecycle, indexTemplatePatch });
return response.data;
}

export async function bootstrap(indexName, aliasName) {
const response = await httpClient.post(`${apiPrefix}/indices/bootstrap`, { indexName, aliasName });
return response.status === 200;
}

export async function getAffectedIndices(indexTemplateName, policyName) {
const path = policyName
? `${apiPrefix}/indices/affected/${indexTemplateName}/${encodeURIComponent(policyName)}`
: `${apiPrefix}/indices/affected/${indexTemplateName}`;
const response = await httpClient.get(path);
return response.data;
}
12 changes: 12 additions & 0 deletions x-pack/plugins/index_lifecycle_management/public/app.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import React from 'react';
import { Wizard } from './sections/wizard';

export const App = () => (
<Wizard/>
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import React from 'react';
import { EuiLink } from '@elastic/eui';
import { ELASTIC_WEBSITE_URL, DOC_LINK_VERSION } from 'ui/documentation_links';

const esBase = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference/${DOC_LINK_VERSION}/`;


export class LearnMoreLink extends React.PureComponent {
render() {
const { href, docPath } = this.props;
let url;
if (docPath) {
url = `${esBase}${docPath}`;
} else {
url = href;
}
return (
<EuiLink href={url} target="_blank">
Learn more
</EuiLink>
);

}
}
8 changes: 8 additions & 0 deletions x-pack/plugins/index_lifecycle_management/public/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import './register_management_section';
import './register_routes';
49 changes: 49 additions & 0 deletions x-pack/plugins/index_lifecycle_management/public/less/main.less
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
@import (reference) "~ui/styles/variables";

#indexLifecycleManagementReactRoot {
background: @globalColorLightestGray;
min-height: 100vh;
}

.euiPageContent.ilmContent {
max-width: 1000px;
width: 100%;
}

.ilmHrule {
// Less has a bug with calcs
width: calc(~"100% + 48px") !important;
margin-left: -24px;
margin-right: -24px;
}

.ilmAlias {
display: inline-block;
background-color: #333;
color: white;
padding: 4px 8px;
}

.ilmDiff__nav {
padding: 16px;
background: #f5f5f5;
}

.ilmDiff__code {

}

.euiAnimateContentLoad {
animation: euiAnimContentLoad $euiAnimSpeedExtraSlow $euiAnimSlightResistance;
}

@keyframes euiAnimContentLoad {
0% {
opacity: 0;
transform: translateY(16px);
}
100% {
opacity: 1;
transform: translateY(0);
}
}
Loading