Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace WzSearchBar component #5366

Closed
wants to merge 40 commits into from
Closed

Conversation

Desvelao
Copy link
Member

@Desvelao Desvelao commented Apr 12, 2023

This pull request is expected to merge after #5363

Description

This pull request replaces the usage of WzSearchBar with the new one.

Changes:

  • Replace the search bar in the following sections:
    • Agent selection modal

    Adapt the external filters

    • Agent section

    Adapt the external filters
    Rename the name of the sessionStorage property from agents_preview_selected_options to wz_page_agents_search_bar_query and change the saved value. This is used to communicate from other sections with the search bar. Renaming avoids errors when the users upgrading due to the change in the schema of saved value.

    • Clean
      • Remove removeFilters props in the AgentsTable component. Adapt AgentsPreview render.
      • Remove unused file: public/controllers/overview/components/select-agent.js

TODO

Issues Resolved

Part of #4312

Evidence

Agent selection modal
image
Agents section
image
image
image
image

Test

⚠️ It is recommended to test some frequent queries in the section where the search bar was replaced. We should use a real Wazuh manager in 4.5.0 or 4.4.x as possible.

Tests
Agent selection modal - The query input should filter the data
Agent selection modal - Change the sort field should fetch the data taking into account this order
Agent selection modal - Change the sort direction should fetch the data taking into account this order
Agent selection modal - Change the current page should fetch the data taking into account this order
Agent selection modal - Change the page size should fetch the data taking into account this order
Agent selection modal - Clicking on a group label should add a query to the search bar and fetch data with that query
Agent section - The query input should filter the data
Agent section - Change the sort field should fetch the data taking into account this order
Agent section - Change the sort direction should fetch the data taking into account this order
Agent section - Change the current page should fetch the data taking into account this order
Agent section - Change the page size should fetch the data taking into account this order
Agent section - Export to CSV should use the filters
Agent section - Clicking on a group label should add a query to the search bar and fetch data with that query
Agent section - Clicking on a status value of visualization in Status panel should add a query input to the search bar and fetch data with the query
Agent section - Clicking on a status value of the indicator in Details panel should add a query input to the search bar and fetch data with the query
Agent section - Go to Modules directory and clicks on an agent status value should redirect to the Agents section and add a query input to the search bar and fetch data with the query

Check List

  • All tests pass
    • yarn test:jest
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

Features:
- Supports multiple query languages
- Decouple the business logic of query languages of the search bar
  component
- Ability of query language to interact with the search bar

Query language implementations
- AQL: custom implementation of the Wazuh Query Language. Include
  suggestions.
- UIQL: simple implementation (as another example)
…ression used in the Wazuh manager API

- Change the implementation of AQL query language to use the regular
  expression decomposition defined in the Wazuh manager API
- Adapt the tests for the tokenizer and getting the suggestions
- Enchance documentation of search bar
- Add documentation of AQL query language
- Add more fields and values for the use example in Agents section
- Add description to the query language select input
…using the Search suggestion in AQL

- Fixes a problem hidding the suggestion popover when using the Search
  suggestion in AQL
- Fixes a problem of input text with undefined value
- Minor fixes
  - Remove `syntax` property of SearchBar component
  - Add disableFocusTrap property to the custom EuiSuggestInput component to be
    forwarded to the EuiInputPopover
  - Replace the inputRef by a reference instead of a state and pass as
    a parameter in the query language run function
  - Move the rebuiding of input text when using some suggestion that changes
    the input to be done when a related suggestion was clicked instead
    of any suggestion (exclude Search).
implemenation

- Add the ability to update the input of the search bar in the example
  implementation
- Enhance the component documentation
- (AQL) Add the fields and an open operator group when there is no input text
link of query language displayed in the popover
- AQL enhancements:
  - documentation:
    - Enhance some descriptions
    - Enhance input processing
    - Remove intermetiate interface of EuiSuggestItem
  - Remove the intermediate interface of EuiSuggestItem. Now it is
    managed in the internal of query language instead of be built by the
    suggestion handler
  - Display suggestions when the input text is empty
  - Add the unifiedQuery field to the query language output
  - Adapt tests

- Search Bar component:
  - Enhance documentation
- Remove UIQL
- Add HAQL query language that is a high-level implementation of AQL
  - Add the query language interface
  - Add tests for tokenizer, get suggestions and
    transformSpecificQLToUnifiedQL method
  - Add documentation about the language
    - Syntax
    - Options
    - Workflow
- Add tests to HAQL and AQL query languages
- Fix suggestions for HAQL when typing as first element a value entity.
  Now there are no suggestions because the field and operator_compare
  are missing.
- Enhance documentation of HAQL and AQL
- Removed unnecesary returns of suggestion handler in the example
  implementation of search bar on Agents section
- Rename query language HAQL to WQL
- Update tests
- Remove AQL usage from the implementation in the agents section
- Add more use cases to the test of WQL query language
- Replace some literals by constants in the WQL query language
  implementation
- WQL
  - add implicit query mode to WQL
  - enhance query language documentation
  - renamed transformUnifiedQuery to transformUQLToQL
  - now wraps the user input if this is defined and there a implicit
    query string
  - fix a problem with the value suggestions if there is a previous
    conjunction
    - add tests cases
  - update tests
- AQL
  - enhance query language documentation
  - renamed transformUnifiedQuery to transformUQLToQL
  - add warning about the query language implementation is not updated
    to the last changes in the search bar component
  - update tests

- Search Bar
  - renamed transformUnifiedQuery to transformUQLToQL
- Set a width for the syntax options popover
- Unify the description in the suggestions of WQL example implementation
- Update tests
- Fix minor bugs in the WQL example implementation in Agents
- WQL
  - Enhance documentation
  - Add partial and "expanded" input validation
  - Add tests
…feat/4312-poc-search-bar-component-multiple-query-language-api-query-language-implementation
Fix an error when the last token in the input was a value and used
a value suggestion whose label contains whitespaces, the value was
not wrapped with quotes.
Replace the WzSearchBar component by the new search bar
Add comments related to TODO and FIX
Enhance request to get the agents data
@Desvelao Desvelao self-assigned this Apr 12, 2023
- Change the sort column or order, or current page or page size doesn't
  fetch the data after change the state. It was moved to the
componentDidUpdate comparing the previous state with the current one.
- Enhance how the sort query parameter is built
- Enhance the application of external query input in the search bar
- Replace the WzSearchBar by SearchBar in the Agents section
- Adapt the external filters
  - Group filter from labels
  - Rename the name of the sessionStorage property from `agents_preview_selected_options`
    to `wz_page_agents_search_bar_query` and change the saved value. This is used
    to communicate from other sections with the search bar. Renaming
    avoids errors when the users upgrading due to the change in the
    schema of saved value.
- Enhance the building of request parameters to fetch the table data.
- Clean:
  - Remove `removeFilters` props in the `AgentsTable` component. Adapt
    `AgentsPreview` render.
  - Remove unused file: public/controllers/overview/components/select-agent.js
@Desvelao Desvelao linked an issue Apr 18, 2023 that may be closed by this pull request
6 tasks
…uery-language-api-query-language-implementation
@Desvelao Desvelao marked this pull request as ready for review April 20, 2023 09:13
@Desvelao Desvelao requested a review from a team as a code owner April 20, 2023 09:13
Copy link
Member

@asteriscos asteriscos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While testing the search bar I found a few issues:

Clicking the refresh button does not update the query status if the input changes. Instead, it uses the previous search parameters. If the user press the "enter" key, then it does update the state.

This also happens if the selected columns change.

Peek 2023-04-20 16-29

If the search input is empty, it doesn't show the Apply Search in the available suggestions. An empty input is still a valid search.

If the user makes a generic search without a date format throughout all the available fields, which includes a Date format field, the API returns a format error.
Maybe date fields should not be used when a generic search is made.

Peek 2023-04-20 16-37

@Desvelao
Copy link
Member Author

Desvelao commented Apr 21, 2023

While testing the search bar I found a few issues:

Clicking the refresh button does not update the query status if the input changes. Instead, it uses the previous search parameters. If the user press the "enter" key, then it does update the state.

This also happens if the selected columns change.

Peek 2023-04-20 16-29 Peek 2023-04-20 16-29

If the search input is empty, it doesn't show the Apply Search in the available suggestions. An empty input is still a valid search.

If the user makes a generic search without a date format throughout all the available fields, which includes a Date format field, the API returns a format error. Maybe date fields should not be used when a generic search is made.

Peek 2023-04-20 16-37 Peek 2023-04-20 16-37

The integration of the search bar in the Agents section is problematic due to the possibility to force refreshing the request data, which was not contemplated that input or selected columns could vary. In this case, we should extract the transformation of the input query to the API query language out of the search bar.

The problem with the search term with the date fields should be discussed. I think removing the date fields of the search it is not a good approach because the user can not search in these fields using the search term mode of the search bar.

@Desvelao Desvelao mentioned this pull request Apr 25, 2023
6 tasks
@Desvelao
Copy link
Member Author

Clicking the refresh button does not update the query status if the input changes. Instead, it uses the previous search parameters. If the user press the "enter" key, then it does update the state.

I am working around this problem. It could need some refactoring or extracting the generation of the Wazuh API parameters out of the search bar.

If the search input is empty, it doesn't show the Apply Search in the available suggestions. An empty input is still a valid search.

I added the search suggestion when the input is empty.

This also happens if the selected columns change.

The current behavior in the Agents section doesn't redo the request if the selected columns are changed. The request included all the available fields. Enable/disable the columns only adds or remove the column to the table.

If the user makes a generic search without a date format throughout all the available fields, which includes a Date format field, the API returns a format error. Maybe date fields should not be used when a generic search is made.

I was researching this problem and we will need to talk with the API coworkers. I did some researching here: #4312 (comment)

@Desvelao
Copy link
Member Author

Clicking the refresh button does not update the query status if the input changes. Instead, it uses the previous search parameters. If the user press the "enter" key, then it does update the state.

I am working around this problem. It could need some refactoring or extracting the generation of the Wazuh API parameters out of the search bar.

Definitely, due how the component of the Agents section, I will need to extract the generation of the API request out of the search component.

I got to solve some problems, but modifying the visible columns meanwhile, the search bar input is in search term mode, if clicking on the Refresh button, causes the API request uses the previous selected fields instead of the new ones.

…earch term fields changed

- Search bar:
  - Add a dependency to run the query language output
  - Adapt search bar documentation to the changes
- WQL
  - Create a new parameter called `options`
  - Moved the `implicitFilter` and `searchTerm` settings to `options`
  - Update tests
  - Update documentation
…age-api-query-language-implementation' of https://github.com/wazuh/wazuh-kibana-app into feat/4312-replace-wzsearchbar
…age-api-query-language-implementation' of https://github.com/wazuh/wazuh-kibana-app into feat/4312-replace-wzsearchbar
@Desvelao
Copy link
Member Author

Desvelao commented May 2, 2023

Update

I refactored the search bar component and now, the current implementations are managing when the user input changes without using the Search suggestion or pressing the Enter key and clicking on the Refresh button on the Agents section.

This enables the Refresh button in the Agents section to take into account the new user input instead of the previous one. For another hand, this behavior causes a bug if using the Export formatted button that should take into account the previous search which is the data that is displayed in the table.

The Agents section has a particular implementation, we should discuss what is the expected behavior, implement it in the reusable TableWzAPI component, and replace the current implementation in the Agents section with the enhanced TableWzAPI. This approach will centralize the behavior and remove the differences between components that seem to be similar.

… `Explore agent` views

- adapt the replacement of search bar in `Agents` and `Explore agent` views
- enhance the `Agents` section
  - session storage management
  - display an `Update` button instead of `Refresh` one when the input
    changed
  - display always the table pagination
  - filter out the date fields from the search term fields in WQL
    (`dateAdd` and `lastKeepAlive`)
  - add a mechanism to redo the same request when using the search bar
  - enhance the management of the visible table columns
- enhance the `Explore agent` section
  - restrict the search term fields to the visibles in the table
  - add a mechanism to redo the same request when using the search bar
@github-actions
Copy link
Contributor

github-actions bot commented May 3, 2023

Code coverage (Jest) % values
Statements 10.01% ( 3735 / 37287 )
Branches 5.66% ( 1638 / 28922 )
Functions 8.99% ( 834 / 9275 )
Lines 10.11% ( 3613 / 35713 )

@Desvelao
Copy link
Member Author

Desvelao commented May 4, 2023

Problem

I am working into replacing the search bar in the Modules/MITRE ATT&CK/Intelligence section and I found a problem.

This section has a main search bar, that searches in all the available fields of the resources the input. Under the hoods, this does an API request:

GET /mitre/{resource}?search={search_bar_input}&limit=5

for each MITRE ATT&CK resource.

image

Using this search bar, displays a count of results. If for a resource there is more than the 5 elements, then the See more button is displayed. Clicking on this button causes a redirection to the resource view, that should pass the filters and the equivalence query in the search bar input. The search query parameter is not supported, so the API query should be translated to the usage of q. So, it is needed to specify each field to search the user input. If we define the fields displayed in the tables, they are name, description and name, there are less results foudn that using search. This means that others fields could match with the user input. Adding all the fields, should solve the problem, but in this case when the user is redirected, the new search bar will display an error because the fields are not supported in the validation of the search bar.
image

We should discuss how to solve this problem.

The current solution was restricting the field to the displayed in the tables: description, external_id and name. This approach will not be able to find resources whose match is in another field.

@Desvelao Desvelao marked this pull request as draft May 15, 2023 12:23
@Desvelao
Copy link
Member Author

This pull request is closed in favor to that replaces the search bar and table components by the reusable TableWzAPI component:

@Desvelao Desvelao closed this May 19, 2023
@Desvelao Desvelao removed a link to an issue May 19, 2023
6 tasks
@AlexRuiz7 AlexRuiz7 deleted the feat/4312-replace-wzsearchbar branch November 29, 2023 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants