[FEATURE] Create/edit rule in YAML format

[kamingleung]

Is your feature request related to a problem?
As a user, I should be able to create/edit a rule in YAML format or with a guided visual editor.

What solution would you like?

• In the Create rule flow, users have the option to define a rule using visual or YAML editor with a toggle.
• Users can toggle between editor and the rule will be translated. For example, if a user started creating the rule in YAML, then toggles over to visual editor, everything from the YAML editor will be translated into the visual editor. The user can continue editing the rule in visual editor.

What alternatives have you considered?
Open to suggestions!

Mockups

1. In the create rule flow, users can click on "YAML editor" to switch from Visual editor to YAML editor. (For toggle, use the EuiButtonGroups component with "single" type)
   

2. Here's how it looks like when toggled to YAML editor. Users can click on "Visual editor" to return the define the rule using the UI. (The "Sigma specification" link should point to: https://github.com/SigmaHQ/sigma-specification)
   
   [Updated screenshot] Added label and description text for the YAML editor:
   Label: Define rule in YAML
   Description text: Use the YAML editor to define a sigma rule. See Sigma specification for rule structure and schema.

[kamingleung]

@djindjic I have updated the second screenshot above with new text for labels and description text.

[djindjic]

Findings

After short sync with @amsiglan we found that these two editor types are maybe not capable to be always connected in "two-way binding" (it means if user change rule name in Visual Editor that should be reflected in YAML as soon as "tab" is changed and vice versa).

There are 3 possible issues:

1. Property names are not 1 on 1 (ex. Rule Name in Visual Editor is title in YAML, )
2. YAML has more fields which does not exist on Visual Editor (ex. date exists only in YAML)
3. logsource is multi value node in YAML -> Log Type is single select dropdown in Visual Editor

Possible solution

Possible solution on Create Rule and Edit Rule page could be initial chooser dialog (ex. Create and Edit Policy on IM Plugin image bellow)

Strictly following that pattern Import Rule and Duplicate Rule pages would have the same logic upfront.

@kamingleung please give us your thoughts.

[kamingleung]

@djindjic

1. The Rule name and Title are referring to the same fields. Can these be 2-way binded?
2. We can add a Date field on the visual editor. For the Modified field, I assume this will be auto-generated?
3. I believe Log type is a new field we are introducing and to be categorized with the detector, should we add this as a required field in the rule schema? I don't think we are leveraging the logsource. @getsaurabh02 @sbcd90, can you provide some insights on how we will handle this?

If we are able to resolve the above 3 issues, what else is preventing this from allowing users to switch between visual and YAML editors?

[djindjic]

@kamingleung @amsiglan
I have managed to handle "2-way-binding" fully aligned with original idea in the Mockup. Base on original YAML file (also visible in mockup) there are few open questions. Some of them are already mentioned:

• date and modified not existing in Visual Editor (we can think of it @kamingleung and create new issue for adding date pickers) - currently ignoring
• logsource yaml property is an object that should have some special mapping (based on my sync with backend devs). Currently mapping logsource.product to Log type dropdown (category key on Rule interface), but we just need to confirm it with someone more involved @amsiglan
• log_source key on Rule interface seem like not in use @amsiglan
  
• tags validation (synced with @petardz, we should always have tags in format namespace.name). This is missing validation on Visual Editor too at the moment @amsiglan
• yaml validation messages - I don't have full list of possible invalid cases and edge cases at the moment, that's part of my next research

Will keep you posted here.

P.S. Just for cleaner communication, I'm adding screen recording of WIP workflow

Screen.Recording.2022-12-06.at.8.53.09.PM.mov

[kamingleung]

@djindjic Thanks for your update! In the video, I noticed the rule name has some naming restrictions. What are the restrictions? Can we provide an inline validation for the visual editor and return an appreciate error message when trying to submit on YAML editor? "Enter valid input for Rule name" is a bit unclear for users.

[djindjic]

@kamingleung good catch, thanks. Since I've already started on next task, I've created 2 new issues based on your finding: #214 and #215. Also added to our internal tracking tool.

@amsiglan @kamingleung Please let me know if #215 is enough high priority for v2.5

[djindjic]

@amsiglan since #201 is merged and I created follow-up tasks, my suggestion would be to close this issue to reflect on the project board, or let me know to move these follow-ups together as part of this issue, eventually.