Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] No findings with test detectors #368

Closed
jaakoob opened this issue Mar 2, 2023 · 28 comments
Closed

[BUG] No findings with test detectors #368

jaakoob opened this issue Mar 2, 2023 · 28 comments
Labels
bug Something isn't working

Comments

@jaakoob
Copy link

jaakoob commented Mar 2, 2023

What is the bug?

We can not get any findings from some test detectors we created.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Create a winlogbeat index with mappings from winlogbeat 7.10.2
  2. Create a test detector with the "Sysmon Configuration Change" shipped rule on that index
  3. Insert the following document:
    POST winlogbeat-7.10.2-test-1/_doc { "@timestamp": "2023-03-02T20:12:59+0000", "host.name": "server01", "winlog.event_id": "16", "winlog.channel": "Microsoft-Windows-Sysmon/Operational", }

What is the expected behavior?

We would expect to see a finding with that doc attached.

What is your host/environment?

  • OS: Almalinux 9.x
  • Version 2.6.0

Do you have any additional context?

We used the auto generated field mapping for the sigma rules and all except from 3 were found in the mapping supplied from winlogbeat.

@jaakoob jaakoob added bug Something isn't working untriaged labels Mar 2, 2023
@petardz
Copy link
Contributor

petardz commented Mar 6, 2023

Seems like you're inserting string value but doc level query is searching numeric value 16

@jaakoob
Copy link
Author

jaakoob commented Mar 8, 2023

The field is mapped as a number, so even if we insert in parantheses, it is ingested as a number. When utilizing uncoder.io to translate the sigma rule to an Opensearch query we are able to find the document though.

@petardz
Copy link
Contributor

petardz commented Mar 8, 2023

Did you try inserting doc like this:

POST winlogbeat-7.10.2-test-1/_doc 
{   
   "@timestamp": "2023-03-02T20:12:59+0000",  
   "winlog.event_id": 16
}

Also, can you verify that you have alias "winlog-event_id" in your winlogbeat-7.10.2-test-1 index mappings

@eirsep eirsep removed the untriaged label Mar 16, 2023
@phipiship
Copy link

We tried that already, unfortunately without success.

GET /_plugins/_security_analytics/mappings/view { "index_name": "winlogbeat-7.10.2-test-1", "rule_topic": "windows" }

Gives me the following output:

........ "winlog-event_id": { "path": "winlog.event_id", "type": "alias" }, ........

But we have three unmapped filed aliases, is that normal?

"unmapped_field_aliases": [ "timestamp", "winlog-computerObject-name", "windows-message" ]

@petardz
Copy link
Contributor

petardz commented Mar 23, 2023

@phipiship Can you please verify that index winlogbeat-7.10.2-test-1 has this alias in mappings:

"winlog-event_id": { "path": "winlog.event_id", "type": "alias" }

Please execute GET winlogbeat-7.10.2-test-1/_mapping to confirm this.

@fquinino
Copy link

I reproduced the issue mentioned above by @jaakoob and he is totally right, alerts are not been generated by security analytics, see the steps I did:

  1. Created the detector
    image

  2. Ensured the detector has the field mapping for event_id:

image

  1. Verified the index mapping:
{
  "os_windows-000002": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "@version": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "CommandLine": {
          "type": "alias",
          "path": "process.command_line"
        },
        "agent": {
          "properties": {
            "ephemeral_id": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "id": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "type": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "version": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "destination": {
          "properties": {
            "ip": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "range": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "user": {
              "properties": {
                "domain": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "id": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            }
          }
        },
        "dns": {
          "properties": {
            "answers": {
              "properties": {
                "data": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "question": {
              "properties": {
                "name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "response_code": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "ecs": {
          "properties": {
            "version": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "event": {
          "properties": {
            "action": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "bytes": {
              "type": "long"
            },
            "code": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "created": {
              "type": "date"
            },
            "id": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "ingested": {
              "type": "date"
            },
            "kind": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "outcome": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "provider": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "reason": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "file": {
          "properties": {
            "hash": {
              "properties": {
                "sha256": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "path": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "pe": {
              "properties": {
                "imphash": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            }
          }
        },
        "host": {
          "properties": {
            "hostname": {
              "type": "keyword",
              "ignore_above": 1024
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "host-hostname": {
          "type": "alias",
          "path": "host.hostname"
        },
        "labels": {
          "properties": {
            "fingerprint": {
              "properties": {
                "flow": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "session": {
              "properties": {
                "id": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            }
          }
        },
        "log": {
          "properties": {
            "level": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "sender": {
              "properties": {
                "name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "type": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            }
          }
        },
        "message": {
          "type": "text",
          "norms": false
        },
        "network": {
          "properties": {
            "protocol": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "transport": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "organization": {
          "properties": {
            "jira": {
              "properties": {
                "id": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "process": {
          "properties": {
            "command_line": {
              "type": "keyword",
              "ignore_above": 1024,
              "fields": {
                "text": {
                  "type": "text",
                  "norms": false
                }
              }
            },
            "executable": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "hash": {
              "properties": {
                "md5": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "sha256": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "parent": {
              "properties": {
                "command_line": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "executable": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "pid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "pe": {
              "properties": {
                "company": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "description": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "file_version": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "imphash": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "original_file_name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "product": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "pid": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "thread": {
              "properties": {
                "id": {
                  "type": "long"
                }
              }
            },
            "working_directory": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "rule": {
          "properties": {
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "server-user-hash": {
          "type": "alias",
          "path": "process.command_line"
        },
        "source": {
          "properties": {
            "geo": {
              "properties": {
                "as": {
                  "properties": {
                    "number": {
                      "type": "long"
                    },
                    "organization": {
                      "properties": {
                        "name": {
                          "type": "text",
                          "fields": {
                            "keyword": {
                              "type": "keyword",
                              "ignore_above": 256
                            }
                          }
                        }
                      }
                    }
                  }
                },
                "geo": {
                  "properties": {
                    "city_name": {
                      "type": "text",
                      "fields": {
                        "keyword": {
                          "type": "keyword",
                          "ignore_above": 256
                        }
                      }
                    },
                    "country_iso_code": {
                      "type": "text",
                      "fields": {
                        "keyword": {
                          "type": "keyword",
                          "ignore_above": 256
                        }
                      }
                    },
                    "country_name": {
                      "type": "text",
                      "fields": {
                        "keyword": {
                          "type": "keyword",
                          "ignore_above": 256
                        }
                      }
                    },
                    "location": {
                      "properties": {
                        "lat": {
                          "type": "float"
                        },
                        "lon": {
                          "type": "float"
                        }
                      }
                    },
                    "postal_code": {
                      "type": "text",
                      "fields": {
                        "keyword": {
                          "type": "keyword",
                          "ignore_above": 256
                        }
                      }
                    }
                  }
                }
              }
            },
            "ip": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "port": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "range": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "user": {
              "properties": {
                "domain": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "id": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            }
          }
        },
        "tags": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "timestamp": {
          "type": "alias",
          "path": "@timestamp"
        },
        "user": {
          "properties": {
            "domain": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "id": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "user_agent": {
          "properties": {
            "device": {
              "properties": {
                "name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            }
          }
        },
        "windows": {
          "properties": {
            "hour": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "weekday": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        "windows-message": {
          "type": "alias",
          "path": "message"
        },
        "winlog": {
          "properties": {
            "activity_id": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "api": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "channel": {
              "type": "keyword"
            },
            "computerObject": {
              "properties": {
                "name": {
                  "type": "keyword"
                }
              }
            },
            "computer_name": {
              "type": "keyword"
            },
            "event_data": {
              "properties": {
                "AccountExpires": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "AdvancedOptions": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "AlgorithmName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "AllowedToDelegateTo": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "AuthenticationPackageName": {
                  "type": "keyword"
                },
                "Binary": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BitlockerUserInputTime": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BootAppStatus": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BootMenuPolicy": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BootMode": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BootStatusPolicy": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BootType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BugcheckCode": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BugcheckInfoFromEFI": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BugcheckParameter1": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BugcheckParameter2": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BugcheckParameter3": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BugcheckParameter4": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "BuildVersion": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CallTrace": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CallerProcessId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CallerProcessName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Checkpoint": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CheckpointStatus": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ClientCreationTime": {
                  "type": "date"
                },
                "ClientProcessId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CommandLine": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Company": {
                  "type": "keyword"
                },
                "Config": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ConfigAccessPolicy": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ConfigurationReader": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ConnectedStandbyInProgress": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CorruptionActionState": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CountNew": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CountOfCredentialsReturned": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CountOld": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CreationUtcTime": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CsEntryScenarioInstanceId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CurrentDirectory": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "CurrentStratumNumber": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Default SD String:": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Description": {
                  "type": "keyword"
                },
                "DestinationHostname": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DestinationIp": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DestinationIsIpv6": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DestinationPort": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DestinationPortName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Detail": {
                  "type": "keyword"
                },
                "Details": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DeviceName": {
                  "type": "keyword"
                },
                "DeviceNameLength": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DeviceTime": {
                  "type": "date"
                },
                "DeviceVersionMajor": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DeviceVersionMinor": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DirtyPages": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DisableIntegrityChecks": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DisplayName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DriveName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DriverName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DriverNameLength": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Dummy": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "DwordVal": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ElevatedToken": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "EnableDisableReason": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "EnabledNew": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "EntryCount": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "EventType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "FailureName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "FailureNameLength": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "FileVersion": {
                  "type": "keyword"
                },
                "FinalStatus": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "FlightSigning": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "GrantedAccess": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Group": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HandleId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Hashes": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HiveName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HiveNameLength": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HomeDirectory": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HomePath": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HypervisorDebug": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HypervisorLaunchType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "HypervisorLoadOptions": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ID": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "IdleImplementation": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "IdleStateCount": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Image": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ImageLoaded": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ImpersonationLevel": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Initiated": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "IntegrityLevel": {
                  "type": "keyword"
                },
                "IpAddress": {
                  "type": "keyword"
                },
                "IpPort": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "IsTestConfig": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "KernelDebug": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "KeyFilePath": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "KeyLength": {
                  "type": "keyword"
                },
                "KeyName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "KeyType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "KeysUpdated": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LastBootGood": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LastBootId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LastShutdownGood": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LmPackageName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LoadOptions": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LogonGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LogonHours": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "LogonId": {
                  "type": "keyword"
                },
                "LogonProcessName": {
                  "type": "keyword"
                },
                "LogonType": {
                  "type": "keyword"
                },
                "MajorVersion": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MandatoryLabel": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MaximumPerformancePercent": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MessageNumber": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MessageTotal": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MinimumPasswordLength": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MinimumPasswordLengthAudit": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MinimumPerformancePercent": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MinimumThrottlePercent": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "MinorVersion": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NewProcessId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NewProcessName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NewSd": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NewSize": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NewThreadId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NewTime": {
                  "type": "date"
                },
                "NewUacValue": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "NominalFrequency": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Number": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ObjectName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ObjectServer": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ObjectType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "OldSd": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "OldTime": {
                  "type": "date"
                },
                "OldUacValue": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Operation": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "OriginalFileName": {
                  "type": "keyword"
                },
                "OriginalSize": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PackageName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ParentCommandLine": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ParentImage": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ParentProcessGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ParentProcessId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ParentProcessName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ParentUser": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PasswordLastSet": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Path": {
                  "type": "keyword"
                },
                "PerformanceImplementation": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Phase": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PowerButtonTimestamp": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PreviousTime": {
                  "type": "date"
                },
                "PrimaryGroupId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PrivilegeList": {
                  "type": "keyword"
                },
                "ProcessCreationTime": {
                  "type": "date"
                },
                "ProcessGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ProcessID": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ProcessId": {
                  "type": "keyword"
                },
                "ProcessName": {
                  "type": "keyword"
                },
                "ProcessPath": {
                  "type": "keyword"
                },
                "Product": {
                  "type": "keyword"
                },
                "ProfilePath": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Protocol": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ProviderName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PuaCount": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "PuaPolicyId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "QfeVersion": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "QueryName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "QueryResults": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "QueryStatus": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ReadOperation": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Reason": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "RemoteEventLogging": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "RestrictedAdminMode": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ReturnCode": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "RuleName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "RunningMode": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SamAccountName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SchemaVersion": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ScriptBlockId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ScriptBlockText": {
                  "type": "keyword"
                },
                "ScriptPath": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ServiceName": {
                  "type": "keyword"
                },
                "ServiceVersion": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ShutdownActionType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ShutdownEventCode": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ShutdownReason": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SidHistory": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Signature": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SignatureStatus": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Signed": {
                  "type": "keyword"
                },
                "SleepInProgress": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceHostname": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceImage": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceIp": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceIsIpv6": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourcePort": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourcePortName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceProcessGUID": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceProcessGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceProcessId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceThreadId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SourceUser": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "StartAddress": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "StartFunction": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "StartModule": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "State": {
                  "type": "keyword"
                },
                "Status": {
                  "type": "keyword"
                },
                "SubjectDomainName": {
                  "type": "keyword"
                },
                "SubjectLogonId": {
                  "type": "keyword"
                },
                "SubjectUserName": {
                  "type": "keyword"
                },
                "SubjectUserSid": {
                  "type": "keyword"
                },
                "SystemSleepTransitionsToOn": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TSId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetDomainName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetFilename": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetImage": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetInfo": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetLinkedLogonId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetLogonGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetLogonId": {
                  "type": "keyword"
                },
                "TargetName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetObject": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetOutboundDomainName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetOutboundUserName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetProcessGUID": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetProcessGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetProcessId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetProcessName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetServerName": {
                  "type": "keyword"
                },
                "TargetSid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetUser": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TargetUserName": {
                  "type": "keyword"
                },
                "TargetUserSid": {
                  "type": "keyword"
                },
                "TerminalSessionId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TestSigning": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TimeSource": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TimeSourceRefId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TokenElevationType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "TransmittedServices": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Type": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UpdateReason": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "User": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UserAccountControl": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UserParameters": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UserPrincipalName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UserSid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UserWorkstations": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UtcTime": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Version": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "VirtualAccount": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "VsmLaunchType": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "VsmPolicy": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Workstation": {
                  "type": "keyword"
                },
                "WorkstationName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "param1": {
                  "type": "keyword"
                },
                "param2": {
                  "type": "keyword"
                },
                "serviceGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "updateGuid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "updateRevisionNumber": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "updateTitle": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "event_id": {
              "type": "long"
            },
            "keywords": {
              "type": "keyword"
            },
            "opcode": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "process": {
              "properties": {
                "pid": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "thread": {
                  "properties": {
                    "id": {
                      "type": "long"
                    }
                  }
                }
              }
            },
            "provider_guid": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "provider_name": {
              "type": "keyword"
            },
            "record_id": {
              "type": "long"
            },
            "task": {
              "type": "keyword"
            },
            "user": {
              "properties": {
                "domain": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "identifier": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "name": {
                  "type": "keyword"
                },
                "type": {
                  "type": "keyword"
                }
              }
            },
            "user_data": {
              "properties": {
                "Address": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Class": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "ListenerName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Param1": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Param2": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Param3": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Reason": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "RmSessionId": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "Session": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "SessionID": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "UTCStartTime": {
                  "type": "date"
                },
                "User": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "binaryData": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "binaryDataSize": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "listenerName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "messageName": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "param1": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "param2": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "xml_name": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }
              }
            },
            "version": {
              "type": "long"
            }
          }
        },
        "winlog-channel": {
          "type": "alias",
          "path": "winlog.channel"
        },
        "winlog-computerObject-name": {
          "type": "alias",
          "path": "winlog.computerObject.name"
        },
        "winlog-computer_name": {
          "type": "alias",
          "path": "winlog.computer_name"
        },
        "winlog-event_data-AuthenticationPackageName": {
          "type": "alias",
          "path": "winlog.event_data.AuthenticationPackageName"
        },
        "winlog-event_data-Company": {
          "type": "alias",
          "path": "winlog.event_data.Company"
        },
        "winlog-event_data-Description": {
          "type": "alias",
          "path": "winlog.event_data.Description"
        },
        "winlog-event_data-Detail": {
          "type": "alias",
          "path": "winlog.event_data.Detail"
        },
        "winlog-event_data-DeviceName": {
          "type": "alias",
          "path": "winlog.event_data.DeviceName"
        },
        "winlog-event_data-FileVersion": {
          "type": "alias",
          "path": "winlog.event_data.FileVersion"
        },
        "winlog-event_data-IntegrityLevel": {
          "type": "alias",
          "path": "winlog.event_data.IntegrityLevel"
        },
        "winlog-event_data-IpAddress": {
          "type": "alias",
          "path": "winlog.event_data.IpAddress"
        },
        "winlog-event_data-KeyLength": {
          "type": "alias",
          "path": "winlog.event_data.KeyLength"
        },
        "winlog-event_data-LogonId": {
          "type": "alias",
          "path": "winlog.event_data.LogonId"
        },
        "winlog-event_data-LogonProcessName": {
          "type": "alias",
          "path": "winlog.event_data.LogonProcessName"
        },
        "winlog-event_data-LogonType": {
          "type": "alias",
          "path": "winlog.event_data.LogonType"
        },
        "winlog-event_data-OriginalFileName": {
          "type": "alias",
          "path": "winlog.event_data.OriginalFileName"
        },
        "winlog-event_data-Path": {
          "type": "alias",
          "path": "winlog.event_data.Path"
        },
        "winlog-event_data-PrivilegeList": {
          "type": "alias",
          "path": "winlog.event_data.PrivilegeList"
        },
        "winlog-event_data-ProcessId": {
          "type": "alias",
          "path": "winlog.event_data.ProcessId"
        },
        "winlog-event_data-ProcessName": {
          "type": "alias",
          "path": "winlog.event_data.ProcessName"
        },
        "winlog-event_data-ProcessPath": {
          "type": "alias",
          "path": "winlog.event_data.ProcessPath"
        },
        "winlog-event_data-Product": {
          "type": "alias",
          "path": "winlog.event_data.Product"
        },
        "winlog-event_data-ScriptBlockText": {
          "type": "alias",
          "path": "winlog.event_data.ScriptBlockText"
        },
        "winlog-event_data-ServiceName": {
          "type": "alias",
          "path": "winlog.event_data.ServiceName"
        },
        "winlog-event_data-Signed": {
          "type": "alias",
          "path": "winlog.event_data.Signed"
        },
        "winlog-event_data-State": {
          "type": "alias",
          "path": "winlog.event_data.State"
        },
        "winlog-event_data-Status": {
          "type": "alias",
          "path": "winlog.event_data.Status"
        },
        "winlog-event_data-SubjectDomainName": {
          "type": "alias",
          "path": "winlog.event_data.SubjectDomainName"
        },
        "winlog-event_data-SubjectLogonId": {
          "type": "alias",
          "path": "winlog.event_data.SubjectLogonId"
        },
        "winlog-event_data-SubjectUserName": {
          "type": "alias",
          "path": "winlog.event_data.SubjectUserName"
        },
        "winlog-event_data-SubjectUserSid": {
          "type": "alias",
          "path": "winlog.event_data.SubjectUserSid"
        },
        "winlog-event_data-TargetLogonId": {
          "type": "alias",
          "path": "winlog.event_data.TargetLogonId"
        },
        "winlog-event_data-TargetServerName": {
          "type": "alias",
          "path": "winlog.event_data.TargetServerName"
        },
        "winlog-event_data-TargetUserName": {
          "type": "alias",
          "path": "winlog.event_data.TargetUserName"
        },
        "winlog-event_data-TargetUserSid": {
          "type": "alias",
          "path": "winlog.event_data.TargetUserSid"
        },
        "winlog-event_data-Workstation": {
          "type": "alias",
          "path": "winlog.event_data.Workstation"
        },
        "winlog-event_data-param1": {
          "type": "alias",
          "path": "winlog.event_data.param1"
        },
        "winlog-event_data-param2": {
          "type": "alias",
          "path": "winlog.event_data.param2"
        },
        "winlog-event_id": {
          "type": "alias",
          "path": "winlog.event_id"
        },
        "winlog-keywords": {
          "type": "alias",
          "path": "winlog.keywords"
        },
        "winlog-provider_name": {
          "type": "alias",
          "path": "winlog.provider_name"
        },
        "winlog-task": {
          "type": "alias",
          "path": "winlog.task"
        },
        "winlog-user-name": {
          "type": "alias",
          "path": "winlog.user.name"
        },
        "winlog-user-type": {
          "type": "alias",
          "path": "winlog.user.type"
        }
      }
    }
  }
}
  1. Manually inserted the document containing the event ID 16 to test if it will trigger the detector alert:
POST os_windows/_doc 
{   
   "@timestamp": "2023-03-24T02:11:00+0000",  
   "winlog.event_id": 16
}
  1. Nothing happened:
    image
    image

@phipiship
Copy link

@petardz here is the output if i execute the following:

GET winlogbeat-7.10.2-test-1/_mapping

I get a very long output, but it contains the following:

.......
"winlog-event_id": {
"type": "alias",
"path": "winlog.event_id"
},
........

@petardz
Copy link
Contributor

petardz commented Mar 24, 2023

Can you show me your Trigger config? You either need to specify rule name or rule tag. Here's example of rule tag:

image

image

@phipiship
Copy link

The two options are set to "any", because many rules are included, so this setting should be okay. If I understand it correctly, I only need to set the name or tag if I want to filter.

grafik

@petardz
Copy link
Contributor

petardz commented Mar 24, 2023

Ah yes, sorry, you're right. This still works for me..

Did you insert this document AFTER creating a detector?

@phipiship
Copy link

Yes, I added it after creating the detector.
I also just set the trigger to the tag and pasted the document again, but still no finding / alert.

In your test it works? Maybe you can provide me with your mapping so that I can try it with it?

@phipiship
Copy link

We now get findings and alerts from the "Raw Disk Access Using Illegitimate Tools" rule.
If I copy a document from these findings, adjust the date and write it manually into the index via

POST winlogbeat-7.10.2-test-1/_doc
{......}

no Finding is generated, although it is the same document as the one that triggered a finding before.
It looks like the detector can't be tested this way, or is that how it works for you @petardz ?

@petardz
Copy link
Contributor

petardz commented Mar 27, 2023

Can't reproduce that. Are you sure that you';re inseting document in index which is monitored by detector?

@phipiship
Copy link

Yes, the documents are written to the correct index.

Do you have a working instance for Windows logs? If yes, how do you test the function of the Sigma rules?

Maybe you can give me your field and alias mappings that I can try with it?

@petardz
Copy link
Contributor

petardz commented Mar 27, 2023

  1. Create log index:
POST windows-logs/_doc
{
  "winlog.event_id": 16
}
  1. Went through Detector creation process. I selected windows-logs index as source, log type as "windows", selected rule "Sysmon Configuration Change" and hit next, next on mappings (1 field automaticlly mapped) and then create. These are requests sent to backend:
POST _plugins/_security_analytics/mappings
{
   "index_name":"windows-logs",
   "rule_topic":"windows",
   "partial":true,
   "alias_mappings":{
      "properties":{
         "winlog-event_id":{
            "type":"alias",
            "path":"winlog.event_id"
         }
      }
   }
}
POST _plugins/_security_analytics/detectors
{
   "type":"detector",
   "detector_type":"windows",
   "name":"test-detector",
   "enabled":true,
   "createdBy":"",
   "schedule":{
      "period":{
         "interval":1,
         "unit":"MINUTES"
      }
   },
   "inputs":[
      {
         "detector_input":{
            "description":"",
            "indices":[
               "windows-logs"
            ],
            "pre_packaged_rules":[
               {
                  "id":"8ac03a65-6c84-4116-acad-dc1558ff7a77"
               }
            ],
            "custom_rules":[
               
            ]
         }
      }
   ],
   "triggers":[
      {
         "name":"test-trigger",
         "sev_levels":[
            
         ],
         "tags":[
            
         ],
         "actions":[
            {
               "id":"",
               "name":"Triggered alert condition:  - Severity: 1 (Highest) - Threat detector: test_detector",
               "destination_id":"",
               "subject_template":{
                  "source":"Triggered alert condition:  - Severity: 1 (Highest) - Threat detector: test_detector",
                  "lang":"mustache"
               },
               "message_template":{
                  "source":"Triggered alert condition: \nSeverity: 1 (Highest)\nThreat detector: test_detector\nDescription: \nDetector data sources:\n\twindows-logs",
                  "lang":"mustache"
               },
               "throttle_enabled":false,
               "throttle":{
                  "value":10,
                  "unit":"MINUTES"
               }
            }
         ],
         "types":[
            "windows"
         ],
         "severity":"1",
         "ids":[
            
         ]
      }
   ]
}
  1. Insert dummy log document to trigger finding/alert generation:
POST windows-logs/_doc
{
  "winlog.event_id": 16
}
  1. After 1 min finding and alert appears for this detector

@fquinino
Copy link

fquinino commented Mar 27, 2023

Hello @petardz ,

I carefully followed all the steps outlined above. While creating the index in the first step, I had to use PUT instead of POST, but despite making this minor adjustment, no alert was triggered. However, I do have all the necessary prints as evidence.

image

image

image

image

image

What is the version of the security analytics plugin you're using?

I'm also putting in the effort to make this work, but unfortunately, I haven't been able to achieve the desired outcome.

Could you share the OpenSearch docker image you're using?

@petardz
Copy link
Contributor

petardz commented Mar 27, 2023

I am using latest main branch (2.6) ran through gradle command: ./gradlew run

@phipiship
Copy link

I have done the steps as described, both on an existing installation that was upgraded from 2.5 to 2.6 in the past, and on a fresh 2.6 installation, both unfortunately did not work.

@petardz
Copy link
Contributor

petardz commented Mar 28, 2023

Did you get any exceptions in logs?

@jaakoob
Copy link
Author

jaakoob commented Mar 28, 2023

I recreated the HTTP requests you posted above in postman (with the modification of the index creation as described by @fquinino). I ran them against a fresh opensearch in a container ("opensearchproject/opensearch:latest"). I did not get any findings nor warnings or errors related to the security analytics module in the logs.

I would appreciate if you can create more documentation for the module as this won´t happen if usable docs exist.

@petardz
Copy link
Contributor

petardz commented Mar 28, 2023

@jaakoob @fquinino Is index autocreation disabled on your cluster? Why step 1 didn't work?

Alternative to step 1 would be:

PUT windows-logs
{
  "mappings": {
    "properties": {
      "winlog.event_id": {
        "type": "long"
      }
    }
  }
}

@petardz
Copy link
Contributor

petardz commented Mar 28, 2023

I am able to reproduce your issue only using official docker images. Some Sigma rules have original field names after parsing instead of mapped, which shouldn't happen. In this case EventID isn't converted to winlog-event_id so this is a reason why it's not working. I'll investigate this.

@fquinino
Copy link

fquinino commented Mar 29, 2023

Hi @petardz ,

I'm currently running the OS 2.6 locally within the security analytics plugin, but encountering an error while attempting to create the detector, as detailed below:

  1. OpenSearch initialization logs:
jackbauer@jsecurity-jump:~/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT$ ./bin/opensearch
[2023-03-29T10:18:30,621][INFO ][o.o.n.Node               ] [jsecurity-jump] version[2.6.0-SNAPSHOT], pid[531], build[tar/7203a5af21a8a009aece1474446b437a3c674db6/2023-03-28T20:18:36.444498Z], OS[Linux/5.15.90.1-microsoft-standard-WSL2/amd64], JVM[Ubuntu/OpenJDK 64-Bit Server VM/11.0.18/11.0.18+10-post-Ubuntu-0ubuntu122.04]
[2023-03-29T10:18:30,626][INFO ][o.o.n.Node               ] [jsecurity-jump] JVM home [/usr/lib/jvm/java-11-openjdk-amd64], using bundled JDK [false]
[2023-03-29T10:18:30,627][INFO ][o.o.n.Node               ] [jsecurity-jump] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-17364001059339799396, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT, -Dopensearch.path.conf=/home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2023-03-29T10:18:30,631][WARN ][o.o.n.Node               ] [jsecurity-jump] version [2.6.0-SNAPSHOT] is a pre-release version of OpenSearch and is not suitable for production
[2023-03-29T10:18:31,790][WARN ][stderr                   ] [jsecurity-jump] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[2023-03-29T10:18:31,790][WARN ][stderr                   ] [jsecurity-jump] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2023-03-29T10:18:31,791][WARN ][stderr                   ] [jsecurity-jump] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2023-03-29T10:18:31,803][INFO ][o.o.s.s.t.SSLConfig      ] [jsecurity-jump] SSL dual mode is disabled
[2023-03-29T10:18:31,803][INFO ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] OpenSearch Config path is /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config
[2023-03-29T10:18:32,168][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] JVM supports TLSv1.3
[2023-03-29T10:18:32,170][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] Config directory is /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/, from there the key- and truststore files are resolved relatively
[2023-03-29T10:18:32,615][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] TLS Transport Client Provider : JDK
[2023-03-29T10:18:32,615][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] TLS Transport Server Provider : JDK
[2023-03-29T10:18:32,616][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] TLS HTTP Provider             : JDK
[2023-03-29T10:18:32,616][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2]
[2023-03-29T10:18:32,616][INFO ][o.o.s.s.DefaultSecurityKeyStore] [jsecurity-jump] Enabled TLS protocols for HTTP layer      : [TLSv1.3, TLSv1.2]
[2023-03-29T10:18:32,626][INFO ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] Clustername: opensearch
[2023-03-29T10:18:32,630][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] Directory /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config has insecure file permissions (should be 0700)
[2023-03-29T10:18:32,630][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] File /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/config.yml has insecure file permissions (should be 0600)
[2023-03-29T10:18:32,630][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] File /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/esnode.pem has insecure file permissions (should be 0600)
[2023-03-29T10:18:32,634][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] File /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/kirk-key.pem has insecure file permissions (should be 0600)
[2023-03-29T10:18:32,634][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] File /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/esnode-key.pem has insecure file permissions (should be 0600)
[2023-03-29T10:18:32,635][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] File /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/root-ca.pem has insecure file permissions (should be 0600)
[2023-03-29T10:18:32,635][WARN ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] File /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/kirk.pem has insecure file permissions (should be 0600)
[2023-03-29T10:18:32,772][INFO ][o.o.i.r.ReindexPlugin    ] [jsecurity-jump] ReindexPlugin reloadSPI called
[2023-03-29T10:18:32,785][INFO ][o.o.i.r.ReindexPlugin    ] [jsecurity-jump] Unable to find any implementation for RemoteReindexExtension
[2023-03-29T10:18:32,811][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [aggs-matrix-stats]
[2023-03-29T10:18:32,811][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [analysis-common]
[2023-03-29T10:18:32,811][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [geo]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [ingest-common]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [ingest-geoip]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [ingest-user-agent]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [lang-expression]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [lang-mustache]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [lang-painless]
[2023-03-29T10:18:32,812][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [mapper-extras]
[2023-03-29T10:18:32,813][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [opensearch-dashboards]
[2023-03-29T10:18:32,813][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [parent-join]
[2023-03-29T10:18:32,813][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [percolator]
[2023-03-29T10:18:32,813][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [rank-eval]
[2023-03-29T10:18:32,813][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [reindex]
[2023-03-29T10:18:32,813][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [repository-url]
[2023-03-29T10:18:32,814][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [systemd]
[2023-03-29T10:18:32,814][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [test-delayed-aggs]
[2023-03-29T10:18:32,814][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded module [transport-netty4]
[2023-03-29T10:18:32,816][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded plugin [opensearch-notifications]
[2023-03-29T10:18:32,816][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded plugin [opensearch-notifications-core]
[2023-03-29T10:18:32,816][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded plugin [opensearch-security]
[2023-03-29T10:18:32,816][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] loaded plugin [opensearch-security-analytics]
[2023-03-29T10:18:32,884][INFO ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml
[2023-03-29T10:18:32,935][INFO ][o.o.e.NodeEnvironment    ] [jsecurity-jump] using [1] data paths, mounts [[/ (/dev/sde)]], net usable_space [935.5gb], net total_space [1006.8gb], types [ext4]
[2023-03-29T10:18:32,936][INFO ][o.o.e.NodeEnvironment    ] [jsecurity-jump] heap size [1gb], compressed ordinary object pointers [true]
[2023-03-29T10:18:33,059][INFO ][o.o.n.Node               ] [jsecurity-jump] node name [jsecurity-jump], node ID [P3LY6niTQEaRB3jc6BFqrw], cluster name [opensearch], roles [ingest, remote_cluster_client, data, cluster_manager]
[2023-03-29T10:18:35,667][WARN ][o.o.s.c.Salt             ] [jsecurity-jump] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
[2023-03-29T10:18:35,720][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Message routing enabled: true
[2023-03-29T10:18:35,844][INFO ][o.o.s.f.SecurityFilter   ] [jsecurity-jump] <NONE> indices are made immutable.
[2023-03-29T10:18:36,166][INFO ][o.o.t.NettyAllocator     ] [jsecurity-jump] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={opensearch.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]
[2023-03-29T10:18:36,263][INFO ][o.o.d.DiscoveryModule    ] [jsecurity-jump] using discovery type [zen] and seed hosts providers [settings]
[2023-03-29T10:18:36,542][WARN ][o.o.g.DanglingIndicesState] [jsecurity-jump] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2023-03-29T10:18:36,865][INFO ][o.o.n.Node               ] [jsecurity-jump] initialized
[2023-03-29T10:18:36,865][INFO ][o.o.n.Node               ] [jsecurity-jump] starting ...
[2023-03-29T10:18:37,055][INFO ][o.o.t.TransportService   ] [jsecurity-jump] publish_address {172.20.99.74:9300}, bound_addresses {[::]:9300}
[2023-03-29T10:18:37,267][INFO ][o.o.b.BootstrapChecks    ] [jsecurity-jump] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2023-03-29T10:18:37,270][INFO ][o.o.c.c.Coordinator      ] [jsecurity-jump] cluster UUID [t-Sf4MYaQbCS9dWe3KIGWg]
[2023-03-29T10:18:37,515][INFO ][o.o.c.s.MasterService    ] [jsecurity-jump] elected-as-cluster-manager ([1] nodes joined)[{jsecurity-jump}{P3LY6niTQEaRB3jc6BFqrw}{6LVFXOHNQY2tbzGKxdEp3w}{172.20.99.74}{172.20.99.74:9300}{dimr}{shard_indexing_pressure_enabled=true} elect leader, _BECOME_CLUSTER_MANAGER_TASK_, _FINISH_ELECTION_], term: 8, version: 92, delta: cluster-manager node changed {previous [], current [{jsecurity-jump}{P3LY6niTQEaRB3jc6BFqrw}{6LVFXOHNQY2tbzGKxdEp3w}{172.20.99.74}{172.20.99.74:9300}{dimr}{shard_indexing_pressure_enabled=true}]}
[2023-03-29T10:18:37,617][INFO ][o.o.c.s.ClusterApplierService] [jsecurity-jump] cluster-manager node changed {previous [], current [{jsecurity-jump}{P3LY6niTQEaRB3jc6BFqrw}{6LVFXOHNQY2tbzGKxdEp3w}{172.20.99.74}{172.20.99.74:9300}{dimr}{shard_indexing_pressure_enabled=true}]}, term: 8, version: 92, reason: Publication{term=8, version=92}
[2023-03-29T10:18:37,652][INFO ][o.o.d.PeerFinder         ] [jsecurity-jump] setting findPeersInterval to [1s] as node commission status = [true] for local node [{jsecurity-jump}{P3LY6niTQEaRB3jc6BFqrw}{6LVFXOHNQY2tbzGKxdEp3w}{172.20.99.74}{172.20.99.74:9300}{dimr}{shard_indexing_pressure_enabled=true}]
[2023-03-29T10:18:37,817][INFO ][o.o.h.AbstractHttpServerTransport] [jsecurity-jump] publish_address {172.20.99.74:9200}, bound_addresses {[::]:9200}
[2023-03-29T10:18:37,817][INFO ][o.o.n.Node               ] [jsecurity-jump] started
[2023-03-29T10:18:37,817][INFO ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] Node started
[2023-03-29T10:18:37,818][INFO ][o.o.s.c.ConfigurationRepository] [jsecurity-jump] Will attempt to create index .opendistro_security and default configs if they are absent
[2023-03-29T10:18:37,819][INFO ][o.o.s.OpenSearchSecurityPlugin] [jsecurity-jump] 0 OpenSearch Security modules loaded so far: []
[2023-03-29T10:18:37,820][INFO ][o.o.s.c.ConfigurationRepository] [jsecurity-jump] Background init thread started. Install default config?: true
[2023-03-29T10:18:37,829][INFO ][o.o.g.GatewayService     ] [jsecurity-jump] recovered [9] indices into cluster_state
[2023-03-29T10:18:37,939][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[.opensearch-sap-detectors-config/sXX5M664TgCqdSjB0A9Igg]
[2023-03-29T10:18:38,145][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[.opensearch-sap-pre-packaged-rules-config/W8mZohmnRa-AEl6SpjoX-w]
[2023-03-29T10:18:38,173][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[security-auditlog-2023.03.29/J5IC6LHVTge4wHmuU1sIdQ]
[2023-03-29T10:18:38,180][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[.opendistro_security/iUmpzzKmQRSnlLiDdXSByQ]
[2023-03-29T10:18:38,348][INFO ][o.o.s.c.ConfigurationRepository] [jsecurity-jump] Index .opendistro_security already exists
[2023-03-29T10:18:38,349][INFO ][o.o.s.c.ConfigurationRepository] [jsecurity-jump] Node started, try to initialize it. Wait for at least yellow cluster state....
[2023-03-29T10:18:38,628][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'config' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/config.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,751][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id config, skipping update.
[2023-03-29T10:18:38,752][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'roles' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/roles.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,767][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id roles, skipping update.
[2023-03-29T10:18:38,771][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'rolesmapping' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/roles_mapping.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,779][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id rolesmapping, skipping update.
[2023-03-29T10:18:38,780][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'internalusers' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/internal_users.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,788][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[.opensearch-notifications-config/5qnd_5GyRRirBOasKIJ6vA]
[2023-03-29T10:18:38,792][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id internalusers, skipping update.
[2023-03-29T10:18:38,793][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'actiongroups' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/action_groups.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,801][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id actiongroups, skipping update.
[2023-03-29T10:18:38,805][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[.kibana_92668751_admin_1/BqQgavWZQPSe4duHbQP3UA]
[2023-03-29T10:18:38,816][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'tenants' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/tenants.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,823][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id tenants, skipping update.
[2023-03-29T10:18:38,824][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[.kibana_1/s2_8z1biQjqhNb1FYpU3wg]
[2023-03-29T10:18:38,825][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'nodesdn' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/nodes_dn.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
[2023-03-29T10:18:38,833][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id nodesdn, skipping update.
[2023-03-29T10:18:38,833][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'whitelist' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/whitelist.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
[2023-03-29T10:18:38,835][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id whitelist, skipping update.
[2023-03-29T10:18:38,836][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'allowlist' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/allowlist.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
[2023-03-29T10:18:38,841][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id allowlist, skipping update.
[2023-03-29T10:18:38,842][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Will update 'audit' with /home/jackbauer/Projects/OpenSearch/build/distribution/local/opensearch-2.6.0-SNAPSHOT/config/opensearch-security/audit.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2023-03-29T10:18:38,881][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[security-auditlog-2023.03.28/oKZ-Efo7TFCX9ErABN_U2Q]
[2023-03-29T10:18:38,900][INFO ][o.o.s.s.ConfigHelper     ] [jsecurity-jump] Index .opendistro_security already contains doc with id audit, skipping update.
[2023-03-29T10:18:39,185][INFO ][stdout                   ] [jsecurity-jump] [FINE] No subscribers registered for event class org.opensearch.security.securityconf.DynamicConfigFactory$NodesDnModelImpl
[2023-03-29T10:18:39,186][INFO ][stdout                   ] [jsecurity-jump] [FINE] No subscribers registered for event class org.greenrobot.eventbus.NoSubscriberEvent
[2023-03-29T10:18:39,186][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing on REST API is enabled.
[2023-03-29T10:18:39,186][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from REST API auditing.
[2023-03-29T10:18:39,187][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing on Transport API is enabled.
[2023-03-29T10:18:39,188][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from Transport API auditing.
[2023-03-29T10:18:39,188][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing of request body is enabled.
[2023-03-29T10:18:39,188][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Bulk requests resolution is disabled during request auditing.
[2023-03-29T10:18:39,189][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Index resolution is enabled during request auditing.
[2023-03-29T10:18:39,189][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Sensitive headers auditing is enabled.
[2023-03-29T10:18:39,189][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing requests from kibanaserver users is disabled.
[2023-03-29T10:18:39,192][WARN ][o.o.s.a.r.AuditMessageRouter] [jsecurity-jump] No endpoint configured for categories [BAD_HEADERS, FAILED_LOGIN, MISSING_PRIVILEGES, GRANTED_PRIVILEGES, OPENDISTRO_SECURITY_INDEX_ATTEMPT, SSL_EXCEPTION, AUTHENTICATED, INDEX_EVENT, COMPLIANCE_DOC_READ, COMPLIANCE_DOC_WRITE, COMPLIANCE_EXTERNAL_CONFIG, COMPLIANCE_INTERNAL_CONFIG_READ, COMPLIANCE_INTERNAL_CONFIG_WRITE], using default endpoint
[2023-03-29T10:18:39,192][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing of external configuration is disabled.
[2023-03-29T10:18:39,193][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing of internal configuration is enabled.
[2023-03-29T10:18:39,194][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing only metadata information for read request is enabled.
[2023-03-29T10:18:39,194][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing will watch {} for read requests.
[2023-03-29T10:18:39,194][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing read operation requests from kibanaserver users is disabled.
[2023-03-29T10:18:39,195][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing only metadata information for write request is enabled.
[2023-03-29T10:18:39,200][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing diffs for write requests is disabled.
[2023-03-29T10:18:39,201][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing write operation requests from kibanaserver users is disabled.
[2023-03-29T10:18:39,204][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Auditing will watch <NONE> for write requests.
[2023-03-29T10:18:39,205][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] .opendistro_security is used as internal security index.
[2023-03-29T10:18:39,205][INFO ][o.o.s.a.i.AuditLogImpl   ] [jsecurity-jump] Internal index used for posting audit logs is null
[2023-03-29T10:18:39,215][INFO ][o.o.s.c.ConfigurationRepository] [jsecurity-jump] Hot-reloading of audit configuration is enabled
[2023-03-29T10:18:39,216][INFO ][o.o.s.c.ConfigurationRepository] [jsecurity-jump] Node 'jsecurity-jump' initialized
[2023-03-29T10:18:39,314][INFO ][o.o.p.PluginsService     ] [jsecurity-jump] PluginService:onIndexModule index:[windows-logs/1TfzFZf2RuuFIDF2Pt-ZmA]
[2023-03-29T10:18:39,383][INFO ][o.o.c.r.a.AllocationService] [jsecurity-jump] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[windows-logs][0]]]).
  1. Adding the detector using the YAML you provided in the previous post:

image

{
  "error": {
    "root_cause": [
      {
        "type": "security_analytics_exception",
        "reason": "Unknown error"
      }
    ],
    "type": "security_analytics_exception",
    "reason": "Unknown error",
    "caused_by": {
      "type": "exception",
      "reason": "org.opensearch.action.search.SearchPhaseExecutionException: "
    }
  },
  "status": 500
}
  1. OpenSearch logging error during the execution above:
[2023-03-29T10:20:32,078][ERROR][o.o.s.u.SecurityAnalyticsException] [jsecurity-jump] Security Analytics error:
org.opensearch.action.search.SearchPhaseExecutionException:
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:663) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:432) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:421) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.FetchSearchPhase.moveToNextPhase(FetchSearchPhase.java:299) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.FetchSearchPhase.lambda$innerRun$1(FetchSearchPhase.java:139) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.FetchSearchPhase.innerRun(FetchSearchPhase.java:151) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.FetchSearchPhase$1.doRun(FetchSearchPhase.java:123) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:806) [opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.IllegalStateException: failed to find action [org.opensearch.action.ActionType@c98b1483] to execute
        at org.opensearch.client.node.NodeClient.transportAction(NodeClient.java:145) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:465) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.commons.alerting.AlertingPluginInterface.indexMonitor(AlertingPluginInterface.kt:44) ~[?:?]
        at org.opensearch.securityanalytics.transport.TransportIndexDetectorAction.createMonitorFromQueries(TransportIndexDetectorAction.java:252) ~[?:?]
        at org.opensearch.securityanalytics.transport.TransportIndexDetectorAction$AsyncIndexDetectorsAction$9.onResponse(TransportIndexDetectorAction.java:1050) ~[?:?]
        at org.opensearch.securityanalytics.transport.TransportIndexDetectorAction$AsyncIndexDetectorsAction$9.onResponse(TransportIndexDetectorAction.java:1021) ~[?:?]
        at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:113) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:107) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.ActionListener$5.onResponse(ActionListener.java:266) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.sendSearchResponse(AbstractSearchAsyncAction.java:657) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.ExpandSearchPhase.run(ExpandSearchPhase.java:132) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:427) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        ... 14 more
[2023-03-29T10:20:32,093][WARN ][r.suppressed             ] [jsecurity-jump] path: /_plugins/_security_analytics/detectors, params: {pretty=true}
org.opensearch.securityanalytics.util.SecurityAnalyticsException: Unknown error
        at org.opensearch.securityanalytics.util.SecurityAnalyticsException.wrap(SecurityAnalyticsException.java:51) ~[?:?]
        at org.opensearch.securityanalytics.transport.TransportIndexDetectorAction$AsyncIndexDetectorsAction.lambda$finishHim$0(TransportIndexDetectorAction.java:1168) ~[?:?]
        at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:73) [opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:88) ~[opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:806) [opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.Exception: org.opensearch.action.search.SearchPhaseExecutionException:
        ... 9 more

Can you provide me with the repository, and specific tag/commit ID you used to successfully build and run the test locally? I'm interested in using this plugin and would like to conduct some testing.

@petardz
Copy link
Contributor

petardz commented Apr 4, 2023

I investigated above issue with no findings and problem was in mappings. Locally, I fixed windows mappings, which are buggy in 2.6. We have "EventId" defined and it no "EventID". Because of this, rule won't work until "EventID" field or alias is defined in index.

@jaakoob
Copy link
Author

jaakoob commented Apr 4, 2023

Thanks for your help! Do you think it´s possible to handle this through an alias in the security analytics mapping:

{ "index_name":"windows-logs", "rule_topic":"windows", "partial":true, "alias_mappings":{ "properties":{ "winlog-event_id":{ "type":"alias", "path":"winlog.event_id" }, "Eventid":{ "type": "alias", "path": "EventID" } } } }

I can´t get it to work with that and a current version.

Is there a prebuilt preview for 2.7.0. available or do we have to build it ourselves?

@petardz
Copy link
Contributor

petardz commented Apr 4, 2023

Yea, because it didn't convert EventID to winlog-event_id, the actual used field in doc level query is EventID. You need to provide either that field or alias named EventID. You can use core's mapping API to apply alias:

PUT my_index/_mapping
{
  "properties": {
    "EventID": {
      "type": "alias",
      "path": "winlog.event_id"
    }
  }
}

Also, be advised that issue was discovered in Alerting plugin when ingesteddocuments are "nested"(non-flattened). This is addressed by this PR: link. This means that for now, you will have to ingest document with flattened fields. Example:

{
...
  "winlog.event_id": 12345,
...
}

@jaakoob
Copy link
Author

jaakoob commented Apr 4, 2023

Ok, thanks for the clarification. I understood it the other way around (between Id and ID). It is now working for us.

Is there an issue for the windows mappings we can link here and close this issue?

@sbcd90
Copy link
Collaborator

sbcd90 commented May 8, 2023

hi @jaakoob , thanks for confirming that the issue is fixed. Can you please let us know if you have more questions on this? i'm closing this issue but feel free to open it again.

@sbcd90 sbcd90 closed this as completed May 8, 2023
riysaxen-amzn pushed a commit to riysaxen-amzn/security-analytics that referenced this issue Feb 20, 2024
…t#368)

* [FEATURE] Detector must have at least one alert set opensearch-project#288

Signed-off-by: Jovan Cvetkovic <[email protected]>

* [FEATURE] Upgrade Vega-tooltips plugin version opensearch-project#254

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Upgrade vega tooltips to use custom tooltip format opensearch-project#368

Signed-off-by: Jovan Cvetkovic <[email protected]>

* [BUG] Create detector | Interval field can be empty opensearch-project#378

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Adjust styling for Finding details flyout opensearch-project#369

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* detector unit tests

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Feature/update vertical domain #372

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Unit tests for public components opensearch-project#383

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Unit tests for public components opensearch-project#383

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Unit tests for public components opensearch-project#383

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Unit tests for public components opensearch-project#383

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Unit tests for public components opensearch-project#383
[BUG] Detector Edit | Custom rule are not selected on update rules opensearch-project#406

Signed-off-by: Jovan Cvetkovic <[email protected]>

* Unit tests for public components opensearch-project#383
[BUG] Detector Edit | Custom rule are not selected on update rules opensearch-project#406

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* PR code review

Signed-off-by: Jovan Cvetkovic <[email protected]>

* [FEATURE] Create detector | Make data source multi-select field opensearch-project#419

Signed-off-by: Jovan Cvetkovic <[email protected]>

* unit tests fix

Signed-off-by: Jovan Cvetkovic <[email protected]>

---------

Signed-off-by: Jovan Cvetkovic <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

6 participants