RFC-Stage-0: Proposal to add a "ticket" schema / field definition to ECS

[kc-comcast]

RFC Stage 0: Strawperson use case for creation of a ticket schema within ECS.

• Have you signed the contributor license agreement? Y
• Have you followed the contributor guidelines? Y
• For proposing substantial changes or additions to the schema, have you reviewed the RFC process? Y
• If submitting code/script changes, have you verified all tests pass locally using make test? N/A
• If submitting schema/fields updates, have you generated new artifacts by running make and committed those changes? N/A for now
• Is your pull request against master? Unless there is a good reason otherwise, we prefer pull requests against master and will backport as needed. Y
• Have you added an entry to the CHANGELOG.next.md? N/A for now

[cla-checker-service]

💚 CLA has been signed

[kc-comcast]

Not sure why it is saying I haven't sign the Contributor Agreement, I completed one yesterday and my company has a Contributor License Agreement with Elastic. Guessing I signed the wrong one yesterday since I selected the company option, just signed the individual as well.

[kgeller]

Hi @kc-comcast ! It appears the check is failing because you signed the CLA with a different email address than is associated with your commit. I think you should be able to just add the email you committed with to your GitHub profile.

[kc-comcast]

Hi @kc-comcast ! It appears the check is failing because you signed the CLA with a different email address than is associated with your commit. I think you should be able to just add the email you committed with to your GitHub profile.

Should all be from my same work email.

[kgeller]

Should all be from my same work email.

They are both your work domain, but the one you signed with follows first_last, and your commit appears more like a username (ends in 00).

[kc-comcast]

Should all be from my same work email.

They are both your work domain, but the one you signed with follows first_last, and your commit appears more like a username (ends in 00).

Didn't realize it used an email alias on me. I signed the agreement with that email as well, and have linked that alias to my GitHub account.

[ebeahan]

Thanks, @kc-comcast; the CLA check is ✅ now!

Sorry for the delay in providing an initial review. We're taking a look and will have some feedback for you shortly.

[kgeller]

Hey @kc-comcast!

There are some big picture concepts here that would definitely benefit ECS, like metadata for Enterprise Search. However, I have some concerns about how broad the fieldset would be if we try to cover all the potentially different types (change requests, incidents, etc) that you mention. Also, the name ticket itself can be an overloaded term, and we, the ECS team, always try to make sure our naming is as clear as possible.

Do you happen to have a high level example of how you would utilize this proposed fieldset?

[kc-comcast]

Agree there is a broad scope here for various types of tickets. I had first thought about handling this on smaller scale focusing on the types of tickets, but the field duplication would be huge. As I thought more about it, I came to the realization we are talking about any type of ticket in a general scope, and almost all have some of the same basic characteristics/properties for the object. There are a great many ticketing tools out there (Jira, ServiceNow, Remedy, Trello, Rally, etc.). Customization on these platforms is extensive to allow users to add their own personalized fields to fit their business case. However, one thing all platforms have in common, is they predominantly all rely on the same core fields being available "out of the box". Those core fields are the things a user are most likely to search on.

ticket.number --> Ticket number
ticket.type --> Ticket type (Incident, Change, Alert, Event, Request, etc.)
ticket.severity --> Severity of the ticket
ticket.priority --> Priority of the ticket
ticket.description.short --> Short description of the ticket
ticket.description.long --> Long/full description of the ticket
ticket.status --> Current status of the ticket
ticket.state --> Current state of the ticket
ticket.submitter --> Who created the ticket
ticket.assignee --> Who is the ticket assigned to
ticket.requester --> Who made the request
ticket.created --> Timestamp when the ticket was created

The sample schema I created in my repo has some more, but the above are fairly constant properties of a ticket which would likely be heavily used. The ticket.type field handles defining the type of ticket and allows for filtering of types for query.

In terms of my use case, my team develops our ITSM ticketing platform. We are working on the observability aspect of our platform by centralizing our logs within Elasticsearch. As I worked to defined the schema for our logging, one key thing came up, which is most troubleshooting or queries we will perform will be based on the ticket entered into our platform. Also, to better understand the success / failure of our platform processing the ticket, we need to understand the details and attributes assigned to it. I proposed, and implemented this light weight schema for us to enable that use. As I was developing the mapping, it occurred to me there is a much larger application for this, especially with Enterprise Search, thus I sought approval for making the OS contribution.

In the Enterprise Search aspect, these same properties of tickets generally exist through all of the various ticketing platforms mentioned above. Additionally, many companies employ multiple ticketing platforms. Trying to know which platform to search to find the ticket to account for an action can be difficult, especially if a company is transitioning ticketing platforms, or isn't standardized on a single one for things like Incidents and Change requests. Depending upon the company, different business policies can be implemented. It is not uncommon to say every change requires a Change Request, and every incident requires an Incident. However, what happens when an Incident creates the need to make a change. Some companies require the other ticket type to be created, while others may say you can act under the open Incident ticket, and there may be other policies out there. By centralizing the tickets for search, you can figure out what ticket was used to approve a particular change going to production. You can also leverage it to identify what recent changes have occurred which may be the root cause of your active Incident. From there, you may want to dig further into the change to identify what functionality was included with the multitude of potential changes that went out during your timeframe to try and isolate the User Story feature which may be the culprit to the active incident. That's probably the use case with the most meat and potatoes around it. I'm sure there are many others and it all depends upon your business policies and your ITSM / ITIL implementations.

[ebeahan]

I think capturing certain ticket management details might make sense in ECS for some use cases.

Like @kgeller, I'm unsure about using ticket.* for the naming, but we can continue to discuss as the RFC advances 😃 .

I do think it'd be preferred if we can leverage other ECS fields where possible versus replicating fields underneath a new field set, like ticket.*. For example, thinking of how to leverage the existing, top-level user.* fields or relying on @timestamp over ticket.created. However, for stage 0 we don't have to get into details about specific fields.