Add documentation for the UBI plugin (opensearch-project#7284)

* gathering potential documentation attempts Signed-off-by: RasonJ <[email protected]> * considering the dashboard tutorial Signed-off-by: RasonJ <[email protected]> * place holder for js data structure usage Signed-off-by: RasonJ <[email protected]> * data-structures placeholder Signed-off-by: RasonJ <[email protected]> * Updating index links Signed-off-by: RasonJ <[email protected]> * adding old doc to be merged Signed-off-by: RasonJ <[email protected]> * Starting to link things together Signed-off-by: RasonJ <[email protected]> * fix broken link Signed-off-by: RasonJ <[email protected]> * respond to vale Signed-off-by: RasonJ <[email protected]> * more vale violations Signed-off-by: RasonJ <[email protected]> * name files consistently with docs site and fix links. Signed-off-by: RasonJ <[email protected]> * vale Signed-off-by: RasonJ <[email protected]> * Minor tweaks. Moved Ubi under SEARCH. Signed-off-by: RasonJ <[email protected]> * add label for versining of spec and OS version Signed-off-by: RasonJ <[email protected]> * try to sort out vale error Signed-off-by: RasonJ <[email protected]> * Converting mermaid diagrams to png's Signed-off-by: RasonJ <[email protected]> * Updating query_id mermaid code Signed-off-by: RasonJ <[email protected]> * Better way to ignore the mermaid scripts in the md files Signed-off-by: RasonJ <[email protected]> * description updates Signed-off-by: RasonJ <[email protected]> * schema.md updating (still need to update the mermaid diagram) Signed-off-by: RasonJ <[email protected]> * schema updates Signed-off-by: RasonJ <[email protected]> * updates Signed-off-by: RasonJ <[email protected]> * Rebuilding main Signed-off-by: RasonJ <[email protected]> * merging in images Signed-off-by: RasonJ <[email protected]> * Updating UBI spec number Signed-off-by: RasonJ <[email protected]> * Use released version Signed-off-by: Eric Pugh <[email protected]> * Update _search-plugins/index.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/index.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/index.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Addressing PR feedback Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Adding dsl intro Signed-off-by: RasonJ <[email protected]> * Adding intro sentence Signed-off-by: RasonJ <[email protected]> * Title adjust Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Addressing PR feedback Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Addressing pr feedback Signed-off-by: RasonJ <[email protected]> * Addressing PR feedback Signed-off-by: RasonJ <[email protected]> * Fixing vale errors Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: RasonJ <[email protected]> * Finishing initial pr feedback Signed-off-by: RasonJ <[email protected]> * Next round of PR feedback Signed-off-by: RasonJ <[email protected]> * Describing chorus workbench link Signed-off-by: RasonJ <[email protected]> * Adding captions for result tables Signed-off-by: RasonJ <[email protected]> * PR clean up Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Adding a few more suggestions Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * updating query filter for laptos Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * PR feedback Signed-off-by: RasonJ <[email protected]> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * PR feedback Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Co-authored-by: Heather Halter <[email protected]> Signed-off-by: RasonJ <[email protected]> * Apply suggestions from code review Edits to all files to comply with OpenSearch standards; nav_order updates Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review Missed a file - more commits to the sql-query topic Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/sql-queries.md Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review Cleaned up the sample query topic Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/dsl-queries.md Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/dsl-queries.md Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review Accepted editorial suggestions. Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Heather Halter <[email protected]> * Update index.md Reformatted table info Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/dsl-queries.md Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review Signed-off-by: Heather Halter <[email protected]> * Update index.md Signed-off-by: Heather Halter <[email protected]> * Update schemas.md Signed-off-by: Heather Halter <[email protected]> * Update index.md Added a missing link and fixed the table. Signed-off-by: Heather Halter <[email protected]> * Update index.md Changed the bold to italics Signed-off-by: Heather Halter <[email protected]> * Update ubi-dashboard-tutorial.md Removed unnecessary note tag. Signed-off-by: Heather Halter <[email protected]> * Update schemas.md Inserted comma Signed-off-by: Heather Halter <[email protected]> * Update sql-queries.md Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review There were some hidden comments that I found in this file. Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/sql-queries.md Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/sql-queries.md Signed-off-by: Heather Halter <[email protected]> * Apply suggestions from code review Signed-off-by: Heather Halter <[email protected]> * Update _search-plugins/ubi/schemas.md Signed-off-by: kolchfa-aws <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> * Update schemas.md Removed links on 'object_id' and 'query_id' Signed-off-by: Heather Halter <[email protected]> * Update sql-queries.md removed a note tag and fixed line 326 Signed-off-by: Heather Halter <[email protected]> * Update sql-queries.md One more table heading Signed-off-by: Heather Halter <[email protected]> --------- Signed-off-by: RasonJ <[email protected]> Signed-off-by: Eric Pugh <[email protected]> Signed-off-by: Heather Halter <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Eric Pugh <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Heather Halter <[email protected]> Co-authored-by: Nathan Bower <[email protected]>
gargharsh3134 · Jul 3, 2024 · 5db02ca · 5db02ca
1 parent f149db5
commit 5db02ca
Show file tree

Hide file tree

Showing 30 changed files with 1,079 additions and 1 deletion.
diff --git a/.github/vale/styles/Vocab/OpenSearch/Plugins/accept.txt b/.github/vale/styles/Vocab/OpenSearch/Plugins/accept.txt
@@ -26,4 +26,5 @@ Search Relevance plugin
 Security plugin
 Security Analytics plugin
 SQL plugin
-Trace Analytics plugin
+Trace Analytics plugin
+User Behavior Insights
diff --git a/.ruby-version b/.ruby-version
@@ -0,0 +1 @@
+3.3.2
diff --git a/_search-plugins/index.md b/_search-plugins/index.md
@@ -70,6 +70,8 @@ OpenSearch provides the following search relevance features:
 
 - [Querqy]({{site.url}}{{site.baseurl}}/search-plugins/querqy/): Offers query rewriting capability.
 
+- [User Behavior Insights]({{site.url}}{{site.baseurl}}/search-plugins/ubi/): Links user behavior to user queries to improve search quality.
+
 ## Search results
 
 OpenSearch supports the following commonly used operations on search results:

diff --git a/_search-plugins/ubi/data-structures.md b/_search-plugins/ubi/data-structures.md
@@ -0,0 +1,204 @@
+---
+layout: default
+title: UBI client data structures
+parent: User Behavior Insights
+has_children: false
+nav_order: 10
+---
+
+# UBI client data structures
+
+Data structures are used to create events that follow the [User Behavior Insights (UBI) event schema specification](https://github.com/o19s/ubi).
+For more information about the schema, see [UBI index schemas]({{site.url}}{{site.baseurl}}/search-plugins/ubi/schemas/).
+
+
+You must provide an implementation for the following functions:
+- `getClientId()`
+- `getQueryId()`
+
+You can also optionally provide an implementation for the following functions:
+- `getSessionId()`
+- `getPageId()`
+
+
+The following JavaScript structures can be used as a starter implementation to serialize UBI events into schema-compatible JSON:
+```js
+/*********************************************************************************************
+ * Ubi Event data structures
+ * The following structures help ensure adherence to the UBI event schema
+ *********************************************************************************************/
+
+
+
+export class UbiEventData {
+  constructor(object_type, id=null, description=null, details=null) {
+    this.object_id_field = object_type;
+    this.object_id = id;
+    this.description = description;
+    this.object_detail = details;
+  }
+}
+export class UbiPosition{
+  constructor({ordinal=null, x=null, y=null, trail=null}={}) {
+    this.ordinal = ordinal;
+    this.x = x;
+    this.y = y;
+    if(trail)
+      this.trail = trail;
+    else {
+      const trail = getTrail();
+      if(trail && trail.length > 0)
+        this.trail = trail;
+    }
+  }
+}
+
+
+export class UbiEventAttributes {
+  /**
+   * Tries to prepopulate common event attributes
+   * The developer can add an `object` that the user interacted with and
+   *   the site `position` information relevant to the event
+   * 
+   * Attributes, other than `object` or `position` can be added in the form:
+   * attributes['item1'] = 1
+   * attributes['item2'] = '2'
+   *
+   * @param {*} attributes: object with general event attributes 
+   * @param {*} object: the data object the user interacted with
+   * @param {*} position: the site position information
+   */
+  constructor({attributes={}, object=null, position=null}={}) {
+    if(attributes != null){
+      Object.assign(this, attributes);
+    }
+    if(object != null && Object.keys(object).length > 0){
+      this.object = object;
+    }
+    if(position != null && Object.keys(position).length > 0){
+      this.position = position;
+    }
+    this.setDefaultValues();
+  }
+
+  setDefaultValues(){
+    try{
+        if(!this.hasOwnProperty('dwell_time') && typeof TimeMe !== 'undefined'){
+          this.dwell_time = TimeMe.getTimeOnPageInSeconds(window.location.pathname);
+        }
+
+        if(!this.hasOwnProperty('browser')){
+          this.browser = window.navigator.userAgent;
+        }
+
+        if(!this.hasOwnProperty('page_id')){
+          this.page_id = window.location.pathname;
+        }
+        if(!this.hasOwnProperty('session_id')){
+          this.session_id = getSessionId();
+        }
+
+        if(!this.hasOwnProperty('page_id')){
+          this.page_id = getPageId();
+        }
+
+        if(!this.hasOwnProperty('position') || this.position == null){
+          const trail = getTrail();
+          if(trail.length > 0){
+            this.position = new UbiPosition({trail:trail});
+          }
+        } 
+        // ToDo: set IP
+    }
+    catch(error){
+      console.log(error);
+    }
+  }
+}
+
+
+
+export class UbiEvent {
+  constructor(action_name, {message_type='INFO', message=null, event_attributes={}, data_object={}}={}) {
+    this.action_name = action_name;
+    this.client_id = getClientId();
+    this.query_id = getQueryId();
+    this.timestamp = Date.now();
+
+    this.message_type = message_type;
+    if( message )
+      this.message = message;
+
+    this.event_attributes = new UbiEventAttributes({attributes:event_attributes, object:data_object});
+  }
+
+  /**
+   * Use to suppress null objects in the json output
+   * @param key
+   * @param value
+   * @returns
+   */
+  static replacer(key, value){
+    if(value == null || 
+      (value.constructor == Object && Object.keys(value).length === 0)) {
+      return undefined;
+    }
+    return value;
+  }
+
+  /**
+   *
+   * @returns json string
+   */
+  toJson() {
+    return JSON.stringify(this, UbiEvent.replacer);
+  }
+}
+```
+{% include copy.html %}
+
+# Sample usage
+
+```js
+export function logUbiMessage(event_type, message_type, message){
+  let e = new UbiEvent(event_type, {
+    message_type:message_type,
+    message:message
+  });
+  logEvent(e);
+}
+
+export function logDwellTime(action_name, page, seconds){
+  console.log(`${page} => ${seconds}`);
+  let e = new UbiEvent(action_name, {
+    message:`On page ${page} for ${seconds} seconds`,
+    event_attributes:{
+      session_id: getSessionId()},
+      dwell_seconds:seconds
+      },
+    data_object:TimeMe
+  });
+  logEvent(e);
+}
+
+/**
+ * ordinal is the number within a list of results
+ * for the item that was clicked
+ */ 
+export function logItemClick(item, ordinal){
+  let e = new UbiEvent('item_click', {
+    message:`Item ${item['object_id']} was clicked`,
+    event_attributes:{session_id: getSessionId()},
+    data_object:item,    
+  });
+  e.event_attributes.position.ordinal = ordinal;
+  logEvent(e);
+}
+
+export function logEvent( event ){
+  // some configured http client 
+  return client.index( index = 'ubi_events', body = event.toJson());
+}
+
+```
+{% include copy.html %}
diff --git a/_search-plugins/ubi/dsl-queries.md b/_search-plugins/ubi/dsl-queries.md
@@ -0,0 +1,101 @@
+---
+layout: default
+title: Example UBI query DSL queries
+parent: User Behavior Insights
+has_children: false
+nav_order: 15
+---
+
+# Example UBI query DSL queries
+
+You can use the OpenSearch search query language, [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/), to write User Behavior Insights (UBI) queries. The following example returns the number of times that each `action_name` event occurs.
+For more extensive analytic queries, see [Example UBI SQL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/sql-queries/). 
+#### Example request
+```json
+GET ubi_events/_search
+{
+  "size":0, 
+  "aggs":{ 
+    "event_types":{
+      "terms": {
+        "field":"action_name", 
+        "size":10
+      }
+    }
+  }
+}
+```
+{% include copy.html %}
+
+#### Example response
+
+```json
+{
+  "took": 1,
+  "timed_out": false,
+  "_shards": {
+    "total": 1,
+    "successful": 1,
+    "skipped": 0,
+    "failed": 0
+  },
+  "hits": {
+    "total": {
+      "value": 10000,
+      "relation": "gte"
+    },
+    "max_score": null,
+    "hits": []
+  },
+  "aggregations": {
+    "event_types": {
+      "doc_count_error_upper_bound": 0,
+      "sum_other_doc_count": 0,
+      "buckets": [
+        {
+          "key": "brand_filter",
+          "doc_count": 3084
+        },
+        {
+          "key": "product_hover",
+          "doc_count": 3068
+        },
+        {
+          "key": "button_click",
+          "doc_count": 3054
+        },
+        {
+          "key": "product_sort",
+          "doc_count": 3012
+        },
+        {
+          "key": "on_search",
+          "doc_count": 3010
+        },
+        {
+          "key": "type_filter",
+          "doc_count": 2925
+        },
+        {
+          "key": "login",
+          "doc_count": 2433
+        },
+        {
+          "key": "logout",
+          "doc_count": 1447
+        },
+        {
+          "key": "new_user_entry",
+          "doc_count": 207
+        }
+      ]
+    }
+  }
+}
+```
+{% include copy.html %}
+
+You can run the preceding queries in the OpenSearch Dashboards [Query Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/). 
+
+A demo workbench with sample data can be found here: 
+[http://chorus-opensearch-edition.dev.o19s.com:5601/app/OpenSearch-query-workbench](http://chorus-OpenSearch-edition.dev.o19s.com:5601/app/OpenSearch-query-workbench).
diff --git a/_search-plugins/ubi/index.md b/_search-plugins/ubi/index.md
@@ -0,0 +1,50 @@
+---
+layout: default
+title: User Behavior Insights
+has_children: true
+nav_order: 90
+redirect_from:
+  - /search-plugins/ubi/
+---
+# User Behavior Insights
+
+**Introduced 2.15**
+{: .label .label-purple }
+
+**References UBI Specification 1.0.0**
+{: .label .label-purple }
+
+User Behavior Insights (UBI) is a plugin that captures client-side events and queries for the purposes of improving search relevance and the user experience.
+It is a causal system, linking a user's query to all of their subsequent interactions with your application until they perform another search.
+
+UBI includes the following elements:
+* A machine-readable [schema](https://github.com/o19s/ubi) that faciliates interoperablity of the UBI specification.
+* An OpenSearch [plugin](https://github.com/opensearch-project/user-behavior-insights) that facilitates the storage of client-side events and queries.
+* A client-side JavaScript [example reference implementation]({{site.url}}{{site.baseurl}}/search-plugins/ubi/data-structures/) that shows how to capture events and send them to the OpenSearch UBI plugin.
+
+<!-- vale off -->
+
+The UBI documentation is organized into two categories: *Explanation and reference* and *Tutorials and how-to guides*:   
+
+*Explanation and reference*
+
+| Link | Description |
+| :--------- | :------- |
+| [UBI Request/Response Specification](https://github.com/o19s/ubi/) | The industry-standard schema for UBI requests and responses. The current version references UBI Specification 1.0.0.  |
+| [UBI index schema]({{site.url}}{{site.baseurl}}/search-plugins/ubi/schemas/) | Documentation on the individual OpenSearch query and event stores. |
+
+
+*Tutorials and how-to guides*
+
+| Link | Description |
+| :--------- | :------- |
+| [UBI plugin](https://github.com/opensearch-project/user-behavior-insights) | How to install and use the UBI plugin. |
+| [UBI client data structures]({{site.url}}{{site.baseurl}}/search-plugins/ubi/data-structures/)  | Sample JavaScript structures for populating the event store. |
+| [Example UBI query DSL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/dsl-queries/)  | How to write queries for UBI data in OpenSearch query DSL. |
+| [Example UBI SQL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/sql-queries/)  | How to write analytic queries for UBI data in SQL. |
+| [UBI dashboard tutorial]({{site.url}}{{site.baseurl}}/search-plugins/ubi/ubi-dashboard-tutorial/) | How to build a dashboard containing UBI data. |
+| [Chorus Opensearch Edition](https://github.com/o19s/chorus-opensearch-edition/?tab=readme-ov-file#structured-learning-using-chorus-opensearch-edition) katas | A series of structured tutorials that teach you how to use UBI with OpenSearch through a demo e-commerce store. |
+
+<!-- vale on -->
+The documentation categories were adapted using concepts based on [Diátaxis](https://diataxis.fr/).
+{: .tip }