[BUG] Teach bulk action to capture no-op update count in IndexingStats.

[r1walz]

Describe the bug

IndexingStats captures statistics related to indexing operations being performed on each shard by the data node. Sample node stats output:

$ http ':9200/_nodes/stats/indices/indexing?filter_path=nodes.*.indices'

HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 174
content-type: application/json; charset=UTF-8

{
    "nodes": {
        "VedlUzbPSY2P3WqbiWJslg": {
            "indices": {
                "indexing": {
                    "delete_current": 0,
                    "delete_time_in_millis": 0,
                    "delete_total": 0,
                    "index_current": 0,
                    "index_failed": 0,
                    "index_time_in_millis": 52,
                    "index_total": 3,
                    "is_throttled": false,
                    "noop_update_total": 0,
                    "throttle_time_in_millis": 0
                }
            }
        }
    }
}

Requests going through TransportUpdateAction mutate the noop_update_total which is the expected behavior but requests coming from TransportBulkAction do not. Teach bulk action to capture no-op update count in IndexingStats.

To Reproduce

Steps to reproduce the behavior:

Checkout e4a1841 from opensearch:main.

# run server
$ ./gradlew run

...

# Create Index
$ http PUT :9200/index

HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 68
content-type: application/json; charset=UTF-8

{
    "acknowledged": true,
    "index": "index",
    "shards_acknowledged": true
}

# Index a Doc
$ http POST :9200/index/_doc/1 name=rohit

HTTP/1.1 201 Created
Location: /index/_doc/1
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 130
content-type: application/json; charset=UTF-8

{
    "_id": "1",
    "_index": "index",
    "_primary_term": 1,
    "_seq_no": 0,
    "_shards": {
        "failed": 0,
        "successful": 1,
        "total": 2
    },
    "_version": 1,
    "result": "created"
}

# Request Update
$ http POST :9200/index/_update/1 doc:='{"name":"rohit"}'

HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 128
content-type: application/json; charset=UTF-8

{
    "_id": "1",
    "_index": "index",
    "_primary_term": 1,
    "_seq_no": 0,
    "_shards": {
        "failed": 0,
        "successful": 0,
        "total": 0
    },
    "_version": 1,
    "result": "noop"
}

# Check IndexingStats
$ http ':9200/_nodes/stats/indices/indexing?filter_path=nodes.*.indices'

HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 176
content-type: application/json; charset=UTF-8

{
    "nodes": {
        "VedlUzbPSY2P3WqbiWJslg": {
            "indices": {
                "indexing": {
                    "delete_current": 0,
                    "delete_time_in_millis": 0,
                    "delete_total": 0,
                    "index_current": 0,
                    "index_failed": 0,
                    "index_time_in_millis": 92,
                    "index_total": 4,
                    "is_throttled": false,
                    "noop_update_total": 1,
                    "throttle_time_in_millis": 0
                }
            }
        }
    }
}

# Create equivalent bulk payload
$ cat noop_update_bulk.json
{"update":{"_id":"1","_index":"index"}}
{"doc":{"name":"rohit"}}

# Do bulk call
$ http POST :9200/_bulk @noop_update_bulk.json

HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 167
content-type: application/json; charset=UTF-8

{
    "errors": false,
    "items": [
        {
            "update": {
                "_id": "1",
                "_index": "index",
                "_primary_term": 1,
                "_seq_no": 0,
                "_shards": {
                    "failed": 0,
                    "successful": 1,
                    "total": 2
                },
                "_version": 1,
                "result": "noop",
                "status": 200
            }
        }
    ],
    "took": 3
}

# Check IndexingStats; it doesn't update `noop_update_total'
$ http ':9200/_nodes/stats/indices/indexing?filter_path=nodes.*.indices'

HTTP/1.1 200 OK
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-encoding: gzip
content-length: 176
content-type: application/json; charset=UTF-8

{
    "nodes": {
        "VedlUzbPSY2P3WqbiWJslg": {
            "indices": {
                "indexing": {
                    "delete_current": 0,
                    "delete_time_in_millis": 0,
                    "delete_total": 0,
                    "index_current": 0,
                    "index_failed": 0,
                    "index_time_in_millis": 92,
                    "index_total": 4,
                    "is_throttled": false,
                    "noop_update_total": 1,
                    "throttle_time_in_millis": 0
                }
            }
        }
    }
}

Expected behavior

NoOp updates in bulk call should update noop_update_total in IndexingStats.

Plugins

Standard.

Screenshots

See To Reproduce.

Host/Environment (please complete the following information):

• EC2 m5.2xlarge latest default ubuntu linux AMI

Additional context

N/A

[gaobinlong]

If no one is working on this, I want to have a try.

[r1walz]

Hi, @gaobinlong. The issue isn't assigned to anyone. You can start by adding a test to verify the bug 🐛. And follow up to fix it. Best!

[gaobinlong]

@r1walz, thanks, I've created a PR for this, could you help to review?

[r1walz]

It's fixed in 2.x with

• Fix noop_update_total metric in indexing stats cannot be updated by bulk API #11485
• [Backport 2.x] Fix noop_update_total metric in indexing stats cannot be updated by bulk API #11865
• Update supported version for fixing the bug of noop_update_total metric cannot be updated by bulk API #11917
• [Backport 2.x] Update supported version for fixing the bug of noop_update_total metric cannot be updated by bulk API #11925