Validate that soft delete settings is enabled at leader

[soosinha]

Description

Replication should occur only when the setting index.soft_deletes.enabled is enabled for the leader index. It is enabled by default and can be set only at index creation. Documentation: https://www.elastic.co/guide/en/elasticsearch/reference/7.x/index-modules.html
Adding validation to ensure that this setting is enabled for leader index before starting replication.

Tested by setting this to true while creating index. Got below exception:

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Cannot Replicate an index where the setting index.soft_deletes.enabled is disabled"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Cannot Replicate an index where the setting index.soft_deletes.enabled is disabled"
    },
    "status": 400
}

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[saikaranam-amazon]

LGTM

[krishna-ggk]

It seems like other static index settings this can be changed by closing and reopening index.
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html

Can we also validate this in resume?

[krishna-ggk]

And add a corresponding integ test?

[soosinha]

This setting cannot be updated even after closing index. I have verified the same. Getting this on trying to add the setting: final my-index setting [index.soft_deletes.enabled], not updateable

[krishna-ggk]

Thanks for validating.