Significant terms ignores background filter

[benwtrent]

Elasticsearch version (bin/elasticsearch --version):
7.15 (and probably earlier)
Description of the problem including expected versus actual behavior:
Significant terms is ignoring the background filter when one index has NO documents containing the term value.

Steps to reproduce:
Here are a long set of steps to reproduce

PUT repeat-test-1
{
  "mappings": {
    "properties": {
      "field1": {
        "type": "keyword"
      },
      "field2": {
        "type": "keyword"
      },
      "field3": {
        "type": "keyword"
      }
    }
  }
}

PUT repeat-test-2
{
  "mappings": {
    "properties": {
      "field1": {
        "type": "keyword"
      },
      "field2": {
        "type": "keyword"
      },
      "field3": {
        "type": "keyword"
      }
    }
  }
}

POST repeat-test-1/_bulk?refresh=true
{ "index" : {} }
{ "field1" : "failed", "field2": "frontend-node", "field3": "foo" }
{ "index" : {} }
{ "field1" : "failed", "field2": "frontend-node", "field3": "bar" }
{ "index" : {} }
{ "field1" : "failed", "field2": "frontend-node", "field3": "bar" }
{ "index" : {} }
{ "field1" : "success", "field2": "frontend-node", "field3": "bar" }
{ "index" : {} }
{ "field1" : "success", "field2": "frontend-node", "field3": "bar" }
{ "index" : {} }

POST repeat-test-2/_bulk?refresh=true
{ "index" : {} }
{ "field1" : "success", "field2": "java-node" }
{ "index" : {} }
{ "field1" : "success", "field2": "java-node" }
{ "index" : {} }
{ "field1" : "success", "field2": "java-node" }
{ "index" : {} }


GET repeat-test*/_search?track_total_hits=true&size=0
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "field2": "frontend-node"
          }
        }
      ]
    }
  },
  "aggs": {
    "sig_term": {
      "significant_terms": {
        "field": "field3",
        "background_filter": {
          "bool": {
            "filter": [
              {
                "term": {
                  "field2": "frontend-node"
                }
              }
            ]
          }
        }
      }
    }
  }
}

With the background filter, one expects the background count to be only 5 docs

But, it responds with:

"aggregations" : {
    "sig_term" : {
      "doc_count" : 5,
      "bg_count" : 8,
      "buckets" : [
        {
          "key" : "bar",
          "doc_count" : 4,
          "score" : 0.4800000000000001,
          "bg_count" : 4
        }
      ]
    }
  }

"bg_count" : 8,

Which means the 3 docs that don't match the filter were matched.

Seems like a bad optimization as those docs also don't contain the term field field3

[elasticmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)