[BUG] Wildcard field type causing errors in query with "fields" wildcard

[Calebjh]

Describe the bug

When trying to incorporate the new wildcard field type into some existing index/query structures, I am encountering an error when using a wildcard in a fields query block.

"reason": "failed to create query: Index 0 out of bounds for length 0",

It seems to be related to the presence of multiple top level properties (my existing data has info split between doc.* fields and metadata.* fields, and when I create a test index with only doc.* fields it seems to work).

Related component

Search

To Reproduce

1. Create an index with more than one subproperty (possibly the incorrect terminology)

PUT test_index
{
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "doc": {
        "properties": {
          "summary": {
            "type": "wildcard"
          }
        }
      },
      "metadata": {
        "properties": {
          "status": {
            "type": "long"
          }
        }
      }
    }
  }
}

2. Make a query against the index with wildcard in the fields block

GET test_index/_search
{
  "from": 0,
  "query": {
    "query_string": {
      "analyze_wildcard": true,
      "fields": [
        "doc.*"
      ],
      "query": "*foo*"
    }
  }
}

3. Receive error response

{
  "error": {
    "root_cause": [
      {
        "type": "query_shard_exception",
        "reason": "failed to create query: Index 0 out of bounds for length 0",
        "index": "test_index",
        "index_uuid": "jkSC-uIXStyOy2db_3ZLtA"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "test_index",
        "node": "GTGj9ZMiT-WUbQTyOtkTdw",
        "reason": {
          "type": "query_shard_exception",
          "reason": "failed to create query: Index 0 out of bounds for length 0",
          "index": "test_index",
          "index_uuid": "jkSC-uIXStyOy2db_3ZLtA",
          "caused_by": {
            "type": "string_index_out_of_bounds_exception",
            "reason": "Index 0 out of bounds for length 0"
          }
        }
      }
    ]
  },
  "status": 400
}

Expected behavior

If the type is not a wildcard, such as a keyword, the query completes with no errors. I would expect either an informative error if this query structure is incompatible with wildcards for some reason, or (ideally) for the query to just work like it does with non-wildcard fields.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):
Querying OpenSearch 2.15 and 2.16 running as a docker container from an opensearch-dashboards 2.15 interface also running via Docker.

Additional context
Add any other context about the problem here.

[hasnain2808]

If we accept this bug in the weekly search triage meeting, I would like to work on this

[mch2]

Thanks @Calebjh for reporting and for volunteering @hasnain2808 have assigned to you. @msfroh FYI.

[msfroh]

@msfroh FYI.

Yup! Sounds like a bug. I'm curious about whether the bug is in the wildcard on the fields block or if it's because of the query_string query. (I don't remember if I added test cases for query_string queries on wildcard fields.)

[hasnain2808]

@Calebjh

It seems to be related to the presence of multiple top level properties (my existing data has info split between doc.* fields and metadata.* fields, and when I create a test index with only doc.* fields it seems to work).

It fails even for a test index with only doc.* fields

[Calebjh]

@hasnain2808 Interesting. I thought that I tested both ways and found it worked for one and not the other, but perhaps I was trying too many different variations and got my test cases mixed up (I was just making the different adjustments by hand to try and see what was causing the failure).
Should I make any adjustment to the original description to correct that, or is the extra clarification down here adequate?

[hasnain2808]

@hasnain2808 Interesting. I thought that I tested both ways and found it worked for one and not the other, but perhaps I was trying too many different variations and got my test cases mixed up (I was just making the different adjustments by hand to try and see what was causing the failure). Should I make any adjustment to the original description to correct that, or is the extra clarification down here adequate?

I retested
it fails for both

we can add a new section with a date and add this as a new update/ new finding

[hasnain2808]

triage:

issue replicates on 2.15, 2.16 but does not replicate on 3.x or 2.x

wild card fields were added to 2.15

Looks like this PR solved the issue and we can close this

Specifically this line

https://github.com/opensearch-project/OpenSearch/pull/15737/files#diff-4fef7944375affd51ec9418a4efbb5c48fb8820dffde9c3ad8f63fe5efbd312dR476

cc: @msfroh @mch2 @Calebjh

[msfroh]

Oh, nice! Thanks @hasnain2808 for working to reproduce.

It looks like I merged that PR after the 2.17 freeze, but it should be fixed in 2.18.

Special thanks to @HUSTERGS, who is fixing all the wildcard field bugs these days.