[BUG] NullPointerException in _geo_distance sort

[janschwarz]

Describe the bug

all shards failed, root_cause: [{null_pointer_exception Cannot invoke "String.endsWith(String)" because "field" is null 

This occurs on Opensearch 3.0 with a geo_point field while sorting by distance.

"location_point": {
    "type": "geo_point"
}

Changing unit from "km" to "m" fixed it on some requests but failed again when changing sort order from "asc" to "desc"

Related component

Search

To Reproduce

1. Have a search index with geo_point field
2. Sort by _geo_distance

{
  "query": { "match_all": {} },
  "sort": [ { "_geo_distance": { "location_point": [9.227400, 49.189800], "order": "desc", "unit": "km", "distance_type": "arc", "mode": "min", "ignore_unmapped": true } } ],
  "size": 10
}

3. Get an Exception all shards failed, root_cause: [{null_pointer_exception Cannot invoke "String.endsWith(String)" because "field" is null 

Expected behavior

Sorting works like in Opensearch 2.18

Additional Details

Plugins
Basic docker image:
Image with Bug: opensearchproject/opensearch:latest
Working image: opensearchproject/opensearch:2.18.0

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: [e.g. iOS]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[sandeshkr419]

@ajleong623 Can you please take an initial look on why it is failing?

[ajleong623]

@janschwarz

Thank you so much for bringing up the issue, but I have not been able to reproduce it.

I have been working on reproducing the error for a bit. One thing I discovered is to create the mapping before putting in the documents, otherwise the sort would be infinity. Otherwise, what I have done was create the mapping for the index:

Put in documents similar to what is shown below

Run the search query

For some reason, the error was not reproduced. Are there any other details that may have been missed?

[janschwarz]

I tried to reproduce it myself today and still get the same error.
image: opensearchproject/opensearch:latest
I used following mapping for my geopointindex:
{ "settings": { "index": { "number_of_shards": 1, "number_of_replicas": 0, "max_result_window": 40000 }, "analysis": { "filter": { "edge_ngram_filter": { "type": "edge_ngram", "min_gram": 1, "max_gram": 20 }, "synonym": { "type": "synonym", "synonyms": [ ], "updateable": true } }, "analyzer": { "autocomplete": { "type": "custom", "tokenizer": "standard", "filter": [ "lowercase", "edge_ngram_filter" ] }, "folding_analyzer": { "tokenizer": "standard", "filter": [ "lowercase", "asciifolding" ] }, "synonym_analyzer": { "type": "custom", "tokenizer": "standard", "filter": [ "lowercase", "synonym" ] } } } }, "mappings": { "properties": { "categories": { "type": "nested", "properties": { "databaseId": { "type": "keyword" }, "division": { "type": "keyword" }, "language": { "type": "keyword" }, "client": { "type": "keyword" }, "categoryId": { "type": "keyword" }, "entry": { "type": "nested", "properties": { "id": { "type": "keyword" }, "value": { "type": "text" } } } } }, "client": { "type": "keyword" }, "closingDate": { "type": "date" }, "company": { "type": "keyword" }, "division": { "type": "keyword" }, "descHeader": { "type": "text" }, "descResponsibilities": { "type": "text" }, "entryDate": { "type": "date" }, "highlight": { "type": "boolean" }, "highlightFrom": { "type": "date" }, "highlightUntil": { "type": "date" }, "jobPortals": { "type": "keyword" }, "jobTemplateId": { "type": "long" }, "language": { "type": "keyword" }, "location_point": { "type": "geo_point" }, "location_city": { "type": "keyword" }, "location_zipcode": { "type": "keyword" }, "location_address": { "type": "keyword" }, "location_name": { "type": "keyword" }, "modifiedTime": { "type": "date" }, "modified_time": { "type": "date" }, "onlineFrom": { "type": "date" }, "onlineUntil": { "type": "date" }, "postingId": { "type": "keyword" }, "recruiter": { "type": "keyword" }, "recruitingUrl": { "type": "keyword" }, "requisition_id": { "type": "keyword" }, "salaryNote": { "type": "text" }, "salaryValue": { "type": "text" }, "templateId": { "type": "keyword" }, "title": { "type": "text", "fields": { "keyword": { "type": "keyword" }, "suggest": { "type": "text", "fielddata": true } } }, "title_completion": { "type": "completion", "analyzer": "folding_analyzer", "search_analyzer": "folding_analyzer" }, "synonym": { "type": "text" }, "suggestions": { "type": "completion" }, "workingHours": { "type": "keyword" }, "deletedAt": { "type": "date" }, "distance": { "type": "double" }, "entry_level_id": { "type": "keyword" }, "entry_level_value": { "type": "keyword" }, "employment_area_id": { "type": "keyword" }, "employment_area_value": { "type": "keyword" }, "contract_duration_id": { "type": "keyword" }, "contract_duration_value": { "type": "keyword" }, "contract_type_id": { "type": "keyword" }, "contract_type_value": { "type": "keyword" }, "autocomplete": { "type": "text", "fielddata": true, "analyzer": "autocomplete", "search_analyzer": "standard" } } } }
and used following search:
{ "query": { "match_all": {} }, "sort": [ { "_geo_distance": { "location_point": [ 9.2274, 49.1898 ], "order": "asc", "unit": "km", "distance_type": "arc", "mode": "min", "ignore_unmapped": true } } ], "size": 20 }
my index contained 12 documents.

when i changed unit from "km" to "m" i get search results

when changing sort order to "desc" i m not getting results for "km" or "m"

I also made sure there were no documents with missing coordinates or non numeric / non float values. The most confusing part for me is, that it is working with unit "m" and sort order "asc". It also works finde with `image: opensearchproject/opensearch:2.18.0`

[ajleong623]

Got it. That’s an interesting error. I will try to reproduce it again with the new schema. Thank you!

[ajleong623]

@janschwarz
An example of 1 document I put in is

{
"autocomplete": "Senior Software Engineer San Francisco",
"categories": [
{
"categoryId": "cat-123",
"client": "acme-corp",
"databaseId": "cat-db-001",
"division": "engineering",
"entry": [
{
"id": "entry-1",
"value": "Software"
},
{
"id": "entry-2",
"value": "Engineering"
}
],
"language": "en"
}
],
"client": "acme-corp",
"closingDate": "2025-07-01T23:59:59Z",
"company": "Acme Corporation",
"contract_duration_id": "permanent",
"contract_duration_value": "Permanent",
"contract_type_id": "full-time",
"contract_type_value": "Full-time",
"deletedAt": null,
"descHeader": "Join our engineering team to build scalable systems",
"descResponsibilities": "Design, develop, and deploy scalable backend systems using Java and Spring Boot.",
"distance": 5.2,
"division": "engineering",
"employment_area_id": "software-dev",
"employment_area_value": "Software Development",
"entryDate": "2025-06-01T10:00:00Z",
"entry_level_id": "mid",
"entry_level_value": "Mid-Level",
"highlight": true,
"highlightFrom": "2025-06-01T00:00:00Z",
"highlightUntil": "2025-06-15T00:00:00Z",
"jobPortals": "Indeed",
"jobTemplateId": 101,
"language": "en",
"location_address": "123 Market Street",
"location_city": "San Francisco",
"location_name": "Acme HQ",
"location_point": {
"lat": 24.984,
"lon": -3.0
},
"location_zipcode": "94105",
"modifiedTime": "2025-06-05T08:00:00Z",
"modified_time": "2025-06-05T08:00:00Z",
"onlineFrom": "2025-06-01T00:00:00Z",
"onlineUntil": "2025-07-01T00:00:00Z",
"postingId": "POST123456",
"recruiter": "jane.doe@acme.com",
"recruitingUrl": "https://careers.acme.com/jobs/POST123456",
"requisition_id": "REQ-98765",
"salaryNote": "Competitive salary based on experience",
"salaryValue": "$120,000 - $140,000",
"suggestions": {
"input": [
"software engineer",
"backend developer"
],
"weight": 8
},
"synonym": "Software Engineer, Backend Developer",
"templateId": "template-01",
"title": "Senior Software Engineer",
"title_completion": {
"input": [
"Senior Software Engineer"
],
"weight": 10
},
"workingHours": "40 hours/week"
}

However this worked just fine. If it would be possible, could share a set of documents that fail? Thank you so much for your patience. I am really want to get to the bottom of this, too so that users could have a better experience.

[janschwarz]

I created a new index with just the document you provided and could reproduce the error.
I'm running opensearch by using the sample docker-compose.yml cluster setup provided here https://docs.opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/ on a mac m4.

[ajleong623]

@janschwarz OK, this is very strange. But it is good the document I gave reproduced it.

I ran the sample docker-compose.yml setup and only disabled the security configurations which should not make a difference. To retrace my steps, I first ran a put request for making the index with the mappings. I used the mapping you gave me earlier. Then I made another put request with the document I gave you. Then I ran the search with the parameters you gave me. Were those the same steps you used or was there something I missed. I am not able to reproduce it after retracing your steps exactly as you told me, I will pass it on to someone else and pair with that person to solve the error.

[janschwarz]

@ajleong623 that are exactly the steps I did. Thanks for all the effort.

[ajleong623]

@janschwarz I will try to see if anyone could reproduce it. This is a pretty strange issue.