Feature: Update results to display non-source fields in search comparison tool

[nung22]

Description

Adds the ability to view non-source fields in the results grid when using the search comparison tool. The result object sourceFields is first populated with fields from _source (if available), then is overlayed with field values from the fields property. If a field is available in both _source and fields, the value in fields overwrites the value in _source.

Ran this script in the Search Comparison tool to populate sample data result documents with a fields property:

{
  "query": {
    "match_all": {}
  },
  "_source": {},
  "script_fields": {
    "FlightNumNew": {
      "script": {
        "lang": "painless",
        "source": "return doc['_id'][0] + 'AAA'"
      }
    },
    "FlightNum": {
      "script": {
        "lang": "painless",
        "source": "return doc['_id'][0] + 'BBB'"
      }
    }
  }
}

Issues Resolved

Closes #201

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Screenshots

Note: Bins in result were grid were expanded specifically for the screenshots to observe changes. No modifications were made to current bin height.

Before: fields property is not displayed

After: unique field FlightNumNew is visible at the bottom of the result grid, while the original value of the _source field FlightNum has been overwritten with the value from fields

[codecov]

Codecov Report

Merging #340 (653ba47) into main (b9a71a4) will increase coverage by 0.06%.
Report is 2 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #340      +/-   ##
==========================================
+ Coverage   87.08%   87.14%   +0.06%     
==========================================
  Files          16       16              
  Lines         209      210       +1     
  Branches       43       43              
==========================================
+ Hits          182      183       +1     
  Misses         26       26              
  Partials        1        1              

Flag	Coverage Δ
dashboards-search-relevance	87.14% <100.00%> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...re/search_result/result_components/result_grid.tsx	91.11% <100.00%> (+0.20%)	⬆️
public/types/index.ts	100.00% <ø> (ø)

... and 2 files with indirect coverage changes

[sejli]

Overall, nice work on the implementation. I left some comments that nitpick a little bit on specifics, may need to engage @kgcreative on some of the UX. Should we indicate a difference between non-source fields and source fields in the comparison tool?

[sejli]

In the event that we have the same field in the source and the fields, which one should take precedence? Or should we find a way to display both and indicate which fields are from mappings/fields and which fields are from the source?

[nung22]

Based on the description in #201, I implemented this so the field from the fields would take precedence. I can change this so that both are displayed though.

[hdhalter]

@nung22 - Is this going into 2.12 and does it need documentation? If so, please create a doc issue. Is the feature tied to this issue (#201)? If so, I'll assign the doc issue to it in the scorecard. Thanks!

[sejli]

Hey @hdhalter, this feature has already been backported to the 2.x branch, so it should be going into the release. I don't think it requires documentation, since the current documentation doesn't specify what kind of results are returned. This PR updates the search results such that it is consistent with what is returned when running a query to the OpenSearch cluster directly.