[Decouple] [BUG] Dev tool console is losing big number precision due to client

[zhongnansu]

Describe the bug

Dev tool console is losing precision when displaying big number.

To Reproduce
Steps to reproduce the behavior:

1. Go to dev tool console in OSD
2. Index one long type data into OS, and send GET request to retrieve it.

   PUT index_0001
   {
   "mappings": {
     "properties": {
       "test_long": {
         "type": "long"
       }
     }
   }
   }
   
   PUT index_0001/_doc/0
   {
   "test_long": 1688275058533285124
   }
   
   GET index_0001/_search
   

3. You'll get below results

   {
     "took": 58,
     "timed_out": false,
     "_shards": {
       "total": 1,
       "successful": 1,
       "skipped": 0,
       "failed": 0
     },
     "hits": {
       "total": {
         "value": 1,
         "relation": "eq"
       },
       "max_score": 1,
       "hits": [
         {
           "_index": "index_0001",
           "_id": "0",
           "_score": 1,
           "_source": {
             "test_long": 1688275058533285000
           }
         }
       ]
     }
   }
   

4. We can see it loses precision comparing original data 1688275058533285124 and the displayed data in OSD 1688275058533285000
5. Also I validated the actual data in OS is stored with correct precision.

Expected behavior
OSD should display the same number as stored in OS, without losing precision

OpenSearch Version
2.5 and later

Dashboards Version
2.5 and later

Additional context

JavaScript does not support 64 bit integer values. Using a number in JavaScript results in the Number type which itself is a 64bit IEEE floating point number, thus leaving us a 52 bit mantisse to store integer values without rounding errors. Everything above what fits into 52 bit will have rounding errors.

Thus fields in OpenSearch that can contain numbers outside that safe JavaScript range(E.g. long) is prone to that rounding error in Dev tool console

[zhongnansu]

The bug is related to some changes we made in dev tool console starting from 2.5

• Prior to 2.5 dev tool console was using Ajax to read and display the response as string, while in Update console to use core.http instead of jQuery.ajax #3080, we replaced ajax with core.http, where it automatically convert the response body into js object causing the loss of precision
• In 2.7, we replaced node http client with opensearch.js client to talk to OS, it was done in this PR Refactor dev tool console to use opensearch-js client to send request #3544. opensearch js client returns a response wrapped as js object, again causing the loss of precision.

  OpenSearch-Dashboards/src/plugins/console/server/routes/api/console/proxy/create_handler.ts

  Lines 114 to 117 in 1dc1060

  	opensearchResponse = await client.transport.request(
  	{ path: toUrlPath(path), method, body: bufferedBody },
  	{ headers: requestHeaders }
  	);

[pjfitzgibbons]

In final resolution, we might need to determine the impact of this on dashboard graphs/visualizations. It may be a challenge to find the impact of a 10^3 delta within a 10^19 number, especially in a visualization displayed on a 100x200 pixel canvas.

[wbeckler]

@pjfitzgibbons If a visualization is based on a delta between two numbers, it could become super obvious. I'm not sure if any visualizations do have that.

[pjfitzgibbons]

Ahh. Excellent use case. In this situation the loss could/would result in a flatline graph where there should be noise.

On Tue, Jul 18, 2023 at 10:55 AM William Beckler ***@***.***> wrote: @pjfitzgibbons <https://github.com/pjfitzgibbons> If a visualization is based on a delta between two numbers, it could become super obvious. I'm not sure if any visualizations do have that. — Reply to this email directly, view it on GitHub <#4536 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACYNXMBFZJMB7PJ34IVBLXQ3E2ZANCNFSM6AAAAAA2E53EWM> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

-- Thanks and Kindest Regards, Peter Fitzgibbons