[MD] Refactor data source server side error handling

[zhongnansu]

Signed-off-by: Su szhongna@amazon.com

Description

Refactor data source server side error handling

• Rename DataSourceConfigError to DataSourceError
• Add helper createDataSourceError() to create data source error from:
  • Opensearch client error
  • legacy client error
  • saved object client error
  • other runtime errors
• Replace usage of new DataSourceError(e) with createDataSourceError(e)
• can't directly import createDataSourceError() to data plugin, have to import from data source plugin setup interface, otherwise, some other UTs in other modules fail
• refactor to move data source error response logic from router.ts to resolve_index.ts. I check all our codebase, only the route handler defined resolve_index is missing error handling. All other internal/xxx/xxx API has err handling. E.g internal/search/, /internal/_msearch, /internal/index-pattern-manage/preview-script

  catch (err) {
          return res.customError({
            statusCode: err.statusCode,
            body: {
              message: err.message,
              attributes: {
                error: err.body.error,
              },
            },
          });
  

Error statusCode casting logic

• If it's response error, we maintain the error statusCode, except 401
• for any 401 error, we cast to 400, because of [MD] multiple datasource doesn't throw 401 error as expected when security plugin enabled  #2591
• if it's client non-response error(without statuscode), cast to 400, while keeping the original error message, such as timeout error, connection error, no living connection error, etc. Because by design, datasource could be created without validation of connectivity. Besides, ds could be updated, we won't be aware of the data source side status, also couldn't keep track of ds domain status~~

Issues Resolved

#2591

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[ashwin-pc]

We cast all search error against data source to 400 bad request, while maintaining the original error message from OS.

This worries me a lil because it is the root of a similar issue we face in other places in Dashboards where we incorrectly assign the wrong error code for a response. 4xx errors are auth errors.

1. Is there really no way to assign the right error response code for an error?
2. If not, why did you choose to use a 4xx error instead of a generic 5xx error?

[zhongnansu]

We cast all search error against data source to 400 bad request, while maintaining the original error message from OS.

This worries me a lil because it is the root of a similar issue we face in other places in Dashboards where we incorrectly assign the wrong error code for a response. 4xx errors are auth errors.

1. Is there really no way to assign the right error response code for an error?
2. If not, why did you choose to use a 4xx error instead of a generic 5xx error?

@ashwin-pc , thanks for the comments, really helpful. I re-worked on this PR. basically doing a refactor of existing error handling logic in data source feature. Now it will keep original status code. But for 401, we choose to cast to 400, due to the limitation of security dashboards plugin. For 5xx error, we cast to 400. Because it doesn't make sense to use 5xx, e.g user's data source domain is down, while trying to connect to it, we get 5xx. But there's nothing wrong with local OSD. Security review also mentioned it's not good to return 5xx for such cases

[ashwin-pc]

@zhongnansu Thanks for that explanation! That logic makes sense to me.

[zengyan-amazon]

can we use instanceofinstead of error.constructor.name ?

[zhongnansu]

importing DataSourceError from plugin module to here(core module) doesn't sound like a good practice. But anyway, I did some refactor to move data source error response logic from router.ts to resolve_index.ts(/internal/index-pattern-manage/resolve-index).

I checked the codebase, only the route handler defined resolve_index is missing error handling. All other internal/xxx/xxx API has err handling. E.g internal/search/, /internal/_msearch, /internal/index-pattern-manage/preview-script

```
catch (err) {
        return res.customError({
          statusCode: err.statusCode,
          body: {
            message: err.message,
            attributes: {
              error: err.body.error,
            },
          },
        });
```   

@zengyan-amazon could you review again