[BUG] patterns command returns type of patterns_field inconsistently when calcite enabled

[paulstn]

What is the bug?

When calcite is enabled, the patterns command switches between the two types of patterns_fields inconsistently.

For the same index, the two types of patterns_fields have been:
(Calcite engine + brain)

<token1> - - [<token2>] "GET <token3> HTTP/<token4><token5>" 200 <token6> "-" "Mozilla/<token7><token8> (<token9>; Linux <token10>_<token11>; rv:<token12><token13><token14>) Gecko/<token15> Firefox/<token16><token17><token18>"

(V2 engine + brain)

<*IP*> - - [<*DATETIME*>] "GET <*> HTTP/<*><*>" 404 <*> "-" "Mozilla/<*><*> (<*>; Linux <*>_<*>; rv:<*><*><*>) Gecko/<*> Firefox/<*><*><*>"

This issue is about how when an aggregation isn't performed on a PPL command with patterns in the calcite engine, it switches from the format seen in calcite to the one seen in V2.

For example, when running the PPL query:

source = opensearch_dashboards_sample_data_logs | patterns `message` method=brain

What's returned has the pattern field in the "V2" format.

When running that PPL query with any kind of aggregation (also noticed with a fields clause):

source = opensearch_dashboards_sample_data_logs | patterns `message` method=brain | stats count() by patterns_field

it returns with the token format.

This behavior is problematic because trying to select documents with a where clause isn't as easy when the patterns_field changes between queries.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

What is the expected behavior?
A clear and concise description of what you expected to happen.

What is your host/environment?

• OS: [e.g. iOS]
• Version [e.g. 22]
• Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

[songkant-aws]

Will make it backward compatible

[penghuo]

@songkant-aws could we close this PR with #4402

[songkant-aws]

@penghuo Yes, we can close it with #4402. BTW, I don't have permission to close such issues.