PPL geoip function

[salyh]

Description

PPL geoip function

Issues Resolved

#672

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[salyh]

@YANG-DB @penghuo @lukasz-soszynski-eliatra @kt-eliatra @dr-lilienthal PPL geoip function syntax proposal

Please review and comment if needed

[YANG-DB]

@salyh thanks
on my end looking good - only question is the datasource definition - how does the user creates a new datasource, can/should the datasource` be one of the following:

• external API
• table
• file
  Lets add these capabilities ...

[YANG-DB]

@LantaoJin @penghuo @dai-chen please add u'r comments here

[salyh]

@salyh thanks on my end looking good - only question is the datasource definition - how does the user creates a new datasource, can/should the datasource` be one of the following:

* external API

* table

* file
  Lets add these capabilities ...

@YANG-DB @LantaoJin @penghuo @dai-chen

datasource refers to the datasources provided by the ip2geo processor https://opensearch.org/docs/latest/ingest-pipelines/processors/ip2geo/

My idea was to leveage the already present capabilities of the ip2geo processor and extend it as necessary. Because it would avoid code duplication. When I read the docs of the processor correct, GeoLite2 is supported by now. I propose to add the above requested functionality to the processor. Then we try to use the code from the processor (as a library) to solve this issue. Otherwise we would reinvent the wheel.

And there is also https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/processors/geoip/ in data prepper which could be facilitated

[YANG-DB]

@salyh thanks on my end looking good - only question is the datasource definition - how does the user creates a new datasource, can/should the datasource` be one of the following:

* external API

* table

* file
  Lets add these capabilities ...

@YANG-DB @LantaoJin @penghuo @dai-chen

datasource refers to the datasources provided by the ip2geo processor https://opensearch.org/docs/latest/ingest-pipelines/processors/ip2geo/

My idea was to leveage the already present capabilities of the ip2geo processor and extend it as necessary. Because it would avoid code duplication. When I read the docs of the processor correct, GeoLite2 is supported by now. I propose to add the above requested functionality to the processor. Then we try to use the code from the processor (as a library) to solve this issue. Otherwise we would reinvent the wheel.

And there is also https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/processors/geoip/ in data prepper which could be facilitated

@salyh this sounds like a good idea !
@LantaoJin @penghuo - any comments ?

[LantaoJin]

As a user doc, could we remove the ANTLR related information, what here needs is just a user readable syntax definition.

[LantaoJin]

Oh, the file is located ppl-lang/planning folder. I think we need an user doc instead of design doc. We can copy this design information to the PR description or issue description. And you can easily update your design without any code changes.

[salyh]

@YANG-DB I did unterstand that the ppl-lang/planning folder is for design docs. When the design is approved it will be converted to a userdoc and moved out of the planning sub-folder. Am I wrong with that assumption?

[LantaoJin]

I thought it was for roadmap. I prefer to using GitHub PR description and comments. It's fine to add design docs in code if you prefer to. But these docs are not for users, we need to write a new one later.

[YANG-DB]

@salyh yes - once design doc is approved we will move it with necessary updates to the regular commands folder

[LantaoJin]

what does datasource mean? Could you use other meaningful usage rather that abc? abc is not straightforward user case.

[salyh]

datasource refers to the datasources provided by the ip2geo processor, see #781 (comment)

[LantaoJin]

datasource = functionArg

Any case for supporting functionArg instead of stringLiteral. Again, what does datasource mean?

[salyh]

no particular case for functionArg. We can change it to stringLiteral for sure

datasource refers to the datasources provided by the ip2geo processor, see #781 (comment)