【Discuss】Whether UltraWarm will be open source?

[jgq2008303393]

Hi,
We’re continuing to look at what the OpenSearch community is doing. Hope to know whether UltraWarm will be open source?

Thanks,
ginger

[CEHENKLE]

Hi Ginger -- thanks for your question. I've flagged it for the UltraWarm team, and I'll make sure you get an update.

Thanks again,
/C

[nirajjetly]

Ginger,

Thanks for your interest in UltraWarm. I am the Engineering manager working on UltraWarm. We are brainstorming options of helping customers drive down their storage costs. Please stay tuned as we share additional details.

[jkowall]

Hello @nirajjetly instead of talking about customers, how about users of OpenSearch? Remember we are in an open source project and this is about the community of users and not just your customers at AWS. In my opinion this is very important for the OpenSearch project to compete and win users. ElasticSearch offers a new frozen index capability after they went non-open source.

We need to have something to draw users into OpenSearch which helps manage warm data. I would dedicate engineering resources on my team to make this happen, but AWS already has done this and it makes sense to move this to OpenSearch.

Comments from AWS?

[jcgraybill]

Hey @jkowall thanks for keeping us honest here. I'll clarify that yes, we're brainstorming options for OpenSearch users to minimize storage costs with some sort of cold/frozen index.

It's common for Amazon employees to use a broad definition of the word "customer" - along the lines of "anybody I'm providing a service to is my customer." This can come across as weird in the context of an open source project, so we're trying to shift away from that terminology in the project. We'll probably slip up sometimes. And of course, the work we're doing in parallel to bring non-Amazon maintainers onboard (update coming on that very soon, I hope!) will help with this.

[jkowall]

@jcgraybill no problem at all, I know it takes time to change the thinking. I have the same challenge between our "products" and "projects". It will be outstanding to see this make it to OpenSearch, but I know it requires a lot of work on your side to refactor the code. I hope it happens, and I hope it happens sooner than later. Once users upgrade their ES versions moving to OpenSearch will be a challenge, and Elastic likely will make this more difficult intentionally.

[jgq2008303393]

Thank you for reply. We hope it happens sooner than later, too.

[frotsch]

https://discuss.opendistrocommunity.dev/t/any-plans-to-backport-ultrawarm-for-amazon-elasticsearch-service/2075/4

[jkowall]

The main reason I updated that thread is that Elastic is linking there to tell users to adopt their non-open source licensed products versus using OpenSearch. I hope we get UltraWarm public ASAP even if it needs a bunch of work to make it usable.

[andrross]

Closing this issue as this work is being tracked in the storage roadmap (#3739). Please feel free to comment there or reopen if there are additional questions.