Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support persistent configuration #52374

Open
murphyatwork opened this issue Oct 28, 2024 · 4 comments · Fixed by #52808 · May be fixed by #54941
Open

Support persistent configuration #52374

murphyatwork opened this issue Oct 28, 2024 · 4 comments · Fixed by #52808 · May be fixed by #54941
Assignees

Comments

@murphyatwork
Copy link
Contributor

Feature request

Is your feature request related to a problem? Please describe.

Current we can change FE configuration through admin set frontend config set('x'='y'), but it's not persistent. After restarted, the FE would read from the original file again.

Describe the solution you'd like

Support persisting the configuration changes into another file to survive the restart.

Describe alternatives you've considered

Additional context

@crossoverJie
Copy link
Contributor

I'm interested in this, please assign it to me, thanks.

@crossoverJie
Copy link
Contributor

After communicating with @kevincai , some details were adjusted.

Phase One(finished):

  1. When saving the configuration, determine if the current node is the leader.
    1.1 true: Persist the configuration and then synchronize it to other nodes.
    1.2 false: Forward the request to the leader node to execute the process in step one.

Phase Two(todo):
To prevent the issue of data inconsistency caused by the follower nodes' inability to persist, the follower nodes proactively synchronize data with the leader node each time they startup.

Flow chart:
image

@kevincai
Copy link
Contributor

the pull request is reverted in #54076.

Reopen this issue for further discussion.

cc @gengjun-git

@gengjun-git
Copy link
Contributor

Using new files to store changed configurations can be confusing if users don't know about this feature, so we should introduce a more compatible solution. And some external management components of StarRocks also need to be compatible with this, which is not very friendly to them.

@crossoverJie Thank you for your contribution, we will work together to find a better solution.

@crossoverJie crossoverJie linked a pull request Jan 10, 2025 that will close this issue
24 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants