HDDS-1596. Create service endpoint to download configuration from SCM #861
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
anuengineer
reviewed
May 29, 2019
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/discovery/DiscoveryUtil.java
Show resolved
Hide resolved
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeService.java
Outdated
Show resolved
Hide resolved
...op-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/discovery/ConfigurationXmlEntry.java
Show resolved
Hide resolved
|
bq. Also add some documentation that scm names is required, and update the config file tag if possible. It's not strictly required as this download is optional as of now. You can use your own ozone-site.xml and in that case you don't need to download any config. (Except with a never-ending loop....) It can be better to use a But I am open to any suggestion. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
Do you have any more comments @anuengineer ? |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
@anuengineer Any more thoughts? ( Today is the 3 months anniversary of this patch. Would be great to either commit or reject and close it.) |
|
Thanks for the reminder; I will get this committed today. |
|
Done, committed in the trunk. |
shanthoosh
pushed a commit
to shanthoosh/hadoop
that referenced
this pull request
Oct 15, 2019
So far we've not seen a lot of use in maintaining separate implementation for ReadableTable and ReadWriteTable, which adds quite a bit complexity. Hence consolidating them. Author: Wei Song <wsong@linkedin.com> Reviewers: Xinyu Liu <xiliu@linkedin.com> Closes apache#861 from weisong44/SAMZA-2043
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As written in the design doc (see the parent issue) it was proposed to download the configuration from the scm by the other services.
I propose to create a separated endpoint to provide the ozone configuration. /conf can't be used as it contains all the configuration and we need only the modified configuration.
The easiest way to implement this feature is:
With this approach we keep the support of the existing manual configuration (ozone-site.xml has higher priority) but we can download the configuration to a separated file during the startup, which will be loaded.
There is no magic: the configuration file is saved and it's easy to debug what's going on as the OzoneConfiguration is loaded from the $HADOOP_CONF_DIR as before.
Possible follow-up steps:
See: https://issues.apache.org/jira/browse/HDDS-1596