HIVE-24838. Reduce FS creation in Warehouse::getDnsPath for object stores #2085
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on its javadoc comment getDnsPath is supposed to change the authority part (ip) of a path to a domain name. To do so it creates a new FileSystem object and gets the default path from it, and uses the authority from the the default path to replace the authority of the input.
There are multiple problems with this. Instantiating a new FileSystem is expensive. Replacing the authority on a blobstore path (s3a) is not needed since s3 uses bucket name instead of a hostname in the path.
Even in HDFS case the original function doesn't do what the original intention might was. The new file system is initialized with the input path so the default FS path is always same as the input path (unless a relative path is used as an input). The only useful thing this function does is transforming a relative path to an absolute path.
For example:
But this can be achieved by doing a simple config lookup (FileSystem.getDefaultUri(conf)) there is no need to create a new FS object every time.