Remove hard-coded special treatment for $path and $bucket #11221
Conversation
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D64180231 |
✅ Deploy Preview for meta-velox canceled.
|
| @@ -226,7 +226,9 @@ class HiveConnectorTestBase : public OperatorTestBase { | |||
| class HiveConnectorSplitBuilder { | |||
| public: | |||
| HiveConnectorSplitBuilder(std::string filePath) | |||
| : filePath_{std::move(filePath)} {} | |||
| : filePath_{filePath.find("/") == 0 ? "file:" + filePath : filePath} { | |||
There was a problem hiding this comment.
Why we need this prefix to ensure it goes to local fs? IIRC, the default goes to local fs.
There was a problem hiding this comment.
I don't know either, maybe some legacy workaround, let me remove it to see if it still works
|
@xiaoxmeng Yes Prestissimo handles it in split conversion code I mentioned in the PR description. Other usages also have the special columns they need set up properly in their code. |
…cubator#11221) Summary: These special columns are engine-specific and should be handled during engine split generation by setting the corresponding values in split `infoColumns`. For example in Prestissimo this is done in https://github.com/prestodb/presto/blob/48f0a0c1d380b1155dfd7c99b134a350627c7260/presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp#L1112-L1120. Reviewed By: xiaoxmeng Differential Revision: D64180231
Yuhta
force-pushed
the
export-D64180231
branch
from
ba9d540
to
a6e4e94
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64180231 |
…cubator#11221) Summary: These special columns are engine-specific and should be handled during engine split generation by setting the corresponding values in split `infoColumns`. For example in Prestissimo this is done in https://github.com/prestodb/presto/blob/48f0a0c1d380b1155dfd7c99b134a350627c7260/presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp#L1112-L1120. Reviewed By: xiaoxmeng Differential Revision: D64180231
Yuhta
force-pushed
the
export-D64180231
branch
from
a6e4e94
to
37b858d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64180231 |
…cubator#11221) Summary: These special columns are engine-specific and should be handled during engine split generation by setting the corresponding values in split `infoColumns`. For example in Prestissimo this is done in https://github.com/prestodb/presto/blob/48f0a0c1d380b1155dfd7c99b134a350627c7260/presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp#L1112-L1120. Reviewed By: xiaoxmeng Differential Revision: D64180231
Yuhta
force-pushed
the
export-D64180231
branch
from
37b858d
to
48fe9a9
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64180231 |
|
This pull request has been merged in 0758d04. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Summary: These special columns are engine-specific and should be handled during engine split generation by setting the corresponding values in split
infoColumns. For example in Prestissimo this is done in https://github.com/prestodb/presto/blob/48f0a0c1d380b1155dfd7c99b134a350627c7260/presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp#L1112-L1120.Differential Revision: D64180231