fix: Enable WASM compilation by making sqlparser's recursive-protection optional #16418
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ility This change addresses the PSM "section too large" error when building DataFusion for WebAssembly by making sqlparser's recursive-protection feature optional. Changes: - Set default-features = false for sqlparser in workspace Cargo.toml - Add sqlparser_std and sqlparser_recursive_protection features to datafusion crate - Include these features in datafusion's default features for backward compatibility This allows WASM builds to exclude the problematic dependency chain: sqlparser → recursive → stacker → psm While maintaining full compatibility for non-WASM builds. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
The std feature is required for sqlparser to compile properly. Without it, basic types and imports are missing.
alamb
reviewed
Jun 16, 2025
datafusion/core/Cargo.toml
Outdated
| @@ -79,6 +81,9 @@ recursive_protection = [ | |||
| "datafusion-physical-optimizer/recursive_protection", | |||
| "datafusion-sql/recursive_protection", | |||
| ] | |||
| # Enable sqlparser's default features for backward compatibility | |||
| sqlparser_std = ["sqlparser/std"] | |||
| sqlparser_recursive_protection = ["sqlparser/recursive-protection"] | |||
There was a problem hiding this comment.
Is there any need to add this new sqlparser_recursive_protection = ["sqlparser/recursive-protection"] feature? could we instead just add it to the recursive_protection existing feature?
recursive_protection = [
"datafusion-common/recursive_protection",
"datafusion-expr/recursive_protection",
"datafusion-optimizer/recursive_protection",
"datafusion-physical-optimizer/recursive_protection",
"datafusion-sql/recursive_protection",
"sqlparser/recursive-protection", # Add this???
]I also don't fully understand why we need a new feature for sqlparser_std as that would seem to already be enabled
There was a problem hiding this comment.
Oh, yeah, good call. Made these updates in 485a505
alamb
reviewed
Jun 16, 2025
There was a problem hiding this comment.
Thank you for this contribution @jonmmease -- I think wasm is a very important target.
alamb
approved these changes
Jun 16, 2025
|
Thanks again @jonmmease |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the WASM compilation issue (#13513) by making sqlparser's
recursive-protectionfeature optional. This allows DataFusion to be compiled for WebAssembly targets without encountering the "LLVM error: section too large" error caused by the PSM (Portable Stack Manipulation) dependency.Background
When compiling DataFusion for WASM (target
wasm32-unknown-unknown), the build fails with:The dependency chain causing this issue is:
The PSM crate is incompatible with WebAssembly because.
As noted in the PSM documentation itself:
The Fix
This PR implements a solution that maintains backward compatibility while enabling WASM builds:
default-features = falsefor sqlparser to prevent automatic inclusion ofrecursive-protectionsqlparser_std→ enablessqlparser/stdsqlparser_recursive_protection→ enablessqlparser/recursive-protectionUsage
After this change:
For WASM builds (no PSM dependency):
For regular builds (unchanged behavior):
Related Issues and Context
recursive-protectionfeature flag #13887 which made recursive_protection optional in datafusion-sqlNotes
While PR #13887 made
recursive_protectionoptional in datafusion-sql, it didn't fully solve the WASM compilation issue because sqlparser itself was still being pulled in with its default features across multiple DataFusion crates. This PR completes the fix by ensuring sqlparser's default features can be controlled at the workspace level.