fix: try older w3up client #2548
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
found memory issues went away with older w3up-client in test, this simply applies the same revert
hannahhoward
force-pushed
the
fix/try-older-client
branch
from
f9a8b8a
to
7a04c47
Compare
gobengo
approved these changes
Apr 2, 2024
There was a problem hiding this comment.
I think this is low risk to try (due to only affecting functionality enabled when feature switch allows), and will be interesting to know whether this helps avoid the OOM.
We had thought we were running with the piece hasher disabled, but this is a good way of trying it with it definitely disabled (i.e. running code from before it was added).
travis
pushed a commit
that referenced
this pull request
Apr 2, 2024
Reverts #2548 Why? * that downgrade of w3up-client didn't fix everything * @hannahhoward mentioned that in local testing, she thinks that the newer version doesn't use extra memory when configured with a `null` pieceHasher, which is what we're doing * I can't think of a good reason to be on old version vs latest version
travis
pushed a commit
that referenced
this pull request
Apr 3, 2024
🤖 I have created a release *beep* *boop* --- ## [4.5.0](api-v4.4.3...api-v4.5.0) (2024-04-02) ### Features * POST /uploads can write uploads to w3up for web3 serving and dagcargoless filecoin deal-making ([#2522](#2522)) ([0c77cb3](0c77cb3)) * uploadCarWithStat avoids copying stat.carBytes via Blob construction ([#2543](#2543)) ([bcb67a3](bcb67a3)) ### Bug Fixes * add a fake piece hasher to try to fix staging upload issues ([#2542](#2542)) ([542983c](542983c)) * avoid car repack ([#2551](#2551)) ([b0d622e](b0d622e)) * avoid car repack ([#2552](#2552)) ([b4ff571](b4ff571)) * change to post method ([#2547](#2547)) ([65e20fb](65e20fb)) * enable `url_standard` compatbility flag ([#2540](#2540)) ([bdde10a](bdde10a)) * enable the `url_standard` compatibility flag ([#2537](#2537)) ([e822bab](e822bab)) * enabled nodejs compat ([#2545](#2545)) ([654cde7](654cde7)) * log out proof ([225cf28](225cf28)) * log proof for debugging ([#2535](#2535)) ([0ecace3](0ecace3)) * more logging ([#2536](#2536)) ([11ae646](11ae646)) * remove debugging console.logs and nodejs_compat flag ([#2553](#2553)) ([6ceb299](6ceb299)) * Revert "fix: try older w3up client" ([#2549](#2549)) ([b31847b](b31847b)) * revert debugging prs ([#2539](#2539)) ([ba22d01](ba22d01)) * try older w3up client ([#2548](#2548)) ([5d94ec5](5d94ec5)) * upgrade @ipld/dag-cbor ([#2534](#2534)) ([964a12c](964a12c)) ### Other Changes * add a bunch of console.logs ([#2544](#2544)) ([2fc34f3](2fc34f3)) * expose simple route to debug ([#2546](#2546)) ([8a9aef5](8a9aef5)) * simple test case to check limits ([#2550](#2550)) ([6a50a70](6a50a70)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Goals
I found memory issues went away with older w3up-client in my upload test.
This latest client appears not to be compatible with whatever APIs exist on local.storage, though perhaps web3.storage is updated -- I was actually trying to resolve this and accidentally discovered that I didn't have a memory crash any more.
Debugging in VS Code put the error in PieceHashing code, so it makes sense that the latest client, which adds filecoin/offer running locally, would potentially consume more memory.