Porting spidermonkey60 support to fdb-layer #2535
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Most of these tests are for quorum and clustered response handling which will no longer exist with FoundationDB. Eventually we'll want to go through these and pick out anything that is still applicable and ensure that we re-add them to the new test suite.
This provides a base implementation of a fabric API backed by
FoundationDB. While a lot of functionality is provided there are a
number of places that still require work. An incomplete list includes:
1. Document bodies are currently a single key/value
2. Attachments are stored as a range of key/value pairs
3. There is no support for indexing
4. Request size limits are not enforced directly
5. Auth is still backed by a legacy CouchDB database
6. No support for before_doc_update/after_doc_read
7. Various implementation shortcuts need to be expanded for full API
support.
This provides a good bit of code coverage for the new implementation. We'll want to expand this to include relevant tests from the previous fabric test suite along with reading through the various other tests and ensuring that we cover the API as deeply as is appropriate for this layer.
This is not an exhaustive port of the entire chttpd API. However, this is enough to support basic CRUD operations far enough that replication works.
This still holds all attachment data in RAM which we'll have to revisit at some point.
When uploading an attachment we hadn't yet flushed data to FoundationDB which caused the md5 to be empty. The `new_revid` algorithm then declared that was because it was an old style attachment and thus our new revision would be a random number. This fix just flushes our attachments earlier in the process of updating a document.
I was accidentally skipping this step around properly serializing/deserializing attachments. Note to self: If someon specifies attachment headers this will likely break when we attempt to pack the value tuple here.
The older chttpd/fabric split configured filters as one step in the coordinator instead of within each RPC worker.
This fixes the behavior when validating a document update that is recreating a previously deleted document. Before this fix we were sending a document body with `"_deleted":true` as the existing document. However, CouchDB behavior expects the previous document passed to VDU's to be `null` in this case.
This was a remnant before we used a version per database.
This changes `chttpd_auth_cache` to use FoundationDB to back the `_users` database including the `before_doc_update` and `after_doc_read` features.
RFC: apache/couchdb-documentation#409 Main API is in the `couch_jobs` module. Additional description of internals is in the README.md file.
Neither partitioned databases or shard splitting will exist in a FoundationDB layer.
This adds the mapping of CouchDB start/end keys and so on to the similar yet slightly different concepts in FoundationDB. The handlers for `_all_dbs` and `_all_docs` have been udpated to use this new logic.
The existing logic around return codes and term formats is labyrinthine. This is the result of much trial and error to get the new logic to behave exactly the same as the previous implementation.
Simple function change to `fabric2_db:name/1`
Previously I was forgetting to keep the previous history around which ended up limiting the revision depth to two.
The old test got around this by using couch_httpd_auth cache in its tests which is fairly odd given that we run chttpd_auth_cache in production. This fixes that mistake and upgrades chttpd_auth_cache so that it works in the test scenario of changing the authentication_db configuration.
Bump ioq to 2.1.3
This ets table was a holdover from when couch_expiring_cache was a non- library OTP application. It is unused, and would prevent multiple users of the library in the same project.
Delete unused ets table creation
reserve search namespace
Add http reporter
There were couple of hacks in test/elixir/lib/couch.ex We've got changes needed to remove them into httpotion 3.1.3. The changes were introduced in: - valpackett/httpotion#118 - valpackett/httpotion#130
Fix b3 header
Co-Authored-By: Jan Lehnardt <jan@apache.org>
Co-Authored-By: Paul J. Davis <paul.joseph.davis@gmail.com>
* fix: avoid segfaults, patch by @davisp * fix: build against sm60 on mac needs extra compiler flags
We've had a number of segfaults in the `make javascript` test suite. The few times we've been able to get core dumps all appear to indicate something wrong in the JIT compiler. Disabling the JIT compilers appears to prevent these segfaults.
Apparently SpiderMonkey 60 changed the behavior of OOM errors to not exit the VM. This updates the SpiderMonkey 60 implementation to match that behavior.
This test is actually checking the behvior of an OOM in `couchjs` now since we lifted the OS process timeout limit.
This changes the couchjs OOM test so that it will trigger more reliably on SpiderMonkey 60. It appears that newer SpiderMonkeys are better at conserving memory usage which takes this test longer to trigger.
and max as macros through a #define Co-authored-by: Joan Touzet <wohali@users.noreply.github.com>
Member
|
@jiangphcn you're going to want https://github.com/apache/couchdb/pull/2534/files#diff-0c65e7bb4bafbb8cf147cc0001ccd436 too, at least L202-207 if not the rest. but if |
Co-authored-by: Joan Touzet <wohali@users.noreply.github.com>
Contributor
Author
|
@wohali thanks for your tips. I added L202-207 for now. This PR arises when working with @garrensmith on one issue in his Mac environment. We got segment fault on Catalina with spidermonkey 1.8.5 (might be related to 32 bit compatibility issue)and work fine after switching to 60. Knowing that this doesn't happen in every development environment, but it is helpful to add spidermonkey 60 support in fdb-layer branch to continue. Thanks for forking |
davisp
force-pushed
the
prototype/fdb-layer
branch
from
b3bd36b to
bdd0578
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Porting spidermonkey60 support to fdb-layer
Testing recommendations
Related Issues or Pull Requests
The codes which are mainly ported are
#2345
Checklist
rel/overlay/etc/default.ini