Meta rebase #3
Closed
Meta rebase #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: GitHub commits: facebook@35c9317 facebook/fb303@ff82ca2 facebook/fbthrift@fd12520 facebook/proxygen@19e6422 facebook/watchman@13b64db facebookexperimental/rust-shed@bcbcd78 Reviewed By: jurajh-fb fbshipit-source-id: 1cae62ca1816999755fa7d02ade5acb0d7d6a1c2
Summary: GitHub commits: facebook@2d00d87 facebook/fb303@64c9ab5 facebook/fbthrift@4d10325 facebook/watchman@e0d358e facebookexperimental/rust-shed@8dd991d facebookincubator/velox@879dbcb pytorch/FBGEMM@49061a2 Reviewed By: jurajh-fb fbshipit-source-id: 4079f877b1f8824d11f1b38e426d902d82032d64
Summary: GitHub commits: facebook/fbthrift@2084e71 facebookresearch/FLSim@25dcc15 Reviewed By: jurajh-fb fbshipit-source-id: 633ebba055ee849f612624dfdb4586e102f6c5e0
Summary: GitHub commits: facebook@9754496 facebook/fb303@c90b698 facebook/fbthrift@09c7898 facebook/watchman@59d2099 facebookexperimental/rust-shed@739451c Reviewed By: jurajh-fb fbshipit-source-id: f27967de0574c05e7a14443a058b77222d51a015
Summary: GitHub commits: facebook/fbthrift@4816073 Reviewed By: jurajh-fb fbshipit-source-id: 98a25f03da94d1a493dc3164ef873f4a7a6c4e7f
Summary: GitHub commits: facebook/fbthrift@f090926 Reviewed By: jurajh-fb fbshipit-source-id: 7d3458299e95ab7202eeb336fbaa8861f763f477
Summary: GitHub commits: facebook/fbthrift@29f5be1 Reviewed By: jurajh-fb fbshipit-source-id: 681ea59613c4a5f67b88cdefcef0b4992f64ecdd
Summary: - `ItemHandle` -> `WriteHandle` codemod in external use cases - Remove `ItemHandle` from cachelib codebase Reviewed By: therealgymmy Differential Revision: D37500348 fbshipit-source-id: 4aaf5bed9b1472064770e8845e2bdbd19af230a4
Summary: GitHub commits: facebook@6fde167 facebook/fb303@75420d5 facebook/fbthrift@c3f7287 facebook/folly@6198adf facebook/litho@f9c68f0 facebook/proxygen@82d1cc3 facebook/rocksdb@e637470 facebook/wangle@feb8186 facebook/watchman@8d674a1 facebookexperimental/edencommon@7b891ae facebookexperimental/rust-shed@450d3f3 facebookincubator/Eigen-FBPlugins@a04ae2c facebookincubator/fizz@2c581c3 facebookincubator/katran@6c1b037 facebook/mvfst@c3a61c5 facebookincubator/velox@c76a9d2 facebookresearch/beanmachine@834f3a7 facebookresearch/multimodal@1c4fa80 pytorch/FBGEMM@a6f5488 pytorch/multipy@5af38eb Reviewed By: wittgenst fbshipit-source-id: 2ad5d5d2f8153b308824a361b41c3d003b2f030d
Summary: GitHub commits: facebook@b55cb24 facebook/fb303@f579f33 facebook/fbthrift@2e1eb49 facebook/proxygen@117513a facebook/rocksdb@2fc6df3 facebook/wangle@f238460 facebook/watchman@19fb934 facebookexperimental/edencommon@a28957c facebookexperimental/rust-shed@bd5a53d facebookincubator/fizz@fa63302 facebookincubator/katran@d1f0274 facebook/mvfst@0e05cc8 facebookincubator/velox@9ea9ba4 facebookresearch/beanmachine@463c58c Reviewed By: wittgenst fbshipit-source-id: fd0f90bc6848a85848bc0d35909a1771f4888845
Summary: GitHub commits: facebook@ad8e718 facebook/fb303@c5c5c73 facebook/fbthrift@b7332da facebook/proxygen@d41b032 facebook/rocksdb@84e9b6e facebook/wangle@634f3dd facebook/watchman@7404ba0 facebookexperimental/rust-shed@3847584 facebookincubator/katran@00bc4ee facebook/mvfst@4985673 facebookresearch/multimodal@52c58ed Reviewed By: wittgenst fbshipit-source-id: b7c04db11d2cf98f76fcd6e940fadebdee3e79cc
Summary: GitHub commits: facebook@8fe6332 facebook/fb303@31df3c7 facebook/fbthrift@dfc34b1 facebook/proxygen@3c36738 facebook/watchman@06bb569 facebookexperimental/rust-shed@bcf941e pytorch/FBGEMM@60a5c2b Reviewed By: wittgenst fbshipit-source-id: 086b16d3d7411255bda0145f97ffa268ac046cd3
Summary: GitHub commits: facebook/fbthrift@8ce919e Reviewed By: wittgenst fbshipit-source-id: 830dd79a01a7084fa6f37ec71b2b8113dd3f64eb
Summary: GitHub commits: facebook/folly@bd3e0f7 Reviewed By: wittgenst fbshipit-source-id: 789ee0df03bc957ab274a27d859c8d3cd7a6e37c
Summary: GitHub commits: facebook@371611a facebook/fb303@240fddc facebook/fbthrift@3d0e4ec facebook/proxygen@dfcffe9 facebook/wangle@3ece116 facebook/watchman@b9da36d facebookexperimental/edencommon@63199b5 facebookincubator/fizz@feef1fb facebookincubator/katran@1526abb facebook/mvfst@ea0e50e facebookincubator/velox@1d66d46 Reviewed By: wittgenst fbshipit-source-id: 9d9714b7e1514b77947009c9d66b940e4d0a929d
Summary: GitHub commits: facebook@84d9cbb facebook/fb303@5e1c47a facebook/fbthrift@b309bab facebook/proxygen@1205190 facebook/wangle@a4f3a14 facebook/watchman@34f0429 facebookexperimental/rust-shed@8087bb5 facebookincubator/katran@f82567b facebook/mvfst@bdd8526 pytorch/FBGEMM@308dd51 Reviewed By: wittgenst fbshipit-source-id: aa40779e080c60d2b0c3cbc2f0877e2a8940695e
Summary: GitHub commits: facebook@3826474 facebook/fb303@d5e6e66 facebook/fbthrift@8d051f0 facebook/litho@80346a6 facebook/proxygen@94863a7 facebook/watchman@0c0f88f facebookexperimental/rust-shed@163e4a1 Reviewed By: wittgenst fbshipit-source-id: fea16b41458db53a94cce907d91fa67c9400f40f
Summary: GitHub commits: facebook/fbthrift@a4fdb4a Reviewed By: wittgenst fbshipit-source-id: f2511ee94b10175cdb76934e663aaad372bb4c13
Summary: GitHub commits: facebook@1b1efaf facebook/fb303@4772c04 facebook/fbthrift@ba4c6ea facebook/watchman@19bd6b4 facebookexperimental/rust-shed@ded0566 facebookresearch/FLSim@f552c50 Reviewed By: wittgenst fbshipit-source-id: 80c528702d5230d5e6c7393f5f7fcc5c065487b4
Summary: GitHub commits: facebook/fbthrift@93deaa2 facebook/litho@2ad905d facebook/watchman@063d421 facebookincubator/velox@0f34e2e Reviewed By: wittgenst fbshipit-source-id: 1f8193cbb4cc8ee7e0781ccc583e2a0c41e508fa
Summary: GitHub commits: facebook/folly@6e11b1e facebookresearch/beanmachine@8bdfb73 Reviewed By: wittgenst fbshipit-source-id: fe32c2aa0dea86fdc52906e03784f0a18cedb03e
Summary: GitHub commits: facebook@0ea6364 facebook/fb303@5590cab facebook/fbthrift@e9f06b8 facebook/proxygen@2649426 facebook/rocksdb@3134471 facebook/wangle@18d96b5 facebook/watchman@390c95f facebookexperimental/edencommon@bb0569b facebookexperimental/rust-shed@34d8ec0 facebookincubator/fizz@7a5a58c facebookincubator/katran@37aaf3c facebook/mvfst@b01a8f5 facebookincubator/velox@be7ca7b Reviewed By: wittgenst fbshipit-source-id: 73248af094ccbd63418d0cd4887116076ccee81c
Summary: GitHub commits: facebook@eb09228 facebook/fb303@e08baba facebook/fbthrift@7c1e720 facebook/folly@3b7c54b facebook/proxygen@e8ed3ca facebook/wangle@0e05dc5 facebook/watchman@413e0d2 facebookexperimental/edencommon@8ba1ed9 facebookexperimental/rust-shed@8867360 facebookincubator/katran@64a3e79 facebook/mvfst@fa7299a facebookincubator/velox@2bcce7f facebookresearch/FLSim@101ce58 facebookresearch/multimodal@d710d46 pytorch/FBGEMM@949420c Reviewed By: wittgenst fbshipit-source-id: 297d3dbc2e777e636440f598252b3f445292d668
Summary: GitHub commits: facebook@e5ce298 facebook/fb303@51c5fec facebook/fbthrift@ffa95ea facebook/proxygen@3a7ff11 facebook/wangle@5605ef6 facebook/watchman@ad48e7e facebookexperimental/edencommon@03096cb facebookexperimental/rust-shed@6048fab facebookincubator/fizz@13cdf6e facebookincubator/katran@bd20ac8 facebook/mvfst@4d56a82 facebookincubator/velox@dacfce2 Reviewed By: wittgenst fbshipit-source-id: 8288bb093f451b36913c6304e1e06b4fb2ba4ce8
Summary: GitHub commits: facebook@daf2f98 facebook/fb303@ffb4c6e facebook/fbthrift@627baed facebook/folly@b756ce7 facebook/proxygen@10a744d facebook/wangle@0b9cc53 facebook/watchman@28a2cd8 facebookexperimental/rust-shed@6323447 facebookincubator/fizz@8af5c40 facebookincubator/katran@505e2e2 facebook/mvfst@a8c0974 facebookincubator/velox@517427f facebookresearch/beanmachine@e2e9ca3 Reviewed By: wittgenst fbshipit-source-id: 552b6c319385063f5e3b8f58b8f7d1cee4cc2665
Summary: GitHub commits: facebook@02bcd69 facebook/fb303@0564c49 facebook/fbthrift@7607fe0 facebook/folly@40e95b6 facebook/proxygen@ce56514 facebook/wangle@d7a58eb facebook/watchman@f142258 facebookexperimental/edencommon@ec543da facebookexperimental/rust-shed@8a083b9 facebookincubator/fizz@90ffa60 facebookincubator/katran@ed60cc1 facebook/mvfst@f157e50 facebookincubator/velox@d0fdadc facebookresearch/multimodal@e890867 Reviewed By: wittgenst fbshipit-source-id: 60f2be957e6d952376335dfef1384989dd392be6
Summary: GitHub commits: facebook@af66133 facebook/fb303@82e2962 facebook/fbthrift@3ec58bd facebook/folly@3d6e679 facebook/proxygen@485be58 facebook/squangle@43c8644 facebook/wangle@ff3dc58 facebook/watchman@dd15e74 facebookexperimental/edencommon@cdaa6a4 facebookexperimental/rust-shed@9c585aa facebookincubator/fizz@c06e209 facebookincubator/katran@de93524 facebook/mvfst@83755a5 facebookincubator/velox@5a9075e Reviewed By: wittgenst fbshipit-source-id: f520b2425bb5979f99f189a85d869611268d38c6
Summary: GitHub commits: facebook@1653d55 facebook/fb303@e515de3 facebook/fbthrift@3e2bbfa facebook/folly@df26892 facebook/proxygen@dd26cd8 facebook/rocksdb@9d7de65 facebook/wangle@f84c916 facebook/watchman@d88f0bd facebookexperimental/edencommon@785c9fa facebookexperimental/rust-shed@6eca381 facebookincubator/fizz@813f959 facebookincubator/katran@fae921c facebook/mvfst@8677d5d facebookincubator/velox@eebb37a Reviewed By: wittgenst fbshipit-source-id: 9efc1f562eaae2094b447999022a67c7f7462088
Summary: GitHub commits: facebook@ee6a029 facebook/fb303@29d7d24 facebook/fbthrift@a8652e8 facebook/proxygen@dad8400 facebook/wangle@091189f facebook/watchman@1424672 facebookexperimental/edencommon@d45c7ac facebookexperimental/rust-shed@b1498a8 facebookincubator/fizz@1be0c95 facebookincubator/katran@57f43d6 facebook/mvfst@8cd45db facebookincubator/velox@20fafbf facebookresearch/beanmachine@6edb227 Reviewed By: wittgenst fbshipit-source-id: 7dbd79942c9ae98dafd74d5425c55186035f28b6
Summary: GitHub commits: facebook@45410de facebook/fb303@67fa3c4 facebook/fbthrift@cade15e facebook/proxygen@a751494 facebook/wangle@ca86d75 facebook/watchman@df824c0 facebookexperimental/rust-shed@8d44cc8 facebookincubator/katran@b275fd3 facebook/mvfst@d4f7cab Reviewed By: wittgenst fbshipit-source-id: 4b12b72db586107f0a1f8bdff9c6b77aeb0518a2
Without this fix removeCb called even in case when Item is moved between tiers.
It fails because CentOS is EOL. We might want to consider using CentOS Streams but for now, just remove it. Right now, we rely on build-cachelib-centos workflow anyway.
Disabled test suite allocator-test-AllocatorTypeTest to skip sporadically failing tests.
Compensation results in ratios being different than originially specified.
Return a sum of sizes of each tier instead of just 1st tier's size.
Compilation of some of the tests was failing with: "undefined reference to `facebook::cachelib::test_util::getRandomAsciiStr[abi:cxx11](unsigned int)'" Fix this by linking those tests with common/TestUtils.cpp Also, for some reason specyfing sources as in add_library(common_test_support INTERFACE [sources...]) did not work. Using target_sources works fine.
Centos8 is EOL
The issue happened when ReadHandleImpl ctor needs to destroy waitContext_ because addWaitContextForMovingItem() returns false. So before destroying waitContext_ we are calling discard method to notify ~ItemWaitContext() that Item is ready.
Do not use shallow clone to make sure Docker rebuild logic works correctly.
Get tier id of item before calling any function on allocator (which needs the tierID).
Remove the item from mmContainer and drop the lock before attempting eviction. Use moving bit for synchronization in findEviction moving bit is used to give exclusive right to evict the item to a particular thread. Originially, there was an assumption that whoever marked the item as moving will try to free it until he succeeds. Since we don't want to do that in findEviction (potentially can take a long time) we need to make sure that unmarking is safe. This patch checks the flags after unmarking (atomically) and if ref is zero it also recyles the item. This is needed as there might be some concurrent thread releasing the item (and decrementing ref count). If moving bit is set, that thread would not free the memory back to allocator, resulting in memory leak on unmarkMoving().
The main purpose of this patch is to better simulate workloads in cachebench. Setting touchValue to true allows to see performance impact of using different mediums for memory cache.
* #75: Use actual tier sizes (rounded down to slab size and decreased by header size) when creating new memory pools * Added getPoolSize method to calculate combined pool size for all tiers; added pool size validation to tests * Explicitly specified type for totalCacheSize to avoid overflow * Minor test change * Reworked tests * Minor change * Deleted redundant tests * Deleted unused constant * First set of changes to cache configuration API to enable multi-tier caches (facebook#138) Summary: These changes introduce per-tier cache configuration required to implement features discussed here: facebook#102. These specific changes enable single DRAM tier configs only which are compatible with the current version of cachelib. Configuration API will be expanded as multi-tier changes in other parts of the library are introduced. Pull Request resolved: facebook#138 Reviewed By: therealgymmy Differential Revision: D36189766 Pulled By: jiayuebao fbshipit-source-id: 947aa0cd800ea6accffc1b7b6b0c9693aa7fc0a5 Co-authored-by: Victoria McGrath <victoria.mcgrath@intel.com>
Hot queue iterator for 2Q. Will start at Hot queue and move to Warm queue if hot queue is exhausted. Useful for promotion semantics if using 2Q replacement. rebased on to develop and added some tests.
vinser52
pushed a commit
that referenced
this pull request
Sep 12, 2022
Summary: AdRanker ASAN canary flagged a possible UBSan violation. ## Error Failed Run: https://fburl.com/servicelab/apytosry ``` #0 0x562e3adb59bc in facebook::cachelib::objcache2::ObjectCacheSizeController<facebook::cachelib::CacheAllocator<facebook::cachelib::LruCacheTrait> >::work() buck-out/v2/gen/fbcode/47d914adeee3d982/cachelib/experimental/objcache2/__object-cache__/headers/cachelib/experimental/objcache2/ObjectCacheSizeController-inl.h #1 0x562de7610f78 in facebook::cachelib::PeriodicWorker::loop() fbcode/cachelib/common/PeriodicWorker.cpp:55 #2 0x7f7632c524e4 in execute_native_thread_routine /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:82:18 #3 0x7f7632f6ec0e in start_thread /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/nptl/pthread_create.c:434:8 #4 0x7f76330011db in clone3 /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 UndefinedBehaviorSanitizer: integer-divide-by-zero buck-out/v2/gen/fbcode/47d914adeee3d982/cachelib/experimental/objcache2/__object-cache__/headers/cachelib/experimental/objcache2/ObjectCacheSizeController-inl.h:33:40 in ``` Reviewed By: jiayuebao Differential Revision: D39024188 fbshipit-source-id: 64ad644c360565e638fa3ca74616a315038382ab
byrnedj
pushed a commit
that referenced
this pull request
Jul 23, 2023
Publish changes from innersouce
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.