Skip to content

settings change in replicators #390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 67 commits into
base: feature/contract
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
9a24b49
Update BUILDLIN.md
AlinaYezhkova Jul 3, 2023
7fd460c
Fix install rpath
cyrillicw Jul 5, 2023
63e0d3a
Merge remote-tracking branch 'origin/develop' into develop
cyrillicw Jul 5, 2023
9bafea6
Added include optional to FsTree.h and ReadEventHandler.h
AlinaYezhkova Jul 5, 2023
0a525c7
Merge pull request #382 from proximax-storage/detached
AlinaYezhkova Jul 5, 2023
f071d72
Submodules
Cempl Jul 7, 2023
1602935
Fix entity types for streaming
Cempl Jul 11, 2023
7f0e5ca
Add DbrbProcess unit tests.
ymol Jul 19, 2023
8cdbe8c
Added loading network config from cache on boot.
otsybizov Jul 26, 2023
022a6e0
fix sirius lib path
shinneng Jul 26, 2023
3337d87
Fixed network config height when reading network config from cache on…
otsybizov Jul 26, 2023
f004731
Merge pull request #385 from proximax-storage/ci-patch-1
shinneng Jul 27, 2023
97cf40f
added config values same on all executors
AlinaYezhkova Aug 2, 2023
fab381f
networkIdentifier removed
AlinaYezhkova Aug 2, 2023
e7f7186
Possible fix of std::out_of_range exception in map::at.
otsybizov Aug 4, 2023
7772fd1
resolve bootstrap addresses
Cempl Aug 11, 2023
b91ef26
Skip tests from third-part libs
Cempl Aug 11, 2023
9d5d20d
Skip tests from third-party libs
Cempl Aug 15, 2023
fabd332
Fixed block time calculation when selecting committee.
otsybizov Aug 18, 2023
520f642
Merge branch 'develop' of https://github.com/proximax-storage/cpp-xpx…
otsybizov Aug 18, 2023
c56620f
changing configs while working
AlinaYezhkova Aug 30, 2023
6207ebd
settings change in replicators
AlinaYezhkova Sep 6, 2023
834081c
added logs ModifyOpinionController.h
AlinaYezhkova Sep 8, 2023
799cfe7
minor changes
AlinaYezhkova Sep 11, 2023
94dd0a2
Fixed the height of config that committee is requesting.
otsybizov Sep 11, 2023
89b1c8c
Save boot config at height 0 when reading config from cache on boot.
otsybizov Sep 11, 2023
c9dbba0
Set blockchain version 1.2.1
otsybizov Sep 11, 2023
a265d72
Temporary removed FF integration tests.
otsybizov Sep 12, 2023
23b7c36
Check host before resolve. Unnecessary checks removed
Cempl Sep 14, 2023
6c804d8
Merge pull request #387 from proximax-storage/dns_lookup
otsybizov Sep 14, 2023
be290a2
pull request fixes
AlinaYezhkova Sep 15, 2023
37ea467
Fixed DBRB issues.
otsybizov Sep 21, 2023
9d1b42c
Merge branch 'develop' of https://github.com/proximax-storage/cpp-xpx…
otsybizov Sep 21, 2023
b89441b
Fix replicator nodes shutdown.
ymol Sep 27, 2023
9696228
Submodules
Cempl Sep 28, 2023
cc9abe1
Submodules
Cempl Sep 29, 2023
cf6328e
Submodules
Cempl Oct 2, 2023
3a89a8f
Rework committee cache patricia tree.
otsybizov Oct 3, 2023
498495a
Submodules
Cempl Oct 11, 2023
d9963f2
Created separate thread pools for DBRB and WV FSM to avoid IO context…
otsybizov Oct 13, 2023
f7b5a85
Fixed DBRB process tests.
otsybizov Oct 13, 2023
7003111
Fix port issue, minor improvements
Cempl Oct 17, 2023
ec9b9cc
updated submodule
Alex-Tsarenko Oct 18, 2023
d0a0ecd
Fixed mosaic levy pruning.
otsybizov Oct 19, 2023
86574a9
Merge branch 'develop' of https://github.com/proximax-storage/cpp-xpx…
otsybizov Oct 19, 2023
b26fe8f
libtorrent switched to 'xpx-work-v4' branch
Alex-Tsarenko Oct 21, 2023
748815b
Fix storage sdk for Debian build
Cempl Oct 25, 2023
668b699
update debian ci
Oct 26, 2023
5b4404b
fix Dockerfile
Oct 26, 2023
8a92999
fix dockerfile
Oct 26, 2023
86b4da7
update debian base img
Oct 26, 2023
4c52cb0
modified: cpp-xpx-storage-sdk
Cempl Oct 26, 2023
58e1a87
updated cpp-xpx-storage-sdk
Alex-Tsarenko Oct 27, 2023
fa4582a
Merge branch 'develop' into fix-deb-ci
Oct 27, 2023
e4a5322
Merge pull request #395 from proximax-storage/fix-deb-ci
shinneng Oct 27, 2023
de5cd2c
submodules updated
Alex-Tsarenko Nov 3, 2023
4ae8f6d
fix in ReplicatorService.cpp (change tcp::endpoint -> udp::endpoint)
Alex-Tsarenko Nov 3, 2023
0ec0af8
updated submodule cpp-storage-sdk
Alex-Tsarenko Nov 7, 2023
afa4d21
Added scripts for stg
Cempl Nov 8, 2023
682f855
Added new dockerfile for linux (local)
Cempl Nov 8, 2023
e1b59a2
Merge branch 'develop' of https://github.com/proximax-storage/cpp-xpx…
Cempl Nov 8, 2023
c0f5669
submodule updated
Alex-Tsarenko Nov 9, 2023
2d26761
Improved docs (rocks db)
Cempl Nov 9, 2023
52313f7
Merge branch 'develop' of https://github.com/proximax-storage/cpp-xpx…
Cempl Nov 9, 2023
0766c81
Used rocksdb v6.6.4 for debian image
Cempl Nov 9, 2023
73d32a5
Merge branch 'develop' into networkConfig
Cempl Nov 15, 2023
5c5a78e
Submodules
Cempl Nov 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ Makefile
build.sh
compile_commands.json
__pycache__
data*/
npm-*.log*
.vscode/
/build/
Expand Down
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,7 @@
path = cpp-xpx-supercontract-sdk
url = https://github.com/proximax-storage/cpp-xpx-supercontract-sdk
branch = develop
[submodule "cpp-xpx-supercontract-sdk"]
path = cpp-xpx-supercontract-sdk
url = https://github.com/proximax-storage/cpp-xpx-supercontract-sdk
branch = develop
1 change: 1 addition & 0 deletions CMakeGlobalSettings.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ include(cmake/sanitizers.cmake)

### enable testing
enable_testing()
configure_file(cmake/CTestCustom.cmake ${CMAKE_BINARY_DIR})

### enable ccache if available
find_program(CCACHE_FOUND ccache)
Expand Down
5 changes: 4 additions & 1 deletion ci/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,10 @@ pipeline {
make -j4 publish
make -j4
cd ..
./scripts/release-script/copyDeps.sh _build/bin/ ./deps
./scripts/release-script/copyDeps.sh _build/bin ./deps
mkdir -p ./deps/sirius
cp -r _build/lib ./deps/sirius/
rm -rf ./deps/home
"""
}
}
Expand Down
2 changes: 1 addition & 1 deletion ci/debian/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ pipeline {
stages {
stage ('Multi-architecture Docker Build and Push'){
when {
tag 'v*-bullseye'
tag 'v*-bookworm'
}
parallel {
stage('Build on AMD64'){
Expand Down
24 changes: 24 additions & 0 deletions cmake/CTestCustom.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
set(CTEST_CUSTOM_TESTS_IGNORE
charclass_test
compile_test
filtered_re2_test
mimics_pcre_test
parse_test
possible_match_test
re2_test
re2_arg_test
regexp_test
required_prefix_test
search_test
set_test
simplify_test
string_generator_test
dfa_test
exhaustive1_test
exhaustive2_test
exhaustive3_test
exhaustive_test
random_test
example
example64
)
2 changes: 1 addition & 1 deletion cpp-xpx-storage-sdk
Submodule cpp-xpx-storage-sdk updated 65 files
+1 −1 .gitmodules
+16 −4 CMakeLists.txt
+1 −1 README.md
+1 −1 cereal
+11 −0 cmake/grpc.cmake
+7 −3 examples/CMakeLists.txt
+25 −7 examples/modifyDriveFlat3/modifyDriveFlat3-repl-status.cpp
+5 −1 examples/standalone-tests/utils.h
+26 −20 examples/streamExample/offchainTest.cpp
+5 −1 examples/supercontract-tests/utils.h
+3 −2 include/drive/ActionList.h
+144 −74 include/drive/ClientSession.h
+71 −45 include/drive/EndpointsManager.h
+2 −1 include/drive/FsTree.h
+1 −0 include/drive/ManualModificationsRequests.h
+3 −1 include/drive/Replicator.h
+30 −1 include/drive/RpcRemoteReplicator.h
+14 −3 include/drive/RpcReplicator.h
+5 −0 include/drive/RpcReplicatorCommands.h
+16 −16 include/drive/Session.h
+58 −16 include/drive/StreamerSession.h
+27 −1 include/drive/Streaming.h
+7 −3 include/drive/Timer.h
+13 −1 include/drive/Utils.h
+28 −3 include/drive/log.h
+1 −0 include/ionet/IoTypes.h
+3 −2 include/types.h
+1 −1 libtorrent
+8 −0 macos/add_ips
+8 −0 macos/del_ips
+7 −5 messenger-server/src/MessageReader.cpp
+8 −8 messenger-server/src/MessageWriter.cpp
+3 −0 messenger-server/src/MessengerServer.cpp
+1 −0 messenger-server/src/ReadEventHandler.h
+12 −1 messenger-server/src/StreamContextKeeper.cpp
+5 −0 messenger-server/src/StreamContextKeeper.h
+1 −1 messenger-server/src/WriteRPCTag.cpp
+4 −0 replicator-service/CMakeLists.txt
+2 −2 src/drive/ActionList.cpp
+6 −2 src/drive/CMakeLists.txt
+8 −9 src/drive/DownloadLimiter.h
+24 −26 src/drive/DriveParams.h
+4 −4 src/drive/DriveTaskBase.h
+4 −6 src/drive/FlatDrive.cpp
+12 −12 src/drive/FsTree.cpp
+26 −16 src/drive/InitializeDriveTask.cpp
+39 −31 src/drive/ManualModificationsTask.cpp
+7 −7 src/drive/ModifyApprovalTaskBase.h
+20 −10 src/drive/ModifyDriveTask.cpp
+19 −5 src/drive/ModifyOpinionController.h
+2 −2 src/drive/RcptSyncronizer.h
+104 −86 src/drive/Replicator.cpp
+83 −64 src/drive/Session.cpp
+109 −32 src/drive/StreamTask.cpp
+12 −11 src/drive/SynchronizationTaskBase.h
+14 −5 src/drive/Utils.cpp
+2 −2 src/drive/VerificationDriveTask.cpp
+6 −0 src/drive/ViewerSession.cpp
+1 −0 src/drive/http_connection.hpp
+1 −0 src/drive/http_reply.hpp
+1 −0 src/drive/http_server.hpp
+7 −2 src/emulator/RpcReplicatorClient.cpp
+1 −1 src/model/CMakeLists.txt
+1 −1 supercontract-server/src/AcceptRequestRPCTag.cpp
+7 −1 supercontract-server/src/CMakeLists.txt
2 changes: 1 addition & 1 deletion cpp-xpx-supercontract-sdk
Submodule cpp-xpx-supercontract-sdk updated 37 files
+5 −0 CMakeLists.txt
+1 −0 cmake/cereal.cmake
+3 −2 cmake/expected.cmake
+12 −0 cmake/grpc.cmake
+1 −0 cmake/magic_enum.cmake
+17 −12 include/rpcExecutorServer/RPCExecutor.h
+4 −2 libs/blockchain/include/blockchain/RPCBlockchainServer.h
+4 −4 libs/blockchain/src/AcceptRequestTag.h
+8 −6 libs/blockchain/src/CachedBlockchain.cpp
+2 −2 libs/blockchain/src/FinishRequestTag.h
+5 −1 libs/blockchain/src/RPCBlockchainClient.cpp
+28 −16 libs/blockchain/src/RPCBlockchainServer.cpp
+1 −1 libs/executor/include/executor/DefaultExecutorBuilder.h
+2 −0 libs/executor/include/executor/Executor.h
+155 −143 libs/executor/include/executor/ExecutorConfig.h
+23 −16 libs/executor/src/BatchExecutionTask.cpp
+5 −4 libs/executor/src/DefaultBatchesManager.cpp
+4 −7 libs/executor/src/DefaultExecutor.cpp
+4 −4 libs/executor/src/DefaultExecutor.h
+2 −2 libs/executor/src/DefaultExecutorBuilder.cpp
+1 −1 libs/executor/src/ExecutorEnvironment.h
+4 −3 libs/executor/src/InternetQueryHandler.cpp
+3 −1 libs/executor/src/InternetQueryHandler.h
+1 −1 libs/executor/src/ManualCallBlockchainQueryHandler.cpp
+0 −1 libs/executor/src/StorageQueryHandler.h
+1 −1 libs/executor/src/SynchronizationTask.cpp
+1 −0 libs/executor/test/StorageUtils.h
+1 −0 libs/utils/include/utils/Random.h
+6 −2 libs/virtualMachine/include/virtualMachine/CallRequest.h
+19 −0 protobuf/executor.proto
+4 −0 src/rpcExecutorClient/ExecutorService.cpp
+12 −0 src/rpcExecutorClient/RPCExecutorClient.cpp
+3 −0 src/rpcExecutorClient/RPCExecutorClient.h
+43 −20 src/rpcExecutorServer/RPCExecutor.cpp
+37 −0 src/rpcExecutorServer/RPCExecutorServerErrorCode.cpp
+40 −0 src/rpcExecutorServer/RPCExecutorServerErrorCode.h
+4 −2 src/rpcExecutorServer/ReadRPCTag.cpp
71 changes: 70 additions & 1 deletion docs/BUILDLIN.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,72 @@
# Building on Ubuntu 22.04 (LTS)

Use the script below:

```
set -e
set -x

cd /tmp

apt-get update -y && apt-get upgrade -y && apt-get clean && \
DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-install-recommends \
cmake \
git \
curl \
wget \
build-essential \
gcc-12 \
software-properties-common \
pkg-config \
libssl-dev \
libsasl2-dev \
libtool \
libboost-dev \
libboost-atomic-dev \
libboost-date-time-dev \
libboost-regex-dev \
libboost-system-dev \
libboost-timer-dev \
libboost-chrono-dev \
libboost-log-dev \
libboost-thread-dev \
libboost-filesystem-dev \
libboost-program-options-dev \
libboost-stacktrace-dev \
libboost-random-dev \
libgtest-dev \
libbenchmark-dev \
librocksdb-dev \
libzmq3-dev \
libmongoc-1.0-0 \
libmongoc-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Replace gcc with gcc-12
rm /usr/bin/gcc && ln -s /usr/bin/gcc-12 /usr/bin/gcc

cd /usr/local/src

curl -OL https://github.com/zeromq/cppzmq/archive/v4.9.0.tar.gz && \
tar xzf v4.9.0.tar.gz && \
cd cppzmq-4.9.0 && \
mkdir build && \
cd build && \
cmake -DCPPZMQ_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local .. && \
make -j4 && \
make install

# Install mongo-cxx
curl -OL https://github.com/mongodb/mongo-cxx-driver/releases/download/r3.7.0/mongo-cxx-driver-r3.7.0.tar.gz && \
tar -xzf mongo-cxx-driver-r3.7.0.tar.gz && \
cd mongo-cxx-driver-r3.7.0/build && \
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DBSONCXX_POLY_USE_BOOST=1 .. && \
make -j4 && \
make install

# Clean Downloads
rm v4.9.0.tar.gz mongo-cxx-driver-r3.7.0.tar.gz
```
# Building on Ubuntu 18.04 (LTS)

Instructions below are for gcc, but project compiles with clang 9 as well.
Expand Down Expand Up @@ -119,7 +188,7 @@ cd rocksdb.git
git checkout v6.6.4

mkdir _build && cd _build
cmake -DCMAKE_BUILD_TYPE=Release -DWITH_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/local ..
cmake -DCMAKE_BUILD_TYPE=Release -DWITH_TESTS=OFF -DWITH_BENCHMARK_TOOLS=OFF -DWITH_TOOLS=OFF -DFAIL_ON_WARNINGS=OFF -DWITH_RUNTIME_DEBUG=OFF -DCMAKE_INSTALL_PREFIX=/usr/local ..
make
sudo make install
```
Expand Down
5 changes: 4 additions & 1 deletion extensions/fastfinality/FastFinalityExtension.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
**/

#include "src/WeightedVotingService.h"
#include "src/dbrb/TransactionSender.h"
#include "catapult/extensions/ProcessBootstrapper.h"
#include "catapult/harvesting_core/ValidateHarvestingConfiguration.h"

Expand All @@ -16,7 +17,9 @@ namespace catapult { namespace fastfinality {
ValidateHarvestingConfiguration(harvestingConfig);
auto dbrbConfig = dbrb::DbrbConfiguration::LoadFromPath(bootstrapper.resourcesPath());

bootstrapper.extensionManager().addServiceRegistrar(CreateWeightedVotingServiceRegistrar(harvestingConfig, dbrbConfig, bootstrapper.resourcesPath()));
auto pTransactionSender = std::make_shared<dbrb::TransactionSender>();
bootstrapper.subscriptionManager().addPostBlockCommitSubscriber(std::make_unique<dbrb::BlockSubscriber>(pTransactionSender));
bootstrapper.extensionManager().addServiceRegistrar(CreateWeightedVotingServiceRegistrar(harvestingConfig, dbrbConfig, pTransactionSender));
}
}
}}
Expand Down
14 changes: 14 additions & 0 deletions extensions/fastfinality/src/CommitteeData.h
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,18 @@ namespace catapult { namespace fastfinality {
return m_sumOfPrevotesSufficient;
}

void setCurrentBlockHeight(const Height& value) {
m_currentBlockHeight = value;
}

void incrementCurrentBlockHeight() {
m_currentBlockHeight = m_currentBlockHeight + Height(1);
}

auto currentBlockHeight() const {
return m_currentBlockHeight;
}

private:
Key m_beneficiary;
std::shared_ptr<harvesting::UnlockedAccounts> m_pUnlockedAccounts;
Expand All @@ -193,5 +205,7 @@ namespace catapult { namespace fastfinality {
bool m_sumOfPrevotesSufficient;

mutable std::mutex m_mutex;

Height m_currentBlockHeight;
};
}}
Loading