Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: restore ServicesMainTest.java #17262

Merged
merged 44 commits into from
Jan 9, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
1c41076
feat: node starts without using Netwoprk.isLocal
edward-swirldslabs Mar 13, 2024
675ac38
missing newline
edward-swirldslabs Mar 13, 2024
a2a49cc
refactored use of nodesToStart
edward-swirldslabs Mar 14, 2024
13a4ae1
Merge branch 'develop' into 11751-determine-node-ids-to-start-without…
edward-swirldslabs Dec 17, 2024
8944f2e
Cleanup Post Merge Of Develop
edward-swirldslabs Dec 17, 2024
4692667
code review.
edward-swirldslabs Dec 17, 2024
de3c113
Merge branch 'develop' into 11751-determine-node-ids-to-start-without…
edward-swirldslabs Dec 17, 2024
e94d18f
Fix Ability To Run Locally
edward-swirldslabs Dec 17, 2024
40f61cb
unused methods.
edward-swirldslabs Dec 17, 2024
99c58bf
adding environment nodesToRun.
edward-swirldslabs Dec 19, 2024
c22de65
Merge branch 'develop' into 11751-determine-node-ids-to-start-without…
edward-swirldslabs Dec 19, 2024
b595450
Merge branch 'develop' into 11751-determine-node-ids-to-start-without…
edward-swirldslabs Dec 20, 2024
4327bd3
Merge branch 'develop' into 11751-determine-node-ids-to-start-without…
edward-swirldslabs Dec 20, 2024
997ff3e
code review and post merge of develop
edward-swirldslabs Dec 20, 2024
e3d685b
Merge branch 'develop' into 11751-determine-node-ids-to-start-without…
edward-swirldslabs Dec 20, 2024
d31f0ee
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Dec 22, 2024
3a1fdbc
cleaning up imports
edward-swirldslabs Dec 23, 2024
6450780
removing broken unit tests
edward-swirldslabs Dec 23, 2024
2f63a17
adding NodeIdConverter
edward-swirldslabs Dec 23, 2024
3bb8795
simplify GUI invocation
edward-swirldslabs Dec 23, 2024
a7ea4d8
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Dec 23, 2024
a4868a1
spotless apply after merging main.
edward-swirldslabs Dec 23, 2024
a3b8a81
updating name of NetworkEndpointConverter
edward-swirldslabs Dec 23, 2024
0fb9830
chore: add consensus node id support to the docker images
nathanklick Dec 31, 2024
8394a9f
chore: add diagnostic output to container images
nathanklick Dec 31, 2024
bc0466f
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Jan 2, 2025
1e0a6d7
chore: apply spotless formatting
nathanklick Jan 3, 2025
ccc462a
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
nathanklick Jan 3, 2025
21e5ef6
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Jan 6, 2025
45e8990
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Jan 7, 2025
4ff5141
Updating log text
edward-swirldslabs Jan 7, 2025
3060319
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Jan 7, 2025
29ba5c2
Update hedera-node/hedera-app/src/main/java/com/hedera/node/app/Servi…
edward-swirldslabs Jan 8, 2025
f9ea609
Update hedera-node/hedera-app/src/main/java/com/hedera/node/app/info/…
edward-swirldslabs Jan 8, 2025
8a8c6b6
Merge branch 'main' into 11751-determine-node-ids-to-start-without-ne…
edward-swirldslabs Jan 8, 2025
9ebbbbe
Merge branch 'refs/heads/main' into 17261-restore-ServicesMainTest.java
edward-swirldslabs Jan 8, 2025
5dd6f24
chore: restoring ServicesMainTest
edward-swirldslabs Jan 8, 2025
30826e0
deleting commented out line
edward-swirldslabs Jan 8, 2025
0224833
updated test names to reflect actual behavior.
edward-swirldslabs Jan 8, 2025
acd05d5
Merge branch 'main' into 17261-restore-ServicesMainTest.java
edward-swirldslabs Jan 8, 2025
fc7388f
fixes the unit test problems.
edward-swirldslabs Jan 8, 2025
940f735
Merge branch 'main' into 17261-restore-ServicesMainTest.java
edward-swirldslabs Jan 8, 2025
367ae25
squash the bug
edward-swirldslabs Jan 9, 2025
7c9e1fc
adding documenting comments.
edward-swirldslabs Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Cleanup Post Merge Of Develop
Signed-off-by: Edward Wertz <edward@swirldslabs.com>
  • Loading branch information
edward-swirldslabs committed Dec 17, 2024
commit 8944f2e212f59cff342d42753a01ec4283074c99
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@
import com.swirlds.state.merkle.MerkleStateRoot;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.time.InstantSource;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.ForkJoinPool;
Expand Down Expand Up @@ -235,7 +236,7 @@ public static void main(final String... args) throws Exception {
final var selfId = ensureSingleNode(nodesToRun, commandLineArgs.localNodesToStart());
final var platformConfig = buildPlatformConfig();
BootstrapUtils.setupConstructableRegistryWithConfiguration(platformConfig);
final var networkKeysAndCerts = initNodeSecurity(diskAddressBook, platformConfig);
final var networkKeysAndCerts = initNodeSecurity(diskAddressBook, platformConfig, new HashSet<>(nodesToRun));
final var keysAndCerts = networkKeysAndCerts.get(selfId);
setupGlobalMetrics(platformConfig);
metrics = getMetricsProvider().createPlatformMetrics(selfId);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import static com.swirlds.platform.test.fixtures.state.FakeMerkleStateLifecycles.FAKE_MERKLE_STATE_LIFECYCLES;
import static com.swirlds.platform.test.fixtures.state.FakeMerkleStateLifecycles.registerMerkleStateRootClassIds;

import com.hedera.hapi.node.state.roster.RosterEntry;
import com.swirlds.common.Console;
import com.swirlds.common.constructable.ClassConstructorPair;
import com.swirlds.common.constructable.ConstructableRegistry;
Expand All @@ -49,6 +50,7 @@
import java.awt.event.KeyEvent;
import java.awt.event.KeyListener;
import java.util.Random;
import java.util.stream.Collectors;

/**
* This demonstrates a cryptocurrency and a stock market. There are 10 stocks, and each member repeatedly generates an
Expand Down Expand Up @@ -163,7 +165,14 @@ public void init(final Platform platform, final NodeId id) {
this.platform = platform;
this.selfId = id;
final int winNum = (int) selfId.id();
this.console = createConsole(platform, winNum, true, platform.getAddressBook().getNodeIdSet()); // create the window, make it visible
this.console = createConsole(
platform,
winNum,
true,
platform.getRoster().rosterEntries().stream()
.map(RosterEntry::nodeId)
.map(NodeId::of)
.collect(Collectors.toSet())); // create the window, make it visible
this.console.addKeyListener(keyListener);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import static com.swirlds.platform.test.fixtures.state.FakeMerkleStateLifecycles.FAKE_MERKLE_STATE_LIFECYCLES;
import static com.swirlds.platform.test.fixtures.state.FakeMerkleStateLifecycles.registerMerkleStateRootClassIds;

import com.hedera.hapi.node.state.roster.RosterEntry;
import com.swirlds.common.Console;
import com.swirlds.common.constructable.ClassConstructorPair;
import com.swirlds.common.constructable.ConstructableRegistry;
Expand All @@ -48,6 +49,7 @@
import com.swirlds.platform.system.status.PlatformStatus;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.nio.charset.StandardCharsets;
import java.util.stream.Collectors;

/**
* This HelloSwirld creates a single transaction, consisting of the string "Hello Swirld", and then goes
Expand Down Expand Up @@ -102,7 +104,13 @@ public void init(final Platform platform, final NodeId id) {
this.selfId = id;
final int winNum = (int) selfId.id();
this.console = createConsole(
platform, winNum, true, platform.getAddressBook().getNodeIdSet()); // create the window, make it visible
platform,
winNum,
true,
platform.getRoster().rosterEntries().stream()
.map(RosterEntry::nodeId)
.map(NodeId::of)
.collect(Collectors.toSet())); // create the window, make it visible
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import static com.swirlds.platform.test.fixtures.state.FakeMerkleStateLifecycles.FAKE_MERKLE_STATE_LIFECYCLES;
import static com.swirlds.platform.test.fixtures.state.FakeMerkleStateLifecycles.registerMerkleStateRootClassIds;

import com.hedera.hapi.node.state.roster.RosterEntry;
import com.swirlds.common.Console;
import com.swirlds.common.constructable.ClassConstructorPair;
import com.swirlds.common.constructable.ConstructableRegistry;
Expand All @@ -57,6 +58,7 @@
import java.util.List;
import java.util.Locale;
import java.util.Random;
import java.util.stream.Collectors;

/**
* This demo collects statistics on the running of the network and consensus systems. It writes them to the
Expand Down Expand Up @@ -243,7 +245,13 @@ public void init(final Platform platform, final NodeId id) {
if (!headless) { // create the window, make it visible
final int winNum = (int) selfId.id();
console = createConsole(
platform, winNum, true, platform.getAddressBook().getNodeIdSet());
platform,
winNum,
true,
platform.getRoster().rosterEntries().stream()
.map(RosterEntry::nodeId)
.map(NodeId::of)
.collect(Collectors.toSet()));
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
Expand Down Expand Up @@ -217,7 +218,7 @@ private static void launchUnhandled(@NonNull final CommandLineArgs commandLineAr
BootstrapUtils.setupConfigBuilder(configBuilder, getAbsolutePath(DEFAULT_SETTINGS_FILE_NAME)));
final Configuration configuration = configBuilder.build();

initNodeSecurity(appDefinition.getConfigAddressBook(), configuration);
initNodeSecurity(appDefinition.getConfigAddressBook(), configuration, new HashSet<>(nodesToRun));
guiEventStorage = new GuiEventStorage(guiConfig, appDefinition.getConfigAddressBook());

guiSource = new StandardGuiSource(appDefinition.getConfigAddressBook(), guiEventStorage);
Expand Down Expand Up @@ -253,7 +254,8 @@ private static void launchUnhandled(@NonNull final CommandLineArgs commandLineAr
nodeId);
final var cryptography = CryptographyFactory.create();
CryptographyHolder.set(cryptography);
final KeysAndCerts keysAndCerts = initNodeSecurity(appDefinition.getConfigAddressBook(), configuration)
final KeysAndCerts keysAndCerts = initNodeSecurity(
appDefinition.getConfigAddressBook(), configuration, new HashSet<>(nodesToRun))
.get(nodeId);

// the AddressBook is not changed after this point, so we calculate the hash now
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,9 @@ private static String getPrivateKeysFileName(final String memberName) {
@NonNull
@Deprecated(since = "0.47.0", forRemoval = false)
static Map<NodeId, KeysAndCerts> loadKeysAndCerts(
@NonNull final AddressBook addressBook, @NonNull final Path keysDirPath, @NonNull final char[] password,
@NonNull final AddressBook addressBook,
@NonNull final Path keysDirPath,
@NonNull final char[] password,
@NonNull Set<NodeId> nodesToStart)
throws KeyStoreException, KeyLoadingException, UnrecoverableKeyException, NoSuchAlgorithmException,
KeyGeneratingException, NoSuchProviderException {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@
*
* <p>
* The {@link EnhancedKeyStoreLoader} class is a replacement for the now deprecated
* {@link CryptoStatic#loadKeysAndCerts(AddressBook, Path, char[])} method. This new implementation adds support for
* {@link CryptoStatic#loadKeysAndCerts(AddressBook, Path, char[], Set<NodeId>)} method. This new implementation adds support for
* loading industry standard PEM formatted PKCS #8 private keys and X.509 certificates. The legacy key stores are still
* supported, but are no longer the preferred format.
*
Expand Down Expand Up @@ -651,7 +651,7 @@ public PublicStores publicStores(@NonNull final AddressBook validatingBook)
.formatted(nodeId, nodeAlias, KeyCertPurpose.SIGNING));
}

if (isLocal(address)) {
if (localNodes.contains(nodeId)) {
// The agreement certificate is loaded by the local nodes and provided to peers through mTLS handshaking
logger.trace(
STARTUP.getMarker(),
Expand Down Expand Up @@ -1347,7 +1347,7 @@ private long extractPrivateKeysAndCertsFromPfxFiles(
final AtomicLong errorCount = new AtomicLong(0);

iterateAddressBook(addressBook, (i, nodeId, address, nodeAlias) -> {
if (isLocal(address)) {
if (localNodes.contains(nodeId)) {
// extract private keys for local nodes
final Path sPrivateKeyLocation = keyStoreDirectory.resolve("s-private-" + nodeAlias + ".pem");
final Path ksLocation = legacyPrivateKeyStore(nodeAlias);
Expand Down Expand Up @@ -1442,7 +1442,7 @@ private long validateKeysAndCertsAreLoadableFromPemFiles(
throws KeyStoreException, KeyLoadingException {
final AtomicLong errorCount = new AtomicLong(0);
iterateAddressBook(addressBook, (i, nodeId, address, nodeAlias) -> {
if (isLocal(address) && pfxCertificates.containsKey(nodeId)) {
if (localNodes.contains(nodeId) && pfxCertificates.containsKey(nodeId)) {
// validate private keys for local nodes
final Path ksLocation = privateKeyStore(nodeAlias, KeyCertPurpose.SIGNING);
final PrivateKey pemPrivateKey = readPrivateKey(nodeId, ksLocation);
Expand Down Expand Up @@ -1494,7 +1494,7 @@ private void rollBackSigningKeysAndCertsChanges(
final AtomicLong cleanupErrorCount = new AtomicLong(0);
iterateAddressBook(addressBook, (i, nodeId, address, nodeAlias) -> {
// private key rollback
if (isLocal(address) && pfxPrivateKeys.containsKey(address.getNodeId())) {
if (localNodes.contains(nodeId) && pfxPrivateKeys.containsKey(address.getNodeId())) {
try {
Files.deleteIfExists(privateKeyStore(nodeAlias, KeyCertPurpose.SIGNING));
} catch (final IOException e) {
Expand Down Expand Up @@ -1551,7 +1551,7 @@ private void cleanupByMovingPfxFilesToSubDirectory() throws KeyStoreException, K
}
}
iterateAddressBook(addressBook, (i, nodeId, address, nodeAlias) -> {
if (isLocal(address)) {
if (localNodes.contains(nodeId)) {
// move private key PFX files per local node
final File sPrivatePfx = legacyPrivateKeyStore(nodeAlias).toFile();
if (sPrivatePfx.exists()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,12 @@
import com.swirlds.common.platform.NodeId;
import com.swirlds.platform.gui.internal.SwirldMenu;
import com.swirlds.platform.roster.RosterUtils;
import com.swirlds.platform.state.address.AddressBookNetworkUtils;
import com.swirlds.platform.system.Platform;
import com.swirlds.platform.system.address.Address;
import com.swirlds.platform.system.address.AddressBook;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.awt.Color;
import java.awt.GraphicsEnvironment;
import java.awt.Rectangle;
import java.util.Set;
import javax.swing.JFrame;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,23 +29,16 @@
import com.swirlds.platform.gossip.sync.SyncOutputStream;
import com.swirlds.platform.network.Connection;
import com.swirlds.platform.network.ConnectionTracker;
import com.swirlds.platform.network.Network;
import com.swirlds.platform.network.NetworkUtils;
import com.swirlds.platform.network.SocketConfig;
import com.swirlds.platform.network.SocketConnection;
import com.swirlds.platform.network.connection.NotConnectedConnection;
import com.swirlds.platform.roster.RosterUtils;
import com.swirlds.platform.system.SoftwareVersion;
import com.swirlds.platform.system.address.Address;
import com.swirlds.platform.system.address.AddressBook;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.io.IOException;
import java.net.ConnectException;
import java.net.InetAddress;
import java.net.Socket;
import java.net.SocketException;
import java.net.SocketTimeoutException;
import java.net.UnknownHostException;
import java.util.Objects;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,6 @@
import com.swirlds.platform.health.filesystem.OSFileSystemChecker;
import com.swirlds.platform.network.Network;
import com.swirlds.platform.state.MerkleRoot;
import com.swirlds.platform.state.address.AddressBookNetworkUtils;
import com.swirlds.platform.network.SocketConfig;
import com.swirlds.platform.state.signed.SignedState;
import com.swirlds.platform.swirldapp.AppLoaderException;
import com.swirlds.platform.swirldapp.SwirldAppLoader;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,9 @@ public static RosterAndCerts loadAddressBookWithKeys(final int size)
NoSuchAlgorithmException, KeyGeneratingException, NoSuchProviderException {
final AddressBook createdAB = createAddressBook(size);
final Map<NodeId, KeysAndCerts> loadedC = EnhancedKeyStoreLoader.using(
createdAB, configure(ResourceLoader.getFile("preGeneratedPEMKeysAndCerts/")))
createdAB,
configure(ResourceLoader.getFile("preGeneratedPEMKeysAndCerts/")),
createdAB.getNodeIdSet())
.scan()
.generate()
.verify()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import java.security.KeyStoreException;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.stream.Stream;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
Expand Down Expand Up @@ -258,7 +259,8 @@ private AddressBook addressBook() {
void migraitonNegativeCaseTest(final String directoryName) throws IOException {
final Path keyDirectory = testDataDirectory.resolve(directoryName);
final AddressBook addressBook = addressBook();
final EnhancedKeyStoreLoader loader = EnhancedKeyStoreLoader.using(addressBook, configure(keyDirectory));
final EnhancedKeyStoreLoader loader =
EnhancedKeyStoreLoader.using(addressBook, configure(keyDirectory), addressBook.getNodeIdSet());

assertThat(keyDirectory).exists().isDirectory().isReadable().isNotEmptyDirectory();

Expand Down
Loading