Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Adds GPU Mining #14

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions GPU-MINING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Besu GPU Mining

## Mining Software

EthMiner is recommended. You can [download EthMiner from Github](https://github.com/ethereum-mining/ethminer), available for Windows, Linux, and MacOS.

## GPU Mining with Besu

To mine with EthMiner, start Besu with HTTP RPC enabled, setting cors to the IP of the miners you want mining from Besu.

If running in a development environment, the starting flags would be the following:

```
./bin/besu --network=dev --gpu-mining-enabled --rpc-http-enabled --rpc-http-cors-origins=all
```

This starts Besu in a developer network with JSON RPC available at localhost:8545, with cors set to all IPs.

Once initiated, you must wait until work is available before starting EthMiner. Once work is availabe, start EthMiner pointing to the JSON RPC endpoint. From the directory containing EthMiner:

```
<EthMiner> -P http://<Your Ethereum Address>@<Host>:<Port>
```

An example on Windows would look like the following:

```
ethminer.exe -P http://0xBBAac64b4E4499aa40DB238FaA8Ac00BAc50811B@127.0.0.1:8545
```

Other computers can mine to this Besu node if cors allows it, by setting the host to the IP of the computer/server running Besu, or the hostname of the server running Besu.

You can test when work is available using CURL in a command line:

```
curl -X POST --data "{"jsonrpc":"2.0","method":"eth_getWork","params":[],"id":73}" localhost:8545
```

From Windows Command Prompt, qoutes are special characters.

```
curl -X POST --data "{\"jsonrpc\":\"2.0\",\"method\":\"eth_getWork\",\"params\":[],\"id\":73}" localhost:8545
```

If work is available for mining, you should see a response similar to the following:

```
{"jsonrpc":"2.0","id":73,"result":["0xc9a815cfff9186b3b1ebb286ff71156b2605c4ca45e2413418c8054688011706","0x0000000000000000000000000000000000000000000000000000000000000000","0x028f5c28f5c28f5c28f5c28f5c28f5c28f5c28f5c28f5c28f5c28f5c28f5c28f"]}
```

If EthMiner disappears after building or downloading, check with your anti-virus software, allow EthMiner to run, and redownload.
68 changes: 51 additions & 17 deletions besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
Original file line number Diff line number Diff line change
Expand Up @@ -544,6 +544,11 @@ void setBannedNodeIds(final List<String> values) {
description = "Set if node will perform mining (default: ${DEFAULT-VALUE})")
private final Boolean isMiningEnabled = false;

@Option(
names = {"--gpu-mining-enabled"},
description = "Set if node will perform mining (default: ${DEFAULT-VALUE})")
private final Boolean isGpuMiningEnabled = false;

@Option(
names = {"--miner-coinbase"},
description =
Expand Down Expand Up @@ -958,6 +963,14 @@ private void issueOptionWarnings() {
!isMiningEnabled,
asList("--miner-coinbase", "--min-gas-price", "--miner-extra-data"));

// Check that GPU Mining Software can access Besu
CommandLineUtils.checkOptionDependencies(
logger,
commandLine,
"--gpu-mining-enabled",
!isMiningEnabled,
asList("--rpc-http-cors-origins", "--rpc-http-enabled"));

CommandLineUtils.checkOptionDependencies(
logger,
commandLine,
Expand Down Expand Up @@ -1031,23 +1044,44 @@ public BesuController<?> buildController() {
public BesuControllerBuilder<?> getControllerBuilder() {
try {
addConfigurationService();
return controllerBuilderFactory
.fromEthNetworkConfig(updateNetworkConfig(getNetwork()), genesisConfigOverrides)
.synchronizerConfiguration(buildSyncConfig())
.ethProtocolConfiguration(ethProtocolOptions.toDomainObject())
.dataDirectory(dataDir())
.miningParameters(
new MiningParameters(coinbase, minTransactionGasPrice, extraData, isMiningEnabled))
.transactionPoolConfiguration(buildTransactionPoolConfiguration())
.nodePrivateKeyFile(nodePrivateKeyFile())
.metricsSystem(metricsSystem.get())
.privacyParameters(privacyParameters())
.clock(Clock.systemUTC())
.isRevertReasonEnabled(isRevertReasonEnabled)
.storageProvider(keyStorageProvider(keyValueStorageName))
.isPruningEnabled(isPruningEnabled)
.pruningConfiguration(buildPruningConfiguration())
.genesisConfigOverrides(genesisConfigOverrides);

if (isGpuMiningEnabled) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not need to be an if block with two clauses. Used a ?: statement, but the odds are it won't be needed as mining will still need a coinbase and not an address of zero.

return controllerBuilderFactory
.fromEthNetworkConfig(updateNetworkConfig(getNetwork()), genesisConfigOverrides)
.synchronizerConfiguration(buildSyncConfig())
.ethProtocolConfiguration(ethProtocolOptions.toDomainObject())
.dataDirectory(dataDir())
.miningParameters(
new MiningParameters(Address.ZERO, minTransactionGasPrice, extraData, isGpuMiningEnabled, isGpuMiningEnabled))
.transactionPoolConfiguration(buildTransactionPoolConfiguration())
.nodePrivateKeyFile(nodePrivateKeyFile())
.metricsSystem(metricsSystem.get())
.privacyParameters(privacyParameters())
.clock(Clock.systemUTC())
.isRevertReasonEnabled(isRevertReasonEnabled)
.storageProvider(keyStorageProvider(keyValueStorageName))
.isPruningEnabled(isPruningEnabled)
.pruningConfiguration(buildPruningConfiguration())
.genesisConfigOverrides(genesisConfigOverrides);
} else {
return controllerBuilderFactory
.fromEthNetworkConfig(updateNetworkConfig(getNetwork()), genesisConfigOverrides)
.synchronizerConfiguration(buildSyncConfig())
.ethProtocolConfiguration(ethProtocolOptions.toDomainObject())
.dataDirectory(dataDir())
.miningParameters(
new MiningParameters(coinbase, minTransactionGasPrice, extraData, isMiningEnabled, isGpuMiningEnabled))
.transactionPoolConfiguration(buildTransactionPoolConfiguration())
.nodePrivateKeyFile(nodePrivateKeyFile())
.metricsSystem(metricsSystem.get())
.privacyParameters(privacyParameters())
.clock(Clock.systemUTC())
.isRevertReasonEnabled(isRevertReasonEnabled)
.storageProvider(keyStorageProvider(keyValueStorageName))
.isPruningEnabled(isPruningEnabled)
.pruningConfiguration(buildPruningConfiguration())
.genesisConfigOverrides(genesisConfigOverrides);
}
} catch (final IOException e) {
throw new ExecutionException(this.commandLine, "Invalid path", e);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ private MiningParameters getMiningParameters() {
// Extradata and coinbase can be configured on a per-block level via the json file
final Address coinbase = Address.ZERO;
final BytesValue extraData = BytesValue.EMPTY;
return new MiningParameters(coinbase, minTransactionGasPrice, extraData, false);
return new MiningParameters(coinbase, minTransactionGasPrice, extraData, false, false);
}

private <T> void importJsonBlocks(final BesuController<T> controller, final Path path)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,8 @@ protected MiningCoordinator createMiningCoordinator(
new DefaultBlockScheduler(
MainnetBlockHeaderValidator.MINIMUM_SECONDS_SINCE_PARENT,
MainnetBlockHeaderValidator.TIMESTAMP_TOLERANCE_S,
clock));
clock),
miningParameters.isGpuMiningEnabled());

final EthHashMiningCoordinator miningCoordinator =
new EthHashMiningCoordinator(protocolContext.getBlockchain(), executor, syncState);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,7 @@ public void envVariableOverridesValueFromConfigFile() {
Address.fromHexString(expectedCoinbase),
DefaultCommandValues.DEFAULT_MIN_TRANSACTION_GAS_PRICE,
DefaultCommandValues.DEFAULT_EXTRA_DATA,
false,
false));
}

Expand All @@ -777,6 +778,7 @@ public void cliOptionOverridesEnvVariableAndConfig() {
Address.fromHexString(expectedCoinbase),
DefaultCommandValues.DEFAULT_MIN_TRANSACTION_GAS_PRICE,
DefaultCommandValues.DEFAULT_EXTRA_DATA,
false,
false));
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ public void extraDataCreatedOnEpochBlocksContainsValidators() {
TestClock.fixed(),
metricsSystem),
proposerKeyPair,
new MiningParameters(AddressHelpers.ofValue(1), Wei.ZERO, vanityData, false),
new MiningParameters(AddressHelpers.ofValue(1), Wei.ZERO, vanityData, false, false),
mock(CliqueBlockScheduler.class),
new EpochManager(EPOCH_LENGTH));

Expand Down Expand Up @@ -133,7 +133,7 @@ public void extraDataForNonEpochBlocksDoesNotContainValidaors() {
TestClock.fixed(),
metricsSystem),
proposerKeyPair,
new MiningParameters(AddressHelpers.ofValue(1), Wei.ZERO, vanityData, false),
new MiningParameters(AddressHelpers.ofValue(1), Wei.ZERO, vanityData, false, false),
mock(CliqueBlockScheduler.class),
new EpochManager(EPOCH_LENGTH));

Expand Down Expand Up @@ -170,7 +170,7 @@ public void shouldUseLatestVanityData() {
TestClock.fixed(),
metricsSystem),
proposerKeyPair,
new MiningParameters(AddressHelpers.ofValue(1), Wei.ZERO, initialVanityData, false),
new MiningParameters(AddressHelpers.ofValue(1), Wei.ZERO, initialVanityData, false, false),
mock(CliqueBlockScheduler.class),
new EpochManager(EPOCH_LENGTH));

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,8 @@ private static ControllerAndState createControllerAndFinalState(
AddressHelpers.ofValue(1),
Wei.ZERO,
BytesValue.wrap("Ibft Int tests".getBytes(UTF_8)),
true);
true,
false);

final StubGenesisConfigOptions genesisConfigOptions = new StubGenesisConfigOptions();
genesisConfigOptions.byzantiumBlock(0);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ private String serialise(final JsonRpcResponse response) {
return EMPTY_RESPONSE;
}

return Json.encodePrettily(response);
return Json.encode(response);
}

@SuppressWarnings("rawtypes")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthProtocolVersion;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthSendRawTransaction;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthSendTransaction;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthSubmitLogin;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthSyncing;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.EthUninstallFilter;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.JsonRpcMethod;
Expand Down Expand Up @@ -249,6 +250,7 @@ public Map<String, JsonRpcMethod> methods(
new EthGetStorageAt(blockchainQueries, parameter),
new EthSendRawTransaction(transactionPool, parameter),
new EthSendTransaction(),
new EthSubmitLogin(),
new EthEstimateGas(
blockchainQueries,
new TransactionSimulator(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ public enum RpcMethod {
ETH_PROTOCOL_VERSION("eth_protocolVersion"),
ETH_SEND_RAW_TRANSACTION("eth_sendRawTransaction"),
ETH_SEND_TRANSACTION("eth_sendTransaction"),
ETH_SUBMIT_LOGIN("eth_submitLogin"),
ETH_SUBSCRIBE("eth_subscribe"),
ETH_SYNCING("eth_syncing"),
ETH_UNINSTALL_FILTER("eth_uninstallFilter"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ public class EthGetWork implements JsonRpcMethod {

private final MiningCoordinator miner;
private static final Logger LOG = getLogger();
private Boolean solverWasPresent = false;
private String[] storedResult;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't store the stored result in the RPC method, query it from the mining co-ordinator.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mining coordinator doesn't store previous results. In-between results, there would be cases where miner.getWorkDefinition().isPresent() would return false, returning JsonRpcError.NO_MINING_WORK_FOUND. This would cause EthMiner connections to abort upon connecting for a few seconds.


public EthGetWork(final MiningCoordinator miner) {
this.miner = miner;
Expand All @@ -54,7 +56,11 @@ public JsonRpcResponse response(final JsonRpcRequest req) {
"0x" + BaseEncoding.base16().lowerCase().encode(dagSeed),
rawResult.getTarget().toHexString()
};
solverWasPresent = true;
storedResult = result;
return new JsonRpcSuccessResponse(req.getId(), result);
} else if (solverWasPresent) {
return new JsonRpcSuccessResponse(req.getId(), storedResult);
} else {
LOG.trace("Mining is not operational, eth_getWork request cannot be processed");
return new JsonRpcErrorResponse(req.getId(), JsonRpcError.NO_MINING_WORK_FOUND);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright 2018 ConsenSys AG.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods;

import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequest;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse;

public class EthSubmitLogin implements JsonRpcMethod {

@Override
public String getName() {
return RpcMethod.ETH_SUBMIT_LOGIN.getMethodName();
}

@Override
public JsonRpcResponse response(final JsonRpcRequest req) {
// Confirm login request
boolean result = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just a stub or should there be more logic here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mining pools use this route to add a miner to their database for purposes such as withholding payouts until a certain threshold is met, collecting fees, and notifying users by email when a miner becomes inactive, while notifying mining software that connection has been established. Here, we're simply accepting all mining software connection requests, notifying mining software connection has been established.

return new JsonRpcSuccessResponse(req.getId(), result);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ public class BlockMiner<C, M extends AbstractBlockCreator<C>> implements Runnabl
private final ProtocolSchedule<C> protocolSchedule;
private final Subscribers<MinedBlockObserver> observers;
private final AbstractBlockScheduler scheduler;
private Boolean gpuMining = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't call it GPU mining, call it externalMining, here and throughout.


public BlockMiner(
final Function<BlockHeader, M> blockCreatorFactory,
Expand Down Expand Up @@ -109,6 +110,11 @@ public Block createBlock(
return blockCreator.createBlock(transactions, ommers, timestamp);
}

public void setGpuMining() {
gpuMining = true;
return;
}

protected boolean mineBlock() throws InterruptedException {
// Ensure the block is allowed to be mined - i.e. the timestamp on the new block is sufficiently
// ahead of the parent, and still within allowable clock tolerance.
Expand All @@ -123,28 +129,41 @@ protected boolean mineBlock() throws InterruptedException {
"Block created, importing to local chain, block includes {} transactions",
block.getBody().getTransactions().size());

final BlockImporter<C> importer =
protocolSchedule.getByBlockNumber(block.getHeader().getNumber()).getBlockImporter();
final boolean blockImported =
importer.importBlock(protocolContext, block, HeaderValidationMode.FULL);
if (blockImported) {
notifyNewBlockListeners(block);
final double taskTimeInSec = stopwatch.elapsed(TimeUnit.MILLISECONDS) / 1000.0;
if (gpuMining) {
LOG.info(
String.format(
"Produced and imported block #%,d / %d tx / %d om / %,d (%01.1f%%) gas / (%s) in %01.3fs",
block.getHeader().getNumber(),
block.getBody().getTransactions().size(),
block.getBody().getOmmers().size(),
block.getHeader().getGasUsed(),
(block.getHeader().getGasUsed() * 100.0) / block.getHeader().getGasLimit(),
block.getHash(),
taskTimeInSec));
String.format(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is formatted incorrectly. ./gradlew spotlessApply will fix all formatting errors.

"Produced block #%,d / %d tx / %d om / %,d (%01.1f%%) gas / (%s) ",
block.getHeader().getNumber(),
block.getBody().getTransactions().size(),
block.getBody().getOmmers().size(),
block.getHeader().getGasUsed(),
(block.getHeader().getGasUsed() * 100.0) / block.getHeader().getGasLimit(),
block.getHash()));
return false;
} else {
LOG.error("Illegal block mined, could not be imported to local chain.");
}
final BlockImporter<C> importer =
protocolSchedule.getByBlockNumber(block.getHeader().getNumber()).getBlockImporter();
final boolean blockImported =
importer.importBlock(protocolContext, block, HeaderValidationMode.FULL);
if (blockImported) {
notifyNewBlockListeners(block);
final double taskTimeInSec = stopwatch.elapsed(TimeUnit.MILLISECONDS) / 1000.0;
LOG.info(
String.format(
"Produced and imported block #%,d / %d tx / %d om / %,d (%01.1f%%) gas / (%s) in %01.3fs",
block.getHeader().getNumber(),
block.getBody().getTransactions().size(),
block.getBody().getOmmers().size(),
block.getHeader().getGasUsed(),
(block.getHeader().getGasUsed() * 100.0) / block.getHeader().getGasLimit(),
block.getHash(),
taskTimeInSec));
} else {
LOG.error("Illegal block mined, could not be imported to local chain.");
}

return blockImported;
return blockImported;
}
}

public void cancel() {
Expand Down
Loading