Skip to content

Releases: NethermindEth/nethermind

v1.29.1

16 Oct 14:03
@kamilchodola kamilchodola
1.29.1
dfea524
Compare
Choose a tag to compare
v1.29.1 Latest
Latest

Release Notes

⚠️ This is a mandatory upgrade for all validators.

Major Highlights

Improved memory usage on Linux

In this version, we fixed a regression issue that caused Linux machines to slowly consume more memory than expected.
In 95% of situations, it did not cause any OOMs; in some specific situations, it restarted a node periodically.

Fixed and improved OP Stack sync

This version also addresses problems with the OP Mainnet sync and makes it faster by removing the snapshot download capability and improving snap sync performance. Now, OP Stack networks are syncing well.
In the meantime, we discovered a few potential further optimizations that will be added to an upcoming release.

Fixed an edge-case block production issue

On the Gnosis chain, we discovered an edge-case scenario in which an incorrect block could be built due to a bug in handling certain rare combinations of transactions. The issue has been fixed, and we can confirm that everything is working well now.

Full Changelog: 1.29.0...1.29.1

Assets 8
Loading

v1.29.0

07 Oct 09:12
@brbrr brbrr
1.29.0
10412d1
This commit was signed with the committer’s verified signature.
brbrr Yaroslav Kukharuk
SSH Key Fingerprint: ZZLrij6mhNrVeQyPUwE9x96IU2N37+ecvU2poXAAzAc Learn about vigilant mode.
Compare
Choose a tag to compare
v1.29.0

Major Highlights

Breaking changes

Configuration flag Discovery.Discv5Enabled is replaced with a more generic flag: Discovery.DiscoveryVersion with V4 as a default value. To enable v5 discovery, use the following: --Discovery.DiscoveryVersion=V5 .

Heuristic transaction censorship detection

We have introduced heuristics-based censorship detection in the Nethermind client. Censorship is detected using two kinds of heuristics, namely, high-paying transaction censorship and address censorship.

In the former, a block is marked potentially censored if the highest non-blob transaction included in the block is worse than the best non-blob transaction available in the pool. The latter, which could also be opted out of, requires specifying a list of addresses that we want to track censorship for. We count the number of tracked addresses to which transactions better than the worst non-blob transaction in the block is sent. If the number of such addresses in the block is less than half the number of these addresses in the pool, we mark the block as potentially censored. If two consecutive blocks are flagged as potentially censored, regardless of the kind, we detect censorship.

Detection of censorship sets the ShouldOverrideBuilder flag to true which acts as a circuit-breaker and communicates to the beacon node that it should fall back to local execution instead of relying on the builder's block.

Performance improvements

On Windows block validation performance used to degrade over several hours due to the default native allocator used by RocksDB. In #7418 we changed this to use the newer SegmentHeap that was introduced in Windows 10 and you can see there is now no throughput degradation over-time and Nethermind maintains a very high tip of chain processing speed.
Ethereum-throughput

Base and Ethereum target MGas/s chain speed are added for reference above.

With this change, as well as 25 other performance improvements, Nethermind maintains a throughput an order of magnitude higher than even Base’s current gaslimit on a gaming or high end workstation PC (with locally PCIe attached Samsung 990 Pro NVMes).

This is also reflected in the end to end new payload processing; receiving and parsing the Json request from the consensus client to producing the validated Json response per block on average in under 20ms.

Slot time for mainnet is 12000ms, and ideally without ePBS the EL should complete in under 4000ms; with ePBS it has the full slot to complete. So Nethermind; on this setup, even at max time is completing the block with 98% of headroom remaining.

image (2)

  • Reduced memory during Archive, Full Sync and head catchup
  • Introduced idle memory cleanup if waiting for new blocks from consensus layer; which can happen during head catchup, depending on consensus peering
  • Introducing a new Macro Op: EXTCODESIZE+POP which was a common pattern during the Shanghai DoS attacks; however the opcode pair combination can also be evaluated to a NOP not requiring to access the account or code for the address referred to.
  • Non-allocating key generation for In64 keys; this becomes impactful as these sync modes process more than 3000 blocks per second.
  • Calculate all the receipts’ blooms together in parallel
  • Fast wait-free and lockless Keccak cache; applied in a targeted manner (as most Keccaks are random), where the same address or UInt256 is keccak’d multiple times.
  • Further improvements to data prewarming for the transaction pipeline processing.

Changelog

New features

Bug fixes and stability

Optimizations:

Other changes

Read more

Contributors

rubo, Scooletz, and 14 other contributors
Assets 8
Loading

v1.28.0

15 Aug 13:38
@kamilchodola kamilchodola
1.28.0
9c4816c
Compare
Choose a tag to compare
v1.28.0

Release Notes

Major Highlights

eth_simulate

eth_simulate is a new JSON-RPC method that expands on eth_call ideas allowing simulation (mocking) of multiple blocks with multiple transactions. Also, it allows users to:

  • Define block variables
  • Replace code and balance of accounts
  • Override ecrecover precompile
  • Event logs and ETH transfer logs (traceTransfers = true/false)

See also eth_simulate in PEEPanEIP and ethereum/execution-apis#484.

Optimism adjustments and fixes

This release enhances Nethermind's support of Optimism with several key updates. JSON-RPC compatibility is improved, the API exposes Optimism-related fields in more places, and the OP transaction processor is now integrated into the trace namespace of JSON-RPC. Fork rules, RIP-7212, gas calculations, and receipt fields are aligned with Optimism's Fjord specifications.

Several fixes for block production, gas calculations, and other areas have been applied, along with numerous general performance improvements. The new discovery protocol (discv5) allows participation in the P2P network and quick node sync.

Additionally, Nethermind can now act as an Optimism Sequencer, increasing its utility within the ecosystem. These updates dramatically enhance Nethermind's reliability and functionality for Optimism.

To run a node, use existing configs like op-mainnet, base-mainnet, op-sepolia, base-sepolia, or assemble your own. Don't forget to enable discv5 and provide the sequencer URL as needed:

--Discovery.Discv5Enabled true --Optimism.SequencerUrl https://sepolia-sequencer.optimism.io

Gnosis and Chiado snap sync enabled by default

Starting from this release, the Gnosis and Chiado chains will be synced much faster from scratch thanks to the Nethermind snap serving feature, which has been active on Gnosis for the last few months. With plenty of nodes capable of serving these data, the team decided to switch on snap sync on both chains.

Below is a comparison of sync time before and after this change:

Gnosis Sync Time 1.27.1 1.28.0
Tip of chain 8 hours 20 minutes 25 minutes
Including historical data 13 hours 5 hours 45 minutes

Pruning.CacheMb values

We recently discovered that the Pruning.CacheMb option with values higher than the default ones is causing performance degradation.
This setting was commonly increased on the previous state DB schema (before v1.26.0) to slow down disk growth, but starting from the beforementioned version, this gives negligible improvements while causing significant performance degradation during in-memory pruning.
So, we recommend reducing it back to 1024 (max to 2048) if the validator connected to the node experiences some missed attestations.

Nethermind Launcher removal

Nethermind Launcher has been removed from distribution packages as obsolete. Nevertheless, it's still available to download from its repo.

Removal of unused plugins

In this version we removed several not used plugins and deprecated implementations to reduce complexity of code base.
Users which are using Nethermind package directly (not through Docker, PPA or other sources) will need to wipe the Nethermind directory except nethermind_db, logs and configs directories. Also any custom plugins in plugins directory should be kept - all others should be removed.

The error message user will face is:

 Failure System.TypeLoadException: Method 'Create' in type 'Nethermind.Mev.MevBlockProducerTransactionsExecutorFactory' from assembly 'Nethermind.Mev, Version=1.27.1.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.

Changelog

New features

  • Add eth_simulate , support array of eth_call for simulation across multiple blocks by @OlegJakushkin in #5530
  • Implement debug_standardTraceBadBlockToFile method by @rubo in #7278

Verkle

  • remove backfill from eip 2935 and serve hashes from state when eip 7709 enabled by @yerke26 in #7140

Performance

Gnosis

Optimism

Bug fixes and stability

Other changes

New Contributors

Read more

Contributors

rubo, flcl42, and 13 other contributors
Assets 8
Loading

v1.27.1

23 Jul 16:27
@kamilchodola kamilchodola
1.27.1
d2d5d4a
Compare
Choose a tag to compare
v1.27.1

Release notes

⚠️
This release is a mandatory upgrade for all nodes operating on the Gnosis chain.
Please update your node to this version to ensure correct node functionality.

This release fixes improperly enabled pre-warming blocks on Aura and OP chains. In a very rare situation, this could cause an issue in block processing.

Full Changelog: 1.27.0...1.27.1

Assets 8
Loading

v1.27.0

21 Jun 14:49
@cbermudez97 cbermudez97
1.27.0
220b5b8
Compare
Choose a tag to compare
v1.27.0

Release notes

Nethermind EVM Throughout MGas/s

Major Highlights

This release brings 159 performance improvements and critical bug fixes to the Nethermind client, significantly enhancing its efficiency and reliability. It is a continuation of big block processing improvements:

  • v1.25.4 to v1.26.0: Improved block processing by about 70 to 80% (from 68 to 107 MGas/s)
  • v1.26.0 to v1.27.0: Improved block processing by about 150% (from 107 to 254 MGas/s)

We are already working on the next improvements, increasing MGas/s throughput even more.

Intra-block cache

One of the key advancements in this release is the implementation of an intra-block cache. This feature optimizes the processing of transactions within blocks. By leveraging caching mechanisms, the system can avoid recalculating the state for transactions within the same block. This not only reduces redundant computations but also accelerates block execution, leading to a notable boost in overall performance.

Building on this, the intra-block cache has been further enhanced with a pre-warming technique during block execution. This optimization pre-loads necessary state data into the cache ahead of time, minimizing delays caused by cache misses during transaction processing. The result is a smoother and more efficient block processing experience, with reduced latency and improved throughput.

Nethermind Mainnet Validator Logs

Changelog

New features

Pectra

Verkle

Optimism

Performance

Bug fixes and stability

Other changes

  • Add a more secure way of estimat...
Read more

Contributors

rubo, flcl42, and 16 other contributors
Assets 8
Loading

v1.26.0

25 Apr 12:02
@kamilchodola kamilchodola
1.26.0
0068729
Compare
Choose a tag to compare
v1.26.0

Release notes

Major highlights

State design upgrade: Half-path

This version introduces a new approach to the state database: half-path. It replaces the previous approach, which we will call hash for these notes.

The goal of half-path is to improve the performance of the existing database without significant codebase changes. It is a middleware between the hash and full-path-based designs (which we are currently working on). It is mainly aimed at validators' performance (we observed a significant improvement in effectiveness) and, as a side effect, improving archive nodes' sync.

Major achievements

  • Block processing time is faster by about 30 to 50% (depending on the hardware setup), but we observed that it improves low-end setups more, so the cost of hosting nodes using Nethermind software may decrease.
  • The state database size is ~25% smaller (the previous approach is now close to 200 GB while the half-path is around 150 GB). This is only for the state database. After snap sync, the entire database will decrease by ~50 GB but still hover around 900 GB.
  • State database growth is significantly reduced—the slower the database grows, the less maintenance around the node is needed.
    We are still observing the exact numbers and applying some optimizations to make it as close as possible to "live pruning", but the database should grow about 10 to 20 times slower (to see the full effect, it may need 1 or 2 weeks—growth ratio will be dropping after that time).
  • Faster archive node sync. As half-path is not meant to optimize archive nodes, we observe that it is quicker than the previous implementation (8mln blocks processed about 30 to 40% faster, and the gap was only increasing—more tests around that are to be conducted).
  • Smaller archive node database. As stated above, the major goal of the half-path wasn't to make the archive node smaller, but what was observed was 2.4 TB vs 1.7 TB at the 8mln blocks checkpoint—and the gap was getting bigger. We are now checking the final numbers.

Making the migration to the half-path hassle-free

  • All newly synced nodes will use half-path as the default state design. If node operators want to use the previous approach, they can specify it with --Init.StateDbKeyScheme Hash.
  • All existing nodes that are synced with the old hash approach will remain on it.

Env variable equivalent of this flag is: NETHERMIND_INITCONFIG_STATEDBKEYSCHEME=HalfPath/Hash

Migration to half-path

  • Full resync. This is the simplest yet most effective way to migrate. As Nethermind syncs quickly back to the head, it may take just 1-2 hours to get back on track.
  • Full pruning. There is an option to migrate while staying online using the full pruning functionality. The downside of that approach is that pruning will take much longer to finish, and users will need to restart their nodes with --Init.StateDbKeyScheme HalfPath— without it, pruning will behave as before, so it will recreate the state with the previous database approach. To reduce slowdown, the user should increase the default full pruning memory budget to at least 8 GB: --Pruning.FullPruningMemoryBudgetMb 8000.

Upgrading from previous versions

  • From v1.25.4. The node will remain on the previous hash design. To migrate to half-path please see the migration instructions above. If the node operator does not want to migrate, it is completely fine to stay on the hash design, as it will still be fully supported.
  • From v1.26.0-exp. The node will remain on the half-path design. No resync is needed between v1.26.0 and its pre-release versions—all optimizations and bugfixes are automatically applied.

Learn more

More details about the half-path and other state redesigns which we are still exploring, can be found here. Please also refer to #6331, which contains all of the changes around the half-path and more in-depth details about specific performance metrics.

Snap serving

We're excited to share that Nethermind now supports snap serving, a capability previously unique to Geth. This is a big step for the Ethereum ecosystem, offering redundancy and easing Geth's considerable responsibility in maintaining network health.

This achievement doesn't diminish Geth's critical role but reinforces our shared goal of a resilient Ethereum. We see this as an opportunity to distribute the workload, ensuring the network's strength through diversity.

This feature was previously present but required a change to the database layout for efficient performance. Nodes running on the old hash approach will not enable snap serving by default. Snap serving's performance is too low on this design and may affect the node's overall performance. If a node operator wants to enable it on hash design, there is an option --Sync.SnapServingEnabled true, although we don't recommend it.

Important

During snap serving for other nodes, the overall performance of the node may be slightly worse. Please reach out to us on Discord with all observations so more optimizations can be discovered and applied. At the same time, it is advised to keep it enabled for the sake of network health.

Rootless and chiseled Docker image

This release introduces a new chiseled rootless Docker image for enhanced security.
These image tags have the -chiseled suffix and run the Nethermind process on behalf of the non-root app user with UID/GID of 64198.

Snappy dependency removal

Nethermind no longer needs the Snappy (libsnappy-dev) to be installed on Linux.

Deprecated options

--Sync.FastBlocks has been deprecated. It is advised to remove this option from your custom configs, environment variables, or flags passed to the client. See #6792 for more details.

Deprecated metrics

Multiple database and networking metrics are now deprecated and will be removed in the next release. These metrics follow the format:

  • nethermind_*_db_size
  • nethermind_*_db_reads
  • nethermind_*_db_writes
  • nethermind_*_disconnects
  • nethermind_*_received
  • nethermind_*_sent

They have been consolidated to labeled metrics:

  • nethermind_db_size{db=*}
  • nethermind_db_reads{db=*}
  • nethermind_db_writes{db=*}
  • nethermind_local_disconnects_total{reason=*}
  • nethermind_remote_disconnects_total{reason=*}
  • nethermind_incoming_p2p_messages{message=*}
  • nethermind_outgoing_p2p_messages{message=*}

Changelog

New features

Optimism

Performance

Bug fixes and stability

  • Fix/flush on snap fin...
Read more

Contributors

rubo, flcl42, and 21 other contributors
Assets 8
Loading

v1.25.4

16 Feb 18:35
@kamilchodola kamilchodola
1.25.4
20b10b3
Compare
Choose a tag to compare
v1.25.4

Release notes

⚠️

This release is a mandatory upgrade for all nodes operating on the Ethereum Mainnet and Gnosis.
Please update your node to this version to ensure correct node functionality.

Major highlights

Mainnet Dencun hard fork

The Mainnet Dencun hard fork is scheduled on Mar 13, 2024 at 13:55:35 UTC (epoch 269568)
⚠️ Execution client and consensus client database sizes can increase by about 150GB over time - please prepare for that!

Gnosis Dencun hard fork

The Gnosis Dencun hard fork is scheduled on Mar 11, 2024 at 18:30:20 UTC (slot 14237696)
⚠️ Execution client and consensus client database sizes can increase by about 150GB over time - please prepare for that!

PPA package

The PPA package has been revised, including the version and installers. Since the version has been fixed to be 1.25.4 instead of 1.2540, installing the latest version requires the manual removal of the previous one as follows:

# ⚠️ If your data directory is in the default location of /usr/share/nethermind, back it up before the package removal.

sudo apt-get purge -y nethermind
sudo apt-get update

Changelog

Cancun

Optimism

  • Add temporary solution for pre-bedrock tx decoding by adding AncientBarriers by @deffrian in #6620

Bugfixes and new features

  • Reduce default peers amount from 100 to 50 on mainnet nodes (not on archive nodes) to reduce a load on it and reduce amount of block processing spikes in #6743
  • Reduce amount of hanging sockets with TIME_WAIT status by @benaadams in #6594
  • Fix ParityTraceActionConverter to output init on create by @LukaszRozmej in #6606
  • Fix TraceStore plugin by @LukaszRozmej in #6609
  • Return BlockForRpc from debug_getBadBlocks by @flcl42 in #6612
  • Revise PPA packaging by @rubo in #6618
  • Improve memory management and reduce traffic while serving data to other peers by @benaadams in #6636
  • Add debug_getBadBlocks support by @kjazgar and @Marchhill in #3838
  • Add Metrics.ExposeHost option so a hostname other than "0.0.0.0" can be exposed by @tgerring in #6528

    ⚠️
    For now we decided to keep default behavior so ExposeHost will default to "+" which means "0.0.0.0" but in the future we are going to change default of that flag to "127.0.0.1" to reduce risk of remote access to metrics.

Full Changelog: 1.25.3...1.25.4

Contributors

rubo, flcl42, and 7 other contributors
Assets 8
Loading

v1.25.3

22 Jan 18:03
@kamilchodola kamilchodola
1.25.3
e64bee7
Compare
Choose a tag to compare
v1.25.3

Release notes

⚠️

This release is a mandatory upgrade for all nodes operating on the following chains: Sepolia, Holesky, and Chiado.
Please update your node to this version to ensure correct node functionality.

Major highlights

Sepolia Dencun hard fork

The Sepolia Dencun hard fork is scheduled on Jan 30, 2024 at 22:51:12 UTC (epoch 132608)

Chiado Dencun hard fork

The Chiado Dencun hard fork is scheduled on Jan 31, 2024 at 18:15:40 UTC (epoch 516608)

Holesky Dencun hard fork

The Holesky Dencun hard fork is scheduled on Feb 07, 2024 at 11:34:24 UTC (epoch 29696)

Full Changelog: 1.25.2...1.25.3

Assets 8
Loading

v1.25.2

21 Jan 22:00
@kamilchodola kamilchodola
1.25.2
78c7bf5
Compare
Choose a tag to compare
v1.25.2

Release notes

This hotfix addresses the consensus issue in Nethermind that was introduced in v1.23.0. This release is mandatory for all Nethermind users.

No resync required. A consensus client restart is required without the need to resync it.
The v1.22.0 and below weren't affected.

A full postmortem will be published soon.

Assets 8
Loading

v1.25.1

16 Jan 18:35
@kamilchodola kamilchodola
1.25.1
e37c10d
Compare
Choose a tag to compare
v1.25.1

⛔ THIS VERSION HAS A CONSENSUS ISSUE. Use v1.25.2 or later instead.

Release notes

Major highlights

Goerli Dencun hard fork

⚠️
This version supports the upcoming Goerli Dencun hard fork that is scheduled on Jan 17, 2024 at 06:32:00 UTC (epoch 231680)
Please update your node to this version to ensure correct node functionality.

Chiado Dencun hard fork

⚠️
This version supports the upcoming Chiado Dencun hard fork that is scheduled on Jan 31, 2024 at 18:15:40 UTC (epoch 516608)
Please update your node to this version to ensure correct node functionality.

Fixed eth_syncing invalid behavior

In v1.25.0, the eth_syncing method misbehaves for those who upgraded from the older version to the latest one (returned "syncing" when a node was fully synced). This version addresses this issue, so it properly returns the sync status.

Fixed extra bodies and receipts downloaded for the Mainnet after upgrading the long-living node

In v1.25.0, in some cases, the old bodies and old receipts started to download even though it should not have been the case on the synced nodes. This usually happens for older nodes that were synced a long time ago without any resync in the meantime.

Full Changelog: 1.25.0...1.25.1

Assets 8
Loading