Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Small fixes for the database #2015

Merged
merged 3 commits into from
Jul 5, 2024
Merged

Small fixes for the database #2015

merged 3 commits into from
Jul 5, 2024

Conversation

xgreenx
Copy link
Collaborator

@xgreenx xgreenx commented Jul 5, 2024

  • Fixed the name for historical columns - Metrics was working incorrectly for historical columns.
  • Added recommended setting for the RocksDB - the source of recommendation is official documentation https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning#other-general-options.
  • Removed repairing since it could corrupt the database if fails - several users reported about the corrupted state of the database after having a "Too many descriptors" error where in logs, repairing of the database also failed with this error creating a lost folder.

Before requesting review

  • I have reviewed the code myself

@xgreenx
Fixed the name for historical columns.
964e594 
Added recommended setting for the RocksDB.
Removed repairing since it could corrupt the database if fails.
@xgreenx xgreenx requested a review from a team July 5, 2024 19:36
@xgreenx xgreenx self-assigned this Jul 5, 2024
MitchTurner
MitchTurner approved these changes Jul 5, 2024
View reviewed changes
Copy link
Member

@MitchTurner MitchTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I'm not sure why we weren't using the inner value from the HistoricalDuplicateColumn enum before, this makes more sense. Was that just a mistake or are we reevaluating intended behavior here?

@xgreenx
Copy link
Collaborator Author

xgreenx commented Jul 5, 2024

It was a mistake=D

@xgreenx xgreenx enabled auto-merge (squash) July 5, 2024 19:49
@xgreenx xgreenx merged commit 4f6f0d3 into master Jul 5, 2024
33 checks passed
@xgreenx xgreenx deleted the feature/small-improvments-to-the-database branch July 5, 2024 20:00
@xgreenx xgreenx mentioned this pull request Jul 5, 2024
Merged
xgreenx added a commit that referenced this pull request Jul 5, 2024
@xgreenx
Release v0.31.0 (#2016)
2faae02 
## Version v0.31.0

### Added
- [#2014](#2014): Added a
separate thread for the block importer.
- [#2013](#2013): Added a
separate thread to process P2P database lookups.
- [#2004](#2004): Added new
CLI argument `continue-services-on-error` to control internal flow of
services.
- [#2004](#2004): Added
handling of incorrect shutdown of the off-chain GraphQL worker by using
state rewind feature.
- [#2007](#2007): Improved
metrics:
  - Added database metrics per column.
  - Added statistic about commit time of each database.
- Refactored how metrics are registered: Now, we use only one register
shared between all metrics. This global register is used to encode all
metrics.
- [#1996](#1996): Added
support for rollback command when state rewind feature is enabled. The
command allows the rollback of the state of the blockchain several
blocks behind until the end of the historical window. The default
historical window it 7 days.
- [#1996](#1996): Added
support for the state rewind feature. The feature allows the execution
of the blocks in the past and the same execution results to be received.
Together with forkless upgrades, execution of any block from the past is
possible if historical data exist for the target block height.
- [#1994](#1994): Added the
actual implementation for the `AtomicView::latest_view`.
- [#1972](#1972): Implement
`AlgorithmUpdater` for `GasPriceService`
- [#1948](#1948): Add new
`AlgorithmV1` and `AlgorithmUpdaterV1` for the gas price. Include tools
for analysis
- [#1676](#1676): Added new
CLI arguments:
    - `graphql-max-depth`
    - `graphql-max-complexity`
    - `graphql-max-recursive-depth`

### Changed
- [#2015](#2015): Small fixes
for the database:
- Fixed the name for historical columns - Metrics was working
incorrectly for historical columns.
- Added recommended setting for the RocksDB - The source of
recommendation is official documentation
https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning#other-general-options.
- Removed repairing since it could corrupt the database if fails -
Several users reported about the corrupted state of the database after
having a "Too many descriptors" error where in logs, repairing of the
database also failed with this error creating a `lost` folder.
- [#2010](#2010): Updated the
block importer to allow more blocks to be in the queue. It improves
synchronization speed and mitigate the impact of other services on
synchronization speed.
- [#2006](#2006): Process
block importer events first under P2P pressure.
- [#2002](#2002): Adapted the
block producer to react to checked transactions that were using another
version of consensus parameters during validation in the TxPool. After
an upgrade of the consensus parameters of the network, TxPool could
store invalid `Checked` transactions. This change fixes that by tracking
the version that was used to validate the transactions.
- [#1999](#1999): Minimize the
number of panics in the codebase.
- [#1990](#1990): Use latest
view for mutate GraphQL queries after modification of the node.
- [#1992](#1992): Parse
multiple relayer contracts, `RELAYER-V2-LISTENING-CONTRACTS` env
variable using a `,` delimiter.
- [#1980](#1980): Add
`Transaction` to relayer 's event filter

#### Breaking
- [#2012](#2012): Bumped the
`fuel-vm` to `0.55.0` release. More about the change
[here](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.55.0).
- [#2001](#2001): Prevent
GraphQL query body to be huge and cause OOM. The default body size is
`1MB`. The limit can be changed by the
`graphql-request-body-bytes-limit` CLI argument.
- [#1991](#1991): Prepare the
database to use different types than `Database` for atomic view.
- [#1989](#1989): Extract
`HistoricalView` trait from the `AtomicView`.
- [#1676](#1676): New
`fuel-core-client` is incompatible with the old `fuel-core` because of
two requested new fields.
- [#1676](#1676): Changed
default value for `api-request-timeout` to be `30s`.
- [#1676](#1676): Now, GraphQL
API has complexity and depth limitations on the queries. The default
complexity limit is `20000`. It is ~50 blocks per request with
transaction IDs and ~2-5 full blocks.

### Fixed
- [#2000](#2000): Use correct
query name in metrics for aliased queries.

## What's Changed
* Generate and publish code coverage reports in the CI by @Dentosal in
#1947
* Gas Price Algorithm by @MitchTurner in
#1948
* Use companies fork of the `publish-crates` action by @xgreenx in
#1986
* Weekly `cargo update` by @github-actions in
#1985
* Implement gas price updater for service by @MitchTurner in
#1972
* Extract `HistoricalView` trait from the `AtomicView` by @xgreenx in
#1989
* Use fresh `ReadView` for mutate queries by @xgreenx in
#1990
* Prevent api spam with GQL complexity limits by @Voxelot in
#1676
* Enable parsing multiple relayer listening contract addresses from
environment variables by @Jurshsmith in
#1992
* Prepare the database to use different types than `Database` for atomic
view by @xgreenx in #1991
* Added the actual implementation for the `AtomicView::latest_view` by
@xgreenx in #1994
* Weekly `cargo update` by @github-actions in
#1998
* Minimize the number of panics in the codebase by @xgreenx in
#1999
* feat: include Transaction events in topic0 filter for download_logs by
@DefiCake in #1980
* Use correct query name for metrics by @xgreenx in
#2000
* Prevent GraphQL query body to be huge and cause OOM by @xgreenx in
#2001
* Adapted the block producer to react on the outdated transactions from
the TxPool by @xgreenx in
#2002
* Process block importer events first under P2P pressure by @xgreenx in
#2006
* Implementation of the state rewind feature for the RocksDB by @xgreenx
in #1996
* Upgraded `fuel-vm` to `0.55.0` by @xgreenx in
#2012
* Improved metrics for the database by @xgreenx in
#2007
* Updated block importer to allow more blocks to be queue by @xgreenx in
#2010
* Added handling of incorrect shutdown of the off-chain GraphQL worker
by @xgreenx in #2004
* Moved P2P database lookups into a separate thread by @xgreenx in
#2013
* Use dedicated thread for the block importer by @xgreenx in
#2014
* Small fixes for the database by @xgreenx in
#2015

## New Contributors
* @Jurshsmith made their first contribution in
#1992
* @DefiCake made their first contribution in
#1980

**Full Changelog**:
v0.30.0...v0.31.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MitchTurner MitchTurner MitchTurner approved these changes

Assignees

@xgreenx xgreenx

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@xgreenx @MitchTurner