Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: stmgr: don't attempt to lookup genesis state #10472

Merged
merged 2 commits into from
Mar 14, 2023

Conversation

magik6k
Copy link
Contributor

@magik6k magik6k commented Mar 14, 2023

Related Issues

Fixes a regression introduced in #10445

Proposed Changes

Don't attempt to get genesis state by walking back to it.

Additional Info

I've noticed that with latest master my mainnet nodes are taking a really long time to start syncing after startup. One splitstore node took 2 hours (!) to begin the process.

This appears to have been caused by trying to get genesis state for the vesting schedule:

goroutine 13001085 [semacquire]:
sync.runtime_Semacquire(0xc0164e8680?)
	/usr/lib/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc015ed0501?)
	/usr/lib/go/src/sync/waitgroup.go:139 +0x52
golang.org/x/sync/errgroup.(*Group).Wait(0xc01bee3e40)
	/home/magik6k/.opt/go/pkg/mod/golang.org/x/sync@v0.0.0-20220907140024-f12130a52804/errgroup/errgroup.go:53 +0x27
github.com/filecoin-project/lotus/chain/store.(*ChainStore).LoadTipSet(0xc000672240, {0x64bafa0?, 0xc005b39da0}, {{0xc046797ef0?, 0xe4?}})
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/store/store.go:842 +0x125
github.com/filecoin-project/lotus/chain/store.(*ChainIndex).walkBack(0xc0339ca390, {0x64bafa0, 0xc005b39da0}, 0xc000616720?, 0x25fbac)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/store/index.go:165 +0xed
github.com/filecoin-project/lotus/chain/store.(*ChainIndex).fillCache(0xc0339ca390, {0x64bafa0, 0xc005b39da0}, {{0xc045471ec0?, 0x96b8cbe4fcef58c4?}})
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/store/index.go:120 +0x178
github.com/filecoin-project/lotus/chain/store.(*ChainIndex).GetTipsetByHeight(0xc0339ca390, {0x64bafa0, 0xc005b39da0}, 0xc015ed0898?, 0x1)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/store/index.go:68 +0x113
github.com/filecoin-project/lotus/chain/store.(*ChainStore).GetTipsetByHeight(0xc000672240, {0x64bafa0, 0xc005b39da0}, 0x1, 0x7dcd86?, 0x0)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/store/store.go:1165 +0xac
github.com/filecoin-project/lotus/chain/stmgr.tryLookupTipsetState({0x64bafa0, 0xc005b39da0}, 0xc000672240, 0xc0194aa700)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/execute.go:77 +0x6c
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).TipSetState(0xc00093bd40, {0x64bafa0?, 0xc013733710?}, 0xc0194aa700)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/execute.go:58 +0x6e5
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).setupGenesisVestingSchedule(0xc00093bd40, {0x64bafa0, 0xc013733710})
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/supply.go:43 +0x155
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).GetFilVested(0xc00093bd40, {0x64bafa0, 0xc013733710}, 0x28ec90)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/supply.go:206 +0x15d
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).GetVMCirculatingSupplyDetailed(0x0?, {0x64bafa0, 0xc013733710}, 0x28ec90, 0x26?)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/supply.go:342 +0x65
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).GetVMCirculatingSupply(0xc015ed10f0?, {0x64bafa0?, 0xc013733710?}, 0x46956c0?, 0xc049ebe701?)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/supply.go:333 +0x3b
github.com/filecoin-project/lotus/chain/vm.defaultFVMOpts({0x64bafa0, 0xc013733710}, 0xc019b21c80)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/vm/fvm.go:250 +0x163
github.com/filecoin-project/lotus/chain/vm.NewFVM({0x64bafa0?, 0xc013733710?}, 0xc019b21c80)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/vm/fvm.go:279 +0x2c
github.com/filecoin-project/lotus/chain/vm.NewVM({0x64bafa0?, 0xc013733710?}, 0x9dbb001?)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/vm/vmi.go:41 +0x45
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).VMConstructor.func1({0x64bafa0?, 0xc013733710?}, 0xc001680c00?)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/stmgr.go:444 +0x2e
github.com/filecoin-project/lotus/chain/consensus.(*TipSetExecutor).ApplyBlocks.func2({{0xc00ff81170?, 0xc013733710?}}, 0x28ec90, 0x640ff140)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/consensus/compute_state.go:291 +0x302
github.com/filecoin-project/lotus/chain/consensus.(*TipSetExecutor).ApplyBlocks(0xc0000143a8, {0x64bafa0, 0xc038220480}, 0xc00093bd40, 0x28ec8f, {{0xc00ff81170?, 0x28ec91?}}, {0xc00fc8b5e0, 0x3, 0x3}, ...)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/consensus/compute_state.go:365 +0xc7c
github.com/filecoin-project/lotus/chain/consensus.(*TipSetExecutor).ExecuteTipSet(0x64bafa0?, {0x64bafa0?, 0xc038220450?}, 0xc00093bd40, 0xc01327e980, {0x0, 0x0}, 0x0?)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/consensus/compute_state.go:548 +0x7ef
github.com/filecoin-project/lotus/chain/stmgr.(*StateManager).TipSetState(0xc00093bd40, {0x64baf30?, 0xc000226008?}, 0xc01327e980)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/stmgr/execute.go:62 +0x733
github.com/filecoin-project/lotus/chain/messagepool.(*mpoolProvider).GetActorAfter(0xc000496300, {{0xc019b241c0?, 0xc002022360?}}, 0xc038220240?)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/messagepool/provider.go:94 +0x21b
github.com/filecoin-project/lotus/chain/messagepool.(*MessagePool).getStateNonce(0xc0164e8340, {0x64baef8?, 0xc0161b36c0?}, {{0xc019b241c0?, 0x0?}}, 0xc01327e980)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/messagepool/messagepool.go:1209 +0x1c6
github.com/filecoin-project/lotus/chain/messagepool.(*MessagePool).addLoaded(0xc0164e8340, {0x64baef8, 0xc0161b36c0}, 0xc019b22090)
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/messagepool/messagepool.go:1058 +0x73
github.com/filecoin-project/lotus/chain/messagepool.(*MessagePool).loadLocal(0xc0164e8340, {0x64baef8, 0xc0161b36c0})
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/messagepool/messagepool.go:1687 +0x28e
github.com/filecoin-project/lotus/chain/messagepool.New.func2()
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/messagepool/messagepool.go:448 +0x6b
created by github.com/filecoin-project/lotus/chain/messagepool.New
	/home/magik6k/github.com/filecoin-project/go-lotus/chain/messagepool/messagepool.go:446 +0xc25

Previously this would be very fast as before #10445 we'd just compute that state, and because genesis is mostly empty, it was very fast.

This PR special-cases genesis state compute to avoid this very long walk in the main sync code path.

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

@magik6k magik6k requested a review from a team as a code owner March 14, 2023 09:25
@magik6k
Copy link
Contributor Author

magik6k commented Mar 14, 2023

Oh, even better, https://github.com/filecoin-project/lotus/pull/10397/files is probably also related to making this issue possible to happen

@magik6k magik6k added this to the v1.21.0 milestone Mar 14, 2023
Copy link
Member

@Stebalien Stebalien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Stebalien Stebalien merged commit 27e3c0b into master Mar 14, 2023
@Stebalien Stebalien deleted the fix/slow-node-startup branch March 14, 2023 13:45
@Stebalien
Copy link
Member

Merging to unbork things, but we should have a test.

@Stebalien
Copy link
Member

@magik6k can you open the backport PR?

@magik6k magik6k mentioned this pull request Mar 14, 2023
7 tasks
@raulk raulk modified the milestones: v1.21.0, v1.20.4 Mar 15, 2023
arajasek added a commit that referenced this pull request Mar 15, 2023
dumikau pushed a commit to protofire/lotus that referenced this pull request Jun 8, 2023
* Fix the overzealous fix (filecoin-project#10366)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* chore: deps: update to actors v10.0.0

* build: v1.20.0: release prep (filecoin-project#10368)

* add upgrade epoch and bump version

* Update changelog

---------

Co-authored-by: Aayush <arajasek94@gmail.com>

* feat: stmgr: cache migrated stateroots

* feat: vm: switch to the new exec trace format (filecoin-project#10372)

This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).

* fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct

* feat: update renew-sectors with FIP-0045 logic

* Hide `lotus-worker set` command

Hide the `lotus-worker set` command and print that this command will be deprecated.

* chain prune hot -- hotstore online gc

* remove accidental add

* fix: EthAPI: Correctly get parent hash

* make debugging windowPoSt-failures human readable

* make gen

* indent output

* post worker sched: Filter out disabled workers correctly

* itests: Test PoSt worker RPC error handling

* post worker sched: Retry on alternative worker on RPC errors

* make gen

* itests: Wait for both workers in TestWindowPostWorkerDisconnected

* Unbump api version as its not necessary

* deps: Update go-jsonrpc to v0.2.2

* Add command to trigger moving GC of hotstore manually

* Timing info and updated docs

* fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct

* fix: EthAPI: Correctly get parent hash

* v1.20.1 prep

* fix: state: short-circuit genesis state computation

* fix: state: short-circuit genesis state computation

* update changelog

* Begin account for size during walks

* Track size of dags relevant to compaction

* Plan out moving GC limiting

* GC respects target for max hotstore space

* lint

* rpcenc: More reliably failing TestReaderRedirectDrop

* rpcenc: Propagate closeOnce in beginPost

* fix: eth API: correct gateway restrictions, drop unimplemented methods

* refactor: EthAPI: Drop unnecessary param from newEthTxReceipt

* feat(deps): update to go-data-transfer v2, still wip

* chore(deps): update go-libipfs

* chore(deps): update to get to compile

* update minimum Go version to 1.19.7

* use go-libipfs/files instead of ipfs/go-ipfs-files

* fix drand test

* Update .github/ISSUE_TEMPLATE/service_developer_bug_report.yml

* Update .github/ISSUE_TEMPLATE/service_developer_bug_report.yml

* Update service_developer_bug_report.yml

* Update service_developer_bug_report.yml

* Update service_developer_bug_report.yml

* un-un-update filecoin-ffi

* try to add statenetworkname method

* add method implementation

* add MPoolGetNonce

* add two more methods: StateCall and StateDecodeParams

* fix logging

* deprecate MarketListRetrievalDeals, make gen

* Fix up approximation and logging

* run make gen

* small doc patch on how to make gen after api changes

* fix TestPrintGroupInfo

* Eth API: make block parameter parsing sounder.

* Eth API: fail when requesting future epochs.

* add a test to verify block parameter soundness.

* register rcmgr metrics

* replace make all with make deps according to review

* fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (filecoin-project#10413)

* cid key size

* computing szPurge leads to deadlock and unneeded, remove

* Review Response

* Lint

* lint

* docsgen-cli

* only register metrics once

* lint

* fix: eth API: return correct txIdx around null blocks (filecoin-project#10419)

* fmt.

* lint

* feat:splitstore:Splitstore enabled by default (filecoin-project#10429)

Discard mode
---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Review Response

* fix lint.

* Configur buffer and threshold

* lint

* make gen

* fix TestEthBlockHashesCorrect_MultiBlockTipset: skip null rounds.

* switch to ChainAPI#ChainGetTipSetByHeight.

* review comments

* simplify test.

* feat: chain: make fetching tipset by height 1000x faster

* feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (filecoin-project#10416)

However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes filecoin-project#10415

* try to add statenetworkname method

* add method implementation

* add MPoolGetNonce

* add two more methods: StateCall and StateDecodeParams

* run make gen

* Eth API: make block parameter parsing sounder.

* Eth API: fail when requesting future epochs.

* add a test to verify block parameter soundness.

* fmt.

* fix lint.

* fix TestEthBlockHashesCorrect_MultiBlockTipset: skip null rounds.

* switch to ChainAPI#ChainGetTipSetByHeight.

* simplify test.

* fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (filecoin-project#10413)

* fix: eth API: return correct txIdx around null blocks (filecoin-project#10419)

* feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (filecoin-project#10416)

However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes filecoin-project#10415

* make gen

* change log

* fiix bakcport error filecoin-project#10427

* retract v1.20.2

* go mod tidy

* make gen

* feat: cli: Add an EVM command to fetch a contract's bytecode

* feat: eth API: reject masked ID addresses embedded in f410f payloads

We'll never get an actor/account deployed to one of these
addresses (although we might get a placeholder). However, converting
such an address to an f4 address is definitely wrong.

* chore: update the FFI for release (filecoin-project#10435)

Specifically, this updates the proofs libraries and enables CUDA by default.

* feat: stmgr: skip tipset execution when possible

* feat: eth: optimize receipt reading

This optimizes the eth APIs (except the fee history one) to lookup the
tipset state/receipts instead of computing the state.

* feat: api: optimize ChainGetParentReceipts

Read the receipts all at once instead of fetching them one-by-one.

* test: eth: reduce chances of chain-reorgs affecting the test

Now that this API is _much_ faster, we're more likely to "catch up" to
the head faster than it can stabilize. I'm pretty sure the test was
intended to be written this way anyways.

* gateway: fix: drop overzealous guard on MsigGetVested

* eth_feeHistory: migrate to using TipSetState.

* Eth API: drop support for 'pending' block parameter.

After transitioning from using StateCompute to loading receipts,
we can no longer handle the 'pending' block without forcing
computation. Eth Core Devs are evaluating a proposal to remove
support on their end too.

* Eth API: make net_version return the chain ID.

* fix imports.

* remove superfluous filter.

* drop irrelevant test.

* Eth API: make eth_getTransactionByBlock* ops return ErrUnsupported.

* fix: itest: avoid failing the test when we race the miner

I've been seeing quite a few tests failing here, so I'm hoping this will
deflake them a bit.

* feat: eth: Avoid StateCompute in EthTxnReceipt lookup (filecoin-project#10460)

Instead:

1. Use the receipt from the message search.
2. Re-compute the gas fees that would have been charged.

fixes filecoin-project#10418

Co-authored-by: raulk <raul.kripalani@gmail.com>

* refactor: update cache to the new generic version (filecoin-project#10463)

- Adds type safety.
- Reduces allocations.
- Fixes the drand cache (was storing by value, but retrieving by pointer)

* fix: gateway: correctly apply the fee history lookback max

This fix ensures that the full fee-history range falls into the
allowable lookback rang.

* Plumb through a proper Flush() method on all blockstores

* feat: consensus: log ApplyBlock timing/gas stats

* fix: stmgr: don't attempt to lookup genesis state (filecoin-project#10472)

* fix: stmgr: don't attempt to lookup genesis state

* stmgr: More correct fix for genesis state compute

* Revert "Eth API: drop support for 'pending' block parameter."

This reverts commit 9412753.

* fix: build: drop drand incentinet servers

* feat: gateway: export StateVerifierStatus

* chore: update go-libp2p-pubsub to v0.9.2

* chore: update ffi to increase execution parallelism (filecoin-project#10480)

This changes the default validation parallelism from 1 to 4, and makes
it configurable with a LOTUS_FVM_CONCURRENCY environment variable.

* fix connection gater integration test

* chore: update go-libp2p to v0.26.2

* make mod-tidy ci happy

* update go-libp2p-pubsub to v0.9.3

* update go-jsonrpc to v0.2.3

* ethrpc: Buffer sub messages if subscriber is slow

* make lint happy

* feat: apply gateway lookback limit to eth API lookback

This change:

1. Introduces new "limited" API endpoints for EthGetTransactionByHash
   and EthGetTransactionReceipt that accept lookback-limits.
2. Implements the gateway version of these API endpoints by calling the
   limited variants with the default message search lookback limit.

fixes filecoin-project#10412

* fix: actually apply limit

* feat: expose limited apis on the gateway

* fix: eth: handle a potential divide by zero in receipt handling

This isn't really possible to hit on-chain at this piont (message won't
be accepted) but we might as well be extra careful.

fixes filecoin-project#10471

* Don't enforce walking receipt tree during compaction

* Demote now common logs (filecoin-project#10516)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* feat:config:force existing users to opt into new defaults (filecoin-project#10488)

* Config default does not comment out EnableSplitstore

* Loadability check

* Remove test used for debugging

* regexp for properly safe check that config is set

* regexp for safely matching the EnableSpitstore field in the config

* Add instructions for undeleting config and remind users to set splitstore false for full archive

* UpdateConfig small docs and functional opts

* make gen

* Lint

* Fix

* nil pointer check on validate

* Unit testing of EnableSplitstore cases

* Address Review

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* release v1.21.0-rc1 prep

* chore: build: Update Changelog for v1.21.0

chore: build: Update Changelog for v1.21.0

* Add RPC API highlight

Add RPC API highlight

* Add callout to exchanges

Add callout to exchanges about the execution traces. Also moving the `execution trace format changes` section to above the contributors section.

* fix: miner: correctly count sector extensions

* fix: miner: call ExtendSectorExpiration2

* chore: all: bump go-libipfs

(cherry picked from commit f48c626)

* Initialize with same length as partition

Initialize the postParam.Partitions slice with the same length as i.Partitions before iterating over it in the loop.

* build: docker: Update GO-version

build: docker: Update GO-version

* build: release: v1.21.0-rc2

build: release: v1.21.0-rc2

* Update go.mod

Update go.mod

* Update changelog for v1.21.0-rc2

Update changelog for v1.21.0-rc2

* perf: eth: gas estimate set applyTsMessages false (filecoin-project#10546)

* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization

* set applyTsMessages = true in CallWithGas call in shed

* update test with new callwithgas api optimization for eth call

* Update chain/stmgr/call.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation

---------

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>

* feat: supply: drop genesis market locked funds

* feat: supply: only grab genesis msig locks for writes

* Update Changelog with backports

Update Changelog with backports

* Bump version in changelog

Bump version in changelog

* Add longer validation time cause to changelog

Add longer validation time cause to changelog

* refactor: stop using deprecated io/ioutil

* Fix error when injecting StateManager

* new actor bundle, fix ci, upgrade fx

* fix fx version

* upgrade go version circle ci

* make linter happy :)

* more go version fixing

* replace deprecated mac build

* removing support for macos build

* Fix ctrl-c for GOOD

---------

Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Travis Person <travisperson@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Phi <orjan.roren@gmail.com>
Co-authored-by: beck <1504068285@qq.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: hannahhoward <hannah@hannahhoward.net>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Arsenii Petrovich <arsenii@protofire.io>
Co-authored-by: Arsenii Petrovich <23522179+ArseniiPetrovich@users.noreply.github.com>
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: ychiao <ychiaoli18@users.noreply.github.com>
Co-authored-by: Travis Person <165274+travisperson@users.noreply.github.com>
Co-authored-by: raulk <raul.kripalani@gmail.com>
Co-authored-by: Peter Rabbitson <ribasushi@leporine.io>
Co-authored-by: vyzo <vyzo@hackzen.org>
Co-authored-by: Jorropo <jorropo.pgm@gmail.com>
Co-authored-by: Mikers <snissn@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
Co-authored-by: Henrique Moniz <1785239+hmoniz@users.noreply.github.com>
dumikau pushed a commit to protofire/lotus that referenced this pull request Jun 8, 2023
…ed (filecoin-project#144)

* use go-libipfs/files instead of ipfs/go-ipfs-files

* fix drand test

* Update .github/ISSUE_TEMPLATE/service_developer_bug_report.yml

* Update .github/ISSUE_TEMPLATE/service_developer_bug_report.yml

* Update service_developer_bug_report.yml

* Update service_developer_bug_report.yml

* Update service_developer_bug_report.yml

* un-un-update filecoin-ffi

* try to add statenetworkname method

* add method implementation

* add MPoolGetNonce

* add two more methods: StateCall and StateDecodeParams

* fix logging

* deprecate MarketListRetrievalDeals, make gen

* Fix up approximation and logging

* run make gen

* small doc patch on how to make gen after api changes

* fix TestPrintGroupInfo

* Eth API: make block parameter parsing sounder.

* Eth API: fail when requesting future epochs.

* add a test to verify block parameter soundness.

* register rcmgr metrics

* replace make all with make deps according to review

* fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (filecoin-project#10413)

* cid key size

* computing szPurge leads to deadlock and unneeded, remove

* Review Response

* Lint

* lint

* docsgen-cli

* only register metrics once

* lint

* fix: eth API: return correct txIdx around null blocks (filecoin-project#10419)

* fmt.

* lint

* feat:splitstore:Splitstore enabled by default (filecoin-project#10429)

Discard mode
---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Review Response

* fix lint.

* Configur buffer and threshold

* lint

* make gen

* fix TestEthBlockHashesCorrect_MultiBlockTipset: skip null rounds.

* switch to ChainAPI#ChainGetTipSetByHeight.

* review comments

* simplify test.

* feat: chain: make fetching tipset by height 1000x faster

* feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (filecoin-project#10416)

However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes filecoin-project#10415

* try to add statenetworkname method

* add method implementation

* add MPoolGetNonce

* add two more methods: StateCall and StateDecodeParams

* run make gen

* Eth API: make block parameter parsing sounder.

* Eth API: fail when requesting future epochs.

* add a test to verify block parameter soundness.

* fmt.

* fix lint.

* fix TestEthBlockHashesCorrect_MultiBlockTipset: skip null rounds.

* switch to ChainAPI#ChainGetTipSetByHeight.

* simplify test.

* fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (filecoin-project#10413)

* fix: eth API: return correct txIdx around null blocks (filecoin-project#10419)

* feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (filecoin-project#10416)

However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes filecoin-project#10415

* make gen

* change log

* fiix bakcport error filecoin-project#10427

* retract v1.20.2

* go mod tidy

* make gen

* feat: cli: Add an EVM command to fetch a contract's bytecode

* feat: eth API: reject masked ID addresses embedded in f410f payloads

We'll never get an actor/account deployed to one of these
addresses (although we might get a placeholder). However, converting
such an address to an f4 address is definitely wrong.

* chore: update the FFI for release (filecoin-project#10435)

Specifically, this updates the proofs libraries and enables CUDA by default.

* feat: stmgr: skip tipset execution when possible

* feat: eth: optimize receipt reading

This optimizes the eth APIs (except the fee history one) to lookup the
tipset state/receipts instead of computing the state.

* feat: api: optimize ChainGetParentReceipts

Read the receipts all at once instead of fetching them one-by-one.

* test: eth: reduce chances of chain-reorgs affecting the test

Now that this API is _much_ faster, we're more likely to "catch up" to
the head faster than it can stabilize. I'm pretty sure the test was
intended to be written this way anyways.

* gateway: fix: drop overzealous guard on MsigGetVested

* eth_feeHistory: migrate to using TipSetState.

* Eth API: drop support for 'pending' block parameter.

After transitioning from using StateCompute to loading receipts,
we can no longer handle the 'pending' block without forcing
computation. Eth Core Devs are evaluating a proposal to remove
support on their end too.

* Eth API: make net_version return the chain ID.

* fix imports.

* remove superfluous filter.

* drop irrelevant test.

* Eth API: make eth_getTransactionByBlock* ops return ErrUnsupported.

* fix: itest: avoid failing the test when we race the miner

I've been seeing quite a few tests failing here, so I'm hoping this will
deflake them a bit.

* feat: eth: Avoid StateCompute in EthTxnReceipt lookup (filecoin-project#10460)

Instead:

1. Use the receipt from the message search.
2. Re-compute the gas fees that would have been charged.

fixes filecoin-project#10418

Co-authored-by: raulk <raul.kripalani@gmail.com>

* refactor: update cache to the new generic version (filecoin-project#10463)

- Adds type safety.
- Reduces allocations.
- Fixes the drand cache (was storing by value, but retrieving by pointer)

* fix: gateway: correctly apply the fee history lookback max

This fix ensures that the full fee-history range falls into the
allowable lookback rang.

* Plumb through a proper Flush() method on all blockstores

* feat: consensus: log ApplyBlock timing/gas stats

* fix: stmgr: don't attempt to lookup genesis state (filecoin-project#10472)

* fix: stmgr: don't attempt to lookup genesis state

* stmgr: More correct fix for genesis state compute

* Revert "Eth API: drop support for 'pending' block parameter."

This reverts commit 9412753.

* fix: build: drop drand incentinet servers

* feat: gateway: export StateVerifierStatus

* chore: update go-libp2p-pubsub to v0.9.2

* chore: update ffi to increase execution parallelism (filecoin-project#10480)

This changes the default validation parallelism from 1 to 4, and makes
it configurable with a LOTUS_FVM_CONCURRENCY environment variable.

* fix connection gater integration test

* chore: update go-libp2p to v0.26.2

* make mod-tidy ci happy

* update go-libp2p-pubsub to v0.9.3

* update go-jsonrpc to v0.2.3

* ethrpc: Buffer sub messages if subscriber is slow

* make lint happy

* feat: apply gateway lookback limit to eth API lookback

This change:

1. Introduces new "limited" API endpoints for EthGetTransactionByHash
   and EthGetTransactionReceipt that accept lookback-limits.
2. Implements the gateway version of these API endpoints by calling the
   limited variants with the default message search lookback limit.

fixes filecoin-project#10412

* fix: actually apply limit

* feat: expose limited apis on the gateway

* fix: eth: handle a potential divide by zero in receipt handling

This isn't really possible to hit on-chain at this piont (message won't
be accepted) but we might as well be extra careful.

fixes filecoin-project#10471

* Don't enforce walking receipt tree during compaction

* Demote now common logs (filecoin-project#10516)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* feat:config:force existing users to opt into new defaults (filecoin-project#10488)

* Config default does not comment out EnableSplitstore

* Loadability check

* Remove test used for debugging

* regexp for properly safe check that config is set

* regexp for safely matching the EnableSpitstore field in the config

* Add instructions for undeleting config and remind users to set splitstore false for full archive

* UpdateConfig small docs and functional opts

* make gen

* Lint

* Fix

* nil pointer check on validate

* Unit testing of EnableSplitstore cases

* Address Review

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* release v1.21.0-rc1 prep

* chore: build: Update Changelog for v1.21.0

chore: build: Update Changelog for v1.21.0

* Add RPC API highlight

Add RPC API highlight

* Add callout to exchanges

Add callout to exchanges about the execution traces. Also moving the `execution trace format changes` section to above the contributors section.

* fix: miner: correctly count sector extensions

* fix: miner: call ExtendSectorExpiration2

* chore: all: bump go-libipfs

(cherry picked from commit f48c626)

* Initialize with same length as partition

Initialize the postParam.Partitions slice with the same length as i.Partitions before iterating over it in the loop.

* build: docker: Update GO-version

build: docker: Update GO-version

* build: release: v1.21.0-rc2

build: release: v1.21.0-rc2

* Update go.mod

Update go.mod

* Update changelog for v1.21.0-rc2

Update changelog for v1.21.0-rc2

* perf: eth: gas estimate set applyTsMessages false (filecoin-project#10546)

* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization

* set applyTsMessages = true in CallWithGas call in shed

* update test with new callwithgas api optimization for eth call

* Update chain/stmgr/call.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation

---------

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>

* feat: supply: drop genesis market locked funds

* feat: supply: only grab genesis msig locks for writes

* Update Changelog with backports

Update Changelog with backports

* Bump version in changelog

Bump version in changelog

* Add longer validation time cause to changelog

Add longer validation time cause to changelog

* refactor: stop using deprecated io/ioutil

* Fix error when injecting StateManager

* new actor bundle, fix ci, upgrade fx

* fix fx version

* upgrade go version circle ci

* make linter happy :)

* more go version fixing

* replace deprecated mac build

* removing support for macos build

* Fix ctrl-c for GOOD

* bypass mpool rm for set membership

---------

Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Arsenii Petrovich <arsenii@protofire.io>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Arsenii Petrovich <23522179+ArseniiPetrovich@users.noreply.github.com>
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: ychiao <ychiaoli18@users.noreply.github.com>
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: Travis Person <travisperson@users.noreply.github.com>
Co-authored-by: Travis Person <165274+travisperson@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: raulk <raul.kripalani@gmail.com>
Co-authored-by: Peter Rabbitson <ribasushi@leporine.io>
Co-authored-by: vyzo <vyzo@hackzen.org>
Co-authored-by: Phi <orjan.roren@gmail.com>
Co-authored-by: Jorropo <jorropo.pgm@gmail.com>
Co-authored-by: Mikers <snissn@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
Co-authored-by: Henrique Moniz <1785239+hmoniz@users.noreply.github.com>
Co-authored-by: Denis Kolegov <dnkolegov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants