update: builtin to builtin11 in wdpost worker #10715

siriusyim · 2023-04-21T04:27:01Z

Related Issues

none

Proposed Changes

storage/sealer/manager_post.go:
"github.com/filecoin-project/specs-actors/v6/actors/builtin" ->
"github.com/filecoin-project/go-state-types/builtin"

Additional Info

This is the log about wd post worker on calibnet

Apr 21 03:49:30 node lotus-miner[54989]: 2023-04-21T03:49:30.121+0800 INFO wdpost wdpost/wdpost_run.go:411 running window post {"cycle": "2023-04-21T03:49:29.754+0800", "chain-random": "u9W8Rko1E2nQs8Qzc59u44V6h3AX1xteF0B7zK0VWVg=", "deadline": {"CurrentEpoch":489793,"PeriodStart":489812,"Index":0,"Open":489812,"Close":489872,"Challenge":489792,"FaultCutoff":489742,"WPoStPeriodDeadlines":48,"WPoStProvingPeriod":2880,"WPoStChallengeWindow":60,"WPoStChallengeLookback":20,"FaultDeclarationCutoff":70}, "height": "489793", "skipped": 0}
Apr 21 03:49:30 node lotus-miner[54989]: 2023-04-21T03:49:30.121+0800 INFO wdpost wdpost/wdpost_run.go:431 computing window post {"cycle": "2023-04-21T03:49:29.754+0800", "batch": 0, "elapsed": 0.000042143, "skip": 0, "err": "get sectors count of partition failed:unsupported proof type: 13:\n github.com/filecoin-project/specs-actors/v6/actors/builtin.PoStProofWindowPoStPartitionSectors\n /home/sirius/go/pkg/mod/github.com/filecoin-project/specs-actors/[email protected]/actors/builtin/sector.go:127", "errVerbose": "get sectors count of partition failed:unsupported proof type: 13:\n github.com/filecoin-project/specs-actors/v6/actors/builtin.PoStProofWindowPoStPartitionSectors\n /home/sirius/go/pkg/mod/github.com/filecoin-project/specs-actors/[email protected]/actors/builtin/sector.go:127:\n github.com/filecoin-project/lotus/storage/sealer.(*Manager).generateWindowPoSt\n /home/sirius/go/src/github.com/siriusyim/lotus/storage/sealer/manager_post.go:118"}
Apr 21 03:49:30 node lotus-miner[54989]: 2023-04-21T03:49:30.121+0800 ERROR wdpost wdpost/wdpost_run.go:433 error generating window post: get sectors count of partition failed:unsupported proof type: 13:
Apr 21 03:49:30 node lotus-miner[54989]: 2023-04-21T03:49:30.121+0800 ERROR wdpost wdpost/wdpost_run.go:98 runPoStCycle failed: running window post failed:
Apr 21 03:49:30 node lotus-miner[54989]: - get sectors count of partition failed:unsupported proof type: 13:
Apr 21 03:49:30 node lotus-miner[54989]: 2023-04-21T03:49:30.122+0800 WARN wdpost wdpost/wdpost_changehandler.go:254 Aborted window post Proving (Deadline: &{CurrentEpoch:489793 PeriodStart:489812 Index:0 Open:489812 Close:489872 Challenge:489792 FaultCutoff:489742 WPoStPeriodDeadlines:48 WPoStProvingPeriod:2880 WPoStChallengeWindow:60 WPoStChallengeLookback:20 FaultDeclarationCutoff:70})

Checklist

Before you mark the PR ready for review, please make sure that:

Commits have a clear commit message.
PR title is in the form of of <PR type>: <area>: <change being made>
- example: fix: mempool: Introduce a cache for valid signatures
- PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
- area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
New features have usage guidelines and / or documentation updates in
- Lotus Documentation
- Discussion Tutorials
Tests exist for new functionality or change in behavior
CI is green

magik6k

Thanks a lot for catching this and the fix! We'll likely include this fix in the release

It also looks like there are missing parameters causing wdpost test to fail:

2023-04-21T04:34:58.457Z	INFO	advmgr	sealer/manager_post.go:85	GenerateWindowPoSt run at lotus-miner
2023-04-21T04:34:58.457Z	ERROR	wdpost	wdpost/wdpost_run.go:433	error generating window post: local prover: No cached parameters found for proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-0-0-0170db1f394b35d995252228ee359194b13199d259380541dc529fb0099096b0 [failure finding /var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-0-0-0170db1f394b35d995252228ee359194b13199d259380541dc529fb0099096b0.params]	{"cycle": "2023-04-21T04:34:58.453Z"}
2023-04-21T04:34:58.458Z	ERROR	wdpost	wdpost/wdpost_run.go:98	runPoStCycle failed: running window post failed:
    github.com/filecoin-project/lotus/storage/wdpost.(*WindowPoStScheduler).runPoStCycle
        /home/circleci/lotus/storage/wdpost/wdpost_run.go:494
  - local prover:
    github.com/filecoin-project/lotus/storage/sealer.(*Manager).GenerateWindowPoSt
        /home/circleci/lotus/storage/sealer/manager_post.go:88

We will look into fixing that as well

arajasek · 2023-04-21T13:50:16Z

Thank you very much for the testing and the catch!

ZenGround0 · 2023-04-21T15:11:46Z

Should we backport this to release/v1.22.0 too?

arajasek · 2023-04-21T15:17:33Z

@ZenGround0 done in #10716. This should get pulled in when we merge releases into the destination branch here

siriusyim · 2023-04-22T05:58:30Z

Hi , @magik6k ,After modifying this problem, I updated the version to test on calibnet, and WdPoSt still failed
lotus version
Daemon: 1.23.0-rc1+calibnet+git.07416e8c0+api1.5.0
Local: lotus version 1.23.0-rc1+calibnet+git.07416e8c0

lotus-miner version
Daemon: 1.23.0-rc1+calibnet+git.07416e8c0+api1.5.0
Local: lotus-miner version 1.23.0-rc1+calibnet+git.07416e8c0

2023-04-22T03:51:00.077 INFO filecoin_proofs::api::post_util > generate_sector_challenges:start
2023-04-22T03:51:00.077 INFO filecoin_proofs::api::post_util > generate_sector_challenges:finish
2023-04-22T03:51:00.077 DEBUG filcrypto::util::types > generate_fallback_sector_challenges: end
2023-04-22T03:51:00.077 DEBUG filcrypto::util::types > generate_fallback_sector_challenges: start
2023-04-22T03:51:00.077 INFO filecoin_proofs::api::post_util > generate_sector_challenges:start
2023-04-22T03:51:00.077 INFO filecoin_proofs::api::post_util > generate_sector_challenges:finish
2023-04-22T03:51:00.077 DEBUG filcrypto::util::types > generate_fallback_sector_challenges: end
2023-04-22T03:51:00.096+0800	WARN	wdpost	wdpost/wdpost_run.go:249	Checked sectors	{"checked": 3, "good": 3}
2023-04-22T03:51:00.096+0800	INFO	wdpost	wdpost/wdpost_run_faults.go:153	attempting recovery declarations for 3 sectors
2023-04-22T03:51:00.096+0800	WARN	wdpost	wdpost/wdpost_run.go:249	Checked sectors	{"checked": 191, "good": 191}
2023-04-22T03:51:00.102+0800	INFO	wdpost	wdpost/wdpost_run.go:411	running window post	{"cycle": "2023-04-22T03:50:59.702+0800", "chain-random": "OQ9CdZKJ0RLuNAiUxhsDt9MULKsrROeLL4bTKAzFgJg=", "deadline": {"CurrentEpoch":492676,"PeriodStart":492692,"Index":0,"Open":492692,"Close":492752,"Challenge":492672,"FaultCutoff":492622,"WPoStPeriodDeadlines":48,"WPoStProvingPeriod":2880,"WPoStChallengeWindow":60,"WPoStChallengeLookback":20,"FaultDeclarationCutoff":70}, "height": "492676", "skipped": 0}
2023-04-22T03:51:00.102+0800	INFO	advmgr	sealer/manager_post.go:137	generateWindowPoSt maxPartitionSize:2349 partitionCount:1
2023-04-22T03:51:00.102 DEBUG filcrypto::util::types > generate_fallback_sector_challenges: start
2023-04-22T03:51:00.102 INFO filecoin_proofs::api::post_util > generate_sector_challenges:start
2023-04-22T03:51:00.103 INFO filecoin_proofs::api::post_util > generate_sector_challenges:finish
2023-04-22T03:51:00.103 DEBUG filcrypto::util::types > generate_fallback_sector_challenges: end
2023-04-22T03:51:00.103+0800	INFO	advmgr	sealer/manager_post.go:219	generateWindowPost	{"index": 0}
2023-04-22T03:51:00.117+0800	WARN	wdpost	wdpost/wdpost_run_faults.go:180	declare faults recovered Message CID	{"cid": "bafy2bzacedm2jak6tx2lussgn6aspzw4zvljvj4eqyh2f7rfwzpotqwg235vo"}
2023-04-22T03:51:10.072+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 2, "forRound": 492677, "baseEpoch": 492676, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T03:51:40.007+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 2, "forRound": 492678, "baseEpoch": 492677, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T03:52:10.007+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 3, "forRound": 492679, "baseEpoch": 492678, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T03:52:40.004+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 1, "forRound": 492680, "baseEpoch": 492679, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T03:53:10.449+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 4, "forRound": 492681, "baseEpoch": 492680, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T03:53:32.760+0800	WARN	advmgr	sealer/manager_post.go:234	generateWindowPost done	{"index": 0, "skipped": 0, "took": "2m32.656581604s", "err": null}
2023-04-22T03:53:32.760 DEBUG filcrypto::util::types > merge_window_post_partition_proofs: start
2023-04-22T03:53:32.760 DEBUG filcrypto::util::types > merge_window_post_partition_proofs: end
2023-04-22T03:53:32.760+0800	INFO	wdpost	wdpost/wdpost_run.go:431	computing window post	{"cycle": "2023-04-22T03:50:59.702+0800", "batch": 0, "elapsed": 152.657857001, "skip": 0, "err": null}
2023-04-22T03:53:32.762 DEBUG filcrypto::util::types > verify_window_post: start
2023-04-22T03:53:32.769 INFO filecoin_proofs::api::window_post > verify_window_post:start
2023-04-22T03:53:32.769 INFO filecoin_proofs::caches > trying parameters memory cache for: WINDOW_POST[34359738368]-verifying-key
2023-04-22T03:53:32.769 INFO filecoin_proofs::caches > no params in memory cache for WINDOW_POST[34359738368]-verifying-key
2023-04-22T03:53:32.821 INFO storage_proofs_core::parameter_cache > parameter set identifier for cache: FallbackPoSt::PublicParams{sector_size: 34359738368, challenge_count: 10, sector_count: 2349}
2023-04-22T03:53:32.821 INFO storage_proofs_core::parameter_cache > ensuring that all ancestor directories for: "/var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-0377ded656c6f524f1618760bffe4e0a1c51d5a70c4509eedae8a27555733edc.vk" exist
2023-04-22T03:53:32.821 INFO storage_proofs_core::parameter_cache > checking cache_path: "/var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-0377ded656c6f524f1618760bffe4e0a1c51d5a70c4509eedae8a27555733edc.vk" for verifying key
2023-04-22T03:53:32.821 INFO storage_proofs_core::parameter_cache > Verify production parameters is false
2023-04-22T03:53:33.967 INFO storage_proofs_core::parameter_cache > read verifying key from cache "/var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-8-0-0377ded656c6f524f1618760bffe4e0a1c51d5a70c4509eedae8a27555733edc.vk"
2023-04-22T03:53:34.625 INFO filecoin_proofs::api::window_post > verify_window_post:finish
2023-04-22T03:53:34.625 DEBUG filcrypto::util::types > verify_window_post: end
2023-04-22T03:53:34.625+0800	INFO	wdpost	wdpost/wdpost_run.go:276	post cycle done	{"cycle": "2023-04-22T03:50:59.702+0800", "took": 154.922569533}
2023-04-22T03:59:40.165+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 3, "forRound": 492694, "baseEpoch": 492693, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T04:00:10.007+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 2, "forRound": 492695, "baseEpoch": 492694, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T04:00:40.313+0800	INFO	miner	miner/miner.go:477	completed mineOne	{"tookMilliseconds": 3, "forRound": 492696, "baseEpoch": 492695, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 0, "lookbackEpochs": 900, "networkPowerAtLookback": "1443885982121984", "minerPowerAtLookback": "0", "isEligible": false, "isWinner": false, "error": null}
2023-04-22T04:01:02.060+0800	ERROR	wdpost	wdpost/wdpost_run.go:697	estimating gas	{"error": "message execution failed: exit 16, reason: message failed with backtrace:\n00: f01035 (method 5) -- invalid post was submitted (16)\n (RetCode=16)"}
2023-04-22T04:01:02.081+0800	ERROR	wdpost	wdpost/wdpost_run.go:179	submit window post failed: pushing message to mpool:
    github.com/filecoin-project/lotus/storage/wdpost.(*WindowPoStScheduler).submitPoStMessage
        /home/sirius/go/src/github.com/siriusyim/lotus/storage/wdpost/wdpost_run.go:658
  - GasEstimateMessageGas error: message execution failed: exit 16, reason: message failed with backtrace:
00: f01035 (method 5) -- invalid post was submitted (16)
 (RetCode=16)
2023-04-22T04:01:02.081+0800	WARN	wdpost	wdpost/wdpost_changehandler.go:494	Aborted window post Submitting (Deadline: &{CurrentEpoch:492692 PeriodStart:492692 Index:0 Open:492692 Close:492752 Challenge:492672 FaultCutoff:492622 WPoStPeriodDeadlines:48 WPoStProvingPeriod:2880 WPoStChallengeWindow:60 WPoStChallengeLookback:20 FaultDeclarationCutoff:70})

siriusyim · 2023-04-22T06:08:58Z

I will update to v1.22.0 and test again

benjaminh83 · 2023-04-23T04:26:07Z

I think I'm in the same boat. I cannot generate windowPoST. I have 0 power.

$ lotus-miner version
Daemon:  1.23.0-rc1+calibnet+git.327760acf+api1.5.0
Local: lotus-miner version 1.23.0-rc1+calibnet+git.327760acf

$ lotus-miner proving compute window-post 0
Took 68.16276ms
ERROR: running window post failed: get sectors count of partition failed:unsupported proof type: 13:
    github.com/filecoin-project/specs-actors/v6/actors/builtin.PoStProofWindowPoStPartitionSectors
        /home/benjamin/go/pkg/mod/github.com/filecoin-project/specs-actors/[email protected]/actors/builtin/sector.go:127

siriusyim · 2023-04-23T08:41:42Z

I will update to v1.22.0 and test again

Submitted issue: #10733

update: builtin to builtin11 in wdpost worker

07416e8

siriusyim requested a review from a team as a code owner April 21, 2023 04:27

magik6k approved these changes Apr 21, 2023

View reviewed changes

magik6k added the release/backport label Apr 21, 2023

arajasek changed the base branch from release/v1.23.0 to release/v1.22.0 April 21, 2023 13:49

arajasek changed the base branch from release/v1.22.0 to release/v1.23.0 April 21, 2023 13:49

arajasek approved these changes Apr 21, 2023

View reviewed changes

arajasek mentioned this pull request Apr 21, 2023

fix: PoSt worker: use go-state-types for proof policies #10716

Merged

7 tasks

rjan90 mentioned this pull request Apr 21, 2023

2021-04-21: Lotus release steps #10717

Closed

21 tasks

magik6k approved these changes Apr 21, 2023

View reviewed changes

magik6k merged commit afc64eb into filecoin-project:release/v1.23.0 Apr 21, 2023

Fatman13 mentioned this pull request Apr 24, 2023

[venus] merge nv19 consensus / NV19 代码合并 filecoin-project/venus#5678

Closed

29 tasks

Stebalien removed the release/backport label Feb 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update: builtin to builtin11 in wdpost worker #10715

update: builtin to builtin11 in wdpost worker #10715

siriusyim commented Apr 21, 2023

magik6k left a comment

arajasek commented Apr 21, 2023

ZenGround0 commented Apr 21, 2023

arajasek commented Apr 21, 2023

siriusyim commented Apr 22, 2023 •

edited

Loading

siriusyim commented Apr 22, 2023

benjaminh83 commented Apr 23, 2023

siriusyim commented Apr 23, 2023 •

edited

Loading

update: builtin to builtin11 in wdpost worker #10715

update: builtin to builtin11 in wdpost worker #10715

Conversation

siriusyim commented Apr 21, 2023

Related Issues

Proposed Changes

Additional Info

Checklist

magik6k left a comment

Choose a reason for hiding this comment

arajasek commented Apr 21, 2023

ZenGround0 commented Apr 21, 2023

arajasek commented Apr 21, 2023

siriusyim commented Apr 22, 2023 • edited Loading

siriusyim commented Apr 22, 2023

benjaminh83 commented Apr 23, 2023

siriusyim commented Apr 23, 2023 • edited Loading

siriusyim commented Apr 22, 2023 •

edited

Loading

siriusyim commented Apr 23, 2023 •

edited

Loading