Index out of bounds

Support Incentivized Testnet
,
1

This is a Centos9 system, compiled myself, sept-25

The farmer didn’t die but that error message doesn’t look healthy.

No special arguments used, the plot size is 1300G

Blockquote
2023-09-29T03:28:18.497833Z INFO single_disk_farm{disk_farm_index=1}: subspace_farmer::reward_signing: Successfully signed reward hash 0x8e92307204c759c08d8cfde28885f9c8f4d1aacb8a7e26f928dce20be1759324
thread ‘thread ‘’ panicked at ‘index out of bounds: the len is 15113 but the index is 32383238’, crates/subspace-proof-of-space/src/chiapos/table.rs:’ panicked at ‘index out of bounds: the len is 15113 but the index is 18446744073677209834’, crates/subspace-proof-of-space/src/chiapos/table.rs:306:37
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
306:37

Blockquote
2023-09-29T03:11:25.527590Z INFO subspace_farmer::piece_cache: Finished cache initialization
2023-09-29T03:12:25.877305Z INFO subspace_networking::node_runner: Public address status changed. old=Unknown new=Private
2023-09-29T03:14:38.819462Z INFO single_disk_farm{disk_farm_index=2}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819503Z INFO single_disk_farm{disk_farm_index=7}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819520Z INFO single_disk_farm{disk_farm_index=3}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819536Z INFO single_disk_farm{disk_farm_index=6}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819550Z INFO single_disk_farm{disk_farm_index=1}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819563Z INFO single_disk_farm{disk_farm_index=0}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819578Z INFO single_disk_farm{disk_farm_index=4}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819592Z INFO single_disk_farm{disk_farm_index=8}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819609Z INFO single_disk_farm{disk_farm_index=9}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:38.819640Z INFO single_disk_farm{disk_farm_index=5}: subspace_farmer::single_disk_farm::plotting: Node is synced, resuming plotting
2023-09-29T03:14:55.753727Z INFO single_disk_farm{disk_farm_index=8}: subspace_farmer::reward_signing: Successfully signed reward hash 0x441b9872dd401fe048a42e9191e8b026ee64ecf007275e131e1dcc8cd2b584ec
2023-09-29T03:15:55.736963Z INFO single_disk_farm{disk_farm_index=8}: subspace_farmer::reward_signing: Successfully signed reward hash 0x10209ecad9a15299fa547c29a8501514f379d67137668e1dff1cb5ef2557c5a6
2023-09-29T03:19:11.680108Z INFO single_disk_farm{disk_farm_index=8}: subspace_farmer::reward_signing: Successfully signed reward hash 0x8bc7bf9bd999e17513500e2e34c47668adc17d9d4c2082c15163bccf53fee4da
2023-09-29T03:22:18.550720Z INFO single_disk_farm{disk_farm_index=4}: subspace_farmer::reward_signing: Successfully signed reward hash 0x61d76ff94d8174c149acf32349154b6c5f40fad3b931a29acb5e66868997d1f3
2023-09-29T03:22:36.961226Z INFO single_disk_farm{disk_farm_index=4}: subspace_farmer::reward_signing: Successfully signed reward hash 0xac20dcac377daf8a8649ed58995a4593ad901aad37f2004c9241f00e2f23ffd3
2023-09-29T03:22:45.498350Z INFO single_disk_farm{disk_farm_index=4}: subspace_farmer::reward_signing: Successfully signed reward hash 0xd5164e9ff1e47cbde0d9851b446fd080dbb061931cf0261e38e0f7435f3eb59d
2023-09-29T03:23:31.863180Z INFO single_disk_farm{disk_farm_index=9}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (2.42%) sector_index=84
2023-09-29T03:24:01.453918Z INFO single_disk_farm{disk_farm_index=2}: subspace_farmer::single_disk_farm::plotting: Sector replotted successfully sector_index=656
2023-09-29T03:24:02.356036Z INFO single_disk_farm{disk_farm_index=4}: subspace_farmer::single_disk_farm::plotting: Sector replotted successfully sector_index=28
2023-09-29T03:24:13.819820Z INFO single_disk_farm{disk_farm_index=3}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (0.34%) sector_index=11
2023-09-29T03:24:15.502113Z INFO single_disk_farm{disk_farm_index=5}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (0.91%) sector_index=31
2023-09-29T03:24:15.727246Z INFO single_disk_farm{disk_farm_index=0}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (15.88%) sector_index=222
2023-09-29T03:24:18.596848Z INFO single_disk_farm{disk_farm_index=7}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (0.43%) sector_index=14
2023-09-29T03:24:22.729189Z INFO single_disk_farm{disk_farm_index=1}: subspace_farmer::single_disk_farm::plotting: Sector replotted successfully sector_index=540
2023-09-29T03:24:23.868000Z INFO single_disk_farm{disk_farm_index=6}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (78.79%) sector_index=2766
2023-09-29T03:24:29.338542Z INFO single_disk_farm{disk_farm_index=8}: subspace_farmer::single_disk_farm::plotting: Sector plotted successfully (84.37%) sector_index=2962
2023-09-29T03:28:18.497833Z INFO single_disk_farm{disk_farm_index=1}: subspace_farmer::reward_signing: Successfully signed reward hash 0x8e92307204c759c08d8cfde28885f9c8f4d1aacb8a7e26f928dce20be1759324
thread ‘thread ‘’ panicked at ‘index out of bounds: the len is 15113 but the index is 32383238’, crates/subspace-proof-of-space/src/chiapos/table.rs:’ panicked at ‘index out of bounds: the len is 15113 but the index is 18446744073677209834’, crates/subspace-proof-of-space/src/chiapos/table.rs:306:37
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
306:37
2023-09-29T03:30:07.497884Z INFO single_disk_farm{disk_farm_index=6}: subspace_farmer::reward_signing: Successfully signed reward hash 0xe48600b732b0b65f4cb6807200402e33e1b19c956b746fd7589db0389c82250d

2

This is an integer underflow, subtraction wrapped the number around. Can you restart the farmer with RUST_LOG=info,subspace_farmer=debug and see if it crashes again? If so I’ll need those more detailed logs.

3

If it does not and replots remaining sectors successfully, it might be a memory issue, running memory check would be helpful in that case.

The values being subtracted must be sorted, thus underflow during subtraction shouldn’t be possible. I double checked the code and I do not immediately see how this could have happened. Not saying there isn’t any, but I’d be surprised if that was the case.