Can we have farmer start to plot when piece cache sync is in progress?

Feedback & Suggestions
1

What we have now is the farmer needs to complete its piece cache sync fully before the plotting can start. With the current data feed for 3h, the required piece to download from peers for large farmer is already about 45GB at start, and to increase more for late farmers. It can take more than a day.

So question is for large farmer, can we have the plotting start when it collect enough piece to plot? If this is implemented, I assume farmer only needs to sync piece for about 1 hour or less before it can plot.

Even nicer, this is applicable to late farmers who join the network at any time. This is very important to not discourage late joiners/farmers later on mainnet.

Thanks.

2

There is no “enough piece to plot” since every sector contains a unique subset of all pieces that exist, so for most efficient plotting you need all pieces locally, which is why piece cache sync happens before initial plotting.

However, that is purely an implementation inefficiency. Part of the solution to improve it already landed earlier today in Farmer plot cache by nazar-pc · Pull Request #2550 · subspace/subspace · GitHub. The follow-up change will be to make piece cache sync and plotting happen in parallel and in case of Space Acres those two will happen in parallel to node sync with all 3 processes sharing pieces with each other if possible, meaning it will not be necessary to block processes on each other or download the same piece more than once in most cases, which will accelerate getting up to speed and farming greatly.

So stay tuned for updates, improvements are still coming to Subspace.

1 Like
3

Space Acres 0.1.5 contains the changes mentioned already and Advanced CLI will have it in the next release too.

4

That’s great. When it’s available in the next release, I surely can test by allocating only 1 blank disk to the farmer for few hours and see how quickly the plotting process can start (I have some blank disk without any plotting yet). This is something I can test out without much impact to my farming reward.

5

There is already a test build for that: Snapshot build · subspace/subspace@df919f9 · GitHub and last memory usage testing build contains it as well

6

I fired up farmer with 3 x 2TB plots (totally fresh one). And I can confirm that the first sector was plot when the piece sync was only about 7% completed. So the new feature is confirmed by my test.

Note: With this build and fresh plots, I noticed intensive write to all disks (100% active) when the farmer started. Only after a long while, then piece sync started. This did NOT happen when I tested using the old plots.