Increased sync speed on initial start, slows down overtime

Issue initially reported by @xorinox in Discord.

Summary

When user restarts the node + farmer the sync and plotting speeds are quicker for some time, but will start to really slow down in the long run. When user resets node + farmer they get some time of quick sync and plotting again. As a result the user will automate the restart for a quicker result.

Users Dialog below.

20 hours later

Aug 18 21:17:29 node-6d subspace-farmer[67025]: 2022-08-19T01:17:29.216703Z  INFO subspace_farmer::archiving: Plotted segment segment_index=56977
Aug 18 21:18:02 node-6d subspace-node[66816]: 2022-08-18 21:18:02 [PrimaryChain] :gear:  Syncing  0.0 bps, target=#567064 (75 peers), best: #19276 (0x1f82…4dcd), finalized #19175 (0x2254…f55f), :arrow_down: 34.6MiB/s :arrow_up: 1.0MiB/s

would you expect more progress?

/usr/local/bin/subspace-farmer --farm hdd=/chia/scratch/disk01/subspace,ssd=/chia/scratch/disk01/subspace,size=14T --farm hdd=/chia/scratch/disk02/subspace,ssd=/chia/scratch/disk02/subspace,size=14T --farm hdd=/chia/scratch/disk03/subspace,ssd=/chia/scratch/disk03/subspace,size=14T farm --reward-address ${REWARD_ADDRESS}


... I still observe, that a restart will kick off increased plotting and node sync activity 
my best guess is, that the node still get's into a situation where it;s only connected with peers that can't really support further syncing
During the past testnet I had the restart of the node/farmer automated to every 30 minutes... this helped to get synced quick 
it felt like this approach lead to faster synced nodes/plotted farmers versus starting the processes and forget
... i see that same behavior.. for past hours nothing was plotted or synced much
the restart 10 min ago. kicked of massive sync and plotting activity
do you have any explanation for that?

System Information

Running via SystemD
Running Aug-17 snapshot pre-release

Steps to reproduce:

  • Start a fresh Subspace Node + Farmer
  • Let sync for few hours
  • Reboot
  • Observe moment of quicker syncing

Actual Results:

Initial sync speeds are fine, but after some time it slows down. If rebooted the node then syncs quicker again for a moment of time then slows down again. best result sync speed wise is to reboot node almost every 30 min.