Certain farmers are saturating internet connection

Support
1

I have three PCs:
PC1: node + 8 farmers (one plot each), IP 192.168.0.10
PC2: 4 farmers (one plot each)
PC3: 1 farmer (two plots)

So farmers on PC2 and PC3 connect to node on PC1.
The issue is, that whenever I start the farmers on PC1, my internet connection fully saturates at 40mbit/s down and 10mbit/s up. When I shut the farmers on PC1 down (obviously leaving the node running so that PC2 and PC3 can continue to plot/farm), the internet transfer rates drop below 2mbit/s (down and up).
I find it very strange that only the farmers on PC1 cause such high traffic.

Node and farmers are dec-11 release. Same issue was present with dec-1 release.

What could be the reason for that?

I start the farmers on PC1 with this command:

./farmer farm --reward-address xyz \
--farm-during-initial-plotting \
--listen-on /ip4/0.0.0.0/tcp/30551 \
--plotting-thread-pool-size 8 \
path=/media/user/SSD/farm1,size=976250000K

Farmers on PC2 and PC3 are started with this command:

./farmer farm --reward-address xyz \
--farm-during-initial-plotting \
--plotting-thread-pool-size 16 \
--sector-downloading-concurrency 3 \
--sector-encoding-concurrency 2 \
--listen-on /ip4/0.0.0.0/tcp/30601 \
--node-rpc-url ws://192.168.0.10:9944 \
path=/media/max/SSD_001/farm1,size=3905000000K \
path=/media/max/SSD_001/farm2,size=3905000000K

The node is started like this:

  ./node \
    --chain gemini-3g \
    --blocks-pruning 256 \
    --state-pruning archive \
    --validator \
    --no-private-ipv4 \
    --base-path "/home/user/subspace/node_data" \
    --rpc-methods unsafe --unsafe-rpc-external --rpc-external --rpc-cors all \
1 Like
2

10Mbps is about the minimum we expect farmer to provide, it is possible and expected that it will be able to saturate significantly more at times.

3

I have a 1Gbps connection, which is saturated at times (15s - 1 minute), I can understand the need to share, but something could be done to limit the bandwidth, even if there is a minimum requirement.

It seems to calm down after a few days, but not being able to connect SSH during these times is a bit annoying.

PS : still plotting at this time.

4

Thanks. 10Mbps per farmer downstream or bidirectional? Many people at home have a much higher downstream bandwidth than upstream.

What happens, if the bandwidth is not enough? Will it impact plotting and/or farming?

5

Are you sure it is upload and not download? You can limit it with any external tools if you really need to.

Bidirectional.

Node may fall behind, plotting might be slower than it should be, etc.

1 Like
6

Thank you for clarification!

That however does not explain why the farmers on the same PC as the node cause a lot more internet traffic than the farmers on other PCs in LAN which connect to the same node.
Even if I start a single farmer on the node’s PC, it already causes more traffic that the other 5 farmers on the two other PCs.

Are local and non-local farmers handled differently?

7

I did a limit to 40mbps using tc -5 hours ago- and this came from upload .
650Mbps (real, not 1gbps commercial) was totally consumed, troubling others services hosted.

Now, my node lose sync sometimes but that keeps other services safe. Subspace is on 4, started 2 days ago (uptimebot looking for a http service)

_month
_month1079Ă—1547 141 KB

8

It doesn’t matter where farmers are, they must be doing something different then

9

Nodes and farms should not be placed together because sometimes the farm may consume all the bandwidth, which can cause your node to go offline. If you’re unlucky, your node might take a long time to resynchronize. Previously, my node was stuck at 0 Bps for half a day. Later, I discovered that it could be due to the farm using excessive bandwidth. Therefore, nodes and farms should be on separate machines; don’t put them together.

10

But does that help if they still share the same internet connection?

11

Just to be clear: Bandwidth requirements scale with number of farmers and not with number of plots, correct?

12

Cannot share bandwidth, please use your switch or router to limit it.

13

Exactly, you shouldn’t need more than one farmer on one machine.

14

Stable at mine for now with launching node/farmer using trickle

trickle -s -d 10240 -u 10240 ./subspace-node ...
trickle -s -d 10240 -u 10240 ./subspace-farmer ...

And limit connections on farmer port (30533 default)

/sbin/iptables -A INPUT -p tcp --syn --dport {FARMER_PORT} -m connlimit --connlimit-above 10 -j REJECT --reject-with tcp-reset
15

Hello everyone… new here :slight_smile:

just wanted to lend my experience here if it can help. i have the same issues as above only i had one farmer node on same pc saturate the network. stuck at 50 mbps down and a few upload. over 3 days i processed over 300 gb of data through my poor starlink… alexa wasnt working my other mining rigs couldnt submit shares. total network saturation.
now that its fully synced and the farmer is done plotting the network traffic is what id expect of a blockchain not trying to sync from scratch. new pc builds from Thanksgiving day sales. ryzen 7900x 32 gb ram 1 tb ssd. same ram ssd psu everything. only difference in the two machines is the offending rig was on starlink and the one that never presented this issues is on fiber. of course i doubt that has anything to do with it.

Screenshot 2023-12-22 195602
Screenshot 2023-12-22 1956022233Ă—767 113 KB

1 Like
16

4 servers (node + farmer) + little home internet traffic

DATE / RX / TX / TOTAL

subspace_traffic
subspace_traffic826Ă—498 40 KB

7 days / 1 server (subspace only) / 1009GB total

server_7days
server_7days975Ă—541 53.3 KB

17

Are you running multiple farmers at the same time on the same machine or is that different versions? Because multiple farmers will consume more network than a single farmer, that is 100% expected.

18

3 servers (1 node + 1 farmer with 5 plots )
1 server (1 node + 1 farmer with 1 plot )

There are different versions on screenshot (from gemini-3g-2023-dec-15 to gemini-3g-2023-dec-22)