Plot is slow and cpu/disk is low

Support
,
1

Issue Report

Environment

  • Ubuntu 22.04.3 LTS
  • Docker:ghcr.io/subspace/farmer:gemini-3g-2024-jan-29-2
  • CPU
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         43 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  48
  On-line CPU(s) list:   0-47
Vendor ID:               AuthenticAMD
  Model name:            AMD EPYC 7402 24-Core Processor
    CPU family:          23
    Model:               49
    Thread(s) per core:  2
    Core(s) per socket:  24
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU max MHz:         2800.0000
    CPU min MHz:         1500.0000
    BogoMIPS:            5599.90
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse
                         3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_p
                         state ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat
                         npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sme sev sev_es
Virtualization features:
  Virtualization:        AMD-V
Caches (sum of all):
  L1d:                   768 KiB (24 instances)
  L1i:                   768 KiB (24 instances)
  L2:                    12 MiB (24 instances)
  L3:                    128 MiB (8 instances)
NUMA:
  NUMA node(s):          4
  NUMA node0 CPU(s):     0-5,24-29
  NUMA node1 CPU(s):     6-11,30-35
  NUMA node2 CPU(s):     12-17,36-41
  NUMA node3 CPU(s):     18-23,42-47
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Mitigation; untrained return thunk; SMT enabled with STIBP protection
  Spec rstack overflow:  Mitigation; safe RET
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected
  • MEM
               total        used        free      shared  buff/cache   available
Mem:           125Gi       2.4Gi       115Gi        12Mi       7.4Gi       122Gi
Swap:          8.0Gi          0B       8.0Gi

Problem

  • node is synced, not error log
  • plot is slow 30m+ one sector, only one farmer running
  • already try run CLI direct not docker, same result
  • how to troubleshoot that? or is it normal for that speed ?

Log

2

@nazar-pc new to subspace, the server machine configure is not correct?

3

You didn’t specify the command you’ve used like on GitHub. It has problematic arguments, I’d recommend you to not specify things you don’t fully understand and use defaults instead.

The reason for slow plotting sectors can be caused by Farmer takes a long time to download pieces · Issue #2398 · subspace/subspace · GitHub and should be resolved in the next release automatically.

4

sorry , i missing that.

  • the node command is
    docker run -d --name subspace-$1 --restart=always \
        --network subspace \
        -v /data/subspace/node:/var/subspace \
        -p 30333:30333/udp -p 30333:30333/tcp -p 30433:30433/udp -p 30433:30433/tcp -p 9944:9944/tcp \
        xxxx.com/subspace/node:3g \
        --chain gemini-3g  \
        --base-path /var/subspace  \
        --blocks-pruning 256 --state-pruning archive-canonical  \
        --port 30333 --dsn-listen-on /ip4/0.0.0.0/udp/30433/quic-v1 --dsn-listen-on /ip4/0.0.0.0/tcp/30433  \
        --rpc-cors all --rpc-methods unsafe  \
        --rpc-external --allow-private-ip --validator \
        --name cny-home-subspace
  • the faramer command is
    docker run -d --name subspace-$1 --restart=always \
        --network subspace \
        -v /data/subspace/store:/var/subspace \
        -p 0:30533/udp -p 0:30533/tcp \
        --add-host=node:192.168.1.5 \
        -e RUST_LOG=info,subspace_farmer=trace,subspace_farmer_components=trace \
        xxx.com/subspace/farmer:3g \
        farm \
        --node-rpc-url ws://node:9944 \
        --listen-on /ip4/0.0.0.0/udp/30533/quic-v1 --listen-on /ip4/0.0.0.0/tcp/30533 \
        --reward-address stAy9rRkZaNa2fijHfH7Pn5KMfpbZCR89R2ri6YLpugLRYsU4 \
        path=/var/subspace/s200/test1,size=16G
  • and try add --sector-downloading-concurrency 4 --sector-encoding-concurrency 20 --plotting-thread-pool-size 48 --replotting-thread-pool-size 24 not work.
5

With a single farm that is so small the extra CLI options you tried make no sense and even contradict each other. On that CPU you ideally need way more space and more farms. Otherwise leave it alone and it will not take too much to finish plotting, but also the reward will likely be tiny as well.

6
  • actually, this machine having 10 * 4T ssd and 8* 16T hdd.
  • that is testing machine working.
  • so what is recommed options? 8 farmer path to 8 ssd with 4T size to math NUMA=8 and all other option is default?
  • the 16T hdd can be using for farming?
7

At least 8, could be more if you want.

No, HDD is not usable