RAM is stable with farming 1 x 1TB plot. There is no much change of RAM whether it is replotting or not.
I will now run with 2 x 1TB plot and report soon in 15 minutes.
This thread is not about unstable memory usage, but rather about high memory usage instead. Please limit conversation here to the minimum number of relevant messages, I don’t need a bunch of unnecessary comments, I only need to know the minimum size of the farm that still uses a lot of RAM as I indicated above so I can reproduce it myself.
Well, this is exactly what I want to report.
Farming 1 x 1TB plot: 23/32 GB RAM used
Farming 2 x 1TB plot: 28/32 GB RAM used
Farming 3 x 1TB plot: 30/32 GB RAM used
I’ve spent my time to test, accepted to win a little bit reward to report in many scenario and see what is your feedback? You don’t have to thank me, but at least, please don’t complain.
I’m sorry, I am thankful! I just don’t have time to read a lot of messages and try to keep the size of the discussion small and free from redundant messages and unnecessary chatter. If it is reproducible with 1T farm then I don’t care about 2x1T and beyond for obvious reasons.
1 Like
I was farming 12 TB on a 128GB machine with 95% RAM usage. I lost 5 TB to a file system corruption, and now it uses 85% of RAM for the remaining 7TB. I think you can replicate the memory leak on a 32GB machine with a 3TB farm
So I tried to plot/farm 1TiB in a Windows 10 VM with 20 cores and 20G of RAM and has not seen such memory usage increase on 3h.
I am still looking for minimal reproduction. What farmer options did you use to consume 23G of RAM with just 1T plot? How much did you have plotted there (I plotted 50 sectors from 0 for testing purposes).
Has this issue been resolved ?
I am also experiencing it.
29.8/31.1GB Ram usage farming 3Tb on Space Acres Windows 10. Plotting is finished.
Those who still observe this issue, do you see ~100% utilization on one of the disks at the same time by any chance?
Hi Nazar - thanks for keeping working on this matter. No, I’ve checked again in all of my windows farm (15 now), none of them has high utilization on disk. All at less than 10-15% active time by the constant audit.
1 Like
PC with 64GB ram win11 cpu 13700k, chrome and few more home task running now with 4+4+4+8 TB disk finished plotting for ~9.5TB now its exit after few minus with out of ram msg:
2024-02-25T15:43:18.446663Z INFO {disk_farm_index=1}: subspace_farmer::single_disk_farm::plotting: Plotting sector (35.92% complete) sector_index=1342
2024-02-25T15:43:18.446681Z INFO {disk_farm_index=0}: subspace_farmer::single_disk_farm::plotting: Replotting sector (0.00% complete) sector_index=1796
2024-02-25T15:43:18.446680Z INFO {disk_farm_index=2}: subspace_farmer::single_disk_farm::plotting: Plotting sector (36.03% complete) sector_index=1346
2024-02-25T15:43:18.446705Z INFO {disk_farm_index=3}: subspace_farmer::single_disk_farm::plotting: Plotting sector (17.96% complete) sector_index=1344
memory allocation of 5677872 bytes failed
Question to those who reproduce this: do you see these issues from the very beginning (0 sectors plotted) or only after some decent number of sectors was plotted?
I’d be curious if someone can start a farmer with empty disks and see if the issue is happening right aware.
I’ve been experimenting with various combinations and the only thing I didn’t have is terabytes already plotted due to the amount of time it takes.
I hope my new disks are delivered soon, so I can test it out.
It seems to scale with sectors plotted. I do not have exact numbers but when replotting for 3h memory usage was not high until well into the replotting process. I have 4tb total plotted
I have 450G plotted and I do see a bit higher memory usage, but nothing alarming yet. Will let it run for longer and hopefully be able to reproduce the issue better.
I was able to reproduce this (sort of) and it has nothing to do with plotting, just farming. More sectors you have plotted - worse it becomes.
I was able to write a small Rust application that demonstrates this issue and so far it looks like a Windows featurebug to me. It basically explodes memory usage somewhere deep inside of Windows and I’m having a hard time finding out where it actually goes. Hopefully someone from Rust community will be able to help with that, for now I’m kind of stuck 
UPD: Also reported to Microsoft here
2 Likes
Does the problem lie with FileExt::seek_read?
Have you tried using CreateFileMappingA from the win_sys crate?
.Net seems to use this to edit large-memory files without occupying excessive memory.
mmap seems to be the solution to this problem
use functions
CreateFileMappingA(
HANDLE(raw_handle as _) ,
ptr::null_mut(),
PAGE_READONLY,
0,
0,
PCSTR(ptr::null()),
);
MapViewOfFile(self._file_handle, FILE_MAP_READ, dwfileoffsethigh, dwfileoffsetlow, page_size)
UnmapViewOfFile();
Because each operation is a page, it needs to be page aligned, which may require pointer offsets. And it will take up more memory