Issue training DS_NeRF. run_nerf.py gets automatically killed by system, possibly due to not enough RAM?

[ChristianMilianti]

Hi Yen-Chen,

I followed your instructions, but seem to be having an issue when running "run_nerf.py" to train DS-NeRF.

The program seems to get up to the get ray part, but I believe my systems runs out of RAM and hence automatically kills the script. I've tried changing N_rand, N_samples and N_importance as well as chunk and netchunk but I am still encountering the same issue.

I was wondering if you had ever encountered a similar issue before or if you knew what the problem could be?

Please see the following output below:
"
(nerf-supervision-test2) christian@christian-hp-zbook:~/Documents/GitHub/nerf-supervision$ python DS_NeRF/run_nerf.py --config DS_NeRF/configs/$OBJECT.txt
Mean Projection Error: 1.336129034424327
Loaded image data (3024, 4032, 3, 38) [3024. 4032. 764.86080901]
near/far: 1.2000000000000002 6.082591371036208
1 2685 1.3361403216637044 4.253687994484856 2.276678718530653
2 2692 1.418229556470065 4.347782976473377 2.367314486659611
3 2754 1.356924325911425 4.513016035740519 2.295840139520376
4 2489 1.3401163400104312 4.421857708472658 2.2684345222017805
5 2510 1.4223369726063062 4.479347166446267 2.3298223449032416
6 2152 1.4537776585416557 4.584575861023723 2.4045069280182316
7 2225 1.5064259150091883 4.460455230286136 2.232687589085591
8 1914 1.4914190077210379 4.274301961650865 2.2466767153793774
9 1794 1.5481049269380658 4.469570934481929 2.3500166500369546
10 1410 1.5844176758604196 4.110607125333729 2.3531117652508726
11 182 1.737093467840505 3.964598613934974 2.64271511210707
12 1430 1.492936372412857 4.015959590654333 2.3460717674220724
13 1989 1.796104046752739 4.655506124839721 2.9786131125585658
14 2238 1.7005383223136286 4.988173454813372 2.851783352301981
15 2882 1.7638375439792615 5.100327573117169 2.9887010407560854
16 2729 1.8799233016958434 5.683747641888692 3.169006711774422
17 2498 1.7813545673169184 5.627622259913167 3.0791895489607475
18 1813 2.112566020062583 6.019759966640099 3.3786384565378094
19 1464 1.8954575836226324 6.03828810712755 3.1700555931268752
20 1510 1.7931354047556982 5.231818042511121 2.8076740817312107
21 1390 2.0681497156843704 5.44989472517439 3.190496712665129
22 1379 1.8951057345071014 4.99178380242684 2.9672091464352013
23 1473 1.8158626178429758 4.517558945081407 2.6722046493836777
24 1730 2.1240441594794564 4.92037384436523 3.007207381559254
25 1433 2.00657874415717 4.618791090239269 2.92972011339423
26 1640 1.7120680584860983 4.1943782931274205 2.6999601992508073
27 724 2.033160932755027 4.615933309370709 3.264174534873625
28 1400 2.097060078751854 4.643794115253744 3.1745723693615115
29 1413 2.013415061140006 4.651377527167284 3.085061591205556
30 637 1.5739971914282882 4.3592319178422745 2.6186279437760778
31 2221 1.5299034813604273 3.633218032669021 2.2444490657886735
32 2376 1.8312957492241713 4.829899143776502 2.776173956882891
33 2010 1.7433228195044195 4.298378888106929 2.63760995927851
34 2070 1.8087965201106648 4.356818917953441 2.6472789929221316
35 1788 1.985549823590858 4.836305275188235 2.9799856589648153
36 3112 1.6300575868290597 4.463178266019346 2.4360249636356186
37 2441 1.5648590767685226 4.703708347121633 2.4210232221175936
38 1697 2.0631896406293455 5.19494490279564 3.1145058424517105
Loaded image data (3024, 4032, 3, 38) [3024. 4032. 764.86080901]
Loaded ./data/fork 2.8637390179169158 13.064216097694247
bds: [2.863739 9.149744]
Data:
(38, 3, 5) (38, 3024, 4032, 3) (38, 2)
HOLDOUT view is 7
Loaded llff (38, 3024, 4032, 3) (120, 3, 5) [3024. 4032. 764.8608] ./data/fork
Auto LLFF holdout, 8
DEFINING BOUNDS
NEAR FAR 1.200000035762787 6.082591533660889
Found ckpts []
Not ndc!
get rays
Killed
"

System:
Ubuntu 20.04
RTX3080 16GB Mobile
32gb RAM
11th gen i9

I've also installed all requirements and dependencies as per instructions.
Also using Ceres Solver v2.1.0, and COLMAP 3.6 with CUDA.

Thanks! :)