Deleting DataFrames does not free up GPU memory

[ayushdg]

import numpy as np
from pygdf import DataFrame

record = np.array([(2,3),(4,5)], dtype = np.dtype('i8,i8'))
gdf = DataFrame.from_records(record)
del(gdf)

Expected: Gpu memory taken up by the dataframe will be freed.

Actual:

Tue Jul 31 11:54:08 2018       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.45                 Driver Version: 396.45                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 710      Off  | 00000000:01:00.0 N/A |                  N/A |
| 50%   47C    P8    N/A /  N/A |     11MiB /  2000MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+
|   1  TITAN V             Off  | 00000000:02:00.0 Off |                  N/A |
| 29%   42C    P8    28W / 250W |   410MiB  / 12066MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0                    Not Supported                                       |
|    1     12145      C   python                                       410MiB |
+-----------------------------------------------------------------------------+

It's not a major issue during data exploration and basic analysis, but when working with multiple dataframes, not having the ability to manually delete the ones which are not needed anymore is resulting in CUDA OOM errors.

[kkraus14]

@ayushdg since Numba is the allocator / deallocator of the memory, it follows Numba's CUDA memory management rules. In this case by default Numba won't try to deallocate memory until you hit 20% of the device memory (~2.4GB in your case). Can you try creating a larger dataframe and then calling the del and seeing if it frees the memory to confirm it's working on your end?

See http://numba.pydata.org/numba-doc/latest/cuda/memory.html#deallocation-behavior for more information.

[ayushdg]

Yeah. Testing on larger dataframes confirmed that when data in the deallocation queue exeeded 20% of the gpu memory it automatically deallocated the memory based on the deallocation behvaiour specified by Numba.