Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU memory is not released. How to release memory for specific field variable? #8131

Closed
qmdxcube opened this issue Jun 5, 2023 · 2 comments
Assignees

Comments

@qmdxcube
Copy link

qmdxcube commented Jun 5, 2023

Describe the bug
GPU memory is not released.

To Reproduce
Please post a minimal sample code to reproduce the bug.

# taichi-1.5.0
import taichi as ti
import numpy as np

ti.init(arch=ti.cuda,kernel_profiler=True,device_memory_GB=5.0)

nelx, nely, nelz = 256, 256, 192
elements_box = (nelx,nely,nelz)
nodes_box = (nelx+1,nely+1,nelz+1)

# design variable vector and sensitivity
x = ti.field(dtype=ti.f64, shape=elements_box)
xTilde = ti.field(dtype=ti.f64, shape=elements_box)
xPhys = ti.field(dtype=ti.f64, shape=elements_box)
xold = ti.field(dtype=ti.f64, shape=elements_box)
dc = ti.field(dtype=ti.f64, shape=elements_box)
dv = ti.field(dtype=ti.f64, shape=elements_box)
dcPhys = ti.field(dtype=ti.f64, shape=elements_box)
dx = ti.field(dtype=ti.f64, shape=elements_box)

F = ti.Vector.field(3, dtype=ti.f64, shape=nodes_box)
supports = ti.field(dtype=ti.i32, shape=nodes_box)

u = ti.Vector.field(3, dtype=ti.f64, shape=nodes_box)
r = ti.Vector.field(3, dtype=ti.f64, shape=nodes_box)
p = ti.Vector.field(3, dtype=ti.f64, shape=nodes_box)
q = ti.Vector.field(3, dtype=ti.f64, shape=nodes_box)

use_multigrid = True
n_mg_levels = 7

def node_shape(l):
	return (nelx//2**l+1, nely//2**l+1, nelz//2**l+1)

def node_number(l):
	return (nelx//2**l+1)*(nely//2**l+1)*(nelz//2**l+1)

A = [ti.Matrix.field(m=3,n=3,dtype=ti.f64,shape=(node_number(l),27)) for l in range(2,n_mg_levels)] # stencils for coarser grids
M = [ti.Vector.field(3, dtype=ti.f64, shape=node_shape(l)) for l in range(n_mg_levels-1)]  # multigird inverse of diagonal of A
Az = [ti.Vector.field(3,dtype=ti.f64,shape=node_shape(l)) for l in range(n_mg_levels)]
rz = [ti.Vector.field(3, dtype=ti.f64, shape=node_shape(l)) for l in range(n_mg_levels)]  # multigird residual for Az=r
rAz = [ti.Vector.field(3, dtype=ti.f64, shape=node_shape(l)) for l in range(n_mg_levels)]  # multigird residual for Az=r
v = [ti.Vector.field(3, dtype=ti.f64, shape=node_shape(l)) for l in range(n_mg_levels)]  # multigird residual for Az=r
z = [ti.Vector.field(3, dtype=ti.f64, shape=node_shape(l)) for l in range(n_mg_levels)]  # multigird solution for Az=r

if __name__ == '__main__':
	change = 1.0
	while change > 0.01:
		change = 0.005

	del x,xTilde,xold,dc,dv,dcPhys,dx
	del F,supports
	del r,p,q
	del A,M,Az,rz,rAz,v,z

	xPhys_to_show = xPhys.to_numpy()
	U_to_show = u.to_numpy()

Log/Screenshots
My GPU has 6GB memory and 5GB was used for this code. Error appears when the last line of the code is running. Strangely, when the last two lines are exchanged, the code can be run successfully. Some fields are only useful for computation, so they can be deleted after computation and the del function was employed in this code. Other fields will be transformed into numpy arrays for further purpose. What is the reliable way to release memory for specific field variable? Thanks.

![traceback](https://github.com/taichi-dev/taichi/assets/67142522/b2763083-1eec-4cdb-ba67-ab706e0bdfc0)

Additional comments
If possible, please also consider attaching the output of command ti diagnose. This produces the detailed environment information and hopefully helps us diagnose faster.

If you have local commits (e.g. compile fixes before you reproduce the bug), please make sure you first make a PR to fix the build errors and then report the bug.

@github-project-automation github-project-automation bot moved this to Untriaged in Taichi Lang Jun 5, 2023
@lin-hitonami lin-hitonami self-assigned this Jun 9, 2023
@lin-hitonami lin-hitonami moved this from Untriaged to Todo in Taichi Lang Jun 9, 2023
@lin-hitonami
Copy link
Contributor

You can use ti.FieldsBuider to manually manage the allocation and destruction of fields. See here for more information.

@lin-hitonami
Copy link
Contributor

Closing this issue. If you have further questions, feel free to reopen it.

@github-project-automation github-project-automation bot moved this from Todo to Done in Taichi Lang Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

2 participants