Skip to content

Commit

Permalink
Merge branch '203-global-maximum-particle-number' into 'master.dev'
Browse files Browse the repository at this point in the history
Resolve "Global maximum particle number"

Closes #203

See merge request piclas/piclas!865
  • Loading branch information
scopplestone committed Nov 29, 2023
2 parents d63356b + a70eecc commit 5c7de28
Show file tree
Hide file tree
Showing 33 changed files with 1,895 additions and 801 deletions.
41 changes: 40 additions & 1 deletion docs/documentation/developerguide/code_extension.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,43 @@ Finally, the `WriteSurfSampleToHDF5` routine writes the prepared `MacroSurfaceVa

IF (ANY(PartBound%SurfaceModel.EQ.1)) CALL AddVarName(Str2DVarNames,nVar2D_Total,nVarCount,'Sticking_Coefficient')

The order of the variable names and their position in the `MacroSurfaceVal` array has to be the same. Thus, make sure to place the `AddVarName` call at the same position, where you placed the calculation and writing into the `MacroSurfaceVal` array, otherwise the names and values will be mixed up.
The order of the variable names and their position in the `MacroSurfaceVal` array has to be the same. Thus, make sure to place the `AddVarName` call at the same position, where you placed the calculation and writing into the `MacroSurfaceVal` array, otherwise the names and values will be mixed up.

## Arrays with size of PDM%maxParticleNumber

If an array is to store particle information, it is usually allocated with

ALLOCATE(ParticleInformation(1:PDM%maxParticleNumber))

But since PDM%maxParticleNumber is dynamic, this behavior must also be captured by this array. Therefore its size has to be changed in the routines `IncreaseMaxParticleNumber` and `ReduceMaxParticleNumber` in `src/particles/particle_tools.f90`.

IF(ALLOCATED(ParticleInformation)) CALL ChangeSizeArray(ParticleInformation,PDM%maxParticleNumber,NewSize, Default)

Default is an optional parameter if the new array memory is to be initialized with a specific value. The same must be done for TYPES of size PDM%maxParticleNumber. Please check both routines to see how to do it.

## Insert new particles

To add new particles, first create a new particle ID using the GetNextFreePosition function contained in `src/particles/particle_tools.f90`

NewParticleID = GetNextFreePosition()

This directly increments the variable PDM%CurrentNextFreePosition by 1 and if necessary adjusts PDM%ParticleVecLength by 1. If this is not desired, it is possible to pass an offset. Then the two variables will not be incremented, which must be done later by the developer. This can happen if the particle generation process is divided into several functions, where each function contains a loop over all new particles (e.g. `src/particles/emission/particle_emission.f90`).

DO iPart=1,nPart
NewParticleID = GetNextFreePosition(iPart)
END DO
PDM%CurrentNextFreePosition = PDM%CurrentNextFreePosition + nPart
PDM%ParticleVecLength = MAX(PDM%ParticleVecLength,GetNextFreePosition(0))

For the new particle to become a valid particle, the inside flag must be set to true and various other arrays must be filled with meaningful data. See SUBROUTINE CreateParticle in `src/particles/particle_operations.f90`. A basic example of the most important variables is given below:

newParticleID = GetNextFreePosition()
PDM%ParticleInside(newParticleID) = .TRUE.
PDM%FracPush(newParticleID) = .FALSE.
PDM%IsNewPart(newParticleID) = .TRUE.
PEM%GlobalElemID(newParticleID) = GlobElemID
PEM%LastGlobalElemID(newParticleID) = GlobElemID
PartSpecies(newParticleID) = SpecID
LastPartPos(1:3,newParticleID) = Pos(1:3)
PartState(1:3,newParticleID) = Pos(1:3)
PartState(4:6,newParticleID) = Velocity(1:3)
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
(sec:particle-initialization-and-emission)=
# Particle Initialization & Emission

The RAM to store the particles is dynamically allocated. However, it is possible to restrict the number of particles per MPI process by setting

Part-MaxParticleNumber=1000000

New memory is allocated in separate chunks because allocating memory for the particle data and copying it to the new memory area is expensive. The chunksize is relative to the particles used and can be set with

Part-MaxPartNumIncrease=0.1

A higher value increases the amount of unnecessary RAM allocated to particles, while a lower value increases the number of memory adjustment operations. The optimal trade-off depends on the simulation and the machine, but it only affects the performance of the simulations, not the quality of the results.

The following section gives an overview of the available options regarding the definition of species and particle initialization
and emission. Simulation particles can be inserted initially within the computational domain and/or emitted at every time step.
First of all, the number of species is defined by

Part-nSpecies=1
Part-MaxParticleNumber=1000000

The maximum particle number is defined per core and should be chosen according to the number of simulation particles you expect,
including a margin to account for imbalances due transient flow features and/or the occurrence of new particles due to chemical
reactions. Example: A node of a HPC cluster has 2 CPUs, each has 12 cores. Thus, the node has 24 cores that share a total of
128GB RAM. Allocating 1000000 particles per core means, you can simulate up to 24 Million particles on a single node in this
example (assuming an even particle distribution). The limiting factor here is the amount of RAM available per node.

Regardless whether a standalone PIC, DSMC, or a coupled simulation is performed, the atomic mass [kg], the charge [C] and the
weighting factor $w$ [-], sometimes referred to as macro-particle factor (MPF), are required for each species.
Expand Down
1 change: 1 addition & 0 deletions regressioncheck/NIG_tracking_DSMC/curved/parameter.ini
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ RefMappingEps = 1e-5
RefMappingGuess = 3
BezierNewtonTolerance = 1e-4
BezierSplitLimit = 0.8
Part-MPI-UNFP-afterPartSend = true
! =============================================================================== !
! OUTPUT / VISUALIZATION
! =============================================================================== !
Expand Down
Loading

0 comments on commit 5c7de28

Please sign in to comment.