Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simulation error at some timesteps #372

Closed
Sonuphd opened this issue Feb 23, 2021 · 23 comments
Closed

Simulation error at some timesteps #372

Sonuphd opened this issue Feb 23, 2021 · 23 comments
Labels

Comments

@Sonuphd
Copy link

Sonuphd commented Feb 23, 2021

Respected Sir,

I am facing a problem with the running of my code in HPC. I am sending you errors.

$ mpirun -np 6   $HOME/Smilei-master/smilei   LWFA.py

Attaching 9320 to 1476299.pbshpc

                    _            _
  ___           _  | |        _  \ \   Version : ??-??
 / __|  _ __   (_) | |  ___  (_)  | |
 \__ \ | '  \   _  | | / -_)  _   | |
 |___/ |_|_|_| |_| |_| \___| |_|  | |
                                 /_/



 Reading the simulation parameters
 --------------------------------------------------------------------------------
 HDF5 version 1.8.16
         Python version 3.6.0
         Parsing pyinit.py
         Parsing ??-??
         Parsing pyprofiles.py
         Parsing LWFA.py
         Parsing pycontrol.py
         Check for function preprocess()
         python preprocess function does not exist
         Calling python _smilei_check
         Calling python _prepare_checkpoint_dir
        [WARNING] Patches distribution: hilbertian


 Geometry: 2Dcartesian
 --------------------------------------------------------------------------------
         Interpolation order : 2
         Maxwell solver : Yee
         (Time resolution, Total simulation time) : (3.183099, 9424.777961)
         (Total number of iterations,   timestep) : (30000, 0.314159)
                    timestep  = 0.806226 * CFL
         dimension 0 - (Spatial resolution, Grid length) : (2.546479, 1884.955592)
                     - (Number of cells,    Cell length)  : (4800, 0.392699)
                     - Electromagnetic boundary conditions: (silver-muller, silver-muller)
                     - Electromagnetic boundary conditions k    : ( [1.00, 0.00] , [-1.00, -0.00] )
         dimension 1 - (Spatial resolution, Grid length) : (0.32, 502.65)
                     - (Number of cells,    Cell length)  : (160, 3.14)
                     - Electromagnetic boundary conditions: (silver-muller, silver-muller)
                     - Electromagnetic boundary conditions k    : ( [0.00, 1.00] , [-0.00, -1.00] )


 Load Balancing:
 --------------------------------------------------------------------------------
         Patches are initially homogeneously distributed between MPI ranks. (initial_balance = false)
         Happens: every 20 iterations
         Cell load coefficient = 1.00
         Frozen particle load coefficient = 0.10


 Vectorization:
 --------------------------------------------------------------------------------
         Mode: off


 Patch arrangement :
 --------------------------------------------------------------------------------


 Patch arrangement :
 --------------------------------------------------------------------------------


 Initializing MPI
 --------------------------------------------------------------------------------
         MPI_THREAD_MULTIPLE not enabled
         Number of MPI process : 6
         Number of patches :
                 dimension 0 - number_of_patches : 32
                 dimension 1 - number_of_patches : 16
         Patch size :
                 dimension 0 - n_space : 150 cells.
                 dimension 1 - n_space : 10 cells.
         Dynamic load balancing: every 20 iterations


 OpenMP
 --------------------------------------------------------------------------------
         Number of thread per MPI process : 2


 Initializing the restart environment
 --------------------------------------------------------------------------------


 Initializing moving window
 --------------------------------------------------------------------------------
         Moving window is active:
                 velocity_x : 1.00
                 time_start : 1696.46



 Initializing species
 --------------------------------------------------------------------------------

         Creating Species #0: electron
                 > Pusher: boris
                 > Density profile: 2D built-in profile `constant` (value: 0.005734)


 Initializing laser parameters
 --------------------------------------------------------------------------------
                Laser #0: separable profile
                        omega              : 1
                        chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                        time envelope      : 1D built-in profile `tgaussian` (start: 0.000000, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                        space envelope (y) : 1D user-defined function
                        space envelope (z) : 1D user-defined function
                        phase          (y) : 1D user-defined function
                        phase          (z) : 1D user-defined function
                delay phase      (y) : 0
                delay phase      (z) : 1.5708
                Laser #1: separable profile
                        omega              : 1
                        chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                        time envelope      : 1D built-in profile `tgaussian` (start: 6.283185, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                        space envelope (y) : 1D user-defined function
                        space envelope (z) : 1D user-defined function
                        phase          (y) : 1D user-defined function
                        phase          (z) : 1D user-defined function
                delay phase      (y) : 0
                delay phase      (z) : 1.5708
                Laser #2: separable profile
                        omega              : 1
                        chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                        time envelope      : 1D built-in profile `tgaussian` (start: 12.566371, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                        space envelope (y) : 1D user-defined function
                        space envelope (z) : 1D user-defined function
                        phase          (y) : 1D user-defined function
                        phase          (z) : 1D user-defined function
                delay phase      (y) : 0
                delay phase      (z) : 1.5708
                Laser #3: separable profile
                        omega              : 1
                        chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                        time envelope      : 1D built-in profile `tgaussian` (start: 18.849556, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                        space envelope (y) : 1D user-defined function
                        space envelope (z) : 1D user-defined function
                        phase          (y) : 1D user-defined function
                        phase          (z) : 1D user-defined function
                delay phase      (y) : 0
                delay phase      (z) : 1.5708


 Initializing Patches
 --------------------------------------------------------------------------------
         First patch created
                 Approximately 10% of patches created
                 Approximately 20% of patches created
                 Approximately 30% of patches created
                 Approximately 40% of patches created
                 Approximately 50% of patches created
                 Approximately 60% of patches created
                 Approximately 70% of patches created
                 Approximately 80% of patches created
                 Approximately 90% of patches created
         All patches created


 Creating Diagnostics, antennas, and external fields
 --------------------------------------------------------------------------------
         Created ParticleBinning #0: species electron
                 Axis moving_x from 0 to 1884.96 in 300 steps
                 Axis ekin from 1 to 500 in 200 steps
         Created ParticleBinning #1: species electron
                 Axis moving_x from 0 to 1884.96 in 300 steps
                 Axis px from -1 to 1 in 100 steps
         Created ParticleBinning #2: species electron
                 Axis y from 0 to 502.655 in 300 steps
                 Axis py from -1 to 1 in 100 steps
         Created ParticleBinning #3: species electron
                 Axis moving_x from 0 to 1884.96 in 300 steps
                 Axis px from -1 to 1 in 100 steps
         Created ParticleBinning #4: species electron
                 Axis y from 0 to 502.655 in 300 steps
                 Axis py from -1 to 1 in 100 steps
         Diagnostic Fields #0  :
                 Ex Ey Bz Jx Rho
         Probe diagnostic #0 created
                 4800 points
                 origin : 0, 251.327
                 corner 0 : 1884.96, 251.327
         Created performances diagnostic


 finalize MPI
 --------------------------------------------------------------------------------
         Done initializing diagnostics, antennas, and external fields


 Applying external fields at time t = 0
 --------------------------------------------------------------------------------


 Applying prescribed fields at time t = 0
 --------------------------------------------------------------------------------


 Initializing diagnostics
 --------------------------------------------------------------------------------


 Running diags at time t = 0
 --------------------------------------------------------------------------------


 Species creation summary
 --------------------------------------------------------------------------------
                 Species 0 (electron) created with 3072000 particles


 Memory consumption
 --------------------------------------------------------------------------------
         (Master) Species part = 24 MB
         Global Species part = 0.143 GB
         Max Species part = 24 MB
         (Master) Fields part = 19 MB
         Global Fields part = 0.115 GB
         Max Fields part = 19 MB
         (Master) ParticleBinning0.h5  = 0 MB
         Global ParticleBinning0.h5 = 0.003 GB
         Max ParticleBinning0.h5 = 0 MB
         (Master) ParticleBinning1.h5  = 0 MB
         Global ParticleBinning1.h5 = 0.001 GB
         Max ParticleBinning1.h5 = 0 MB
         (Master) ParticleBinning2.h5  = 0 MB
         Global ParticleBinning2.h5 = 0.001 GB
         Max ParticleBinning2.h5 = 0 MB
         (Master) ParticleBinning3.h5  = 0 MB
         Global ParticleBinning3.h5 = 0.001 GB
         Max ParticleBinning3.h5 = 0 MB
         (Master) ParticleBinning4.h5  = 0 MB
         Global ParticleBinning4.h5 = 0.001 GB
         Max ParticleBinning4.h5 = 0 MB
         (Master) Probes0.h5  = 0 MB
         Global Probes0.h5 = 0.001 GB
         Max Probes0.h5 = 0 MB


 Expected disk usage (approximate)
 --------------------------------------------------------------------------------
         WARNING: disk usage by non-uniform particles maybe strongly underestimated,
            especially when particles are created at runtime (ionization, pair generation, etc.)

         Expected disk usage for diagnostics:
                 File Fields0.h5: 1.15 G
                 File Probes0.h5: 7.42 M
                 File Performances.h5: 110.98 K
                 File scalars.txt: 78.42 K
                 File ParticleBinning0.h5: 18.34 M
                 File ParticleBinning1.h5: 9.18 M
                 File ParticleBinning2.h5: 9.18 M
                 File ParticleBinning3.h5: 9.18 M
                 File ParticleBinning4.h5: 9.18 M
         Total disk usage for diagnostics: 1.21 G



 Cleaning up python runtime environement
 --------------------------------------------------------------------------------
         Checking for cleanup() function:
         python cleanup function does not exist
         Calling python _keep_python_running() :
                 Closing Python


 Time-Loop started: number of time-steps n_time = 30000
 --------------------------------------------------------------------------------
      timestep       sim time   cpu time [s]   (    diff [s] )
     100/30000     3.1573e+01     1.4792e+01   (  1.4792e+01 )
     200/30000     6.2989e+01     2.9028e+01   (  1.4236e+01 )
     300/30000     9.4405e+01     4.3285e+01   (  1.4257e+01 )
     400/30000     1.2582e+02     5.7528e+01   (  1.4243e+01 )
     500/30000     1.5724e+02     7.1448e+01   (  1.3921e+01 )
     600/30000     1.8865e+02     8.5600e+01   (  1.4152e+01 )
     700/30000     2.2007e+02     9.9746e+01   (  1.4146e+01 )
     800/30000     2.5148e+02     1.1586e+02   (  1.6111e+01 )
     900/30000     2.8290e+02     1.3010e+02   (  1.4240e+01 )
    1000/30000     3.1432e+02     1.4373e+02   (  1.3633e+01 )
    1100/30000     3.4573e+02     1.5803e+02   (  1.4300e+01 )
    1200/30000     3.7715e+02     1.7229e+02   (  1.4263e+01 )
    1300/30000     4.0856e+02     1.8575e+02   (  1.3457e+01 )
    1400/30000     4.3998e+02     1.9881e+02   (  1.3059e+01 )
    1500/30000     4.7140e+02     2.1253e+02   (  1.3716e+01 )
    1600/30000     5.0281e+02     2.2714e+02   (  1.4619e+01 )
    1700/30000     5.3423e+02     2.4308e+02   (  1.5933e+01 )
    1800/30000     5.6564e+02     2.5894e+02   (  1.5866e+01 )
    1900/30000     5.9706e+02     2.7478e+02   (  1.5839e+01 )
    2000/30000     6.2848e+02     2.9124e+02   (  1.6462e+01 )
    2100/30000     6.5989e+02     3.0850e+02   (  1.7253e+01 )
    2200/30000     6.9131e+02     3.2501e+02   (  1.6514e+01 )
    2300/30000     7.2272e+02     3.3793e+02   (  1.2922e+01 )
    2400/30000     7.5414e+02     3.5004e+02   (  1.2111e+01 )
Stack trace (most recent call last) in thread 26197:
#10   Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#9    Object "/lib64/libc.so.6", at 0x2aff55d6988c, in clone
#8    Object "/lib64/libpthread.so.0", at 0x2aff53cd2e64, in
#7    Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2aff554b1dac, in
#6    Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2aff554860d9, in
#5    Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2aff554869d6, in
#4    Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2aff554b1ab2, in __kmp_invoke_microtask
#3    Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x7cc21f, in main
#2    Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x6f2bb1, in VectorPatch::dynamics(Params&, SmileiMPI*, SimWindow*, RadiationTables&, MultiphotonBreitWheelerTables&, double, Timers&, int)
#1    Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x7f2f24, in Species::dynamics(double, unsigned int, ElectroMagn*, Params&, bool, PartWalls*, Patch*, SmileiMPI*, RadiationTables&, MultiphotonBreitWheelerTables&, std::vector<Diagnostic*, std::allocator<Diagnostic*> >&)
#0    Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x732f19, in Projector2D2Order::currentsAndDensityWrapper(ElectroMagn*, Particles&, SmileiMPI*, int, int, int, bool, bool, int, int, int)
Segmentation fault (Address not mapped to object [0x2afb7a096e90])

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 26177 RUNNING AT khas007.hpc.iitd.ac.in
=   EXIT CODE: 139
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
@Sonuphd Sonuphd added the bug label Feb 23, 2021
@mccoys
Copy link
Contributor

mccoys commented Feb 23, 2021

Is it possible for you to share your input file ?

@Sonuphd
Copy link
Author

Sonuphd commented Feb 23, 2021

import math


l0 = 2.0*math.pi # laser wavelength [in code units]
t0 = l0  # optical cycle
Lsimx = 300.*l0  # length of the simulation box in x (= 240 mum)
Lsimy = 80.*l0  # length of the simulation box in y (= 64 mum)
Tsim = 1500.*t0  # duartion of the simulation (= 4 ps)
resx = 16.
resy = 2.  # nb of cells in on laser wavelength
rest = resx/0.8  # nb of timesteps in one optical cycle
dt = t0/rest
nx = 300*16  # nb of cells along x-axis
npatchx = 32 # nb of patches along x-axis

# -----------
# n0max = 5.734e-3 # electron density (code units =>1=plasma critical density)

# initial density profile of electrons
# def n0_electron(x,y):
# return 1./(1.+math.exp((abs(x-x0)-widthx)/Lx))*n0max

#------------



# DEFINING SMILEI's VARIABLES
# All in "bloacks"

Main(
    geometry = "2Dcartesian",
    interpolation_order = 2,
    timestep = dt,
    simulation_time = Tsim,
    cell_length = [l0/resx,l0/resy],
    grid_length = [Lsimx,Lsimy],
    number_of_patches = [32,16],
    clrw = nx/npatchx,
    reference_angular_frequency_SI = 2.0*math.pi*3e8/0.8e-6,
    EM_boundary_conditions = [["silver-muller","silver-muller"],["silver-muller","silver-muller"],],
    solve_poisson = False,
    print_every = 100,
    random_seed = smilei_mpi_rank
    )

    

MovingWindow(
    time_start = 0.9*Main.grid_length[0],
    velocity_x = 0.9997
    )

LoadBalancing(
        initial_balance = False,
    every  = 20,
    cell_load = 1.,

    )

    
Species(
    name = "electron",
    position_initialization = "regular",
    momentum_initialization = "cold",
    particles_per_cell = 4,
    mass = 1.0,
    charge = -1.0,
    number_density = 0.005734,# electron density (code units =>1=plasma critical density)
    mean_velocity = [0.0,0.0,0.0],
    pusher = "boris",
    time_frozen = 0.0,
    boundary_conditions = [["remove","remove"],["remove","remove"],],
    )

FWHMinI = 12.5*l0 #(= 10 mum)
waistinI = FWHMinI/(2.0*math.sqrt(math.log(2.0)))
waistinE = FWHMinI/math.sqrt(2.0*math.log(2.0))
FWHMtinI = 8.0*t0  #(= 21.3 fs)
FWHMtinE = FWHMtinI*math.sqrt(2.0)  #(= 30 fs)

diagEvery = int(37.5*t0/dt)  # frequency of outputs(= 100 fs)


LaserGaussian2D(
box_side = "xmin",
a0 = 2.6,  # intensity 1.45 e+19
omega = 1.,
focus = [0.,Main.grid_length[1]/2.],
waist = waistinE,

time_envelope = tgaussian(start=0.,duration=2.*FWHMtinE,fwhm=FWHMtinE,center=FWHMtinE)


)

LaserGaussian2D(
box_side = "xmin",
a0 = 2.6,  # intensity 1.45 e+19
omega = 1.,
focus = [0.,Main.grid_length[1]/2.],
waist = waistinE,

time_envelope = tgaussian(start=2.0*math.pi,duration=2.*FWHMtinE,fwhm=FWHMtinE,center=FWHMtinE)


)

LaserGaussian2D(
box_side = "xmin",
a0 = 2.6,  # intensity 1.45 e+19
omega = 1.,
focus = [0.,Main.grid_length[1]/2.],
waist = waistinE,

time_envelope = tgaussian(start=4.0*math.pi,duration=2.*FWHMtinE,fwhm=FWHMtinE,center=FWHMtinE)


)

LaserGaussian2D(
box_side = "xmin",
a0 = 2.6,  # intensity 1.45 e+19
omega = 1.,
focus = [0.,Main.grid_length[1]/2.],
waist = waistinE,

time_envelope = tgaussian(start=6.0*math.pi,duration=2.*FWHMtinE,fwhm=FWHMtinE,center=FWHMtinE)


)
list_fields = ['Ex','Ey','Bz','Rho','Jx']



DiagFields(
every = diagEvery,
fields = list_fields
)

DiagProbe(
    every = diagEvery,
    origin = [0.,Main.grid_length[1]/2.],
    corners = [[Main.grid_length[0],Main.grid_length[1]/2.],
           ],
    number = [nx],
    fields = list_fields
    )

DiagScalar(
    every = int(diagEvery/10),
    vars=['Uelm','Ukin_electron',
          'ExMax','ExMaxCell','EyMax','EyMaxCell','RhoMin',
          'RhoMinCell','Ukin_bnd','Uelm_bnd','Ukin_out_mvw','Ukin_inj_mvw','Uelm_out_mvw','Uelm_inj_mvw','Utot']
    )

DiagParticleBinning(
    deposited_quantity = "weight",
    every = diagEvery,
    species =  ["electron"],
    axes = [
        ["moving_x",0., Lsimx, 300],
        ["ekin",1.,500.,200]
        ]
    )
DiagParticleBinning(
    deposited_quantity = "weight",
    every = diagEvery,
    species =  ["electron"],
    axes = [
        ["moving_x",0., Lsimx, 300],
        ["px",-1.,1.,100]
        ]
    )
DiagParticleBinning(
    deposited_quantity = "weight",
    every = diagEvery,
    species =  ["electron"],
    axes = [
        ["y",0., Lsimy, 300],
        ["py",-1.,1.,100]
        ]
    )

DiagParticleBinning(
    deposited_quantity = "weight",
    every = diagEvery,
    species =  ["electron"],
    axes = [
        ["moving_x",0., Lsimx, 300],
        ["px",-1.,1.,100]
        ]
    )
DiagParticleBinning(
    deposited_quantity = "weight",
    every = diagEvery,
    species =  ["electron"],
    axes = [
        ["y",0., Lsimy, 300],
        ["py",-1.,1.,100]]

        )
DiagPerformances(
    every = diagEvery
    )

@Sanjeev273
Copy link

Check the line "nx = 30016 # nb of cells along x-axis" in your code. is it correct value of nx?

@Sonuphd
Copy link
Author

Sonuphd commented Feb 23, 2021

nx=300*16 is correct.

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

Respected Sir,

I have loaded these modules before simulation in my environment.

$ module list
Currently Loaded Modulefiles:

  1. apps/anaconda/3
  2. compiler/python/3.6.0/ucs4/gnu/447
  3. pythonpackages/3.6.0/ucs4/gnu/447/pip/9.0.1/gnu
  4. pythonpackages/3.6.0/ucs4/gnu/447/appdirs/1.4.3/gnu
  5. pythonpackages/3.6.0/ucs4/gnu/447/six/1.10.0/gnu
  6. pythonpackages/3.6.0/ucs4/gnu/447/pyparsing/2.2.0/gnu
  7. pythonpackages/3.6.0/ucs4/gnu/447/packaging/16.8/gnu
  8. pythonpackages/3.6.0/ucs4/gnu/447/setuptools/34.3.2/gnu
  9. pythonpackages/3.6.0/ucs4/gnu/447/wheel/0.30.0a0/gnu
  10. suite/intel/parallelStudio/2015
  11. lib/fftw/3.2.2/intel
  12. suite/intel/parallelStudio/default
  13. lib/phdf/5/1.8.16/intel
  14. smilei

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

Do you know if this error always occurs at the same time ?

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

yes, if I use this input file.

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

I am unable to reproduce the segfault unfortunately. Could you remove all diagnostics and try again ? Then remove the LoadBalancing and try again ?

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

Okay sir

@Sanjeev273
Copy link

yes, if I use this input file.

There is no error in your script. I tested it. working fine. Some times the simulation terminate before completion, when there is memory issue. Check if you have sufficient memory to run the simulation and to save the output data. it was happen to me in the past.

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

Thank you sir for your suggestion. I check this.

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

I am facing the same problem. Please solve my problem.

This is the command for job submission.

qsub -q low -I -P physics -N test -M $[email protected] -l select=4:ncpus=8 -l walltime=01:00:00

-l select= n # request n number of nodes/slots
-l ncpus= n # request n number of cpus on a node

[phz178389@khas026 ~/LWFA]
$ export OMP_NUM_THREADS=2
[phz178389@khas026 ~/LWFA]
$ mpirun -np 4 $HOME/Smilei-master/smilei LWFA.py

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

Please specify who you are responding to. Same problem as what ?

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

Simulation error at some timesteps.

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

Please explain better. What did you do to obtain the same problem.

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

I am attaching you error found during simulation.

$ mpirun -np 2 $HOME/Smilei-master/smilei LWFA.py
Attaching 27479 to 1477879.pbshpc
_ _
___ _ | | _ \ \ Version : ??-??
/ | _ __ () | | ___ () | |
__ \ | ' \ _ | | / -) _ | |
|
/ |||| || || ___| || | |
/_/

Reading the simulation parameters

HDF5 version 1.8.16
Python version 3.6.0
Parsing pyinit.py
Parsing ??-??
Parsing pyprofiles.py
Parsing LWFA.py
Parsing pycontrol.py
Check for function preprocess()
python preprocess function does not exist
Calling python _smilei_check
Calling python _prepare_checkpoint_dir
[WARNING] Patches distribution: hilbertian

Geometry: 2Dcartesian

     Interpolation order : 2
     Maxwell solver : Yee
     (Time resolution, Total simulation time) : (3.183099, 9424.777961)
     (Total number of iterations,   timestep) : (30000, 0.314159)
                timestep  = 0.806226 * CFL
     dimension 0 - (Spatial resolution, Grid length) : (2.546479, 1884.955592)
                 - (Number of cells,    Cell length)  : (4800, 0.392699)
                 - Electromagnetic boundary conditions: (silver-muller, silver-muller)
                 - Electromagnetic boundary conditions k    : ( [1.00, 0.00] , [-1.00, -0.00] )
     dimension 1 - (Spatial resolution, Grid length) : (0.32, 502.65)
                 - (Number of cells,    Cell length)  : (160, 3.14)
                 - Electromagnetic boundary conditions: (silver-muller, silver-muller)
                 - Electromagnetic boundary conditions k    : ( [0.00, 1.00] , [-0.00, -1.00] )

Load Balancing:

     Patches are initially homogeneously distributed between MPI ranks. (initial_balance = false)
     Happens: every 150 iterations
     Cell load coefficient = 1.00
     Frozen particle load coefficient = 0.10

Vectorization:

     Mode: off

Patch arrangement :

Patch arrangement :

Initializing MPI

     MPI_THREAD_MULTIPLE not enabled
     Number of MPI process : 2
     Number of patches :
             dimension 0 - number_of_patches : 32
             dimension 1 - number_of_patches : 16
     Patch size :
             dimension 0 - n_space : 150 cells.
             dimension 1 - n_space : 10 cells.
     Dynamic load balancing: every 150 iterations

OpenMP

     Number of thread per MPI process : 2

Initializing the restart environment

Initializing moving window

     Moving window is active:
             velocity_x : 1.00
             time_start : 1696.46

Initializing species

     Creating Species #0: electron
             > Pusher: boris
             > Density profile: 2D built-in profile `constant` (value: 0.005734)

Initializing laser parameters

            Laser #0: separable profile
                    omega              : 1
                    chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                    time envelope      : 1D built-in profile `tgaussian` (start: 0.000000, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                    space envelope (y) : 1D user-defined function
                    space envelope (z) : 1D user-defined function
                    phase          (y) : 1D user-defined function
                    phase          (z) : 1D user-defined function
            delay phase      (y) : 0
            delay phase      (z) : 1.5708
            Laser #1: separable profile
                    omega              : 1
                    chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                    time envelope      : 1D built-in profile `tgaussian` (start: 6.283185, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                    space envelope (y) : 1D user-defined function
                    space envelope (z) : 1D user-defined function
                    phase          (y) : 1D user-defined function
                    phase          (z) : 1D user-defined function
            delay phase      (y) : 0
            delay phase      (z) : 1.5708
            Laser #2: separable profile
                    omega              : 1
                    chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                    time envelope      : 1D built-in profile `tgaussian` (start: 12.566371, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                    space envelope (y) : 1D user-defined function
                    space envelope (z) : 1D user-defined function
                    phase          (y) : 1D user-defined function
                    phase          (z) : 1D user-defined function
            delay phase      (y) : 0
            delay phase      (z) : 1.5708
            Laser #3: separable profile
                    omega              : 1
                    chirp_profile      : 1D built-in profile `tconstant` (0.000000)
                    time envelope      : 1D built-in profile `tgaussian` (start: 18.849556, duration: 142.172254, sigma: 1822.570154, center: 71.086127, order: 2.000000)
                    space envelope (y) : 1D user-defined function
                    space envelope (z) : 1D user-defined function
                    phase          (y) : 1D user-defined function
                    phase          (z) : 1D user-defined function
            delay phase      (y) : 0
            delay phase      (z) : 1.5708

Initializing Patches

     First patch created
             Approximately 10% of patches created
             Approximately 20% of patches created
             Approximately 30% of patches created
             Approximately 40% of patches created
             Approximately 50% of patches created
             Approximately 60% of patches created
             Approximately 70% of patches created
             Approximately 80% of patches created
             Approximately 90% of patches created
     All patches created

Creating Diagnostics, antennas, and external fields

     Diagnostic Fields #0  :
             Ex Ey Bz Jx Rho

finalize MPI

     Done initializing diagnostics, antennas, and external fields

Applying external fields at time t = 0

Applying prescribed fields at time t = 0

Initializing diagnostics

Running diags at time t = 0

Species creation summary

             Species 0 (electron) created with 3072000 particles

Memory consumption

     (Master) Species part = 73 MB
     Global Species part = 0.143 GB
     Max Species part = 73 MB
     (Master) Fields part = 59 MB
     Global Fields part = 0.115 GB
     Max Fields part = 59 MB

Expected disk usage (approximate)

     WARNING: disk usage by non-uniform particles maybe strongly underestimated,
        especially when particles are created at runtime (ionization, pair generation, etc.)

     Expected disk usage for diagnostics:
             File Fields0.h5: 1.15 G
             File scalars.txt: 0 bytes
     Total disk usage for diagnostics: 1.15 G

Cleaning up python runtime environement

     Checking for cleanup() function:
     python cleanup function does not exist
     Calling python _keep_python_running() :
             Closing Python

Time-Loop started: number of time-steps n_time = 30000

  timestep       sim time   cpu time [s]   (    diff [s] )
 100/30000     3.1573e+01     1.6565e+01   (  1.6565e+01 )
 200/30000     6.2989e+01     3.3032e+01   (  1.6467e+01 )
 300/30000     9.4405e+01     4.9520e+01   (  1.6488e+01 )
 400/30000     1.2582e+02     6.6169e+01   (  1.6648e+01 )
 500/30000     1.5724e+02     8.2885e+01   (  1.6716e+01 )
 600/30000     1.8865e+02     9.9741e+01   (  1.6856e+01 )
 700/30000     2.2007e+02     1.1668e+02   (  1.6944e+01 )
 800/30000     2.5148e+02     1.3424e+02   (  1.7555e+01 )
 900/30000     2.8290e+02     1.5134e+02   (  1.7103e+01 )
1000/30000     3.1432e+02     1.6861e+02   (  1.7264e+01 )
1100/30000     3.4573e+02     1.8603e+02   (  1.7420e+01 )
1200/30000     3.7715e+02     2.0348e+02   (  1.7453e+01 )
1300/30000     4.0856e+02     2.2117e+02   (  1.7692e+01 )
1400/30000     4.3998e+02     2.3899e+02   (  1.7818e+01 )
1500/30000     4.7140e+02     2.5717e+02   (  1.8179e+01 )
1600/30000     5.0281e+02     2.7605e+02   (  1.8885e+01 )
1700/30000     5.3423e+02     2.9675e+02   (  2.0696e+01 )
1800/30000     5.6564e+02     3.1743e+02   (  2.0682e+01 )
1900/30000     5.9706e+02     3.3803e+02   (  2.0594e+01 )
2000/30000     6.2848e+02     3.5813e+02   (  2.0107e+01 )
2100/30000     6.5989e+02     3.7771e+02   (  1.9576e+01 )
2200/30000     6.9131e+02     3.9673e+02   (  1.9019e+01 )
2300/30000     7.2272e+02     4.1580e+02   (  1.9068e+01 )
2400/30000     7.5414e+02     4.3716e+02   (  2.1362e+01 )

Stack trace (most recent call last):
#11 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#10 Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x45ef98, in
#9 Object "/lib64/libc.so.6", at 0x2b3427daa504, in __libc_start_main
#8 Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x7cd72a, in main
#7 Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2b3427578fd4, in __kmpc_fork_call
#6 Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2b34275a5031, in __kmp_fork_call
#5 Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2b34275a39d6, in
#4 Object "/home/soft/intel2015/composer_xe_2015.3.187/compiler/lib/intel64/libiomp5.so", at 0x2b34275ceab2, in __kmp_invoke_microtask
#3 Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x7cc21f, in main
#2 Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x6f2bb1, in VectorPatch::dynamics(Params&, SmileiMPI*, SimWindow*, RadiationTables&, MultiphotonBreitWheelerTables&, double, Timers&, int)
#1 Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x7f2f24, in Species::dynamics(double, unsigned int, ElectroMagn*, Params&, bool, PartWalls*, Patch*, SmileiMPI*, RadiationTables&, MultiphotonBreitWheelerTables&, std::vector<Diagnostic*, std::allocator<Diagnostic*> >&)
#0 Object "/home/physics/phd/phz178389/Smilei-master/smilei", at 0x732f19, in Projector2D2Order::currentsAndDensityWrapper(ElectroMagn*, Particles&, SmileiMPI*, int, int, int, bool, bool, int, int, int)
Segmentation fault (Address not mapped to object [0x7ffa9a705a10])

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 26957 RUNNING AT khas026.hpc.iitd.ac.in
= EXIT CODE: 139
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES

APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

You already sent the error before. I asked what did you do to obtain it. Did you modify something?

Please make an effort to be clearer

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

I am unable to reproduce the segfault unfortunately. Could you remove all diagnostics and try again ? Then remove the LoadBalancing and try again ?

I have done these changes but problem is same.

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

I have done these changes.

#LoadBalancing(
  #   initial_balance = False,
  #every  = 150,
  #  cell_load = 1.,

  # )
#DiagFields(
#every = diagEvery,
#fields = list_fields
#)

#DiagProbe(
    #every = diagEvery,
    #origin = [0.,Main.grid_length[1]/2.],
    #corners = [[Main.grid_length[0],Main.grid_length[1]/2.],
     #      ],
    #number = [nx],
    #fields = list_fields
    #)

#DiagScalar(
   # every = int(diagEvery/10),
   # vars=['Uelm','Ukin_electron',
   #       'ExMax','ExMaxCell','EyMax','EyMaxCell','RhoMin',
    #      'RhoMinCell','Ukin_bnd','Uelm_bnd','Ukin_out_mvw','Ukin_inj_mvw','Uelm_out_mvw','Uelm_inj_mvw','Utot']
    #)

#DiagParticleBinning(
   # deposited_quantity = "weight",
    #every = diagEvery,
    #species =  ["electron"],
    #axes = [
      #  ["moving_x",0., Lsimx, 300],
        #["ekin",1.,500.,200]
        #]
  #  )
#DiagParticleBinning(
   # deposited_quantity = "weight",
   # every = diagEvery,
   # species =  ["electron"],
    #axes = [
        #["moving_x",0., Lsimx, 300],
       # ["px",-1.,1.,100]
       # ]
    #)
#DiagParticleBinning(
   # deposited_quantity = "weight",
   # every = diagEvery,
    #species =  ["electron"],
    #axes = [
      #  ["y",0., Lsimy, 300],
      #  ["py",-1.,1.,100]
      #  ]
    #)

#DiagParticleBinning(
 #   deposited_quantity = "weight",
 #   every = diagEvery,
 #   species =  ["electron"],
  #  axes = [
   #     ["moving_x",0., Lsimx, 300],
    #    ["px",-1.,1.,100]
     #   ]
   # )
#DiagParticleBinning(
 #   deposited_quantity = "weight",
  #  every = diagEvery,
  #  species =  ["electron"],
    #axes = [
    #    ["y",0., Lsimy, 300],
     #   ["py",-1.,1.,100]]

      #  )
#DiagPerformances(
   # every = diagEvery
   # )

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

Ok
I am unable to reproduce your error on 2 different machines. I even tried your exact MPI+OMP configuration. I suspect that your installation has problems, or that you have an old version.

  • try a new version
  • try another machine

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

This is working on my personal laptop but not working on HPC.

@mccoys
Copy link
Contributor

mccoys commented Feb 24, 2021

This means the cluster has installed something wrong. You can try asking them if they have a different version of the compilers.

@Sonuphd
Copy link
Author

Sonuphd commented Feb 24, 2021

Thank you for your help, I will ask them.

@mccoys mccoys closed this as completed Mar 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants