SPARTA (23 Dec 2017) KOKKOS mode is enabled (../kokkos.cpp:39) using 0 GPU(s) per MPI task using 2 thread(s) per MPI task package kokkos package kokkos reduction parallel/reduce comm classic # advect particles via VSS collisional flow on a uniform grid # particles reflect off global box boundaries variable x index 10 variable y index 10 variable z index 10 variable lx equal $x*1.0e-5 variable lx equal 128*1.0e-5 variable ly equal $y*1.0e-5 variable ly equal 160*1.0e-5 variable lz equal $z*1.0e-5 variable lz equal 160*1.0e-5 variable n equal 10*$x*$y*$z variable n equal 10*128*$y*$z variable n equal 10*128*160*$z variable n equal 10*128*160*160 seed 12345 dimension 3 global gridcut 1.0e-5 boundary rr rr rr create_box 0 ${lx} 0 ${ly} 0 ${lz} create_box 0 0.00128 0 ${ly} 0 ${lz} create_box 0 0.00128 0 0.0016 0 ${lz} create_box 0 0.00128 0 0.0016 0 0.0016 Created orthogonal box = (0 0 0) to (0.00128 0.0016 0.0016) create_grid $x $y $z create_grid 128 $y $z create_grid 128 160 $z create_grid 128 160 160 WARNING: Could not acquire nearby ghost cells b/c grid partition is not clumped (../grid.cpp:381) Created 3276800 child grid cells parent cells = 1 CPU time = 0.019453 secs create/ghost percent = 62.6324 37.3676 balance_grid rcb part Balance grid migrated 3225600 cells CPU time = 0.19434 secs reassign/sort/migrate/ghost percent = 33.2284 0.857664 32.861 33.0529 species ar.species Ar mixture air Ar vstream 0.0 0.0 0.0 temp 273.15 global nrho 7.07043E22 global fnum 7.07043E6 collide vss air ar.vss create_particles air n $n create_particles air n 32768000 Created 32768000 particles CPU time = 0.191104 secs stats 100 compute temp temp stats_style step cpu np nattempt ncoll c_temp # first equilibrate with large timestep to unsort particles # then benchmark with normal timestep timestep 7.00E-8 run 30 Memory usage per proc in Mbytes: particles (ave,min,max) = 68.9175 68.9175 68.9175 grid (ave,min,max) = 11.5451 11.0764 12.0139 surf (ave,min,max) = 0 0 0 total (ave,min,max) = 80.4627 79.9939 80.9314 Step CPU Np Natt Ncoll c_temp 0 0 32768000 0 0 273.13748 30 8.2003369 32768000 32870375 23136537 273.13748 Loop time of 8.2004 on 64 procs for 30 steps with 32768000 particles Particle moves = 983040000 (983M) Cells touched = 4888792842 (4.89B) Particle comms = 78028405 (78M) Boundary collides = 26610646 (26.6M) Boundary exits = 0 (0K) SurfColl checks = 0 (0K) SurfColl occurs = 0 (0K) Surf reactions = 0 (0K) Collide attempts = 951846999 (952M) Collide occurs = 691077101 (691M) Reactions = 0 (0K) Particles stuck = 0 Particle-moves/CPUsec/proc: 1.87308e+06 Particle-moves/step: 3.2768e+07 Cell-touches/particle/step: 4.97314 Particle comm iterations/step: 3 Particle fraction communicated: 0.0793746 Particle fraction colliding with boundary: 0.0270697 Particle fraction exiting boundary: 0 Surface-checks/particle/step: 0 Surface-collisions/particle/step: 0 Surf-reactions/particle/step: 0 Collision-attempts/particle/step: 0.968269 Collisions/particle/step: 0.703 Reactions/particle/step: 0 Move time (%) = 4.44333 (54.1843) Coll time (%) = 2.16579 (26.4108) Sort time (%) = 0.931299 (11.3568) Comm time (%) = 0.645997 (7.87764) Modfy time (%) = 0 (0) Outpt time (%) = 0.0138943 (0.169435) Other time (%) = 8.53501e-05 (0.0010408) Particles: 512000 ave 513377 max 509846 min Histogram: 1 2 1 5 8 13 13 8 11 2 Cells: 51200 ave 51200 max 51200 min Histogram: 64 0 0 0 0 0 0 0 0 0 GhostCell: 13515 ave 18496 max 8776 min Histogram: 8 0 16 8 0 0 24 0 0 8 EmptyCell: 4469.14 ave 6830 max 1936 min Histogram: 3 0 16 3 0 26 5 2 0 9 timestep 7.00E-9 run $t run 100 Memory usage per proc in Mbytes: particles (ave,min,max) = 68.9175 68.9175 68.9175 grid (ave,min,max) = 11.5451 11.0764 12.0139 surf (ave,min,max) = 0 0 0 total (ave,min,max) = 80.4627 79.9939 80.9314 Step CPU Np Natt Ncoll c_temp 30 0 32768000 32870375 23136537 273.13748 100 7.4904809 32768000 3145274 2314267 273.13748 130 10.65359 32768000 3188475 2316568 273.13748 Loop time of 10.6537 on 64 procs for 100 steps with 32768000 particles Particle moves = 3276800000 (3.28B) Cells touched = 4577351082 (4.58B) Particle comms = 26518622 (26.5M) Boundary collides = 8863205 (8.86M) Boundary exits = 0 (0K) SurfColl checks = 0 (0K) SurfColl occurs = 0 (0K) Surf reactions = 0 (0K) Collide attempts = 306994572 (307M) Collide occurs = 229966631 (230M) Reactions = 0 (0K) Particles stuck = 0 Particle-moves/CPUsec/proc: 4.80586e+06 Particle-moves/step: 3.2768e+07 Cell-touches/particle/step: 1.3969 Particle comm iterations/step: 1 Particle fraction communicated: 0.00809284 Particle fraction colliding with boundary: 0.00270484 Particle fraction exiting boundary: 0 Surface-checks/particle/step: 0 Surface-collisions/particle/step: 0 Surf-reactions/particle/step: 0 Collision-attempts/particle/step: 0.0936873 Collisions/particle/step: 0.0701802 Reactions/particle/step: 0 Move time (%) = 6.216 (58.3462) Coll time (%) = 1.3089 (12.2859) Sort time (%) = 2.9093 (27.308) Comm time (%) = 0.172622 (1.62031) Modfy time (%) = 0 (0) Outpt time (%) = 0.046464 (0.436132) Other time (%) = 0.000366542 (0.00344053) Particles: 512000 ave 513200 max 510615 min Histogram: 5 4 5 5 3 14 8 8 10 2 Cells: 51200 ave 51200 max 51200 min Histogram: 64 0 0 0 0 0 0 0 0 0 GhostCell: 13515 ave 18496 max 8776 min Histogram: 8 0 16 8 0 0 24 0 0 8 EmptyCell: 4469.14 ave 6830 max 1936 min Histogram: 3 0 16 3 0 26 5 2 0 9