SPARTA (23 Dec 2017) KOKKOS mode is enabled (../kokkos.cpp:39) using 0 GPU(s) per MPI task using 1 thread(s) per MPI task package kokkos package kokkos reduction parallel/reduce comm classic # advect particles via VSS collisional flow on a uniform grid # particles reflect off global box boundaries variable x index 10 variable y index 10 variable z index 10 variable lx equal $x*1.0e-5 variable lx equal 512*1.0e-5 variable ly equal $y*1.0e-5 variable ly equal 320*1.0e-5 variable lz equal $z*1.0e-5 variable lz equal 640*1.0e-5 variable n equal 10*$x*$y*$z variable n equal 10*512*$y*$z variable n equal 10*512*320*$z variable n equal 10*512*320*640 seed 12345 dimension 3 global gridcut 1.0e-5 boundary rr rr rr create_box 0 ${lx} 0 ${ly} 0 ${lz} create_box 0 0.00512 0 ${ly} 0 ${lz} create_box 0 0.00512 0 0.0032 0 ${lz} create_box 0 0.00512 0 0.0032 0 0.0064 Created orthogonal box = (0 0 0) to (0.00512 0.0032 0.0064) create_grid $x $y $z create_grid 512 $y $z create_grid 512 320 $z create_grid 512 320 640 WARNING: Could not acquire nearby ghost cells b/c grid partition is not clumped (../grid.cpp:381) Created 104857600 child grid cells parent cells = 1 CPU time = 0.095862 secs create/ghost percent = 19.4529 80.5471 balance_grid rcb part Balance grid migrated 104834919 cells CPU time = 1.74363 secs reassign/sort/migrate/ghost percent = 58.1638 0.123526 26.4486 15.2641 species ar.species Ar mixture air Ar vstream 0.0 0.0 0.0 temp 273.15 global nrho 7.07043E22 global fnum 7.07043E6 collide vss air ar.vss create_particles air n $n create_particles air n 1048576000 Created 1048576000 particles CPU time = 0.264155 secs stats 100 compute temp temp stats_style step cpu np nattempt ncoll c_temp # first equilibrate with large timestep to unsort particles # then benchmark with normal timestep timestep 7.00E-8 run 30 Memory usage per proc in Mbytes: particles (ave,min,max) = 32.8179 32.8179 32.8179 grid (ave,min,max) = 5.98596 5.32638 6.26388 surf (ave,min,max) = 0 0 0 total (ave,min,max) = 38.8038 38.1443 39.0818 Step CPU Np Natt Ncoll c_temp 0 0 1048576000 0 0 273.15653 30 8.1248963 1048576000 1052011401 740501652 273.15653 Loop time of 8.12789 on 4608 procs for 30 steps with 1048576000 particles Particle moves = 31457280000 (31.5B) Cells touched = 157235934477 (157B) Particle comms = 4302751195 (4.3B) Boundary collides = 278258089 (278M) Boundary exits = 0 (0K) SurfColl checks = 0 (0K) SurfColl occurs = 0 (0K) Surf reactions = 0 (0K) Collide attempts = 30459686146 (30.5B) Collide occurs = 22114714690 (22.1B) Reactions = 0 (0K) Particles stuck = 0 Particle-moves/CPUsec/proc: 839906 Particle-moves/step: 1.04858e+09 Cell-touches/particle/step: 4.9984 Particle comm iterations/step: 3.83333 Particle fraction communicated: 0.136781 Particle fraction colliding with boundary: 0.00884559 Particle fraction exiting boundary: 0 Surface-checks/particle/step: 0 Surface-collisions/particle/step: 0 Surf-reactions/particle/step: 0 Collision-attempts/particle/step: 0.968287 Collisions/particle/step: 0.703008 Reactions/particle/step: 0 Move time (%) = 4.47441 (55.0501) Coll time (%) = 1.9338 (23.7921) Sort time (%) = 0.66024 (8.12314) Comm time (%) = 1.03732 (12.7625) Modfy time (%) = 0 (0) Outpt time (%) = 0.0197029 (0.242411) Other time (%) = 0.00242105 (0.0297869) Particles: 227556 ave 229258 max 226027 min Histogram: 27 122 416 841 1192 1136 618 195 56 5 Cells: 22755.6 ave 22756 max 22755 min Histogram: 2048 0 0 0 0 0 0 0 0 2560 GhostCell: 11015.4 ave 15084 max 4377 min Histogram: 11 61 155 273 749 720 1143 911 281 304 EmptyCell: 3721.3 ave 6844 max 0 min Histogram: 21 162 326 640 732 1014 649 738 158 168 timestep 7.00E-9 run $t run 100 Memory usage per proc in Mbytes: particles (ave,min,max) = 32.8179 32.8179 32.8179 grid (ave,min,max) = 5.98596 5.32638 6.26388 surf (ave,min,max) = 0 0 0 total (ave,min,max) = 38.8038 38.1443 39.0818 Step CPU Np Natt Ncoll c_temp 30 0 1048576000 1052011401 740501652 273.15653 100 6.8534421 1048576000 100634858 74052699 273.15653 130 9.8096961 1048576000 101970355 74068110 273.15653 Loop time of 9.81415 on 4608 procs for 100 steps with 1048576000 particles Particle moves = 104857600000 (105B) Cells touched = 146667680200 (147B) Particle comms = 1794458722 (1.79B) Boundary collides = 92761165 (92.8M) Boundary exits = 0 (0K) SurfColl checks = 0 (0K) SurfColl occurs = 0 (0K) Surf reactions = 0 (0K) Collide attempts = 9824154600 (9.82B) Collide occurs = 7358642424 (7.36B) Reactions = 0 (0K) Particles stuck = 0 Particle-moves/CPUsec/proc: 2.31865e+06 Particle-moves/step: 1.04858e+09 Cell-touches/particle/step: 1.39873 Particle comm iterations/step: 1 Particle fraction communicated: 0.0171133 Particle fraction colliding with boundary: 0.000884639 Particle fraction exiting boundary: 0 Surface-checks/particle/step: 0 Surface-collisions/particle/step: 0 Surf-reactions/particle/step: 0 Collision-attempts/particle/step: 0.0936904 Collisions/particle/step: 0.0701775 Reactions/particle/step: 0 Move time (%) = 5.94948 (60.6214) Coll time (%) = 1.22516 (12.4836) Sort time (%) = 2.04746 (20.8623) Comm time (%) = 0.549425 (5.59829) Modfy time (%) = 0 (0) Outpt time (%) = 0.0395245 (0.40273) Other time (%) = 0.00310112 (0.0315984) Particles: 227556 ave 229114 max 225845 min Histogram: 8 67 234 608 1070 1224 880 384 108 25 Cells: 22755.6 ave 22756 max 22755 min Histogram: 2048 0 0 0 0 0 0 0 0 2560 GhostCell: 11015.4 ave 15084 max 4377 min Histogram: 11 61 155 273 749 720 1143 911 281 304 EmptyCell: 3721.3 ave 6844 max 0 min Histogram: 21 162 326 640 732 1014 649 738 158 168