Weak scaling performance, Collide benchmark, Broadwell, 16M particles/node
Performance in millions of particle-timesteps / second / node

Nodes CPU (mpi,hyper) Kokkos/OMP (mpi,hyper,thread) Kokkos/serial (mpi,hyper)
1 167.3 (72,2) 168.9 (36,2,2) 176.8 (72,2)
2 170 (72,2) 166.5 (36,2,2) 176 (72,2)
4 168.9 (72,2) 165.7 (36,2,2) 175.5 (72,2)
8 168.5 (72,2) 162.4 (36,2,2) 171.7 (72,2)
16 167.8 (72,2) 158.6 (36,2,2) 172.4 (72,2)
32 152.3 (72,2) 152.8 (36,2,2) 169.5 (72,2)
64 163.8 (72,2) 154.9 (36,2,2) 166.9 (72,2)

Run commands and logfile links for column CPU

1 mpiexec -np 72 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=1.mpi=72.hyper=2
2 mpiexec -np 144 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 160 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=2.mpi=72.hyper=2
4 mpiexec -np 288 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 256 -v y 160 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=4.mpi=72.hyper=2
8 mpiexec -np 576 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 256 -v y 160 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=8.mpi=72.hyper=2
16 mpiexec -np 1152 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 256 -v y 320 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=16.mpi=72.hyper=2
32 mpiexec -np 2304 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 512 -v y 320 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=32.mpi=72.hyper=2
64 mpiexec -np 4608 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 512 -v y 320 -v z 640 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=cpu.kind=weak.size=16M.node=64.mpi=72.hyper=2

Run commands and logfile links for column Kokkos/OMP

1 mpiexec -np 36 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=1.mpi=36.thread=2.hyper=2
2 mpiexec -np 72 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 160 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=2.mpi=36.thread=2.hyper=2
4 mpiexec -np 144 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 256 -v y 160 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=4.mpi=36.thread=2.hyper=2
8 mpiexec -np 288 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 256 -v y 160 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=8.mpi=36.thread=2.hyper=2
16 mpiexec -np 576 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 256 -v y 320 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=16.mpi=36.thread=2.hyper=2
32 mpiexec -np 1152 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 512 -v y 320 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=32.mpi=36.thread=2.hyper=2
64 mpiexec -np 2304 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 512 -v y 320 -v z 640 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_omp.kind=weak.size=16M.node=64.mpi=36.thread=2.hyper=2

Run commands and logfile links for column Kokkos/serial

1 mpiexec -np 72 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=1.mpi=72.hyper=2
2 mpiexec -np 144 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 160 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=2.mpi=72.hyper=2
4 mpiexec -np 288 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 256 -v y 160 -v z 160 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=4.mpi=72.hyper=2
8 mpiexec -np 576 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 256 -v y 160 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=8.mpi=72.hyper=2
16 mpiexec -np 1152 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 256 -v y 320 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=16.mpi=72.hyper=2
32 mpiexec -np 2304 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 512 -v y 320 -v z 320 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=32.mpi=72.hyper=2
64 mpiexec -np 4608 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 512 -v y 320 -v z 640 -v t 100 -in in.collide.steps -log log.sparta.date=23Dec17.model=collide.machine=serrano.pkg=kokkos_serial.kind=weak.size=16M.node=64.mpi=72.hyper=2