Strong scaling performance, Sphere benchmark, Broadwell, 8M particles
Performance in millions of particle-timesteps / second / node

Nodes

CPU (mpi,hyper)

Kokkos/OMP (mpi,hyper,thread)

Kokkos/serial (mpi,hyper)

1

236.3 (72,2)

223.5 (36,2,2)

250.4 (72,2)

2

253.6 (72,2)

255.5 (36,2,2)

261.6 (72,2)

4

267.6 (72,2)

240.3 (36,2,2)

264 (72,2)

8

296.4 (36,1)

228.9 (18,2,4)

252.5 (72,2)

16

328.5 (36,1)

223.3 (18,2,4)

311.4 (36,1)

32

268.3 (72,2)

160.8 (18,1,2)

228.8 (36,1)

64

204.3 (36,1)

104.2 (18,1,2)

177.3 (36,1)


Run commands and logfile links for column CPU

1

mpiexec -np 72 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=1.mpi=72.hyper=2

2

mpiexec -np 144 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=2.mpi=72.hyper=2

4

mpiexec -np 288 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=4.mpi=72.hyper=2

8

mpiexec -np 288 -npernode 36 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=8.mpi=36.hyper=1

16

mpiexec -np 576 -npernode 36 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=16.mpi=36.hyper=1

32

mpiexec -np 2304 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=32.mpi=72.hyper=2

64

mpiexec -np 2304 -npernode 36 --oversubscribe --bind-to core ./spa_serrano_cpu -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=cpu.kind=strong.size=8M.node=64.mpi=36.hyper=1


Run commands and logfile links for column Kokkos/OMP

1

mpiexec -np 36 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=1.mpi=36.thread=2.hyper=2

2

mpiexec -np 72 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=2.mpi=36.thread=2.hyper=2

4

mpiexec -np 144 -npernode 36 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=4.mpi=36.thread=2.hyper=2

8

mpiexec -np 144 -npernode 18 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 4 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=8.mpi=18.thread=4.hyper=2

16

mpiexec -np 288 -npernode 18 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 4 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=16.mpi=18.thread=4.hyper=2

32

mpiexec -np 576 -npernode 18 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=32.mpi=18.thread=2.hyper=1

64

mpiexec -np 1152 -npernode 18 --oversubscribe --bind-to socket ./spa_serrano_kokkos_omp -sf kk -k on t 2 -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_omp.kind=strong.size=8M.node=64.mpi=18.thread=2.hyper=1


Run commands and logfile links for column Kokkos/serial

1

mpiexec -np 72 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=1.mpi=72.hyper=2

2

mpiexec -np 144 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=2.mpi=72.hyper=2

4

mpiexec -np 288 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=4.mpi=72.hyper=2

8

mpiexec -np 576 -npernode 72 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=8.mpi=72.hyper=2

16

mpiexec -np 576 -npernode 36 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=16.mpi=36.hyper=1

32

mpiexec -np 1152 -npernode 36 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=32.mpi=36.hyper=1

64

mpiexec -np 2304 -npernode 36 --oversubscribe --bind-to core ./spa_serrano_kokkos_serial -sf kk -k on -pk kokkos reduction parallel/reduce comm classic -v x 128 -v y 80 -v z 80 -v t 100 -in in.sphere.steps -log log.sparta.date=23Dec17.model=sphere.machine=serrano.pkg=kokkos_serial.kind=strong.size=8M.node=64.mpi=36.hyper=1