Let's run Python on a Supercomputer
Speaker | David Perry |
---|---|
Time | 2017-08-04 14:15 |
Conference | PyCon Au 2017 |
Let’s run Python on a Supercomputer
Fast networking more important than fast CPUs.
Schedular. Slurm most popular. PBS.
Job types:
- Independant parallel jobs.
- Multi-core single-node jobs. Multi-threaded.
- Multi-core multi-node jobs. Multi-threaded with communication between nodes.
- GPU jobs.
OpenMP - Shared memory.
OpenMPI - Message passing.
Make Python go fast: Profiling, NumPy, PyPy, Numba, Cython.
Cloud HPC, AWS spot pricing.
In practise “good enough” is often sufficient, no need to get optimal performance.