Deposited in DRO : 23 July 2018 Version of attached le : Accepted Version Peer-review status of attached

We introduce a new code for cosmological simulations, PHoToNs , which has features on performing massive cosmological simulations on heterogeneous high performance Computer (HPC) and threads oriented programming. PHoToNs adopts a hybrid scheme to compute gravity force, with the conventional PM to compute the long-range force, the Tree algorithm to compute the short range force, and the direct summation PP to compute the gravity from very close particles. A self-similar space filling Peano-Hilbert curve is used to decompose computing domain. Threads programming is highly used to more flexibly manage the domain communication, PM calculation and synchronization, as well as Dual Tree Traversal on the CPU+MIC platform. The scalability of the PHoToNs performs well and the efficiency of PP kernel achieves 68.6% of peak performance on MIC and 74.4% on CPU platforms. We also test the accuracy of the code against the much used Gadget-2 in the community and found excellent agreement.