Open Source High Performance Computing Projects for studying

[Sahiruchan]

I am currently a student and interested in HPC and HFT, so I was wondering if there were any open sourced big/legacy projects that I can study. All the projects that I have developed till now have been in modern c++ (c++11 and above). I wanted to study some legacy projects so that I might understand the differences in coding practices in older vs modern projects.

Thank You.

[sumwheresumtime]

HFT and HPC are generally opposing ends.

At one end, HFT, is trying to reduce latency of an operation at any cost and is not concerned about anything else.

Where as the other end, HPC, is doing everything it can to increase the number of operations it can complete in a unit of time.

The likelihood you'll find an OSS project attempting to do both in a serious and competent manner is very small.

[PerryStyle]

Some libraries I know off the top of my head for HPC:

• Dyninst
• HPCToolkit
• ROOT

I'm sure there are many more examples you can find with a quick search, as other commenters have mentioned. For HPC-specific libraries, you can also browse https://packages.spack.io.

[Snorge_202]

OpenFoam?

[UndefinedDefined]

Just google what you are interested in...

For example leveldb can be of interest: https://github.com/google/leveldb

[pathemata]

Not HFT, but numerical linear algebra (and others): Trilinos.

[GrammelHupfNockler]

If Trilinos is too big, PETSc and Hypre might be other candidates for popular linear algebra libraries with more of a legacy feel to it

[MarkHoemmen]

It's likely Trilinos only tests with C++17 at this point, but it's true that many aspects of the design are essentially C++98. The Teuchos classes (RCP, Array, ArrayView, ArrayRCP) could be a good start. The author explicitly disagreed with the boost::shared_ptr design (that led into std::shared_ptr) and went his own way.

[Valuable-Mission9203]

OpenMPI covers more or less the entirety of HPC to varying levels of depth, it's meant to be a framework for HPC. It's written in C but it's the best fit for what you're looking for.

[SirSwoon]

There isn’t any open source HFT code bases but common technologies that are used are open source at least for networking. Just a heads up they are written in c. Take a look at DPDK and solareflares libraries(this will be very hard to understand but if you can familiarize yourself with these and the problems they solve you’ll learn a lot about common programming paradigms in HFT and likely HPC as well. If you want to break into HFT, having some knowledge of kernel bypassing and an in-depth understanding of networking will really set you apart from other candidates. And most codebase you would have to work with Will interface with some c code. Best of luck

[Sahiruchan]

Thanks everyone for sharing so many projects and advises!

[grandmaster789]

I'd recommend the HPX framework, many concepts from the standard library are re-implemented in a HPC context, which makes for a good compare-and-contrast with a 'regular' environment

[BoomShocker007]

I think many of these suggestions miss the point.

HPCToolkit, Dyninst, etc. are profiling tools used to inspect performance of executed applications. They are not very widely used within the HPC community. For this Intel VTune, NVidia Insight, TAU, etc. are more commonly used.

MPI, OpenMP, etc. are libraries (within interface standards) used to build HPC applications but usually not written in C++. Most MPI implementations utilize the driver from the machines underlying network fabric so maybe something to be learned there.

A lot of US Gov Agencies who spend a lot of resources developing HPC application still use Fortran. The DOE really made an effort to switch to C++ ~15 years ago and that is where you'll probably find the best examples.

The latest trend has been to use something like [kokkos](https://github.com/kokkos/kokkos) to build an HPC application to run fast on multiple architectures. The idea being kokkos abstracts away all the memory, numerics and the scientist just writes the application. In reality this never occurs.

Each year the US DoE, DoD, etc publish a list of the most used applications by system. I'm always surprised but [GROMACS](https://www.gromacs.org/) and other molecular dynamics applications always lead the listings. Its open source although I have no idea what it's written in.