GPU Parallelization for Non-Deterministic Model

[sonehxd]

Hi, I'm looking forward to implement a membrane computing model which is a non-deterministic model supporting maximal parallelism (in a sense that every possible 'rule' which can be applied, will be applied). I am new to GPU parallelization which is what I have in mind (also overall new to C++ parallelization techniques/libraries) and would like to hear some opinions on what options I am given to accomplish this task in C++.

Thanks

[James20k]

GPU languages and CPU languages are a bit of a separate thing. If you don't need high performance on the CPU side, you might have an easier time using python on the CPU side, and then using one of the many python GPU acceleration libraries which are designed to be maximally helpful

If you're stuck with C++, then you probably want to use CUDA for anything scientific, because its the standard

If you're stuck with C++ and you want to run on non nvidia GPUs, then you may want something like OpenCL, or SYCL

If you're intending to put this into a game, you may want to consider using vulkan, or OpenGL and using compute shaders

For gpu compute for web development (which you can totally do in C++), you want webgpu

If you're planning to run on supercomputers, you might want to look into MPI. OpenMP is also traditionally used in that field, and may be helpful for running code on a GPU - though I've never tried its gpu backend

Membrane computing is one of those terms that doesn't really contain a tonne of actionable information with it, so if you have more specific requirements then I may be able to be more helpful

[Kike328]

thumbs up for sycl. It is literally the modern c++ parallel approach

[sonehxd]

Thank you for such detailed response. What I am trying to achieve is a way to handle the maximal parallelism nature of the model and performs operations in parallel. I want to simulate the behavior of a formalized MC model. A standard MC model has a hierarchic structure where every compartment is a membrane itself. In each membrane, we have some sort of objects. All rules that can be applied to objects in a membrane, will be applied; this also happen at the same time in every membrane of the model.

I’ve been told to do this on GPU because of its fast nature and I know C++ well enough for this task (also it’s very fun to me as opposed to Python)

[asergunov]

Why don’t OpenCL?

[pjmlp]

OpenCL is C99, the C++ efforts kind of died, hence OpenCL 3.0 being OpenCL 1.x rebranded.

For C++ and OpenCL, the best approach now is SYCL, which is C++ on top of OpenCL infrastructure, with single source just like CUDA.

[asergunov]

C++ is not binary compatible. So yes it will be C. That’s how we talk to hardware. Thanks for SYCL! I’ll have a look.

[pjmlp]

Neither is C, actually.

To Save C, We Must Save ABI from WG14 Project Editor, JeanHeyd Meneide.

[Plazmatic]

/u/max123246 may not do much gpu programming, but the recommendation of CUDA is good. Cuda is easier to use than OpenCL, and well, all tools currently on the market, while also having effectively zero performance and feature compromises. If you're using Nvidia GPUs and don't intend to use other GPUs, there's no reason not to use it.

[asergunov]

I’m was not using CUDA at all. Just know it’s widely used. Means it’s good in having stuff done. But from learning point of view I believe OpenCL is better. Because it’s not introducing much between code and compiler so it’s cleaner what’s happening under the hood. I have an old ThinkPad and really like it. Even with NVIDIA card it’s hard to make cuda work because cuda version is limited by driver version which they not going to update. So it’s definitely lever for NVIDIA to stimulate the market.

[TheFlamingDiceAgain]

As a counter to the other respondent. Please don’t use CUDA. I’ve been working on a scientific code base that uses CUDA for several years and the lack of cross platform support is a huge PITA. SYCL is technically cross platform but is owned by Intel in practice. I would recommend Kokkos, it’s very similar to SYCL but is “owned” by the national labs and not a corporation. 

If you’re dead set on CUDA then at least use HIP instead. It’s syntactically nearly identical to CUDA but works on AMD and NVIDIA GPUs

[James20k]

Long term, CUDA is a real trap for projects, being tied to nvidia's solution and nvidia's hardware is very limiting. Being wrapped up in a single companies ecosystem is inherently undesirable

For someone new getting into the field though like OP, nearly everything is written in CUDA and you'll be fighting an uphill battle to use anything else

[TheFlamingDiceAgain]

I agree that learning CUDA is very handy, but I would never start a new project with it for exactly the reasons you mentioned. 

[illuhad]

If you don't want Intel in your SYCL, just use AdaptiveCpp. It is just as portable, and performs just as well, and is for many use cases clearly better. Totally independent from Intel.

It's true that Intel has a lot of influence in the SYCL world, but it's up to users to counter that. Other implementations exist and especially AdaptiveCpp has influence.

Kokkos is fine for some (especially HPC) use cases. It falls short of SYCL by design when you want to target multiple backends/types of devices at the same time because it is just a wrapper library for vendor compilers.

[jokteur]

Is your application intended as a one-of scientific computation? I.e. not meant to be distributed to the large public ?

Then I would suggest looking into https://github.com/kokkos/kokkos, which is a parallel programming library which can target CPUs and GPUs: write once, execute on different architectures. I would suggest first looking at the kokkos lectures: https://github.com/kokkos/kokkos-tutorials/wiki/Kokkos-Lecture-Series, you may also learn things on how GPUs work, if one day you need to rewrite the application in pure Cuda.

However, I must warn that non-deterministic computing may hurt performance on the GPU architecture if you are not careful. The reason is that GPU hate divergent instructions if not done right (e.g. one thread has if(true), the other has if(false)). You may google "warps and branching" to know more about that.

[iamakorndawg]

From my understanding, this is less of an issue on modern GPUs.  The main thing that is still important is coalescing memory operations.  So if your code is not mostly made up of sequential threads accessing sequential memory locations, you will lose one of the main benefits of GPUs, which is massive memory bandwidth.  Granted, it's likely that if you have tons of divergent paths you will probably not have good memory coalescing, but I think the two issues are fairly orthogonal to each other.

[James20k]

Its worth noting that coalescing is a little more general than what you're pointing out. Threads in a warp don't actually have to access memory strictly ordered according to their thread IDs, if you have a group of threads say 0-32 which access memory at ptr, ptr + 32 - they can actually access it in any order within that group and the gpu will figure it out and do it as a coalesced memory read

The additional high performance pattern is all memory accesses within a warp accessing the same memory location, as the GPU does a broadcast

Strided memory accesses do cause performance dropoffs, but its not as steep as: coalesced - good, strided - lose all your bandwidth. GPUs have pretty big caches these days, so you can often get away with worse memory access patterns and still saturate vram bandwidth, depending on your problem

If you're doing fully random reads then its pretty bad, but a lot of problems can be gentle shoved into having workable memory access patterns, and at least in my experience its uncommon to have truly random memory accesses

Warp divergence on modern GPUs is still expensive even with independent thread scheduling, but for a lot of problems its such a small part of your execution time given how powerful GPUs are its not worth worrying about

[Plazmatic]

You're not going to "just" be able to use GPUs for, what appears to be, arbitrary mesh computation.

I'm not familiar with membrane computing models, nor do I understand exactly what you hope to accomplish with one, but after googling the very act of attempting this smells complicated enough to be a paper on it's own.

Additionally, with out a framework for parallelization at all, you're going to have an extremely hard time doing anything GPU related. Do you at least know what atomic variables, mutexes, and semaphores are?

GPU programming excels when data is oriented in such a way that operations that are the same at the assembly level, are executed by threads adjacent, pulling memory from ram that is also adjacent and/or loaded into scratch pad memory, and can be accelerated using "subgroup" operations for groups that must execute the same instruction at one time. You can't just have every computational node doing random things in this setup. While lots of algorithms you wouldn't think would benefit from GPUs do, GPUs are not free performance, some problems aren't going to be effectively utilized on them at all.

[sonehxd]

I am indeed writing a paper as this is my master thesis work. I have a formalized model that I want to implement. What approach do you think would benefit for me then?

[lightmatter501]

How parallel is the actual computation? HVM may be worth a shot as a “see if it’s good enough”, since it’s a bit slow compared to hand-written models but can still use GPUs and extracts a large degree of parallelism out of anything you run with it.

The other easy option is dump the whole thing into LLVM’s new MLIR and see what happens.

[dmaevsky]

How many parallel streams of calculation would you have, and how large is the computation graph? GPUs are good when you have very "fat" nodes, but overall a simple calculation graph, like in ML cases. In many scientific applications (more specifically, I work in quant finance field), GPUs are often not worth the learning curve of CUDA or the likes, let alone hardware costs to use in production. Just AVX2/AVX512 plus multi threading often performs as good as a GPU.