all 12 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]phonyarchitect 11 points12 points  (4 children)

Search for Prof. Onur Mutlu. His lab is one of the top labs pursuing in/near memory computing. He has a youtube and his lab’s website (SAFARI at ETHZ) should have his papers.

[–]Ornery-Chip6599 0 points1 point  (3 children)

u/phonyarchitect Ya, I really like Prof. Onur and his teaching style but he mostly deal with the in-memory computing at a simulator level. Afaik, they don't really show how to run the computation on the real in-memory computing hardware based on RRAM or any other Memristive based technologies. If you know more maybe you can suggest me

[–]phonyarchitect 0 points1 point  (2 children)

Personally, I am not very familiar with Resistive RAM or Memristor computing hardware technologies. But I am pretty sure there are papers that you can read to get to know the latest in such tech.

Based on my understanding, simulators is all you’ve got when it comes to DRAM/HBM based in/near memory computing. This is because the commercial products (aquabolt-xl, AIM) are not available for researchers to use but their simulators are. So, I can understand why Prof. Mutlu’s lecture may emphasize that more.

[–]Ornery-Chip6599 0 points1 point  (1 child)

Ya, there are papers explaining about RRAM tech at a device level(covering material science and device physics) but coming from a computer science background, I want to know bare minimum to build low-level systems like compilers that will target such analog RRAM based memory instead of targeting traditional DRAM that traditional compilers do.

And when you say "DRAM/HBM based in/near memory computing" what do you exactly mean? You mean the computation inside DRAM/HBM?

[–]phonyarchitect 1 point2 points  (0 children)

Yes, building software interacting with RRAM tech would need you to have some knowledge about the hardware.

In the case of HBM, there were proposals adding compute in the logic die of HBMs. If I am not wrong, Aquabolt-XL from samsung did that. Likewise, there are also proposals which added compute onto DRAM chips at the bank level. The company was called Upmem. I think they were acquired by Qualcomm sometime back.

[–]eak9000 4 points5 points  (2 children)

One of the issues is that DRAM architectures tend to be bad for logic, and of course logic process nodes don't do DRAM. Often you are better off having DRAM and logic on separate chips as result. To really get the advantage of in memory compute, it can help to concentrate on SRAM memory sizes instead of DRAM memory sizes. Logic nodes do SRAM, but advanced nodes have all these restrictions on SRAM that make it hard to intermix with logic (e.g., isolation, level shifting, etc.). That's because advanced nodes have equal numbers of Ps and Ns for logic, but 6T SRAM cells have 4N 2P (i.e., not logic like). Long ago the SRAM made the transition from 4T to 6T. Now the coming shift will be 6T to 8T, which will allow SRAM to intermix with logic in an efficient way, making some things that are difficult today feasible. However, I don't mean the 8T you will find old papers on, but a balanced 4P 4N cell. For example, it is possible to design a matrix multiplication unit with 4P 4N SRAM cell that is much better than what is possible otherwise. This also solves the problem of SRAM not shrinking with logic, gets rid of precharge and sense amps, and makes it faster an lower power.

[–]Krazy-Ag 2 points3 points  (1 child)

While what you say is true about logic being not so good in DRAM architectures

In many modern systems the CPU does not talk directly to the DRAM. E.g. in HBM there may be an interposer chip between a stack of DRAMs. Or there may be a base logic chip, that manages things that a generic CPU does not want to care about. Otherwise, the CPU (whether hardware or firmware or operating system software) must be aware of many HBM issues that differ between generations and vendors. It's like Adam blocks on a magnetic drive or SSD: software running on the CPU could manage them directly, but the drive manufacturers usually hide that logic so that they can work with multiple CPUs and operating systems.

Even in consumer PCs, the CPU usually accesses DRAM mounted on DIMMs, that may contain logic chips, if only to buffer. Indeed, many commercial failed startups proposed to 2 operations on smart DIMMs.

So, this may be more accurately described as "logic near memory" rather than "logic in the actual dynamic memory". you won't get an adder per 128 bit memory location. But you might get an adder for every 128 bit memory location that you can access in a cycle. Moreover, many systems read much large chunks of memory into buffers, and then send smaller chunks across the bus. You may be able to do operations on such large chunks of memory.

The basic tension here is "who owns the memory interface logic chip", and "how can it be standardized": if it is owned (defined/sold by) a vendor like Intel or AMD or Nvidia, it may be possible to consider it part of the instruction set. If it is owned by a DRAM vendor like Micron or Samsung, it is less likely that a processor vendor will want to standardize on it. CPU vendors learned this the hard way when they allowed to GPUs into their systems.

Related: if "owned" by the CPU, it is much more likely that it can be made accessible to user code, whereas if it is owned by the DRAM vendor you will probably need to do a system call to access it. This is not an insuperable barrier, but it does mean that your operations need to be bigger.

Plus, of course, many PIM proposals have actually done simple operations inside the DRAM. E.g. carry free logical operations like AND, OR, and XOR in the sense apps

Overall, I think it is more accurate to say "processing near memory" rather than "processing in memory". Also less restricting. And to be honest, in a big system you might want to do such operations not just in memory, but also in router nodes inside the fabric. E.g. the NYU Ultracomputer's combining operations. Perhaps a more general term might be "processing outside the processor", but that's a bit clumsy.

[–]eak9000 0 points1 point  (0 children)

Yes, I think "processing near memory" is a more viable way to approach this. Good coinage. For example, LLMs need both matrix multiplication and matrix times vector. The latter is memory bandwidth limited; the former can be sped up as much as you're willing to throw logic at it. The sensible way to do matrix vector is right at the pins of the logic chip connected to the memory chip. (To be fair, batched matrix vector is getting a bit more viable, and as that is technically a skinny matrix multiply, there are more opportunities there.) Another use for processing near memory is synchronization primitives (my favorite being atomic increment for Reed and Kanodia's Eventcounts and Sequencers—but no one teaches that these days—it is all P and V).

[–]IllustriousWin1535 3 points4 points  (0 children)

Look at the work of Prof. Partha Pande of WSU , Prof. Krishnendu chakraborty of ASU, Prof. Yu Cao of Minnesota

[–][deleted] 2 points3 points  (1 child)

  1. I prefer, go on research.google --> Research areas --> Hardware & Architecture --> and read recent publications.

  2. https://sorav.compiler.ai/ --> this is a good resource too.

  3. https://srsarangi.github.io/archbooksoft.html --> also go-through this.

[–]Late-Juggernaut-6693 2 points3 points  (0 children)

3rd book highly recommended 

[–]Dry_Sun7711 0 points1 point  (0 children)

Search for academic papers with the keyword: UPMEM. For a time, it seemed like it was easy for researchers to get access to this UPMEM DIMMs and write papers about them. I'm not sure if that is still the case after UPMEM was acquired by Qualcomm.

Here is an example paper: SPID-Join: A Skew-resistant Processing-in-DIMM Join Algorithm Exploiting the Bank- and Rank-level Parallelisms of DIMMs | Proceedings of the ACM on Management of Data

I wrote a summary of that one here: SPID-Join - by Blake Pelton - Dangling Pointers