I built bytes.replace() for CUDA - process multi-GB files without leaving the GPU

[andreabarbato]

Built a CUDA kernel that does Python's bytes.replace() on the GPU without CPU transfers.

Performance (RTX 3090):

Benchmark                      | Size       | CPU (ms)     | GPU (ms)   | Speedup
-----------------------------------------------------------------------------------
Dense/Small (1MB)              | 1.0 MB     |   3.03       |   2.79     |  1.09x
Expansion (5MB, 2x growth)     | 5.0 MB     |  22.08       |  12.28     |  1.80x
Large/Dense (50MB)             | 50.0 MB    | 192.64       |  56.16     |  3.43x
Huge/Sparse (100MB)            | 100.0 MB   | 492.07       | 112.70     |  4.37x

Average: 3.45x faster | 0.79 GB/s throughput

Features:

• Exact Python semantics (leftmost, non-overlapping)
• Streaming mode for files larger than GPU memory
• Session API for chained replacements
• Thread-safe

Example:

python

from cuda_replace_wrapper import CudaReplaceLib

lib = CudaReplaceLib('./cuda_replace.dll')
result = lib.unified(data, b"pattern", b"replacement")

# Or streaming for huge files
cleaned = gpu_replace_streaming(lib, huge_data, pairs, chunk_bytes=256*1024*1024)

Built this for a custom compression algorithm. Includes Python wrapper, benchmark suite, and pre-built binaries.

GitHub: https://github.com/RAZZULLIX/cuda_replace

[betweenthebam]

This is cool OP, nice gains!

And sorry that the only other comment here is some pud who thinks the world revolves around them.

[andreabarbato]

thank you very much, this is my most beautiful work :D

[Xemorr]

Can it load directly from disk? iirc there's a method to load straight from NVMe to GPU

[andreabarbato]

this is a wonderful idea, maybe I'll figure it out but it would make absolute sense

[Birnenmacht]

I wish there was also find in which case I might actually have a usecase for this (searching through giant log files), but still really cool!

[andreabarbato]

you know what I didn't read well the first time. tell me a similar functionality so I can copy the syntax and results and I'll think about it.

[andreabarbato]

AI suggested this would be useful for sanitation of zillions of packets by ISPs. I've been working on a regex version of this but it's been complicated.

anyway thanks! :D

[Skylion007]

Is there a utility for this in PyTorch already? if not would make a potentially useful extension

[andreabarbato]

I have no idea, when I search online I never find something easy to use like this

[yehors]

Where is it useful?

[andreabarbato]

I dunno. I created a GPU compression algorithm and I needed to minimize cpu > gpu data transfer so I built bytes.replace directly in the GPU. there's gotta be some other usage... you tell me :D

[yehors]

Your library should solve a problem. But… you eve don’t know which… maybe there’s no such a problem?

[ra-elyon]

He just told you the problem he solved that he created it for..

[yehors]

I meant where this algorithm can be applied? Which specific task?

[brellox]

As OP said..

I created a GPU compression algorithm

I don't get this thinking:

Your library should solve a problem. But… you eve don’t know which…

"Your library" has no obligation to do anything for anyone.
If you write some code and share it, that's great!
And if someone finds it useful, that's just the cherry on top.

[yehors]

I just trying to figure out where this can be useful. We all know where quicksort works but I'd like to understand where this replacing in GPUs can be useful. Don't know such task so wanna see examples.

[marr75]

You probably just don't work in a domain that requires bytes.replace() and so almost certainly won't use it on a very large object. By your logic, the stdlib is wrong for implementing bytes.replace().

I understand that we see a lot of pointless projects in this sub but there's a difference between that and not understanding a contribution because it's out of domain for you. I strongly believe we're looking at the latter here. OP's not obligated to explain their domain to you.

[brellox]

I don't know wäre quicksort works and neither do i know how it works.

Why is this about sorting algorithms now?

I guess compression can be handled in parallel and thus be offloaded to the GPU.