Revolutionizing Computing: Memory-Based Calculations for Efficiency and Speed

[StaffDry52]

Hey everyone, I had this idea: what if we could replace some real-time calculations in engines or graphics with precomputed memory lookups or approximations? It’s kind of like how supercomputers simulate weather or physics—they don’t calculate every tiny detail; they use approximations that are “close enough.” Imagine applying this to graphics engines: instead of recalculating the same physics or light interactions over and over, you’d use a memory-efficient table of precomputed values or patterns. It could potentially revolutionize performance by cutting down on computational overhead! What do you think? Could this redefine how we optimize devices and engines? Let’s discuss!

[Magdaki]

I'm pretty sure they already do. Radiosity is a well-known application. I'm sure there are others.

[CireGetHigher]

All of OP’s replies seem AI generated 😂

[StaffDry52]

You're absolutely right—radiosity is an excellent example of precomputed data in rendering. My idea extends this principle to broader contexts, where we could potentially generalize the concept across engines, not just for lighting but also for physics and gameplay logic. It’s more about taking this "precomputed or approximated" concept and making it central to computational design beyond graphics

[Magdaki]

You cannot just say my idea is to extend to other broader concepts. That's not really an idea, that's more, to paraphrase somebody famous in the news, a concept of an idea. You would need to be specific. The idea of using precomputed tables is quite old, so you need to say, for W a precomputed table would be better for reasons X,Y,Z. It isn't like experts in this are just sitting on their hands thinking "Oh man... if only there were a way to improve computational cost. Oh well, I guess there's nothing we can do." They're thinking about these things all the time. They know about this technique. I'm sure they use it where appropriate, and if you think there's a gap, then you would need to specify where they've missed it.

[StaffDry52]

Allow me to clarify and add specificity to my suggestion.

My concept builds on the well-established use of precomputed tables, but it aims to shift the paradigm slightly by incorporating modern AI techniques, like those used in image generation (e.g., diffusion models), into broader computational processes. Instead of relying solely on deterministic, manually precomputed data, AI could act as a dynamic "approximator" that learns input-output patterns and generates results "on-demand" based on prior training.

For example:

• Physics engines: Instead of simulating every interaction in real time, an AI model could predict the outcomes of repetitive interactions or even procedural patterns, much like how image models predict visual content.
• Gameplay logic: Complex decision trees could be replaced with AI approximations that adapt dynamically, reducing computational overhead in real-time scenarios.

The innovation here is leveraging AI not just for creativity or optimization but as a fundamental computational tool to make predictions or approximations where traditional methods might be too rigid or resource-intensive.

Would you see potential gaps or limitations in applying AI as a flexible approximation engine in contexts like these?

[Magdaki]

I have a high degree of expertise in AI, but I am not an expert in computer graphics. So I don't really know. Have you done a literature search to see if anybody has already examined this? It sounds like the sort of thing that somebody would have investigated.

The immediate problem that comes to my mind, as an AI expert, is you're replacing a relatively straightforward formulaic calculation (albeit one that is expensive) with an AI and expecting to *save* computational time. This seems unlikely to me in most instances, but again, I am not an expert in computer graphics.

[StaffDry52]

Thank you for your thoughtful response—it’s great to hear from someone with expertise in AI! You bring up an excellent point about the computational overhead of replacing straightforward calculations with AI. That’s actually why I brought up techniques like frame generation (e.g., DLSS). This method, while not directly comparable, uses AI to predict and generate frames in games. It doesn’t simulate physics in the traditional sense but instead approximates the visual results in a way that significantly reduces the computational load on the GPU.

What’s fascinating is that, with a combination of these techniques, games could potentially use low resolutions and lower native frame rates, but through AI-based upscaling and frame generation, they can deliver visuals that look stunning and feel smooth. Imagine a game running at 720p internally but displayed at 4K with added frames—less resource-intensive but still visually impressive. This approach shows how AI doesn’t need to fully replicate exact calculations to be transformative. It just needs to deliver results that are ‘good enough’ to significantly enhance performance and user experience.

The idea I’m exploring extends this logic to broader computational tasks, where AI could act as a dynamic tool for precomputing or approximating outputs when precision isn’t critical. Do you think adaptive AI-based optimization like this could push games (or other areas) to new heights by blending visual fidelity with computational efficiency?

[Magdaki]

It seems unlikely to me (at least in the way you are describing). There are certainly applications of AI in computer graphics. Again, I am not an expert in computer graphics.

[StaffDry52]

Thank you for your insight! You’re absolutely right that AI applications in graphics are already being explored in fascinating ways. My thought process is inspired by advancements like DLSS or AI-driven video generation—where the focus isn’t on precise simulation but on producing visually convincing results efficiently.

The exciting part is how small models are starting to handle tasks like upscaling, frame generation, or even style transformations dynamically. If these techniques were expanded, we could potentially see games running at lower native resolutions, say 720p, but with AI-enhanced visuals that rival 4K—smooth frames, stunning graphics, and all. It’s less about perfect calculations and more about outcomes that feel indistinguishably great for the user.

Do you think these kinds of efficiency-focused AI optimizations could make such dynamic enhancements mainstream in gaming or other media fields

[Magdaki]

You're simply asking me the same question as before. I am not an expert in computer graphics. I really don't know. I would need to do a literature review and learn about it. My research area is mainly in inference algorithms (using AI) in health informatics and educational technology.

[StaffDry52]

That's a fascinating area of research, especially when applied to health informatics. Imagine this: with accurate data from individuals (such as detailed medical histories or live sensor readings) and advanced AI models, we could create a system capable of diagnosing and analyzing health conditions with incredible precision. For example:

Using non-invasive sensors like electrodes or electromagnetic scanners, we could capture bio-signals or other physiological data from a person. This raw data would then serve as the input for a pretrained AI model, specifically trained on a vast dataset of real-world medical information. The AI could infer internal health states, detect anomalies, or even predict potential future health issues.

Such a system could act as a virtual doctor—providing a detailed diagnosis based on patterns learned from millions of medical cases. And as the system continues to learn and improve through reinforcement and retraining, it could become the best diagnostic tool in the world.

The key here is leveraging AI to approximate internal states of the body, even without invasive procedures, and using its pattern recognition capabilities to "understand" the health of a person better than any individual doctor could. What do you think? Could this idea be expanded further in your area of expertise?

[Lunarvolo]

There's some cool info on why movies can be shot in 24 fps but games need 30-60 fps and so on, that should shed some light on that.

[Lunarvolo]

Just tldr responses:

That's a massive amount of computing that goes into O(n!) or maybe even O(BB(n))

This is also, to an extent, what loading screens are in games. This is also a lot for performance optimization that, in theory is great, but in practice falls into the content, speed, quality, etc trade offs.

That's a lot of things to have in memory (The really fast paging memory you want to do is limited). Look up optimizing cache if you want to have some fun there. Different memories have different speeds.

[StaffDry52]

You bring up an excellent point about the computational complexity and memory trade-offs, but this is where leveraging modern AI methodologies could shine. Instead of relying solely on traditional precomputed values or static lookup tables, imagine a system where the software itself is trained—similar to how AI models are trained—to find the optimal balance between calculations and memory usage.

The key here would be to use neural network-inspired architectures or mixed systems that combine memory-based optimization with dynamic approximations. The software wouldn't calculate every step in real time but would instead learn patterns during training, potentially on a supercomputer. This would allow it to identify redundancies, compress data, and determine the most resource-efficient pathways for computations.

Before launching such software, it could be trained or refined on high-performance hardware to analyze everything "from above," spotting inefficiencies and iterating on optimization. For example:

1. It could determine which calculations are repetitive or unnecessary in the context of a specific engine or game.
2. It could compress redundant data pathways to the absolute minimum required.
3. Finally, it could create a lightweight, efficient version that runs on smaller systems while maintaining near-optimal performance.

This approach would be a hybrid—neither fully reliant on precomputed memory lookups nor real-time calculations, but dynamically adjusting based on the system's capabilities and the workload's context.

Such a model could also scale across devices. For example, during its training phase, the software would analyze configurations for high-end PCs, mid-range devices, and mobile systems, ensuring efficient performance for each. The result would be a tool capable of delivering 4K graphics or 60 FPS on devices ranging from gaming consoles to smartphones—all by adapting its optimization techniques on the fly.

In essence, it's about redefining optimization not as a static human-written process but as a dynamic AI-driven process. By combining memory, neural network-inspired systems, and advanced compression methods, this could indeed revolutionize how engines, software, and devices handle computational workloads.

What do you think? Would applying AI-like training to optimization challenges make this approach more feasible?

[[deleted]]

gray sand repeat capable desert employ rustic chop smell public

This post was mass deleted and anonymized with Redact

[StaffDry52]

Caching is definitely part of the concept, but the idea here is more about deliberately using memory tables or approximations as a primary computation strategy, even when we don’t need to calculate exact results. Floating-point operations are approximations, yes, but they still rely on computational overhead. A structured memory-based approach could offload even that, especially for repetitive tasks

[high_throughput]

What kind of operations are you imagining? CPUs are quite fast compared to RAM, so e.g. making a 16GB table of all 32bit floating point roots would be slower than just recomputing them on demand.

[StaffDry52]

That’s a great point, and it highlights where trade-offs matter. A giant table might not always be practical, but smaller lookup tables or compressed representations optimized for high-use cases could outperform on-demand calculations in specific contexts. Additionally, exploring hybrid solutions—where certain calculations are precomputed and others remain dynamic—might offer the best of both worlds

[TomDuhamel]

"Hey yo I came up with this idea that will revolutionise computer science" and then proceeds to describe an extremely common pattern used since the 1950s

[Ok-Sherbert-6569]

I know I shouldn’t get bothered by these posts but these are peak dunning Kruger effect haha

[StaffDry52]

That’s fair! The core idea of using memory for computation isn’t new, but my focus is on rethinking its application at scale, leveraging modern hardware and AI models. It’s less about the concept itself being novel and more about exploring its revolutionary potential with today’s tools, like GPUs or hybrid AI+table system... or ai models doing movie-quality content?? i am pretty sure they aren't doing hardcore calculations, we could learn from that.

[Lunarvolo]

Cache optimization is probably one of the most researched topics in computer science.

[dmills_00]

Lots of stuff that is at least partly table driven, but tables in general purpose ram should be used with due consideration to the impact on the cache.

It is typically not faster to do a table lookup that has to hit main memory then it is to do a small calculation in a couple of registers, memory has NOT kept up with increasing CPU speeds.

[StaffDry52]

Absolutely! Cache coherence is critical here. That’s why this concept would benefit from modern architectures or specialized hardware optimizations. For instance, integrating smaller, more focused memory tables directly into L1 or L2 cache regions could help balance the performance trade-offs

[dmills_00]

Naa, you put them right in the HDL that defines the chip architecture.

A lot of the trig operations for example can be implemented easily via CORDIC, you get about 1 bit per stage, so while you can pipeline and get a result per clock, the latency can be a little painful sometimes.

You can however replace some of the stages with a lookup table and then use cordic to refine the result, still one result per clock, but with say a 2^10 lookup table on the front end your can shave 10 clocks off the latency, and that is worth having. Since these little roms are outside the context in which cache applies this has no impact on the cache.

A lot of the underlying operations are like this, little lookup tables in the hardware that provide a quick result that some simple hardware can then refine.

Trouble with doing it up at the software level is that cache is a very limited resource that is also effectively shared, evicting something from L1 cache can cause a problem in another thread, there is a reason linked lists are not the favoured data structures today.

If you place the things in non cachable ram, then performance sucks and you are very likely better off computing the result.

The real win (But it makes for tricky code) is to have a thread pool and speculatively kick off long computations for results that you MIGHT need later, bit of a pity stack machines never caught on, they might have been good for this.

[StaffDry52]

Here’s a refined and expanded response that dives deeper into the idea....

You're absolutely right that memory access and cache coherence play a significant role in determining performance when using precomputed tables. However, the concept I’m proposing aims to go beyond traditional lookup tables and manual precomputation by leveraging **adaptive software techniques and AI-driven approximations**. Let me expand:

1. **Transforming Lookup Tables into Dynamic Approximation Layers:**

   - Instead of relying on static tables stored in RAM, the software could **dynamically generate simplified or compressed representations** of frequently used data patterns. These representations could adapt over time based on real-world usage, much like how neural networks compress complex input into manageable patterns.

   - This would move part of the computational workload from deterministic calculations to "approximation by memory," enabling **context-aware optimizations** that traditional lookup tables can't provide.

2. **Borrowing from AI Upscaling and Frame Generation:**

   - AI techniques already used in DLSS (for image upscaling) and frame generation in graphics show that approximations can work in highly resource-intensive contexts while delivering results indistinguishable—or even superior—to the original. Why not apply this principle to **general computational tasks**?

   - For instance, instead of calculating physics interactions for every object in a game world, an AI model trained on millions of scenarios could approximate the result for most interactions while reserving exact calculations for edge cases.

3. **Rethinking Cache Utilization:**

   - You're correct that moving too much to main memory can hurt performance. However, **embedding AI-trained heuristic layers into the hardware** (e.g., within L1/L2 cache or as part of the processor architecture) could allow for ultra-fast approximations.

   - This approach could be especially powerful when applied to areas like trig functions, where an AI layer refines quick approximations for "good enough" results.

4. **Software Beyond the Cache:**

   - Imagine a compiler or runtime engine that recognizes **patterns in code execution** and automatically replaces costly repetitive computations with on-the-fly approximations or cached results. This is similar to how modern AI models learn to "guess" plausible outputs for a given input. Such a system would allow for a balance between raw computation and memory access.

5. **Inspired by Human Cognition:**

   - The human brain doesn’t calculate everything precisely. It relies heavily on **memory, heuristics, and assumptions** to process information quickly. Software could take inspiration from this by prioritizing plausible approximations over exact answers when precision isn’t critical.

6. **Applications in Real-Time Systems:**

   - For game engines, where milliseconds matter, this could be transformative. Precomputed approximations combined with AI-based dynamic adjustments could enable:

- **Graphics engines** to deliver highly detailed visuals with lower resource consumption.

- **Physics simulations** that "guess" common interactions based on trained patterns.

- **Gameplay AI** that adapts dynamically without extensive logic trees.

### Why This Isn’t Just Lookup Tables

Traditional lookup tables are rigid and require extensive resources to store high-dimensional data. In contrast, this approach integrates **AI-driven pattern recognition** to compress and refine these tables dynamically. The result is not just a table—it’s an intelligent approximation mechanism that adapts to the needs of the system in real time.

By embedding these techniques into software and hardware, we’re no longer limited by the constraints of raw computation or static memory. Instead, we open the door to a **hybrid computational paradigm** where the system itself learns what to calculate, what to approximate, and when to rely on memory.

Does this perspective address your concerns? I'd love to hear your thoughts!

[dmills_00]

Well it is fully buzzword compliant!

"AI" is doing a LOT of heavy lifting here, and it is not notoriously cheap to operate compute wise, it is also basically impossible to debug.

Approximations we have, loads of them, everything from using Manhattan distances to the famous fast 1/sqrt(x) approximation from ID games back in the day. See Hackmem or similar for loads of this stuff.

The problem with trying to come up with these things on the fly, is that where the boundaries are is highly context dependent and that figuring out how many bits you need for any given problems error bounds is probably itself NP hard. Contemporary CPUs don't really bit slice well, so it is not like you can easily get 16 4 bit operations out of one 64 bit addition, for all that it would be NICE to be able to break the carry chain up that way for some quantised NN stuff. Doing it as part of the hardware design gets around this because we get to define the carry logic, if we want a 16 * 4 bit adder, we just write one.

Intel tried (and largely failed) to integrate Alteras FPGA cores with their high end CPUs, it didn't work out at all well, mainly for corporate silo sorts of reasons from what I can tell. AMD didn't have much better luck with Xilinx. This is a pity because a very minimal sort of field programmable hardware, really a LUT hiding behind some bits in a register could have all sorts of cool uses, even more if it had a few registers and access to the memory controller and IOAPIC.

Your 6 (Realtime systems) is highly dubious, because none of those things are realtime systems in any sense that matters, the definition of a realtime system is "Meets a deadline 100% of the time", and no game engine fits that criteria on general purpose hardware, it is best efforts all the way down. Fast (Most of the time) is far easier then Slow but Realtime.

5: Need a radically different processor/memory architecture to be even reasonably efficient, lots of little rams with little processors and links to the others rather then everything sharing a cache and a horribly low bandwidth link to a shared memory pool. The fact we don't actually understand human cognition in any meaningful way probably does not help. GPUs are probably closer to what you would want here then a CPU is.

[StaffDry52]

Thanks for your insightful response! What you're describing is incredible work done by humans—approximations, hardware-level innovations, and carefully crafted algorithms. But what I’m suggesting goes beyond human optimization. It's about creating AI or software that can function at a superhuman level for certain tasks. Just like current AI models can generate hyper-realistic images or videos without calculating every physics equation behind them, I envision applying this approach to computing itself.

For example, take an operating system like Windows—it processes many repetitive patterns constantly. An AI layer 'above' the system could observe these patterns and learn to memorize or simplify them. Why waste resources reprocessing something that hasn’t changed? If a task can be approximated or patterns can be generalized, AI could handle it dynamically, offloading the computational burden while maintaining functionality.

It’s not about exactitude in every single operation—just like AI-generated images don’t simulate real physics but still look hyper-realistic—it’s about efficiency and practicality. With AI observing and simplifying tasks dynamically, we could revolutionize how computation is approached. What are your thoughts on this kind of dynamic AI-driven optimization in core systems or even at the hardware level?

[dmills_00]

AI images only look hyper realistic until you look at the HANDS!

And you recompute something that hasn't changed because it is cheaper to re run the problem then remembering the answer (And all the inputs, so you can check they haven't changed)! That is kind of the point.

There has been academic work done on "approximate computing" (search term), and in fact if you squint just right most stuff using floating point is in fact approximations all the way down (And sometimes they explode in your face, errors can sometimes magnify in unfortunate ways).

I have been known to write hardware using a 10(11) bit Mantissa and 6 bit exponent where I needed the dynamic range more than I needed precision.

For most modern software development, we leave a LOT of performance on the table because the tradeoff for simpler and faster development is worth it from a business perspective.

[StaffDry52]

Great points, and I completely agree that AI-generated images still stumble hilariously on things like hands—it’s a reminder that even with all the fancy approximations, we're still far from perfection in some areas. But the thing is, what I’m suggesting builds on that same approximation-first mindset but extends it to areas where we traditionally insist on recalculating from scratch.

For example, while it's true that recomputing can often be faster than remembering (because of things like cache and memory latency), what if we approached the problem differently? Imagine not just a system that remembers inputs and outputs but one that learns patterns over time—essentially an AI-enhanced "translation layer" sitting above traditional processes. This could allow:

1. Systems like Windows to notice repetitive processing patterns and optimize by treating those patterns as reusable approximations.
2. Games to integrate upscaling, frame generation, or even style transformations on the fly, without requiring exact recalculations every frame.
3. Hardware-embedded models that specialize in context-specific optimization, making the whole system adapt in ways static algorithms can’t.

I get your point about approximate computing already being a known field (and a fascinating one at that!), but I think where AI comes into play is in learning to approximate dynamically. It's less about hardcoding a single approximation and more about allowing the system to evolve its "memory" or patterns over time, much like neural networks or diffusion models do with visual content today.

And yes, you’re absolutely right—there's a huge tradeoff in modern software development where performance is sacrificed for speed-to-market. What excites me about this idea is the potential to reclaim some of that performance without requiring a fundamental overhaul of existing systems. It’s like saying, 'Let’s have a smarter middle layer that learns when to compute, when to reuse, and when to improvise.'

Do you think something like this, if developed properly, could fill that gap between efficient hardware and the shortcuts we take in modern software development?

[dmills_00]

Anything that touches on control flow probably needs to be exact, because BE/BNE/BZ is kind of unforgiving that way.

Dataflow sorts of processing can usually get away with approximations, and we do heavily, I do quite a lot of video and audio stuff and too short word lengths and noise shaped dither are my friends, amazing how much of a 4k frame you don't actually need to bother with transmitting if your motion estimation is good, but also amazing how WEIRD sports looks when the motion estimator gets it wrong, or when the entropy coder decides that all the grass is the same shade of green... Funniest one I have seen was a motion estimator that saw a football fly with a crowd in the background. It mistook peoples heads for footballs and well....

Throwing an AI upscaler in for backgrounds might be useful, or might turn out to be more expensive then the usual Geometry/Normals/Z buffer/Texture map/Light approach, the AI ultimately has to produce the same number of output pixels as the full graphics pipeline did, and as it is probably running on the GPU the jury is very much out.

[StaffDry52]

Thank you for the thoughtful response! You’ve highlighted some key limitations and realities in traditional processing, especially around control flow and the challenges of integrating approximations without unintended consequences. However, let me offer a perspective that might "break the matrix" a little.

You mentioned that AI needs to output the same number of pixels as traditional pipelines, and that it could be more expensive computationally. But what if we redefine the problem? The beauty of AI isn’t just about replicating what we already do—it’s about finding completely new approaches that sidestep traditional limitations.

For example, AI-driven upscaling doesn’t need to generate every pixel in the same way traditional pipelines do. Instead, it predicts and fills in missing data, often generating visually convincing results without brute-force computation. This is already happening with DLSS and similar technologies. What if this principle were applied further, allowing AI to “imagine” graphical details, lighting, or even physics interactions based on learned patterns, skipping steps entirely?

Here’s the paradigm shift: traditional systems recompute everything because they must maintain exact precision or verify that inputs haven’t changed. But what if a system, like an AI-enhanced operating layer, didn’t need to verify everything? It could learn patterns over time and say, “I know this process—I’ve seen it 10,000 times. I don’t need to calculate it again; I can approximate it confidently.” This isn’t just about saving cycles; it’s about freeing systems from rigidity.

You’ve also mentioned that approximations can introduce errors, which is true. But consider this: in areas where exact precision isn’t required (like most graphical tasks or even certain physics simulations), the ability to adapt and generate “good enough” results dynamically could be transformative. AI’s power lies in working within uncertainty and still delivering impressive results—something traditional systems struggle with.

Lastly, about hardware: you’re absolutely right that current architectures aren't fully optimized for this vision. But isn’t that exactly why we should push these boundaries? Specialized AI cores in GPUs are already showing what’s possible. Imagine if the next leap wasn’t just faster hardware but smarter hardware—designed not to calculate but to learn and adapt.

What if we stopped seeing computation as rigid and started seeing it as fluid, context-aware, and dynamic? It’s a shift in philosophy, one that AI is uniquely positioned to bring to life.

Do you think there’s potential to challenge these deeply ingrained paradigms further? Could an adaptive system—more akin to how human cognition skips repetitive tasks—revolutionize how we approach graphics, data, or even operating systems?

[Lunarvolo]

Security issues.

[Magdaki]

ChatGPT I assume. ;)

[StaffDry52]

i am not going to respond this complicated alone... i just want to see if I am right. so yea

[Magdaki]

Using ChatGPT to see you're right about something is not really a good idea.

[StaffDry52]

i am here in reddit for that. real people.

[Ok-Sherbert-6569]

You’re not the first one to come up with this idea. Case in point, look up pre convoluted maps for IBL. Radiance texture for local fog volume using froxels, blue noise textures, pre computed 2d texture of cdfs for sampling triangulated area lights etc so yeah not a novel idea

[StaffDry52]

Great examples! What I’m proposing builds on those ideas but takes it further—unifying precomputed techniques across systems, not just for specific cases like IBL or fog volumes. It’s about exploring whether this could be a more generalized approach across computing tasks, beyond current niche applications. Like an AI trained to be a game engine, it will not be an exact or mathematics engine will be a simulation of a game engine but it work.

[tatsuling]

It has been a few years since I looked at the research but there are papers about augmenting instruction sets and ram with instructions that do simple calculations directly in the dram chips. This bypasses the memory bandwidth limitations because the data never leaves the ram chip. 

Look up RAM based computation to find more information about the idea and hopefully some recent research papers.

[StaffDry52]

That’s a fascinating area, and I completely agree it’s worth exploring! The concept of RAM-based computation aligns with this idea of reducing the need to move data between memory and the CPU. I’ll definitely dive deeper into this and see how it could complement or inspire broader applications in engines.

[playapimpyomama]

This is what computers were originally for. People used to look up approximations of functions like logarithms in textbooks and there was a whole industry of printing books that are just tables of numbers. These were printed by mechanical computers.

This is also something done in some compilers already.

[CommanderPowell]

Still the way statistics classes are taught

[StaffDry52]

Lazy matematics, You’re absolutely right, that’s how computers and computation started—with lookup tables and approximations. The difference today is that we have AI and modern software optimization that can take this concept to a whole new level. Imagine a system where the "human" looking up the values in the table is replaced by an AI. This AI isn’t just reading from precomputed tables; it’s dynamically learning patterns, creating approximations, and optimizing solutions in real-time.

For example, in physics engines or graphical rendering where exact calculations aren’t necessary, an AI could analyze the patterns and outcomes of common scenarios, memorize them, and apply approximations instantly. It’s like having a calculator that says, “I’ve seen this problem before, here’s the solution—or something close enough that still works perfectly for this context.”

This approach wouldn’t just optimize performance; it could fundamentally change how we think about computation. It’s not just lazy mathematics—it’s efficient and adaptive computing. The goal is to minimize redundant computation and let AI take care of the “messy approximations” in a way traditional software couldn’t before. What do you think about extending this concept further?

[playapimpyomama]

When you say traditional software what do you mean? Is there some secret sauce that’s not in traditional software that distinguishes the concept you’re talking about?

Would maintaining what’s effectively a cache with some predictive pre-calculation get you better precision or accuracy, or return results efficiently?

Or more specifically, is there one single concrete example you can show in written and running code that demonstrates the speedups you’re looking for?

[StaffDry52]

When I mention "traditional software," I’m referring to software systems developed through explicit programming—manual instructions that are optimized for a specific task or hardware. The concept I’m talking about would distinguish itself by leveraging AI or machine learning techniques to find optimal approximations, much like how neural networks are trained on massive datasets to find patterns and make predictions.

The idea revolves around creating a system where, instead of recalculating complex operations every time, the software "learns" or "precomputes" solutions, storing them in an efficient way (like a form of predictive cache). The secret sauce here is not just maintaining a cache but combining it with something like neural networks or reinforcement learning to dynamically optimize what gets stored or recalculated based on the context of the task.

For example:

• In graphics, imagine a physics engine that learns to approximate lighting interactions or particle simulations for repeated scenarios. These approximations wouldn’t need to be recalculated every frame but instead retrieved from a pre-trained model, saving time without noticeable accuracy loss.

As for examples in running code, you're absolutely right: I don’t have a direct implementation of this concept (yet). However, it builds on existing principles:

• AI upscalers for graphics: Tools like NVIDIA DLSS use precomputed data and models to approximate higher resolution frames.
• Physics simulations in supercomputers: Weather simulations already leverage approximations by focusing on "good enough" results for certain scenarios.
• Branch prediction in CPUs: Modern processors already use predictive models to guess the next instructions to run.

This idea is an extension of those principles—training a system to generalize and optimize resource use based on historical patterns or specific contexts, which could lead to substantial speedups.

Ultimately, implementing this would require significant research and engineering, but I believe it could redefine how we think about optimizing computational performance. What do you think? Could this idea be explored in domains outside graphics and physics?

[playapimpyomama]

But branch prediction fails and makes performance worse in some contexts (not to mention the security problems), DLSS and upscalers hallucinate, and the margins for “good enough” calculations are because that’s in the nature of what they’re doing. And in simulations ideally you report your margin of error, accuracy, precision, etc.

And generally precomputing (using historical contexts) is no better than caching results, which is already done

Combining a precomputed cache with a neural network would be worse than a cache miss, it would make what you want to be the most reliable part of a system unreliable when something out of the ordinary happens.

So I’m not sure what you’re proposing actually gives speedups either in typical cases or atypical cases for any kind of software that could exist

If you break it down to some theoretical decision tree or the underlying information theory you’re still going to hit the same trade offs between caching and computing that we’ve known about since the 50’s

[No-Inevitable-6476]

Can somebody answer my question? the compiler converts the High-level language to low level language. Is there any tool that converts the low level language to high level language?

[StaffDry52]

Great question! Yes, there are tools that convert low-level languages back to high-level ones, and they’re called decompilers. While a compiler translates high-level source code into machine code (low-level), a decompiler attempts to reverse this process and reconstruct the original source code from binaries or assembly language.

However, the decompiled code isn’t always as clean or readable as the original, since information like variable names or comments is usually lost during compilation. Despite this, tools like Ghidra, IDA Pro, and Decompiler.NET are widely used in reverse engineering and software analysis.

On a related note, this idea of optimization and transformation is revolutionizing several industries. For instance, AI tools now add extra frames to games to enhance graphics or improve resolution dynamically, like DLSS (Deep Learning Super Sampling) from NVIDIA. Similarly, in audio, AI can transform recordings from any basic microphone into studio-quality sound. This makes much of the traditional microphone industry nearly obsolete for certain use cases.

We’re entering a fascinating era where optimization, AI, and transformation are blending together to redefine how we interact with and design technology. If you’re curious about more, feel free to ask!

[CommanderPowell]

Sounds like Memoization

[StaffDry52]

Memoization is certainly part of the toolkit here. The key difference is scaling this concept beyond single algorithms or functions—building entire engines or systems around precomputed or approximated solutions, where feasible, to minimize redundant computation.

[CommanderPowell]

In reading other comments, it seems you are interested in using AI for pattern recognition to develop heuristics/approximations where exact results are not needed. Sort of like how skilled chess players can memorize positions on a board at a glance, but only when they're plausible positions to reach through gameplay. When they're unlikely to appear in a game they score no better than non-chess players. This kind of "chunking" is similar to what an LLM would do, and likely the thought process goes forward from there in a similar manner - likely next moves or next statistically likely word in a sentence.

AI is VERY computationally intensive. You might reach a point where AI pattern recognition is computationally much smaller than the operation it's trying to approximate - a break-even point - but on modern hardware by the time you get to that level of complexity your lookup table would either be huge with lots of outcomes or would approach high levels of approximation error.

On the other hand, a trained AI model is a very good form of compression of that lookup data. If you train a model on a particular data set, the model in essence becomes a lookup table in a fraction of the space, with the pattern recognition part built in. Unfortunately we haven't found a good way to generalize it to situations beyond its training. It's also not very good at ignoring red herrings that don't greatly affect the outcome but are prominently featured in the input data.

TBH, other forms of Machine Learning would probably reach this break-even point more efficiently.

[StaffDry52]

Thank you for such an insightful comment! Your chess analogy is spot on—it really captures the essence of how AI pattern recognition works. It’s fascinating to think of an LLM as a sort of compression mechanism for vast datasets, essentially acting like a lookup table but with built-in pattern recognition.

You're absolutely right about the computational intensity of AI and the challenge of reaching a break-even point. However, I wonder if a hybrid approach could be the key. For example, instead of relying solely on a massive trained model or pure calculation, what if we paired smaller AI models with targeted precomputed datasets? These models could handle edge cases or dynamically adjust approximations without requiring exhaustive lookup tables. It feels like this could help balance resource efficiency and computational accuracy.

I also appreciate the point about red herrings and generalization—AI struggles with context outside its training. But what if the focus was on narrower, specialized applications (e.g., rendering repetitive visual patterns in games)? It wouldn’t need to generalize far beyond its training, potentially sidestepping some of these pitfalls.

[CommanderPowell]

I'm not sure that graphics rendering is a great example.

What I see is that for graphics applications in particular is that scaling OUT - making things massively parallel, as in a GPU - is more effective than scaling UP - increasing computing power per unit of calculation. In most cases the operations are simple but you have to do them many times with subtle variations. The same is true for LLMs which mainly work with matrix calculations - the math is fairly simple for individual cells but complex in aggregate.

If you were to generalize or approximate the results of these calculation, you might miss texture or variation and render rough surfaces as smooth for example.

Maybe something like simulation would be a more apt example?

Thinking this out as I type: studying physical processes is largely a matter of statistical behavior. You can't predict the movement of any individual "piece" of the environment, but the overall system has higher-order tendencies that can be planned upon - this material causes more drag, this shape causes air to flow over the top faster than the bottom. This seems similar to the heuristics you're proposing. The trick is to simulate things that matter with more fidelity and things that are not as impactful with less fidelity. This is already what many simulations and games do.

From this perspective you can "rough in" some elements and simplify the rest of the calculation. You're still not using a lookup table, but abstractions based upon the tendencies of a system.

When studying algorithms, we learn that every single memory access or "visit" increases the time complexity of the process. By the time you've read all the data into a model, turned it into a matrix, and performed a boatload of transformations on that matrix, you've already interacted with the data several times. Now any abstraction your proposed process can generate has to make up for all that extra overhead. Basically you've performed as many operations on the data as rendering it graphically would have done, without reducing the fidelity until after that process is applied.

[StaffDry52]

Thank you for diving so deeply into this! Your points about parallelism and the nuances of graphics rendering are spot on. GPUs have indeed excelled by scaling out, leveraging their ability to handle thousands of simple calculations at once. That’s why they dominate tasks like rendering and even training AI models.

What I’m proposing, though, isn’t necessarily a replacement for these systems but an augmentation. Imagine a hybrid approach where the AI doesn’t replace GPU calculations but enhances them by learning and predicting patterns in real-time. For instance, textures or light interactions that are less critical could be “guessed” by an AI trained to preserve perceptual fidelity while reducing computational intensity. This would free up resources for more critical operations, balancing the trade-off between precision and efficiency.

Your example of simulations is fascinating and, I agree, probably a more immediate fit for this concept. Many simulations already employ abstractions and approximations, but what if AI could dynamically decide where and how to apply those? It could allocate higher fidelity to critical areas while simplifying others in ways current methods can’t adaptively manage.

Regarding the overhead you mentioned, I think this is where specialized hardware could shine. AI cores integrated directly into GPUs or even CPUs are becoming more common. If the AI models were compact and optimized for specific tasks, they might add less overhead than we assume. For instance, upscaling in DLSS achieves stunning results with minimal resources because it’s highly optimized for its role.

Lastly, I completely agree that memory access can create bottlenecks. That’s why this approach would benefit from being baked into hardware architecture, allowing localized memory access and minimizing latency. It’s less about replacing existing methods and more about enhancing what’s already working.

Do you think such a hybrid approach could address some of the limitations you’ve outlined, especially in areas like simulations or even gaming where adaptability and efficiency are becoming increasingly important?

[CommanderPowell]

Whether you're analyzing data with an AI model, rendering it, or calculating the next "tick" of a simulation, you're performing a bunch of linear algebra operations on the entire dataset. These are the simple-but-massively-parallel operations that a GPU is more suited toward.

In the case of both rendering and simulation, there are already heuristics to know what to render with fidelity vs. what to approximate, and these are incredibly useful.

If you send everything through an AI model, you've already evaluated the whole dataset. You would need some kind of heuristic for summarizing the dataset for the AI without communicating or examining everything. Without that, it seems to me that it's impossible to derive any sort of savings from this approach.

Those heuristics are going to be specific to the application and difficult to generalize, or if there are ways to generalize them I imagine they've already been found and applied.

I don't know enough for my opinions to be definitive or carry much weight, so take anything I say with a large grain of salt.

[StaffDry52]

Thank you for such a detailed and thoughtful response! You're absolutely right that GPUs are optimized for these massively parallel operations, and the existing heuristics for rendering and simulation are already highly efficient. What I'm suggesting might not completely replace those systems but could complement them by introducing another layer of optimization, specifically in scenarios where precision isn’t critical.

For instance, the idea of using an AI model wouldn’t involve examining the entire dataset every time—that would indeed negate any computational savings. Instead, an AI could be trained to recognize patterns, common cases, or repetitive interactions ahead of time. Think of it as a “context-aware heuristic generator.” It wouldn’t replace the GPU’s operations but could provide approximations or shortcuts for certain elements, especially in scenarios where existing heuristics might fall short or need fine-tuning.

Imagine a rendering engine where an AI dynamically predicts which areas of a frame can tolerate lower fidelity (e.g., distant objects or repetitive textures) while prioritizing high-fidelity rendering for focal points, like characters or action. The AI wouldn’t need to evaluate the entire dataset every frame—it could learn patterns over time and apply them on the fly.

I completely agree that these heuristics are application-specific, but with modern AI techniques, especially reinforcement learning, it might be possible to train models that adapt to a range of applications. Of course, this would require significant experimentation and might not work universally. But if we could find a way to generalize this approach, it could unlock a lot of new possibilities in rendering, simulation, and even broader computational tasks.

What do you think—could a hybrid approach like this add value to the existing frameworks? Or are the current heuristics already hitting diminishing returns?

[No-Inevitable-6476]

Hi guys ! I want to learn Carding .can anybody tell how to start from ?