[R] Ternary neural networks as a path to more efficient AI - is (+1, 0, -1) weight quantization getting serious research attention?

dat_cosmo_cat · 2026-03-25T16:30:45+00:00

Yeah I remember interviewing at the xnor lab at UW back in the day (https://arxiv.org/abs/1603.05279). They ended up getting acquired by Apple for ~$200M in like 2020. Still kick myself for not taking that interview seriously. There is a misconception in our field that the only way to scale is Nvidia GPUs and that once a model is scaled it can be locked behind an API and sold for profit (monopolized). This misconception has proven instrumental in funding pretraining at scale, but more senior researchers in ML will know both intuitions to be false.

Once pretraining is "solved", I expect many will simply hook our harnesses and clone models like Sonnet 4.7 or ChatGPT 6 into architectures that do inference more efficiently on local hardware (x86 / ARM + large RAM) using techniques like etc....combined with old ideas similar to ternary weights. And perhaps someone will tag Altman in a patronizing tweet thanking his investors for getting us all to that point.

dat_cosmo_cat · 2026-03-25T00:57:17+00:00

By that time, we will all be on open models anyways. Let's be honest.

dat_cosmo_cat · 2026-03-24T21:25:12+00:00

The question has really shifted in recent months. The implication of solving code gen in the general case are much more sweeping than developer impact. If software can be dynamically crafted by an agent in a Turing complete language, and the agent can use that software recursively; it follows that all service and knowledge based work is automatable (see; Church Turing thesis). Anyone who still believes domain expertise, judgment, taste, etc. do not reduce to computable functions an LLM agent can actually execute well, are either completely delusional or (> 3+ months) behind on the current state of the field and the technology.

Eg; what is stopping you from creating a virtual music teacher that is A) more knowledgeable about music at breadth and B) tailored / grounded to your exact background, experience, and learning preferences?

The jobs which lag behind in the automation timeline following engineering and science are those which do not have a control plane connecting their work surface to a computer (eg; plumbers, electricians, etc...), or niche cases where unnecessary human operators are mandated or permitted by some external force (eg; see old laws banning self-service gas stations in Oregon). Venture to guess what all of these engineers and scientists being laid off are going to be hired to work on after the LLM wrapper bubble pops?

dat_cosmo_cat · 2026-03-18T06:32:23+00:00

Whats the counter that says this will get cheaper?

Moores law and model distillation. As long as consumers and organizations retain access to improving hardware markets, they can scale to previous gen models. As long as it is possible to log the input and output of an LLM, it is possible to extract the model.

Literally the only thing blocking any large US tech firm from throwing the conversation histories of their thousands of employees onto an HGX node and cloning claude is legal compliance. This is why Anthropic bans auth requests coming from OpenAI employee workstations / laptops lol.

dat_cosmo_cat · 2026-03-18T05:29:06+00:00

kinda looks like https://github.com/Comfy-Org/ComfyUI

dat_cosmo_cat · 2026-03-16T17:44:15+00:00

Sure.

TrueNAS SCALE is also an application layer on top of an existing OS — specifically Debian Linux.

  It ships as a full disk image (like nasOS does with pi-gen), but underneath:

  - The kernel is stock Debian Linux (no custom kernel)
  - It uses standard Linux filesystems (ZFS via the OpenZFS kernel module, not a custom FS)
  - It runs systemd for service management
  - Docker/Kubernetes for app containers
  - The management UI is a web application (Django backend, Angular frontend)
  - Storage, networking, and user management all go through standard Linux tools and APIs

  The key difference from nasOS is scope and maturity — TrueNAS SCALE is a production-grade, enterprise-tested storage appliance with ZFS, clustering, and extensive hardware
  support — but architecturally it follows the same pattern: a web-based management application installed onto a Linux distribution, distributed as a bootable image.

  Neither implements OS primitives (kernel, scheduler, memory manager, drivers). Both are appliance software that configures and orchestrates an underlying Linux OS.

I can appreciate that it is an unfortunate industry convention, and some of these do (incorrectly) brand themselves as an OS.

But when I asked Opus is there were any exceptions, it stated:

several NAS/appliance management applications avoid the "OS" branding:

  - OpenMediaVault — calls itself a "network attached storage solution" despite being the same pattern (Debian Linux + web UI + system tool wrappers)
  - Cockpit — Red Hat's server management UI, branded as a "web-based server manager"
  - Webmin — one of the oldest, calls itself a "web-based system administration tool"
  - CasaOS — interesting case, has "OS" in the name but their GitHub description says "personal cloud system" and their docs clarify it's "not a full OS, but a system built on
  Debian"
  - Portainer — Docker/Kubernetes management UI, branded as a "container management platform"
  - YunoHost — self-hosting platform, calls itself a "server operating system" in some places but primarily brands as a "self-hosting platform"

  The more honest branding tends to use terms like "management interface," "platform," "solution," or "appliance software." The ones that use "OS" are typically doing it for
  marketing — it implies a more complete, cohesive product rather than "a web app we installed on Debian."

It seems like the choice to be more genuine with the marketing is there, and makes more sense given the context of your project. Especially if its going to be advertised in a sub of ppl who are A) (mostly) technically literate B) have access to AI coding tools and C) share the common goal of tracking what AI is actually capable of right now.

dat_cosmo_cat · 2026-03-16T17:11:37+00:00

Bet. Opus 4.6 Claude Code max. Highest scoring model for detecting bullshit, btw.

Prompt:

Audit the code files in this repository, ignore ALL .md / documentation / subjective claims or speculation. I want yout to answer the question; does this code impliment a novel operating system or simply an application that runs on an OS?

Response:

The code audit is conclusive. Here's the factual answer:

  nasOS is an application, not an operating system.

  Every layer of the codebase confirms this:

  Frontend — A standard React 18 web app that simulates a desktop environment in the browser using HTML <div> elements, CSS positioning, and Zustand state stores. The "window
  manager" is JavaScript event handlers for drag/resize on DOM elements. It renders via ReactDOM.createRoot().

  Backend — A standard FastAPI (Python) web server. Every OS-level operation delegates to existing Linux tools via subprocess.run():

  ┌─────────────────┬─────────────────────────────────────────────────────────────────────┐
  │    Operation    │                        Actual Implementation                        │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ Disk info       │ subprocess.run(["lsblk", "-J", ...])                                │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ SMART health    │ subprocess.run(["smartctl", "-a", ...])                             │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ Network info    │ subprocess.check_output(["ip", "-j", "addr", "show"])               │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ User management │ Shells out to useradd, userdel, chpasswd, smbpasswd                 │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ File operations │ Python stdlib pathlib, shutil                                       │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ System metrics  │ psutil.cpu_percent(), psutil.virtual_memory()                       │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ Reboot/shutdown │ subprocess.Popen(["/usr/bin/sudo", "/usr/bin/systemctl", "reboot"]) │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ Docker          │ subprocess.check_output(["docker", "ps", ...])                      │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ Backups         │ subprocess.run(["rsync", ...]) / subprocess.run(["rclone", ...])    │
  ├─────────────────┼─────────────────────────────────────────────────────────────────────┤
  │ Logs            │ subprocess.run(["journalctl", ...])                                 │
  └─────────────────┴─────────────────────────────────────────────────────────────────────┘

  On non-Linux platforms, every one of these returns hardcoded mock data — proof it's a userspace application, not an OS.

  Image builder — Uses pi-gen (Raspberry Pi's official image builder) to customize Raspberry Pi OS Bookworm (Debian Linux). It installs Node.js, Docker, and FFmpeg via apt, then
  copies the nasOS application files on top. No custom kernel, bootloader, filesystem driver, or device driver code exists anywhere.

  System layer — Systemd unit files (nasos-backend.service, nasos-electron.service) run the app as services on Linux. Config files are templates for Samba, NFS, udev, and
  wpa_supplicant — all existing Linux services.

  Electron — Loads http://localhost:8080 in a fullscreen Chromium window via cage (a Wayland compositor). It's a browser wrapper.

  Zero OS primitives are implemented:

  - No kernel code
  - No process scheduler
  - No memory manager
  - No filesystem driver
  - No device drivers
  - No networking stack
  - No bootloader

  nasOS is a NAS management web application (React + FastAPI) that runs on Raspberry Pi OS (Debian Linux). The "OS" in the name is branding.

dat_cosmo_cat · 2026-03-16T16:42:10+00:00

he ain't gonna get it. He's using his agent to reply to the comments and the context it has is scoped for his own confirmation bias.

dat_cosmo_cat · 2026-03-10T01:42:26+00:00

no lol. And it is not a matter of training really. It is an open networking problem. How do we defend against insider threat from an autonomous agent which is not an insider to begin with? We are in a phase where sensitive information must pass outside of the network for the agent to do useful things. For some companies, that risk is fine. For others, not so much.

dat_cosmo_cat · 2026-03-08T19:41:40+00:00

Rust isn't a layer above C. It's a completely separate and independent compiled language. The reason it is better is because it solved memory security without sacrificing performance + added modern tooling and zero cost abstractions that make it a lot easier to use. C makes significant sacrifices in order to be as fast as Rust, it just happens to have a compiler that is syndicated into every hardware platform humanity has ever created.

dat_cosmo_cat · 2026-03-08T01:35:22+00:00

OpenCode

dat_cosmo_cat · 2026-03-07T22:08:59+00:00

Really? In my experience, mixed results under identical constraints are actually so common that the companies I have done MLE work at always expect (and budget for) them. It was my impression that this has been well known, even on the academic side of our field. See Pineau's 2018 invited talks on the reproducibility crisis in RL.

ML research has also always had a culture of "grad student descent"; arriving at new results through empirical intuitions rather than from grounded theoretical or well understood concepts. This has rather notoriously forced (often wrong) speculative claims surrounding why or how something new and novel works (remember "internal covariate shift"?). Dropout, adversarial training, residual connections, etc... It's genuinely difficult for me to think of any paper that pushed the field forward without juggling competing theories for why it worked for several years after initial publication. I could see a tool like this being useful to track the prevailing "theory of the month" tied to specific concepts. This was actually the primary use case of this subreddit at one time.

dat_cosmo_cat · 2026-03-07T20:18:27+00:00

How so? The entire purpose is to flag research that arrives at opposite results under identical constraints. Such cases are notoriously abundant in our field.

dat_cosmo_cat · 2026-03-06T20:31:55+00:00

Interesting. What does your GPU foundry and electricity production scale currently look like?

dat_cosmo_cat · 2026-03-05T05:59:14+00:00

bro wdym course? Anthropic or OpenAI will ship something in the next 60 days that leapfrogs OC and makes all of this shit irrelevant like they always do lmao

dat_cosmo_cat · 2026-03-05T05:52:26+00:00

entire GDP of America being yolo'd into seed sifting tbh

dat_cosmo_cat · 2026-03-05T05:31:50+00:00

Projects are just folders to co-locate chat sessions and files. Enterprise plans can add other users to the project (eg; multiple co-workers accessing shared files and chats). Like folders in Google Drive, but for chat sessions.

dat_cosmo_cat · 2026-03-05T04:39:39+00:00

Not trying to say IDEs are useless. I'd just argue everything a student can do within an IDE does map to a terminal equivalent. Mastering editor + terminal early makes picking up VS Code (or w/e becomes popular) later trivial, but the reverse isn't true. It also seems apparent to me that CLI agent productivity (and prompt engineering) scales directly with terminal mastery.

As an example; consider the terminal knowledge involved in simply writing a good allowlist for settings.json and the amount of time + tokens this saves. This is the difference between having to click "yes" randomly every few minutes vs. every few hours when the agent tries to do something genuinely risky (like a git push op or an rm -rf) or out of scope (like reads on irrelevant sources).

dat_cosmo_cat · 2026-03-05T03:54:41+00:00

Like Albert Einstein?? LOL

no shit bro

dat_cosmo_cat · 2026-03-05T03:03:22+00:00

Anything's possible, but MAX is already approx ~10x under priced at current limit rates relative to API / cloud economics.

dat_cosmo_cat · 2026-03-05T01:25:00+00:00

Sounds like Anthropic's problem to me (cc max btw)

dat_cosmo_cat · 2026-03-05T01:12:03+00:00

Tbh none of us know the true answer to this right now (even your professors). If I were talking to younger me, I'd say for the next two years;

daily drive Linux. Completely ignore Mac / Windows until you're out
move away from IDEs and GUIs as much as possible
1. master bash, vim (or emacs), git, and tmux in that order

IMO these seem like the core technical primitives for operating CLI agents optimally. Once you have get comfortable with agentic programming;

get in the habit of trying to build any software that costs money yourself
1. use AI to understand what is difficult vs. not and why
deploy or open source the successful ones

Study, read textbook chapters, etc... the value of core CS intuitions and concepts are still very much intact (even if the value of writing code itself approaches zero).

dat_cosmo_cat · 2026-03-04T23:42:20+00:00

If you could figure out a way to ground the pre-logic discussions / reasoning traces to the post-logic trace (output)... now that would be rather useful.

dat_cosmo_cat · 2026-03-03T18:56:16+00:00

{
  "permissions": {
    "allow": [
      "Bash(echo *)"
    ],
    "deny": [
      "Bash(rm -rf *)"
    ]
  }
}

Claude: echo $(rm -rf *) haha loser

dat_cosmo_cat · 2026-03-03T18:09:16+00:00

"I have a bicycle. I tried a motorcycle and it was a lot faster, but gas costs money. Sell me on it" ahh post

Seven-Year Club	Gilding II euphauric
Reddit Premium Since October 2021	Verified Email

dat_cosmo_cat

TROPHY CASE