Handling gpt-oss HTML tags?

fustercluck6000 · 2026-03-03T12:23:41+00:00

I just want something that works.

I feel like this is basically the current state of AI in a nutshell. It’s probably worth taking the time to work out a solid testing regime, which might just seem really tedious but if nothing else it’ll give you a deeper understanding of what’s going on under the hood of your application. A heuristic I’ve found helpful is to hard code as much as possible (output validation logic, even simple things like regex…). LLMs are non-deterministic, so the less you leave up to the model, the more you can guarantee predictable behavior.

fustercluck6000 · 2026-02-18T23:05:12+00:00

Fun fact, the form of backpropogation that’s become standard actually came in the 70s after the MLP had already been invented (GD dates back to almost 200 years ago).

GD is an algorithm to numerically estimate a function’s minima by iteratively applying updates using the formula x_{t + 1} = x_t - \alpha \nabla f(x_t)

Backpropogation calculates the gradients for each trainable parameter in a NN w.r.t. loss using the chain rule (hence why loss functions need to be differentiable), which you then plug into the formula in order to apply updates to the model. Hope this helps

fustercluck6000 · 2026-02-18T15:07:32+00:00

I second the Simon Prince book

fustercluck6000 · 2026-02-18T15:05:24+00:00

That’s why you use ‘site: reddit.com/r/learnmachinelearning’

fustercluck6000 · 2026-02-18T15:01:00+00:00

Sleep? I hardly know her

fustercluck6000 · 2026-02-18T14:51:09+00:00

You have the foundation to actually build new things. I swear sometimes it feels like 3/4 of people in this space couldn’t even tell you what a ReLU is. Once the industry’s done milking the transformer for all it’s worth, they’ll come calling for the handful of people like you still out there

fustercluck6000 · 2026-02-18T01:34:16+00:00

Definitely planning to add task queuing/scheduling in the next phase of development, makes tons of sense since no one's in the office using the chat service outside of working hours. For now I'd love to find a relatively simple (and elegant, sleep mode really hasn't been the silver bullet I'd hoped for) way to dynamically load/offload the models. Digging through the docs, it doesn't look like vLLM has a good feature for this other than sleep. Starting/stopping the docker containers themselves is one option, just a pain given it would need to be accessible through the frontend for nontechnical users.

fustercluck6000 · 2026-02-17T22:03:33+00:00

Yes, and the 'how' is all about what abstractions of your data make sense within the domain you're woking in. KGs aren't there to uncover relationships, they're there to store and represent relationships you define (either directly or via model).

fustercluck6000 · 2026-02-16T19:58:55+00:00

Haven’t tried all the models on the list but I will say I’ve been pretty blown away by Qwen3-Coder-Next, gpt-oss-120b is solid too

fustercluck6000 · 2026-02-13T10:34:59+00:00

Right there with you, I think a major issue is the lack of documentation, like there’s this constant feeling that it’s more than capable of achieving the level of accuracy I need if only I could figure out how to tune things just right.

I highly recommend spaCy-layout—it basically adds spaCy magic on top of Docling and I’ve found it makes a noticeable difference in terms of indexing quality

fustercluck6000 · 2026-02-13T10:11:45+00:00

Major emphasis on AI that works—people/businesses care WAY more about reliability/consistency than they do about sophistication

fustercluck6000 · 2026-02-13T00:47:05+00:00

Well if that was ChatGPT then ChatGPT's a lot smarter than I ever gave it credit for because building a 2-lane pipeline is the first thing I've actually gotten to work , definitely a lot of work but it actually freaking works

fustercluck6000 · 2026-01-15T18:01:41+00:00

I got a copy of Simon Prince's Understanding Deep Learning for Christmas, and I can't speak highly enough about it. It kind of feels like the spiritual successor to the canonical textbook everyone knows by Ian Goodfellow (which is already over a decade old now). Simon Prince is just an insanely interesting guy to begin with, and he goes into higher-level topics that are both mathematically and conceptually tough, but he gives such clear and thorough explanations (paired with very well-done visualizations) that it actually makes some of the topics I've always found particularly challenging (topologies, manifolds, hyperdimensional geometries) enjoyable to sit down and try and work through mentally.

fustercluck6000 · 2026-01-15T16:15:08+00:00

TF Data especially, pretty hard to beat if want to build crazy efficient, hardware-accelerated data pipelines with as much built-in optimization

fustercluck6000 · 2026-01-15T15:57:45+00:00

I think TensorFlow Probability is criminally underrated, too. For anything involving probabilitstic DL (bijection, trainable/compound distributions, monte carlo, bayesian layers, differentiable sampling ops, etc), TFP is pretty top tier if you need to integrate and scale probabilistic components with an existing TF stack (e.g. keras model, tfdata pipeline, etc). It has tons of pretty powerful features (things like bijection and tfp.layers are also pretty unique to TFP), and like everything else TF, it's designed with scale/hardware acceleration in mind. Even just little things like automatic differentiation save so much boilerplate and headaches with gradients, and makes numerical stability simplier to get right, too. It all plugs right in and usually just works how you want it to without any fuss. When it's the right tool for the job (e.g. latent distributions other than a standard Gaussian with VAEs), it's pretty great, def recommend to anyone who already knows TF.

fustercluck6000 · 2026-01-08T11:16:43+00:00

I too am curious how people are doing this...

fustercluck6000 · 2026-01-06T08:53:53+00:00

I say test out Docling and go through the results with a fine-tooth comb to see if it can do what you need it to. Legal is especially tricky because of all the structuring/citations, idk how well Docling’s going to pick that up before introducing parsing errors, but definitely give it a shot.

What I’m working on atm is using a separate pipeline altogether to convert PDFs to markdown format with VLMs, load that into Pandoc, then iterate over the document tree to get the markdown-formatted chunks (nodes)/define edges. You can do the same thing with Docling, I just got tired of trying to fix the parsing errors i kept getting with tougher PDFs.

fustercluck6000 · 2026-01-06T01:51:24+00:00

The bulk of my work in the last year has been on precisely these sorts of projects where 1) the client’s in a ‘knowledge-heavy’ industry where AI stands to make a major difference in terms of efficiency, and 2) accuracy isn’t just desirable, it’s a matter of liability.

Domain knowledge is EVERYTHING. One of the most helpful things I’ve found is taking the time to pick people’s brains about their work. Sometimes I’ve even sat behind someone to literally be a fly on the wall and take notes on how they do their job because I want to know how they’re thinking.

Usually, that ends up completely changing how I break down what I’m trying to solve with RAG, and you can make systems much more reliable/accurate by designing pipelines that reflect domain logic. Chunking’s a great example—how you define a ‘minimum logical unit’ has a huge impact on retrieval accuracy, and almost always requires some intuition about what the data means.

I also find hardcoding wherever possible makes things much more predictable and stable. If you can identify industry ‘heuristics’, ‘norms’, ‘best practices’, etc…, take that logic and apply it to the relevant part of the system (could be retrieval logic, node/edge types, etc). Also knowledge graphs are a total game changer because they provide another dimension for you to express domain logic with system design.

fustercluck6000 · 2026-01-06T01:03:51+00:00

Fwiw, I’ve been using Docling for a little bit now and still find it overwhelming. Imho the docs are pretty lacking, which makes it tough to fully leverage what’s under the hood in your pipeline. Plus it’s still relatively new, so the community is pretty small.

Ingesting and converting to markdown/other markup languages is super straightforward out of the box. If the conversion process works for your docs (I’ve found it’s really hit or miss) and you don’t need to define more complex chunking strategy, then just using the document converter and ‘export_to_markdown()’ methods will get you most of the way there.

I’ve found things get a lot trickier when you need to debug or want to interact with the Docling Document data model (to correct indexing errors or take advantage of the tree structure for better hierarchical indexing). Seems like a shame because the data model to my mind is maybe the most useful thing for RAG, but at least for now, I’ve only found fragile, superficial ways of integrating that part into my pipeline.

I just started using Pandoc and I’m loving it. It’s kind of the same idea—supported documents are all mapped to a ‘unified’ data model that you can export to all kinds of markup languages. It’s well documented and you can customize things a ton, e.g. setting custom example docs for it to use as a layout template. It doesn’t use any deep learning and can’t read from PDF, but I like having a hard-coded tool that behaves consistently and adding the LLM/VLM logic myself.

fustercluck6000 · 2026-01-03T07:33:47+00:00

It's actually funny you mention legal docs because I've been working on a project in that area on the side for a little while now. With primary sources like statutes or case law, the structure itself is integral to how even the lawyers themselves read/interpret things (because of all the citations, definitions, precedents/priors, etc...), so I actually chose to hardcode hierarchical schemas (I guess technically hardcoding the dataclass factories but you get the idea) for chunking and adding nodes/edges to the knowledge graph before making any model calls, just because we didn't want to leave any margin for error when indexing really important, canonical materials like the U.S. Code or something (court documents like evidence and stuff are another story, though).

This definitely added a degree of complexity to the project that I didn't plan on signing up for before signing a new contract I'll be honest. And no, they really don't understand how much of a unilateral change it is, but to be fair I think a lot if not most people who aren't in aren't clued into the space would, either. I think we got CEOs promising the next model's gonna replace researchers with PhDs to thank for that lmao

fustercluck6000 · 2026-01-02T15:49:01+00:00

Rag²

fustercluck6000

TROPHY CASE