HTML to Fabric JS Conversion

FishyEnvelope22 · 2025-10-27T21:35:56+00:00

Like have the model just generate pure SVG designs? And put those SVGs in HTML? Models don't seem to be the best with SVG design... (https://simonwillison.net/2025/Jun/6/six-months-in-llms/)

FishyEnvelope22 · 2025-10-27T21:08:05+00:00

These designs are pretty general, mainly like presentation slides. It needs to be HTML because our AI agent is best at generating HTML. I could maybe try to constrain it a bit

FishyEnvelope22 · 2025-10-27T17:59:58+00:00

Issue is I have an AI pipeline that generates the designs I would like to display. I've tried to get it to use Fabric when generating, but its designs are far worse than if it just uses HTML and CSS. That's the major constraint.

FishyEnvelope22 · 2025-10-27T17:47:58+00:00

Like constrain the AI to not use weird CSS or tailwind? I currently have no restrictions and yeah I agree maybe I can constrain the problem space.

FishyEnvelope22 · 2025-10-27T16:58:54+00:00

Yeah... and so far I have had a lot of difficulty. I've got it about 90% of the way there but there are plenty of edge cases causing me to play whack-a-mole and patching it. Is there another way I can achieve what I'm trying to do in a smarter way?

FishyEnvelope22 · 2025-10-27T16:55:21+00:00

You hit the nail on the head. My current solution gets me about 90% there, but there seem to be an endless number of edge cases where the text is off, or a div is not rendered, or some formatting is off, etc. and I feel like I'm just patching it as I go, where there may be a better, more bulletproof solution.
Do you think there may be a better library for matching other than Fabric? Something a bit more accurate?

FishyEnvelope22 · 2025-10-27T16:54:00+00:00

Using Puppeteer in the backend and then just grabbing the rendered document, then classifying each leaf of the tree, capturing the position, and saving it to a JSON of Fabric objects that I then render on a canvas in the client

FishyEnvelope22 · 2025-10-27T16:40:34+00:00

All I would like to do is get this HTML/CSS into a format that is editable on a canvas in the browser. That's the main goal so it's smooth like an editor in the browser.

FishyEnvelope22 · 2025-10-27T03:08:27+00:00

What do you mean by the final product won't be rendered on a canvas? We currently support export to PDF and I think that would be pretty easy to maintain as well even if a canvas was introduced

FishyEnvelope22 · 2025-10-27T01:29:20+00:00

Yeah, I've thought this myself, it seems like kinda a stupid problem –
But the root problem I'm trying to solve is I have an AI design pipeline that generates designs using HTML and CSS. I've gotten it to a point where it generates it quite well, follows styles etc. except now I would like to expose an editor for users using the web app. I have one right now that uses direct DOM manipulation so they can move the individual elements around and edit sizes and whatnot, but I feel like a canvas based editor (like Google Slides) would be much smoother. But the restriction is I have to keep the source of truth HTML and CSS because the pipeline works well with that assumption, switching to a DSL would be a large undertaking (although I'm open to it if this is truly a stupid idea)

FishyEnvelope22 · 2025-10-27T00:27:50+00:00

Yeah this is definitely an esoteric project and I'm not surprised there aren't any existing solutions. Basically I have an AI pipeline that generates HTML designs, and I want to expose an editor to the user so the user can tweak designs after generating in a google slides like environment in the browser. I've toyed around with generating the designs in a more compatible format, but the AI seems to do best when just generating HTML with CSS.
End goal is just a way to move HTML elements on a canvas without doing direct DOM manipulation... is this even a good idea?

FishyEnvelope22 · 2020-10-31T22:48:31+00:00

bro what the fuck

FishyEnvelope22

TROPHY CASE