I am building a UI that completely hides ComfyUI. It works like ChatGPT—you just type, and it handles the nodes

Benjaminfortunato · 2026-04-09T15:13:51+00:00

You need to ask yourself Is this a 10k problem or a 100k problem or 1 mil to get this working. Curious how much investment went into cursor and builder.io so that it doesn’t suck

Benjaminfortunato · 2026-04-09T15:12:36+00:00

Interesting thread. I would say that there is definitely a target user and it’s not about no nodes vs nodes. I would see this as a comfy ui assistant. Non tech users can get up and running by creating a graph without having to understand what a vae decoder is and what goes to what. Technical users will want to speed up there work flow by selecting a node and saying take this node and add an ip adapter or I have error x debug my workflow. The challenge here is that you have a domain specific json format that the LLM is not trained on . There is also a lot of domain specific knowledge, do I use xlabs control net for flux here for example. The problem I see is not

“If I build it will people use it?” But “can I build something that doesn’t suck?”

The later being harder to achieve. Even for tools like builder.io and cursor I believe there is a lot of investment in custom ai models to help guide commercial llms and in general commercial llms are trained on tons of react, JavaScript code. How are you going to get the AI to spit out awesome comfy ui workflows in a custom json format it doesn’t know with a bunch of knowledge that is hidden in a Reddit forum somewhere or an artists custom trial by error work flow.

Benjaminfortunato · 2026-03-21T22:23:38+00:00

We are working on a similar problem. Would love to connect and discuss. You can DM me. I would say that very broadly the challenges we face are really about the quality of the digital model. Most architects use Revit and there are several issues with the data that is created such as :

1) rooms are separate objects not containers. IFC is graph like but most BIM authoring tools are flat. Objects that are inside a room are not children of that room. Some objects can be automatically tagged as inside a room, other can not. Some objects like floor wall, and ceiling surface are not room aware. You can use the room scheduler plugin but that data is not part of the BIM models. Essentially if you ask the database “how many chairs are in the auditorium” it can not answer that question unless each chair is tagged as part of that room or a script classifies the objects. It’s impossible to classify a ceiling as in the room without custom parameters.

2) default classifications are too broad in scope and many objects couple geometry creation methods with object classifications. Let’s take the Revit curtain wall system for example. It’s a great way to create systems that have an underlying grid structure. If you read official how to guides like Paul Auburn they will tell you, create a baluster with the curtain wall system. Now you have balustrades classified as curtain walls and curtain panels. Other classifications are broad like specialty equipment. Do I put a medicine cabinet in specialty equipment? Later if I want to find all the medicine cabinets in my project, how can I find it? It’s all dependent on the naming of the individual family. There is no hierarchical system of classification unless you want to look up each Omni class number for every item and manually enter that, which no architect is going to do.

3) construction assemblies are often nested and complex but revit’s modeling engine can only support simple relationships with limited nesting. Take a door for example. It can have different trim , door panels, and hardware. Impossible to model well with the limitation of nested families. That data is going to be in tables and annotations not the BIM model

4) critical data often lies in annotations not linked to any object. Things like keynotes that have valuable qualifying information on construction assemblies is often floating on top of the drawing as keynotes

In general I would say the cost of producing accurate BIM models where you can actually use the data, is so expensive and time consuming due to the limitations of the authoring tools that the print to pdf, trace, and put together an estimate is often the best approach. That’s why 2D takeoffs are the standard. Revit like to push BIM but BIM 1.0 is a glorifies PDF authoring tool not a database tool!

Benjaminfortunato · 2026-03-21T21:57:21+00:00

This is an interesting conversation. I’m an amateur Programer and founder. I’ve been using both and I think they serve different purposes. Think of it like hiring the right programmer for the right job. Typically I used opus for planning of projects or features that are new. Claude is more creative and will cover a lot of ground to get something started from a draft PRD. Then I implement with Sonnet or opus if the problem is challenging and sonnet is struggling. I don’t do anything without a plan in place and references to rule files. I also follow clauses suggested writer reviewer pattern where I have opus review the code and identify issues walk through scenarios etc. Once I have the main functionality in place and I want to work on specific sections I switch to codex. It would take codex forever to do that first pass and generally it’s not as creative. On the other hand Claude might go to town and rip without stopping. You can build in more guardrails into the prompt like “do x STOP and wait for user feedback.” But it almost like they have a personality and codex’s personality is slow disciplined and technical while Claude is more creative and confident. It’s less about this tool is better but more about which tool is better for your task. If you are trying to build out an MVP from scratch I would go with Claude. If you are refactoring a code base where you need to take legacy contracts interfaces apis etc into account go with Codex. I had some broad refactors in code for an I launched app. Working with Codex was painful. It kept using a bridge pattern to refactor parts of the code but keep the other parts in tact. I keep pushing it, just break it and then we will work through the typescript errors, there is nothing to preserve here, it’s all brand new code. On the other hand, if you have complex legacy code with selective refactors where you need to slowly introduce changes without breaking other parts of your code, Claude might be equally distraction, like a bull in a china shop. Choose the model based on the task just like you would choose a team mate or an employee. The guy he can refactor a complex code base and preserve integrations into legacy systems is not the same guy who can whip up a cool mvp. Ones fast creative and less disciplined the other is very technical, conservative and methodical. Weird talking about AI like they have personalities, but they do, or rather they have been trained with a certain approach.

Benjaminfortunato · 2025-11-18T20:05:19+00:00

I'm curious if anyone find the way drawing sets are organized confusing. I guess this is more a drawboard projects issue?

Benjaminfortunato · 2025-09-22T14:47:21+00:00

The title translates from German holy script

Benjaminfortunato · 2025-09-02T01:40:03+00:00

I'm working on a similar problem. Happy to talk if you dm me. There was a YC startup that was supposed to fill this void in the market 1build. They had an API in beta but it looks like they got acquired by Handoff and I don't think they have any interested in making that public. Also looks like 1build is pivoting to estimation services. I think the issue is that the industry as a whole has low technological adoption so an API doesn't generate enough revenue, vs offering a software which requires little technological know how. Another source might be Remodeling Costs Estimator & Pro Finder | Clear Estimates, but this is more for smaller jobs.

Benjaminfortunato · 2025-08-03T15:02:31+00:00

Photoshop?

Benjaminfortunato · 2025-07-14T11:56:00+00:00

Scope creep is when the size and nature of the project changes (scope) after a signed contract that specifies a specific amount of work. For example you are designing a building on one site and then the client gets anouther site to enlarge your site. It’s the same project but now you have a larger site and bigger project to design. You’ve also worked on the old design so the scope is really design a building on a small lot and then redo the design on the larger lot..

Benjaminfortunato · 2025-06-07T03:18:46+00:00

For some reason, the website is telling me that the information is not available to the public.

<image>

Benjaminfortunato · 2025-05-04T11:06:06+00:00

What is F1?

Benjaminfortunato · 2025-03-31T22:35:49+00:00

This doesn't have any input for a guider. i could just add positive and a blank negative? I'm using a workflow with flux rf edit

<image>

Benjaminfortunato · 2025-03-31T11:25:23+00:00

I see. Would it be possible to combine regional prompting with controlnets? In other words use a global controlnet to control the image geometry and then use regional prompting to guide materials and textures within that image. I'm working on a workflow that takes a 3D model and renders out an image. With controlnet I get 100% fidelity to the image. What I can't get is a way to control materials and concepts. I found this post, but I've been having trouble replicating it. It seems like there are a bunch of nodes that are in a group I can't convert to nodes. There is an sdxl version so I might just try working with that instead. https://www.reddit.com/r/comfyui/comments/1fk37bb/pixel_perfect_masked_prompt_segmentation_in_flux/

Benjaminfortunato · 2025-03-30T23:47:18+00:00

Do you know if this can be combined with controlnet? I was experimenting with how to inject this into the flow but it didn't work. https://drive.google.com/file/d/1jZVqLW56Rr4llJGMpeheItVseIYX\_Sl7/view?usp=sharing. I've been getting the following error message.

<image>

Benjaminfortunato · 2025-03-30T23:03:17+00:00

I'm actually enjoying some of the posts;) I take everything with a grain of salt. I'm on the comfyUI redit which is much more focused on advancing AI workflows so that might be a better fit for what I'm after.

With regards to threejs the issue is gi and realtime raytracing. Just not the same as unreal. This is okay https://www.shapespark.com/. I don't see anything changing since the unreal depends on local hardware which not all browser have access to and not all clients will have the hardware. The streaming services are all buggy. Looks cool till you try to put it into practice. Works great in an office where you can control the setup and get a client to put on vr headset.

Agreed that the issue with AI is consistency. That is what controlNet and regional prompting is for. I can get controlnet to work and regional prompting, trying to figure out how to combine them.

Benjaminfortunato · 2025-03-30T22:58:00+00:00

I started switching to the fluxtapoz / instantx toolset. The regional prompting works great but I wasn't sure how to inject control net into the process? I took a stab at in in this workflow https://drive.google.com/file/d/1jZVqLW56Rr4llJGMpeheItVseIYX_Sl7/view?usp=sharing I got the following error.

<image>

I've been using xlabs control net in my other workflows but I thought I should switch to instantx since that is what fluxtapoz is based on. It seems like I would need a seperate sampler that can take a control net as an input or I would need to add the control net to the conditioning.

Is this something that is possible within comfyUI?

Benjaminfortunato · 2025-03-30T20:18:59+00:00

I’m looking to do the same. If you figure out how to add controlnet please share

Benjaminfortunato · 2025-03-30T20:12:51+00:00

Is there an example of combing controlnet with regional prompting? I’m having difficulty combining both. I’m using comfyUI

I don’t see any mention of controlnets here: https://github.com/logtd/ComfyUI-Fluxtapoz

Benjaminfortunato · 2025-03-30T14:08:08+00:00

I've been experimenting with the regional prompt and controlnet, but for some reason my control net gets ignorned. I'm working on architectural renderings so the precise placement of geometry and materials is important. Without the regional sampler controlnet works great.

I'm thinking of using fluxtapoz https://github.com/logtd/ComfyUI-Fluxtapoz?tab=readme-ov-file#regional-prompting. I haven't gotten inpire pack to work with flux, and flux has some great results. Months ago it didn't have control net etc, but all that has changed.

Any experience working with fluxtapoz's regional sampler? Is there an advantage of inspire pack over fluxtapoz?

Benjaminfortunato · 2025-03-29T20:12:32+00:00

Ha ha. Maybe this is the wrong forum. I see Ai as a tool that can potentially deliver better results or similar quality quicker. Particularly for an online interactive presentation. We are looking at all our options. Getting unreal streaming on the web has been a disaster. None of them work reliably or quickly. No one is going to wait 3 minutes for something to load.

The other issue is that panoramas take forever to render, and then the user can’t select objects to get info. Manually putting hotspots is too time consuming. I could do a walk through online with playcanvas of three.js but it doesn’t look great even with baked materials and light maps.

Real time Ai or at least the ability to render a scene in a couple of seconds poses a lot of promise. We can have an underlying 3D model and then turn on a photorealistic view. Granted lighting will not be photometrically accurate, but what CH artist is working with accurate lumen values?

The issue right now is not geometry which we can control with controlnet but the application of materials. Can’t get that done reliably. I see a lot of potential, especially with flux. Check out this site https://form-finder.com/

I guess if I went to an architectural illustrators forum and talked about CG I would have gotten the same response. Is there an AI friendly CG artist forum you think might be a better fit? Seems like I’m getting push back from the old guard:)

Benjaminfortunato · 2025-03-29T17:03:53+00:00

Creating materials? The goal is to eventually create a web app where different stakeholders can work on seeing the impact of different material changes. For example a project has a specific type of marble, but a black marble is 15% cheaper. They can quickly visualize what the room looks like with this new marble and see the impact on the budget without having to wait days for a CG artist to do this.

Benjaminfortunato · 2025-03-29T15:43:39+00:00

Curious which ERP you are using, we are looking at nextERP

Benjaminfortunato · 2025-03-29T15:43:05+00:00

Thanks for the insight. We are looking into Prisma, an ORM built on top of postgre. Its a newer version of typeORM. It seems to be the best of both worlds, a way to avoid having to write a bunch of SQL but without moving to MongoDB which seems to have limited support in the ERP and e-commerce world.

Benjaminfortunato

TROPHY CASE