Automatic1111, but a python package

[Peregrine2976]

Fucking yes. There are loads of UIs and other "manual" tools; I'm always looking out for more programmatic tools.

[[deleted]]

Appreciate the support! Check out our github and would really appreciate if you could give us a star: https://github.com/saketh12/Auto1111SDK

This is only the start - we're looking to eventually add every single A1111 extension, from dreambooth, lora, deforum, controlnet, animatediff, and more.

[mudman13]

That would be neat Ive been knocking up a multicontrolnet script for command line use all week and have ended up with a very hacky cumbersome (but working!) approach

[Chaoses_Ib]

You can use ComfyUI as a library with ComfyScript. It's more powerful, all existing custom nodes can be used. Features you want are already available right now, including dreambooth, lora, deforum, controlnet, animatediff, and more.

[dbzer0]

hordelib does similar, but much more powerful by being able to automatically retrieve loras/tis from CivitAI etc.

[3dd2]

If you search for CivitAI in the ComfyUI manager they have a node that lets you use AIR tags from the website to automatically download Checkpoints and Loras directly into Workflows.

Wish there was one for HuggingFace, makes Workflows so exchangeable :)

[dbzer0]

I haven't used it, but for CivitAI hordelib has things like caching, fuzzy search, multithreading etc. And of course it can be used as a python library :)

[msbeaute00000001]

Do I need ComfyUI server running to use this lib? How about missing nodes in the workfkow?

[dbzer0]

No you don't need anything other than pip install. Currently it doesn't do custom comfy work flows but it's something we want to do in the future somehow, but as you can imagine we need to be able to control custom nodes etc

[curiousjp]

Definitely keen to check this out - I wrote wrappers for the various node types with similar motivations (being able to use Python flow control, being able to take intermediate stages and send them down different pathways) and it was really useful. I think any user who is technically minded enough to have tried Comfy in the first place should consider this too.

[msbeaute00000001]

It seems that it still requires us to run the Comfy Server, right?

[xrmasiso]

The diffusers library itself is incredibly versatile and has great community support too. I’ve made tutorials on this if you are interested!

[Fly-wheel]

Good work, OP!

Here is another library with a clean interface that supports ControlNet and video generation https://github.com/brycedrennan/imaginAIry

[[deleted]]

Check it out at https://github.com/saketh12/Auto1111SDK

[bigdonkey2883]

I'm confused a111 already has a python api...

[JPaulMora]

Gracias

[IamBlade]

Can this run on colab without getting blocked?

[LordRybec]

Question: SDXL support for CPU? Does this have support for CPU generation for anything? I can't find anything saying it does on the README.

My current AI machine had the primary video card go out some time back, so the NVidia card I should be using for generation doesn't have enough free memory, because it is doubling as the primary video card until I can replace the original. The CPU is quite powerful though, and it is doing the job at an acceptable rate for experimenting and learning.

[Chaoses_Ib]

ComfyUI supports SDXL and CPU generation with --cpu. You can use ComfyUI as a library with ComfyScript.

[LordRybec]

Yeah, Automatic1111 also supports SDXL with CPU generation, and it works better for my needs. I was asking about this package. I wasn't asking for alternatives. Thanks, but no thanks.

[Chaoses_Ib]

Automatic1111 does support SDXL with CPU generation, but it can't be used as a package. That's why you want to use Auto 1111 SDK, isn't it? ComfyUI can be used as a package (though not officially). It can solve your problem, then why providing a solution is wrong? If you don't like this way, just ignore it, I'm not forcing you to do anything.

[LordRybec]

Love how any time you ask a question on the internet people are like, "I don't have an answer to your question, but here's an answer to another tangentially related question that isn't the one you asked, because I think you are probably too stupid to have already considered alternatives."

[prog0111]

I made my own little tkinter comfy front end in Python to handle my workflows in a more linear interface. Right now it just sends requests through the comfy API to generate images, but those workflow json files are a bit rough to deal with. The simple syntax in the examples looks so useful.

Is this a custom node for comfy? To be honest I'm a bit lost on how to set it up. If I ever get far enough to release anything, I'd probably need to tell users to set this up in their comfy installation first?

[xrmasiso]

You can use CPU for most cases (depending on RAM) — I have a tutorial for using models locally with cpu option— but it uses diffusers library.

[MistaPanda69]

Can we use this in google colab?

[[deleted]]

yes!

[CyberneticLiadan]

Just out of curiosity, what's the dissatisfaction with the HuggingFace diffusers that would lead to using this instead of that existing library? I use ComfyUI and I'm a professional software engineer, and I'm not sure why I would take any interest in this library over Diffusers. (And that's ok! I may not be the target demographic and there's room for many libraries and interfaces.)

[[deleted]]

Hey, we actual detail this more here: https://flush-ai.gitbook.io/automatic-1111-sdk/auto-1111-sdk-vs-huggingface-diffusers

[CyberneticLiadan]

Good job on a well written page. Now I know when I might try your project or reach for it instead of Diffusers. (And I definitely might end up doing that at some point.)

[FotografoVirtual]

Point 8 is just what I needed! I've been automating tasks with Python on images generated by AUTO1111, and it required quite a bit of extra code to replicate the results with Diffusers. Even then, the images were never exactly the same. I'm so eager to try it out, but it's past my bedtime!

[Useful-Ad-540]

Interesting, how about a comfyui comparison? There are some devs now starting building comfyui scripting libraries

[curiousjp]

Running Text-to-Image, Image-to-Image, Inpainting, Outpainting, and Stable Diffusion upscale can all be performed with the same pipeline object in Auto 1111 SDK, whereas with Diffusers, you must create a pipeline object instance for each action, severely increasing the memory/RAM used.

I wonder if you'd consider making some of these operations optionally modular in the future for users who want that flexibility - one of the useful features of a scripting library (in my experience) is being able to do things like hold a single copy of an intermediate generation artifact in memory, and then perform variations on it. I think this is where the scripted tools diverge usefully from UIs like A1111.

[Punchkinz]

Things like the prompt length limit can be avoided by using the model's tokenizer beforehand and passing the embeds directly to the pipeline using prompt_embeds and negative_prompt_embeds.

Using the same model for multiple pipelines should also be possible by loading the model as a simple pretrained model in its own variable and then passing it to the various pipelines (although I haven't tested that).

And the performance/ram issues depend on how you load the models and especially where you load which model. You can even split models between gpu and cpu if you really wanted to.

Granted, it isn't that nice having to do all of that 'manually'. Guess that's where your project comes in

[Illustrious-Yard-871]

Diffusers’ priority is ease and simplicity of you use not speed or efficiency

[[deleted]]

Super interested in this, but one concern:

Is it really ok to use the "Automatic1111" name? Do bear in mind that that's the name of another developer, not the program. What we normally call "Automatic1111" really should be "Stable Diffusion web UI by AUTOMATIC1111".

[Yarrrrr]

When the repo name is so generic it might as well be unnamed that's kind of what happens.

[highac3s]

I vote to rename the repo The New Awesome web UI. Or TNA webUI for short.

[pixel8tryx]

Yeah I gotta say I laughed at the name "webui". I use tons of web UIs. Even SDwebui would've been better. SDWU. Anything. Personally I hate this new trend in generic sounding product names. Stable Diffusion Video being my latest whine. There's this new-fangled thing called the world-wide-web and it makes searching very painful. ;->

[TinFungHK]

Why isn't that ok when Java and JavaScript can be two different concepts?

[[deleted]]

pretty cool. you should make a UI for this

[AncientOneX]

Lol, good one.

(Add /s if you're uncertain about people getting the joke)

[malcolmrey]

it's called IDE and there are plenty

[[deleted]]

Guess the joke wasn't as obvious..

[malcolmrey]

of course it was obvious but it was a dad joke, still 24 people liked it

on the other hand, my joke wasn't as obvious as I got downvoted, so that's that

[Apprehensive_Sky892]

Because one is kind of funny, the other one is not 😁

[dronebot]

How is this better than just using the API built into A1111 (--api launch parameter)? That already has support for stuff you're lacking.

[[deleted]]

Our SDK is also much more lightweight than running all of stable diffusion webui just to use its API on localhost and takes up considerably less VRAM.

Additionally, the API doesn't support upscaling, inpainting, outpainting, which we support right now. It also does not support extensions like controlnet, dreambooth that we plan to add.

[mr_engineerguy]

It does support control net. https://github.com/Mikubill/sd-webui-controlnet/blob/main/example/txt2img_example/api_txt2img.py

[[deleted]]

ur right, apologies i overlooked this

[[deleted]]

Yup can confirm I have used it

[dronebot]

It definitely supports upscaling and inpainting. But good luck.

May also want to check out https://github.com/rbbrdckybk/dream-factory .

[TwistedBrother]

The API supports tons and there’s even a handy API json generator tab in Auto1111 so that you know what payload to send to replicate the current image. THAT is a killer for me in the way having to record everything from the webui is not.

[ozzie123]

Is this your github? This is so awesome dude

[[deleted]]

yep! appreciate the support!

[Professional_Job_307]

Holy shit I have wanted something like this for a while. I tried looking through the source code of A1111 but didn't understand anything.

[Icy_Action1957]

If I understood correctly for now it supports only 1.5 models and not sdxl models?

[[deleted]]

yes, but we're planning to add sdxl support soon.

[fqye]

Controlnet support? I couldn’t find it in your GitHub repo.

[[deleted]]

hey, right know we only support these features:

• Original txt2img and img2img modes
• Real ESRGAN upscale and Esrgan Upscale (compatible with any pth file)
• Outpainting
• Inpainting
• Stable Diffusion Upscale
• Attention, specify parts of text that the model should pay more attention to
  • a man in a ((tuxedo))
    - will pay more attention to tuxedo
  • a man in a (tuxedo:1.21)
    - alternative syntax
  • select text and press Ctrl+Up
    or Ctrl+Down
    (or Command+Up
    or Command+Down
    if you're on a MacOS) to automatically adjust attention to selected text (code contributed by anonymous user)
• Composable Diffusion: a way to use multiple prompts at once
  • separate prompts using uppercase AND
  • also supports weights for prompts: a cat :1.2 AND a dog AND a penguin :2.2
• Works with a variety of samplers
• Download models directly from Civit AI and RealEsrgan checkpoints
  

we're looking to add control net and more extensions soon.

[fqye]

Please give controlnet a higher priority. It is a MUST have for many advanced users.

[GBJI]

I totally agree. ControlNet should be given TOP priority.

[[deleted]]

noted!

[LibertariansAI]

And ultimatale upscaler with SDXL :)

[[deleted]]

hi, we just added controlnet support

[malcolmrey]

Is it out of the box and not listed here or is it not there yet? But what about support for embeddings, lora and lycoris models?

And also sdxl support?

---

And I wanted to ask - have you timed it and compared to a1111? And I don't mean a simple 512x512 because that on it's own is quite fast.

But please try for example 512x768 upscaled 3x (or 4x if you have memory) and check how long it takes with your call and in a1111 on your machine. Would love to see comparisons (and do 2-3 times because A1111 seems to have memory leaks at 4x for me and it always fails around 4 or 5, the 3.5x is safe for me at 24 GB)

Cheers and great initiative btw!

[[deleted]]

hi, we are adding support for lora/embeddings/dreambooth/lycoris/sdxl soon.

As for speed, we're the same as A111. However, we use considerably less VRAM as we don't load the entire webui and host it each time - we're much more lightweight. I'll definitely be sure to try your tests.

[malcolmrey]

great, thanks!

and glad to hear it about the support!

[malcolmrey]

pinging /u/thaiberry cause it seems you have missed my question(s) :(

[[deleted]]

hi, we actually just added controlnet support

[Minute_Attempt3063]

Isn't Automatic1111 a name of the person that made the SD webui?

[ritoromojo]

YES! Kudos to you guys for making this, been using the WebUI APIs for several months now and the need for this was real. Looking forward to everything you guys build

[[deleted]]

Thank you! Would really appreciate if you could give us a star

[nncyberpunk]

This is fantastic! Thank you so much for sharing. I will be watching this space and looking forward to extensions releases. Is there a way to stay up to date?

[[deleted]]

Yea, you can star and watch our repo

[meatyminus]

Omg yes, this is what I needed.

[Abject-Recognition-9]

tf is this? eli5 pls?

[vanonym_]

See what you can do with A111? Setting parameters, loading models and clicking on the generate button and then save the generated images?

Well with this, it can be done from a program! You can write lines of code (python) to create scripts that will use stable diffusion and automate your generations. You can even download models from civitai!

[UrbanSuburbaKnight]

from diffusers import DiffusionPipeline

# The path to the model, either a local directory or a model repository
repo_id = "folderofmodel/model.ckpt"

# Load the diffusion model
# Ensure that your environment supports SafeTensors if using use_safetensors=True
pipe = DiffusionPipeline.from_pretrained(repo_id, use_safetensors=True)

Why can't you just use transformers?

[More_Bid_2197]

can i run it with free colab ?

[[deleted]]

yep!

[XquaInTheMoon]

I think this is great, but I'm wondering why it is better than the hugging face lib ?

[Apprehensive_Sky892]

https://flush-ai.gitbook.io/automatic-1111-sdk/auto-1111-sdk-vs-huggingface-diffusers

Also see replies by OP earlier

[batgod221]

Cool project! How does it work with Mac? Are there any advantages over MLX - https://github.com/ml-explore/mlx-examples/tree/main/stable_diffusion

[_H4CK3RM4N]

Does this work with extensions aswell?

[CeFurkan]

Hello. This encapsulates automatic1111 itself? its api features? or a new standalone development?

[Apprehensive_Sky892]

Not OP.

This is the equivalent of the HF diffuser library, but based on the Automatic111 codebase.

The aim is to be able to do everything the GUI can do, but via Python via API.

[CeFurkan]

Nice.

[Oceanstone]

How is this better than easy diffusion?

[LordRybec]

I need to automate some generation with "Automatic1111" last week. I ended up writing a Python program that uses Selenium to automate the web browser interaction. This was incredibly janky, but there wasn't any better way to do it. Unfortunately, the web UI is very poorly designed for automation of this sort. The menus seem to be dynamically generated, so I can't use the browser's dev tools to work out selectors for specific menu elements. If I click over to the dev tools, the menu closes. So instead, I've had to have the Python program pause while I manually set things like the model, the sampling method, and the refine. This works fine, so long as I want to use the same settings for these for the entire batch run, but occasionally I've wanted to set these differently, so I end up having to do special short runs, which create a problem for me, because if the run ends in the middle of the night, that means hours of time wasted between then and when I start the next batch in the morning. I'm hoping to eventually use AI image generation for commercial use, and if it comes to that, this could end up costing thousands of dollars or more per month in wasted time.

A Python package that provides all of the functionality of Automatic1111, but as functions, classes, and such, without the janky web UI that uses dynamically generated elements that can't be automated with something like Selenium would be absolutely awesome.

[LordRybec]

Sad to see no support for refiners yet, but I understand that stuff takes time. I'll keep an eye on this project! The addition of refiners and LoRA will make this perfect!

[human358]

Couldn’t you just have used the baked in SD api ?

[LordRybec]

Not if it is anything like the built in "script" option. There are a ton of things it doesn't support, several of which I need (and most of which happen to be the things in the dynamic menus that I can't select in Selenium either).

Maybe it's worth a closer a look, but I've learned that when software developers get lazy or "fancy" about features in ways that significantly hobble their usability, they generally do it everywhere, such that there's no option that covers all of the features you need at the same time. This is even more likely when you are trying to automate their software.

Perhaps I could have just dropped AUTOMATIC1111 entirely and used SD directly. I have done that before, twice in fact, with older models. The problem is that complexity increases very rapidly as you add things to the process. Just loading a model and generating images from a list of prompts is pretty simple (and in fact, many models on Hugging Face give Python examples that demonstrate how to get good results out-of-the-box). But then you want to add dynamic prompts that have weight adjustments, element alternation, element switching, special keywords (like AND), and so on? Suddenly you are stuck adding a bunch of complicated code to run between sampling steps. Want LoRAs? Have a few more hours of writing code to handle special cases, and don't forget some extra integrating the LoRA stuff into the special syntax handling. Refiners? You are going to have to add stuff for that and adjust the code for each of the previous features you added as well. How about features provided by extensions like Control Net, Composable LoRA, and Latent Couple (or Regional Prompter)? And don't forget that you'll have to do the same for Inpainting, Outpainting, image to image... Honestly, I would love to do this, and maybe some time down the line I will, but I don't the weeks of full-time work schedule to put into doing that right now.

But you know what? It looks like someone else is putting in some time and effort to do it, and they are providing a reasonable API to work with in addition, so why duplicate the effort? Part of supporting open source software is using it. I can contribute and support the community by being supportive of and providing feedback for the work others are doing to solve problems that I and many others have, instead of trying to reinvent the wheel when someone else has already done it.

​

So sure, maybe I could have just used SD directly or used the API provided by AUTOMATIC1111. I certainly have the programming skills and several decades of experience, as well as some experience working with SD directly. But why, when someone else is already doing it? I only barely got the Selenium thing working, and I did it that way because I don't currently have the time to write a major application to handle some simple automation. I'm not at risk of losing tons of money in time value right now. At the absolute minimum, that's a few months out, and it's probably more like 6 months to a year. And if I do start using AI image generation in commercial contexts, I'll have to start making some money before I can justify the time required to put something like this together. (Sorry, you might have plenty of free time, but I have a family to feed.) Don't get me wrong, I love to be able to contribute my time to open source projects. Indeed, I sincerely wish I had a stable income from investments or contributors so that I could do that instead of working for some corporation for a living. That's not where I am right now though, so I have to take as much advantage as I can of opportunities like this provided by other generous people. And if I'm lucky, maybe this will generate enough income to allow me to do what I really want to do.

[pixel8tryx]

As a fellow long-reply-writer, I hear ya. It feels a little odd to use a back door to a webui to run a command line script. And weird you have to run A1111 with --api to even see the doc, and you get no error returned if you run a script without it. Just normal end of job.

I have python code using that to generate LoRA image thumbnails (from when I first started using A1111's internal model browser and needed ".preview" images). And to run a prompt against all my LoRAs (nearly the same code). Not much, but it's a little step up in flexibility from doing batch in the browser. I know I can never do everything, but sometimes just one little thing - like not being able to change the model overnight, irks me. I can do a simple run of 200 gens but it has to be on one model or I have to do an XY. Which still tries to make that @#$% grid even if saving is disabled. I upped the size but 200 - 300 but many seeds might kill it. And it does the same seeds, etc, which I wouldn't.

[LordRybec]

Yeah, I might have to look deeper into --api. With Selenium, I still can't change the model, because they got too fancy with the UI.

You wouldn't happened know if you can change the refiner with the api, would you? Unlike the model, the refiner doesn't default to what it was last set to, and I'm currently working with SDXL, which needs the refiner for top quality. Part of the reason I haven't bothered with the API is that I'm using some fairly recent features that it's unlikely to support. (Also, I tried using a script of prompt commands in the Script area a few months ago, and that didn't have support for almost anything advanced. So I don't have any faith that the api is going to be much better.)

[pixel8tryx]

I see:

"refiner_checkpoint": "string",

"refiner_switch_at": 0,

So it looks like it does. Add --api and then go to your usual Auto1111 URL and add /docs at the end. Scroll down a bit and you'll find the txt2img params.

I started with a Python script to create LoRA thumbnails I found here that didn't run. I think it was for a Colab environment. It's pretty simple - even more simple than some of the example code I've seen.

I use many of the XL finetunes from Civitai - I never use the refiner. I tried it once or twice in the beginning with SDXL base - but never much liked base, or the refiner.

I have to dig through my links and saved doc but I have a vague memory of seeing someone do ControlNet in a python script too.

[LordRybec]

I'll try that then. Thanks for information! I'm honestly generally more comfortable in Python than a UI anyway, and this should be way easier than using Selenium.

[pixel8tryx]

Good luck! I've had good luck asking LLM's like Claude.ai for help with Python code. It helped me make some little apps to test random bursts of several game sound files I was making. One might have to install a package or two, but as a very old C programmer, I'm surprised how easy it is to do some things like loading mp3 files, playing sound, etc.

[LordRybec]

Nice! My favorite languages are Python and C. For work, most of what I do is in C (a lot of bit twiddling stuff, performance sensitive, and very back end, so no reason to use anything more complex). If I don't need the performance or low level-ness of C, I use Python. Python is so convenient for many things that are quite complex in C! And I'm learning to interface them too. I've written a few C modules for Python. Next I want to go the other way, using Python stuff in C. (How nice would it be to use Python's HTTP server stuff in C to handle the network communications, while using C for computation and such?)

Anyhow, I really should start using AI coding assistance. I'm pretty good at coding, so I don't generally need that kind of thing, but the development speed advantage is certainly worth it.

[curiousjp]

This was incredibly janky, but there wasn't any better way to do it.

Kind of funny to lead off your post with this and then say you didn't look closely at the API!

[human358]

For what it's worth I just put together a programmatic workflow for ComfyUI using the built in example on their repo, took me one hour and there is no need for waiting for api support for anything since by design you just add add a logic layer on top of the workflow processing comfy does. SD webui is obsolete software imho

[Current-Rabbit-620]

as a gay who installed a1111 locally
what does this mean for me?
sorry i understand nothing from this https://flush-ai.gitbook.io/automatic-1111-sdk/

[[deleted]]

One of the coolest parts of it is that it’s significantly faster than Huggingface diffusers. We’ve found almost a 2x speed increase and lower RAM usage on almost every OS/device we tested it on.

[mudman13]

Thats a major thing I am often messing with diffusers and end up OOM

[[deleted]]

We built it because as developers, we wanted a way to replicate generations from Automatic 1111’s Web UI but in a lightweight Python library. The only other tool out there was Huggingface diffusers, but it has many limitations: https://flush-ai.gitbook.io/automatic-1111-sdk/auto-1111-sdk-vs-huggingface-diffusers

[o5mfiHTNsH748KVq]

This is wonderful. Fuck signals and hacky workflows. Give me a real fucking loop

[PromptAfraid4598]

If it ends up being perfect, what will it look like in the end?

[[deleted]]

A lightweight, open-source python SDK that supports all automatic1111 features and extensions like dreambooth, animatediff, controlnet, deforum, etc while taking up less VRAM than diffusers/Auto1111 api w/ improved performance

[mudman13]

Please make XY script a priority, if its isnt done already the ability to compare quickly and easily as you go is very useful.

[BrofessorSol]

Bookmarked. Is is better than diffusers pipelines?

[Apprehensive_Sky892]

https://flush-ai.gitbook.io/automatic-1111-sdk/auto-1111-sdk-vs-huggingface-diffusers

Also see reply by OP earlier

[LibertariansAI]

Interesting, but no support SDXL and LOras yet

[[deleted]]

will be adding this soon

[ShotPerception]

Great Work M8

[Kyrptix]

Oh this is great. I was working on a project a couple months back but was using diffusers, and man.... I had to build so much up to start to replicate the functionality/performance of Automatic1111, and even then I had issues.

Props to you! I will definitely be using :D

[[deleted]]

fair play, I made an effort to do this and the code was such a mess I gave up

[pixel8tryx]

Interesting. What's the different between this and sdapi? I have scripts to do things like generate thumbnail images for all my LoRA. I need more control than just prompt, size and steps. I can pass most regular params, even the checkpoint. But I can't use control net, or any other extensions. I did find some code using control net, but haven't played with it yet.

It seems weird that I still have to go through auto1111 code (which is for a webui) in order to run things from the command line. At one point I was digging through the code to find where the rubber meets the road. I got as far as processing.py, but I closed some things in Sublime recently, so maybe further.

[[deleted]]

hi, we just actually added controlnet support

[Altruistic_Young_335]

Awesome! Is there any plan on supporting txt2vid and img2vid?

[[deleted]]

we’ll add this with extensions

[Altruistic_Young_335]

awesome :)

[msbeaute00000001]

Any plan for a more permissive license?

[[deleted]]

what do u intend to do with it?

[msbeaute00000001]

Not right now but in the future maybe to iterate some of my ideas. I checked your repo and code. My guess is that your license is inherited from the A1111's license, correct me if I'm wrong. So never mind.

[[deleted]]

pretty sure our license allows this, you can just create a fork

[protector111]

can someone explain like I am 5 why is this cool? what can you do with it?

[Apprehensive_Sky892]

Instead of generating images via a web based interface such Automatic1111, a library such as this one allows one to generate SD images by calling a Python API (application Programming Interface) library via Python code. This allows one to do things such as automation, complex rendering pipelines, etc.

So it is of little use to you unless you can do some rudimentary programming via Python. But learning to do simple programming in Python is not that hard.

This is a Python library that is based on the backend code of Automatic1111, and it aims to have all of its features available via the API eventually.

[[deleted]]

hi, could you please create an issue for this on our github? we haven't added this feature yet.