Local AI image generation. Stability Matrix

✨ Introduction: About Stability Matrix

We’ll be using Stability Matrix, a cross-platform package manager for Stable Diffusion that works on Windows, Linux, and macOS. Additionally, Stability Matrix supports installing various panels for managing image generation, including those that work with Flux models.

Stability Matrix handles everything for you, so you don’t need to install multiple dependencies separately and manually. This is nice because all necessary dependencies are kept in one place and won’t interfere with your operating system.

For example, if you already have a specific version of Python installed, Stability Matrix will install the required version anyway and use only that one without modifying yours. If you don’t have Python installed, it won’t appear in your system after installing a panel via Stability Matrix – the version installed by Stability Matrix will be used exclusively within it.

🖼️ Pictures

Images generated using flux1-dev-bnb-nf4-v2 model.

⚙️ Installing Stability Matrix

Go to GitHub page of Stability Matrix repository and download it from Releases page:

Or from the buttons in the README:

You can install Stability Matrix or run it without installation (portable). I recommend portable. To do this, simply create a folder for Stability Matrix and run the executable file from that folder.

Of course, agree the license agreement – without even reading it:

Next, choose the installation path or use Portable Mode:

At the end of installation, Stability Matrix will suggest you install panel right away.

If you’re not sure what you need, click Skip first-time setup:

🚀 Installing and Launching first panel

Panels can be installed from the Packages tab:

Click Add Package:

Here, you’ll find a list of panels available for installation. Each panel specifies its compatibility: CPU, NVIDIA, DirectML, AMD (Linux), macOS, etc.

If this is your first time, I recommend “Stable Diffusion WebUI” by AUTOMATIC1111 or “Stable Diffusion WebUI Forge” by lllyasviel.

Both panels, “Stable Diffusion WebUI” and “Stable Diffusion WebUI Forge” are very similar.

However, Forge version offers more features and supports not only SD (Stable Diffusion) models, also Flux.

Starting Flux models locally

You do not have CLIP state dict!

nozsh

When you’ve chose the panel you need, click Install:

Installation may take a lot of time, depending of the panel.

When installation is complete, click 🚀 Launch. The panel startup may also take some time.

If everything runs good and there are no critical errors, output should look something like:

1
2
3

...
local URL:  http://127.0.0.1:7860
...

If this URL does not open automatically in your browser, click manually or copy and paste into your browser.

You can learn how to use WebUI by AUTOMATIC1111 in the article about. Stable Diffusion (Text in Russian)

Or watch video on YouTube:

Low VRAM Warning

1
2
3
4
5
6

[Low VRAM Warning] You just set Forge to use 100% GPU memory (**** MB) to load model weights.
[Low VRAM Warning] This means you will have 0% GPU memory (0.00 MB) to do matrix computation. Computations may fallback to CPU or go Out of Memory.
[Low VRAM Warning] In many cases, image generation will be 10x slower.
[Low VRAM Warning] To solve the problem, you can set the 'GPU Weights' (on the top of page) to a lower value.
[Low VRAM Warning] If you cannot find 'GPU Weights', you can click the 'all' option in the 'UI' area on the left-top corner of the webpage.
[Low VRAM Warning] Make sure that you know what you are testing.

In some cases, you may get Low VRAM Warning.

This warning indicates that you have allocated too much video memory to the panel, which may cause the generation speed degrade over time.

Solution: Reduce allocated memory by 500-1500 MB. For example, if you have 12 GB of VRAM, allocate 11 GB for panel.

How to do this depends of the panel, but here’s how it’s make in WebUI Forge:

🔍 Stability Matrix Overview

How to install models

Models, LoRAs, encoders, VAEs, and everything else for all panels are located in /Data/Models. The main directories are:

• /StableDiffusion/ - Models (any, not only SD)
• /Lora/ - LoRAs
• /CLIP/ - Encoders
• /VAE/ - VAEs

Packages

The Packages tab, as you’ve probably guessed, here only for managing packages (panels). This includes installation, updates, removal, launching, etc.

The Inference sub-tab (inside Add Package) is used for installing panels primarily designed for running models.

The Training sub-tab (inside Add Package) is for installing panels designed for training models.

Inference

Inference is a redesigned interface for Stable Diffusion, built into Stability Matrix.

Some might find it useful, but self, I prefer using the stock interface of the installed panel.

Checkpoint Manager

If Packages is a package (panels) manager, then Checkpoint Manager is a manager for all your installed models, LoRAs, encoders, etc.

Model Browser

In Model Browser, you can browse models and download them from Hugging Face.

Additionally, you can link your Civitai account (Settings - Accounts) to browse and download models, LoRAs, and other resources from Civitai from Stability Matrix.

Output Browser

In Output Browser, you can view all generated images.

Workflows

Workflow ready-made processes and scenarios to simplify making of some tasks.

📦 Where to find models

Probably the best website for finding models is Civitai.

Civitai not only offers a big pick of models but also allows creators to publish images. Users can also upload images generated with these models, making it easy to see what you can got.

You can also find models on Hugging Face.

And of course, you can find models in the internet.