The world of AI image generation is constantly evolving, and one of the most exciting new developments is the emergence of Flux models. Developed by the brilliant minds at Black Forest Labs, Flux has taken the AI art scene by storm, generating jaw-dropping images that flood your social media feeds. But Flux is more than just pretty pictures – it’s a revolution in AI art accessibility, bringing high-quality image generation to a wider audience.
In this post, we'll delve deeper into the magic of Flux, exploring its different variations and understanding what sets it apart. Most importantly, we'll guide you through harnessing its power, even without a top-of-the-line PC, thanks to the innovative NF4 quantization and the versatile ComfyUI interface.
A Family of Models for Every Need
Flux is not just one model but a family of models, each with its own strengths and target users. This allows users to choose the best model for their needs, from rapid prototyping to high-quality final renders:
Flux.1 [schnell]: The speedster, ideal for quick tests and experiments. Perfect for those dipping their toes into the Flux waters.
Flux.1 [dev]: The all-rounder, striking a balance between speed and image quality. This is ideal for most artistic endeavors.
Flux.1 [pro]: The powerhouse, offering the highest performance and detail. This is the choice for professionals and those seeking absolute peak performance.
Overcoming the VRAM Hurdle
There's a catch, though. These models, especially the unquantized versions, are real VRAM hogs. The full, unquantized experience can require 20GB of VRAM or more! This is where the magic of NF4 quantization comes in.
NF4 quantization, introduced by the creator of Stable Diffusion Forge, allows you to run Flux even with a modest 4-10GB of VRAM. These NF4 versions are not just lighter on your system, they're faster too! This opens up a whole new world of possibilities for artists who don't have access to high-end hardware.
If you prefer to watch the video, you can watch it here
Introducing ComfyUI: Your AI Art Playground
To unleash the power of Flux, especially these amazing NF4 versions, we need a playground, a digital canvas. That's where ComfyUI comes in. ComfyUI is a rapidly growing platform for AI image generation, gaining popularity as the go-to GUI for Stable Diffusion. While ComfyUI can seem intimidating at first, especially if you're used to more user-friendly interfaces, fear not! I've created a super simple method to install ComfyUI along with a pre-made workflow specifically designed for these new quantized Flux NF4 models.
Setting Up Your AI Art Studio
Follow these steps to get ComfyUI up and running with those awesome Flux models:
Install ComfyUI: Download the portable standalone build for Windows. You can find the link in the description below. Extract the contents of the downloaded file to a convenient location on your hard drive.
Install Git: If you don’t have Git installed, download it from the official Git website. Install it like any other program.
Install ComfyUI Manager:
Navigate to your ComfyUI custom nodes folder (e.g., C:\ComfyUI_windows_portable_nvidia\ComfyUI\custom_nodes).
Type CMD in the address bar and press Enter.
Type git clone followed by the link to the ComfyUI Manager repository (link in the description below).
Press Enter and let Git do its magic.
Install the NF4 Extension:
Type cd.. to go back one folder.
Type cd.. again to return to the main ComfyUI directory.
Type git clone followed by the link to the ComfyUI Bits and Bytes NF4 repository (link in the description below).
Press Enter.
Install Necessary Python Library:
Type python embedded\python.exe -m pip install -r ComfyUI/custom_nodes/ComfyUI_bitsandbytes_NF4/requirements.txt into your command prompt and press Enter.
Download Flux NF4 Models:
Head over to the ComfyUI Bits and Bytes NF4 GitHub page (link in the description below).
Click the download button for both the NF4 Flux Dev and NF4 Flux Schnell checkpoints.
Place the Models in the Checkpoints Folder:
Find your main ComfyUI folder, then the models folder.
Open the checkpoints folder.
Move the downloaded model files into this folder.
Download and Save the Workflow:
Download the pre-made workflow from the link in the description below.
Create a separate folder for your workflows and save the downloaded file there.
Bringing Your AI Art to Life
Now you're ready to create! Go back to your ComfyUI portable folder and double-click the run_nvidia_gpu.bat file. ComfyUI will launch in your web browser.
Don't be intimidated by all the nodes and connections. That's just ComfyUI's way of being super powerful and flexible. Click "Load" on the right-hand side and choose the workflow you downloaded.
You should see the model checkpoint you downloaded in the checkpoint loader node. You can switch models by clicking the arrows in the loader node until you land on the model you want.
If you're using the Dev model, you can just hit "Queue Prompt" and watch the magic happen. No need for further configurations. If you're using the Schnell model, remember to change the "Steps" in the basic scheduler node to 4 instead of 20, then hit the Queue button.
Depending on your PC, generations might take anywhere from a few seconds to several minutes. Experiment with the settings and see what kind of amazing art you can create. Feel free to bump up the resolution, but be prepared for longer generation times.
Pro Tip: To keep ComfyUI and all those custom nodes up to date, just click the "Manager" button on the right. It's super easy to install updates or add new models and nodes as they come out. The ComfyUI Manager is your best friend for keeping your AI Art studio cutting edge.
So there you have it! We've just unlocked the incredible power of Flux, even on more modest hardware thanks to those NF4 models. Get out there, explore, experiment, and most importantly, have fun creating stunning AI art!
🔗 Links
ComfyUI Github Repo - https://github.com/comfyanonymous/ComfyUI ComfyUI Manager Github
Repo - https://github.com/ltdrdata/ComfyUI-Manager ComfyUI bitsandbytes NF4 Github Repo - https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4?tab=readme-ov-file
bitsandbytes command prompt - python_embedded\python.exe -m pip install -r ComfyUI\custom_nodes\ComfyUI_bitsandbytes_NF4\requirements.txt
Flux ComfyUI Workflow: https://drive.google.com/file/d/1Q1DTyaRWqD-GIE4B5RavgYFZivMqLU3y/view?usp=sharing
0 Comments