What is GGUF and Why Should You Care?
GGUF stands for GPT-Generated Unified Format. Developed for the LLaMa.cpp framework, GGUF was initially focused on compressing and managing large language models (LLMs), making them accessible even on less powerful hardware. But here's the game-changer: GGUF is now being used to quantize state-of-the-art image generation models like Flux, resulting in a massive efficiency boost.
Think about it: previously, loading the NF4 quantized Flux Dev and Schnell models on my NVIDIA 4050 took around 5-10 minutes for Dev and 2 minutes for Schnell. With GGUF quantization, those times are obliterated! We're now talking a lightning-fast 1.5 minutes for Dev and under 30 seconds for Schnell. That's insane!
And the best part? The quality remains phenomenal. We're achieving close to the same quality as higher-precision versions, but with a fraction of the load time. This is truly a revolution in AI image generation.
If you would like to watch the tutorial video instead, here is the link:
Getting Started with Flux GGUF in ComfyUI
Excited to experience this speed and quality for yourself? Let's get these blazing-fast Flux GGUF models up and running in ComfyUI! Here's a step-by-step guide:
1. Install ComfyUI
If you haven't already, head over to the ComfyUI Github repository (linked in the video description below) and download the ComfyUI Windows portable version. Extract the zip file to your desired location. You might need a zip extractor like 7-Zip, available for free online.
2. Gather Your Models
We'll need a few models to get started. Don't worry, I've got you covered! You'll find links to all the necessary downloads in the video description. Here's a breakdown:
Flux GGUF Models: Download the Flux Dev Q4 GGUF model from Hugging Face. This offers a fantastic balance of speed and performance, especially when using Loras. Feel free to experiment with other quantization levels (Q5, 6K, Q8) to find the sweet spot for your needs.
CLIP Text Encoders: We need two CLIP text encoder models:
CLIP ViT Large Patch 14: Download this from OpenAI on Hugging Face.
T5 XXL FP8: Download the FP8 version (for storage efficiency) from Comfy Anonymous on Hugging Face.
Flux VAE: Download this from the Flux Dev model page on Hugging Face. This is crucial for decoding those beautiful images.
Lora Model (Optional): Download any Flux Lora model from CivitAI or Hugging Face. I'm using the XLabs Flux Realism Lora for a photorealistic touch.
3. Organize Your Models
Let's organize those downloaded files within the ComfyUI directory:
Flux GGUF Model: Go to your main ComfyUI folder, open the models directory, and place the Flux Dev Q4 GGUF file inside the unet folder.
CLIP Text Encoders: Place both the CLIP ViT Large Patch 14 and T5 XXL FP8 files inside the clip folder.
Flux VAE: Place the Flux VAE file inside the vae folder.
Lora Model: Place your downloaded Lora file inside the loras folder.
4. Install the GGUF Extension
Time to install the ComfyUI GGUF extension that allows us to load GGUF models:
Navigate to Custom Nodes: In your main ComfyUI folder, open the custom_nodes folder.
Open Command Prompt: Type CMD in the file explorer address bar and press Enter. This opens a command prompt within the custom nodes directory.
Clone the Repository: Type git clone followed by the link to the ComfyUI GGUF repository (from the video description). Hit Enter to clone the repository.
Go Back to Main Directory: Type cd.. twice to navigate back to the main ComfyUI directory.
Install Python Libraries: Copy the Python command for installing GGUF dependencies from the repository's Readme file and paste it into the terminal. Press Enter to execute the command.
5. Launch ComfyUI and Load the Workflow
Almost there! Now, head back to your ComfyUI Windows portable folder and run either run_cpu.bat or run_nvidia_gpu.bat, depending on your setup. This will finalize the installation and launch the ComfyUI web interface in your browser.
Once in ComfyUI:
Load the Workflow: Click on Load in the right-hand menu and select the Flux GGUF workflow you downloaded.
Enter Prompts: Type your prompts in the CLIP text encode boxes: positive prompts up top and negative prompts below.
Adjust Settings: Dial down the steps in the K Sampler node to around 4 if using Schnell. Feel free to crank up the resolution if you desire higher-quality outputs, but note that this will increase generation time.
Start Generating: Hit Queue Prompt to unleash the magic!
Embrace the Speed and Creativity
With your Flux GGUF models up and running, you're ready to experience a whole new level of AI image generation! Experiment with different Lora models, fine-tune settings, and watch as stunning images come to life faster than ever before.
🔗 Links
ComfyUI Github Repo - https://github.com/comfyanonymous/ComfyUI
ComfyUI GGUF Github Repo - https://github.com/city96/ComfyUI-GGUF
Flux Models -
https://huggingface.co/city96/FLUX.1-dev-gguf/tree/main
https://huggingface.co/city96/FLUX.1-schnell-gguf/tree/main
Clip Models -
https://huggingface.co/openai/clip-vit-large-patch14/tree/main
https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main
Flux Vae -
https://huggingface.co/black-forest-labs/FLUX.1-dev/tree/main/vae
https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main/vae
(TO USE LORA: Connect the GGUF model loader node to the Lora node, then connect the Lora node to the Ksampler node. Be advised that you will need to also make sure there's always Lora loaded to use the workflow. If you no longer want to use the Lora revert back to the default workflow.)
xLabs Realism Lora - https://huggingface.co/XLabs-AI/flux-RealismLora/tree/main
Flux ComfyUI GGUF Workflow:
https://drive.google.com/file/d/1lWsaVtESydGudEakK0CW9C28qHSVOlcu/view?usp=sharing
0 Comments