1.2. Introduce

Chapter 1.Background

Why We Made This: Reason for Creating the Document

As generative AI has only recently emerged, it has quickly become a valuable tool for creators. While numerous services are offering various useful features, many still have their limitations. Among these, Stable Diffusion and ComfyUI have gained significant popularity due to their open-source nature. Their strength lies in the fact that users can freely import or create models, use workflows, and custom nodes, adapting them for tailored outcomes suitable for both individuals and businesses. Although Stable Diffusion may not be the absolute best tool, most services use the original Stable Diffusion model as their foundation.

However, due to the high level of flexibility, new users can easily struggle without a proper understanding of key concepts. This is not a flaw of the ComfyUI software itself, but rather a challenge inherent to the nature of generative AI, where the models are developed first and the methods for using them are developed later. Through over a year of experience working on more than 30 corporate projects and communicating with numerous users, our team has come to understand the challenges and needs users face. This led us to create a document to help those who are motivated to learn ComfyUI by providing them with a resource they can use for self-study.

We hope this document will be a valuable resource for those learning ComfyUI. We've made an effort to thoroughly cover the basics, keeping in mind that many existing workflows can be created by effectively combining the elements of Chapter 4: Core and Chapter 5: Templates. With enough practice, you'll become an expert at using ComfyUI in no time. The core concepts can be mastered in as little as 2-3 days. However, since this document was created in a short period, some parts may be lacking, and certain complex technical concepts might have been omitted. Please feel free to point out any issues, and we'll make corrections. We aim to reflect all feedback appropriately and continually improve this resource.

How to Use: Guide for Using the Document

Let's get comfortable with this in an enjoyable way. The difficulty level is not high at all— it's just that the concepts and terms might feel unfamiliar at first. Once you try it and read through it a couple of times, you'll become familiar quickly!

Strategic Use of the Document

First, refer to Chapter 5 and Chapter 7 or external communities to understand what ComfyUI is and what it can achieve.

During this process, use the table of contents in Chapter 4 and Chapter 6 to get an idea of how to grasp the key concepts and what topics to study. Skip over any sections you're already familiar with and strategically extract only the necessary parts. Once you feel reasonably confident, lightly read through the document to check if you've missed or misunderstood anything.

Alternatively, you can read through and practice step by step from the beginning, building and creating your workflows as you go, which won't take much time. If you have any questions, feel free to leave a comment. Others who have had similar experiences may offer advice, and our team will also provide as much assistance as possible.

Setting Goals

Before strategizing, briefly consider what level of skill you want to achieve in each category.

  • Changing the input or coefficient values of existing workflows. (Reaching this level is already quite sufficient.)
  • Creating your own workflow from scratch. (Achieving this level is also quite easy.)
  • Creating and using custom nodes or models, or building large-scale systems for corporate projects. (We welcome business inquiries.)

Background Knowledge

Image Generation AI Models

Models that generate new images based on learned data.

Stability AI's Stable Diffusion is widely used as a general-purpose model.

Stable Diffusion

An open-source image generation AI model released by Stability AI.

As the technology behind image generation AI evolves, various versions have been released.

Although SD3.0 has been released recently, SD1.5 and SDXL remain the most widely used versions.

(This document focuses on SDXL but also includes explanations for SD1.5.)

New models like Flux and Pony are gaining attention, and an ecosystem is developing around them (as of August 2024).

Once this ecosystem is fully developed, it is expected that the main model will shift from SDXL to other models.

(However, the concepts explained in SDXL are quite widely applicable elsewhere as well.)

Many AI image generation models have been derived from Stable Diffusion.

ComfyUI

An open-source program that allows you to run image generation AI models using a node system.

Node System

A system that creates desired results by connecting various blocks with lines.

It is widely used in computer graphics and game development programs, with Unreal Engine and Blender being prominent examples.

Compared to traditional programming, it's easier to understand, making it ideal for designing complex logic.

Workflow

A workflow refers to the connections between individual nodes and their configuration values, all saved in a single JSON file.

License: Licensing and Copyright

This document is subject to the following licensing terms:

  • Personal use for educational purposes is strongly encouraged.
  • Use as teaching material by instructors is permitted, but the source must be credited.
  • When sharing this material, please be sure to include the source (https://nordy.ai).
  • Redistribution or use for commercial purposes is prohibited.