How to use Stable Diffusion was actually shockingly easy.

MLBoy
2 min readFeb 24, 2024

--

  • Stable Diffusion is an AI that generates images from text, and I had the impression that
    it would be difficult .
    The name seems complicated (stable diffusion model), and I get the impression that the guys who generate beautiful images have to hack and defeat them, and the WebUI seems difficult to set up.
    There are so many different models out there that I’m confused as to which one to use.
    So I was scared for about a year,
  • It’s simple and
    easy to use if you can use python.
  • This is serious.
    So how to use it with impressively simple code.

Simple usage

install diffusers

pip install diffusers

find checkpoint

Download model checkpoints from the web.
It is a several GB file with the extension .safetensors or .ckpt.

There are several web pages with checkpoints, such as HugginFace Model Hub .

As an example, instead of using the general StableDiffusion checkpoints,
I will use checkpoints trained with anime-specific data.

Initialize pipeline

Initialize by passing the checkpoint file.

from diffusers import StableDiffusionXLPipeline

pipe = StableDiffusionXLPipeline.from_single_file("animagine-xl-3.0.safetensors")

execution

image = pipe('a machine learning boy').image[0]
image.save('mlboy.png')

That’s all there is to it.

a boy, in the snow

StableDiffusion has a large model called xl and a regular StableDiffusion, and the above example is for xl.
If you want to use regular StableDiffusion checkpoints, you can use StableDiffusionPipeline in the same way.

Also, you can create a pipeline from the repository name.

DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1")

There is also a reason. But wouldn’t it be easier to initialize from checkpoint? That’s what I think, but what do you think?
It seems like there are many ways to use it, but for now.

🐣

I’m a freelance engineer.
Work consultation
Please feel free to contact us with a brief development description.
rockyshikoku@gmail.com

I am creating applications using machine learning and AR technology.

I send machine learning / AR related information.

GitHub

Twitter
Medium

--

--