Text-to-Image

Use Cases

Data Generation

Businesses can generate data for their their use cases by inputting text and getting image outputs.

Immersive Conversational Chatbots

Chatbots can be made more immersive if they provide contextual images based on the input provided by the user.

Creative Ideas for Fashion Industry

Different patterns can be generated to obtain unique pieces of fashion. Text-to-image models make creations easier for designers to conceptualize their design before actually implementing it.

Architecture Industry

Architects can utilise the models to construct an environment based out on the requirements of the floor plan. This can also include the furniture that has to be placed in that environment.

Task Variants

You can contribute variants of this task here.

Inference

You can use diffusers pipelines to infer with text-to-image models.

from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler

model_id = "stabilityai/stable-diffusion-2"
scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0]

You can use Model Database.js to infer text-to-image models on Model Database Hub.

import { HfInference } from "@Model Database/inference";

const inference = new HfInference(HF_ACCESS_TOKEN);
await inference.textToImage({
  model: 'stabilityai/stable-diffusion-2',
  inputs: 'award winning high resolution photo of a giant tortoise/((ladybird)) hybrid, [trending on artstation]',
  parameters: {
    negative_prompt: 'blurry',
  }
})

Useful Resources

• Model Database Diffusion Models Course
• Getting Started with Diffusers
• Text-to-Image Generation
• MinImagen - Build Your Own Imagen Text-to-Image Model
• Using LoRA for Efficient Stable Diffusion Fine-Tuning
• Using Stable Diffusion with Core ML on Apple Silicon
• A guide on Vector Quantized Diffusion
• 🧨 Stable Diffusion in JAX/Flax
• Running IF with 🧨 diffusers on a Free Tier Google Colab

This page was made possible thanks to the efforts of Ishan Dutta, Enrique Elias Ubaldo and Oğuz Akif.