Skip to content
2 changes: 1 addition & 1 deletion docs/source/en/api/pipelines/pag.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ pipeline.enable_model_cpu_offload()
```

> [!TIP]
> The `pag_applied_layers` argument allows you to specify which layers PAG is applied to. Additionally, you can use `set_pag_applied_layers` method to update these layers after the pipeline has been created. Check out the [pag_applied_layers](#pagappliedlayers) section to learn more about applying PAG to other layers.
> The `pag_applied_layers` argument allows you to specify which layers PAG is applied to. Additionally, you can use `set_pag_applied_layers` method to update these layers after the pipeline has been created. Check out the [pag_applied_layers](#pag_applied_layers) section to learn more about applying PAG to other layers.

If you already have a pipeline created and loaded, you can enable PAG on it using the `from_pipe` API with the `enable_pag` flag. Internally, a PAG pipeline is created based on the pipeline and task you specified. In the example below, since we used `AutoPipelineForText2Image` and passed a `StableDiffusionXLPipeline`, a `StableDiffusionXLPAGPipeline` is created accordingly. Note that this does not require additional memory, and you will have both `StableDiffusionXLPipeline` and `StableDiffusionXLPAGPipeline` loaded and ready to use. You can read more about the `from_pipe` API and how to reuse pipelines in diffuser [here](https://huggingface.co/docs/diffusers/using-diffusers/loading#reusing-models-in-multiple-pipelines).

Expand Down
2 changes: 1 addition & 1 deletion docs/source/en/api/pipelines/stable_diffusion/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ The table below summarizes the available Stable Diffusion pipelines, their suppo
</tr>
<tr>
<td class="px-4 py-2 text-gray-700">
<a href="./stable_diffusion_safe">StableDiffusionPipelineSafe</a>
StableDiffusionPipelineSafe (deprecated; no doc page)
</td>
<td class="px-4 py-2 text-gray-700">filtered text-to-image</td>
<td class="px-4 py-2"><a href="https://huggingface.co/spaces/AIML-TUDA/unsafe-vs-safe-stable-diffusion"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue"/></a>
Expand Down
6 changes: 3 additions & 3 deletions docs/source/zh/optimization/speed-memory-optims.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ http://www.apache.org/licenses/LICENSE-2.0
> [!TIP]
> 查看 [torch.compile](./fp16#torchcompile) 指南以了解更多关于编译以及如何在此处应用的信息。例如,区域编译可以显著减少编译时间,而不会放弃任何加速。

对于图像生成,结合量化和[模型卸载](./memory#model-offloading)通常可以在质量、速度和内存之间提供最佳权衡。组卸载对于图像生成效果不佳,因为如果计算内核更快完成,通常不可能*完全*重叠数据传输。这会导致 CPU 和 GPU 之间的一些通信开销。
对于图像生成,结合量化和[模型卸载](./memory#模型卸载)通常可以在质量、速度和内存之间提供最佳权衡。组卸载对于图像生成效果不佳,因为如果计算内核更快完成,通常不可能*完全*重叠数据传输。这会导致 CPU 和 GPU 之间的一些通信开销。

对于视频生成,结合量化和[组卸载](./memory#group-offloading)往往更好,因为视频模型更受计算限制。
对于视频生成,结合量化和[组卸载](./memory#组卸载)往往更好,因为视频模型更受计算限制。

下表提供了优化策略组合及其对 Flux 延迟和内存使用的影响的比较。

Expand Down Expand Up @@ -113,7 +113,7 @@ pipeline(
</hfoption>
<hfoption id="group offloading">

[组卸载](./memory#group-offloading) 将单个管道组件(如变换器模型)的内部层移动到 GPU 进行计算,并在不需要时将其卸载。同时,它使用 [CUDA 流](./memory#cuda-stream) 功能来预取下一层以执行。
[组卸载](./memory#组卸载) 将单个管道组件(如变换器模型)的内部层移动到 GPU 进行计算,并在不需要时将其卸载。同时,它使用 [CUDA 流](./memory#cuda-stream) 功能来预取下一层以执行。

通过重叠计算和数据传输,它比模型 CPU 卸载更快,同时还能节省内存。

Expand Down
2 changes: 1 addition & 1 deletion docs/source/zh/tutorials/using_peft_for_inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ pipeline.unet = torch.compile(pipeline.unet, mode="reduce-overhead", fullgraph=T
pipeline("A bowl of ramen shaped like a cute kawaii bear").images[0]
```

如果你想在编译模型后配合多个 LoRA 一起使用,又不想每次都重新编译,可以查看下文的 [hotswapping](#hotswapping) 部分。
如果你想在编译模型后配合多个 LoRA 一起使用,又不想每次都重新编译,可以查看下文的 [hotswapping](#热切换) 部分。

## 权重缩放

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
4 changes: 2 additions & 2 deletions examples/community/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ Please also check out our [Community Scripts](https://github.com/huggingface/dif
| Stable Diffusion Mixture Tiling Pipeline SDXL | A pipeline generates cohesive images by integrating multiple diffusion processes, each focused on a specific image region and considering boundary effects for smooth blending | [Stable Diffusion Mixture Tiling Pipeline SDXL](#stable-diffusion-mixture-tiling-pipeline-sdxl) | [![Hugging Face Space](https://img.shields.io/badge/🤗%20Hugging%20Face-Space-yellow)](https://huggingface.co/spaces/elismasilva/mixture-of-diffusers-sdxl-tiling) | [Eliseu Silva](https://github.com/DEVAIEXP/) |
| Stable Diffusion MoD ControlNet Tile SR Pipeline SDXL | This is an advanced pipeline that leverages ControlNet Tile and Mixture-of-Diffusers techniques, integrating tile diffusion directly into the latent space denoising process. Designed to overcome the limitations of conventional pixel-space tile processing, this pipeline delivers Super Resolution (SR) upscaling for higher-quality images, reduced processing time, and greater adaptability. | [Stable Diffusion MoD ControlNet Tile SR Pipeline SDXL](#stable-diffusion-mod-controlnet-tile-sr-pipeline-sdxl) | [![Hugging Face Space](https://img.shields.io/badge/🤗%20Hugging%20Face-Space-yellow)](https://huggingface.co/spaces/elismasilva/mod-control-tile-upscaler-sdxl) | [Eliseu Silva](https://github.com/DEVAIEXP/) |
| FABRIC - Stable Diffusion with feedback Pipeline | pipeline supports feedback from liked and disliked images | [Stable Diffusion Fabric Pipeline](#stable-diffusion-fabric-pipeline) | [Notebook](https://github.com/huggingface/notebooks/blob/main/diffusers/stable_diffusion_fabric.ipynb)| [Shauray Singh](https://shauray8.github.io/about_shauray/) |
| sketch inpaint - Inpainting with non-inpaint Stable Diffusion | sketch inpaint much like in automatic1111 | [Masked Im2Im Stable Diffusion Pipeline](#stable-diffusion-masked-im2im) | - | [Anatoly Belikov](https://github.com/noskill) |
| sketch inpaint xl - Inpainting with non-inpaint Stable Diffusion | sketch inpaint much like in automatic1111 | [Masked Im2Im Stable Diffusion XL Pipeline](#stable-diffusion-xl-masked-im2im) | - | [Anatoly Belikov](https://github.com/noskill) |
| sketch inpaint - Inpainting with non-inpaint Stable Diffusion | sketch inpaint much like in automatic1111 | [Masked Im2Im Stable Diffusion Pipeline](#masked-im2im-stable-diffusion-pipeline) | - | [Anatoly Belikov](https://github.com/noskill) |
| sketch inpaint xl - Inpainting with non-inpaint Stable Diffusion | sketch inpaint much like in automatic1111 | [Masked Im2Im Stable Diffusion XL Pipeline](#masked-im2im-stable-diffusion-pipeline-xl) | - | [Anatoly Belikov](https://github.com/noskill) |
| prompt-to-prompt | change parts of a prompt and retain image structure (see [paper page](https://prompt-to-prompt.github.io/)) | [Prompt2Prompt Pipeline](#prompt2prompt-pipeline) | [Notebook](https://github.com/huggingface/notebooks/blob/main/diffusers/prompt_2_prompt_pipeline.ipynb) | [Umer H. Adil](https://twitter.com/UmerHAdil) |
| Latent Consistency Pipeline | Implementation of [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://huggingface.co/papers/2310.04378) | [Latent Consistency Pipeline](#latent-consistency-pipeline) | - | [Simian Luo](https://github.com/luosiallen) |
| Latent Consistency Img2img Pipeline | Img2img pipeline for Latent Consistency Models | [Latent Consistency Img2Img Pipeline](#latent-consistency-img2img-pipeline) | - | [Logan Zoellner](https://github.com/nagolinc) |
Expand Down
2 changes: 1 addition & 1 deletion examples/controlnet/train_controlnet_sd3.py
Original file line number Diff line number Diff line change
Expand Up @@ -585,7 +585,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=77,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--dataset_preprocess_batch_size", type=int, default=1000, help="Batch size for preprocessing dataset."
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_flux.py
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=77,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_flux.py
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_flux2.py
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--text_encoder_out_layers",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_flux2_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_flux2_klein.py
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--text_encoder_out_layers",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_flux_kontext.py
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_lumina2.py
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=256,
help="Maximum sequence length to use with with the Gemma2 model",
help="Maximum sequence length to use with the Gemma2 model",
)
parser.add_argument(
"--system_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_sana.py
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=300,
help="Maximum sequence length to use with with the Gemma model",
help="Maximum sequence length to use with the Gemma model",
)
parser.add_argument(
"--complex_human_instruction",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_sd3.py
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=77,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_lora_z_image.py
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=512,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)

parser.add_argument(
Expand Down
2 changes: 1 addition & 1 deletion examples/dreambooth/train_dreambooth_sd3.py
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=77,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=300,
help="Maximum sequence length to use with with the Gemma model",
help="Maximum sequence length to use with the Gemma model",
)
parser.add_argument(
"--validation_prompt",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ def parse_args(input_args=None):
"--max_sequence_length",
type=int,
default=77,
help="Maximum sequence length to use with with the T5 text encoder",
help="Maximum sequence length to use with the T5 text encoder",
)
parser.add_argument(
"--validation_prompt",
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/optimization.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ def get_cosine_schedule_with_warmup(
The number of steps for the warmup phase.
num_training_steps (`int`):
The total number of training steps.
num_periods (`float`, *optional*, defaults to 0.5):
num_cycles (`float`, *optional*, defaults to 0.5):
The number of periods of the cosine function in a schedule (the default is to just decrease from the max
value to 0 following a half-cosine).
last_epoch (`int`, *optional*, defaults to -1):
Expand Down
3 changes: 0 additions & 3 deletions src/diffusers/pipelines/flux/pipeline_flux_controlnet.py
Original file line number Diff line number Diff line change
Expand Up @@ -368,9 +368,6 @@ def encode_prompt(
pooled_prompt_embeds (`torch.FloatTensor`, *optional*):
Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting.
If not provided, pooled text embeddings will be generated from `prompt` input argument.
clip_skip (`int`, *optional*):
Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that
the output of the pre-final layer will be used for computing the prompt embeddings.
lora_scale (`float`, *optional*):
A lora scale that will be applied to all LoRA layers of the text encoder if LoRA layers are loaded.
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ def encode_prompt(
torch device
batch_size (`int`):
batch size of prompts, defaults to 1
num_images_per_prompt (`int`):
num_videos_per_prompt (`int`):
number of images that should be generated per prompt
prompt_embeds (`torch.Tensor`, *optional*):
Pre-generated text embeddings. If not provided, text embeddings will be generated from `prompt` input
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,7 @@ def encode_prompt(
torch device
batch_size (`int`):
batch size of prompts, defaults to 1
num_images_per_prompt (`int`):
num_videos_per_prompt (`int`):
number of images that should be generated per prompt
prompt_embeds (`torch.Tensor`, *optional*):
Pre-generated text embeddings. If not provided, text embeddings will be generated from `prompt` input
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/kandinsky5/pipeline_kandinsky.py
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ def __init__(

self.prompt_template = "\n".join(
[
"<|im_start|>system\nYou are a promt engineer. Describe the video in detail.",
"<|im_start|>system\nYou are a prompt engineer. Describe the video in detail.",
"Describe how the camera moves or shakes, describe the zoom and view angle, whether it follows the objects.",
"Describe the location of the video, main characters or objects and their action.",
"Describe the dynamism of the video and presented actions.",
Expand Down Expand Up @@ -326,7 +326,6 @@ def _encode_prompt_qwen(
Args:
prompt (str | list[str]): Input prompt or list of prompts
device (torch.device): Device to run encoding on
num_videos_per_prompt (int): Number of videos to generate per prompt
max_sequence_length (int): Maximum sequence length for tokenization
dtype (torch.dtype): Data type for embeddings

Expand Down Expand Up @@ -395,7 +394,6 @@ def _encode_prompt_clip(
Args:
prompt (str | list[str]): Input prompt or list of prompts
device (torch.device): Device to run encoding on
num_videos_per_prompt (int): Number of videos to generate per prompt
dtype (torch.dtype): Data type for embeddings

Returns:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ def __init__(
tokenizer_2=tokenizer_2,
scheduler=scheduler,
)
self.prompt_template = "<|im_start|>system\nYou are a promt engineer. Based on the provided source image (first image) and target image (second image), create an interesting text prompt that can be used together with the source image to create the target image:<|im_end|><|im_start|>user{}<|vision_start|><|image_pad|><|vision_end|><|im_end|>"
self.prompt_template = "<|im_start|>system\nYou are a prompt engineer. Based on the provided source image (first image) and target image (second image), create an interesting text prompt that can be used together with the source image to create the target image:<|im_end|><|im_start|>user{}<|vision_start|><|image_pad|><|vision_end|><|im_end|>"
self.prompt_template_encode_start_idx = 55

self.vae_scale_factor_spatial = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ def __init__(

self.prompt_template = "\n".join(
[
"<|im_start|>system\nYou are a promt engineer. Describe the video in detail.",
"<|im_start|>system\nYou are a prompt engineer. Describe the video in detail.",
"Describe how the camera moves or shakes, describe the zoom and view angle, whether it follows the objects.",
"Describe the location of the video, main characters or objects and their action.",
"Describe the dynamism of the video and presented actions.",
Expand Down
Loading
Loading