feat: Add Motif-Video model and pipelines

[waitingcheung]

What does this PR do?

This PR adds support for Motif-Video - a text-to-video (T2V) and image-to-video (I2V) diffusion model from Motif Technologies. The implementation includes the transformer architecture, both pipeline variants, guiding configurations, and comprehensive documentation.

Changes

New Files

• Model: src/diffusers/models/transformers/transformer_motif_video.py - MotifVideoTransformer3DModel
• Pipelines:
  • src/diffusers/pipelines/motif_video/pipeline_motif_video.py - Text-to-Video
  • src/diffusers/pipelines/motif_video/pipeline_motif_video_image2video.py - Image-to-Video
• Output: src/diffusers/pipelines/motif_video/pipeline_output.py
• Tests:
  • tests/pipelines/motif_video/test_motif_video.py
  • tests/pipelines/motif_video/test_motif_video_image2video.py
• Documentation:
  • docs/source/en/api/models/motif_video_transformer_3d.md
  • docs/source/en/api/pipelines/motif_video.md

Key Features

• Architecture: DiT-based transformer with T5Gemma2Encoder for text encoding
• Flow Match: Uses FlowMatchEulerDiscreteScheduler
• Guiding: Supports ClassifierFreeGuidance, SkipLayerGuidance, and AdaptiveProjectedGuidance
• Video Processing: Wan-style VAE for video encoding/decoding

Version Requirements

• transformers>=5.1.0 - Required for T5Gemma2Encoder (critical bug fix in PR #43633)
• The pipeline includes a version check that raises a clear error with upgrade instructions if the transformers version is too old

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[waitingcheung]

@yiyixuxu @asomoza @sayakpaul

Quick ping for visibility. This PR adds Motif-Video (T2V/I2V + new transformer and pipelines).

Would appreciate your feedback, especially on dependency/version constraints:

• transformers>=5.1.0 for T5Gemma2Encoder (currently enforced via an assertion with an upgrade message)
• compel requiring transformers<5, which may conflict with diffusers usage

This is currently blocking some diffusers-side integration, so your input would help.

A working branch for this integration is available here.

[sayakpaul]

transformers>=5.1.0 for T5Gemma2Encoder (currently enforced via an assertion with an upgrade message)

I think we can guard the transformers import in the pipeline with something like is_transformers_version("<", "5.1.0")?

compel conflict is fine IMO.

[waitingcheung]

transformers>=5.1.0 for T5Gemma2Encoder (currently enforced via an assertion with an upgrade message)

I think we can guard the transformers import in the pipeline with something like is_transformers_version("<", "5.1.0")?

compel conflict is fine IMO.

We have something like this at the top of the pipeline code to guide the users to upgrade the transformers package before importing T5Gemma2Encoder

# Check transformers version before importing T5Gemma2Encoder
if not is_transformers_version(">=", "5.1.0"):
    import transformers

    raise ImportError(
        f"MotifVideoPipeline requires transformers>=5.1.0. "
        f"Found: {transformers.__version__}. "
        "Please upgrade transformers: pip install transformers --upgrade"
    )

[sayakpaul]

Then it will cut it.

[waitingcheung]

@dg845 , @yiyixuxu
I would appreciate your feedback on this PR when you have a moment.

[dg845]

Thanks for iterating! I think this PR is close to merge. I also have the following comments:

1. The following model offloading tests fail for the I2V pipeline: test_pipeline_level_group_offloading_inference, test_sequential_cpu_offload_forward_pass, and test_sequential_offload_forward_pass_twice. I think the reason is that T5Gemma2Encoder's vision_tower currently doesn't support either block-level or leaf-level offloading. So I think it's fine to skip these tests for now.
2. For the _keep_in_fp32_modules issue (#13551 (comment)), I believe there are some bugs in the way GGUF interacts with _keep_in_fp32_modules. I will open a separate PR for this (EDIT: opened at #13697).

I'm not sure what to do about the HF Hub CI test failures or the PR documentation build failure, which look like they are both due to the fact that Motif-Video requires transformers>=5.1.0 for T5Gemma2Encoder. @sayakpaul do you have any ideas?

[waitingcheung]

@dg845
Thanks for the detailed review and suggestions!
I’ve now skipped the failing model offloading tests for the I2V pipeline as suggested.

@sayakpaul
For the transformers>=5.1.0 compatibility issue, I’ve followed up both on the upstream PR and with the repo author regarding the Transformers 5 integration for Compel:
damian0815/compel#129

I’m also open to other suggestions or approaches here if there’s a better way to handle the HF Hub CI and docs build failures in the meantime.