Diffusers documentation

ZImageTransformer2DModel

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.42.1).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

ZImageTransformer2DModel

用于 Z-Image 图像类数据的 Transformer 模型。

ZImageTransformer2DModel

class diffusers.ZImageTransformer2DModel

< >

( all_patch_size = (2,)all_f_patch_size = (1,)in_channels = 16dim = 3840n_layers = 30n_refiner_layers = 2n_heads = 30n_kv_heads = 30norm_eps = 1e-05qk_norm = Truecap_feat_dim = 2560siglip_feat_dim = Nonerope_theta = 256.0t_scale = 1000.0axes_dims = [32, 48, 48]axes_lens = [1024, 512, 512] )

forward

< >

( x: listtcap_feats: listreturn_dict: bool = Truecontrolnet_block_samples: dict[int, torch.Tensor] | None = Nonesiglip_feats: list[list[torch.Tensor]] | None = Noneimage_noise_mask: list[list[int]] | None = Nonepatch_size: int = 2f_patch_size: int = 1joint_attention_kwargs: dict[str, typing.Any] | None = None )

参数

  • x (list of torch.Tensor or nested list of torch.Tensor) — 输入潜变量。在标准模式下运行时为扁平列表,在 omni 模式下运行时为嵌套列表。
  • t (torch.Tensor) — 用于指示去噪步骤。
  • cap_feats (list of torch.Tensor or nested list of torch.Tensor) — 要使用的条件字幕嵌入(从输入条件(例如提示词)计算得出的嵌入)。
  • return_dict (bool,可选,默认为 True) — 是否返回 ~models.transformer_2d.Transformer2DModelOutput 而不是普通的元组。
  • controlnet_block_samples (dict,键为 int,值为 torch.Tensor,可选) — 从块索引到张量的映射。如果指定,这些张量将被添加到 Transformer 块的残差中。
  • siglip_feats(list of list of torch.Tensor,可选)— 可选的 SigLIP 图像特征,用作额外的条件输入。
  • image_noise_mask (list of list of int,可选) — 在 omni 模式下,指示每张图像中噪声 token 与干净 token 的逐图像噪声掩码。
  • patch_size (int,可选,默认为 2) — 用于将输入潜变量分块的空间块大小。
  • f_patch_size (int, 可选,默认为 1) — 用于对输入潜变量进行分块的时间维度分块大小。
  • joint_attention_kwargs (dict, 可选) — 一个 kwargs 字典。仅使用其中的 scale 项,用于缩放 LoRA 层。

ZImageTransformer2DModel 的前向方法。

流程:patchify -> t_embed -> x_embed -> x_refine -> cap_embed -> cap_refine -> [siglip_embed -> siglip_refine] -> build_unified -> main_layers -> final_layer -> unpatchify

patchify_and_embed

< >

( all_image: listall_cap_feats: listpatch_size: intf_patch_size: int )

基础模式下的 Patchify:每个批次项对应单张图像。

patchify_and_embed_omni

< >

( all_x: listall_cap_feats: listall_siglip_feats: listpatch_size: intf_patch_size: intimages_noise_mask: list )

全模态模式下的 Patchify:每个批次项包含多张图像及噪声掩码。

Update on GitHub