Diffusers documentation

P-H3-Image

Diffusers

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.42.1).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

P-H3-Image

P-H3-Image is an image model by Pruna AI based on MiniMax-H3, for text-to-image generation and instruction-based image editing. You can find the checkpoint at PrunaAI/P-H3-Image.

import torch
from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")  # or "mps", "xpu"

pipe = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)

prompt = (
    "A German Expressionist oil painting of a Berlin street at night in the manner of Ernst Ludwig Kirchner, "
    "with gaunt figures hurrying past a glowing gas lamp."
)
image = pipe(prompt=prompt, height=3360, width=2496, generator=torch.Generator().manual_seed(3), output="images")[0]
edited = pipe(prompt="let it snow on the street", image=image, output="images")[0]

Every request goes through the prompt upsampler first. With every component on the accelerator, the pipeline needs about 101 GB in bfloat16 at 8 megapixels, so the examples use auto offloading: the ComponentsManager moves each component onto the accelerator when a block needs it, and the pipeline runs on an 80 GB GPU.

Components

Component	Class
`transformer`	MiniMaxH3Transformer3DModel
`text_encoder`	`Qwen3VLForConditionalGeneration` (Qwen3-VL-4B)
`text_connector`	PH3ImageTextConnector
`vae`	AutoencoderKLPH3Image
`tokenizer`, `processor`	The text encoder’s tokenizer, and a `Qwen3VLProcessor` with MiniMax-H3’s image processor
`scheduler`, `audio_scheduler`	MiniMaxH3Scheduler
`prompt_upsampler`, `prompt_upsampler_processor`	`Qwen3_5ForConditionalGeneration` (based on Qwen3.5-9B) and its processor

Workflows

The workflow is selected on the image input:

text2image generates from prompt. height and width default to 2912 and are rounded to multiples of 32. The model works best at 8 megapixels, for example 2912x2912, 3872x2176 or 2176x3872.
image_conditioned edits image by the instruction in prompt, and works best at 4 or 8 megapixels. The image is encoded at about vision_pixels (one megapixel) for the text encoder and at its own size for the VAE, scaled up to reference_min_pixels (one megapixel) or down to reference_max_pixels (eight megapixels) when outside that range. Without height and width, the edit is generated at the size of the first image, which is scaled up to canvas_min_pixels (four megapixels) if smaller. Passed height and width, rounded to multiples of 32, change the canvas but not the images. image can also be a list of up to 8 images: the first is edited and the others are references, as <Picture 2>, <Picture 3> and so on.

Prompt upsampling

P-H3-Image is meant to render full prompts rather than short requests, so the pipeline runs the prompt upsampler first on every request. It thinks about the request and then writes the prompt: a detailed text-to-image prompt for the aspect ratio of height and width, or, when editing, a brief of what to keep and what to change in the images. The prompt it wrote comes back as the prompt output.

out = pipe(prompt=prompt, height=3360, width=2496, output=["images", "prompt"])
print(out["prompt"])

max_new_tokens caps the reasoning and the prompt together, 1024 tokens for text-to-image and 1536 for edits by default. If the upsampler does not return a usable prompt, the pipeline raises an error rather than rendering it.

To render prompt as given, pop the step from the blocks. The popped block also runs on its own, for example to show the prompt to a user before rendering it:

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

blocks = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image").blocks
upsampler_block = blocks.sub_blocks.pop("prompt_upsampler")

upsampler = upsampler_block.init_pipeline("PrunaAI/P-H3-Image", components_manager=manager)
upsampler.load_components(dtype=torch.bfloat16)
upsampled = upsampler(prompt=prompt, height=3360, width=2496, output="prompt")

pipe = blocks.init_pipeline("PrunaAI/P-H3-Image", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)
image = pipe(prompt=upsampled, height=3360, width=2496, output="images")[0]

Without the step, an edit prompt in either form the upsampler writes, the six-section brief or a paragraph that starts with Edit <Picture 1>., is encoded as written; any other instruction gets that prefix.

Sampling

The 8 steps run over the checkpoint’s sample_sigmas. Pass sigmas to use another grid.
The model is guidance-distilled, so there is no guider, negative_prompt or guidance_scale.

PH3ImageModularPipeline

class diffusers.PH3ImageModularPipeline

< source >

( blocks: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = Nonepretrained_model_name_or_path: str | os.PathLike | None = Nonecomponents_manager: diffusers.modular_pipelines.components_manager.ComponentsManager | None = Nonecollection: str | None = Noneworkflow: str | None = Nonemodular_config_dict: dict[str, typing.Any] | None = Noneconfig_dict: dict[str, typing.Any] | None = None**kwargs )

A ModularPipeline for P-H3-Image, an image model based on MiniMax-H3, for text-to-image (text2image) and instruction-based editing (image_conditioned).

The image is a single latent frame, denoised in one packed sequence with two audio latents that are discarded.

import torch
from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

pipe = ModularPipeline.from_pretrained("PrunaAI/P-H3-Image", components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)
prompt = (
    "A German Expressionist oil painting of a Berlin street at night in the manner of Ernst Ludwig Kirchner, "
    "with gaunt figures hurrying past a glowing gas lamp."
)
image = pipe(prompt=prompt, height=3360, width=2496, output="images")[0]

PH3ImageBlocks

class diffusers.PH3ImageBlocks

< source >

( )

Modular pipeline blocks for P-H3-Image: text2image, and instruction-based editing (image_conditioned) when image is passed. prompt goes through the prompt upsampler first; pop the prompt_upsampler step to render it as given.

Supported workflows:

text2image: requires prompt
image_conditioned: requires prompt, image

Components: image_processor (VaeImageProcessor) prompt_upsampler (Qwen3_5ForConditionalGeneration): The prompt upsampler. prompt_upsampler_processor (AutoProcessor): Its processor. text_encoder (Qwen3VLForConditionalGeneration) text_connector (PH3ImageTextConnector) tokenizer (Qwen2Tokenizer) processor (Qwen3VLProcessor) vae (AutoencoderKLPH3Image) scheduler (MiniMaxH3Scheduler) audio_scheduler (MiniMaxH3Scheduler) transformer (MiniMaxH3Transformer3DModel)

Configs: reference_min_pixels (default: 1048576) reference_max_pixels (default: 8388608) canvas_min_pixels (default: 4194304) vision_pixels (default: 1000000) sample_sigmas (default: [1.0, 0.984753, 0.967905, 0.951688, 0.925372, 0.881923, 0.810083, 0.528718, 0.0]): The checkpoint’s sampling grid, used when sigmas is not passed.

Inputs: image (Image | list, optional): The image to edit, or a list of at most 8 images: the first is edited, the others are references. height (int, optional): Height of the generated image in pixels, rounded to a multiple of 32. Defaults to 2912 for text2image, which works best at 8 megapixels. An edit defaults to the first image’s size, at least canvas_min_pixels, and works best at 4 or 8 megapixels. width (int, optional): Width of the generated image in pixels, rounded to a multiple of 32. Defaults to 2912 for text2image, which works best at 8 megapixels. An edit defaults to the first image’s size, at least canvas_min_pixels, and works best at 4 or 8 megapixels. prompt (str): The prompt, or the edit instruction when image is passed. A single string. max_new_tokens (int, optional, defaults to 1536 or 1024, depending on the workflow): The most tokens the upsampler may generate, its reasoning included. vision_images (list, optional): The images of the edit resized for the text encoder, the image to edit first. reference_images (list, optional): The images of the edit resized for the VAE, the image to edit first. condition_latents (list, optional): The conditioning latents of the images of the edit. Their shapes give the geometry of the reference blocks. generator (Generator, optional): The generator of the request. The conditioning noise is drawn from it first, one draw per condition, before the noise of the generated rows. latents (Tensor): Pre-generated image noise of shape (1, latent_channels, 1, latent_height, latent_width), used instead of the draw. sigmas (list, optional): A custom sampling grid, strictly decreasing and ending at 0. Defaults to the checkpoint’s sample_sigmas. *denoiser_input_fields (None,optional): The structural description of the packed sequence the transformer reads by name: token_tags, position_ids and the three row-index tensors. attention_kwargs (dict,optional): Additional kwargs for attention processors. output_type (str,optional*, defaults to pil): Output format: ‘pil’, ‘np’, ‘pt’.

Outputs: images (list): Generated images.

PH3ImageTextConnector

class diffusers.PH3ImageTextConnector

< source >

( hidden_state_indices: tuple = (13, 19, 25, 31, 36)text_hidden_size: int = 2560out_channels: int = 5120 )

Parameters

hidden_state_indices (tuple[int, ...], defaults to (13, 19, 25, 31, 36)) — Which entries of the text encoder’s hidden_states are read, in the numbering of output_hidden_states (0 is the embedding output, i the output of decoder layer i, and the last one the final normed hidden state).
text_hidden_size (int, defaults to 2560) — The hidden size of the text encoder.
out_channels (int, defaults to 5120) — The number of channels of the conditioning MiniMax-H3’s transformer reads.

Maps hidden states of P-H3-Image’s Qwen3-VL text encoder to the text conditioning its transformer reads.

The hidden states listed in hidden_state_indices are concatenated, standardized, projected linearly and destandardized. The first token is an attention sink with the same conditioning for every prompt, so it is replaced by sink_embedding.

forward

< source >

( hidden_states: tuple ) → torch.Tensor

Parameters

hidden_states (tuple[torch.Tensor, ...]) — The hidden states of the text encoder, as output_hidden_states=True returns them: one tensor of shape (batch_size, num_tokens, text_hidden_size) per entry.

Returns

torch.Tensor

the conditioning, of shape (batch_size, num_tokens, out_channels), in the dtype of the connector’s parameters.

Update on GitHub

←Ovis-Image PAG→