Native transparent PNGs and exact prompt reuse
The FP8 model natively generates RGBA images. --transparent adds a transparency instruction to your prompt and requires a PNG output, preserving the original alpha channel. It does not remove backgrounds after generation or change the model, VAE, denoising steps or precision. Transparency remains prompt-dependent; the CLI reports alpha statistics.
python generate.py --prompt "A cute mint-green baby dragon mascot, full body" --transparent --width 1024 --height 1024 --steps 40 --warmup --output dragon.png
For several variations of the same text prompt in one loaded process, --cache-prompts reuses the exact encoder outputs. It holds at most eight prompts, never caches image edits, and does not reuse denoising features. New prompts still run the encoder. Six paired checks, including 2048 output and two transparent-image edits, produced bit-identical final latents and RGBA pixels versus the existing optimized FP8 runtime.
| Resolution | Fresh FP8, encode every request | Exact prompt cache hit |
|---|---|---|
| 1024 × 1024 | 5.964 s | 5.951 s |
| 2048 × 2048 | 37.240 s | 37.230 s |
Warmed batch 1, all 40 steps, CFG 1, same calibrated FP8 weights and BF16 attention/encoder/VAE on the RTX PRO 6000. Five interleaved measurements per mode at 1024 and three at 2048. Times include the complete generation path; cached mode omits the repeated encoder computation on cache hits. Load, compilation/warmup and PNG writing are excluded. These timings show no material general speedup. This does not reach one image per second, and cached timings do not apply to new prompts or edits.
Transparent corgi PNG · Transparent dragon PNG · 2048 watercolor PNG · Measurements and further optimization trials.
Checkerboard previews below are display aids. The linked PNGs retain native transparency.
Further speed trials
The existing optimized FP8 arithmetic remains unchanged. Additional experiments tested immutable-prefix metadata reuse, whole-transformer CUDA graph replay, an FP32-output FP8 GEMM plus exact scale epilogue, quantizer launch sizes, and aggressive pointwise compiler tuning. None provided a useful validated general speedup worth promoting under the quality priority. The native 40-step path remains enabled.
The interleaved latent-only compiler trial measured baseline 5.6820s, tuned 5.6771s and graph 5.6566s. Tuning changed the final latent (NRMSE 0.006562 versus the existing runtime) with negligible speed difference, so it was rejected. Graph replay was bit-identical but its mean improvement was below 0.5%, within the observed run variation; its added complexity and capture cost were not adopted. The broader all-directions/radius 2 compiler search was stopped after seven minutes without a result; the reported trial uses radius 1. These are exploratory latent-only timings, not the end-to-end table above. See the trial JSON files for exact outputs and individual runs. FlashInfer's inspected low-latency FP8 API targets tiny M/other GPU constraints; its SM120 groupwise GEMM requires scales of (1 or 128,128,128), incompatible with the existing per-token/per-channel scales without changing quantization.
The previous actual 1024 denoising-step trace spends roughly 93.4 ms in 224 FP8 GEMMs and 27.1 ms in 32 BF16 attention calls. Across 40 steps, matrix multiplication alone accounts for roughly 3.7 seconds at that measured kernel speed. This explains why removing setup and Python overhead cannot yield 1 image/second here. It is not a hardware lower-bound proof.
Repeated prompts
from fp8_runtime import load_pipeline
from acceleration import accelerate_pipeline
from prompt_cache import cache_prompt_embeddings
pipe = cache_prompt_embeddings(accelerate_pipeline(load_pipeline('Qwen/Qwen-Image-2.1', './transformer')))
The cache is for an immutable inference pipeline. Clear pipe._image21_prompt_cache after changing the text encoder, adapters or tokenizer. Image edits, batched prompts, explicit embeddings, custom image masks, autograd and autocast calls bypass the cache. Model movement changes its device key. Standard one-shot calls still pay model loading and warmup; use one loaded process for repeated generation.
The four native transparency samples and two edits were compared against uncached FP8 with matched prompts, seeds, 40 steps and precision. Full latents and all RGBA bytes match exactly in these six cases. Alpha reports distinguish transparent (0–15), translucent (16–239), opaque (240–255) pixels. The evaluation wording is frozen in the script and recorded verbatim in quality.json. The general CLI adds only background/alpha instructions and permits user-requested lettering; HELLO is an additional CLI smoke sample. This validates the new helpers in these cases; the underlying original FP8 quantization remains lossy relative to BF16, as documented elsewhere in the model card. The glass sample has a transparent surrounding canvas but a mostly opaque interior; native alpha is not a physically exact transmission model.
Experiment sources retain workstation path assumptions. Public runtime helpers are portable within the already documented environment. No extra dependency, model weight change, reduced resolution, reduced-step adapter, approximate feature reuse or attention quantization was introduced.