Spectral Progressive Diffusionfor Efficient Image and Video Generation

Computational demands for visual generative models are increasing rapidly as image resolutions and video sequence lengths continue to grow. This trend reveals a fundamental scaling crisis: while scaling model and data sizes consistently yields better generation quality, the underlying cost of self-attention in Diffusion Transformers scales quadratically with the number of generated tokens.

Diffusion models have been shown to implicitly generate visual content autoregressively in the frequency domain, where low-frequency components are generated earlier in the denoising process while high-frequency details emerge only in later timesteps. This structure offers a natural opportunity for efficient generation, as high-resolution computation on noise-dominated frequencies is largely redundant.

We propose Spectral Progressive Diffusion, a general framework that progressively grows resolution along the denoising trajectory of pretrained diffusion models. To this end, we develop a spectral noise expansion mechanism and derive an optimal resolution schedule from the model’s power spectrum. Our framework supports training-free acceleration and a fine-tuning recipe that further improves efficiency and quality, enabling significant speedups on state-of-the-art pretrained image and video generation models while preserving visual quality.

Our optimal resolution schedule determines when to grow resolution by matching the frequencies that are beginning to emerge from noise with the spatial resolutions that can represent them. Using the model’s measured power spectrum and each resolution’s Nyquist limit, we estimate when high-frequency bands become informative rather than noise-dominated. This yields principled transition times from a single error tolerance parameter, reducing brittle hyperparameter tuning while avoiding unnecessary high-resolution computation early in denoising.

Motivated by the spectral autoregression property of diffusion, we progressively increase image resolution by injecting higher-frequency components along the denoising trajectory using a spectral transformation. Our framework applies natively to existing pretrained image and video generation models, without additional custom modules or architectural modifications.

Please refer to our paper for further method details.

Spectral Progressive Diffusion accelerates pretrained image generation models while preserving visual quality across both latent- and pixel-space settings. In training-free inference, our method improves the speed-quality tradeoff on FLUX.1-dev, reaching up to 7.09× wall-clock speedup and 7.36× FLOPs reduction. With lightweight fine-tuning, the same framework further improves quality and efficiency on Z-Image and PixelGen, showing that spectral progressive generation can be used both as a plug-and-play acceleration method and as a fine-tunable recipe. The framework also extends naturally to video generation. On WAN 2.1 latent-space video generation, Spectral Progressive Diffusion achieves more than 2× speedup while maintaining high-fidelity VBench performance.

Please refer to our paper for additional qualitative and quantitative results, including ablations and results on frequency-based editing.