Forcing Diffusion Models to Respect the Rules: Constrained Posterior Sampling for Time Series

Proceedings of the 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

TLDR: Constrained Posterior Sampling (CPS) is a diffusion-based sampling algorithm that aims to generate time series samples that can satisfy user-specified constraints.

arXiv

Motivation

Imagine you are a quantitative researcher who wants to stress-test trading strategies. You would want access to a tool that can precisely generate high-fidelity stock price time series for prompts like

Generate Tesla’s stock price for the next month with 5% volatility.

Additionally, for the stocks domain, there are some inherent rules that a generated sample should adhere to, such as the opening and closing prices for a day should be bounded by the highest and lowest prices for that day. This problem exists in almost all engineering domains, where certain domain-specific constraints may arise due to the laws of physics or the underlying process. We refer to this as the constrained time series generation problem.

Expectations Of A Desired Approach

Training-free: Training a generative model for specific constraints lacks scalability. For example, a model trained for mean constraints cannot easily adapt to argmax constraints.
Hyperparameter-free: The choice of guidance weights in guidance-based approaches with diffusion models significantly affects the sample quality. However, optimizing these weights becomes computationally intractable when dealing with hundreds of constraints.
Should be Independent of external models: Prior works often employ external models, like the discriminator in Generative Adversarial Networks (GANs), alongside the generative model to enforce realism, necessitating additional training and complex sampling procedures.

High level description plot — Figure 1 - High-Level Description of Constrained Posterior Sampling (CPS): Here, we show an example where CPS generates a daily stock price time series with natural constraints, such as the bounds on the opening and closing prices of the stock.

This post features Constrained Posterior Sampling, a training-free, plug-and-play approach that enforces hard constraints on the outputs from your diffusion generative models without compromising sample quality.

Constrained Posterior Sampling

CPS builds on diffusion models – a class of generative models that iteratively refine noise into data. In a diffusion model, you start with random noise and repeatedly “denoise” it to produce a sample. The key insight of CPS is: after each small denoising step, gently push the output back to satisfy the constraints before further denoising. In other words, at every step of the generation, we enforce the rules, at least a little. By the end, the output should follow all the rules by construction.

Under the Hood: How CPS Enforces Constraints in Diffusion Sampling

Given a specific constraint set from which we aim to generate a realistic time series sample, we perform the following steps:

Estimate the posterior mean. At each denoising step, we obtain an estimate of the clean sample.
Project the posterior mean estimate. We project the posterior mean estimate towards the constraint set. This is performed using off-the-shelf optimization routines.
Perform DDIM sampling. Using the projected posterior mean estimate, we obtain the noisy sample for the next denoising step using DDIM sampling. We refer the reader to our manuscript for further technical details.

This process continues for all the denoising steps. During projection, CPS performs an unconstrained optimization step that ensures minimal perturbation of the posterior mean estimate such that the constraint violation is reduced. This is concisely depicted in Figure 2.

CPS Approach — Figure 2 - Constrained Posterior Sampling: We show the graphical model for one step of denoising in CPS: check Algorithm 1 in our manuscript.

To ensure minimal effects on the sample quality, we introduce penalty coefficients that minimize the perturbations of the posterior mean estimate during the initial denoising steps when the signal-to-noise ratio is very low. Towards the final denoising steps, the penalty coefficients are very large to ensure constraint satisfaction.

Note that CPS satisfies all the requirements listed above for a desired approach.

CPS is a training-free procedure and can be extended to any constraint class.
The proposed approach is hyperparameter-free. The penalty coefficients are fixed using the noise schedule of the diffusion model, and CPS leverages existing off-the-shelf optimization solvers to perform the projection step.
CPS does not require any external model to improve sample realism. In fact, we note that the adverse effects of projection are mitigated through the subsequent denoising steps.

Applications and Results Across Domains:

In our work, we showcase the efficiency of CPS in terms of sample quality and diversity on six diverse datasets, spanning environmental, traffic, and finance domains. Particularly, in terms of sample realism and utility metrics (check Figure 3), we show that CPS outperforms SOTA approaches by 10%.

CPS Main Qualitative Results — Figure 3 - Main Qualitative Results: CPS provides high-fidelity synthetic time series samples that match real time series data. The real test samples from which the constraints are extracted are shown in blue. The samples generated using the extracted constraints are shown in red. Across all datasets, the baselines suffer from the adversarial effects of the projection step, whereas CPS generates high-quality samples.

Additionally, in terms of tracking real time series samples, we show that CPS outperforms SOTA methods by 42%. Specifically, CPS does not suffer from sample quality degradation for a large number of constraints, while other approaches break down in such settings (Figure 4). We refer the readers to Figures 3 and 4 to observe the sample quality and tracking abilities of CPS.