🎲📈 Continuous Time Markov Chain (CTMC) Models: Incorporating Stochasticity into Compartmental Epidemiology

Continuous Time Markov Chain (CTMC) models form a foundational class of stochastic compartmental models in mathematical epidemiology. In contrast to deterministic ordinary differential equation (ODE) models, which describe average behavior in large populations, CTMC models explicitly incorporate randomness through probabilistic transition mechanisms. This allows them to capture demographic stochasticity, chance extinction, and variability in outbreak trajectories that are especially important in small populations or during the early stages of an epidemic.

In a CTMC framework, individuals transition between compartments such as Susceptible, Exposed, Infectious, and Recovered through random events that occur in continuous time. Each transition is governed by an instantaneous rate, and the timing of events follows exponential waiting-time distributions.

────────────────────────────────────────────

🔄 1. Compartmental Structure and Flow

CTMC models typically adopt classical compartmental structures, most commonly the Susceptible–Infectious–Recovered (SIR) or Susceptible–Exposed–Infectious–Recovered (SEIR) formulations. Unlike deterministic models, the state variables in a CTMC are integer-valued, representing the exact number of individuals in each compartment at a given time.

Agent State Flow Governed by Probabilistic Events:

1. Infection (S → I or S → E):
   Susceptible individuals become infected through contact with infectious individuals. The transition rate is proportional to the number of susceptible–infectious pairs, reflecting mass-action mixing.
2. Progression (E → I):
   Exposed individuals progress to the infectious state at a constant rate σ, defining the latent period.
3. Recovery or Removal (I → R):
   Infectious individuals recover or are removed at a constant rate γ, defining the infectious period.

A defining characteristic of CTMC models is that the waiting time spent in each compartment is exponentially distributed, reflecting the memoryless (Markovian) nature of the process.

────────────────────────────────────────────

🧪 2. Mathematical Formulation

CTMC models are defined by specifying the set of possible transitions and their associated intensity (hazard) rates. The system evolves as a stochastic process in continuous time, with events occurring one at a time.

Consider a basic SIR CTMC with population counts S(t), I(t), and R(t), and total population size N.

Transition Intensities (Event Rates):

Transition Event	Change in State	Instantaneous Intensity Rate
Infection (S → I)	S → S − 1, I → I + 1	λ_inf = β · S · I / N
Recovery (I → R)	I → I − 1, R → R + 1	λ_rec = γ · I

At any given time, the total event rate is λ_total = λ_inf + λ_rec.
The time until the next event occurs is exponentially distributed with mean 1 / λ_total.
The type of event (infection or recovery) is selected probabilistically in proportion to its intensity.

This event-driven formulation underlies simulation algorithms such as Gillespie’s direct method, which are commonly used to generate sample epidemic trajectories.

────────────────────────────────────────────

📋 3. Parameter Definitions

The parameters in CTMC models have clear epidemiological interpretations and correspond directly to biological and behavioral processes.

Parameter	Definition	Role in the Model
β	Transmission rate	Governs the rate at which susceptible–infectious contacts result in new infections
σ	Latent period rate (1 / T_inc)	Controls the transition from Exposed to Infectious in SEIR-type models
γ	Recovery rate (1 / T_inf)	Controls the transition from Infectious to Recovered
N	Total population size	Determines the scale at which stochastic effects operate

────────────────────────────────────────────

📊 4. Parameter Ranges (General Viral Disease Context)

CTMC parameter values are typically aligned with those used in deterministic compartmental models, but their stochastic interpretation highlights variability around mean behavior.

Parameter	Typical Range	Unit	Context
Recovery rate (γ)	0.07 – 0.14	day⁻¹	Corresponds to an infectious period of approximately 7 to 14 days
Transmission rate (β)	0.3 – 0.6	day⁻¹	Depends on pathogen transmissibility and contact intensity
Latency rate (σ)	0.14 – 0.25	day⁻¹	Corresponds to an incubation period of approximately 4 to 7 days
Population size (N)	Small to large	individuals	CTMC effects are most pronounced for small N

────────────────────────────────────────────

🛑 5. Applicability and Limitations

Applicability – When to Use CTMC Models:

1. Small Population Dynamics:
   CTMCs are essential when modeling disease spread in small or closed populations, where random fluctuations strongly influence outcomes.
2. Extinction and Establishment Analysis:
   They are particularly well suited for estimating the probability that an outbreak dies out by chance or successfully establishes itself, especially when the basic reproduction number is close to one.
3. Early Outbreak Behavior:
   CTMCs provide realistic descriptions of the highly variable early phase of epidemics, when case numbers are low and stochastic effects dominate.

Key Assumptions and Weaknesses:

1. Homogeneous Mixing:
   Standard CTMC compartmental models assume mass-action mixing, ignoring contact networks, spatial structure, or individual heterogeneity.
2. Markovian Assumption:
   The exponential waiting-time assumption implies memoryless transitions, which may not accurately represent biological processes with more concentrated duration distributions.
3. Computational Cost:
   For large populations, simulating every stochastic event can be computationally expensive, making deterministic or approximate methods more practical.

────────────────────────────────────────────

📚 References

Allen, L. J. S. (2008). An introduction to stochastic epidemic models.

Anderson, R. M., & May, R. M. (1991). Infectious Diseases of Humans: Dynamics and Control.

Andersson, H., & Britton, T. (2000). Stochastic Epidemic Models and Their Statistical Analysis.

Becker, N. G. (1977). On a general stochastic epidemic model.

Britton, T., Pardoux, E., Ball, F., et al. (2019). Stochastic Epidemic Models with Inference.