Synthesis and next steps

Closing session

Estimated time: 20 minutes

Slides

Introduction

You have now met every family of method this course covers, starting with a likelihood evaluated on a grid and ending with a particle filter inside a Metropolis-Hastings sampler. Each answers something the one before it could not, and each is slower, more approximate, or harder to tune.

This session puts them side by side and asks which of them your own problem needs.

Objectives

By the end of this session you will be able to

  1. say what each method in the course can do, and what makes it slow or approximate
  2. decide, for a model of your own, whether a deterministic likelihood is enough
  3. recognise the two questions that settle the choice: is the likelihood tractable, and is the model differentiable
  4. name the failure mode you should expect from the method you have chosen

The two questions

Almost every choice in this course reduces to two properties of the model in front of you.

Can you write down the likelihood? For a deterministic model with an observation distribution, yes: that is the first session, and everything from MCMC through model checking follows. For a stochastic model the likelihood integrates over every path the epidemic could have taken, and you cannot. That is where particle filters and ABC come in, from opposite directions: the filter estimates the intractable likelihood, ABC replaces it with a distance.

Can you differentiate it? If you can, NUTS will usually just work and you should reach for it first. A discrete-event simulator, a resampling step, or a hard threshold all break differentiation, and you fall back to random-walk methods with an adaptive proposal.

Working through your own problem

Bring a model you actually want to fit. The discussion works better on real problems than on the worked examples, because the awkward parts of a real problem are exactly the parts that decide the method.

For your model, answer in turn:

  1. Is the process deterministic or stochastic? If stochastic, does the stochasticity matter at the population size you are working with?
  2. Can you write a likelihood for the observations given a trajectory?
  3. Is the simulator differentiable?
  4. How expensive is one simulation, and how many can you afford?
  5. What would tell you that the fit had failed?

People skip the last one. The model checking and MCMC diagnostics sessions exist for it.

TipExercise: place your own problem

Take the model you brought and work through the five questions above. Then say out loud which method you would use and what you expect to go wrong with it.

Then compare with the person next to you. Disagreement is the useful part: if two of you reach different methods for the same problem, one has made an assumption the other has not.

Most real problems stop at question 2 or 3.

If you can write the likelihood and differentiate the model, you are in the world of the first four sessions and should use NUTS. That covers most deterministic compartmental models fitted to case counts, which is more of epidemiology than it sounds.

If the process is genuinely stochastic and the population is small enough that chance changes the answer, as on Tristan da Cunha or in a care home or an early outbreak, you need particle MCMC, and you should expect it to be slow and to need its particle count calibrated.

If you cannot even write the observation model down, ABC is the fallback, and expect your answer to depend on the distance and the tolerance you chose.

Question 4 decides whether any of this is feasible at all. A model that takes a second to simulate and needs 100,000 particle filter evaluations is a week of computing, and that is a modelling decision, not a computing one.

Where to go next

Four further topics sit below this one in the sidebar, and each stands on its own:

  • Variational inference replaces sampling with an optimisation, for when you need an answer quickly. It needs only the MCMC session as background.
  • Neural posterior estimation trains a network to map data straight to a posterior, amortising the cost across many datasets.
  • Universal differential equations put a neural network inside the model, for when part of the mechanism is unknown.
  • Beyond Turing is a tour of the wider Julia ecosystem for anyone who wants to keep working in it.

The course structure shows what each of them assumes you have already done.

Going further

  • Everything here stays online, and the material is MIT licensed, so you are free to reuse it with attribution.
  • Nowcasting and forecasting infectious disease dynamics covers the intricacies of working with epidemiological data and using them to make predictions.
  • The session timetable and course structure are the quickest way back into a specific topic.
  • The Discussion board is the place to ask about anything in the sessions, say how you got on, or show what you have fitted. It is open to everyone using the material, wherever they came across it.
  • For something in the course that is wrong, unclear, or broken, open an issue. That is genuinely useful to us and to the next cohort.
TipLearning points
  • The choice of inference method follows from two properties of your model: whether the likelihood is tractable, and whether the model is differentiable.
  • Each method in this course exists because of a limitation of the one before it, and pays for that capability in speed, in approximation, or in tuning.
  • A method you cannot diagnose is not a method you should use. Decide in advance what would tell you the fit had failed.
  • Most epidemiological models are simpler than the machinery available: use the simplest thing that survives your own checks.