| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0 | 0 | 0 |
| D_CHF | 0 | 0 | 0 |
| D_OTH | 0 | 0 | 0 |
5. Structuring the Markov Model
Learning Objectives and Outline
Learning Objectives
- Understand differences between rates and probabilities, hazard rates, relative risks, and other relevant model inputs.
- Understand rate-to-probability conversion formulas and transition probability embedding.
- Explain how to embed a transition probability matrix with a defined timestep.
- Explore age-, time- and country/region-specific adjustments to a Markov model.
Outline
- Rates, probabilities, and other model inputs.
- Constructing the transition probability matrix.
- Model dynamics
- Non-stationary rates: mortality and other time/age-varying parameters.
- Tunnel states and other disease-specific transition dynamics
- Working backwards: adapting an existing model to your country/region.
Goal
What Never Happens
Problem
- Need transition probabilities but literature-based parameters are reported as:
- Rates
- Hazard ratios
- Odds ratios
- Relative risks
- Transition probabilities (rare!)
How can we deal with this?
Solution
- Need “common ground” where we can combine and transform different model inputs.
- This “common ground” is often found in a transition rate matrix.
Solution
Solution
Solution
Solution
Transition Rate Matrix
- The central “hub” of a Markov model.
- Straightforward to convert rate matrix into a transition probability matrix.
- Can be used to change the cycle length.
- Facilitates modeling using alternative techniques:
- Continuous time Markov
- Discrete event simulation
Four-Step Process
- Use data inputs/published literature to define a rate matrix R.
- Make strategy-specific adjustments to R as needed.
- “Embed” the transition probability matrix using the rate matrix
- Make further overall or strategy-specific adjustments to P as needed.
What is the difference between a rate and a probability?
Rates vs. Probabilities
- Rate: number of occurrences of an event per unit of time.
- Probability: Liklihood that an event will occur for an in individual over a defined time period.
- Major difference is in denominator: rates take into account time at risk while probabilities do not.
Rates vs. Probabilities
Rates
- Number of events divided by the total time at risk experienced by all people followed.
- Ranges from 0 to ∞.
- #events in time periodTotal time period experienced by all subjects followed
Rates
Probabilities
Rates vs. Probabilities: Example1
- Suppose a study followed 100 people with congestive heart failure for 4 years.
- At the end of 4 years, 40 had died.
- The probability of death over 4 years is 40/100=0.40.
Rates vs. Probabilities: Example1
- A rate takes into acccount the time each person was at risk.
- The 60 who survived were at risk the entire 4 years and contributed 60×4=240 years at risk.
- Once a person dies, he/she is no longer at risk.
Rates vs. Probabilities: Example1
- When a study does not report time at risk, the conventional assumption is that the events were spread evenly over the time period.
- Using this assumption, the average time at risk for the 40 who died was 2 years, adding 40 × 2 = 80 years at risk.
Rates vs. Probabilities: Example1
- Total time at risk for the cohort of 100 people is 320 person-years (240+80).
- Thus, the rate of death from CHF is 40/320 = 0.125 deaths per person-year.
- Let’s now construct a rate matrix with three states: CHF, Death from CHF, and Death from other causes.
Rates vs. Probabilities: Example
Rates vs. Probabilities: Example
- 0.125 per person year is the rate we’d enter into our rate matrix.
- We could also think of a separate death rate from background causes (e.g., 0.006 per person-year).1
Rates vs. Probabilities: Example
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0 | 0.125 | 0.006 |
| D_CHF | 0 | 0.000 | 0.000 |
| D_OTH | 0 | 0.000 | 0.000 |
Rates vs. Probabilities: Example
- The diagonal elements in a rate matrix are just the negative sum of the off-diagonal elements.
- In this example, the diagonal value for the 1st row would be -0.131 = -(0.125+0.006)
- We can leave all other rate matrix values at 0 because the rate of progression from death to other states is zero.
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| Death_CHF | 0 | 0.000 | 0.000 |
| Death_Other | 0 | 0.000 | 0.000 |
Summary: Rates vs. Probabilities
| Statistic | Evaluates | Range | Applicable Domain |
|---|---|---|---|
| Rate | #events in time periodTotal time period experienced by all subjects followed | 0 to ∞ | Rate matrix |
| Probability/risk | #events in time period#people followed for time period | 0-1 | Probability matrix |
1. Place rates in a rate matrix R.
1. Place rates in a rate matrix R.
- Country/region-specific background mortality rate (0.006)
- Disease mortality rates from existing population-based epidemiological study. (e.g., 0.125 = 40 cases per 320 person-years)
- Diagonal value that is the negative sum of the off-diagonal values in each row (-0.131).
“Natural History” (i.e., “do nothing”) rate matrix:
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| Death_CHF | 0.000 | 0.000 | 0.000 |
| Death_Other | 0.000 | 0.000 | 0.000 |
1. Place rates in a rate matrix R.
- What we have just done is contruct a rate matrix for a “natural history” model of the disease (CHF).
- This is a version of the model in which we allow the disease process to play out naturally, with no further intervention.
- Sometimes called a “do nothing” strategy.
1. Place rates in a rate matrix R.
- The “natural history” model is useful because it can help us verify that the model matches what we see in the “real world.”
- The “natural history” model also can be used to calibrate transition rates to different countries/contexts.
- For example, we could recalibrate the model so the CHF mortality transition rate matches our country-specific data.
1. Place rates in a rate matrix R.
- Alternatively, suppose we “borrow” a model developed in another country/context.
- That model will be based on underlying rates specific to that context.
- The underlying rate matrix can be used to “swap in” transition rates that apply to our country/context.
- Example: change the background mortality rate to match you country’s.
2. Make adjustments to R as needed.
2. Make adjustments to R as needed.
- Suppose that a new strategy can reduces the risk of CHF mortality by 20% (i.e., hazard ratio = 0.8).
- We can simply apply this hazard ratio directly to construct a rate matrix for strategy A.
- For the Strategy A rate matrix, the rate of CHF death is 0.8∗0.125=0.1
- Make sure that the diagonal element is adjusted to account for this change!
“Natural History” rate matrix
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| Death_CHF | 0.000 | 0.000 | 0.000 |
| Death_Other | 0.000 | 0.000 | 0.000 |
2. Make adjustments to R as needed.
- Suppose that a new strategy can reduces the risk of CHF mortality by 20% (i.e., hazard ratio = 0.8).
- We can simply apply this hazard ratio directly to construct a rate matrix for strategy A.
- For the Strategy A rate matrix, the rate of CHF death is 0.8∗0.125=0.1
- Make sure that the diagonal element is adjusted to account for this change!
“Natural History” rate matrix
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| Death_CHF | 0.000 | 0.000 | 0.000 |
| Death_Other | 0.000 | 0.000 | 0.000 |
“Strategy A” rate matrix
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.106 | 0.1 | 0.006 |
| Death_CHF | 0 | 0 | 0.000 |
| Death_Other | 0 | 0 | 0.000 |
2. Make adjustments to R as needed.
- Another adjustment we could make at this stage is the time cycle length.
- Suppose our rate matrix is defined in terms of a one-year time cycle, but we want to convert to a monthly cycle.
- In that event, we’d simply divide each rate in the matrix by 12, and the resulting matrix would be for a monthly timestep.
“Natural History” rate matrix (one-year timestep)
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| Death_CHF | 0.000 | 0.000 | 0.000 |
| Death_Other | 0.000 | 0.000 | 0.000 |
2. Make adjustments to R as needed.
- Another adjustment we could make at this stage is the time cycle length.
- Suppose our rate matrix is defined in terms of a one-year time cycle, but we want to convert to a monthly cycle.
- In that event, we’d simply divide each rate in the matrix by 12, and the resulting matrix would be for a monthly timestep.
“Natural History” rate matrix (one-year timestep)
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| Death_CHF | 0.000 | 0.000 | 0.000 |
| Death_Other | 0.000 | 0.000 | 0.000 |
“Natural History” rate matrix (one-month timestep)
| CHF | Death_CHF | Death_Other | |
|---|---|---|---|
| CHF | -0.0109 | 0.0104 | 5e-04 |
| Death_CHF | 0 | 0 | 0 |
| Death_Other | 0 | 0 | 0 |
3. “Embed” the transition probability matrix using the rate matrix
3. “Embed” the transition probability matrix using the rate matrix
- Our next step is to convert the transition rate matrix into a transition probability matrix.
- Common practice is to use rate-to-probability conversion formulas
3. “Embed” the transition probability matrix using the rate matrix
- Our next step is to convert the transition rate matrix into a transition probability matrix.
- Common practice is to use rate-to-probability conversion formulas
p=1−exp(−rt) where r is the rate and t is the time-step.
3. “Embed” the transition probability matrix using the rate matrix
- This formula works fine when there is only one possible state an individual can transition to.
- The formula will not calculate the correct transition probability if there are two or more states someone can transition to.
- See the workshop blog posting for more.
3. “Embed” the transition probability matrix using the rate matrix
- We will cover two approaches for constructing a transition probability matrix.
- The first is technically incorrect, but is widely used and easier to implement because it ignores compound transitions (i.e., multiple transitions within a cycle).
3. “Embed” the transition probability matrix using the rate matrix
The two approaches
- 3a. Construct a transition probability matrix using rate-to-probability conversion formulas.
- 3b. Embed the transition probability matrix using the rate matrix exponential (i.e., P=eR).
3a. Rate-to-probability conversion
The probability of transitioning from health state A to health state B is:
pAB=rAB∑SrAS(1−e−(∑SrAS)t) where S captures all the health states (i.e., columns) in the transition rate matrix, and t is the time-step (e.g., t=1 if 1 year, t=1/12 if one month, etc.).
3a. Rate-to-probability conversion
Let’s build on our chronic heart failure example from earlier. Here is the transition rate matrix we constructed:
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| D_CHF | 0.000 | 0.000 | 0.000 |
| D_OTH | 0.000 | 0.000 | 0.000 |
3a. Rate-to-probability conversion
Let’s now calculate the probability of transitioning from the CHF state to D_CHF (death from heart failure).
3a. Rate-to-probability conversion
Annual probability of transitioning from CHF to death from CHF:
pCHF,D_CHF=rCHF,D_CHFrCHF,D_CHF+rCHF,D_OTH(1−e−(rCHF,D_CHF+rCHF,D_OTH)×1)
We can find each of these rates in our transition rate matrix…
3a. Rate-to-probability conversion
Annual probability of transitioning from CHF to death from CHF:
pCHF,D_CHF=0.1250.125+0.006(1−e−(0.125+0.006)×1)=0.1172
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| D_CHF | 0.000 | 0 | 0 |
| D_OTH | 0.000 | 0 | 0 |
3a. Rate-to-probability conversion
Let’s now calculate the probability of transitioning from the CHF state to D_OTH (death from other causes).
3a. Rate-to-probability conversion
Annual probability of transitioning from CHF to death from other causes:
pCHF,D_CHF=0.0060.125+0.006(1−e−(0.125+0.006)×1)=0.00562
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | -0.131 | 0.125 | 0.006 |
| D_CHF | 0.000 | 0 | 0 |
| D_OTH | 0.000 | 0 | 0 |
3a. Rate-to-probability conversion
We now have the quantities needed to complete the first row of our transition rate matrix.
Recall that the diagonal elements of the transition probability matrix are just 1 minus the other transition probabilities.
3a. Rate-to-probability conversion
Calculated transition probability matrix:
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8772 | 0.1172 | 0.0056 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
3b. Exponentiate the transition rate matrix
The most technically correct approach for “embedding” a transition probability matrix is using the rate matrix exponential.
This is a matrix analogue to the cellwise rate-to-probability matrix process we just went through.
3b. Exponentiate the transition rate matrix
P=eR
3b. Exponentiate the transition rate matrix
Pros:
Embedding the transition probability matrix in this way ensures that the correct transition probabilities are calculated.
Without going into too many details, this approach ensures that some compound transitions are not “hidden” in the Markov cycle.
This ensures that our discrete time Markov model accurately represents the underlying continuous time process.
3b. Exponentiate the transition rate matrix
Cons:
A major drawback is that this approach can create some “jumpover” states that are seemingly inconsistent with the underlying model (see blog for more).
Accounting for “hidden” transitions and jumpover states requires augmenting the transition probability matrix (again, see blog for details).
3b. Exponentiate the transition rate matrix
Cons, cont’d:
Excel does not easily do matrix exponentiation, however you can use an approximation (we’ll do this in a case study).
Modern (free) statistical software can easily exponentiate a matrix:
3b. Exponentiate the transition rate matrix
Cons, cont’d:
Excel does not easily do matrix exponentiation, however you can use an approximation (we’ll do this in a case study).
Modern (free) statistical software can easily exponentiate a matrix:
3b. Exponentiate the transition rate matrix
For our chronic heart failure example, the exponentiated matrix yields a very similar answer to the first approach:
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8772 | 0.1172 | 0.0056 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
So Where Are We Now?
So Where Are We Now?
By constructing our model using the “roots” of a transition rate matrix, we can incorporate disparate sources of information.
Facilitates country/region-specific background mortality.
Facilitates standardizing inputs measured at different time intervals.
So Where Are We Now?
Not all literature-based parameters operate on transition rates.
You will often find that the strategies you want to model have inputs based on odds ratios, relative risks, risk differences, etc.
CHF Example, Continued
Recall from earlier that we constructed two transition rate matrices:
- “Natural History” rate matrix
- Strategy A (“New Drug”) rate matrix based on a hazard ratio (for CHF mortality) of 0.8.
CHF Example, Continued
These two rate matrices can be used to construct the following transition probability matrices:
m_P_NH =
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8772 | 0.1172 | 0.0056 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
m_P_A =
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8994 | 0.0949 | 0.0057 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
CHF Example, Continued
- Now suppose we wanted to model a second strategy (“B”) based on a randomized trial of another drug.
- That trial reports an odds ratio (OR) of CHF death of 0.75.
- The probability of CHF death in the placebo (control) arm in that trial was 0.15.1
CHF Example, Continued
- An odds ratio is based on the odds of an outcome happening, which is directly related to the probability.
- Odds = Probability of Outcome1−Probability of Outcome
- Odds Ratio = Odds of outcome in exposedOdds of outcome in unexposed
CHF Example, Continued
Important
Not all literature-based parameters operate on the rate scale. Some operate on the probability scale!
4. Make adjustments to P as needed.
4. Make adjustments to P as needed.
- Constructing our final transition probability matrix—for natural history or more specifically for for a stratgey under consideration–may require further adjustment.
- We must be careful about the scale on which these parameters apply.
- Odds ratios, relative risks, and risk differences all operate on the probability scale.
4. Make adjustments to P as needed.
| Statistic | Evaluates | Range | Applicable Domain |
|---|---|---|---|
| Rate | #events in time periodTotal time period experienced by all subjects followed | 0 to ∞ | Rate matrix |
| Hazard Ratio | Hazard rate of outcome in exposedHazard rate of outcome in unexposed | 0 to ∞ | Rate matrix |
| Probability/risk | #events in time period#people followed for time period | 0-1 | Probability matrix |
| Odds | Probability of Outcome1−Probability of Outcome | 0 to ∞ | Probability matrix |
| Odds Ratio | Odds of outcome in exposedOdds of outcome in unexposed | 0 to ∞ | Probability matrix |
| Relative Risk | Probability of outcome in exposedProbablity of outcome in unexposed | 0 to ∞ | Probability matrix |
| Risk Difference | Probability of outcome in exposed−Probablity of outcome in unexposed | -1 to 1 | Probability matrix |
4. Make adjustments to Ps as needed.
CHF Example, Continued
Let’s turn back to our CHF example.
- Suppose we wanted to model a second strategy (“B”) based on a randomized trial of another drug.
- That trial reports an odds ratio (OR) of CHF death of 0.75.
- The probability of CHF death in the placebo (control) arm in that trial was 0.15.
CHF Example, Continued
- We can convert an odds ratio to a relative risk (i.e., ratio of probabilities) if we know the baseline (unexposed) probability of the outcome, p0.
- In this case we were able to find p0=0.15 in the underlying clinical trial.
- If we didn’t have this information, we might assume its the same (0.1172) as in our underlying natural history probability matrix1
CHF Example, Continued
- We can convert an odds ratio to a relative risk (RR).1
- RR=Probability of outcome in exposedProbablity of outcome in unexposed=p1p0=OR(1−p0+(p0×OR))
- A relative risk is the ratio of the probability of the outcome in the exposed group to the probability of the outcome in the unexposed group.
CHF Example, Continued
- RR=p1p0
- p1=RR×p0
- pDCHF=0.75×.15=0.1125
CHF Example, Continued
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8772 | 0.1172 | 0.0056 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8994 | 0.0949 | 0.0057 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
| CHF | D_CHF | D_OTH | |
|---|---|---|---|
| CHF | 0.8819 | 0.1125 | 0.0056 |
| D_CHF | 0.0000 | 1.0000 | 0.0000 |
| D_OTH | 0.0000 | 0.0000 | 1.0000 |
Full Process
Transition Dynamics
Transition Dynamics
- We have focused on “static” transition rate/probability matrices.
- Often, transitions vary as a function of time.
Transition Dynamics
- Nonstationary rates: background mortality rate rises with age.
- Nonstationary rates: transition to disease status may vary over different ages.
- Time-dependent event transitions: rate of adverse events after disease onset is higher in first cycle, lower in subsequent cycles.
Age-Varying Background Mortality
- Useful data source is life table data.
- May be available from vital statistics division in your country.
- Also available (by region) from the UN and other organizations.
Background Mortality
- You can fit a Gompertz survival model to each age’s life table data.1
- The Gompertz model will estimate two parameters: a shape (a) and a rate (b) parameter.
- Gompertz parameters (by gender) are shown in the table.
Background Mortality
- You can fit a Gompertz survival model to each age’s life table data.
- The Gompertz model will estimate two parameters: a shape (a) and a rate (b) parameter.
- Gompertz parameters (by gender) are shown in the table on next slide.
Background Mortality
| Age | a_shape | b_rate |
|---|---|---|
| 0 | 0.08386 | 8e-05 |
| 1 | 0.08630 | 7e-05 |
| 2 | 0.08644 | 8e-05 |
| 3 | 0.08653 | 8e-05 |
| 4 | 0.08658 | 9e-05 |
| 5 | 0.08661 | 1e-04 |
Background Mortality
- Gompertz fit to mortality data for two selected ages (40- and 90-year old U.S. males)
Background Mortality
- Suppose we have a model of males starting at age 40.
- Each age has its own estimated shape (a) and rate (b) parameter.
- We can find the values of a and b for our 40 year old male cohort:
Background Mortality
- Suppose we have a model of males starting at age 40.
- Each age has its own estimated shape (a) and rate (b) parameter.
- We can find the values of a and b for our 40 year old male cohort:
| Age | a_shape | b_rate |
|---|---|---|
| 40 | 0.0908323 | 0.0017132 |
Background Mortality
- These shape and rate values are now fixed for our model
- That is, we only need these two values, as they allow us to model the remaining life expectancy of our 40-year-old male cohort as it ages through our model!
Background Mortality
As our cohort ages through the model (40, 41, 42, …), the background death rate at each subsequent age can be calculated using a simple formula.
Given the a and b values for a 40-year-old males, we can calcualte the death rate at any subsequent age as re=ba(eat2−eat1)
Background Mortality
- The Gompertz parameters for 40-year old males (US life table data) are:
- a = 0.0908
- b = 0.0017
Background Mortality
- At t=0 (i.e., age 40), the mortality rate for the cohort over the next year is:
- rt=0=0.0017934=0.00171320.0908323(e0.0908323⋅1−e0.0908323⋅(0)))
- Using a rate-to-probability conversion formula, this corresponds to a probability of death in the next year of p=1−exp(−r)=1−exp(−0.0017934)=0.0017918
Verifying Accuracy
- Using this mortality rate, we can (randomly) draw death times (years from age 40) for 1 million 40 year old males.
- The average life expectancy is the mean of this value.
- In this example, the life expectancy is 38.3 years.
Background Mortality
- This life expectancy closely matches other published data
Background Mortality
- We can now repeat this exercise for every cycle in our model.
- This gives us a different background mortality rate for every cycle.
| cycle | age | a_shape | b_rate | r_mort | p_death |
|---|---|---|---|---|---|
| 0 | 40 | 0.0908 | 0.0017 | 0.0018 | 0.0018 |
| 1 | 41 | 0.0908 | 0.0017 | 0.0020 | 0.0020 |
| 2 | 42 | 0.0908 | 0.0017 | 0.0022 | 0.0021 |
| 3 | 43 | 0.0908 | 0.0017 | 0.0024 | 0.0024 |
| 4 | 44 | 0.0908 | 0.0017 | 0.0026 | 0.0026 |
| 5 | 45 | 0.0908 | 0.0017 | 0.0028 | 0.0028 |
Background Mortality
- We can now repeat this exercise for every cycle in our model.
- This gives us a different background mortality rate for every cycle.
| cycle | age | a_shape | b_rate | r_mort | p_death |
|---|---|---|---|---|---|
| 50 | 90 | 0.0908 | 0.0017 | 0.1683 | 0.1549 |
| 51 | 91 | 0.0908 | 0.0017 | 0.1843 | 0.1683 |
| 52 | 92 | 0.0908 | 0.0017 | 0.2018 | 0.1828 |
| 53 | 93 | 0.0908 | 0.0017 | 0.2210 | 0.1983 |
| 54 | 94 | 0.0908 | 0.0017 | 0.2420 | 0.2150 |
| 55 | 95 | 0.0908 | 0.0017 | 0.2650 | 0.2328 |
Nonstationary Rates
- You may have other parameters that vary by age.
- Disease specific mortality rate:
Source: Russell et al. (2016)
Nonstationary Rates
- You may have other parameters that vary by age.
- Disease specific mortality rate:
- Age-specific intervention rates/probabilities
Source: Russell et al. (2016)
Nonstationary Rates
- It is straightforward to incorporate these dynamics into a Markov model.
- Essentially, you recalculate the transition probability matrix in each cycle.
- This works long as the underlying rates/probabilities change by age.
Nonstationary Rates
- This makes the Excel a bit more complicated, because you have to incorporate visual basic code to recalculate.
- We have constructed an optional case study that draws on age-specific background mortality rates – see the workshop website for more!
Other Transition Dynamics
- Suppose, instead, that event/transition rates, costs, quality of life vary over the course of a disease or other process in our model.
- Example: cost, quality of life weight, or risk of adverse event (e.g., death) is different in first year after disease onset, and changes thereafter.
- These types of dynamics are more challenging to implement – but you can often do it!
Other Transition Dynamics
One option is to simply build a decision tree around events that occur at higher rates shortly after disease onset, surgery, drug initiation, etc.
The decision tree can follow people until they reach a “steady state” — then the Markov model can pick up from there.
Other Transition Dynamics
Another option is “tunnel states” — simply expand the health state to follow people over the first few cycles of onset.
Tunnel states are “non-markovian,” so they need to be added once you have the transition probability matrix defined.
Tunnel States: Healthy, Sick, Dead
- Let’s take a simple healthy, sick, dead model.
- Illness onset rate = 0.0314
- Background mortality rate = 0.0094
- Illness increases the risk of death by a hazard ratio of 5.45.
Tunnel States: Healthy, Sick, Dead
Transition rate matrix: R
| H | S | D | |
|---|---|---|---|
| H | -0.0408 | 0.0314 | 0.0094 |
| S | 0.0000 | -0.0513 | 0.0513 |
| D | 0.0000 | 0.0000 | 0.0000 |
Tunnel States: Healthy, Sick, Dead
Transition probability matrix: exp(R)
| H | S | D | |
|---|---|---|---|
| H | 0.96 | 0.03 | 0.01 |
| S | 0.00 | 0.95 | 0.05 |
| D | 0.00 | 0.00 | 1.00 |
Tunnel States: Healthy, Sick, Dead
Now suppose that the probability of death from disease varies by time since disease onset.
- 0.08 in first year
- 0.06 in second year
- 0.04 in third year onwards
We can add tunnel states to our transition probability matrix to accomodate this.
Tunnel States: Healthy, Sick, Dead
- Here is the new transition probability matrix.
- Notice how the tunnel states “force” people to transition either to the next disease state if they do not die of the disease in the cycle.
| H | S1 | S2 | S | D | |
|---|---|---|---|---|---|
| H | 0.96 | 0.03 | 0.00 | 0.00 | 0.01 |
| S1 | 0.00 | 0.00 | 0.92 | 0.00 | 0.08 |
| S2 | 0.00 | 0.00 | 0.00 | 0.94 | 0.06 |
| S | 0.00 | 0.00 | 0.00 | 0.96 | 0.04 |
| D | 0.00 | 0.00 | 0.00 | 0.00 | 1.00 |
Tunnel States: Healthy, Sick, Dead
- If we run out the Markov trace for five cycles, here are the results.
- You can find a nice discussion of how to incorporate tunnel states into an Excel model here.
| cycle | H | S1 | S2 | S | S_total | D |
|---|---|---|---|---|---|---|
| 0 | 1000 | 0 | 0 | 0 | 0 | 0 |
| 1 | 960 | 30 | 0 | 0 | 30 | 10 |
| 2 | 922 | 29 | 28 | 0 | 57 | 22 |
| 3 | 885 | 28 | 26 | 26 | 80 | 35 |
| 4 | 849 | 27 | 25 | 50 | 102 | 49 |
| 5 | 815 | 25 | 24 | 72 | 121 | 63 |
Working Backwards
Working Backwards
- Suppose someone has already constructed a detailed Markov model for the decision problem you are working on.
- However, this model was constructed for another country/region/context.
- Rather than start from “scratch” with your own model, you might be able to adapt it!
Working Backwards
Essentially, this means working backwards!
Working Backwards
Essentially, this means working backwards!
Working Backwards
- We won’t go into the (very mathematical) details, but we will show you how to do this!
- Essentially, we will “solve” for what is called the generator matrix for the (existing) transition probability matrix.
- This is done through a process known as “eigenvalue decomposition.”
- Note that this is not always possible, depending on the matrix!
Working Backwards
Let’s start with the example transition probability matrix from earlier:
m_P =
| H | S | D | |
|---|---|---|---|
| H | 0.96 | 0.03 | 0.01 |
| S | 0.00 | 0.95 | 0.05 |
| D | 0.00 | 0.00 | 1.00 |
Working Backwards
To obtain the rate matrix (m_R) from an existing transition probability matrix (m_P):
Working Backwards
To obtain the rate matrix (m_R) from an existing transition probability matrix (m_P):
m_R =
| H | S | D | |
|---|---|---|---|
| H | -0.040822 | 0.0314139 | 0.0094081 |
| S | 0.000000 | -0.0512933 | 0.0512933 |
| D | 0.000000 | 0.0000000 | 0.0000000 |
Working Backwards
- We now have the rate matrix that is the “hub” or “root” of the underlying Markov model!
- From here, we could adapt rates using country-specific parameters.
- For example, we could adapt the rate of background mortality to match our country’s/region’s mortality rate.
- We could also add additional states, or make other changes as needed.
Working Backwards
- Once we make the changes we need, we just go back to our usual process.
- “Embed” the new transition probability matrix using rate-to-probability conversion formulas, or using exponentiation.
Now You Try It!
References
Gidwani, Risha, and Louise B. Russell. 2020. “Estimating Transition Probabilities from Published Evidence: A Tutorial for Decision Modelers.” PharmacoEconomics 38 (11): 1153–64. https://doi.org/10.1007/s40273-020-00937-z.
Russell, Louise B., Sri Ram Pentakota, Cristiana Maria Toscano, Ben Cosgriff, and Anushua Sinha. 2016. “What Pertussis Mortality Rates Make Maternal Acellular Pertussis Immunization Cost-Effective in Low-and Middle-Income Countries? A Decision Analysis.” Clinical Infectious Diseases 63 (suppl_4): S227–35.
Zhang, Jun, and Kai F. Yu. 1998. “What’s the Relative Risk?A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes.” JAMA 280 (19): 1690–91. https://doi.org/10.1001/jama.280.19.1690.
5. Structuring the Markov Model