WEBVTT 1 00:00:29.090 --> 00:00:34.350 David Stephens, Prof: Okay, so we'll pick up for the second part of today 2 00:00:34.720 --> 00:00:36.670 David Stephens, Prof: where we left off, which is 3 00:00:38.120 --> 00:00:48.270 David Stephens, Prof: thinking about continuous treatments. So for continuous treatments. As we were saying, you can have models that depend on more general functions of 4 00:00:48.570 --> 00:00:51.660 David Stephens, Prof: Z, say, Z squared, or we could have interactions. 5 00:00:51.960 --> 00:00:56.930 David Stephens, Prof: And the message overall is that our correction or adjustment procedures 6 00:00:57.080 --> 00:01:00.990 David Stephens, Prof: for continuous treatments operate in the same way they do for 7 00:01:01.810 --> 00:01:08.910 David Stephens, Prof: binary treatments, in that we can think about propensity, score, regression, g, estimation, and inverse probability weighting. 8 00:01:09.590 --> 00:01:12.460 David Stephens, Prof: and the details really only 9 00:01:14.130 --> 00:01:28.310 David Stephens, Prof: differ slightly from case to case. So we might have to model, for example, the expected value of Z. Given x, or the expected value of Z. Squared. Given X. Give 2 balancing functions, b, 1 and B 2. In general, these will be different functions, of course. 10 00:01:29.800 --> 00:01:30.730 David Stephens, Prof: But 11 00:01:31.730 --> 00:01:44.499 David Stephens, Prof: if we had a model data generating model that looked like this, then we could estimate psi 0 and psi one successfully by doing a straightforward propensity score regression, just adding in 12 00:01:44.790 --> 00:01:48.369 David Stephens, Prof: the balancing score based on b 1 13 00:01:49.020 --> 00:01:58.309 David Stephens, Prof: right. We estimated this model using ordinary least squares that would give us consistent estimation of the psi 0 and psi one quantities. 14 00:01:59.780 --> 00:02:04.660 David Stephens, Prof: and that's because of the principles we learned about 15 00:02:04.820 --> 00:02:14.129 David Stephens, Prof: when discussing dags right at the start of the module, because it turns out that there's only one backdoor path that is problematic in this 16 00:02:15.790 --> 00:02:20.199 David Stephens, Prof: setting, one biasing path that's in this setting. 17 00:02:20.390 --> 00:02:23.560 David Stephens, Prof: So if we had. This is the general confounding bag. 18 00:02:24.630 --> 00:02:28.289 David Stephens, Prof: The balancing score that we've just constructed would sit there. 19 00:02:30.370 --> 00:02:35.200 David Stephens, Prof: But in the quadratic model that we was, we were proposing. 20 00:02:35.440 --> 00:02:39.830 David Stephens, Prof: That means that there is an effect of z squared on y. 21 00:02:40.270 --> 00:02:55.320 David Stephens, Prof: Now, z squared, we're going to treat here as a separate, random, variable. But of course, it is a deterministic function of Z, which means that the cause of Z squared is deterministically Z, that is, if we know Z. We know Z squared. 22 00:02:55.570 --> 00:03:01.200 David Stephens, Prof: So the appropriate dag in this case is this, but notice that 23 00:03:01.740 --> 00:03:04.550 David Stephens, Prof: there is only one confounding path 24 00:03:05.070 --> 00:03:08.389 David Stephens, Prof: from Z, and indeed Z squared that goes to Y, 25 00:03:09.330 --> 00:03:14.269 David Stephens, Prof: and it's the one that goes through X, so we can block that path by the single intervention of 26 00:03:15.090 --> 00:03:24.279 David Stephens, Prof: using the b 1 function right? I only need to introduce one propensity score here, because there is no other confounding path 27 00:03:24.640 --> 00:03:25.670 David Stephens, Prof: in the dag. 28 00:03:26.700 --> 00:03:35.439 David Stephens, Prof: So that means that this model is sufficient to get estimation of the site parameters correct. 29 00:03:38.330 --> 00:03:43.440 David Stephens, Prof: If I, however, I had an interaction between Z squared and x 1, say. 30 00:03:44.190 --> 00:03:48.380 David Stephens, Prof: then, what that does actually is to introduce 2 31 00:03:48.550 --> 00:03:55.189 David Stephens, Prof: biasing paths, 2 open and directed paths between the Z terms and y. 32 00:03:55.550 --> 00:04:02.150 David Stephens, Prof: and both of those terms need to be blocked. Both of those paths need to be blocked. So what I need to do is reintroduce. 33 00:04:04.150 --> 00:04:07.739 David Stephens, Prof: A term to handle the 34 00:04:09.250 --> 00:04:19.020 David Stephens, Prof: X contributions. Sorry the Z contributions and a term to handle the Z squared contributions. 35 00:04:19.870 --> 00:04:23.280 David Stephens, Prof: So here I need both models 36 00:04:23.710 --> 00:04:28.709 David Stephens, Prof: b, 1 and B 2, both balancing scores in order to block all the paths 37 00:04:30.110 --> 00:04:37.400 David Stephens, Prof: right. And it turns out that if we just block on b 1 block using b 1, 38 00:04:37.660 --> 00:04:46.220 David Stephens, Prof: then that's no longer sufficient. I need to use a propensity score regression model which contains these terms. 39 00:04:48.190 --> 00:04:54.700 David Stephens, Prof: these terms, and all of these terms, okay. 40 00:04:55.170 --> 00:05:04.459 David Stephens, Prof: so that's 1 slight difference from the binary treatment case, for inverse probability weighting the procedure 41 00:05:04.890 --> 00:05:14.320 David Stephens, Prof: goes exactly the same as in the binary case. Remember that these were the formulae for the average potential outcome estimator. Here's the inverse weight quantity. 42 00:05:19.520 --> 00:05:24.950 David Stephens, Prof: the difficulty in implementing this exact formula in the continuous case is that 43 00:05:25.530 --> 00:05:29.909 David Stephens, Prof: this indicator here picks off the target treatment 44 00:05:32.240 --> 00:05:36.069 David Stephens, Prof: subset of the observed treated zeds. 45 00:05:36.230 --> 00:05:49.170 David Stephens, Prof: and if we have a continuous treatment, then that indicator may not hit any of the observed Z's. It may admit it may hit one of the observed Z's, and therefore the averaging that we need to do won't 46 00:05:49.640 --> 00:05:51.289 David Stephens, Prof: be any use to us. 47 00:05:51.710 --> 00:06:01.799 David Stephens, Prof: So what that means is that this, if this is a a density function here. F of Z given X is a density rather than a probability. The 48 00:06:02.460 --> 00:06:05.949 David Stephens, Prof: the inverse weighting won't be operationalizable. 49 00:06:07.900 --> 00:06:14.509 David Stephens, Prof: What we do, instead, in the numerator is to think about a little window around the target. Z, 50 00:06:15.080 --> 00:06:17.480 David Stephens, Prof: so we say, we look at Z, 51 00:06:18.050 --> 00:06:26.549 David Stephens, Prof: the observed Z values that lie between the target z minus d and the target z plus d, so that's a d neighborhood of Z, 52 00:06:26.860 --> 00:06:33.460 David Stephens, Prof: okay? So that takes a small window around the target treatment and collects all the data from that window. 53 00:06:33.920 --> 00:06:36.670 David Stephens, Prof: okay? And then proceeds in the same way. 54 00:06:37.210 --> 00:06:40.839 David Stephens, Prof: So we just have to implement inverse probability weighting 55 00:06:41.020 --> 00:06:49.689 David Stephens, Prof: with the same inverse probability. Weight form now worked out for a density, but now essentially collecting more terms into the numerator. 56 00:06:51.300 --> 00:06:57.159 David Stephens, Prof: But other than that, the principles are exactly the same. This inverse probability, weighting approach works 57 00:06:57.620 --> 00:07:00.339 David Stephens, Prof: perfectly successfully for continuous treatments. 58 00:07:02.340 --> 00:07:02.740 David Stephens, Prof: Okay. 59 00:07:03.380 --> 00:07:16.959 David Stephens, Prof: we can also do what's called stabilized weight estimation for the continuous case which looks at a weight formed in this way, basically relying on me coming up with a data. Sorry, a data, an estimate of the 60 00:07:17.320 --> 00:07:19.869 David Stephens, Prof: treatment assignment model marginally. 61 00:07:20.260 --> 00:07:31.230 David Stephens, Prof: So, this is just the probability of getting treatment at a certain level. Say so. I need to estimate that. And then I use a weight which is based on the ratio of that quantity to the 62 00:07:31.800 --> 00:07:36.729 David Stephens, Prof: to the original, the original inverse probability. Weight. 63 00:07:38.880 --> 00:07:39.630 David Stephens, Prof: Okay. 64 00:07:40.220 --> 00:07:44.946 David Stephens, Prof: But again, that can be augmented and augmented. 65 00:07:45.940 --> 00:07:49.610 David Stephens, Prof: Ipw, or outcome regression could also be used 66 00:07:50.670 --> 00:07:53.319 David Stephens, Prof: all right. So the message here is that really the 67 00:07:54.180 --> 00:08:00.250 David Stephens, Prof: procedures go forward in exactly the same way in the continuous case 68 00:08:01.880 --> 00:08:09.639 David Stephens, Prof: as opposed to the discrete case that we saw earlier on G estimation, propensity, score, regression, inverse probability, weighting. 69 00:08:10.130 --> 00:08:14.960 David Stephens, Prof: augmented Ipw and augmented outcome regression. 70 00:08:15.270 --> 00:08:18.849 David Stephens, Prof: all operate in the same way for continuous treatments. 71 00:08:20.700 --> 00:08:21.164 David Stephens, Prof: Okay. 72 00:08:21.790 --> 00:08:30.890 David Stephens, Prof: if we're going to use augmented outcome regression, or indeed, Ipw, we can use, in fact, a weighted least squares approach, because that returns the same estimate. So we form a 73 00:08:31.200 --> 00:08:41.079 David Stephens, Prof: a model for the effect of treatment, and we weight it by the appropriate inverse probability weights. 74 00:08:41.429 --> 00:08:47.560 David Stephens, Prof: and then use ordinary approaches here weightedly squares to estimate. Psi. 75 00:08:48.190 --> 00:08:51.750 David Stephens, Prof: Now this approach requires. 76 00:08:51.900 --> 00:08:52.850 David Stephens, Prof: So in the 77 00:08:53.140 --> 00:09:06.009 David Stephens, Prof: this, this estimator actually returns the Ipw estimator properly, but it perhaps is easier to fit because you can fit weightedly squares more easily than the original estimator. Perhaps. 78 00:09:07.290 --> 00:09:10.359 David Stephens, Prof: Okay, so the only 79 00:09:11.340 --> 00:09:30.709 David Stephens, Prof: other thing to note in the continuous treatment case is about positivity violation. So we mentioned positivity in the binary treatment case. That meant that we couldn't have deterministic prediction of the treatment so that we didn't have a probability of Z equals one or z equals 0 for any X. 80 00:09:31.060 --> 00:09:44.099 David Stephens, Prof: We have more or less the same thing here, except that, again, what's appearing in the denominator here is now a density rather than a probability. So it might be more prone to getting very close to 0. 81 00:09:44.830 --> 00:09:51.380 David Stephens, Prof: Okay, so and if, of course it's very close to 0, then the estimator is poorly behaved. 82 00:09:52.200 --> 00:09:59.979 David Stephens, Prof: so we need to take special care in the continuous treatment case, for to handle positivity violations. 83 00:10:00.570 --> 00:10:04.220 David Stephens, Prof: but the principles of all the correction methods, the adjustment methods 84 00:10:04.940 --> 00:10:09.079 David Stephens, Prof: really are exactly the same in the continuous cases for the discrete case. 85 00:10:12.850 --> 00:10:42.349 David Stephens, Prof: So before we go into into some examples. I'll talk a bit more about the statistical methodologies associated with estimating these models. So we've touched on most of them already. We typically only have to use standard approaches. So standard distributional assumptions are normal or binomial. Or if we have a time to event outcome, we can do the same kinds of calculation with time to event distributions like the exponential or the Y-ball. 86 00:10:43.300 --> 00:10:48.069 David Stephens, Prof: we have to perform regression at least one regression in each of the 87 00:10:48.490 --> 00:10:58.160 David Stephens, Prof: analyses. But typically that's a very standard form of regression, either a linear model or ordinary least squares analysis. Or 88 00:10:58.300 --> 00:11:02.939 David Stephens, Prof: we might have to use a glm for estimating the propensity score and 89 00:11:03.280 --> 00:11:06.569 David Stephens, Prof: logistic regression some other form of binary regression. 90 00:11:07.968 --> 00:11:20.499 David Stephens, Prof: We might have a survival outcome, so we might need to use survival methods. But really, at this stage, once we've corrected for the confounding, either by including the right regressor or by inverse weighting. 91 00:11:21.620 --> 00:11:26.439 David Stephens, Prof: The remainder of the calculation is straightforward. You just have to use standard tools, essentially. 92 00:11:32.900 --> 00:11:39.680 David Stephens, Prof: something that's really coming up in the literature. More and more, though, is the advanced modeling 93 00:11:40.000 --> 00:11:48.549 David Stephens, Prof: for the model components. That is a proposed conditional, mean model or propensity or balancing model. 94 00:11:48.720 --> 00:11:51.570 David Stephens, Prof: right? And what I'm thinking about here are. 95 00:11:51.960 --> 00:11:58.790 David Stephens, Prof: oh, that should be an X there. Sorry sophisticated models for 96 00:11:59.900 --> 00:12:09.150 David Stephens, Prof: Mu or E or B that go beyond standard multiple multiple regression type models. 97 00:12:10.160 --> 00:12:13.669 David Stephens, Prof: So this is occurring. This is coming up more and more in the literature. 98 00:12:14.170 --> 00:12:16.750 David Stephens, Prof: So I'm there, I'm thinking about 99 00:12:17.560 --> 00:12:23.659 David Stephens, Prof: more sophisticated linear or generalized linear models or flexible models using splines. 100 00:12:24.320 --> 00:12:36.540 David Stephens, Prof: models based on prediction approaches so algorithms, more algorithmic type modeling. So machine learning approaches classification and regression trees 101 00:12:37.446 --> 00:12:40.783 David Stephens, Prof: neural network models, deep, deep models, 102 00:12:42.860 --> 00:12:57.349 David Stephens, Prof: different sorts of flexible approaches, boostings, and so on. So all of these approaches have been proposed in the literature. So Random Forest is another common one that's used right now. So all of these sort of algorithmic approaches 103 00:12:57.680 --> 00:13:08.080 David Stephens, Prof: reflecting modern statistics have been used. And then this other sort of ensemble approach, which basically takes a rag bag collection of these previous methods. 104 00:13:08.870 --> 00:13:15.200 David Stephens, Prof: So you could use all of these methods to fit an outcome model. 105 00:13:15.960 --> 00:13:24.510 David Stephens, Prof: a conditional, mean model that's going to appear in an Aipw analysis. The propensity score model or balancing score model. 106 00:13:26.440 --> 00:13:31.660 David Stephens, Prof: All of these approaches could be used and have been used fairly successfully. 107 00:13:31.970 --> 00:13:44.850 David Stephens, Prof: There's this famous paper on analysis of what's called his structural parameters. But it's really the same kind of causal parameters that we're thinking about in biostatistics and epidemiology. This is a 108 00:13:44.990 --> 00:13:48.859 David Stephens, Prof: multi-authored paper with a collect us 109 00:13:49.570 --> 00:13:55.620 David Stephens, Prof: galaxy of star authors. And it talks about the 110 00:13:57.180 --> 00:14:03.270 David Stephens, Prof: potential for using machine learning methods in estimating all of these quantities. 111 00:14:07.400 --> 00:14:11.370 David Stephens, Prof: When we start talking about advanced modeling of that sort, though 112 00:14:11.920 --> 00:14:16.089 David Stephens, Prof: we've got to be aware still of of the statistical limitations. 113 00:14:17.230 --> 00:14:24.080 David Stephens, Prof: and for those I would identify the following key points to consider. 114 00:14:24.470 --> 00:14:30.150 David Stephens, Prof: So if because all of these methods well, a lot of these methods are based on 115 00:14:30.850 --> 00:14:42.610 David Stephens, Prof: prediction, prioritizing prediction that essentially comes with a limited understanding of the associated uncertainty. So 116 00:14:42.810 --> 00:14:45.169 David Stephens, Prof: none of the methods give 117 00:14:46.200 --> 00:14:57.460 David Stephens, Prof: very straightforward variance calculations, for example. So if we want to understand the sampling distribution of the estimator, or compute a confidence interval for our average treatment effect, and so on. 118 00:14:58.660 --> 00:15:01.500 David Stephens, Prof: These methods don't readily give that 119 00:15:02.480 --> 00:15:09.400 David Stephens, Prof: right. It's quite difficult to understand in in also these multi-stage estimation settings, just how much. 120 00:15:09.840 --> 00:15:17.780 David Stephens, Prof: how how wide we should make the the confidence interval, for example. So we've always got access to the Bootstrap 121 00:15:18.030 --> 00:15:22.799 David Stephens, Prof: to do confidence, interval, type calculations or standard error calculations. 122 00:15:23.230 --> 00:15:33.540 David Stephens, Prof: But of course, in many settings, and especially if you're using high-dimensional, flexible models to do fitting the Bootstrap is computationally burdensome. 123 00:15:36.360 --> 00:15:38.370 David Stephens, Prof: so the methods are very clever. 124 00:15:38.600 --> 00:15:46.440 David Stephens, Prof: and they can do many things, many aspects very well, but they do have limitations in terms of representing uncertainty. 125 00:15:47.050 --> 00:15:58.729 David Stephens, Prof: The other important thing that I would point out is the danger of positivity violations, which is that if you're using these advanced methods, these flexible methods to 126 00:15:59.350 --> 00:16:06.750 David Stephens, Prof: construct the propensity score, that is, to get an understanding of the Z given X model, then 127 00:16:07.180 --> 00:16:11.719 David Stephens, Prof: that on the surface looks like a prediction problem predicting Z given X. 128 00:16:13.230 --> 00:16:17.350 David Stephens, Prof: And these flexible methods are very good at doing that. 129 00:16:17.660 --> 00:16:20.550 David Stephens, Prof: But in the causal estimation problem 130 00:16:20.980 --> 00:16:24.450 David Stephens, Prof: predicting Z from X is not the fundamental goal. 131 00:16:26.230 --> 00:16:33.290 David Stephens, Prof: You want to understand the confounding structure and induce balance using the propensity score. 132 00:16:33.960 --> 00:16:44.940 David Stephens, Prof: So prediction of the treatment is not the fundamental goal. And there's a danger with flexible modeling procedures that you can be overly good in terms of prediction. 133 00:16:45.590 --> 00:16:49.520 David Stephens, Prof: that is, you can construct a model that predicts Z. Precisely. 134 00:16:50.050 --> 00:16:58.120 David Stephens, Prof: If you if you have that, of course, then you are immediately led to think about positivity, violations and lack of comparability. 135 00:17:00.120 --> 00:17:09.089 David Stephens, Prof: Okay, so one needs to be a bit wary when applying these methods in classical propensity score settings. 136 00:17:11.119 --> 00:17:23.410 David Stephens, Prof: Of course you can target your objective function in these flexible approaches to represent an understanding of balance or an appreciation of balance inducement, and some some methods have been proposed to do that 137 00:17:27.980 --> 00:17:33.810 David Stephens, Prof: understanding the theoretical guarantees for these advanced model approaches is 138 00:17:33.910 --> 00:17:36.510 David Stephens, Prof: much more difficult than for standard approaches. 139 00:17:37.590 --> 00:17:44.890 David Stephens, Prof: It's hard to understand the nature of the statistical operating characteristics. 140 00:17:45.000 --> 00:17:51.070 David Stephens, Prof: to study consistency, to study asymptotic normality, to construct confidence intervals for large samples. 141 00:17:51.370 --> 00:17:54.759 David Stephens, Prof: and in some cases, and indeed in math. 142 00:17:55.090 --> 00:18:05.879 David Stephens, Prof: Double Ml. Paper, that I referenced, they make the point that a lot of the standard statistical machinery doesn't apply. So you need more sophisticated theoretical techniques. 143 00:18:07.090 --> 00:18:17.289 David Stephens, Prof: And this is still being understood by the theoreticians. Some of these methods are available in our packages. 144 00:18:17.570 --> 00:18:22.329 David Stephens, Prof: but the theory behind them, and therefore the 145 00:18:22.470 --> 00:18:28.090 David Stephens, Prof: ability to use them in practice in a fully successful way is limited. I think at the moment. 146 00:18:28.420 --> 00:18:33.789 David Stephens, Prof: It's a big area for current research, especially with the advances in machine learning, for example. 147 00:18:34.380 --> 00:18:45.850 David Stephens, Prof: but understanding the theoretical properties which is key in statistics, I think, for key for statistical applications, that that theory is not very well developed yet. 148 00:18:48.630 --> 00:18:59.499 David Stephens, Prof: And so we wrote a paper on this topic. My coauthors and I published in 2019 in statistics in medicine, pointing out that 149 00:18:59.840 --> 00:19:09.639 David Stephens, Prof: some of the advanced methods, and including in particular the ensemble method. So the standard method that people were using for propensity score construction 150 00:19:09.820 --> 00:19:13.509 David Stephens, Prof: doesn't really do any better than simpler models. So we compared 151 00:19:14.290 --> 00:19:24.040 David Stephens, Prof: logistic regression against more complicated advanced modeling approaches and showed that the logistic regression, even in 152 00:19:25.070 --> 00:19:28.369 David Stephens, Prof: quite complicated real examples, could do a good 153 00:19:29.220 --> 00:19:32.020 David Stephens, Prof: job in doing the confounding correction. 154 00:19:36.420 --> 00:19:40.090 David Stephens, Prof: So the next topic I want to mention is about variance estimation 155 00:19:40.350 --> 00:19:46.449 David Stephens, Prof: which is related to the things I was just mentioning. And this is a a. 156 00:19:48.280 --> 00:19:52.339 David Stephens, Prof: a topic that's that comes up in many statistical settings. 157 00:19:52.570 --> 00:20:02.120 David Stephens, Prof: so many of you may have come across the idea of sandwich estimation or robust variance estimation which is a method for developing 158 00:20:02.340 --> 00:20:10.320 David Stephens, Prof: confidence intervals based on minimal distribution assumptions using frequent asymptotic theory. 159 00:20:10.990 --> 00:20:17.849 David Stephens, Prof: So the sandwich estimator for the variance of an of a particular kind of estimator 160 00:20:20.030 --> 00:20:27.410 David Stephens, Prof: is is the standard construction, and we'll be seeing some examples of that in the applications coming up. 161 00:20:28.446 --> 00:20:35.920 David Stephens, Prof: The Sandwich estimator, which I'll go through the details of in a second can be applied in almost all of the cases that we've been looking at. 162 00:20:39.680 --> 00:20:42.170 David Stephens, Prof: And I'll show some implementations of that. 163 00:20:42.310 --> 00:20:49.569 David Stephens, Prof: Some causal packages have built in robust variance estimation. So we'll see Dr. GE. For doing G. Estimation. 164 00:20:50.300 --> 00:20:54.060 David Stephens, Prof: One topic that I'll also discuss in 165 00:20:54.870 --> 00:21:07.970 David Stephens, Prof: the simulation on the example section is the accounting for the estimation of nuisance parameters. You'll remember that in the 1st stage of a statistical analysis for propensity score regression, we need to estimate the propensity score from the data. 166 00:21:08.860 --> 00:21:13.410 David Stephens, Prof: And the question is, do we need to account for that estimation when reporting 167 00:21:13.810 --> 00:21:18.810 David Stephens, Prof: the variance or the uncertainty in the average treatment effect. 168 00:21:19.730 --> 00:21:26.160 David Stephens, Prof: Okay? And it turns out that in certain classes of models we can actually ignore that uncertainty. 169 00:21:28.570 --> 00:21:45.390 David Stephens, Prof: Okay? And for propensity, score regression, gestimation, and in some inverse probability, weighting settings. It turns out that we can essentially use the Plugin estimates of the nuisance parameters, and that doesn't affect the variance at all. We'll see that in some examples 170 00:21:45.812 --> 00:21:50.485 David Stephens, Prof: we could always use Bootstrap for for all of these methods as well, but that is 171 00:21:51.180 --> 00:22:01.459 David Stephens, Prof: especially in the regression. And Ipw approaches. That's a standard workhorse tool that is always available to us, but does introduce a considerable computational 172 00:22:02.260 --> 00:22:03.200 David Stephens, Prof: overhead. 173 00:22:06.570 --> 00:22:11.230 David Stephens, Prof: So to dig a little bit more into the variance estimation through sandwich approaches. 174 00:22:11.580 --> 00:22:20.100 David Stephens, Prof: So the kind of semi parametric models that I'm thinking of and semiparametric semi-parametric estimation 175 00:22:24.250 --> 00:22:35.529 David Stephens, Prof: can be achieved through estimating equations approaches. So these approaches make no parametric assumptions about the distributions of the data, but instead make assumptions about the the mean 176 00:22:35.840 --> 00:22:37.369 David Stephens, Prof: and the variance structure. 177 00:22:38.070 --> 00:22:52.919 David Stephens, Prof: In any time we can write something as an estimating equation with certain properties. The resulting estimators have typically good asymptotic properties, and you can estimate the asymptotic variance using 178 00:22:53.300 --> 00:22:55.190 David Stephens, Prof: the sandwich based approach. 179 00:22:55.710 --> 00:22:59.590 David Stephens, Prof: So here's a bit of a summary of that, and we'll see this in some of the 180 00:22:59.810 --> 00:23:03.520 David Stephens, Prof: in the simulation approaches. Simulation studies. 181 00:23:04.143 --> 00:23:16.800 David Stephens, Prof: If we have an estimating function, bold u for parameter theta. What we end up having to do is to solve this system. That's the sum of the estimating function evaluated at the 182 00:23:16.960 --> 00:23:19.940 David Stephens, Prof: data for parameter theta equal to 0. 183 00:23:21.300 --> 00:23:37.119 David Stephens, Prof: Okay? So ordinarily, squares works like this. And then the resulting estimator has good properties. Typically it's asymptotically, normally distributed, centered at the truth, but with a variance asymptotic variance that can be derived in terms of U. 184 00:23:37.240 --> 00:23:38.530 David Stephens, Prof: In particular. 185 00:23:38.750 --> 00:23:52.890 David Stephens, Prof: the asymptotic variance of the estimator has this form, so a inverse BA inverse transpose which I'll write like that where a is minus the expected value of U dot, the first, st 186 00:23:53.570 --> 00:24:01.399 David Stephens, Prof: the matrix of partial derivatives which I'll mention in a second, and B is essentially the fisher information like quantity. 187 00:24:01.930 --> 00:24:05.290 David Stephens, Prof: Right? So this is the sandwich estimation form 188 00:24:05.440 --> 00:24:08.630 David Stephens, Prof: in terms of properties of the estimating function. 189 00:24:09.860 --> 00:24:17.960 David Stephens, Prof: So all I have to do when implementing any of these causal estimators based on the propensity score is to write down the estimating equation 190 00:24:18.660 --> 00:24:30.790 David Stephens, Prof: compute, using this formula and then estimate the 2 matrices from the data in this formula. 191 00:24:32.460 --> 00:24:35.969 David Stephens, Prof: No, that should be a theta 0 there. Sorry it's a 0 192 00:24:36.380 --> 00:24:44.679 David Stephens, Prof: in this formula. Theta 0 is the true data generating parameter u.is the matrix of 1st partial derivatives of U with respect to theta 193 00:24:46.840 --> 00:24:52.600 David Stephens, Prof: This matrix of 1st partial derivatives is not a symmetric matrix necessarily. 194 00:24:53.830 --> 00:25:01.819 David Stephens, Prof: and in practice A and B are estimated using the sample averages where Theta North is replaced by Theta hat. 195 00:25:02.470 --> 00:25:09.360 David Stephens, Prof: and I'll show some illustrative calculations of that nature in the, in the practical examples that are coming up. 196 00:25:11.900 --> 00:25:14.049 David Stephens, Prof: So there's a question in the chat there. 197 00:25:14.320 --> 00:25:18.930 David Stephens, Prof: Do bootstop approaches, give similar variances to the variant solutions 198 00:25:19.070 --> 00:25:25.390 David Stephens, Prof: close to invariant solutions when those solutions exist often. Yes. 199 00:25:26.640 --> 00:25:30.921 David Stephens, Prof: but as we'll see in some of the examples, the 200 00:25:34.690 --> 00:25:45.090 David Stephens, Prof: the closed form sandwich type estimators can be Conf can be compromised in the 201 00:25:45.470 --> 00:25:52.530 David Stephens, Prof: propensity score setting. If there are some specific configurations for the for certain parts of the model. 202 00:25:52.960 --> 00:25:58.379 David Stephens, Prof: So in the examples I've got coming up, I'll show the a Monte Carlo study. 203 00:25:58.760 --> 00:26:02.190 David Stephens, Prof: and I'll show the sandwich based estimation 204 00:26:02.530 --> 00:26:10.210 David Stephens, Prof: and show that the sandwich-based estimation doesn't always correspond to the empirical variance of the estimator. 205 00:26:11.160 --> 00:26:18.530 David Stephens, Prof: Okay, now the idea is that in these regression methods Bootstrap approaches will always give. Give a good approximation. 206 00:26:19.230 --> 00:26:24.560 David Stephens, Prof: Right? So to go to the second part of your question. 207 00:26:25.790 --> 00:26:29.189 David Stephens, Prof: I think you're right. That there's 208 00:26:32.140 --> 00:26:36.269 David Stephens, Prof: going with the Bootstrap solution is always pretty safe 209 00:26:36.630 --> 00:26:39.980 David Stephens, Prof: for the regression, and inverse probability weighting approaches 210 00:26:40.960 --> 00:26:48.870 David Stephens, Prof: where you have to be careful, though, is for some of the other approaches like matching and stratification. 211 00:26:49.950 --> 00:26:50.700 David Stephens, Prof: Right? 212 00:26:50.810 --> 00:27:02.460 David Stephens, Prof: So for for more standard sorts of statistical adjustment, regression based adjustment or estimating equation based estimation relying on the bootstrap is generally 213 00:27:04.260 --> 00:27:05.160 David Stephens, Prof: okay, 214 00:27:07.370 --> 00:27:10.900 David Stephens, Prof: So computational consideration is really the main factor. 215 00:27:11.970 --> 00:27:18.380 David Stephens, Prof: I guess the other thing I would say is that the conditions under which the Bootstrap is guaranteed to work 216 00:27:19.000 --> 00:27:25.100 David Stephens, Prof: also can exclude the use of advanced modeling for the propensity score 217 00:27:25.470 --> 00:27:32.229 David Stephens, Prof: all right. So the the conditions under which the Bootstrap distribution matches the sampling distribution. 218 00:27:33.510 --> 00:27:39.119 David Stephens, Prof: Sometimes. That the Bootstrap approach 219 00:27:39.400 --> 00:27:41.179 David Stephens, Prof: can be shown not to work. 220 00:27:41.390 --> 00:27:45.940 David Stephens, Prof: and typical cases where you have high dimensional predictor vectors 221 00:27:46.380 --> 00:27:56.260 David Stephens, Prof: and you want to do inference for the propensity score with the number of confounders is high. So in those cases you need to do a bit something a bit more sophisticated than the Bootstrap. Actually. 222 00:27:56.610 --> 00:27:59.709 David Stephens, Prof: So, there are methods which are called sample splitting. 223 00:28:00.620 --> 00:28:02.060 David Stephens, Prof: That's a good question, though. 224 00:28:02.170 --> 00:28:11.819 David Stephens, Prof: Yeah. And I I think for for certain settings the param. Sorry the the robust standard error approaches that I've outlined here can still work 225 00:28:11.990 --> 00:28:16.450 David Stephens, Prof: pretty well, all right. So 226 00:28:17.031 --> 00:28:28.640 David Stephens, Prof: yeah. So here's the case for linear regression just working through the the constructions of A and B, and then we can estimate. This is this is the the sandwich estimator, for for 227 00:28:29.020 --> 00:28:33.290 David Stephens, Prof: of the variance asymptotic variance of a ordinary least squares estimator. 228 00:28:35.050 --> 00:28:56.919 David Stephens, Prof: So what are some key considerations before we start looking at the example. So the key assumptions we've made, or the key things that we've looked at are. I've listed them in the following way. So 1st of all, the importance of no unmeasured confounding. So nothing with a propensity score will work if you have unmeasured confounding, but then not many statistical methods would work. If you have unmeasured confounding 229 00:28:57.610 --> 00:29:03.939 David Stephens, Prof: right? So you need to make that assumption. It's pretty critical, at least in the causal world. We're upfront about making that assumption 230 00:29:06.760 --> 00:29:10.570 David Stephens, Prof: for the outcome regression model. We can afford to be flexible there. 231 00:29:11.070 --> 00:29:15.670 David Stephens, Prof: And if we have a correct specification, we don't need causal adjustments. 232 00:29:17.030 --> 00:29:32.350 David Stephens, Prof: in this outcome model. We can think about using conventional, variable selection methods, because that's going to predict the right sort of terms. It's going to predict predictors of it's going to include predictors of outcome, and we'll select, therefore, the confounders 233 00:29:33.400 --> 00:29:39.699 David Stephens, Prof: in finite sample, though if you have a weak, confounder, weak predictor of the outcome. It may omit that 234 00:29:40.170 --> 00:29:41.309 David Stephens, Prof: if we use 235 00:29:41.540 --> 00:29:54.050 David Stephens, Prof: model selection, so we need to be a little bit aware of that. But the bias would be minimal if the confounder is a weak confounder. So for outcome regression using model selection methods is perfectly fine. 236 00:29:54.700 --> 00:29:58.039 David Stephens, Prof: However, for the propensity score, we shouldn't do that. 237 00:29:58.580 --> 00:30:00.409 David Stephens, Prof: That's my overall message. 238 00:30:00.570 --> 00:30:03.459 David Stephens, Prof: So there are a number of approaches 239 00:30:03.580 --> 00:30:06.550 David Stephens, Prof: that we could use. We could either throw in all 240 00:30:07.030 --> 00:30:16.710 David Stephens, Prof: confounders that are available and all interactions between them into the propensity score model. Well, that's dangerous, because we 241 00:30:17.080 --> 00:30:22.430 David Stephens, Prof: might induce positivity. Violations 242 00:30:22.810 --> 00:30:30.760 David Stephens, Prof: be overconfident about our prediction of Z, when, in fact, what we need to match is on purely the sorry, the model, basically only on the confounders. 243 00:30:31.970 --> 00:30:40.019 David Stephens, Prof: We might also use up data in estimating the propensity score 244 00:30:40.310 --> 00:30:50.679 David Stephens, Prof: and therefore inflate the variance of our estimator, so the danger of of constructing a too complicated propensity score is that it may inflate our variance 245 00:30:51.600 --> 00:30:54.730 David Stephens, Prof: for the estimation of the parameter of interest. 246 00:30:55.260 --> 00:31:06.939 David Stephens, Prof: We should really focus on the confounders. We definitely need to ignore instruments. Those variables that are predictors of treatment only should not be included in the propensity score 247 00:31:11.190 --> 00:31:15.950 David Stephens, Prof: when we construct the propensity score. Almost the 1st thing we should look at is 248 00:31:16.240 --> 00:31:24.339 David Stephens, Prof: that we've constructed a balancing score, that is that the distributions of the axes are similar 249 00:31:25.040 --> 00:31:29.529 David Stephens, Prof: in different strata of the propensity score. If we don't have balance, then 250 00:31:29.660 --> 00:31:36.250 David Stephens, Prof: there's we. We have literally no guarantee. And it means, really, we've constructed the propensity scoring correctly. 251 00:31:37.630 --> 00:31:46.563 David Stephens, Prof: We've also got to be aware of positivity and overlap can overlap issues. So we need certainly to predict, be able to 252 00:31:48.292 --> 00:31:54.740 David Stephens, Prof: have a model that doesn't assign probability one or 0 to either of the binary treatments 253 00:31:55.230 --> 00:32:05.469 David Stephens, Prof: conditional on X, and we also need to make sure that the strata of propensity score that we're gonna that we could look at contain enough treated and untreated individuals. 254 00:32:05.620 --> 00:32:18.679 David Stephens, Prof: So we need to make sure that the propensity score distribution for the treated individuals overlaps with the propensity, score distribution for the untreated individuals. We'll see some pictures of that in the, in the plots that are upcoming. 255 00:32:19.800 --> 00:32:28.290 David Stephens, Prof: we need to do effective model selection. And really, the what this is saying is that running ordinary 256 00:32:29.930 --> 00:32:43.690 David Stephens, Prof: model selection methods just on the propensity score. Construction really is the wrong thing to do, because what it's going to do is to predict strong predictors of treatment, irrespective of whether they're irrespective of whether they're confounders. 257 00:32:46.160 --> 00:32:51.179 David Stephens, Prof: So we can think of the the variables in the system that look like this. 258 00:32:51.300 --> 00:32:56.779 David Stephens, Prof: The X's are the confounders. Xi. Are the instruments they predict. Z. Only 259 00:32:57.040 --> 00:33:01.239 David Stephens, Prof: XO are the pure predictors of outcome. They predict y only. 260 00:33:02.920 --> 00:33:09.269 David Stephens, Prof: So we need to omit xi. In the construction of the propensity score, and only use X 261 00:33:10.990 --> 00:33:16.080 David Stephens, Prof: ideally. We should use Xo in the outcome model along with X 262 00:33:16.220 --> 00:33:19.730 David Stephens, Prof: and Z. That'll give the best, the best results. 263 00:33:22.390 --> 00:33:26.069 David Stephens, Prof: So just to reiterate conventional model selection should not be used 264 00:33:26.690 --> 00:33:33.920 David Stephens, Prof: right? So whether this is based on information, criteria or spar selection methods or step isolation whatever. 265 00:33:35.890 --> 00:33:44.560 David Stephens, Prof: because these methods typically prioritize the prediction of treatment, give an axe, and 266 00:33:46.010 --> 00:33:57.890 David Stephens, Prof: that's fine. If the X contains only the confounders. But if we have spurious terms in there. We'll have positivity violations potentially, and also that will inflate the variance of the 267 00:33:59.540 --> 00:34:01.470 David Stephens, Prof: quantity of interest. Estimator. 268 00:34:02.660 --> 00:34:08.030 David Stephens, Prof: Okay? Because really accurate prediction of the treatment is not the goal. 269 00:34:08.420 --> 00:34:15.189 David Stephens, Prof: We're only using the propensity score to induce balance or conditional independence between X and Z. 270 00:34:15.989 --> 00:34:19.289 David Stephens, Prof: Right predicting Z from X is not really the goal. 271 00:34:24.100 --> 00:34:31.750 David Stephens, Prof: So the other method I mentioned of an apparent conservative approach is to build very rich versions of these models 272 00:34:32.469 --> 00:34:40.383 David Stephens, Prof: and that can work fine, but of course it pro it 273 00:34:41.139 --> 00:34:45.769 David Stephens, Prof: prioritizes bias elimination over variance inflation. 274 00:34:46.639 --> 00:34:50.049 David Stephens, Prof: So you might think that bias elimination is a good thing. 275 00:34:50.620 --> 00:34:52.880 David Stephens, Prof: Well, and in the sense it is a good thing. 276 00:34:53.960 --> 00:35:00.040 David Stephens, Prof: provided you don't want to report. Say a confidence interval for your estimator or your estimate 277 00:35:00.700 --> 00:35:02.599 David Stephens, Prof: from your estimate right. 278 00:35:02.770 --> 00:35:06.450 David Stephens, Prof: So the more data you use, the richer models you use here 279 00:35:06.610 --> 00:35:15.330 David Stephens, Prof: you are going to be able to reduce the bias. But the corresponding variance uncertain, the uncertainty associated with the corresponding inference 280 00:35:16.030 --> 00:35:18.449 David Stephens, Prof: is going to be compromised because of that. 281 00:35:19.130 --> 00:35:26.349 David Stephens, Prof: So really rich models here containing large number of complicated terms. 282 00:35:27.384 --> 00:35:30.800 David Stephens, Prof: Is probably not the the way to go. 283 00:35:34.600 --> 00:35:54.180 David Stephens, Prof: The final thing I guess we need to think about is the causal measure required. Is it a risk difference, a risk ratio. Is it an odds ratio average treatment effect on the treated average treatment effect or the average potential outcome for different treatment levels? Right? And for most of the methods. We can get all of those things from 284 00:35:54.580 --> 00:36:02.839 David Stephens, Prof: a single analysis. But we, when reporting our results, we need to make sure we have the right quantity. 285 00:36:03.730 --> 00:36:10.969 David Stephens, Prof: All right. So with that, with those caveats in mind, we're gonna spring on to the 286 00:36:12.650 --> 00:36:20.849 David Stephens, Prof: numerical results. So the 1st one I'm going to look at is the Nhanes example. So this is in a couple of forms on the website. 287 00:36:22.010 --> 00:36:24.539 David Stephens, Prof: It's an analysis of a real data set. 288 00:36:28.370 --> 00:36:29.780 David Stephens, Prof: The 289 00:36:38.900 --> 00:37:05.599 David Stephens, Prof: the file. So there are 2 forms. One is the spreadsheet, one is the slides version, and one is the the worksheet version. This is the slides version. So Nhanes is a publicly available data set. I got it from our Us. National health and Nutrition examination survey, long running multi-wave survey. And from my analysis that I'm going to show here. I collect. I extracted these variables. 290 00:37:05.880 --> 00:37:08.710 David Stephens, Prof: and I looked at a subset of individuals 291 00:37:09.120 --> 00:37:17.200 David Stephens, Prof: adults in the second wave of the survey, and my task is going to be to to see how current smoking status 292 00:37:17.570 --> 00:37:20.539 David Stephens, Prof: influences systolic blood pressure. 293 00:37:20.870 --> 00:37:29.790 David Stephens, Prof: Right? So I've got these treatment and outcome variables. And then I've got a number of potential confounders. 294 00:37:33.240 --> 00:37:36.879 David Stephens, Prof: All right. So just load up so you can 295 00:37:37.070 --> 00:37:43.199 David Stephens, Prof: extract this code or copy this code and implement it yourself. I'm going to go through the analysis. 296 00:37:44.535 --> 00:37:45.330 David Stephens, Prof: Here. 297 00:37:45.470 --> 00:37:50.950 David Stephens, Prof: So the 1st bit loads up the R libraries, and we've got a number of specialist libraries there. 298 00:37:52.140 --> 00:37:59.340 David Stephens, Prof: The 1st thing I'm going to do is to do processing of the data set. So the Nhanes data is contained in this library 299 00:37:59.470 --> 00:38:07.809 David Stephens, Prof: and or this package. And I'm going to basically extract the data that I need select the variables and 300 00:38:07.980 --> 00:38:14.100 David Stephens, Prof: use the resulting data set right? So that bit of code does that. And then. 301 00:38:14.567 --> 00:38:19.429 David Stephens, Prof: If I'm going to look for the effect of smoking on systolic blood pressure. 302 00:38:21.790 --> 00:38:25.779 David Stephens, Prof: my 1st go-to analysis would be based on a regression. Perhaps 303 00:38:26.030 --> 00:38:29.790 David Stephens, Prof: I could do a 2 group comparison. But the regression based on Lm. 304 00:38:29.970 --> 00:38:32.130 David Stephens, Prof: Effectively implements that for me. 305 00:38:32.790 --> 00:38:34.710 David Stephens, Prof: So if I just regress 306 00:38:35.300 --> 00:38:41.540 David Stephens, Prof: systolic blood pressure on smoking, this is the result I get. I get an estimate of the 307 00:38:41.720 --> 00:38:44.749 David Stephens, Prof: quantity which is minus 3.6 7 9, 308 00:38:45.580 --> 00:38:54.439 David Stephens, Prof: with a standard error of point 9 6. Right. So my naive, unadjusted analysis gives me this negative coefficient. 309 00:38:54.880 --> 00:38:59.789 David Stephens, Prof: It's significantly different from 0, as the T value and the p-value show. 310 00:39:00.520 --> 00:39:04.370 David Stephens, Prof: So it looks as if there's a significant negative effect 311 00:39:05.350 --> 00:39:11.550 David Stephens, Prof: of current smoking status. So smoky. Status, yes, is one. 312 00:39:12.450 --> 00:39:16.459 David Stephens, Prof: And so this indicates a lowering of systolic blood pressure. 313 00:39:18.250 --> 00:39:19.830 David Stephens, Prof: If you currently smoke. 314 00:39:20.650 --> 00:39:32.590 David Stephens, Prof: Now, that's an unadjusted analysis. The next thing I could do would be to try to fit a regression model. So I'm going to do that by including the smoking status variable. But then all the other variables as main effects. 315 00:39:35.770 --> 00:39:48.860 David Stephens, Prof: Now in this linear model, if I do that, the coefficient of the smoking variable is the estimate of the average treatment effect. So here I get that it's now shrunk to minus 1.0 9 7 7. 316 00:39:50.370 --> 00:39:57.230 David Stephens, Prof: Right? So what that tells me is the adjustment for the potential confounders here is really changing the 317 00:39:57.580 --> 00:40:00.849 David Stephens, Prof: the value of the estimate. It's now not significant. 318 00:40:02.670 --> 00:40:09.750 David Stephens, Prof: Standard error remains about the same. So if I was happy with the regression analysis here. 319 00:40:10.910 --> 00:40:14.549 David Stephens, Prof: these would be my results. I'm clearly getting more 320 00:40:18.260 --> 00:40:25.850 David Stephens, Prof: sorry clear. There is some effect of these potential confounders in the analysis. And it's changing the estimate value. 321 00:40:26.510 --> 00:40:31.379 David Stephens, Prof: So my causal analysis is going to try to check these results. 322 00:40:32.050 --> 00:40:39.340 David Stephens, Prof: I'm going to use various propensity score approaches to see what the comparative results are. 323 00:40:41.350 --> 00:40:47.630 David Stephens, Prof: How am I going to do that? Well, I'm going to think about fitting propensity score models, and I'm going to think about quantifying 324 00:40:47.790 --> 00:40:48.850 David Stephens, Prof: balance. 325 00:40:49.350 --> 00:40:56.140 David Stephens, Prof: And my balance measure is going to be either the standardized mean difference in the covariates. 326 00:40:58.210 --> 00:41:05.310 David Stephens, Prof: unweighted or weighted forms. So that's the difference in the 2 means weighted by the average of the variance of the 327 00:41:05.490 --> 00:41:08.660 David Stephens, Prof: in the 2 groups, the treated and untreated groups. 328 00:41:08.990 --> 00:41:11.629 David Stephens, Prof: Right? So this measure 329 00:41:12.130 --> 00:41:22.690 David Stephens, Prof: is going to be a measure of balance. It's a imperfect measure of balance, but it gives a basic idea. It's the measuring, the the difference in the mean level on some standardized scale 330 00:41:23.010 --> 00:41:25.060 David Stephens, Prof: between the treated and untreated groups. 331 00:41:26.360 --> 00:41:34.769 David Stephens, Prof: The empirical Cdf method is going to be the other method. So I'm going to look at the empirical Cdf in treated and untreated groups for the variables or the weighted versions of the variables. 332 00:41:39.060 --> 00:41:49.970 David Stephens, Prof: Now using the create table one function which is in one of the which is in the survey package. I believe I can look at 333 00:41:50.810 --> 00:41:58.990 David Stephens, Prof: a typical table, one as in a paper to do a comparison of treated and untreated groups. So if I look at age, it's clear that 334 00:41:59.210 --> 00:42:12.050 David Stephens, Prof: why, it's not clear. But there's an implication that age is different age profile is different in the 2 smoking groups, so current smokers are younger than non-current smokers 335 00:42:13.000 --> 00:42:18.889 David Stephens, Prof: standardized mean difference in the original sample. In the 2 groups is 0 point 5 9 2. That's 336 00:42:19.090 --> 00:42:26.740 David Stephens, Prof: large. We think about a standardized mean difference of being of less than one as being indicating a a. 337 00:42:26.850 --> 00:42:28.770 David Stephens, Prof: A balance that's reasonable. 338 00:42:28.980 --> 00:42:38.449 David Stephens, Prof: Okay, here. This is way away from that to the if there's no adjustment, then it's clear that age is not balanced in the 2 treatment groups. 339 00:42:39.350 --> 00:42:41.260 David Stephens, Prof: So if it were a confounder. 340 00:42:41.940 --> 00:42:44.429 David Stephens, Prof: if it were something that affects the outcome. 341 00:42:44.780 --> 00:42:51.059 David Stephens, Prof: which in this case it probably does, then this would be cause for concern in the original analysis. 342 00:42:52.970 --> 00:42:53.640 David Stephens, Prof: Okay. 343 00:42:54.210 --> 00:43:01.719 David Stephens, Prof: so do that for privileges. I don't show the output here. I think I included it in the Pdf version, the other Pdf version. Okay. 344 00:43:04.590 --> 00:43:13.510 David Stephens, Prof: another way to look at that is to look at the empirical Cdfs rage in the 2 smoking groups. So blue. Here is the smoking 345 00:43:15.790 --> 00:43:23.580 David Stephens, Prof: non-current smokers, and red is the current smokers. So again, we see this difference in the profile in the 2 groups. 346 00:43:23.800 --> 00:43:26.040 David Stephens, Prof: So at the moment there's no balance. 347 00:43:29.990 --> 00:43:36.640 David Stephens, Prof: So we we know we need some sort of causal adjustment. And the way I'm going to start thinking about that is to 1st build a propensity score. 348 00:43:36.920 --> 00:43:41.040 David Stephens, Prof: So in the propensity score, I'm going to predict the 349 00:43:42.590 --> 00:43:47.450 David Stephens, Prof: treatment variable on the basis of well, in this case all of the other 350 00:43:49.140 --> 00:43:58.149 David Stephens, Prof: potential confound variables. So here I'm not thinking about doing model selection. I could do that. It's plausible that all of these things are confounders. So I'm going to look at 351 00:43:58.440 --> 00:44:00.620 David Stephens, Prof: all of them as predictors of Z. 352 00:44:01.050 --> 00:44:08.550 David Stephens, Prof: The treatment. So I do the glm. And then from that I obtain the fitted versions of the propensity scores 353 00:44:08.850 --> 00:44:10.499 David Stephens, Prof: using the predict function. 354 00:44:12.780 --> 00:44:17.129 David Stephens, Prof: So okay, let's now, I fitted the propensity school. 355 00:44:17.660 --> 00:44:29.789 David Stephens, Prof: So the next thing I need to do is to check whether I can use that propensity score to any good purpose in the estimation procedure. So the 1st thing I look at is the breakdown of the 356 00:44:30.520 --> 00:44:34.489 David Stephens, Prof: sample into propensity score Quintiles. 357 00:44:35.550 --> 00:44:36.610 David Stephens, Prof: So 358 00:44:37.030 --> 00:44:49.730 David Stephens, Prof: 1st quintile, so propensity score is always between 0 and one. In this case I'm going to break it down into the empirical quintile. So the 1st quintile is here, second, quintile, 3, rd quintile, 4, th and 5.th 359 00:44:50.020 --> 00:44:57.500 David Stephens, Prof: And what this table tells us is that there is good representation of both the treated and untreated in this case 360 00:44:57.650 --> 00:45:04.519 David Stephens, Prof: exposed and unexposed, so current smoker or non-current smoker in each of the quintiles. 361 00:45:05.900 --> 00:45:12.770 David Stephens, Prof: All right, so that tells me that there is comparability. There is overlap between the propensity score distributions in the 2 362 00:45:13.840 --> 00:45:14.720 David Stephens, Prof: groups. 363 00:45:16.050 --> 00:45:24.880 David Stephens, Prof: I can look at that in a histogram. So here this is the code for producing the following histogram. So I see the blue bars 364 00:45:25.230 --> 00:45:26.390 David Stephens, Prof: colored in 365 00:45:26.990 --> 00:45:35.330 David Stephens, Prof: this light blue color, indicating the no smoking group and the red bars indicating the smoking group. So these are the propensity scores 366 00:45:36.080 --> 00:45:39.700 David Stephens, Prof: in this sample, total sample size of 1, 3, 7, 7. 367 00:45:41.130 --> 00:45:48.580 David Stephens, Prof: The the pink bars are the or or red bars are the the. 368 00:45:49.220 --> 00:45:54.929 David Stephens, Prof: It'll represent the propensity score histogram for the smoking group, and the sort of dark purple 369 00:45:55.650 --> 00:45:58.850 David Stephens, Prof: that we see here is the overlap of pink and blue. 370 00:45:59.060 --> 00:46:04.310 David Stephens, Prof: So the point is here we see an overlap between the distribution of the blue and the distribution of the red. 371 00:46:05.790 --> 00:46:12.759 David Stephens, Prof: So what that tells me is that actually, I'm going to get. If I were to use this propensity score model, I'm going to get at least 372 00:46:14.210 --> 00:46:18.759 David Stephens, Prof: the potential for good estimation, because I've got overlap between the 2 groups. 373 00:46:21.170 --> 00:46:28.310 David Stephens, Prof: Okay, that's not the end of the story, though, because I'm still gonna have to assess balance. 374 00:46:30.050 --> 00:46:37.510 David Stephens, Prof: Next thing I can look at is a box plot of the 2 propensity score 375 00:46:37.640 --> 00:46:51.169 David Stephens, Prof: subset distributions. So here we see the box plot for the smoking equals 0 group. Here. The box plot for the smoking equals one group, and they're different, and the the propensity to smoke is greater amongst the smokers. That's not surprising. 376 00:46:51.820 --> 00:46:55.379 David Stephens, Prof: but there is still sufficient overlap between the 2 groups that the 377 00:46:55.540 --> 00:47:01.200 David Stephens, Prof: the red lines here are the quartiles, I think of the the overall distribution. 378 00:47:03.250 --> 00:47:08.499 David Stephens, Prof: No, yeah, the sorry, not the quartiles, the the boundaries of the quintiles. 379 00:47:11.670 --> 00:47:13.439 David Stephens, Prof: So there seems to be good overlap. 380 00:47:14.680 --> 00:47:20.839 David Stephens, Prof: We infer from this and from this that this is potentially a good propensity score 381 00:47:21.250 --> 00:47:23.549 David Stephens, Prof: construction to use for our adjustment. 382 00:47:26.640 --> 00:47:27.650 David Stephens, Prof: So 383 00:47:28.170 --> 00:47:36.850 David Stephens, Prof: the next question we ask is, well, okay, does the propensity score induce balance? So within the quintile groups, the 5 384 00:47:37.160 --> 00:47:42.019 David Stephens, Prof: subsets, the 5 strata. Do I get balance amongst the 385 00:47:42.370 --> 00:47:48.380 David Stephens, Prof: covariance? So here are the 5 covariates, 5 sorry, 6, 6 covariants. 386 00:47:49.226 --> 00:47:54.760 David Stephens, Prof: When I look at overall the data. 387 00:47:55.200 --> 00:48:00.969 David Stephens, Prof: I see there's that point 5 9 that I mentioned before. This is an imbalance of age. 388 00:48:01.170 --> 00:48:08.150 David Stephens, Prof: but there are so, although the gender difference is not too great, this is standardized. Mean difference, calculation 389 00:48:08.260 --> 00:48:10.070 David Stephens, Prof: for all the other variables. 390 00:48:10.290 --> 00:48:13.340 David Stephens, Prof: If I don't break it down by princely score 391 00:48:13.840 --> 00:48:19.049 David Stephens, Prof: quintile. Then I see significantly high standardized mean difference. 392 00:48:19.950 --> 00:48:22.160 David Stephens, Prof: When I look within the quintiles 393 00:48:22.370 --> 00:48:24.349 David Stephens, Prof: the picture is not much better. 394 00:48:26.940 --> 00:48:33.159 David Stephens, Prof: Okay, generally the the imbalance measure is reduced. 395 00:48:33.920 --> 00:48:39.099 David Stephens, Prof: but in some cases there are still relatively high values of the standardized mean difference. 396 00:48:39.380 --> 00:48:43.969 David Stephens, Prof: So what that's saying is that for stratification purposes 397 00:48:44.320 --> 00:48:47.219 David Stephens, Prof: this propensity score is not quite doing the job. 398 00:48:49.550 --> 00:48:52.600 David Stephens, Prof: and the standardized mean differences are still too large. 399 00:48:54.190 --> 00:48:56.269 David Stephens, Prof: Balance does not seem to be achieved. 400 00:48:57.750 --> 00:48:59.160 David Stephens, Prof: But I look at the 401 00:49:01.970 --> 00:49:05.310 David Stephens, Prof: If I break down the data by the 402 00:49:05.740 --> 00:49:16.959 David Stephens, Prof: same quintile strata. These are the empirical Cdfs for age, and I should see these lying on top of each other, and they really don't. So that's saying that 403 00:49:17.430 --> 00:49:21.249 David Stephens, Prof: the propensity score Quintiles are not balanced on 404 00:49:21.860 --> 00:49:25.139 David Stephens, Prof: do not yield balance for the age variable. 405 00:49:26.050 --> 00:49:31.169 David Stephens, Prof: I could take a finer gradation, and that might improve things. But then quickly, I run out of data. 406 00:49:32.510 --> 00:49:52.699 David Stephens, Prof: Okay, so the function I'm using here is match balance. Well, sorry. A function I'm going to use now is the match balance function which is in the matching library, and you can use that to give a lot more of the comparisons between the groups. Once you've broken it down by whatever propensity score strata you want to use. So there is a call for that in the other in the 407 00:49:52.840 --> 00:49:54.319 David Stephens, Prof: and that version, I think 408 00:49:57.440 --> 00:49:58.420 David Stephens, Prof: so. 409 00:49:58.550 --> 00:50:05.109 David Stephens, Prof: Talking about Matching, I can then try a second approach to causal adjustment which is matching on the propensity score. 410 00:50:06.950 --> 00:50:11.239 David Stephens, Prof: So to do that I use the match function in the matching library. 411 00:50:12.550 --> 00:50:21.259 David Stephens, Prof: And so I run that I have to supply the treatment variable. The Confounder variables. 412 00:50:21.870 --> 00:50:25.469 David Stephens, Prof: I'm going to tell it that the S demand is the average treatment effect. 413 00:50:27.390 --> 00:50:32.160 David Stephens, Prof: Okay? And then I'm going to do an analysis of that. 414 00:50:36.082 --> 00:50:41.289 David Stephens, Prof: proposed matching strategy using the match balance function. 415 00:50:42.040 --> 00:50:46.669 David Stephens, Prof: Okay? So it's based this phone. I don't show the results here. But it's in a bigger handout. 416 00:50:47.130 --> 00:50:54.189 David Stephens, Prof: This is going to check for me things like standardized mean difference amongst matched individuals. 417 00:50:54.750 --> 00:50:59.370 David Stephens, Prof: So the idea is that for each treated individual we're gonna find 418 00:50:59.910 --> 00:51:03.610 David Stephens, Prof: one potentially more matched, untreated individuals. 419 00:51:03.940 --> 00:51:08.810 David Stephens, Prof: And we're going to see when we collect when we construct that matched sample. 420 00:51:09.320 --> 00:51:13.579 David Stephens, Prof: how comparable individuals in within the matching are. 421 00:51:17.050 --> 00:51:22.019 David Stephens, Prof: And so I've summarized it here in terms of the age 422 00:51:23.610 --> 00:51:33.150 David Stephens, Prof: covariate. I could do it for the other covariates as well. But this illustrates the point that when I carried out this matching. So I've matched on the propensity score. 423 00:51:33.450 --> 00:51:38.610 David Stephens, Prof: Each treated individual with one untreated individual. 424 00:51:40.150 --> 00:51:45.519 David Stephens, Prof: I get actually a very good agreement in age. 425 00:51:47.250 --> 00:51:47.980 David Stephens, Prof: Okay? 426 00:51:48.260 --> 00:52:00.449 David Stephens, Prof: So matching via the propensity score is looking as if it's a potential method for doing the analysis, whereas stratification on the propensity score didn't quite do the job. 427 00:52:01.960 --> 00:52:04.799 David Stephens, Prof: And if I look at the standardized mean difference 428 00:52:06.270 --> 00:52:09.700 David Stephens, Prof: quantities for the variables based on the matched sample. 429 00:52:10.600 --> 00:52:13.060 David Stephens, Prof: I see that they've all been reduced. 430 00:52:15.720 --> 00:52:24.550 David Stephens, Prof: So do we need a specific sample size here. Now, this will work for arbitrary sample sizes. Of course, as you reduce the sample size. 431 00:52:24.700 --> 00:52:43.759 David Stephens, Prof: the reliability of any of the empirical estimation will be, will be lower. Right? So you need. So matching will work in principle with any sample size but the the resulting summary statistics, and then subsequent ate estimates will be more variable. It's inevitable. 432 00:52:47.510 --> 00:52:52.568 David Stephens, Prof: Okay? So matching looks as if it works and then 433 00:52:54.100 --> 00:52:56.600 David Stephens, Prof: the next method that we could think about 434 00:52:56.820 --> 00:52:59.790 David Stephens, Prof: for assessing matching is through inverse waiting. 435 00:52:59.980 --> 00:53:11.130 David Stephens, Prof: So what we're going to do here is we're going to weight individuals by their inverse probability given by the propensity score. So that's what I form here. Pslr weight 436 00:53:12.110 --> 00:53:16.950 David Stephens, Prof: to that treat treatment variable over PS. 437 00:53:18.020 --> 00:53:24.529 David Stephens, Prof: Plus one minus treatment variable over one minus Ps, so that assigns weight one over PS. 438 00:53:24.920 --> 00:53:28.430 David Stephens, Prof: To the treated individuals, and one minus 439 00:53:28.670 --> 00:53:32.470 David Stephens, Prof: sorry one over one minus Ps. To the untreated individuals. 440 00:53:33.610 --> 00:53:40.780 David Stephens, Prof: Okay? And then this is just a convenient way to do that. And then I'm gonna so I'm gonna 441 00:53:41.180 --> 00:53:49.839 David Stephens, Prof: use a software to imagine reweighting the samples. And then I'm going to compute the standardized mean difference for the reweighted samples. 442 00:53:51.560 --> 00:53:55.459 David Stephens, Prof: Okay? So when I do that and look at 443 00:53:56.070 --> 00:54:01.360 David Stephens, Prof: version of the empirical. Cdf, so this is basically a weighted version of the empirical Cdf, 444 00:54:03.420 --> 00:54:10.519 David Stephens, Prof: I see that there's reasonable match reasonable balance across the 2 groups. 445 00:54:11.870 --> 00:54:16.400 David Stephens, Prof: And if I look at the standardized mean difference for the inverse probability weighted 446 00:54:16.610 --> 00:54:21.279 David Stephens, Prof: new data set. I see that the standardized mean differences have really been reduced a lot. 447 00:54:22.240 --> 00:54:26.060 David Stephens, Prof: So what that tells me is that this propensity score 448 00:54:26.680 --> 00:54:34.370 David Stephens, Prof: provides good, the potential for good causal adjustment using matching and waiting approaches. 449 00:54:36.270 --> 00:54:37.120 David Stephens, Prof: Okay? 450 00:54:37.540 --> 00:54:45.499 David Stephens, Prof: Because I get when I do Matching, or I do inverse probability, waiting. I end up with a sample 451 00:54:45.950 --> 00:54:48.380 David Stephens, Prof: in the treated group which is comparable 452 00:54:48.670 --> 00:54:53.300 David Stephens, Prof: with the sample in the untreated group which is my goal all along. 453 00:54:59.810 --> 00:55:03.790 David Stephens, Prof: So we need to create balance. 454 00:55:05.100 --> 00:55:08.350 David Stephens, Prof: We're going to use the propensity score to do that, to assess that. 455 00:55:09.493 --> 00:55:15.849 David Stephens, Prof: It doesn't necessarily mean that the model that we fit for the propensity score is gonna work 456 00:55:16.900 --> 00:55:19.539 David Stephens, Prof: to guarantee balance, and indeed, for some 457 00:55:19.840 --> 00:55:23.149 David Stephens, Prof: in some forms of the assessment in our case it didn't. 458 00:55:23.810 --> 00:55:30.810 David Stephens, Prof: But for other methods it seemed to be useful. 459 00:55:32.270 --> 00:55:43.699 David Stephens, Prof: So now we're going to go on to estimating the ate with these methods for the Nhanes data set. So I'm going to use these 5 methods. I'm going to use outcome regression. 460 00:55:44.250 --> 00:55:47.330 David Stephens, Prof: propensive score stratification matching. 461 00:55:48.780 --> 00:55:51.840 David Stephens, Prof: Ps regression and then inverse probability weighting. 462 00:55:55.080 --> 00:56:11.339 David Stephens, Prof: So what can happen is that the inverse probability and aipw estimators will suffer if the inverse probability weights are too extreme, that is, if the propensity score for fitted over 463 00:56:13.100 --> 00:56:22.420 David Stephens, Prof: the fitted propensity score for treated or for untreated, is too near 0, then that's going to inflate the variance of the estimator. So what people do sometimes is to 464 00:56:22.610 --> 00:56:32.079 David Stephens, Prof: focus on the region banded away from 0, so they omit individuals that have a too low or too high probability of being treated. 465 00:56:33.130 --> 00:56:39.300 David Stephens, Prof: and you can do that in a number of ways you can think about truncating. So set everybody 466 00:56:39.630 --> 00:56:51.089 David Stephens, Prof: who has a propensity score lower than this to point 0 0 1 higher than this to be 0 point 9 9, or you could just throw away those samples. Different different approaches are used, and there are 467 00:56:51.840 --> 00:56:56.879 David Stephens, Prof: benefits and disadvantages to any of those methods. 468 00:56:58.350 --> 00:56:59.650 David Stephens, Prof: Okay, so 469 00:57:00.080 --> 00:57:04.830 David Stephens, Prof: and now we're going to compare the estimates. So the unadjusted version that we saw. 1st of all, is this 470 00:57:05.670 --> 00:57:12.119 David Stephens, Prof: just the unadjusted analysis we think about the method with 471 00:57:12.690 --> 00:57:21.989 David Stephens, Prof: the main main effects and no interactions. Again, the method can give me this if I fit interactions, though. 472 00:57:32.333 --> 00:57:39.620 David Stephens, Prof: Sorry I haven't done that yet is I'm just showing that we can get this estimate, either as the coefficient attached into the 473 00:57:40.280 --> 00:57:46.240 David Stephens, Prof: regresses in this linear model, or by going through the prediction exercise which 474 00:57:46.390 --> 00:57:54.859 David Stephens, Prof: takes the predicted outcome under treatment, the predicted outcome, and no treatment, and then looks at the average of those 475 00:57:55.440 --> 00:58:00.540 David Stephens, Prof: and looks at the difference average. That gives us the same answer as before in this model. 476 00:58:00.860 --> 00:58:11.740 David Stephens, Prof: Okay. But I'm showing you that method because we need to use this method. If I have interactions between treatment and the other covariates fitted in the outcome model. 477 00:58:13.390 --> 00:58:23.590 David Stephens, Prof: So if I have interactions, let's say I'm going to fit the interaction between smoking and household income and gender and age. 478 00:58:23.760 --> 00:58:35.009 David Stephens, Prof: then I can't pick off a single coefficient in this analysis to get the average treatment effect. What I need to do is to predict under treatment predict under no treatment to take the difference. 479 00:58:35.590 --> 00:58:43.560 David Stephens, Prof: So that's basically what this code here is doing. We see it here implemented for the new model. This is the collection of coefficients that we get. 480 00:58:44.000 --> 00:58:47.379 David Stephens, Prof: including a main effect, but also interactions. 481 00:58:47.740 --> 00:58:58.140 David Stephens, Prof: So you look at this, and you think, well, how how am I going to pick off a single average treatment effect from this? Well, the method is to use predictions from this model average the predictions and then take the difference 482 00:58:59.280 --> 00:59:00.579 David Stephens, Prof: as it says here. 483 00:59:02.130 --> 00:59:06.370 David Stephens, Prof: Okay, so we do that. And we end up with an average treatment effect 484 00:59:06.610 --> 00:59:08.700 David Stephens, Prof: of minus 1.4 0. 485 00:59:10.280 --> 00:59:10.950 David Stephens, Prof: Okay. 486 00:59:12.690 --> 00:59:19.150 David Stephens, Prof: we can use propensity score stratification. Here's the code that does that. So I'm going to use quintile stratification. 487 00:59:19.940 --> 00:59:24.243 David Stephens, Prof: I'm gonna place or I'm gonna 488 00:59:25.920 --> 00:59:31.809 David Stephens, Prof: Recognize that each individual, whether they're treated or untreated falls into one of the 5 strata. 489 00:59:32.560 --> 00:59:36.270 David Stephens, Prof: I'm gonna basically do a comparison within each strata 490 00:59:36.740 --> 00:59:44.699 David Stephens, Prof: stratum and then combine the estimates across strata to get an overall estimate. So this is the code that does that 491 00:59:44.920 --> 00:59:48.860 David Stephens, Prof: standard base R code here. And I end up with 492 00:59:49.140 --> 00:59:52.589 David Stephens, Prof: an average tumort effect estimate of minus 1.8 1. 493 00:59:53.640 --> 00:59:56.660 David Stephens, Prof: Okay, that's a stratification estimator. 494 00:59:57.140 --> 00:59:59.420 David Stephens, Prof: If I do a matching estimator 495 01:00:00.160 --> 01:00:05.349 David Stephens, Prof: based on the propensity score, I can also get that from the match function. 496 01:00:06.410 --> 01:00:10.120 David Stephens, Prof: And my estimate that I get here is much lower. 497 01:00:12.480 --> 01:00:21.949 David Stephens, Prof: Okay, this is how I would implement that. I use the match function to create the match, and then I use the matched data to estimate in a 2 sample comparison 498 01:00:22.290 --> 01:00:24.429 David Stephens, Prof: the overall difference in meetings. 499 01:00:26.020 --> 01:00:35.289 David Stephens, Prof: Okay, so again, I've provided this code. I don't expect you to read it and go through it now. But it's provided there for you to implement on your own data sets. Maybe 500 01:00:36.600 --> 01:00:37.490 David Stephens, Prof: all right. 501 01:00:37.790 --> 01:00:50.860 David Stephens, Prof: So we can now think about propensity score regression methods. So here are 3 propensity score regression, the base propensity score regression method. And then with some more complicated terms, including splines. 502 01:00:51.600 --> 01:01:01.179 David Stephens, Prof: Right. So I've got a spline here on the propensity. So what I'm going to do is to take that previously fitted propensity score model and include it into these 3 models and see what the 503 01:01:01.340 --> 01:01:04.380 David Stephens, Prof: results are for the coefficient of 504 01:01:04.570 --> 01:01:10.959 David Stephens, Prof: site one or the average treatment effect arising from Psr. 2 or Psr. 3. 505 01:01:12.650 --> 01:01:15.230 David Stephens, Prof: If I use the 1st method, I get a 506 01:01:15.500 --> 01:01:19.520 David Stephens, Prof: estimate of minus 1.1 0 7. 507 01:01:21.200 --> 01:01:25.200 David Stephens, Prof: Again, using the similar method as before, taking the the 508 01:01:25.950 --> 01:01:29.659 David Stephens, Prof: fitted propensity score model and then averaging the predictions under it. 509 01:01:30.480 --> 01:01:31.730 David Stephens, Prof: If I use the second 510 01:01:31.920 --> 01:01:36.440 David Stephens, Prof: second propensity score model that gives me an estimate of minus 1.6 8. 511 01:01:36.760 --> 01:01:42.740 David Stephens, Prof: If I take the 3, rd I get an estimate of minus 1.9 7 fitting. The model is 512 01:01:42.940 --> 01:01:50.569 David Stephens, Prof: basically an exercise in linear regression modeling in this case, because the outcome is continuous is just depend on what terms we include in the 513 01:01:50.700 --> 01:01:52.525 David Stephens, Prof: in the formula, okay? 514 01:01:54.305 --> 01:01:58.109 David Stephens, Prof: I can then look at Ipw estimation. So if you remember. 515 01:01:58.430 --> 01:02:04.620 David Stephens, Prof: back at the start, we said that Matching and Ipw estimation, possibly with a 516 01:02:05.000 --> 01:02:08.599 David Stephens, Prof: most reliable because we could guarantee balance 517 01:02:08.730 --> 01:02:14.230 David Stephens, Prof: using those methods, the Ipw estimator can be obtained using 518 01:02:15.200 --> 01:02:25.510 David Stephens, Prof: weightedly squares with a weight determined by these forms here, so, treatment over probability of treatment, no treatment over one minus probability of 519 01:02:26.040 --> 01:02:34.220 David Stephens, Prof: treatment, and we fit these models first.st Use this weight in the weighted least squares. 520 01:02:34.590 --> 01:02:39.410 David Stephens, Prof: The 2 stand 2 versions of the weight estimator given by 521 01:02:39.950 --> 01:02:45.969 David Stephens, Prof: the simple difference in means or the weight standardized difference in means. 522 01:02:47.200 --> 01:02:53.650 David Stephens, Prof: This is the version. Actually, this estimator here is what you get if you use weighted least squares. 523 01:02:54.120 --> 01:03:01.400 David Stephens, Prof: Here's the here the results. So here I'm doing just the direct, weighted average difference. 524 01:03:01.740 --> 01:03:07.370 David Stephens, Prof: So my ps.lr, dot, wait is formed in the same way as before. 525 01:03:07.660 --> 01:03:14.709 David Stephens, Prof: I look at the difference in means between the smoking group and the non-smoking group, and I get minus 1.9 2. 526 01:03:15.240 --> 01:03:24.010 David Stephens, Prof: If I use the standardized weight version, which is going to be slightly different in finite sample, I get a broadly comparable answer. 527 01:03:26.180 --> 01:03:36.719 David Stephens, Prof: I could use inverse, perverturating with augmentation aipw. And for that I need to specify a mean regression model to augment the previous estimator. 528 01:03:37.900 --> 01:03:43.290 David Stephens, Prof: Okay, so I do that I fit these M models and then 529 01:03:43.430 --> 01:03:45.400 David Stephens, Prof: form the estimator in this way. 530 01:03:47.480 --> 01:03:54.280 David Stephens, Prof: Here I'm doing precisely that calculation. 531 01:03:56.320 --> 01:04:00.100 David Stephens, Prof: Using 1st of all a fit 532 01:04:00.300 --> 01:04:06.530 David Stephens, Prof: of the M model using Lm, F. MoD, I is the interaction model for the outcome. 533 01:04:07.540 --> 01:04:20.050 David Stephens, Prof: Okay, I then take the propensity score treatment. Sorry the propensity score. Wait, boomed my. 534 01:04:20.190 --> 01:04:31.830 David Stephens, Prof: oh, so the Ms probability weight formed by treatment over propensity school, or one minus treatment over 535 01:04:33.760 --> 01:04:43.049 David Stephens, Prof: one minus propensity score for the untreated group. Do that here. Actually, in this case, I'm implementing truncation as well, because I want to get rid of some of the extreme weights. 536 01:04:43.720 --> 01:04:47.800 David Stephens, Prof: So then, I look at the difference in the 2 average 537 01:04:48.430 --> 01:04:52.700 David Stephens, Prof: potential outcomes formed by that waiting reweighting of the samples. 538 01:04:52.890 --> 01:04:58.619 David Stephens, Prof: And then I add, on the average predictions under the M model. 539 01:04:58.870 --> 01:05:04.079 David Stephens, Prof: So, going through that exercise, I end up with an estimate of minus 1.8 5. 540 01:05:04.570 --> 01:05:07.330 David Stephens, Prof: Right? So all of these things are broadly comparable. 541 01:05:09.160 --> 01:05:13.370 David Stephens, Prof: which tells us that the estimate is gonna be in that range. Roughly. 542 01:05:13.750 --> 01:05:20.429 David Stephens, Prof: I then use augmented outcome regression which forms the clever covariates included in the formula. Here 543 01:05:22.200 --> 01:05:28.270 David Stephens, Prof: estimate the coefficients for those, and then takes the difference in means of the predictions from that model. 544 01:05:31.470 --> 01:05:35.450 David Stephens, Prof: And I end up with an estimate of minus 1.8 1. 545 01:05:35.790 --> 01:05:36.610 David Stephens, Prof: Right? 546 01:05:36.810 --> 01:05:41.389 David Stephens, Prof: So again, broadly in the same range. 547 01:05:43.010 --> 01:05:49.890 David Stephens, Prof: Okay, some things to remember here is that something to remember here is that all of the school approaches. Consider 548 01:05:50.070 --> 01:06:01.130 David Stephens, Prof: the use of substitution estimators. That means when I estimate the propensity score, I take the fitted value, and I plug that in. I don't account for any other variation. 549 01:06:03.130 --> 01:06:12.790 David Stephens, Prof: Right? We plug in the estimated propensity score as a covariate, or in in the form of the weights, and it turns out that in many cases that is a sufficient approach. 550 01:06:16.090 --> 01:06:19.830 David Stephens, Prof: In general, we need to account for it. But in many cases the 551 01:06:20.688 --> 01:06:24.870 David Stephens, Prof: the adjustment due to that 1st stage estimation is minimal. 552 01:06:27.270 --> 01:06:28.380 David Stephens, Prof: We can 553 01:06:28.550 --> 01:06:39.670 David Stephens, Prof: make sure of that using bootstrap approaches. But we've got to remember that the bootstrap is questionable when it comes to certain sorts of statistical procedure, such as, for example. 554 01:06:40.330 --> 01:06:47.030 David Stephens, Prof: matching for matching. You've got a bit bit more careful about the 555 01:06:50.630 --> 01:06:52.960 David Stephens, Prof: about the implementation of the bootstrap 556 01:06:53.250 --> 01:06:56.029 David Stephens, Prof: and the conclusions you can make from that analysis. 557 01:06:56.490 --> 01:07:01.729 David Stephens, Prof: The match function does report standard errors. 558 01:07:02.518 --> 01:07:11.460 David Stephens, Prof: Directly. So you don't have to actively compute them. So I haven't talked much about the compute computation of the standard errors that would be in my second example. 559 01:07:13.770 --> 01:07:17.649 David Stephens, Prof: So here I did a bootstrap study, using all of those methods on these data. 560 01:07:17.860 --> 01:07:25.929 David Stephens, Prof: right? So we see a significant difference between the unadjusted analysis and all of the other analyses. 561 01:07:27.109 --> 01:07:31.789 David Stephens, Prof: But what we get the picture of is that essentially the estimate 562 01:07:32.640 --> 01:07:41.000 David Stephens, Prof: for all of them is comparable. So maybe we could be convinced that this is A reasonable estimate of the true effect. 563 01:07:42.010 --> 01:07:47.180 David Stephens, Prof: Some other things to note are that the matching estimator is more variable. That's typical. 564 01:07:48.228 --> 01:07:56.389 David Stephens, Prof: The Ipw estimators are slightly higher variance than the propensity score regression estimators, and there are theoretical reasons for that. 565 01:07:58.930 --> 01:08:04.620 David Stephens, Prof: But broadly, there's a comparability between the between the conclusions. 566 01:08:07.360 --> 01:08:12.930 David Stephens, Prof: There's a summary of the estimates and the standard errors, computing using those methods. 567 01:08:14.210 --> 01:08:27.569 David Stephens, Prof: Okay? So I went through that fairly quickly. But you can look back and look at the individual analysis and implement them for yourself. I'll upload. I didn't upload yet the net file that I used, but I can upload that as well, so you can just run that netter directly. 568 01:08:28.830 --> 01:08:29.789 David Stephens, Prof: All right. 569 01:08:30.130 --> 01:08:38.480 David Stephens, Prof: So the the next collection of practical results I want to show you are to do with. 570 01:08:39.990 --> 01:08:49.830 David Stephens, Prof: Sorry to do with this simulation study, which 571 01:08:50.160 --> 01:08:54.980 David Stephens, Prof: pairs, statistication, matching regression and inverse weighting in a single analysis. 572 01:08:55.279 --> 01:09:03.439 David Stephens, Prof: Okay, so I do a simulation here of a sample size of 2,000, according to a very specific model that has 573 01:09:03.590 --> 01:09:08.469 David Stephens, Prof: x 1 x. 2, and the interaction between x. 1 and x 2 in the 574 01:09:09.870 --> 01:09:14.589 David Stephens, Prof: fancy score model and in the outcome model. Through this interaction 575 01:09:14.990 --> 01:09:19.589 David Stephens, Prof: collection, right? So the average treatment effect is going to be 576 01:09:20.270 --> 01:09:29.930 David Stephens, Prof: computed by computing the estimates for psi. 0 through psi. 3, and then averaging the predictions 577 01:09:30.180 --> 01:09:33.010 David Stephens, Prof: under the different X values that we observe. 578 01:09:33.160 --> 01:09:34.920 David Stephens, Prof: So there's the simulation. 579 01:09:35.905 --> 01:09:38.640 David Stephens, Prof: I can compute the theoretical quantities 580 01:09:38.810 --> 01:09:46.699 David Stephens, Prof: in this model very easily. So again, I'll leave you to look through that. And then I go through the same steps again, unadjusted analysis 581 01:09:48.920 --> 01:09:55.750 David Stephens, Prof: showing bias some discussion of the sandwich estimator there. 582 01:09:56.447 --> 01:10:05.020 David Stephens, Prof: Doing a more advanced outcome regression analysis on the correct specification. I get the right answer under incorrect specification. I get the wrong answer. 583 01:10:06.590 --> 01:10:22.463 David Stephens, Prof: Okay, here I go into some details about how to compute the estimated sorry. The the Sandwich estimated standard error for the effective interest, and it's a little bit involved, but it's essentially a manipulation of this, the 584 01:10:23.510 --> 01:10:30.630 David Stephens, Prof: sandwich variance that we get for from the from the ols, okay? 585 01:10:30.800 --> 01:10:37.720 David Stephens, Prof: And what I end up with is in this example. So the true value here in this simulation is 586 01:10:38.480 --> 01:10:44.660 David Stephens, Prof: given here. The true average treatment effect is minus 1.8 8 7 5. 587 01:10:44.870 --> 01:10:49.549 David Stephens, Prof: For my unadjusted analysis. I get minus 1.5 1 significantly wrong. 588 01:10:50.792 --> 01:11:02.349 David Stephens, Prof: For my outcome regression. I get exactly the right analysis, or exactly the right answer from a correct specification and a very wrong analysis from an incorrect specification as shown here. 589 01:11:03.760 --> 01:11:12.950 David Stephens, Prof: Okay, so all everything up to that point is really just a standard statistical analysis. 590 01:11:13.100 --> 01:11:20.770 David Stephens, Prof: then I start using propensity, score stratification. So I look at the true propensity score distribution. It looks like this in this case. 591 01:11:21.830 --> 01:11:22.760 David Stephens, Prof: and 592 01:11:23.220 --> 01:11:31.860 David Stephens, Prof: what I can verify is that there's enough data in each of the propensity score quintile strata to facilitate good estimation 593 01:11:34.130 --> 01:11:47.520 David Stephens, Prof: here. I'm using a correct, the correct propensity score. So I know that balance is going to be achieved. So I progress straight to the estimation. Here is the code to do the stratification estimator, and here I. You see, I get the right answer. 594 01:11:50.390 --> 01:11:58.190 David Stephens, Prof: I can then, or a a reasonably good answer. I can then think about making that more. 595 01:11:58.410 --> 01:12:11.059 David Stephens, Prof: or implementing that through a linear model here, I do it by hand, basically. But here I'm going to use the Lm function in order to implement the same analysis, so that minus 1.8 4 for the 596 01:12:11.190 --> 01:12:18.080 David Stephens, Prof: so the main difference there is the ate and I get the same answer. If I use the Lm approach. 597 01:12:19.050 --> 01:12:29.550 David Stephens, Prof: Okay, I can then use the sandwich variance competition to think about the 598 01:12:30.300 --> 01:12:40.579 David Stephens, Prof: difference between the model based standard error and the sandwich standard error. And and in this case the sandwich standard error is smaller than the model based version. 599 01:12:44.070 --> 01:12:51.570 David Stephens, Prof: Then I implement propensity score analysis, and that's given there and then. 600 01:12:53.150 --> 01:13:08.709 David Stephens, Prof: Matching and then propensity, score regression and inverse probability weighting augmented Ipw 601 01:13:16.180 --> 01:13:18.030 David Stephens, Prof: augmented outcome regression. 602 01:13:19.960 --> 01:13:22.090 David Stephens, Prof: And then finally, gestimation. 603 01:13:23.020 --> 01:13:25.259 David Stephens, Prof: Okay? So all of the methods 604 01:13:25.450 --> 01:13:32.630 David Stephens, Prof: with a little bit of summary of the theory is given in this handout again. I won't go through the details now. 605 01:13:32.890 --> 01:13:37.130 David Stephens, Prof: You can use that email me if you have any questions about what's being implemented there. 606 01:13:40.310 --> 01:13:48.859 David Stephens, Prof: I have another simulation handout which is really more focused on computing the astronotic variance. 607 01:13:49.523 --> 01:13:56.279 David Stephens, Prof: calculations using simulation and comparing the theoretical with the Monte Carlo based 608 01:13:56.380 --> 01:13:59.850 David Stephens, Prof: estimation. So that's there for a couple of 609 01:14:00.320 --> 01:14:11.220 David Stephens, Prof: cases. So again, the theory gets a bit more advanced in here. But if you're interested in looking at the construction of sandwich estimators for this type of this type of model, it's included in this handout. 610 01:14:16.350 --> 01:14:17.060 David Stephens, Prof: Okay? 611 01:14:17.690 --> 01:14:24.539 David Stephens, Prof: And then I've got one other simulation study uploaded to the 612 01:14:25.750 --> 01:14:34.730 David Stephens, Prof: to the website. And this goes through different settings, and it attempts to show the difference in performance of the different estimators. 613 01:14:36.300 --> 01:14:41.480 David Stephens, Prof: And again, I won't go through the details here. You can find the code on the website if you want to implement it yourself. 614 01:14:41.590 --> 01:14:43.880 David Stephens, Prof: But the point of this message is that 615 01:14:45.750 --> 01:14:49.210 David Stephens, Prof: Matching has got a higher variance than. 616 01:14:49.610 --> 01:14:55.020 David Stephens, Prof: and there's probability weighting which has got a higher variance than parenthesical regression 617 01:14:56.310 --> 01:14:59.169 David Stephens, Prof: right? They all come with different assumptions. 618 01:15:00.220 --> 01:15:10.069 David Stephens, Prof: But if the propensity score regression assumptions are correct, that is, you have the correct model for the outcome component, the sorry. The treatment influence on the outcome model. 619 01:15:10.310 --> 01:15:13.390 David Stephens, Prof: then prevent score regression is the best thing to do. 620 01:15:13.870 --> 01:15:18.480 David Stephens, Prof: Okay? And that's what this series of simulations is meant to illustrate. 621 01:15:20.200 --> 01:15:25.029 David Stephens, Prof: Okay, so now gonna go back to the to finish off the slides. 622 01:15:26.990 --> 01:15:27.890 David Stephens, Prof: And 623 01:15:28.310 --> 01:15:38.640 David Stephens, Prof: so hopefully, in what I've covered them. The materials I've given you from the in those handouts will show you how to implement all of the methods that I mentioned. 624 01:15:38.930 --> 01:15:44.100 David Stephens, Prof: and hopefully, I've been able to click. Make clear why each of those methods works. 625 01:15:45.140 --> 01:15:49.710 David Stephens, Prof: and I think understanding why they work is useful. 626 01:15:50.880 --> 01:16:00.440 David Stephens, Prof: not only because it gives a a better understanding of the limitations, perhaps, but also because if you want to go on developing 627 01:16:00.670 --> 01:16:04.004 David Stephens, Prof: new related methods, then I think 628 01:16:05.240 --> 01:16:14.010 David Stephens, Prof: understanding at this level why these methods work in the way they do will greatly assist in any future research development. At least that's what I found. 629 01:16:14.610 --> 01:16:17.719 David Stephens, Prof: It took me a long time to understand exactly why 630 01:16:18.020 --> 01:16:26.970 David Stephens, Prof: Ipw and Aipw worked until I saw the connection with Monte Carlo and important sampling change of measure type ideas. 631 01:16:27.990 --> 01:16:29.859 David Stephens, Prof: Okay? So just to finish off 632 01:16:30.040 --> 01:16:32.149 David Stephens, Prof: gonna look at some quick extensions. 633 01:16:33.370 --> 01:16:34.220 David Stephens, Prof: Oops. 634 01:16:36.060 --> 01:16:36.880 David Stephens, Prof: Sorry. 635 01:16:39.180 --> 01:16:43.720 David Stephens, Prof: So in longitudinal. So the main extension is going to be in longitudinal studies. 636 01:16:44.730 --> 01:16:48.499 David Stephens, Prof: So in longitudinal studies, we've got 637 01:16:48.750 --> 01:16:53.700 David Stephens, Prof: to estimate the effect of a collection of treatments on an outcome. 638 01:16:53.840 --> 01:16:57.580 David Stephens, Prof: or a collection of outcomes collected longitudinally 639 01:16:58.630 --> 01:17:06.349 David Stephens, Prof: so. We might be interested in the direct effect of treatment at an individual time point, or the total effect of a collection of treatments. 640 01:17:08.690 --> 01:17:18.349 David Stephens, Prof: So the the corresponding dag for a 5 time point data collection exercise might look like this. 641 01:17:18.540 --> 01:17:26.860 David Stephens, Prof: the Z's are the longitudinal treatments. The X are the longitudinal confounders, and the Y are the intermediate and indeed terminal 642 01:17:27.240 --> 01:17:28.070 David Stephens, Prof: outcome. 643 01:17:28.980 --> 01:17:31.469 David Stephens, Prof: So there are more complicated models that I could 644 01:17:31.630 --> 01:17:35.610 David Stephens, Prof: represent using dags like this. But this is a a typical one. 645 01:17:36.710 --> 01:17:53.660 David Stephens, Prof: So what we want to understand is what Z effect on each of the variables. So there's a direct effect of z on y 1, but also an indirect effect of z 1 on y. 2 that passes through either y, 1 or z, 2 646 01:17:54.570 --> 01:17:57.080 David Stephens, Prof: z. 2 has an effect on z. 3, 647 01:17:57.530 --> 01:18:03.509 David Stephens, Prof: and therefore on y. 3, but it also has effect on z. 2, and therefore y. 3 through another path. 648 01:18:03.750 --> 01:18:11.440 David Stephens, Prof: So, as you can see, the more complicated we make this longitudinal picture, the greater the challenge in 649 01:18:11.610 --> 01:18:17.339 David Stephens, Prof: extracting causal effects is going to be because we need to make sure we block exactly the right paths 650 01:18:17.450 --> 01:18:19.370 David Stephens, Prof: that caused confounding. 651 01:18:24.450 --> 01:18:30.980 David Stephens, Prof: So we have this issue of mediation of the effect of earlier treatments through later variables. 652 01:18:31.620 --> 01:18:34.879 David Stephens, Prof: And we have the issue of time varying, confounding. 653 01:18:36.710 --> 01:18:43.520 David Stephens, Prof: So here's the notation. And all of this is basic. I'm going through this quite quickly, because we've only got about 654 01:18:43.690 --> 01:18:52.830 David Stephens, Prof: 12 min left. But basically, the the notational framework extends from the single time point case to the multiple time point case we think about 655 01:18:52.940 --> 01:18:58.399 David Stephens, Prof: counterfactuals in a longitudinal fascial potential outcomes to a pattern of treatments. 656 01:19:01.755 --> 01:19:08.089 David Stephens, Prof: We can extend the propensity score to the generalized propensity score in a fairly straightforward fashion. 657 01:19:10.070 --> 01:19:16.850 David Stephens, Prof: requiring the modeling of the treatment at interval. J. Based on all previous variables. 658 01:19:18.130 --> 01:19:26.910 David Stephens, Prof: This multivariate generalized propensity score will allow us to get a direct effect in the same way that it does in the univariate single time point setting. 659 01:19:27.230 --> 01:19:32.149 David Stephens, Prof: We have all the same considerations about knowing, measuring founders and positivity 660 01:19:32.370 --> 01:19:35.309 David Stephens, Prof: in the construction of this quantity. 661 01:19:37.730 --> 01:19:44.550 David Stephens, Prof: Okay, whenever we're thinking about longitudinal data, we could also use 662 01:19:45.320 --> 01:19:47.969 David Stephens, Prof: a mixed or random effects model 663 01:19:48.140 --> 01:19:54.030 David Stephens, Prof: to capture the joint structure embedded into the into the dag. 664 01:19:54.200 --> 01:19:58.550 David Stephens, Prof: So if we've got longitudinal observations, then. 665 01:20:00.460 --> 01:20:10.520 David Stephens, Prof: or repeated measures data, we could always think about using random effects. Type approaches, and it turns out that there are some random effects approaches that can be used in the causal setting 666 01:20:10.720 --> 01:20:14.129 David Stephens, Prof: to do exactly the right kind of adjustment to 667 01:20:14.460 --> 01:20:16.320 David Stephens, Prof: get at the cause of the fact. 668 01:20:19.120 --> 01:20:34.629 David Stephens, Prof: The example I'm going to go into a little bit more, though, is perhaps the most common kind of longitudinal setting, which is where we have a pattern of treatments over a number of intervals, and we're interested in a total effect of that pattern on a terminal outcome. 669 01:20:35.090 --> 01:20:47.230 David Stephens, Prof: So this leads us to what's called the marginal structural model. So the marginal structural model is basically based on a longer multi interval, inverse probability weighting approach. 670 01:20:49.460 --> 01:20:53.050 David Stephens, Prof: So I've got this is my dag here in the 5 Time point case. 671 01:20:53.320 --> 01:20:59.619 David Stephens, Prof: I've got this collection of treatments influencing a terminal outcome 672 01:21:00.040 --> 01:21:07.700 David Stephens, Prof: through maybe some intermediate outcomes and time varying and founders 673 01:21:08.810 --> 01:21:14.429 David Stephens, Prof: different ways. People have approached this. Sometimes the survival type approach is used 674 01:21:14.710 --> 01:21:18.359 David Stephens, Prof: and pulled logistic regression to get it. 675 01:21:18.680 --> 01:21:26.130 David Stephens, Prof: The the terminal binding outcome. Our models are constructed in 676 01:21:26.760 --> 01:21:36.729 David Stephens, Prof: the following way. So essentially, what we end up doing is proposing a marginal model for the outcome of interest. So here, if y is a binary response. 677 01:21:36.920 --> 01:21:39.369 David Stephens, Prof: I propose a model for the log odds. 678 01:21:40.510 --> 01:21:43.209 David Stephens, Prof: and I do that based on 679 01:21:43.430 --> 01:21:50.310 David Stephens, Prof: an intercept in this case, and a treatment effect captured through a parameter site. 680 01:21:51.260 --> 01:21:59.459 David Stephens, Prof: and the treatment effect relates to the aggregate treatment. The sum of the Zj's from J. Equals one to M. 681 01:22:00.380 --> 01:22:10.500 David Stephens, Prof: So it's phrased here through a counterfactual model, basically saying that the counterfactual outcome for that particular treatment pattern 682 01:22:10.860 --> 01:22:16.830 David Stephens, Prof: will be a binary variable where the log odds is captured through this right-hand side. 683 01:22:17.740 --> 01:22:26.050 David Stephens, Prof: So my interest is in estimating side. This is a marginal model, because it doesn't explicitly depend on the conditionally on the axis. 684 01:22:26.530 --> 01:22:38.659 David Stephens, Prof: So if I believe this was the model I wanted to fit, and I wanted to capture the treatment effect. This aggregate treatment effect through this model, and I wanted to estimate psi. I couldn't do that from the original data 685 01:22:39.080 --> 01:22:49.659 David Stephens, Prof: by using a regression approach regressing Y on observed treatments, because that would be subject to confounding and mediation, time bearing, confounding and mediation. 686 01:22:51.280 --> 01:22:52.449 David Stephens, Prof: as it says here. 687 01:22:52.600 --> 01:22:55.909 David Stephens, Prof: So what I need to do is to do an inverse weighting analysis. 688 01:22:56.490 --> 01:22:59.980 David Stephens, Prof: and under some assumptions similar to the assumptions in the 689 01:23:01.870 --> 01:23:10.799 David Stephens, Prof: single time point setting extended to have conditional independence longitudinally of the so no one measure can find us in a sequential fashion. 690 01:23:12.480 --> 01:23:14.379 David Stephens, Prof: I can use the same approach 691 01:23:14.740 --> 01:23:17.520 David Stephens, Prof: of inverse weighting by the treatment pattern 692 01:23:17.680 --> 01:23:19.850 David Stephens, Prof: to estimate the parameter of interest. 693 01:23:21.710 --> 01:23:27.559 David Stephens, Prof: So I won't go through the details, I think, because I'm almost out of time. But the but this method 694 01:23:29.140 --> 01:23:32.819 David Stephens, Prof: allows me to construct estimators of 695 01:23:32.990 --> 01:23:40.430 David Stephens, Prof: average potential outcomes simply by reweighting the observed outcomes according to 696 01:23:41.110 --> 01:23:44.849 David Stephens, Prof: the inverse probability, weight associated with a treatment pattern. 697 01:23:45.460 --> 01:23:58.199 David Stephens, Prof: Right? So here I have the observed pattern of treatments. If I want the average potential outcome for a particular proposed treatment pattern. I just need to pick off 698 01:23:58.640 --> 01:24:05.559 David Stephens, Prof: the individuals that followed that treatment pattern and reweight them by the probability of receiving those treatments. 699 01:24:07.360 --> 01:24:14.110 David Stephens, Prof: Okay, so the mechanism for adjustment here is precisely the same as in the single time. Point case. 700 01:24:15.610 --> 01:24:19.729 David Stephens, Prof: Okay, I can consider the the pattern of treatments 701 01:24:20.420 --> 01:24:27.370 David Stephens, Prof: as a single quantity that I want to compute the probability of, and I reweight by that. 702 01:24:27.610 --> 01:24:33.590 David Stephens, Prof: So I'm up waiting according to their observed treatment pattern. 703 01:24:34.980 --> 01:24:44.409 David Stephens, Prof: That's not the only way I can do it. I can do a model based analysis which proposes these marginal models, and then requires me to estimate parameters in those models. 704 01:24:45.030 --> 01:24:48.380 David Stephens, Prof: and then, in order to learn about these parameters. 705 01:24:48.740 --> 01:24:53.339 David Stephens, Prof: I can use weightedly squares with the weights as given by the 706 01:24:55.260 --> 01:24:57.650 David Stephens, Prof: form in the marginal structural model. 707 01:24:58.240 --> 01:24:58.990 David Stephens, Prof: Okay? 708 01:25:01.646 --> 01:25:02.679 David Stephens, Prof: And then 709 01:25:02.960 --> 01:25:14.259 David Stephens, Prof: this weightedly squares can be implemented fairly, simply. So the key idea here is. And this is the I I guess the standard workhorse method. These 2 approaches the Msm. And the 710 01:25:14.480 --> 01:25:21.060 David Stephens, Prof: weighted least squares approach. These are the the 2 workhorse method for longitudinal treatments and causal adjustment. 711 01:25:21.640 --> 01:25:27.340 David Stephens, Prof: Okay, the principle of inverse probability is being applied inverse probability, weighting is being applied. 712 01:25:27.590 --> 01:25:32.140 David Stephens, Prof: And it's following from the same theory that we saw for the single time. Point case. 713 01:25:32.620 --> 01:25:43.710 David Stephens, Prof: Okay, we have to adjust for confounding, break the confounding, and indeed the the ignore, the time varying mediation 714 01:25:43.820 --> 01:25:49.829 David Stephens, Prof: by introducing these probability weights that upweight 715 01:25:50.110 --> 01:25:53.389 David Stephens, Prof: the individual observations in exactly the right way. 716 01:25:57.353 --> 01:26:04.729 David Stephens, Prof: And yeah. And then this is just the details of that. So we need typically parametric models to represent this joint distribution of the treatment pattern 717 01:26:07.555 --> 01:26:10.315 David Stephens, Prof: to skip over the details here, because it's 718 01:26:13.690 --> 01:26:20.560 David Stephens, Prof: we're getting up to the end of the course. But if anybody's got any questions about this, I'll stick around afterwards and answer them. 719 01:26:25.380 --> 01:26:28.731 David Stephens, Prof: Okay, so the the to finish off. Then 720 01:26:29.980 --> 01:26:37.200 David Stephens, Prof: where are the current challenges in implementation of these methods. Well, really, it's about complexity and high dimensionality of the data sets. 721 01:26:37.710 --> 01:26:41.050 David Stephens, Prof: So we see in more and more settings that 722 01:26:41.260 --> 01:26:52.689 David Stephens, Prof: the setting of Rosenbaum and Ruben, where we're thinking of a low number of covariates, and a single treatment is is really not being reflected in modern data sets of the electronic health records. 723 01:26:52.960 --> 01:27:01.190 David Stephens, Prof: real time health monitoring pharmacoa, and so on. Where we have a large number of potential confounders to adjust for 724 01:27:03.250 --> 01:27:11.709 David Stephens, Prof: And we've seen the principles, and I've mentioned some caveats about applying them in high dimensional settings as we've gone through. 725 01:27:15.240 --> 01:27:17.542 David Stephens, Prof: What we have to remember, though, is that 726 01:27:18.280 --> 01:27:20.859 David Stephens, Prof: the decisions that we make when we 727 01:27:22.290 --> 01:27:28.079 David Stephens, Prof: decide to include or exclude data of sometimes profound effects on the results that we report. 728 01:27:29.270 --> 01:27:32.550 David Stephens, Prof: Different databases may lead to different conclusions. 729 01:27:33.210 --> 01:27:35.790 David Stephens, Prof: and that's not because of the 730 01:27:36.320 --> 01:27:39.249 David Stephens, Prof: method that we use, but it is because of the 731 01:27:39.730 --> 01:27:42.580 David Stephens, Prof: maybe the inclusion or exclusion criteria 732 01:27:43.130 --> 01:27:45.289 David Stephens, Prof: used to construct the raw data. 733 01:27:45.920 --> 01:27:52.539 David Stephens, Prof: and the key task in all of this, for the statistician is to represent uncertainty in a coherent fashion. 734 01:27:52.880 --> 01:27:56.940 David Stephens, Prof: So this include should include uncertainty induced due to 735 01:27:57.160 --> 01:28:01.070 David Stephens, Prof: the model. Fitting, certainly, but also the incompatibility in the data sources. 736 01:28:03.260 --> 01:28:11.580 David Stephens, Prof: We've also got the usual challenges of statistical modeling. So missing data, informative missingness, truncation. 737 01:28:12.130 --> 01:28:18.030 David Stephens, Prof: selection, bias, I guess, and noncompliance in randomized trials. 738 01:28:19.550 --> 01:28:25.159 David Stephens, Prof: So all of those sort of approach, the things that are common in any high, dimensional modern statistical 739 01:28:25.400 --> 01:28:28.870 David Stephens, Prof: application are still challenges here. 740 01:28:30.110 --> 01:28:34.810 David Stephens, Prof: So to finish off. Then I've got a couple of slides of conclusions. 741 01:28:35.770 --> 01:28:36.700 David Stephens, Prof: And 742 01:28:36.820 --> 01:28:51.169 David Stephens, Prof: again, the the implementation of the methods that I've mentioned. I think it'll it really takes some time to go away and digest what I've mentioned there. So please do that with the worksheets that I've or the handouts that I've provided. 743 01:28:51.460 --> 01:28:52.240 David Stephens, Prof: But 744 01:28:53.110 --> 01:29:00.250 David Stephens, Prof: we recognize that causal inference methods provide answers to important questions concerning the impact of hypothetical exposures or treatments. 745 01:29:01.080 --> 01:29:02.070 David Stephens, Prof: We. 746 01:29:02.220 --> 01:29:07.539 David Stephens, Prof: I'll formulate problems using dags, causal representations through graphs. 747 01:29:09.860 --> 01:29:15.009 David Stephens, Prof: We can deploy standard statistical methods in most cases to make causal inferences. 748 01:29:16.220 --> 01:29:21.390 David Stephens, Prof: But we have to do it with an eye on the confound issue of confounding. 749 01:29:21.900 --> 01:29:28.730 David Stephens, Prof: so we assess and reassess the presence of confounding 750 01:29:29.010 --> 01:29:35.430 David Stephens, Prof: by thinking about balance in particular balance. 751 01:29:36.110 --> 01:29:39.160 David Stephens, Prof: It's not necessarily present in our original data. 752 01:29:39.780 --> 01:29:44.819 David Stephens, Prof: but we can induce balance and therefore render causal analysis 753 01:29:45.850 --> 01:29:53.730 David Stephens, Prof: possible by using the propensity score we can use it in a number of different ways, matching, waiting regression, and so on. 754 01:29:57.270 --> 01:30:00.620 David Stephens, Prof: We will generally use simple 755 01:30:01.430 --> 01:30:11.630 David Stephens, Prof: model components, perhaps using flexible forms. But the outcome regression, propensity score model terms. We need to do statistical constructions of 756 01:30:12.500 --> 01:30:13.360 David Stephens, Prof: okay. 757 01:30:14.710 --> 01:30:24.390 David Stephens, Prof: But now that we've got access to all of these tools from machine learning other algorithmic approaches. 758 01:30:24.940 --> 01:30:27.060 David Stephens, Prof: if we want to deploy those. 759 01:30:27.400 --> 01:30:30.499 David Stephens, Prof: then we really need to be aware of 760 01:30:30.950 --> 01:30:37.000 David Stephens, Prof: the possibilities and the limitations of the of those methods, and 761 01:30:37.370 --> 01:30:53.510 David Stephens, Prof: if they're deployed correctly, then they can be extremely useful, but they can easily be deployed in an incorrect fashion. So approach those analyses with care, whereas with more standard statistical methods, understanding things like uncertainty is much more straightforward. 762 01:30:55.700 --> 01:31:00.699 David Stephens, Prof: We have the issue of model selection in high dimension that's going to be relevant, I think. 763 01:31:01.130 --> 01:31:08.929 David Stephens, Prof: And also as data sets grow, the the structure is going to be more and more complex. 764 01:31:11.250 --> 01:31:18.519 David Stephens, Prof: So really, that's the end of the of the module. These are the people that I've worked with in constructing some of the examples that 765 01:31:19.010 --> 01:31:32.849 David Stephens, Prof: I've mentioned. My funding over the years from Nserc and Cihr has facilitated, which are the Canadian Research Councils have facilitated some of the research that I've been talking about in this module. 766 01:31:33.510 --> 01:31:39.030 David Stephens, Prof: And then at the end, I've included some key references 767 01:31:41.090 --> 01:31:49.680 David Stephens, Prof: listed there in different areas. And there's the full reference. So if somebody asked me about Rosenbaum and Ruben, there's the reference there. 768 01:31:50.590 --> 01:31:54.430 David Stephens, Prof: Central role of the propensity score in the observational studies for causal effects. 769 01:31:56.660 --> 01:32:00.040 David Stephens, Prof: Okay? And then at the end, I provided some of the appendices. 770 01:32:00.440 --> 01:32:07.550 David Stephens, Prof: Okay, so I'll I'll stop there at 3 0 1, thank you very much for attending. I'm gonna turn the recording off 771 01:32:07.830 --> 01:32:14.949 David Stephens, Prof: and then stick around to answer any questions that you might have. The recording will be posted 772 01:32:15.622 --> 01:32:23.119 David Stephens, Prof: probably in about an hour or so. It came up in an hour yesterday on my zoom repository. So thank you very much.