WEBVTT 1 00:00:15.550 --> 00:00:18.800 David Stephens, Prof: Okay then, so we'll pick up with the second half of 2 00:00:19.350 --> 00:00:22.099 David Stephens, Prof: Today's, module, the second session. 3 00:00:22.460 --> 00:00:29.960 David Stephens, Prof: And, the scene setting so far has put us in a position where we recognize that 4 00:00:31.930 --> 00:00:33.579 David Stephens, Prof: We want to make inference. 5 00:00:34.240 --> 00:00:37.470 David Stephens, Prof: a bad… Causal quantity. 6 00:00:38.450 --> 00:00:45.690 David Stephens, Prof: And that's best framed in the world of… Interventions, through experimental assignment, of treatment. 7 00:00:46.760 --> 00:00:52.630 David Stephens, Prof: Intervention to set a particular value done independently of the… predictors, or covariates, X. 8 00:00:54.000 --> 00:00:55.729 David Stephens, Prof: But we recognize that. 9 00:00:56.500 --> 00:00:59.269 David Stephens, Prof: That's not a realistic situation, in that… 10 00:00:59.720 --> 00:01:02.639 David Stephens, Prof: Most often, we're presented with observational data. 11 00:01:02.960 --> 00:01:09.419 David Stephens, Prof: Where… the independence between the predictors X and the treatment Z is not present. 12 00:01:10.670 --> 00:01:16.299 David Stephens, Prof: So now what we're going to look at is how we can adjust the calculations to take into account the 13 00:01:16.940 --> 00:01:18.950 David Stephens, Prof: The observational nature of the data. 14 00:01:20.930 --> 00:01:29.030 David Stephens, Prof: So the key challenge is that exposure is not any longer necessarily assigned independently of the other variables. 15 00:01:30.070 --> 00:01:33.170 David Stephens, Prof: So maybe some of the… predictors. 16 00:01:33.550 --> 00:01:37.119 David Stephens, Prof: Lead to… are associated with, treatment, let's say. 17 00:01:38.370 --> 00:01:45.740 David Stephens, Prof: As we know from ordinary regression logic, then… 18 00:01:46.630 --> 00:01:55.190 David Stephens, Prof: If those predictors also predict the outcome variable, so they predict treatment, and they predict the outcome, then… 19 00:01:55.950 --> 00:02:00.950 David Stephens, Prof: Any misspecification of the model associating those predictors to the 20 00:02:01.830 --> 00:02:06.580 David Stephens, Prof: Treatment, sorry, to the outcome, will lead to a mis-estimation. 21 00:02:06.760 --> 00:02:12.100 David Stephens, Prof: Bias estimation of the Coefficients associated with those variables. 22 00:02:14.150 --> 00:02:17.639 David Stephens, Prof: So the DAG that we saw early on, the page 15 DAG, 23 00:02:18.190 --> 00:02:22.459 David Stephens, Prof: Where X was a predictor of treatment, but it was also a predictor of the outcome. 24 00:02:23.980 --> 00:02:33.460 David Stephens, Prof: That situation is exactly the situation that we're most commonly faced with. Or we are faced with in the observational world. So the question is, what do we do about it? 25 00:02:37.320 --> 00:02:38.769 David Stephens, Prof: So this is the DAG. 26 00:02:39.510 --> 00:02:42.020 David Stephens, Prof: That we've been… talking about. 27 00:02:42.130 --> 00:02:51.219 David Stephens, Prof: X has an arrow going into Y, X has an arrow going into Z. The arrow that we're interested in is the arrow from Z to Y. That is, what is the effect of changing Z on Y? 28 00:02:51.460 --> 00:02:57.620 David Stephens, Prof: Imagining that we were working in an interventional world, but recognizing that actually we were in this world. 29 00:02:59.870 --> 00:03:04.949 David Stephens, Prof: So, let's think about what this means in probability terms. Basically, it means that 30 00:03:06.070 --> 00:03:08.829 David Stephens, Prof: X predicts Y in the presence of Z. 31 00:03:09.760 --> 00:03:18.579 David Stephens, Prof: That is… The conditional distribution for Y given Z and X Can't simply be written. 32 00:03:18.740 --> 00:03:24.890 David Stephens, Prof: In terms of Right? We need that X there to help predict Y. 33 00:03:26.150 --> 00:03:30.749 David Stephens, Prof: Also, we have that X predicts Z, that is. 34 00:03:31.410 --> 00:03:39.270 David Stephens, Prof: The modeling of Z needs to be conditional rather than marginal, that is, the distribution of Z depends on X, conditionally. 35 00:03:41.210 --> 00:03:50.800 David Stephens, Prof: The contrary to that would be that the marginal distribution for Z is… does not depend on X. 36 00:03:52.420 --> 00:03:56.310 David Stephens, Prof: Right? So we have these two situations. So this… 37 00:03:57.040 --> 00:04:01.299 David Stephens, Prof: really corresponds to the presence of this arrow, given that Z is there. 38 00:04:01.640 --> 00:04:05.910 David Stephens, Prof: And this… Corresponds to the presence of this arrow. 39 00:04:08.410 --> 00:04:13.199 David Stephens, Prof: Okay, so this is the general confounding situation. 40 00:04:15.660 --> 00:04:26.210 David Stephens, Prof: And in that earlier example, we can easily, and actually very commonly in observational data settings, we can think of these, dependencies being 41 00:04:26.590 --> 00:04:35.429 David Stephens, Prof: present in the real data. So, for example, we might think of socioeconomic status as being something that plays the role of X in the confounding graph. 42 00:04:37.470 --> 00:04:42.630 David Stephens, Prof: Okay? So X influences, let's say, smoking, which leads to… 43 00:04:43.700 --> 00:04:49.710 David Stephens, Prof: Sorry, could be smoking, could be pollution, could be, health. 44 00:04:49.980 --> 00:04:51.289 David Stephens, Prof: behaviour. 45 00:04:51.540 --> 00:04:59.000 David Stephens, Prof: All of those things could be factors that are dependent on X, but also will 46 00:05:00.090 --> 00:05:02.069 David Stephens, Prof: Sorry, that will be dependent on… 47 00:05:02.400 --> 00:05:08.160 David Stephens, Prof: So, we'll be influencing Y, but also influencing… diet through. 48 00:05:08.480 --> 00:05:10.819 David Stephens, Prof: The socioeconomic status background. 49 00:05:11.450 --> 00:05:12.220 David Stephens, Prof: Okay? 50 00:05:12.400 --> 00:05:19.730 David Stephens, Prof: So… The key problem, then, is that when we have 51 00:05:20.550 --> 00:05:22.830 David Stephens, Prof: this dependence between X and Z. 52 00:05:24.100 --> 00:05:32.280 David Stephens, Prof: It fundamentally means that The two groups indexed by the treatment Label. 53 00:05:32.420 --> 00:05:34.309 David Stephens, Prof: Are not… are not comparable. 54 00:05:34.720 --> 00:05:35.829 David Stephens, Prof: That is… 55 00:05:36.300 --> 00:05:45.329 David Stephens, Prof: If you take the treated group and look at their X characteristics, there's no guarantee that they'll be the same as the X characteristics that we see for the Z equals 1 group. 56 00:05:48.010 --> 00:05:53.480 David Stephens, Prof: The treated individuals could be fundamentally different from the untreated individuals. 57 00:05:54.950 --> 00:05:58.790 David Stephens, Prof: Those groups are no longer necessarily directly comparable. 58 00:06:00.490 --> 00:06:01.849 David Stephens, Prof: For example, the… 59 00:06:02.570 --> 00:06:11.219 David Stephens, Prof: the distribution of axes could be different. The effect of the confounder X on the outcome is different in the treated and untreated groups. 60 00:06:13.140 --> 00:06:17.610 David Stephens, Prof: So this lack of exchangeability, if you like, the lack of 61 00:06:17.750 --> 00:06:23.840 David Stephens, Prof: direct… the lack of an ability to directly compare the groups is… Fundamentally. 62 00:06:24.580 --> 00:06:28.230 David Stephens, Prof: What is at the heart of the… confounding. 63 00:06:29.740 --> 00:06:37.119 David Stephens, Prof: issue. And it means that we have to do some sort of adjustment in order to be able to make causal conclusions 64 00:06:37.330 --> 00:06:39.090 David Stephens, Prof: From the observational data. 65 00:06:41.460 --> 00:06:50.099 David Stephens, Prof: So it's this lack of direct comparability that's critical. Now, remember in the experimental setting, where we were randomly allocating treatment, say. 66 00:06:51.180 --> 00:06:55.339 David Stephens, Prof: Because treatment was assigned irrespective of the covariates. 67 00:06:55.720 --> 00:07:00.110 David Stephens, Prof: The two groups of individuals were comparable in terms of their covariance. 68 00:07:00.460 --> 00:07:02.300 David Stephens, Prof: Right? The two groups are exchangeable. 69 00:07:04.060 --> 00:07:05.850 David Stephens, Prof: Now we're no longer in that setting. 70 00:07:07.810 --> 00:07:10.920 David Stephens, Prof: And what that means in statistical terms is that… 71 00:07:11.270 --> 00:07:14.970 David Stephens, Prof: We're no longer in a position to be able to directly compare the sample means. 72 00:07:17.730 --> 00:07:25.010 David Stephens, Prof: Comparison of the sample means gives a misleading effect, or… 73 00:07:25.200 --> 00:07:27.280 David Stephens, Prof: Yeah, because it's a misleading effect. 74 00:07:27.710 --> 00:07:30.790 David Stephens, Prof: Misleading idea of the effect of treatment on the outcome. 75 00:07:35.910 --> 00:07:44.669 David Stephens, Prof: So, causal inference then, because of the setup we've built, is really fundamentally about comparing exposure subgroups 76 00:07:44.980 --> 00:07:46.569 David Stephens, Prof: That are comparable. 77 00:07:49.720 --> 00:07:54.129 David Stephens, Prof: that we can compare them on an equal footing. There's no influence of the other predictors. 78 00:07:54.380 --> 00:07:58.739 David Stephens, Prof: That's what the causal calculation… Means in our mathematical setup. 79 00:08:00.150 --> 00:08:03.849 David Stephens, Prof: So it's possible to do that in the randomized study case. 80 00:08:04.050 --> 00:08:06.919 David Stephens, Prof: Because we've broken the connection between X and Z. 81 00:08:07.080 --> 00:08:14.210 David Stephens, Prof: But in the observational case, in general, we haven't. In the presence of confounding, That direct comparison. 82 00:08:14.450 --> 00:08:15.609 David Stephens, Prof: It's not possible. 83 00:08:19.370 --> 00:08:21.050 David Stephens, Prof: And just to note that… 84 00:08:21.360 --> 00:08:26.700 David Stephens, Prof: This is not quite the same as the marginalization or collapsibility idea that we saw earlier on. 85 00:08:29.220 --> 00:08:38.539 David Stephens, Prof: It will always be the case, in general, that the interpretation of a conditional parameter is different from a marginal parameter, and that's true whether we've got confounding or not. 86 00:08:39.750 --> 00:08:42.100 David Stephens, Prof: That's a model specification issue. 87 00:08:42.630 --> 00:08:46.740 David Stephens, Prof: Confounding concerns, the interrelationship between the variables. 88 00:08:47.100 --> 00:08:52.509 David Stephens, Prof: And it's not really to do with the models that are being used, although that has an impact on what we can estimate. 89 00:08:54.840 --> 00:08:58.130 David Stephens, Prof: Okay, so non-collapsibility, it's probably the model 90 00:08:59.760 --> 00:09:02.760 David Stephens, Prof: Not the design, really. Not the presence of confounding. 91 00:09:07.340 --> 00:09:12.210 David Stephens, Prof: So here's a little example to illustrate the point in mathematical terms. 92 00:09:17.830 --> 00:09:19.920 David Stephens, Prof: And we'll see, as we go through. 93 00:09:21.910 --> 00:09:27.050 David Stephens, Prof: Where the critical differences are between this and the experimental setting. 94 00:09:27.260 --> 00:09:31.280 David Stephens, Prof: So, suppose that we've got 3 binary variables, so outcome. 95 00:09:31.400 --> 00:09:35.329 David Stephens, Prof: Treatment and X are all binary variables. 96 00:09:35.890 --> 00:09:38.369 David Stephens, Prof: So we've got a 2x2x2 study. 97 00:09:38.480 --> 00:09:39.900 David Stephens, Prof: design, basically. 98 00:09:40.620 --> 00:09:44.900 David Stephens, Prof: And… Let's say that the expected outcome 99 00:09:45.680 --> 00:09:49.590 David Stephens, Prof: Which is basically the probability that Y is equal to 1. 100 00:09:50.530 --> 00:09:56.549 David Stephens, Prof: Given… X and Z is given by this formula. 101 00:09:57.500 --> 00:09:59.139 David Stephens, Prof: So, a simple regression setup. 102 00:09:59.940 --> 00:10:03.310 David Stephens, Prof: This explains the relationship between YX and Z. 103 00:10:04.140 --> 00:10:10.539 David Stephens, Prof: by telling us what happens to Y if I fix levels of X and Z. 104 00:10:10.730 --> 00:10:12.119 David Stephens, Prof: Given by this formula. 105 00:10:13.070 --> 00:10:13.820 David Stephens, Prof: Okay. 106 00:10:14.170 --> 00:10:20.500 David Stephens, Prof: Suppose also that the probability that the covariate takes the value 1 is Q. 107 00:10:23.280 --> 00:10:24.330 David Stephens, Prof: then… 108 00:10:25.740 --> 00:10:32.650 David Stephens, Prof: In terms of our causal estimate, what we need to do is to take the expectation of this thing. 109 00:10:33.290 --> 00:10:39.269 David Stephens, Prof: over the distribution of X, Having fixed treatment to be some particular value. 110 00:10:39.880 --> 00:10:45.170 David Stephens, Prof: So, we fix the little red Z here, and then we take the expectation over X. 111 00:10:46.440 --> 00:10:48.630 David Stephens, Prof: Now, because X and Z are independent. 112 00:10:50.190 --> 00:10:54.489 David Stephens, Prof: in the hypothetical world where I'm doing the calculation. 113 00:10:54.830 --> 00:10:56.929 David Stephens, Prof: That means that the expected 114 00:11:00.080 --> 00:11:04.340 David Stephens, Prof: Potential outcome, the average potential outcome in this model is given by this. 115 00:11:04.990 --> 00:11:07.619 David Stephens, Prof: This is the numerical value. 116 00:11:07.760 --> 00:11:10.549 David Stephens, Prof: Once I've got numerical values of Z and Q. 117 00:11:10.670 --> 00:11:13.100 David Stephens, Prof: And if I look at the treatment effect. 118 00:11:13.590 --> 00:11:17.980 David Stephens, Prof: That's the difference between this when Z equals 0, and when Z equals 1. 119 00:11:18.310 --> 00:11:24.600 David Stephens, Prof: So all I need to do is to plug Z equals 0 and Z equal to 1 into that formula, take the difference, and I end up with 0.2. 120 00:11:24.850 --> 00:11:25.530 David Stephens, Prof: Right? 121 00:11:25.740 --> 00:11:31.100 David Stephens, Prof: So, in this model, The average treatment effect is 0.2. 122 00:11:33.910 --> 00:11:36.410 David Stephens, Prof: Now, if I were to carry out a randomized study. 123 00:11:37.550 --> 00:11:39.859 David Stephens, Prof: where Z and X were independent. 124 00:11:41.900 --> 00:11:48.760 David Stephens, Prof: Collect the data, collect the Y's, Collect the axis, Randomly allocate the Zs. 125 00:11:49.380 --> 00:12:01.449 David Stephens, Prof: after randomly allocated the Zs, and I was to take the sample mean in the untreated group, sample mean in the treated group, take the difference, I would be able to consistently estimate this quantity. 126 00:12:03.410 --> 00:12:04.880 David Stephens, Prof: Alright? So that's good. 127 00:12:09.520 --> 00:12:13.389 David Stephens, Prof: Nothing in that calculation, though, talks about the, 128 00:12:13.630 --> 00:12:21.749 David Stephens, Prof: the relationship between Z and X, right? What is the joint distribution? The assumption, the calculation is done assuming that Z or X are independent. 129 00:12:22.290 --> 00:12:29.229 David Stephens, Prof: Why can I do that? Well, because basically I'm fixing the value of Z here. It's not arising randomly anymore. 130 00:12:29.670 --> 00:12:36.120 David Stephens, Prof: I'm fixing that value, so I only need to worry about the distribution of Now… 131 00:12:37.880 --> 00:12:45.020 David Stephens, Prof: Let's suppose the data were generated, though, from a model that was of this nature, so the probability that Z equal to 1 132 00:12:46.080 --> 00:12:49.630 David Stephens, Prof: for a given X value was like this. 133 00:12:49.840 --> 00:12:57.130 David Stephens, Prof: So it takes the value 1 with probability P0, when x is 0, and P1 when x is 1. 134 00:13:00.340 --> 00:13:04.790 David Stephens, Prof: So this specifies the two different cases. This… 135 00:13:05.390 --> 00:13:08.650 David Stephens, Prof: It's just a parsimonious formula for writing this down. 136 00:13:08.760 --> 00:13:13.460 David Stephens, Prof: So what this is saying is the probability of being treated In the first… 137 00:13:13.820 --> 00:13:19.539 David Stephens, Prof: X subgroup, X equals 0, is different from the probability in the second. 138 00:13:19.780 --> 00:13:20.750 David Stephens, Prof: subgroup. 139 00:13:21.940 --> 00:13:24.740 David Stephens, Prof: So X and Z are related, they're not independent. 140 00:13:26.650 --> 00:13:30.010 David Stephens, Prof: The distribution of Z depends on X. 141 00:13:31.490 --> 00:13:39.550 David Stephens, Prof: Alright? So also from this, I can work out what the probability of being treated overall is just by using standard methods. Turns out to be some… 142 00:13:39.710 --> 00:13:43.570 David Stephens, Prof: a combination of the Q's and the P's, which I'll just call P. 143 00:13:45.050 --> 00:13:47.080 David Stephens, Prof: Alright, so that's the observational. 144 00:13:48.330 --> 00:13:49.430 David Stephens, Prof: calculation. 145 00:13:51.970 --> 00:14:04.070 David Stephens, Prof: And what we can do, and the full details are given in the appendix again, so if you click on that link there, it'll take you to the full detailed calculations. What you can show is that if you take the sample mean. 146 00:14:04.670 --> 00:14:10.470 David Stephens, Prof: From the treated group, And take the sample mean from the untreated group, that is. 147 00:14:11.950 --> 00:14:16.720 David Stephens, Prof: this calculation and this calculation. So this is the statistic we use. 148 00:14:18.580 --> 00:14:19.270 David Stephens, Prof: Right? 149 00:14:19.380 --> 00:14:24.439 David Stephens, Prof: You can do the analysis of this Using statistical means. 150 00:14:24.580 --> 00:14:29.590 David Stephens, Prof: And it turns out that this thing has got expectation equal to this. 151 00:14:32.740 --> 00:14:41.050 David Stephens, Prof: So… Critical thing is that… The average treatment effect, the true value, is 0.2. 152 00:14:43.890 --> 00:14:52.040 David Stephens, Prof: But the estimator has got expectation equal to this, dependent on the Q's, the P's, on the right-hand side. 153 00:14:53.160 --> 00:14:57.399 David Stephens, Prof: So it's 0.2 minus something, which is non-zero in general. 154 00:14:58.140 --> 00:15:02.580 David Stephens, Prof: So that means the unadjusted estimator that we've got here is biased. 155 00:15:05.690 --> 00:15:10.169 David Stephens, Prof: So what this tells us straight away is that if you go for the simplest possible setup. 156 00:15:11.020 --> 00:15:20.119 David Stephens, Prof: In this data generating mechanism, The simple difference in sample means between the treated and untreated groups 157 00:15:20.290 --> 00:15:21.789 David Stephens, Prof: Gives you the wrong answer. 158 00:15:23.270 --> 00:15:31.710 David Stephens, Prof: confounding plays an important role. We can see here the presence of these P's and the Q here. This tells you that confounding is 159 00:15:31.820 --> 00:15:38.289 David Stephens, Prof: taking you to the wrong value. So if you just were to take the difference in the sample means, you would get the wrong answer. 160 00:15:39.180 --> 00:15:42.249 David Stephens, Prof: All the calculations are laid out in the appendix. 161 00:15:43.970 --> 00:15:49.789 David Stephens, Prof: So it's not a surprise that that happens. Fundamentally, it comes back to the fact that 162 00:15:51.080 --> 00:15:55.190 David Stephens, Prof: There's a difference in the treatment probabilities. 163 00:15:55.640 --> 00:15:58.039 David Stephens, Prof: in the different categories of X. 164 00:16:01.170 --> 00:16:06.400 David Stephens, Prof: So the treated group, And the untreated group are no longer comparable. 165 00:16:06.930 --> 00:16:09.269 David Stephens, Prof: So a simple comparison of means… 166 00:16:09.670 --> 00:16:11.339 David Stephens, Prof: It's not gonna do the job. 167 00:16:14.780 --> 00:16:15.570 David Stephens, Prof: Okay. 168 00:16:16.250 --> 00:16:27.930 David Stephens, Prof: Yep, so the bias is caused by the fact that the two observed subsamples with Z equals 0 and Z equals 1, they're not directly comparable. They have different profiles in terms of X. 169 00:16:33.440 --> 00:16:41.229 David Stephens, Prof: We know, in fact, that if, by simple base theorem calculation, the probability that X is 1 in the Z equals 1 group. 170 00:16:41.380 --> 00:16:43.150 David Stephens, Prof: It's given by this. 171 00:16:43.570 --> 00:16:50.239 David Stephens, Prof: And the probability that X equals 1 in the Z equals 0 group is this, and in general, those two things are different. 172 00:16:54.760 --> 00:16:58.299 David Stephens, Prof: So, conditioning on z equals 1 and Z equals 0, in turn. 173 00:16:59.100 --> 00:17:02.530 David Stephens, Prof: leads to a different composition of X values in the two subsamples. 174 00:17:04.810 --> 00:17:07.719 David Stephens, Prof: Because of the association between Z and X, 175 00:17:09.660 --> 00:17:15.159 David Stephens, Prof: The treated individuals have a different X profile, Compared to the untreated individual. 176 00:17:16.609 --> 00:17:22.890 David Stephens, Prof: That means that the two groups are no longer directly comparable, and that's what leads to the bias. 177 00:17:23.250 --> 00:17:25.619 David Stephens, Prof: In the simple difference of means estimator. 178 00:17:28.810 --> 00:17:29.540 David Stephens, Prof: Okay? 179 00:17:35.290 --> 00:17:39.280 David Stephens, Prof: In passing, I want to point out the… the… 180 00:17:41.640 --> 00:17:46.730 David Stephens, Prof: The role of what are sometimes called instruments in this overall calculation. 181 00:17:48.220 --> 00:17:52.150 David Stephens, Prof: So, a term, a predictor exercise called an instrument. 182 00:17:52.670 --> 00:17:55.729 David Stephens, Prof: If it predicts Z, but doesn't predict Y. 183 00:17:56.060 --> 00:17:58.299 David Stephens, Prof: That is, it's a predictor of treatment. 184 00:17:58.960 --> 00:18:00.800 David Stephens, Prof: But it's not a predictor of the outcome. 185 00:18:01.050 --> 00:18:06.799 David Stephens, Prof: So in this case, we've got X, which is a confounder. It predicts both Y and Z. 186 00:18:07.420 --> 00:18:11.319 David Stephens, Prof: But Xi is only an instrument because it only predicts Z. 187 00:18:13.980 --> 00:18:16.009 David Stephens, Prof: What that means, of course, is that… 188 00:18:20.090 --> 00:18:26.889 David Stephens, Prof: Xi… is only related to Y, through Zed. 189 00:18:37.120 --> 00:18:40.169 David Stephens, Prof: So this… what we're gonna see as we go through is that 190 00:18:40.780 --> 00:18:45.680 David Stephens, Prof: We've talked about removing this arrow in order to look at the… 191 00:18:46.510 --> 00:18:56.379 David Stephens, Prof: the, experimental study, right? In the observational study, this arrow is still there, so there's an arrow going from X into Z. We've mainly focused on this. 192 00:18:56.630 --> 00:19:00.249 David Stephens, Prof: But it turns out that there's going to be a role for these instruments as well. 193 00:19:00.490 --> 00:19:05.000 David Stephens, Prof: The instrument is where the predictor predicts Z, but it doesn't predict the outcome. 194 00:19:07.850 --> 00:19:15.209 David Stephens, Prof: we'll see that the treatment of this X and the treatment of this X is somewhat different in the analyses. 195 00:19:17.560 --> 00:19:23.640 David Stephens, Prof: And you sort of get an idea from the previous calculation what, what's going to turn out here, in that 196 00:19:24.640 --> 00:19:29.310 David Stephens, Prof: if Xi… doesn't predict why. 197 00:19:30.170 --> 00:19:36.199 David Stephens, Prof: then it really doesn't matter that we're unbalanced in terms of… in terms of the XIs. 198 00:19:36.910 --> 00:19:44.619 David Stephens, Prof: just because Xi predicts Z, If it doesn't predict Y also, then we can essentially forget about it. 199 00:19:45.600 --> 00:19:49.539 David Stephens, Prof: We'll see that formalized as we go through the subsequent calculations. 200 00:19:54.650 --> 00:20:03.449 David Stephens, Prof: So, where we might see an instrument arising is in the case of a randomized study with incomplete. 201 00:20:03.570 --> 00:20:05.929 David Stephens, Prof: Concordance, or non-compliance. 202 00:20:06.550 --> 00:20:16.370 David Stephens, Prof: So, in a randomized study, it might be that Z records the treatment received, but that doesn't necessarily reflect the treatment assigned. 203 00:20:17.970 --> 00:20:18.730 David Stephens, Prof: Right. 204 00:20:19.090 --> 00:20:22.969 David Stephens, Prof: So it might be that there's a random coin flip. 205 00:20:23.200 --> 00:20:28.489 David Stephens, Prof: To assign somebody to a treatment group, But they, in reality. 206 00:20:28.820 --> 00:20:30.889 David Stephens, Prof: Turn out to take the other treatment. 207 00:20:33.140 --> 00:20:40.550 David Stephens, Prof: So, in that case, the assigned… Treatment. 208 00:20:40.690 --> 00:20:44.870 David Stephens, Prof: And the received treatment are not exactly identical. 209 00:20:46.920 --> 00:20:52.529 David Stephens, Prof: So in that case, Xi is an instrument, because it does predict Z, presumably. 210 00:20:52.930 --> 00:20:57.459 David Stephens, Prof: It does influence some people to take the treatment that they were assigned to, but it doesn't predict it perfectly. 211 00:20:59.830 --> 00:21:03.910 David Stephens, Prof: In this case, They're not… instruments are not confounders. 212 00:21:04.350 --> 00:21:13.460 David Stephens, Prof: And, what we can see pretty quickly is that they are going to have a different impact. 213 00:21:14.340 --> 00:21:15.310 David Stephens, Prof: on. 214 00:21:17.990 --> 00:21:19.950 David Stephens, Prof: The performance of estimators. 215 00:21:21.360 --> 00:21:28.199 David Stephens, Prof: So let's say we're in this instrument case, so we've got the model for Y given X and Z 216 00:21:28.800 --> 00:21:31.919 David Stephens, Prof: isn't… a function of X. 217 00:21:34.000 --> 00:21:37.209 David Stephens, Prof: So we've still got the Z influences Y. 218 00:21:38.020 --> 00:21:40.260 David Stephens, Prof: But X no longer influences Y. 219 00:21:42.330 --> 00:21:44.910 David Stephens, Prof: Okay, so this situation is this. 220 00:21:45.420 --> 00:21:49.880 David Stephens, Prof: But we've still got the model that says that treatment depends on X. 221 00:21:55.270 --> 00:22:00.210 David Stephens, Prof: Well, we can very quickly see that if we take the expectation over 222 00:22:00.500 --> 00:22:03.329 David Stephens, Prof: this model, for a fixed Z, 223 00:22:03.490 --> 00:22:10.430 David Stephens, Prof: We're always going to get the same answer, irrespective of X. It's always going to be 0.2 plus .2 times Z. 224 00:22:10.890 --> 00:22:12.039 David Stephens, Prof: Little Red said. 225 00:22:13.880 --> 00:22:20.139 David Stephens, Prof: So the fact that Z and X are related, Turns out to be irrelevant. 226 00:22:22.740 --> 00:22:29.169 David Stephens, Prof: So, for instruments, Even though there can be a ZX relationship. 227 00:22:30.430 --> 00:22:36.980 David Stephens, Prof: Provided the instrument doesn't influence the outcome, which it doesn't by definition, then… 228 00:22:37.500 --> 00:22:42.880 David Stephens, Prof: It doesn't matter that we don't have The same distribution of… 229 00:22:44.080 --> 00:22:47.129 David Stephens, Prof: The instrument in the treated and untreated groups. 230 00:22:49.370 --> 00:22:58.300 David Stephens, Prof: So, back to the DAG. The fact that there's an arrow here, but no arrow here, means that we don't need to worry about the presence of this X. 231 00:22:59.090 --> 00:23:01.750 David Stephens, Prof: We'll come back to this in some simulations later on. 232 00:23:07.690 --> 00:23:16.820 David Stephens, Prof: One thing that the causal analysis that I've been sort of leading up to really does depend on, though, is the assumption of no unmeasured confounding. 233 00:23:18.010 --> 00:23:21.319 David Stephens, Prof: That is, the measured predictors X that we have. 234 00:23:21.630 --> 00:23:23.779 David Stephens, Prof: Or maybe a subset of them, at least. 235 00:23:24.350 --> 00:23:34.809 David Stephens, Prof: are all the variables that confound the effect of Z and Y. So the variables that sit in that X spot in the confounding graph are all available to us for analysis. 236 00:23:35.200 --> 00:23:37.339 David Stephens, Prof: There are no other confers. 237 00:23:39.950 --> 00:23:41.180 David Stephens, Prof: So that's, 238 00:23:44.160 --> 00:23:52.239 David Stephens, Prof: there's the DAG that represents that, right? So, an unmeasured confounder is something, so I put it here in this ghostly dotted form. 239 00:23:53.010 --> 00:23:59.260 David Stephens, Prof: Right? So we observe the X's, and we treat X as a confounder of the relationship between Y and Z. 240 00:23:59.400 --> 00:24:02.889 David Stephens, Prof: Unmeasured confounders might sit here, and these dotted lines 241 00:24:03.220 --> 00:24:10.179 David Stephens, Prof: would be there if there was a variable U that was there in the background, but we don't have access to in the study. 242 00:24:16.240 --> 00:24:20.200 David Stephens, Prof: So we have to assume In the analysis that I'm going to talk about. 243 00:24:20.550 --> 00:24:24.480 David Stephens, Prof: That there are no such unmeasured ews. 244 00:24:26.290 --> 00:24:34.480 David Stephens, Prof: So you could argue, and people do argue, that this is a strong assumption, possibly it's unrealistic. 245 00:24:38.760 --> 00:24:45.260 David Stephens, Prof: But I would point out that it's a common assumption that's made in almost all statistical analyses, based on regression. 246 00:24:52.330 --> 00:25:00.219 David Stephens, Prof: So it's the kind of thing that sort of slips under the radar in many types of analysis, but that's actually necessary to be assumed. 247 00:25:01.170 --> 00:25:07.710 David Stephens, Prof: The other argument I would make is that if you work with subject matter experts. 248 00:25:08.070 --> 00:25:11.239 David Stephens, Prof: Then they should be able to identify at the start of the study 249 00:25:11.400 --> 00:25:14.029 David Stephens, Prof: The appropriate set of confounding variables. 250 00:25:15.840 --> 00:25:19.669 David Stephens, Prof: Those variables that influence are likely to influence treatment and outcome. 251 00:25:21.260 --> 00:25:25.800 David Stephens, Prof: And then, once they've made that commitment to collect those variables, 252 00:25:26.610 --> 00:25:30.990 David Stephens, Prof: Really, the focus is on how they should be used in the analysis. 253 00:25:32.070 --> 00:25:40.250 David Stephens, Prof: Acknowledging the fact that although that assessment might be perfect… might be imperfect, the conclusions that you make are contingent on that assumption. 254 00:25:44.470 --> 00:25:46.119 David Stephens, Prof: If you did want to… 255 00:25:46.350 --> 00:25:55.230 David Stephens, Prof: try to acknowledge the presence of unmatched commanders, and you… what people have done is to use sensitivity analyses based on proposed 256 00:25:55.650 --> 00:26:01.530 David Stephens, Prof: Unmeasured competitors, and see what influence that has on the, on the resulting estimates. 257 00:26:05.950 --> 00:26:09.360 David Stephens, Prof: Okay, so… 258 00:26:15.690 --> 00:26:17.649 David Stephens, Prof: When we talked about the, 259 00:26:18.830 --> 00:26:22.880 David Stephens, Prof: Instrumental… sorry, not instrumental, the experimental study earlier on. 260 00:26:23.210 --> 00:26:25.639 David Stephens, Prof: And built those sample mean estimators. 261 00:26:25.870 --> 00:26:32.230 David Stephens, Prof: Just averaging the data observed without any other modeling, just taking simple sample averages. 262 00:26:33.170 --> 00:26:38.560 David Stephens, Prof: That, we can think of as what's called a non-parametric, or model-free analysis. 263 00:26:40.930 --> 00:26:44.580 David Stephens, Prof: That is, we acknowledge the presence of this joint distribution. 264 00:26:45.150 --> 00:26:53.639 David Stephens, Prof: that generates the data, but we never have to do any actual modeling of it. We don't have to specify a normal distribution, or a particular form of the mean, or whatever. 265 00:26:57.370 --> 00:27:14.070 David Stephens, Prof: That can be somewhat limiting, as we'll see, so it's more common, perhaps, to make, semi-parametric specifications, so that means, effectively, we end up building means for the expectation, so something like a regression model. 266 00:27:14.300 --> 00:27:15.930 David Stephens, Prof: for Y given X. 267 00:27:17.900 --> 00:27:26.519 David Stephens, Prof: So we model the mean, but then we don't make assumptions about the distributions concerned, so we don't assume normality of the residual errors or something like that. 268 00:27:27.470 --> 00:27:37.099 David Stephens, Prof: So we're very quickly going to get drawn down the route of this semi-parametric estimation when we want to perform causal analyses, because we want to acknowledge 269 00:27:37.380 --> 00:27:40.709 David Stephens, Prof: The roles of the different variables in the model. 270 00:27:41.170 --> 00:27:48.550 David Stephens, Prof: And… Although, ultimately, a non-parametric analysis, a model-free analysis, would be ideal. 271 00:27:49.120 --> 00:27:55.770 David Stephens, Prof: We often are limited in the data that we have access to, the amount of data in particular, that makes this not really feasible. 272 00:27:56.040 --> 00:28:10.869 David Stephens, Prof: So, as in most of the areas of statistics, we make some parametric compromises, that is, we pick models for these things, and then we work assuming that that conditional specification, that conditional mean model is correctly specified. 273 00:28:13.420 --> 00:28:20.229 David Stephens, Prof: So, for example, we might use a regression approach that said that this conditional expectation is equal to some function mu. 274 00:28:20.590 --> 00:28:22.380 David Stephens, Prof: Usually it's parametric. 275 00:28:22.660 --> 00:28:25.879 David Stephens, Prof: So we have some parameters theta that we want to estimate. 276 00:28:27.320 --> 00:28:28.040 David Stephens, Prof: Okay? 277 00:28:35.560 --> 00:28:45.369 David Stephens, Prof: And… One of the critical things about the no-measure-confounders assumption Is that it allows us 278 00:28:46.030 --> 00:28:53.510 David Stephens, Prof: To move from this hypothetical world, of… expected counterfactuals. 279 00:28:53.840 --> 00:28:55.550 David Stephens, Prof: To the observational world. 280 00:28:56.640 --> 00:29:00.239 David Stephens, Prof: That is… The structural model that we build. 281 00:29:00.380 --> 00:29:02.749 David Stephens, Prof: Is the same as the observed data model. 282 00:29:05.230 --> 00:29:18.620 David Stephens, Prof: So, if we make… No emotional confander's assumption, then moving from this to this… Is a legitimate step. 283 00:29:19.300 --> 00:29:32.920 David Stephens, Prof: And if you remember back to the structural equation modeling I was talking about before, then that was, I think a reasonable collection of assumptions there. We were assuming that the variables on the right-hand side were generating the variables on the left-hand side. 284 00:29:33.350 --> 00:29:39.600 David Stephens, Prof: All we're assuming here is that that… structural model for Y is… 285 00:29:40.360 --> 00:29:54.509 David Stephens, Prof: corresponding to the observational model. In particular, what that means is that it doesn't matter how Z and X arise jointly, once they get together to cause the variation in Y, it's the same in the experimental world as in the observational world. 286 00:29:56.910 --> 00:29:59.400 David Stephens, Prof: The only difference between the two DAGs 287 00:29:59.550 --> 00:30:06.030 David Stephens, Prof: for the experimental and the observational is that in the observational world, the arrow between X and Z is removed. 288 00:30:06.660 --> 00:30:11.759 David Stephens, Prof: But the arrows going from X and Z into Y remain unchanged. 289 00:30:12.140 --> 00:30:16.329 David Stephens, Prof: That's essentially what this assumption is allowing us to do. 290 00:30:22.890 --> 00:30:25.460 David Stephens, Prof: So, one important first message is that 291 00:30:27.370 --> 00:30:30.250 David Stephens, Prof: Under correct specification of this model. 292 00:30:32.700 --> 00:30:35.930 David Stephens, Prof: We don't need special causal methods. 293 00:30:36.150 --> 00:30:42.269 David Stephens, Prof: We don't need… To think about moving from one world to the other world. 294 00:30:42.550 --> 00:30:45.020 David Stephens, Prof: We don't have to think about the propensity score. 295 00:30:45.240 --> 00:30:50.059 David Stephens, Prof: If this model is correctly specified, Through a regression, say. 296 00:30:50.470 --> 00:30:58.620 David Stephens, Prof: Then, no special techniques are needed, we could just use the regression. We regress Y on X and Z, and interpret the mean model. 297 00:31:03.770 --> 00:31:10.920 David Stephens, Prof: Even if the observational design, even if we have observational data. 298 00:31:11.790 --> 00:31:14.870 David Stephens, Prof: Then regression can solve all our problems for us. 299 00:31:16.390 --> 00:31:24.740 David Stephens, Prof: Under the key assumption that this model, this conditional mean model, the regression function that you use, is correctly specified. 300 00:31:26.950 --> 00:31:32.450 David Stephens, Prof: Now, we know from regression, however, from regression theory. 301 00:31:32.610 --> 00:31:39.680 David Stephens, Prof: standard, again, undergraduate regression theory, that If that model is misspecified. 302 00:31:41.080 --> 00:31:48.480 David Stephens, Prof: If the regression function is misclassified, then that can lead to biased estimation of the coefficients, in some circumstances. 303 00:31:49.890 --> 00:31:54.590 David Stephens, Prof: What we've seen today, really, backs that up through a confounding argument. 304 00:31:58.000 --> 00:32:01.220 David Stephens, Prof: But if we have correct specification, regression does the job. 305 00:32:04.770 --> 00:32:09.719 David Stephens, Prof: To estimate the average potential outcome, all we need to do is to get the fitted values from the model. 306 00:32:11.090 --> 00:32:12.280 David Stephens, Prof: And average them. 307 00:32:12.390 --> 00:32:14.150 David Stephens, Prof: for that given target Z. 308 00:32:17.730 --> 00:32:20.759 David Stephens, Prof: That's essentially what we saw earlier on in the structural model. 309 00:32:21.260 --> 00:32:26.190 David Stephens, Prof: We plug in the little red Z into the function mu that we've specified back here. 310 00:32:27.410 --> 00:32:31.770 David Stephens, Prof: Plug in the little red Z, Compute the outcome. 311 00:32:32.550 --> 00:32:33.530 David Stephens, Prof: Maine? 312 00:32:33.930 --> 00:32:35.690 David Stephens, Prof: Average that over the data. 313 00:32:37.990 --> 00:32:40.720 David Stephens, Prof: If this regression model is correctly specified. 314 00:32:41.120 --> 00:32:47.089 David Stephens, Prof: This averages the average… this estimates the average potential outcome completely correctly. 315 00:32:48.410 --> 00:32:49.680 David Stephens, Prof: Our job is done. 316 00:32:55.990 --> 00:32:57.629 David Stephens, Prof: How does that work in the… 317 00:32:58.340 --> 00:33:04.810 David Stephens, Prof: In the, in the, in the world of DAGs and causal graphs and, 318 00:33:08.960 --> 00:33:11.569 David Stephens, Prof: Observational setups, confounding setups. 319 00:33:11.730 --> 00:33:13.810 David Stephens, Prof: Well, it does this. 320 00:33:15.220 --> 00:33:18.060 David Stephens, Prof: We're interested in the connection between Z and Y. 321 00:33:18.370 --> 00:33:23.989 David Stephens, Prof: We acknowledge that X is present, there is a path going from Z to X to Y, 322 00:33:24.690 --> 00:33:32.730 David Stephens, Prof: That path is open, and therefore it's potentially Problematic, it induces confounding, Produces bias. 323 00:33:33.350 --> 00:33:36.120 David Stephens, Prof: So what we need to do, simply, is to block it. 324 00:33:36.690 --> 00:33:39.559 David Stephens, Prof: And we block it by conditioning on X. 325 00:33:41.940 --> 00:33:45.010 David Stephens, Prof: If we block the path at X by conditioning. 326 00:33:45.350 --> 00:33:56.449 David Stephens, Prof: then there is no longer a path from Z to Y that goes through X, because that path is now blocked. The only path from Z to Y is down the direct path. 327 00:33:57.420 --> 00:34:00.239 David Stephens, Prof: So our causal conclusions will be correct. 328 00:34:08.370 --> 00:34:12.880 David Stephens, Prof: So the key thing is blocking this so-called backdoor path. 329 00:34:14.290 --> 00:34:18.890 David Stephens, Prof: the path that allows the effect of Z to sneak out. 330 00:34:19.300 --> 00:34:20.669 David Stephens, Prof: to influence Y. 331 00:34:21.070 --> 00:34:23.979 David Stephens, Prof: Okay, we're blocking that path by conditioning on X. 332 00:34:27.969 --> 00:34:35.330 David Stephens, Prof: And, again, we can think of that as just being as simple as including the regression, in the regression, the X variable. 333 00:34:45.340 --> 00:34:47.920 David Stephens, Prof: Correct specification is a strong assumption, though. 334 00:34:50.679 --> 00:34:55.320 David Stephens, Prof: So the third challenge is that correct specification can't be guaranteed. 335 00:34:55.860 --> 00:34:56.989 David Stephens, Prof: That is. 336 00:34:57.170 --> 00:35:01.969 David Stephens, Prof: In order for the previous calculation to work, we need to know exactly what the mu function is. 337 00:35:02.170 --> 00:35:07.590 David Stephens, Prof: we need to be able to say that the expected value of Y conditional on X and Z is captured by mu. 338 00:35:09.390 --> 00:35:12.280 David Stephens, Prof: With parameters that are known in there. 339 00:35:14.120 --> 00:35:16.190 David Stephens, Prof: But we can't guarantee that that holds. 340 00:35:16.810 --> 00:35:18.100 David Stephens, Prof: In a real setting. 341 00:35:19.470 --> 00:35:24.560 David Stephens, Prof: We might use the wrong model. We might use M instead of mu, so M is a different model. 342 00:35:26.490 --> 00:35:32.099 David Stephens, Prof: In which case… This logic doesn't work. We get the wrong answer again. 343 00:35:39.190 --> 00:35:42.240 David Stephens, Prof: So, because correct specification can't be guaranteed. 344 00:35:42.720 --> 00:35:45.670 David Stephens, Prof: We need new methods that allow us to 345 00:35:46.800 --> 00:35:51.650 David Stephens, Prof: Still deploy the observational data to get at the causal quantities of interest. 346 00:35:55.890 --> 00:36:01.090 David Stephens, Prof: So, to recap, in the experimental world, direct comparison of the two group means works. 347 00:36:01.480 --> 00:36:02.959 David Stephens, Prof: Gives you the right answer. 348 00:36:04.070 --> 00:36:05.860 David Stephens, Prof: In the observational world. 349 00:36:06.330 --> 00:36:12.129 David Stephens, Prof: If we can correctly specify the conditional mean of Y given X and Z, the correct regression model. 350 00:36:12.760 --> 00:36:17.480 David Stephens, Prof: Then we can use that method to estimate the causal quantities and the average treatment effect. 351 00:36:19.210 --> 00:36:23.249 David Stephens, Prof: If we can't guarantee a correct specification, though, in the observational world. 352 00:36:23.740 --> 00:36:28.380 David Stephens, Prof: We need to work a bit harder, and that's what the Propensity score allows us to do. 353 00:36:30.970 --> 00:36:33.860 David Stephens, Prof: Okay, so that leads us on to the construction. 354 00:36:34.450 --> 00:36:36.429 David Stephens, Prof: And use of the Propensity score. 355 00:36:37.680 --> 00:36:43.739 David Stephens, Prof: So the whole first part, the first section there is really meant to justify why we look at the propensity score. 356 00:36:43.850 --> 00:36:50.729 David Stephens, Prof: It's to allow previously uncomparable things, or incomparable things, to become comparable. 357 00:36:54.880 --> 00:36:59.100 David Stephens, Prof: So… Let's think about the construction of the propensity score. 358 00:37:01.120 --> 00:37:02.180 David Stephens, Prof: So… 359 00:37:04.600 --> 00:37:15.939 David Stephens, Prof: motivate the Propensity score, what I'm going to do is to think back to the randomized study setting, and to identify the key aspects of the randomized study that were useful to us in estimating causal quantities. 360 00:37:16.720 --> 00:37:22.919 David Stephens, Prof: Here's the setup again. We obtain a random sample of size n from the target population, measure their X values. 361 00:37:24.190 --> 00:37:25.890 David Stephens, Prof: According to some random… 362 00:37:26.580 --> 00:37:33.350 David Stephens, Prof: According to some random assignment mechanism, we intervene to assign Treatment Z to individuals, and then measure the outcome. 363 00:37:34.050 --> 00:37:38.129 David Stephens, Prof: The link between X and Z is broken because of the random allocation. 364 00:37:39.710 --> 00:37:46.860 David Stephens, Prof: And if we did that, then direct comparison of the sample means in the treated and untreated groups gave us a perfectly good answer. 365 00:37:48.970 --> 00:37:52.140 David Stephens, Prof: Equation 1, that's where we took the actual sample means. 366 00:37:52.330 --> 00:38:03.099 David Stephens, Prof: In the treated and untreated groups took the difference. Equation 2, that was where we substituted in the denominator the expected sample sizes, and not the actual sample size. 367 00:38:03.240 --> 00:38:10.910 David Stephens, Prof: Both of them are actually perfectly good, unbiased estimators of the truth under the… experimental setting. 368 00:38:14.300 --> 00:38:16.730 David Stephens, Prof: So the important feature of the randomized study 369 00:38:17.220 --> 00:38:23.709 David Stephens, Prof: is that we have this independence between X and Z. That is, for all confounders, all predictors. 370 00:38:23.830 --> 00:38:29.069 David Stephens, Prof: The distribution of X given Z Doesn't depend on Z. 371 00:38:29.840 --> 00:38:37.279 David Stephens, Prof: That is, the distribution of X for Z equals 1 is the same as the distribution for X given Z for Z equals 0. 372 00:38:38.060 --> 00:38:41.120 David Stephens, Prof: that holds… uniformly enacts. 373 00:38:44.910 --> 00:38:49.860 David Stephens, Prof: In the case of a binary confounder, what that means is that the probability that X equals 1 374 00:38:50.610 --> 00:38:52.400 David Stephens, Prof: Given that Z equals 1, 375 00:38:53.220 --> 00:38:59.329 David Stephens, Prof: is equal to the probability that Z equals 1 given X, X equals 1 given Z equals 0. 376 00:39:00.360 --> 00:39:04.049 David Stephens, Prof: Right? This is a feature of the randomized study. 377 00:39:05.700 --> 00:39:11.990 David Stephens, Prof: In this case, the distribution of X is what we can term as balanced across the two exposure groups. 378 00:39:12.120 --> 00:39:15.230 David Stephens, Prof: There's no difference in the distribution of X's here. 379 00:39:16.310 --> 00:39:17.330 David Stephens, Prof: And here. 380 00:39:18.860 --> 00:39:29.609 David Stephens, Prof: Right, so you look at the treatment group, you look at the non… look at the treated group, look at the untreated group, look at their X characteristics, the profiles are the same. 381 00:39:31.510 --> 00:39:34.370 David Stephens, Prof: That's induced by the randomization. 382 00:39:35.700 --> 00:39:38.669 David Stephens, Prof: And in probability terms, we say that X and Z are independent. 383 00:39:39.550 --> 00:39:45.929 David Stephens, Prof: Remember, this is where we want our inference, right? We want inference in this world. 384 00:39:46.170 --> 00:39:51.070 David Stephens, Prof: Right? In the world where X and Z are independent. However, In the observational… 385 00:39:51.870 --> 00:39:54.860 David Stephens, Prof: setting. We don't have that independence. 386 00:39:58.000 --> 00:40:03.120 David Stephens, Prof: In the observational world, There's a possibility that two exposure groups are… 387 00:40:03.620 --> 00:40:12.010 David Stephens, Prof: systematically not balanced, that is, the distribution of the X's for the Z equals 1 group is different from the distribution 388 00:40:13.340 --> 00:40:15.699 David Stephens, Prof: of the X is for the Z equals 0 group. 389 00:40:18.070 --> 00:40:21.370 David Stephens, Prof: In the binary case, these two conditional probabilities are not equal. 390 00:40:22.090 --> 00:40:27.129 David Stephens, Prof: The X profiles are fundamentally different in the two groups. 391 00:40:28.090 --> 00:40:29.350 David Stephens, Prof: Indexed by treatment. 392 00:40:31.820 --> 00:40:42.049 David Stephens, Prof: So that lack of balance on its own, when combined with the fact that X influences Y, is what leads to confounding, it's what leads to bias in the estimators. 393 00:40:44.040 --> 00:40:47.129 David Stephens, Prof: It fundamentally means that the two groups are not. 394 00:40:47.270 --> 00:40:48.250 David Stephens, Prof: Comparable. 395 00:40:52.950 --> 00:40:58.849 David Stephens, Prof: So that's the problem we're trying to solve. And we're going to build up to a solution based on the propensity score. 396 00:41:07.260 --> 00:41:11.829 David Stephens, Prof: Our requirement for a full comparison is that balance needs to hold. 397 00:41:13.480 --> 00:41:23.269 David Stephens, Prof: And in the observational study, we have no guarantee that it does. But it might be that local balance holds within certain strata of the sample. Let's say, let's take that optimistic view. 398 00:41:24.510 --> 00:41:30.849 David Stephens, Prof: So let S be some identified stratum in the sample space for X's, that is, a particular collection of X values. 399 00:41:32.700 --> 00:41:34.670 David Stephens, Prof: a portion of the X space. 400 00:41:37.280 --> 00:41:43.089 David Stephens, Prof: And suppose that… In that stratum, we do have balance. 401 00:41:43.210 --> 00:41:49.549 David Stephens, Prof: That is, if we identify a particular subset of the X's, In that subset. 402 00:41:55.590 --> 00:42:02.010 David Stephens, Prof: Or, rather, for that subset, For that portion of the space. 403 00:42:02.380 --> 00:42:12.520 David Stephens, Prof: There is no distinction between the treated individuals and the untreated individuals. That is, in that local part of the X space, you can't tell the treated from the untreated. 404 00:42:14.450 --> 00:42:27.600 David Stephens, Prof: That's what this notation means, basically, here. So it's… it's sort of like a local independence. It's not technically independence, but it's sort of saying, if the X world was boiled down into a smaller subset. 405 00:42:28.870 --> 00:42:30.470 David Stephens, Prof: We could find this. 406 00:42:30.670 --> 00:42:31.790 David Stephens, Prof: Balance. 407 00:42:33.560 --> 00:42:38.430 David Stephens, Prof: So, that's not going to hold in general, if X and Z are dependent. 408 00:42:38.540 --> 00:42:39.640 David Stephens, Prof: By definition. 409 00:42:39.800 --> 00:42:44.700 David Stephens, Prof: But it might be that in some parts of the X space, We can detect this independence. 410 00:42:47.690 --> 00:42:50.320 David Stephens, Prof: So, for the individuals in that stratum. 411 00:42:51.210 --> 00:42:55.770 David Stephens, Prof: We can do a direct comparison between treated and untreated individuals, because 412 00:42:56.420 --> 00:42:58.899 David Stephens, Prof: The profile of the axes is the same. 413 00:42:59.630 --> 00:43:01.689 David Stephens, Prof: In the treated and untreated groups. 414 00:43:04.230 --> 00:43:11.830 David Stephens, Prof: So then what we might be able to do is to say, well, we'll do causal inference, we'll do the average treatment effect, but restricted to S. 415 00:43:17.000 --> 00:43:21.980 David Stephens, Prof: Okay, so that's the idea, that's the first idea. We might be able to find local balance 416 00:43:22.180 --> 00:43:24.319 David Stephens, Prof: In a subset of the X space. 417 00:43:28.480 --> 00:43:35.299 David Stephens, Prof: And a trivial, extreme case to think about is, suppose that the confounder only took a single value X0. 418 00:43:37.040 --> 00:43:38.500 David Stephens, Prof: In that case… 419 00:43:40.440 --> 00:43:46.880 David Stephens, Prof: the portion of the X space is in the entirety of the space, it's that particular X value, and in that case. 420 00:43:47.280 --> 00:43:55.790 David Stephens, Prof: there's clearly no systematic differences between the X's in the treated and untreated groups, whatever they are, right? Because X always takes the same value. 421 00:43:57.670 --> 00:44:01.069 David Stephens, Prof: So it has to take the same value in treated and untreated individuals. 422 00:44:01.540 --> 00:44:04.110 David Stephens, Prof: So the profile is the same, we do have balance. 423 00:44:04.810 --> 00:44:10.580 David Stephens, Prof: Now, that's an extreme case, right? We're not gonna have this case, and… 424 00:44:11.810 --> 00:44:17.639 David Stephens, Prof: I mean, it's not really useful to look at such a confounder and look at its impact on the outcome. 425 00:44:18.960 --> 00:44:24.480 David Stephens, Prof: But it tries to convey the idea that we can get this, perhaps in some settings, local. 426 00:44:24.710 --> 00:44:25.970 David Stephens, Prof: independence. 427 00:44:31.080 --> 00:44:36.440 David Stephens, Prof: If all of our confounders were discrete, Then… 428 00:44:37.000 --> 00:44:39.450 David Stephens, Prof: We do have a way to achieve balance. 429 00:44:40.770 --> 00:44:45.840 David Stephens, Prof: What we could do is to define strata where the X values are precisely matched. 430 00:44:48.550 --> 00:44:51.230 David Stephens, Prof: So, if the X's are discrete, that is, take 431 00:44:51.610 --> 00:44:59.279 David Stephens, Prof: Whole number values, let's say they're integers between 1 and 10, Then we could look at… 432 00:44:59.800 --> 00:45:04.730 David Stephens, Prof: All of the 1s, all of the 2s, all of the 3s, dot dot dot, all of the 10s. 433 00:45:05.160 --> 00:45:11.640 David Stephens, Prof: And look at the treated individuals and the untreated individuals within each of those defined strata. 434 00:45:13.210 --> 00:45:16.590 David Stephens, Prof: Right? So the X equals 1 group defines a stratum. 435 00:45:16.860 --> 00:45:19.380 David Stephens, Prof: X equals 2 group defines a stratum. 436 00:45:22.130 --> 00:45:27.419 David Stephens, Prof: And within those stratum, We may have treated and untreated individuals. 437 00:45:28.340 --> 00:45:30.120 David Stephens, Prof: So what we can do is… 438 00:45:30.760 --> 00:45:33.320 David Stephens, Prof: Within each stratum, do a comparison. 439 00:45:34.470 --> 00:45:38.650 David Stephens, Prof: And then, at the end, Combine across all strata. 440 00:45:39.710 --> 00:45:42.209 David Stephens, Prof: For some… for some overall estimate. 441 00:45:43.090 --> 00:45:51.940 David Stephens, Prof: Right? So we define strata where the axes are precisely matched, we compare the outcomes for treated and untreated individuals within those strata. 442 00:45:52.500 --> 00:46:00.009 David Stephens, Prof: Perform this comparison For multiple strata, and then combine them. 443 00:46:05.990 --> 00:46:11.760 David Stephens, Prof: So this gives us a way of constructing balance, at least locally. 444 00:46:15.340 --> 00:46:22.440 David Stephens, Prof: So for discrete confounders, We're in a position to manufacture balance. 445 00:46:23.100 --> 00:46:23.880 David Stephens, Prof: Actually. 446 00:46:24.920 --> 00:46:26.689 David Stephens, Prof: If we've got enough data. 447 00:46:27.970 --> 00:46:33.230 David Stephens, Prof: If we've got enough treated and untreated individuals within each stratum defined by the X values. 448 00:46:33.620 --> 00:46:39.530 David Stephens, Prof: Then we can do those local comparisons, and then combine them into a global comparison. 449 00:46:41.630 --> 00:46:43.740 David Stephens, Prof: Here's the sort of formula for doing that. 450 00:46:44.210 --> 00:46:56.650 David Stephens, Prof: So we say that the average treatment effect is the average of the average treatment effects within each stratum, weighted by the probability of lying within that stratum. 451 00:46:58.350 --> 00:47:01.800 David Stephens, Prof: This is just an iterated expectation calculation, formally. 452 00:47:06.470 --> 00:47:08.270 David Stephens, Prof: The idea is that we can… 453 00:47:09.530 --> 00:47:14.330 David Stephens, Prof: Estimate this just by counting the proportion of X's that lie in that stratum. 454 00:47:14.810 --> 00:47:20.499 David Stephens, Prof: And then here, we just do a direct comparison of means between the treated and untreated individuals. 455 00:47:21.610 --> 00:47:28.110 David Stephens, Prof: This overall calculation allows us then to return an estimate of the overall average treatment effect. 456 00:47:30.530 --> 00:47:36.480 David Stephens, Prof: So that's the first method that allows us to overcome the observational Modeling. 457 00:47:36.760 --> 00:47:40.849 David Stephens, Prof: So, sorry, the observational… Data collection. 458 00:47:42.760 --> 00:47:46.089 David Stephens, Prof: We can get a causal comparison in this setting. 459 00:47:46.430 --> 00:47:51.639 David Stephens, Prof: By using this particular iterated expectation calculation. 460 00:47:54.140 --> 00:48:00.019 David Stephens, Prof: This could just be a simple comparison of means. It could also be that we have some modeling in here. 461 00:48:00.170 --> 00:48:01.280 David Stephens, Prof: We could do that. 462 00:48:01.640 --> 00:48:06.139 David Stephens, Prof: This is just estimated as the proportion of X's that fall into the stratum. 463 00:48:07.020 --> 00:48:15.380 David Stephens, Prof: So everybody's got the idea of what we're trying to do here, right? So the X's are just a discrete set of confounders, just taking discrete values. 464 00:48:16.150 --> 00:48:24.709 David Stephens, Prof: We match on the values of the X's, look at the treated and untreated individual within each matched stratum. 465 00:48:25.630 --> 00:48:31.249 David Stephens, Prof: And then combine the overall comparisons into a global comparison. 466 00:48:35.900 --> 00:48:36.740 David Stephens, Prof: Okay. 467 00:48:40.330 --> 00:48:42.250 David Stephens, Prof: Of course, that's a bit too simplistic. 468 00:48:42.590 --> 00:48:45.549 David Stephens, Prof: Because we're often faced with continuous confounders. 469 00:48:46.700 --> 00:48:50.920 David Stephens, Prof: Right? So matching on the value of a continuous confounder. 470 00:48:51.880 --> 00:48:58.430 David Stephens, Prof: age, measured to the day, or whatever. In a finite data set, that's gonna be… Hard. 471 00:49:00.230 --> 00:49:06.480 David Stephens, Prof: But we can mimic the calculation by saying, well, okay, let's assume that we've broken down into K categories. 472 00:49:06.640 --> 00:49:11.589 David Stephens, Prof: And we're going to assume within each stratum, We have a match. 473 00:49:14.210 --> 00:49:17.120 David Stephens, Prof: Alright, so we break down the stratum. 474 00:49:17.550 --> 00:49:20.909 David Stephens, Prof: Sorry, break down the overall sample space into these strata. 475 00:49:23.710 --> 00:49:26.579 David Stephens, Prof: We do a local comparison within each stratum. 476 00:49:27.060 --> 00:49:28.539 David Stephens, Prof: And then we combine. 477 00:49:34.220 --> 00:49:40.960 David Stephens, Prof: So, that also has a… has a mechanism for breaking the… confounding issue. 478 00:49:42.210 --> 00:49:47.270 David Stephens, Prof: Now it becomes a bit more subject to correct specification assumptions. For example. 479 00:49:47.670 --> 00:49:50.319 David Stephens, Prof: We have to model this, typically. 480 00:49:50.700 --> 00:49:53.259 David Stephens, Prof: So that means that a local regression 481 00:49:53.400 --> 00:49:56.230 David Stephens, Prof: Within this stratum needs to be correctly specified. 482 00:49:59.570 --> 00:50:04.449 David Stephens, Prof: But again, in principle, we've manufactured this balance by matching within the stratum. 483 00:50:07.410 --> 00:50:11.730 David Stephens, Prof: Making things directly comparable, when previously they were not. 484 00:50:16.540 --> 00:50:25.649 David Stephens, Prof: And again, in this case, only in the previous case, we're looking at a causal comparison, but we're restricted to the region covered by the strata, so the union of these… 485 00:50:27.070 --> 00:50:34.269 David Stephens, Prof: sub-portions of the X space delimit where we're looking at the average treatment effect. 486 00:50:40.070 --> 00:50:43.760 David Stephens, Prof: However, there are… more complicated problems. 487 00:50:46.450 --> 00:50:56.619 David Stephens, Prof: The first one I've mentioned a bit, so exact matching with continuous variables is not really very straightforward, so you need to think about, well, what does a match mean if you've got a continuous covariate? 488 00:50:59.880 --> 00:51:06.940 David Stephens, Prof: So the whole literature on matching is actually quite well developed in statistics. 489 00:51:07.130 --> 00:51:16.100 David Stephens, Prof: outside of the causal setting. We'll see later on that matching Is a perfectly good causal inference method. 490 00:51:17.510 --> 00:51:24.920 David Stephens, Prof: Okay? So this… Matching, which induces the balance that we need to make things 491 00:51:25.110 --> 00:51:32.790 David Stephens, Prof: comparable when they previously were not comparable. That is, a well-established method that we'll deploy. 492 00:51:33.570 --> 00:51:36.229 David Stephens, Prof: The other complication, though, is that 493 00:51:36.460 --> 00:51:42.119 David Stephens, Prof: Often, X is not low-dimensional, but it's in fact high-dimensional. 494 00:51:43.870 --> 00:51:50.029 David Stephens, Prof: So, we may not… we may run out of data quickly. We might have difficulty defining what the strata are. 495 00:51:51.520 --> 00:51:58.829 David Stephens, Prof: We need enough data within each of the strata to do a comparison of treatment, treated individuals and non-treated individuals. 496 00:51:58.940 --> 00:52:05.380 David Stephens, Prof: So, with high-dimensional and continuous Xs, this is not really implementable. 497 00:52:06.460 --> 00:52:09.579 David Stephens, Prof: Right? So the principle is very good. 498 00:52:10.080 --> 00:52:12.299 David Stephens, Prof: But in practice, it's not implementable. 499 00:52:15.620 --> 00:52:19.649 David Stephens, Prof: So matching in this way, and this sort of local modeling. 500 00:52:20.280 --> 00:52:26.000 David Stephens, Prof: It's, in principle, a good idea, but we need… we need to think harder about how we can implement it. 501 00:52:27.540 --> 00:52:31.580 David Stephens, Prof: We've removed the problem of confounding, Through matching. 502 00:52:33.920 --> 00:52:36.070 David Stephens, Prof: Almost equivalently, stratification. 503 00:52:37.480 --> 00:52:42.449 David Stephens, Prof: But achieving the matching, or the stratification itself, is not easy. 504 00:52:43.590 --> 00:52:46.389 David Stephens, Prof: So it turns out this is where the propensity score comes in. 505 00:52:51.800 --> 00:52:56.340 David Stephens, Prof: The Propensity score is really introduced to overcome those two things that I've mentioned. 506 00:52:58.020 --> 00:53:01.679 David Stephens, Prof: It's a way that facilitates causal comparison. 507 00:53:02.210 --> 00:53:06.250 David Stephens, Prof: By constructing balance in a clever way. 508 00:53:08.700 --> 00:53:18.169 David Stephens, Prof: And it's built to address The problems that I identified, continuous predictors, And… Predictors in high dimension. 509 00:53:19.130 --> 00:53:24.220 David Stephens, Prof: So I'm gonna take a couple of minutes break now before we finish today. I'm gonna have a… 510 00:53:24.910 --> 00:53:32.800 David Stephens, Prof: drink on the go here, so I'm gonna take a… take a drink, and then, we'll come back, start again at, 28 minutes past. 511 00:58:38.690 --> 00:58:44.110 David Stephens, Prof: Okay, so we'll pick up then for the last half an hour with some… 512 00:58:45.070 --> 00:58:47.499 David Stephens, Prof: Insights into what the Propensity score… 513 00:58:47.750 --> 00:58:50.750 David Stephens, Prof: How the Propensity score is constructed, how it functions. 514 00:58:50.870 --> 00:58:56.709 David Stephens, Prof: And then we'll look in more detail tomorrow about how it's actually used in statistical methods. 515 00:58:58.330 --> 00:59:04.240 David Stephens, Prof: So what we'll see is that the propensity score is fundamentally a way of constructing balance. 516 00:59:04.370 --> 00:59:08.820 David Stephens, Prof: that is more straightforward than some of the methods that I've just mentioned. 517 00:59:13.200 --> 00:59:17.469 David Stephens, Prof: So remember that our goal is to mimic the construction of the randomized study. 518 00:59:18.280 --> 00:59:23.149 David Stephens, Prof: Because that facilitates direct comparison between treated between treated and untreated groups. 519 00:59:23.970 --> 00:59:34.189 David Stephens, Prof: we, I've seen how to deploy matching-type ideas, but I've also recognized that matching itself might be 520 00:59:34.450 --> 00:59:35.639 David Stephens, Prof: Difficult to implement. 521 00:59:39.020 --> 00:59:45.409 David Stephens, Prof: One of the points about the balance construction, though, is that we're looking to create strata. 522 00:59:46.520 --> 00:59:49.380 David Stephens, Prof: Where there is this sort of local independence. 523 00:59:50.180 --> 00:59:56.159 David Stephens, Prof: That is, within those strata, you can't tell the treated individuals from the untreated individuals. 524 00:59:56.460 --> 00:59:57.970 David Stephens, Prof: In terms of their axis. 525 01:00:05.120 --> 01:00:07.330 David Stephens, Prof: Okay, so… 526 01:00:07.760 --> 01:00:21.770 David Stephens, Prof: Balance, essentially, corresponds to independence between X and Z, so what we're really driving at is that we're looking for balance locally between the treated and untreated groups with respect to the X variable. That is, within 527 01:00:21.940 --> 01:00:27.149 David Stephens, Prof: the range within a stratum S of the X space. 528 01:00:28.060 --> 01:00:33.610 David Stephens, Prof: The distribution of X given Z equals 1, and the distribution of X given Z equals 0 are the same. 529 01:00:35.500 --> 01:00:36.510 David Stephens, Prof: That is. 530 01:00:36.950 --> 01:00:38.320 David Stephens, Prof: We can't tell. 531 01:00:39.590 --> 01:00:44.969 David Stephens, Prof: Whether somebody is untreated, Or treated, just by looking at their X's within that stratum. 532 01:00:50.420 --> 01:00:51.469 David Stephens, Prof: So let's… 533 01:00:52.080 --> 01:00:58.859 David Stephens, Prof: imagine that instead of using all of the X's to do this, we define the stratum in terms of a summary statistic. 534 01:00:59.890 --> 01:01:01.390 David Stephens, Prof: I'm gonna call it E. 535 01:01:01.760 --> 01:01:03.840 David Stephens, Prof: This is a sans serifi. 536 01:01:05.420 --> 01:01:09.380 David Stephens, Prof: If you look closely, you'll see no… Serifs? 537 01:01:09.810 --> 01:01:13.349 David Stephens, Prof: Saying that this is some summary of the axis. 538 01:01:16.630 --> 01:01:24.930 David Stephens, Prof: Okay? And let's say that we're going to define E somehow, such that the conditional distribution 539 01:01:26.610 --> 01:01:29.199 David Stephens, Prof: of X given Z and E. 540 01:01:31.990 --> 01:01:38.620 David Stephens, Prof: Is… Constructed in such a way, that… 541 01:01:40.390 --> 01:01:50.009 David Stephens, Prof: This distribution no longer depends on Z on the right-hand side. So conditioning on E is sufficient to make X independent of Z. 542 01:01:54.610 --> 01:01:57.369 David Stephens, Prof: So, E is going to be used to define the stratum. 543 01:01:59.680 --> 01:02:05.030 David Stephens, Prof: X could be high dimensional, Continuous components and whatever. 544 01:02:05.930 --> 01:02:07.989 David Stephens, Prof: E is gonna be low dimensional. 545 01:02:08.320 --> 01:02:10.910 David Stephens, Prof: Ideally, it will be one-dimensional. 546 01:02:11.720 --> 01:02:21.249 David Stephens, Prof: So let's suppose I can define the stratum in terms of the E, a one-dimensional quantity, such that when I condition upon E, the distribution of X 547 01:02:21.420 --> 01:02:27.610 David Stephens, Prof: Doesn't depend on Z. That is, within the strata, you can't tell the treated from the untreated individuals. 548 01:02:28.910 --> 01:02:30.959 David Stephens, Prof: So let's suppose I have such an E. 549 01:02:33.210 --> 01:02:39.930 David Stephens, Prof: then I could proceed as on the… as in the previous calculations, but using E instead of X. 550 01:02:43.020 --> 01:02:47.889 David Stephens, Prof: Within strata of E, the treated and untreated groups are directly comparable. 551 01:02:48.110 --> 01:02:50.430 David Stephens, Prof: So I can just go back to the previous… 552 01:02:50.740 --> 01:02:54.330 David Stephens, Prof: Local comparisons being combined into a global comparison. 553 01:02:56.730 --> 01:02:59.259 David Stephens, Prof: So that sounds great. If we could do that. 554 01:02:59.580 --> 01:03:01.980 David Stephens, Prof: Our job would be much easier. 555 01:03:02.450 --> 01:03:04.320 David Stephens, Prof: In estimating the causal quantity. 556 01:03:07.870 --> 01:03:13.410 David Stephens, Prof: So let's think about the nature of E, and again, I'll notice, I'll, 557 01:03:13.990 --> 01:03:20.249 David Stephens, Prof: I'll point out that here I've got E, which is the random variable, or the function of the X's. 558 01:03:20.780 --> 01:03:30.589 David Stephens, Prof: So basically, all this means is that I take the X's and form a one-dimensional function of them. If the X's are random, E is random, so E is a random variable. 559 01:03:31.460 --> 01:03:40.289 David Stephens, Prof: And the, so those are the sans-serifs E here that I'll write there. Whereas this little E here, that's a numerical value. This is a regular E. 560 01:03:41.020 --> 01:03:43.110 David Stephens, Prof: So basically what this says is that 561 01:03:43.520 --> 01:03:46.640 David Stephens, Prof: The con… so, for conditional independence. 562 01:03:47.170 --> 01:03:52.990 David Stephens, Prof: Z independent… Sorry, X independent of Z given E, What we have is… 563 01:03:53.270 --> 01:03:55.749 David Stephens, Prof: that the distribution of X given Z and E 564 01:03:56.850 --> 01:03:59.580 David Stephens, Prof: can be written only as the distribution of X, given E. 565 01:04:00.380 --> 01:04:04.340 David Stephens, Prof: Or equivalently, the distribution of Z given X and E 566 01:04:05.070 --> 01:04:07.030 David Stephens, Prof: is the distribution of Z given E. 567 01:04:08.010 --> 01:04:10.690 David Stephens, Prof: Those two things are equivalent, those two equations. 568 01:04:11.630 --> 01:04:12.760 David Stephens, Prof: Alright. 569 01:04:14.760 --> 01:04:19.910 David Stephens, Prof: So, to get such a, so I'm looking for this function E. Can I find… 570 01:04:20.070 --> 01:04:22.129 David Stephens, Prof: an E that does this job. 571 01:04:22.990 --> 01:04:24.910 David Stephens, Prof: If I could, my life is easy. 572 01:04:26.320 --> 01:04:33.890 David Stephens, Prof: Well, let's think about the construction. So, we know that Z is binary, so the distribution of Z is a Bernoulli distribution. 573 01:04:35.790 --> 01:04:41.260 David Stephens, Prof: So what I need is that the distribution of Z given E can be written in this way. 574 01:04:42.240 --> 01:04:44.069 David Stephens, Prof: That's the Bernoulli distribution. 575 01:04:46.020 --> 01:04:53.509 David Stephens, Prof: So the probability of being Z being equal to 1 is just a function of E. The probability that Z equals 0 is just a function of E. 576 01:04:54.980 --> 01:04:55.990 David Stephens, Prof: Alright. 577 01:04:56.350 --> 01:05:04.880 David Stephens, Prof: So that's fine. What this is saying is that I need E to characterize the distribution of Z given X. 578 01:05:06.350 --> 01:05:11.440 David Stephens, Prof: So I can write the distribution of Z given X, Entirely in terms of E. 579 01:05:19.570 --> 01:05:24.949 David Stephens, Prof: Which is what this says here. So I need the distribution of Z given X, that is, the thing that… 580 01:05:25.830 --> 01:05:28.860 David Stephens, Prof: Makes the observational model. 581 01:05:29.640 --> 01:05:32.339 David Stephens, Prof: Different from the experimental model. 582 01:05:32.950 --> 01:05:35.100 David Stephens, Prof: The distribution of Z given X, 583 01:05:35.380 --> 01:05:38.429 David Stephens, Prof: Must be written this, like this. 584 01:05:40.820 --> 01:05:41.570 David Stephens, Prof: Okay? 585 01:05:41.740 --> 01:05:45.450 David Stephens, Prof: If I can do that, then I've found myself an E. 586 01:05:48.950 --> 01:05:50.150 David Stephens, Prof: Can I do that? 587 01:05:50.840 --> 01:05:51.950 David Stephens, Prof: Absolutely. 588 01:05:52.590 --> 01:05:58.390 David Stephens, Prof: If Z is binary, then I can just define A… 589 01:05:58.750 --> 01:06:02.130 David Stephens, Prof: to be the probability that Z equals 1 given X. 590 01:06:04.500 --> 01:06:09.099 David Stephens, Prof: Because then I have that the distribution of Z given E is entirely written in terms of E. 591 01:06:09.930 --> 01:06:11.750 David Stephens, Prof: No longer depends on X. 592 01:06:13.590 --> 01:06:16.520 David Stephens, Prof: So, irrespective of the distribution of the X's. 593 01:06:16.780 --> 01:06:19.119 David Stephens, Prof: The, the dimension of the axis. 594 01:06:20.420 --> 01:06:26.660 David Stephens, Prof: the distribution of Z given X, Is a univariate distribution. 595 01:06:28.270 --> 01:06:35.049 David Stephens, Prof: So this probability here, this conditional probability that Z equals 1 given X, the probability of being treated. 596 01:06:35.280 --> 01:06:39.660 David Stephens, Prof: Given a particular X vector, defines me my E. 597 01:06:41.220 --> 01:06:43.740 David Stephens, Prof: then the distribution of Z given X, 598 01:06:43.850 --> 01:06:46.419 David Stephens, Prof: can be expressed entirely in terms of E. 599 01:06:50.760 --> 01:06:52.690 David Stephens, Prof: That's the fundamental idea. 600 01:06:53.480 --> 01:06:56.369 David Stephens, Prof: We boil all the relevant information. 601 01:06:56.970 --> 01:07:01.230 David Stephens, Prof: For the Z given X distribution, Down into E. 602 01:07:03.460 --> 01:07:09.330 David Stephens, Prof: Then we're going to use conditioning on E, To build our causal comparisons. 603 01:07:11.900 --> 01:07:16.300 David Stephens, Prof: So this is a construction That we can use. 604 01:07:16.490 --> 01:07:20.710 David Stephens, Prof: To aid our stratification, our balance construction. 605 01:07:23.020 --> 01:07:25.220 David Stephens, Prof: And that thing's called the Propensity score. 606 01:07:26.810 --> 01:07:30.710 David Stephens, Prof: Alright, so the Propensity score is really just a way of manufacturing balance. 607 01:07:31.830 --> 01:07:34.370 David Stephens, Prof: If we condition upon the propensity score. 608 01:07:34.760 --> 01:07:38.049 David Stephens, Prof: The treated and untreated individuals within that. 609 01:07:38.410 --> 01:07:42.579 David Stephens, Prof: Within strata defined by the Propensity score, or matched 610 01:07:42.890 --> 01:07:47.170 David Stephens, Prof: individuals, based on the Propensity score, have the same X profiles. 611 01:07:50.230 --> 01:08:02.090 David Stephens, Prof: It's a balancing score, conditional on E, Z and X are independent, exactly what we're after. It's a scalar quantity, because it's, as we see here, this is a single probability, a single numerical value. 612 01:08:04.000 --> 01:08:10.319 David Stephens, Prof: So it's therefore much easier to do stratification matching on the basis of that propensity score. 613 01:08:15.350 --> 01:08:31.550 David Stephens, Prof: There are some other results in the paper that introduced the Propensity score, this Rosenbaum and Rubin paper in 1983, which point out the other beneficial properties of the propensity score as a balancing score mechanism, in that it's the coarsest, and therefore the 614 01:08:31.550 --> 01:08:37.000 David Stephens, Prof: the best form of dimension reduction of the X space into a univariate. 615 01:08:37.029 --> 01:08:38.040 David Stephens, Prof: Quantity. 616 01:08:38.290 --> 01:08:40.700 David Stephens, Prof: So-called courses balancing score. 617 01:08:41.910 --> 01:08:46.010 David Stephens, Prof: That has statistical advantages, because it turns out to be the best. 618 01:08:46.430 --> 01:08:49.760 David Stephens, Prof: way of summarizing the X's to get balance. 619 01:08:50.510 --> 01:08:55.539 David Stephens, Prof: Okay? Somewhat like minimal sufficiency in the ordinary statistical language. 620 01:08:56.689 --> 01:09:01.180 David Stephens, Prof: So this is the propensity score for binary treatments. This is how we construct it. 621 01:09:03.300 --> 01:09:08.660 David Stephens, Prof: And I also think that a DAG visualization of this is also useful. 622 01:09:09.420 --> 01:09:21.670 David Stephens, Prof: So, we had the X, YXZ, and ZY DAG before. This is the confounding setup. Where the Propensity score sits is exactly here. It sits on the path between X and Z. 623 01:09:23.229 --> 01:09:29.630 David Stephens, Prof: And by conditioning on E, We again block the problematic path. 624 01:09:30.600 --> 01:09:35.370 David Stephens, Prof: the back door… Biasing path that was present previously. 625 01:09:38.649 --> 01:09:48.969 David Stephens, Prof: This double arrow here means it's a deterministic function, right? So, basically, the EX is just some summary of the X's, it's a particular summary of the X's. 626 01:09:49.240 --> 01:09:52.000 David Stephens, Prof: So this is one-dimensional, whereas this can be. 627 01:09:52.220 --> 01:09:54.980 David Stephens, Prof: Arbitrary high-dimensional quantity. 628 01:09:56.100 --> 01:09:56.800 David Stephens, Prof: Okay? 629 01:09:57.520 --> 01:10:03.789 David Stephens, Prof: So this is how the Propensity score… this is how the Propensity score constructed, and this DAG tells us why it works. 630 01:10:04.340 --> 01:10:11.149 David Stephens, Prof: Right? Conditioning on E, rather than conditioning on X, achieves the same goal. It breaks the confounding. 631 01:10:20.500 --> 01:10:29.699 David Stephens, Prof: Now, there's an assumption, again, which I'll talk about, and I'll talk more about correct specification through this few ideas in more detail tomorrow. 632 01:10:29.980 --> 01:10:37.280 David Stephens, Prof: It does require that this model be correctly specified. So we've substituted one correct specification assumption 633 01:10:37.960 --> 01:10:47.000 David Stephens, Prof: Which is that the mu is correctly specified, the conditional mu model for Y given X and Z is correctly specified, with an assumption that this is correctly specified. 634 01:10:49.260 --> 01:10:53.160 David Stephens, Prof: The argument is that this is an easier model to… 635 01:10:53.690 --> 01:10:55.929 David Stephens, Prof: Get a handle on, in practice. 636 01:10:59.490 --> 01:11:07.160 David Stephens, Prof: There are special cases that… We can, think about if X is entirely discrete components. 637 01:11:07.510 --> 01:11:10.879 David Stephens, Prof: Then we can get away with non-parametric modeling of this. 638 01:11:11.150 --> 01:11:20.280 David Stephens, Prof: But if X has continuous components, then we need to think about parametric specification, so something like a logistic regression in the binary case. 639 01:11:22.660 --> 01:11:27.699 David Stephens, Prof: The difference here, though, is that this model is uncontroversially estimable from the data. 640 01:11:36.030 --> 01:11:42.780 David Stephens, Prof: The other key result in Rosenbaum and Rubin is to do with unconfattedness, which is that If… 641 01:11:42.880 --> 01:11:48.350 David Stephens, Prof: We start off with the assumption of… Independence of the… 642 01:11:51.050 --> 01:11:57.199 David Stephens, Prof: counterfactual outcomes and Z given X. That is, the original DAG works. 643 01:11:57.310 --> 01:12:05.180 David Stephens, Prof: then… Elementary… so that allows us to shift around the conditioning in this setting to say that 644 01:12:05.380 --> 01:12:12.750 David Stephens, Prof: The target joint distribution of interest, in the hypothetical world, of counterfactuals. 645 01:12:12.940 --> 01:12:15.110 David Stephens, Prof: Is linked directly. 646 01:12:15.840 --> 01:12:17.600 David Stephens, Prof: To the observational world. 647 01:12:19.170 --> 01:12:27.200 David Stephens, Prof: It allows us to remove… to… change this. Otherwise, complicated looking. 648 01:12:27.530 --> 01:12:34.119 David Stephens, Prof: distribution of the counterfactual outcome given X into this observational type. 649 01:12:34.430 --> 01:12:36.049 David Stephens, Prof: Y given X and Z? 650 01:12:36.500 --> 01:12:41.459 David Stephens, Prof: model, right? That's what… No own magic founder allows us to do. 651 01:12:42.660 --> 01:12:45.810 David Stephens, Prof: Well, the critical result of Rosenbaumin is that 652 01:12:46.050 --> 01:12:54.830 David Stephens, Prof: If we've got no measured confounders and unconfoundedness given X, then we also have unconfoundedness given E. 653 01:12:58.690 --> 01:13:07.400 David Stephens, Prof: And this is important, so what this… this really backs up the fact that we can replace conditioning on X by conditioning on E. 654 01:13:11.240 --> 01:13:15.779 David Stephens, Prof: Having gone down the route of balance, we're now in a position to exploit that. 655 01:13:15.920 --> 01:13:25.689 David Stephens, Prof: base our modeling on E rather than X, which, in principle, is a simpler exercise. If we're happy to make the assumption of correct specification here. 656 01:13:26.320 --> 01:13:31.430 David Stephens, Prof: then… That allows us to do the causal comparisons. 657 01:13:32.010 --> 01:13:36.550 David Stephens, Prof: deploying E, rather than… The whole of Acts. 658 01:13:38.080 --> 01:13:40.719 David Stephens, Prof: So that's the construction of the propensity score. 659 01:13:41.020 --> 01:13:48.810 David Stephens, Prof: How do we use it in practice? Well, we can think of a number of ways, and we'll dig into this in more detail, in full detail tomorrow, and see some examples of it. 660 01:13:49.720 --> 01:13:57.600 David Stephens, Prof: We could do stratified estimation based on the propensity score. Stratifying on E is a lot more easy than stratifying on X. 661 01:13:58.110 --> 01:14:00.150 David Stephens, Prof: So we stratify on E, 662 01:14:00.350 --> 01:14:07.690 David Stephens, Prof: And then do local comparisons based on the strata, and then an overall combination of those factors based on the… 663 01:14:08.420 --> 01:14:10.100 David Stephens, Prof: the local estimates. 664 01:14:11.010 --> 01:14:12.150 David Stephens, Prof: We could do that. 665 01:14:12.260 --> 01:14:19.380 David Stephens, Prof: So stratifying on X, stratifying on a single variable, is much easier than stratifying on a high-dimensional variable. 666 01:14:22.560 --> 01:14:24.990 David Stephens, Prof: We still have to do some local modeling. 667 01:14:25.730 --> 01:14:29.230 David Stephens, Prof: So we still have to think about local comparisons within the strata. 668 01:14:30.040 --> 01:14:36.620 David Stephens, Prof: So this is a… so we can, what we're assuming, essentially, is that the comparison of… 669 01:14:37.350 --> 01:14:47.509 David Stephens, Prof: The treated and untreated individuals In the local strata defined by the propensity score, That comparison is… 670 01:14:48.300 --> 01:14:50.320 David Stephens, Prof: Possible. It's feasible. 671 01:14:50.770 --> 01:14:52.679 David Stephens, Prof: We have enough data to do it. 672 01:14:52.810 --> 01:15:03.290 David Stephens, Prof: And that, the role of X is completely captured, that is, we can do a direct comparison of the treated and untreated individuals by comparison of means. 673 01:15:04.280 --> 01:15:04.970 David Stephens, Prof: Okay. 674 01:15:05.080 --> 01:15:14.740 David Stephens, Prof: So that means that we need sufficient number of the treated individuals and a sufficient number of the untreated individuals within the strata, within each stratum. 675 01:15:21.680 --> 01:15:24.630 David Stephens, Prof: Which is what's called an overlap condition. 676 01:15:25.190 --> 01:15:29.220 David Stephens, Prof: And we'll, again, see this in practice tomorrow. Basically, it means that… 677 01:15:29.700 --> 01:15:42.969 David Stephens, Prof: We can't have that, X predicts Z precisely, because if it did, we wouldn't be able to compare the treated individuals with the untreated individuals, right? We would have strata. 678 01:15:43.180 --> 01:15:48.020 David Stephens, Prof: That only had treated, or only had untreated in them. 679 01:15:50.870 --> 01:15:52.019 David Stephens, Prof: That's the first. 680 01:15:52.240 --> 01:15:56.329 David Stephens, Prof: First… Idea, stratification. 681 01:15:56.490 --> 01:15:58.430 David Stephens, Prof: We could also think about matching. 682 01:15:59.570 --> 01:16:04.979 David Stephens, Prof: on the propensity score. So previously, we thought about matching on the axes. We saw that that could induce balance. 683 01:16:06.310 --> 01:16:11.180 David Stephens, Prof: Now we're gonna consider… Matching on the… 684 01:16:11.600 --> 01:16:15.270 David Stephens, Prof: Propensity score. That also can induce balance. 685 01:16:15.830 --> 01:16:20.930 David Stephens, Prof: So, again, we can think about a way to To do that, 686 01:16:21.330 --> 01:16:31.189 David Stephens, Prof: we could do one-to-one matching, for example. We could say, we'll take, a propensity score… sorry, we'll take an individual who's treated. 687 01:16:31.850 --> 01:16:33.769 David Stephens, Prof: Compete that Propensity school. 688 01:16:35.560 --> 01:16:42.620 David Stephens, Prof: Look for a non-treated individual with a similar propensity score, and regard those two individuals as matched. 689 01:16:44.780 --> 01:16:50.039 David Stephens, Prof: So it's the equivalent of matching on X, but instead now we're matching on E. 690 01:16:51.710 --> 01:16:57.910 David Stephens, Prof: Okay? And there are a number of different ways that we can implement this. We'll see this in the… in the constructions tomorrow. 691 01:17:02.220 --> 01:17:07.319 David Stephens, Prof: This, matching kind of strategy is, in fact, widely used in statistics. 692 01:17:07.560 --> 01:17:13.390 David Stephens, Prof: Already. And really, the only role of the propensity score is to make the matching easier. 693 01:17:13.850 --> 01:17:20.540 David Stephens, Prof: Okay, so matching, like regression, matching is a good way to approach causal inference. 694 01:17:21.570 --> 01:17:26.409 David Stephens, Prof: The practical difficulty of having a high-dimensional predictive vector to match on is solved 695 01:17:26.710 --> 01:17:29.459 David Stephens, Prof: Partly by using the propensity score. 696 01:17:33.680 --> 01:17:48.089 David Stephens, Prof: So the way we might do this, then, in practice is to say that we've got two individuals, I1 and I0. I1 is a treated individual, I0 is an untreated individual. The difference between their outcomes 697 01:17:48.680 --> 01:17:54.670 David Stephens, Prof: is, A good representation of the… Causal effect of treatment. 698 01:17:54.780 --> 01:18:01.440 David Stephens, Prof: Right? The only difference between these individuals, once they've been matched, Is their treatment assignment. 699 01:18:02.340 --> 01:18:06.090 David Stephens, Prof: So we can… Do that at the individual level. 700 01:18:06.480 --> 01:18:16.470 David Stephens, Prof: We can aggregate those, across a… a mapped sample, And that sample mean 701 01:18:18.160 --> 01:18:21.400 David Stephens, Prof: Unbiasedly estimates the average treatment effect. 702 01:18:23.660 --> 01:18:31.930 David Stephens, Prof: Okay, that's the basic idea. By performing this matching on the propensity score, we've removed any influence of X. 703 01:18:32.630 --> 01:18:40.730 David Stephens, Prof: We have essentially induced balance at that individual level, To make sure that… these two individuals… 704 01:18:41.130 --> 01:18:44.959 David Stephens, Prof: The only systematic difference between them is their treatment status. 705 01:18:46.080 --> 01:18:52.079 David Stephens, Prof: So we do that across all individuals, take the average, and that's an unbiased estimate of the treatment effect. 706 01:18:53.780 --> 01:18:54.450 David Stephens, Prof: Okay? 707 01:18:56.400 --> 01:19:03.709 David Stephens, Prof: So there are different ways we can think about implementing this. We can do many to one, so one treated to M untreated. 708 01:19:04.490 --> 01:19:20.729 David Stephens, Prof: We could do caliper matching, so within a tolerance. We could do nearest neighbor matching, or kernel matching, based on the propensity score. We could do matching with replacements, and as we'll see tomorrow, that there are software packages that implement this. I'm going to show one in particular that's very good. 709 01:19:20.800 --> 01:19:26.919 David Stephens, Prof: the matching package in R, but there are other… other packages that can be used pretty, 710 01:19:27.610 --> 01:19:33.830 David Stephens, Prof: Pretty effectively to… Achieve this causal… approach. 711 01:19:34.420 --> 01:19:35.990 David Stephens, Prof: Causal estimation approach. 712 01:19:40.210 --> 01:19:44.499 David Stephens, Prof: So I'll finish off today by saying we've focused on the binary case. 713 01:19:45.170 --> 01:19:47.140 David Stephens, Prof: And most of the literature 714 01:19:47.760 --> 01:19:57.620 David Stephens, Prof: up until, I guess, about 20 years ago, was on the binary case, so we had 20 years of Rosenbaum and Rubin inspiring the binary treatment analysis. 715 01:19:57.880 --> 01:20:03.339 David Stephens, Prof: What I want to say, though, is that actually all of the same ideas go across to more general types of treatment. 716 01:20:05.400 --> 01:20:09.289 David Stephens, Prof: And… I think I'll just say that… 717 01:20:09.630 --> 01:20:18.530 David Stephens, Prof: the principle that we're appealing to is really the same, in that what we want to do is to construct a balancing score B, 718 01:20:18.700 --> 01:20:22.429 David Stephens, Prof: Such that conditional on B, Z and X are independent. 719 01:20:26.270 --> 01:20:27.350 David Stephens, Prof: Now… 720 01:20:28.630 --> 01:20:38.390 David Stephens, Prof: what we can think of that as really saying is that the conditional distribution of Z given X is entirely characterized by some function B of X. 721 01:20:39.370 --> 01:20:44.709 David Stephens, Prof: In general, we can think about functions of both Z and X. We might have to think of that. 722 01:20:44.880 --> 01:20:46.780 David Stephens, Prof: But to mimic the Propensity 723 01:20:46.970 --> 01:20:56.120 David Stephens, Prof: Propensity score construction, we're looking for the balancing score B that summarizes appropriately the X distribution, such that when I condition on that B, 724 01:20:56.540 --> 01:20:58.880 David Stephens, Prof: All dependence on X is removed. 725 01:21:00.010 --> 01:21:01.400 David Stephens, Prof: In the conditional model. 726 01:21:02.900 --> 01:21:07.510 David Stephens, Prof: So that might, again, seem like a tall obstacle. 727 01:21:08.570 --> 01:21:10.669 David Stephens, Prof: tall order. But… 728 01:21:11.950 --> 01:21:23.369 David Stephens, Prof: First of all, people started working on what's called the generalized Propensity Score, which is something that's based on the same construction as the ordinary propensity score, it's just that Z is allowed to be continuous. 729 01:21:24.580 --> 01:21:30.300 David Stephens, Prof: Right? So we build a continuous model for Z, given X, 730 01:21:31.530 --> 01:21:37.119 David Stephens, Prof: Okay? And then we just deploy B in the same way that we do as the ordinary propensity score. 731 01:21:38.460 --> 01:21:52.620 David Stephens, Prof: So this conditional model of Z given X is still the key to achieving balance. We might match on this, we might include it in other analysis methods, but it's really the same kind of construction. 732 01:21:53.040 --> 01:22:02.609 David Stephens, Prof: There are some challenges with this approach to do with the correct specifications, so now this method, which was proposed, I guess, about in 2003, 2004, 733 01:22:03.020 --> 01:22:10.160 David Stephens, Prof: This kind of method has been superseded by a simpler method, which is basically that we… 734 01:22:10.570 --> 01:22:16.270 David Stephens, Prof: Can really focus on model specifications. 735 01:22:16.410 --> 01:22:21.829 David Stephens, Prof: such that the function B, the balancing score B, is only a function of X. 736 01:22:24.320 --> 01:22:27.510 David Stephens, Prof: So instead of having this more general 737 01:22:27.800 --> 01:22:30.600 David Stephens, Prof: Balancing score, which depends on Z and X. 738 01:22:31.050 --> 01:22:38.539 David Stephens, Prof: which is the most general kind of construction. Those kind of constructions are sometimes hard to… to make. Instead, we focus on 739 01:22:38.930 --> 01:22:43.549 David Stephens, Prof: B's, which are purely functions of X, so again, mimicking the propensity score. 740 01:22:44.110 --> 01:22:49.489 David Stephens, Prof: And a sort of example that I'm thinking of here is if we have, say, continuous valued exposures. 741 01:22:50.070 --> 01:22:55.000 David Stephens, Prof: We might say that Z given X is normally distributed using some model. 742 01:22:55.650 --> 01:23:01.910 David Stephens, Prof: That is… the distribution of Z given X follows this normal regression-like construction. 743 01:23:03.000 --> 01:23:09.139 David Stephens, Prof: Okay? So in this case, X alpha is a balancing score. 744 01:23:10.060 --> 01:23:15.360 David Stephens, Prof: That is, irrespective of the individual values of the X's. 745 01:23:16.410 --> 01:23:21.140 David Stephens, Prof: It's the linear combination X times alpha, That is… 746 01:23:22.490 --> 01:23:25.660 David Stephens, Prof: The only thing that specifies this model. 747 01:23:27.240 --> 01:23:37.880 David Stephens, Prof: So we can use this model, which is indeed the conditional expectation of Z given X, as the balancing score instead of the propensity score. And once you see that general idea. 748 01:23:38.120 --> 01:23:40.179 David Stephens, Prof: Then it's easy to see that 749 01:23:40.350 --> 01:23:44.860 David Stephens, Prof: really, conditional models for Z given X are often as good as 750 01:23:45.070 --> 01:23:47.140 David Stephens, Prof: probabilities of a dead given X. 751 01:23:49.100 --> 01:23:51.420 David Stephens, Prof: So the idea here is simply that 752 01:23:51.570 --> 01:23:55.019 David Stephens, Prof: Our task in constructing a balancing score B 753 01:23:55.410 --> 01:24:02.200 David Stephens, Prof: is really to have something that characterizes the conditional distribution of Z given X precisely. 754 01:24:02.430 --> 01:24:08.030 David Stephens, Prof: That is, again, somewhat like a sufficiency-type argument. You only need 755 01:24:08.390 --> 01:24:17.559 David Stephens, Prof: to know B, you don't need to know the individual values of the X's, so all X's that give rise to the same B are considered matched. 756 01:24:19.150 --> 01:24:33.080 David Stephens, Prof: Okay? So actually, this extension… so I used to do a lot more on… when I taught this course, taught this module on… I used to do a lot more on continuous exposures, but actually, the logic is exactly the same as the original propensity score. 757 01:24:33.470 --> 01:24:39.310 David Stephens, Prof: So really, there's not much else to be done. We'll see some examples of continuous exposures in the examples tomorrow. 758 01:24:43.430 --> 01:24:48.379 David Stephens, Prof: Yeah, so the… it's just summarizing that. So, 759 01:24:50.410 --> 01:24:58.269 David Stephens, Prof: What I'll, so I think I'll break off there for today. We're at 5 to… 5 to the hour, 5 to 3 my time. 760 01:24:58.700 --> 01:25:07.310 David Stephens, Prof: what I'll do, first of all, is to… so we'll pick up tomorrow with some more explicit use of the Propensity score in statistical modeling methods. I'll give a, 761 01:25:08.120 --> 01:25:14.190 David Stephens, Prof: a nod to the analysis of the NHANES data, so you can look through this maybe before, 762 01:25:14.520 --> 01:25:20.310 David Stephens, Prof: Before we start tomorrow, we'll get to this around the end of the first session tomorrow morning. 763 01:25:22.790 --> 01:25:40.590 David Stephens, Prof: this shows the real implementation of these kinds of methods, and then also this simulated study, which gets at all of the issues that I've raised today. The importance of inducing balance, the relevance of modeling, the relevance of regression, correct specification, incorrect specification. 764 01:25:40.810 --> 01:25:46.390 David Stephens, Prof: And these handouts also, go on to the more advanced methods that we'll talk about. 765 01:25:46.890 --> 01:25:52.610 David Stephens, Prof: Tomorrow. So you can see there the kinds of methods that we're gonna study. 766 01:25:53.610 --> 01:26:00.530 David Stephens, Prof: In particular, we'll look at… Regression-type methods, varieties of that, and inverse weighting methods. 767 01:26:02.380 --> 01:26:09.789 David Stephens, Prof: Okay, so I'll stop there for today, then, and, I'll, upload the recording for today. 768 01:26:10.240 --> 01:26:13.700 David Stephens, Prof: In the next couple of hours, it takes a little bit of time to, 769 01:26:13.810 --> 01:26:19.870 David Stephens, Prof: filter through the Zoom system at McGill, but it should be up in a couple of hours, so… 770 01:26:20.610 --> 01:26:26.570 David Stephens, Prof: I'll sign off there for today. If anybody's got any immediate questions, I can answer them, but we can pick up tomorrow. 771 01:26:27.350 --> 01:26:28.070 David Stephens, Prof: Thanks.