[MUSIC] Hi everyone, welcome back to the systematic review and meta-analysis class. Today we're going to talk about statistical methods for meta-analysis. In the goals for today, we're going to describe the fixed effect and random effects models and the underlying assumptions. So these two types of models are the commonly used statistical models for meta-analysis. And then we're going to show you how to compute the summary effect, the diamond down on the forest plot, using a fixed effect in a random effects model. So to begin with, let's have a quick review of what is a meta-analysis. Well, meta-analysis is an optional component of a systematic review. Which means not every single systematic review would have a meta-analysis, although every single systematic review would have the qualitative synthesis. There are classic definitions for meta-analysis, which is the statistical analysis for a large collection of analysis results from individual studies for the purpose of integrating the findings. A better alternative definition is that meta-analysis is the statistical analysis which combines the results of several independent studies considered by the analyst to be combinable. So the second part of this definition, considered by the analyst to be combinable, is very important, because you as a systematic reviewer has to make the decision whether the studies are similar enough to be put together in a meta-analysis. Meta-analysis is not that complicated. Let's demystify what is a meta-analysis. So for meta-analysis, you're including a set of individual independent studies. And in each study, those individual studies, the measures of association or the effect estimate of the treatment effect, or the association, is usually summarized as a risk ratio, as an odds ratio, mean difference, prevalence, or regression coefficient. For example, let's say we're doing a study comparing vitamin D versus placebo for preventing fracture, and then the fracture event will be a binary outcome. We can summarize that using risk ratio or odds ratio. And then you will have a set of studies to be included into your meta-analysis. Each study will have that summary measure. And now what we're doing a meta-analysis is to estimate the overall measure effect as an average. There are different ways to average a bunch of numbers. Arithmetic mean is one way, but in meta-analysis, we're taking weighted average. So the weight reflects the varying importance of each study in your meta-analysis. As you can imagine, we're going to assign more weight if there's more information in that particular study. And the more information is primarily affected by your sample size, as well as the number of events that occurred in that study. So more information will lead to increased precision in your individual study, and more precision overall will lead to increased precision in your meta-analysis results. In today's lecture, we're going to introduce two types of models for meta-analysis. We're going to start with the fixed effect model, because that's simpler to describe and to introduce the concept to you, and then we will move on to the random effects model. Under the fixed effect model, we assume that all studies, you're including your meta-analysis are measuring the same, common (true) effect size. Well, what does it mean? It means that, if it's not for random or sampling error all results in those individual studies would be identical. And now we are going to denote the true (unknown) effect size by theta, okay? So basically, the idea is saying, we observe some effects from each individual study, and now we have to come up with the summary effect. And how we're going to do it, we're going to make some assumptions. And under the fixed effect model, we assume that each individual study estimated a true common effect size. Here is another way to look at the assumption. In this example, we have three studies. Each circle on the figure represents the true effect in each study. And again, under the fixed effect model, we assume all the circles coincide with each other, and we're trying to estimate that combined true effect. Well, the problem is, if we already know the true effect size in each study, we don't have to do any research at all, because we just take where that circle is and go with it. Again, the true effect we denoted use theta, and in this particular example, the theta is 0.6. So we don't know the true effect size in each study. What we have from your data is the observed effect size. And it usually varies from one study to the next. And under that model, the fixed effect model, we assume that there's only one source of variation which is the random errors inherent in each study. If you look at the figure, the second figure on the plot, where we have the square, the black square to represent the observed effect size from each study. So, here, that's what you have actually from your study. Going back to the example, let's say we're comparing vitamin D versus placebo to prevent fracture, and you have three studies. You may observe the effect size of 0.4 in Study 1, effect size of 0.7 in Study 2, and effect size of 0.5 in Study 3. That's what you actually have from your data. And the purpose of doing a meta-analysis is trying to use those observed effect size to make the best guess of where the common combined true effect size is. Again, here for example, the observed effect in Study 2 is circled on the figure, and it's 0.7 in this example. The observed effects size varies from one study to the next only because of the random error inherent in each study. Thus, the assumption for the fixed effect model meta-analysis. And the sampling error, which is denoted as epsilon in this example, is -0.2, 0.1, and -0.1 respectively in Study 1, 2, and 3. Another way to look at this data, so for Study 1, the effect size you observe is actually 0.4. So Y1 would equal 0.4. And how is that derived? It's using the overall mean theta, 0.6 minus 0.2. That's your Y1. And if you look at the second study, the observed effect size in that study is 0.7. And it can be written as using the overall theta, 0.6 plus the 0.1, which is epsilon 2. And you can do the same for the third study. More generally, you can write this out, right? The observed effect size Yi for any study It's given by the population mean, plus the sampling error in that study. So again, in a more general way, you can write Yi equals theta plus the epsilon Y. Then, Epsilon Y is the random arrow, or the sampling arrow inherent in that study. If you have three studies in your meta-analysis, the i would be one, two, or three. Just to summarize, under a fixed effect model, there is only one source of variance. And that is the random errors inherent in the study. Here, if you look at this figure, we have the distribution of the random errors inherent in the study. Each normal curve on the figure reflects the amount of variance in that study. Comparing Study 1, 2, 3, Which study has the largest variance? Well it's the first study because the normal distribution has the widest spread. And the study two is the most precise because the normal curve has a shorter distribution. And under the fixed effect model. We assume all the circles on the figure which are the true effect from each study, but you don't see them, coincide with each other. Okay? So you don't actually see the true effect from each study. All you see or all you have from your data is the observed effect sites, which are represented as the square on this plot. So meta-analysis is basically you have all these numbers, you get a point estimate from each study, as well as some variance estimate from that study reflected as the width of the distribution of the normal curve on this figure. And now you're saying, basically you're telling your computer model or software program, Well, I'm going to assume all the true effects in the studies lie on the same line together. And we're going to use that model to try to figure out where that theta is. So that's what you're doing on their fixed effect model. You're making the assumption, all the circles, the true effect that the all observes true effect in each study, lying on top of each other. So they're identical basically. And to do a meta-analysis, you start with your observed effects and try to estimate the population effect through computing a weighted mean. Okay. So we say you have that estimate from each individual study, and now we're going to take a weighted mean to try to figure out where those circles are. And the weight assigned to each study in a fixed effect meta-analysis equals the inverse of the Wi inv Vy study variance. That's the first equation on this slide. Wi equals the inverse of the variance for that study i. Okay? So, now you get the weight for each study, which is the inverse of the Wi inv Vy study variance. And the weighted mean can be computed using the second equation on this slide, which equals the summation of Yi, which is the effect size from that study i. Times the weight i from the study. And you're going to sum cross all studies including your meta-analysis. That's your numerator. And the denominator basically is the summation of all weight. So let's say going back to the previous example where you have three studies, then you will use the effect size from study one. Times the weight from study one plus the effect size from study two times the weight from study two plus the effect size from the third study, the last study, times the weight of that study. So that's your numerator. You do the same for your denominator. And that's how you compute the weighted mean. Getting a weighted mean is not sufficient because we also want to know how precise is that weighted mean, That's why you need to get the variance for the summary effect, which is the third equation on our slide. So the variance for the summary effect, the weighted mean, equals one over the summation of weight i. Again, if you have three studies, then your denominator would be weight 1 plus weight 2 plus weight 3. Once you get the variance, the standard error is basically the square root of your variance. That's the last equation on the slide. So those are the equations, all you need to know of how to calculate a meta-analysis result under a fixed effect model and with those numbers, you can derive the 95 percent confidence interval. Again, the lower and upper limits for the summary effect would be the mean minus 1.96 times the standard error or the mean plus 1.96 times the standard error for the upper limit. And you can use again the mean and variance or the standard error to test for the null hypothesis that the common true effects on theta is zero. And from there you can get the P value. So the equations on this slides are standard equations probably you have seen already from your EPI or biostats classes. Now we're going to plug in some numbers to the equations I just showed you and see how these play out. So here we have a data set where each row is the number or the result from one study. Together we have six studies and for each trial there are treated participants and untreated participants, the comparison group. And you will have the number of events and non-events from each group, denoted as A, B, C, D, in this data set. For example, for the first study, under the treated group, there are 12 events, and 53 non-events. And under the control group, there are 16 events and 49 non events. Going back to the vitamin D example, this could be the number of fractures. There are 12 fractures under those treated by vitamin D, and 16 fractures under those who are treated by placebo. And that's the the number you're going to get from each study and altogether we have six of them. Now, once you have these numbers, you can calculate the measures for the treatment effect. One of the measure is odds ratio. An odds ratio is defined as the odds of the event under one group over the odds of event for the other group. And, if you remember the equation from your Biostats class, so here the odds ratio basically equals AD/BC. And again for the first study, if you plug in the four numbers, 12, 53, 16, and 49, you'll get an odds ratio 0.69. So that's the first number on the second table here. And because this is ratio and we work on the log scale. So you're going to take the log of that odds ratio and get a minus 0.37 for the first study. And the variance for odds ratio, again, you can go back to your biostats materials to figure out the equation. Again, you plug in those 4 numbers, you get the variance for that study. And remember from the previous slide, we said under the fixed effect meta- analysis model the weight equals one over the variance. So for the first study, if you use 1 divided by 0.19, you're going to get the weight for that study, which is 5.4, okay? And once you multiply your weight from that study with the point estimate from that study, you get your last column. So weight i, weight one, is 5.4 and Yi is minus 0.37. So you times those two numbers- multiply those two numbers you're going to get minus 1.97. So, that's what you get from your first study. And you do the same for the second to the sixth studies. You're going to fill in the numbers to this table following this same procedure in the equations. You don't have to do this by hand, I mean the easy way to do it would be to do it in an Excel spreadsheet where you plug in the equations and it will do it automatically for you. Or you'd use statistical software we're going to introduce later for example using Stata for the analysis. You don't have to do this by hand. The most important thing on this slide is the sum of the numbers. So if you look at the last two columns, and those are the two columns we have to sum them across studies, to get the summation. And why we need those two numbers? It's because remember the equation to get the Pooled odds ratio? So for the meta analysis you have to take the weighted average of those numbers. And the equation says the log odds ratio for the meta analysis equals the summation Yi times Wi divided by the summation of Wi. So that's why those two sums are important. And if you go back to the previous slide or if you still remember the two numbers from the previous slide. They're minus 30.59 divided by 42.25. So that's how you get the log odds ratio of minus 0.72. And if we exponentiate that number we get the odds ratio on the natural scale. So that's how you get your meta-analysis results. Remember we said you not only need the point estimate, you still need some estimation of the variability of that estimate. So how precise is that odds ratio? And in order to get the variance and confidence intervals, the variance of that odds ratio equals one over the summation of Wi. This is not a new equation, you have seen it on one of the previous slides. Again, we're just plugging the number from the example and using 1 divided by 42.25, you get the variance. Square root of the variance is your standard arrow, and now again plugging the numbers to the equations you're pretty familiar with by now. You will get the lower and upper confidence limit for the alteration. Fortunately, we don't have to do all this by hand, but I do want to show you how the calculation was done behind the scenes. If you are going to do this using statistical software, this is what you will get most likely, which is a forest plot. Here again, each row represent one study. We have to point estimate the odds ratio from each individual study, as well as the relative weight that study's taking. So we calculated odds ratio for the first study, which is 0.69. And the relative weight basically is the weight the absolute weight of that study divided by the summation of weights crossed or studies. So the first study takes about 13% of all weight in a combined estimate. And each square on the right hand side basically plots where the odds ratio is and the 95% confidence interval for that study. You will the notice the size of square differ and it's reflected by the relative weight. So, the fourth study, the Lane study, which takes about 41% of the total weight, is the largest. So it dominates other studies. That's why it has the larger square. And then the summary effect is shown on the bottom with the diamond, and the length of that diamond is the confidence interval for the results. So here the summary estimate is .46, and it ranges from 0.3 something to maybe 0.6 or 0.7. And you can get that number from our previous slide. So, that's it for a fixed effect meta-analysis. And I think the most important thing you should take away from the fixed effect model is the assumption. Basically we're assuming all studies in analysis share a common true effect. And remember all the circles on the plot that I showed you early on, we are assuming all the true effects line up on top of each other. And the meta-analytic result is our estimate of this common effect. In other sense, all the observed variation reflects the sampling arrow. So the true effects lie on top of each other and all the differences you actually see from one study to the next is only because of the sampling error or the random error of each individual study. And the study weights are assigned proportional to the inverse of within studies variance. So how precisely the estimates are. So the larger the study, the smaller the variance. So the larger study will contribute more information to the meta-analysis results. So it will take up more weight in your pooled estimate. That's a summary of the fixed effect model.
