Since we are interested in how these two measures of variability compare, we simply look at the ratio, and that ratio is called the F statistic. Under the null hypothesis where there is no difference in treatment means, this ratio should be about 1. Of course, it will be not exactly 1 due to sampling variability. In fact, it follows what's called an F-distribution with k - 1 and N - k degrees of freedom. So, the F-distribution is a little bit like the student's T-distribution in that it comes with degrees of freedom. But in this case, there are actually two degrees of freedom. One for the numerator, which is k - 1, and one for the denominator, which is N - k. Remember, our idea is to assess whether the variability between the treatment means is large compared to the variability within the groups. So, we would reject our null hypothesis if this F statistic is large. And that means that if F is in the right 5% tail, that is then the p-value is smaller than 5%. Typically, the results of an analysis of variance are summarized in an ANOVA table. Remember, we looked at two sources of variation, the treatment and the error, but just the variability within the groups. We saw the treatment has k - 1 degrees of freedom and the corresponding treatment sum of squares are denoted by SST. The mean square is simply the treatment sum of squares divided by the degrees of freedom. The same thing applies to the second line where we look at the error sum of squares and the mean square error. And finally, we saw that the F statistic is simply the ratio of these two mean squares. And then, the analysis will produce a p-value, which goes in this last cell here. There's also a third line in the ANOVA table, which simply sums up the degrees of freedom and the sum of squares, and we will talk about that later. In the previous example about peer assessment, when we do the ANOVA for the data in the right-hand display, we get the following ANOVA table. The p-value comes out as 9.7%. Since that is not smaller than 5%, there is not sufficient evidence to reject the null hypothesis. Remember, that was also our impression when we looked at the box plots. The idea behind the ANOVA table is that each observation is generated as the sum of a treatment mean, mu j, and an error term, epsilon. You can think of the error term as a measurement error, and the assumption for ANOVA is that these error terms at least roughly follow the normal curve, and they are independent with mean 0, and a common variance, sigma square. Remember, our null hypothesis is that all those treatment means are the same. Instead of looking at these treatment means, mu j, it's common to look at deviations from an overall mean, mu. That deviation is called tau j. It's simply the difference of the jth treatment mean, from the overall mean mu. So, if we rewrite that model, we get that an observation yij is a sum of the overall treatment mean mu, the treatment effect of the jth group, and the error term epsilon. Since we are now looking at deviations from the overall treatment mean mu, the null hypothesis becomes that all these deviations are 0. Of course, we would estimate the overall mean mu by the grand mean y bar bar. Then, the estimate of the treatment effect tau j simply becomes yj bar - y bar bar. All we did there was plug in the estimates for mu j and mu. Finally, the estimate of our error term epsilon is simply the residual. The residual is defined just the same way as in regression. So, now, when we look at our model where y is the sum of an overall mean mu, a treatment effect, tau j, and an error term, we can plug in our estimates and we get this equation, y is the sum of three terms. Now, of course, this is kind of obvious because you see that y bar bar cancels out here, and the jth treatment mean also cancels out. So, this identity simply says that yij = yij. But it's useful to write it that way, because it tells us how we think that yij comes about from the various sources of variation. It turns out that this type of decomposition is also true after squaring terms. That is, after we move y bar bar to the left side by subtracting it off, and then, if we square and take sums, we get this identity here. We already know what the last term there is, that's simply the error sum of squares. Likewise, the second to last term is the treatment sum of squares. We haven't discussed the first term. It's called total sum of squares. If you look at this, you see what we are doing here, we look at all the observations, look at the squared differences from the overall mean, and sum up. So, this is like a formula from computing a sample variance. What this means is that the total variation can be split into two sources, the treatment sum of squares and the error sum of squares. This is the decomposition that's behind the ANOVA table.
