Some definitions for ANOVA and Regression

Some definitions: Factor, treatment, level, blocks, the model, partitioning of variance, residual mean square, experimental error,  F distribution, F tests, mean separation, interaction, main effects. (Click for Glossary)

The  ANOVA model is an additive model which aims to  partitions the total variance into variation from known sources (Factors, Blocks and Error). the Total Sums of Squares divided by the degrees of freedom give estimates of Mean Squares for the known sources of variation.

ANOVA set out as treatment  categories (Factors – of interest)  and the continuous response variable

  • The overall  experimental mean is also called the Grand Mean
  • Typically a treatment is entered in the experiment as a number of levels.
  • One Factor or more Factors can be studied within one experiment.
  • These are called Main factors and many be labelled A and B
  • An interaction is the significant effect of  Factor A at different levels on Factor  B. This can be tested in the ANOVA with an Interaction Source of variation and it’s own F test.
  • Blocks are aspects of the experimental material that can be better estimated by taking into account nuisance aspects such as size differences in animals, soil differences in a field or environmental gradients of heat, cold or humidity in a glass house. We can reduce the Residual Mean Square (also known as Error Mean Square)  by partitioning out known variability.
  • The Error Mean Square (EMS) or  Residual Mean Square (RMS) is an estimate of our experimental error or pooled variance for the whole experiment.  When EMS is low we are more likely to find treatment differences in our factors of interest.


  • Response variable
  • Independent variable,

An ANOVA table is made of the following columns

  • Source of variation – from factors of interest and residual
  • degrees  freedom df (eg in a simple one way the number of factor levels minus one)
  • Sums of Squares
  • Mean Squares
  • F statistic from the experiment for that particular source

It may also show:

  • The probability of the calculated F statistic from the software
  • The F statistics is a ratio of variances, in  ANOVA it is a  ratio of treatment MS divided by Residual MS

ANOVA for Regression

Regression coefficients,

  • slope,
  • intercept
  • standard errors of coefficients
  • confidence intervals of coefficients

Model fit

  • residual analysis
  • Advanced features of model fit are covered in the regression (module 3) and the advanced module 7

Interpretation of the results

  • Examine the results of the ANOVA table.
  • Only look at mean separation if the F test shows significance
  • Plot means, present as a table of means
  • Regression look at model fit
  • Regression look at prediction of  a new value

Multivariate regression look at the way to build models and determine the best model

  • Forward selection, add one variable at a time, with rules to keep in
  • Backwards elimination, put all variables in and gradually remove
  • Compare the model with the baseline model

Leave a Reply