This paper presents a new method for characterizing brain responses in both PET and fMRI data. The aim is to capture the correlations between the scans of an experiment and a set of external predictor variables that are thought to affect the scans, such as type, intensity, or shape of stimulus response. Its main feature is a Canonical Variates Analysis (CVA) of the estimated effects of the predictors from a multivariate linear model (MLM). The advantage of this over current methods is that temporal correlations can be incorporated into the model, making the MLM method suitable for fMRI as well as PET data. Moreover, tests for the presence of any correlation, and inference about the number of canonical variates needed to capture that correlation, can be based on standard multivariate statistics, rather than simulations. When applied to an fMRI data set previously analyzed by another CVA method, the MLM method reveals a pattern of responses that is closer to that detected in an earlier non-CVA analysis.