Observation n°1 :
The tests were carried out by Covance Laboratory (reference 6103-293 of the December 17th, 2002), at Monsanto's request ; and the experimental plan seems to have been followed correctly, in particular the randomization of the distribution of the 400 rats (200 males and 200 females in the ten studied groups) guarantees there has been no skew in sampling. This is translated for example, at the beginning of the experiment, by the absence of significant differences in rat's' weights, by sex, between the ten groups. However, the range of rat's weights is very large, maybe more than usually.
On the other hand, the crude file reveals that the statistical analysis of the data for this experiment was carried out by Monsanto's statistics center. This is likely to harm seriously the independence of expertise on the results, for instance even at the level of the choice of the techniques and the statistical tests used.
Observation n°2 (on the experimental plan itself) :
The goal of the experiment being the study of the toxicological effect of the introduction of the genetic construction producing an insecticide, into the genome of corn, it should be guaranteed that the only sources of variability in the results relate to the presence, or not, of this transgene apart from purely random effects.
It is the reason why the presence of the 6 groups of references fed with other commercial varieties of corn is not necessary, and likely to introduce some distortion in the experimental data. It would have been preferable to restrict, to the groups 1 to 4, whose feed includes 11% or 33% from MON 863 maize or its genetic equivalent (nontransgenic) LH82A634.
Then, the restriction on the number of studied groups could have been accompanied by an increase in the number of individuals by group (40 or 50) and by a prolongation of the study beyond 14 weeks.
Observation n°3 :
One wonders why hematologic and urinary measurements were made only on ten rats in each group and not on all the rats.
It should be noted that most of the statistical tools used by the Monsanto center required conditions which are generally not satisfied if the sample size is too small ; this is the case for ten animals measured per group. In fact, only the first ten rats of each group underwent hematologic and urinary analyses, and Monsanto's statisticians make comparisons using Student tests between samples sizes of ten, an error in methodology : they should use another method like non parametric tests (Wilcoxon, Mann Whitney) with such small groups.
Observation n°4 :
One lays out, at the end of the study for each rat of each group, a great number of measurements of quantitative and qualitative variables. The statistical analysis used should have been right away multidimensional. Then, why did Monsanto's statisticians not use standard multivariate methods ? Instead of that, they have been satisfied most of the time with a simple Analysis Of Variance (ANOVA) with one factor, week after week, character after character, preferring to work on the margins, and not on the joint data, and thus, losing a great part of the information, in particular possible correlations, covariations and interactions between statistical differences concerning different organs and factors, and the principal factors of variability. Differences which could be biologically significant are then likely to be randomly neglected. Only a few of them will be detected as statistical significant.
As an example, one can quote the study of increase in the rat's weights. The analysis which is proposed is a ANOVA week after week, without having an overall vision of the curves of increase of the weights expressed in grams. A simple graph illustrates this question. The curves of average growth by sex for the four groups (1 to 4) raise a certain number of questions which do not seem to have been seen, nor studied by Monsanto's statisticians : For the males fed with transgenic corn, the curves of average growth of the rat's weights remain permanently lower than those fed with nontransgenic corn. Moreover, the highest difference is obtained for the curve corresponding to an amount of 11% of GMO in the regimen. For the females fed with transgenic corn, conversaly, the curves of average growth of their weight remain permanently higher than those fed with nontransgenic corn.
Moreover, the highest difference is obtained for the curve corresponding to an amount of 33% of GMO in the regimen. What does that mean ? The phenomena tend to develop during weeks. What would occur if the study was prolonged over a one year period ?
Without having studied the general data (now our present work), one cannot decide if these differences are significant or not. But it is astonishing that Monsanto's statisticians did not underline this phenomenon, nor used multidimensional methods to compare these curves of growth : for example using Multivariate Analysis Of Variance (instead of ANOVA ) or Time Series Analysis . This small example is important because it underlines the lack of serious and the conceptual errors in Monsanto's statistical analysis.
In addition, it highlights opposite tendencies in the males and the females. This phenomenon is sufficiently serious (even if one cannot say yet if it is significant) for a study to be made on a possible causes for the hormonal effects of the consumed food, and for the problem not to be ignored, as Monsanto does by affirming that, since the tendencies are opposing, this can only be chance.
The same applies, concerning the percentage of GMO in the regimen since things do not occur in an identical way for males and females : is there a sex or dose effect ?
Observation n°5 :
One can make the same remarks with the other statistical significant differences appearing for the weights of some parameters, or haematological data. For Monsanto, they are not biologically significant and undoubtedly due to chance. What is the use of the statistics if each time that a significant effect appears it is considered to be a random effect ? There are again obviously statistical methodological errors at this level.
In addition of the sample size, if the statistical techniques used do not seem sufficiently suitable, the conditions of use must be respected and checked . Finally, any statistical approach should normally begin with a classical analysis of the data, as in a PCA (Principal Components Analysis), as the first stage of a multidimensional analysis.
Temporary conclusion :
The following findings clearly indicate major failures of statistical analyses as performed by Monsanto :
- introduction of irrelevant variability sources as use of additional animal groups likely to dilute biological effects,
-methodological errors such as wrong test system in general which is not suitable to detect very important effects,
-statistical techniques not performed properly ; such as Student test with too small animal groups that do not allow all significant effects to be seen,
-differences in average growth and weight not mentioned by Monsanto,
-conceptual errors in Monsanto's statistical analysis.
In conclusion, after the above remarks, it is essential for Monsanto's whole statistical analysis to be done again, before any decision about market access can be taken. In a second step further analyses from other feeding studies delivered to EU authorities should be done to find out if there are further indications that Bt toxins influence animals's state of health. If market approval for these kind of products is soug, new tests need to be developed, such as initial studies of these new Bt toxins on human cells. Moreover, a new experimental plan including several types of rats, several species of mammals and more samples, during a longer period, could be proposed, as is done for other pesticides. In any case, the ethical question of wether GMO plants such as MON863 really justify animal experiments should be evaluated by the EU and national authorisations, taking civil society into account. For the moment it is only clear that if animal experiments are done, they should be designed in a way that they can produce meaningful results concerning safety aspects, which is hardly the case with the data as discussed here.