Skip to main content Accessibility help

Statistical procedures for testing hypotheses of equivalence in the safety evaluation of a genetically modified crop

  • Q. KANG (a1) and C. I. VAHL (a2)


Safety evaluation of a genetically modified crop entails assessing its equivalence to conventional crops under multi-site randomized block field designs. Despite mounting petitions for regulatory approval, there lack a scientifically sound and powerful statistical method for establishing equivalence. The current paper develops and validates two procedures for testing a recently identified class of equivalence uniquely suited to crop safety. One procedure employs the modified large sample (MLS) method; the other is based on generalized pivotal quantities (GPQs). Because both methods were originally created under balanced designs, common issues associated with incomplete and unbalanced field designs were addressed by first identifying unfulfilled theoretical assumptions and then replacing them with user-friendly approximations. Simulation indicated that the MLS procedure could be very conservative in many occasions irrespective of the balance of the design; the GPQ procedure was mildly liberal with its type I error rate near the nominal level when the design is balanced. Additional pros and cons of these two procedures are also discussed. Their utility is demonstrated in a case study using summary statistics derived from a real-world dataset.


Corresponding author

* To whom all correspondence should be addressed. Email:


Hide All
Chiu, S. T., Tsai, P. Y. & Liu, J. P. (2010). Statistical evaluation of non-profile analyses for the in vitro bioequivalence. Journal of Chemometrics 24, 617625.
Codex Alimentarius Commission (2009). Foods Derived from Modern Biotechnology, 2nd edn, Rome: Joint FAO/WHO Food Standards Programme.
Davit, B. M., Chen, M. L., Conner, D. P., Haidar, S. H., Kim, S., Lee, C. H., Lionberger, R. A., Maklouf, F. T., Nwakama, P. E., Patel, D. T., Schuirmann, D. J. & Yu, L. X. (2012). Implementation of a reference-scaled average bioequivalence approach for highly variable generic drug products by the US Food and Drug Administration. The AAPS Journal 14, 915924.
Dragalin, V., Fedorov, V., Patterson, S. & Jones, B. (2003). Kullback-Leibler divergence for evaluating bioequivalence. Statistics in Medicine 22, 913930.
EFSA (2010). Scientific opinion on statistical considerations for the safety evaluation of GMOs. EFSA panel on GMOs. EFSA Journal 8(1), 1250. doi: 10.2903/j.efsa.2010.1250.
EFSA (2014). Explanatory statement for the applicability of the Guidance of the EFSA Scientific Committee on conducting repeated-dose 90-day oral toxicity study in rodents on whole food/feed for GMO risk assessment. EFSA Journal 12(10), 3871. doi: 10.2903/j.efsa.2014.3871.
EFSA (2015 a). Outcome of the Public Consultation on the Draft Guidance on the Agronomic and Phenotypic Characterization of Genetically Modified Plants. EFSA supporting publication 2015: EN-829. Parma, Italy: EFSA.
EFSA (2015 b). Guidance on the agronomic and phenotypic characterization of genetically modified plants. EFSA Journal 13(6), 4128. doi: 10.2903/j.efsa.2015.4128.
Endrenyi, L., Taback, N. & Tothfalusi, L. (2000). Properties of the estimated variance component for subject-by-formulation interaction in studies of individual bioequivalence. Statistics in Medicine 19, 28672878.
FDA (2014). Guidance for Industry: Bioavailability and Bioequivalence Studies Submitted in NDAs or INDs – General Considerations. Draft Guidance. Rockville, MD: Food and Drug Administration, Center for Drug Evaluation and Research (CDER). Available from: (verified 3 December 2015).
Graybill, F. A. & Wang, C. M. (1980). Confidence intervals on nonnegative linear combinations of variances. Journal of the American Statistical Association 75, 869873.
Hannig, J., Iyer, H. & Patterson, P. (2006). Fiducial generalized confidence intervals. Journal of the American Statistical Association 101, 254269.
Harrigan, G. G., Culler, A. H., Culler, M., Breeze, M. L., Berman, K. H., Halls, S. C. & Harrison, J. M. (2013). Investigation of biochemical diversity in a soybean lineage representing 35 years of breeding. Journal of Agricultural and Food Chemistry 61, 1080710815.
Harrison, J. M., Howard, D., Malven, M., Halls, S. C., Culler, A. H., Harrigan, G. G. & Wolfinger, R. D. (2013). Principle variance component analysis of crop composition data: a case study on herbicide-tolerant cotton. Journal of Agricultural and Food Chemistry 61, 64126422.
Herman, R. A., Scherer, P. N., Phillips, A. M., Storer, N. P. & Krieger, M. (2010). Safe composition levels of transgenic crops assessed via a clinical medicine model. Biotechnology Journal 5, 172182.
Hothorn, L. A. & Oberdoerfer, R. (2006). Statistical analysis used in the nutritional assessment of novel food using the proof of safety. Regulatory Toxicology and Pharmacology 44, 125135.
Howe, W. G. (1974). Approximate confidence limits on the mean of X + Y where X and Y are two tabled independent random variables. Journal of the American Statistical Association 69, 789794.
Hyslop, T., Hsuan, F. & Holder, D. J. (2000). A small sample confidence interval approach to assess individual bioequivalence. Statistics in Medicine 19, 28852897.
John, J. A. & Mitchell, T. J. (1977). Optimal incomplete block designs. Journal of the Royal Statistical Society, Series B (Methodological) 39, 3943.
Kang, Q. & Vahl, C. I. (2014). Statistical analysis in the safety evaluation of genetically modified crops: equivalence tests. Crop Science 54, 21832200.
Khuri, A. I., Mathew, T. & Sinha, B. K. (1998). Statistical Tests for Mixed Linear Models. New York: Wiley-Interscience.
König, A., Cockburn, A., Crevel, R. W. R., Debruyne, E., Grafstroem, R., Hammerling, U., Kimber, I., Knudsen, I., Kuiper, H. A., Peijnenburg, A. A. C. M., Penninks, A. H., Poulsen, M., Schauzu, M. & Wal, J. M. (2004). Assessment of the safety of foods derived from genetically modified (GM) crops. Food and Chemical Toxicology 42, 10471088.
Krishnamoorthy, K. & Lian, X. D. (2012). Closed-form approximate tolerance intervals for some general linear models and comparison studies. Journal of Statistical Computation and Simulation 82, 547563.
Krishnamoorthy, K. & Mathew, T. (2004). One-sided tolerance limits in balanced and unbalanced one-way random models based on generalized confidence intervals. Technometrics 46, 4452.
Krishnamoorthy, K. & Mathew, T. (2009). Statistical Tolerance Regions: Theory, Applications, and Computation. Hoboken, NJ: Wiley.
Lee, Y. H., Shao, J. & Chow, S. C. (2004). Modified large-sample confidence intervals for linear combinations of variance components: extension, theory, and application. Journal of the American Statistical Association 99, 467478.
Liao, C. T., Lin, T. Y. & Iyer, H. K. (2005). One- and two-sided tolerance intervals for general balanced mixed models and unbalanced one-way random models. Technometrics 47, 323335.
McNally, R. J., Iyer, H. & Mathew, T. (2003). Tests for individual and population bioequivalence based on generalized P values. Statistics in Medicine 22, 3153.
OECD (1993). Safety Evaluation of Foods Derived by Modern Biotechnology: Concepts and Principles. Paris, France: Organization for Economic Cooperation and Development.
Park, D. J. & Burdick, R. K. (2003). Performance of confidence intervals in regression models with unbalanced one-fold nested error structures. Communication in Statistics – Simulation and Computation 32, 717732.
Quiroz, J., Ting, N., Wei, G. C. G. & Burdick, R. K. (2002). Alternative confidence intervals for the assessment of bioequivalence in four-period cross-over designs. Statistics in Medicine 21, 18251847.
SAS Institute Inc. (2011). SAS/STAT® 9·3 User's Guide. Cary, NC: SAS Inst. Inc.
Schuirmann, D. J. (1987). A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics 15, 657680.
Searle, S. R. (1987). Linear Models for Unbalanced Data. New York: John Wiley and Sons.
Ting, N., Burdick, R. K., Graybill, F. A., Jeyaratnam, S. & Lu, T. F. C. (1990). Confidence interval on linear combinations of variance components that are unrestricted in sign. Journal of Statistical Computation and Simulation 35, 135143.
Tsui, K. W. & Weerahandi, S. (1989). Generalized P values in significance testing of hypotheses in the presence of nuisance parameters. Journal of the American Statistical Association 84, 602607.
Vahl, C. I. & Kang, Q. (2015). Equivalence criteria for the safety evaluation of a genetically modified crop – a statistical perspective. The Journal of Agricultural Science, Cambridge. doi: S0021859615000271.
Van der Voet, H., Perry, J. N., Amzal, B. & Paoletti, C. (2011). A statistical assessment of differences and equivalences between genetically modified and reference plant varieties. BMC Biotechnology 11, 15. doi: 10.1186/1472–6750-11-15.
Venkatesh, T. V., Breeze, M. L., Liu, K., Harrigan, G. G. & Culler, A. H. (2014). Compositional analysis of grain and forage from MON 87427, an inducible male sterile and tissue selective glyphosate-tolerant maize product for hybrid seed production. Journal of Agricultural and Food Chemistry 62, 19641973.
Ward, K. J., Nemeth, M. A., Brownie, C., Hong, B., Herman, R. A. & Oberdoerfer, R. (2012). Comments on the paper “A statistical assessment of differences and equivalences between genetically modified and reference plant varieties” by van der Voet et al. 2011. BMC Biotechnology 12, 13. doi: 10.1186/1472–6750-12-13.
Weerahandi, S. (1993). Generalized confidence intervals. Journal of the American Statistical Association 88, 899905.
Whent, M., Hao, J. J., Slavin, M., Zhou, M., Song, J. Z., Kenworthy, W. & Yu, L. L. (2009). Effect of genotype, environment, and their interaction on chemical composition and antioxidant properties of low-linolenic soybeans grown in Maryland. Journal of Agricultural and Food Chemistry 57, 1016310174.


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed