No CrossRef data available.
Article contents
Assessing assessors: proposal for a guidance for evaluating the scientific performance of a pesticide regulatory authority
Published online by Cambridge University Press: 16 April 2021
Abstract
External evaluations by an international committee of the scientific risk assessment and decision-making processes of the Netherlands Competent Authority for the authorisation of plant protection products and biocides (Ctgb) were conducted in 2013 and 2018. Based on the experience drawn from them, a general guidance for such visitations is suggested. An international visitation committee examined the structure and management of the Authority, its human resources and staff policy, the scientific processes and output, the documentation and communication of its decisions and the mechanisms for keeping up to date with international scientific developments. Attention was paid to the degree of openness and transparency throughout the organisation and in particular when dealing with confidential information. From the experience gained it can be concluded that visitations not aiming at finding mistakes and omissions but instead focusing on recommendations and constructive suggestions will result in cooperation, mutual trust and acceptance of the recommendations made. A follow-up visitation after a few years can be effective in maintaining a traceable, high-level scientific output. In view of the strong drive towards the European Union-wide harmonisation of the regulatory practices of hazardous chemicals, a voluntary evaluation of regulatory authorities’ scientific performance is recommended as a means for organisational learning.
- Type
- Articles
- Information
- Copyright
- © The Author(s), 2021. Published by Cambridge University Press
Footnotes
The international visitation committee acknowledge the support and assistance offered by all Board for the Authorisation of Plant Protection Products and Biocides (Ctgb) staff members and would like to thank them all for their patience and willingness to spend their precious time with us for interviews and technical help. In particular, we would like to thank the chairman and the members of the board, as well as the director of the Ctgb, for their trust in the capacity, capability and expertise of the international visitation committee membership. We are also grateful to the project manager for providing us with all necessary documents and information. He has been our beacon, always patient and tireless, when some of us got lost in the sea of documents. Finally, we thank the unknown peer reviewer for suggesting significant improvements to this article. The authors declare that they have no competing interests.
References
1 T Tweedale, A Lysimachou and H Muilerman, “Missed & dismissed, pesticide regulators ignore the legal obligation to use independent science for deriving safe exposure levels” (2014) Report published by Pesticide Action Network Europe and Generation Futures, 29 pp.
2 O Renn, Risk Governance. Coping with Uncertainty in a Complex World (London, Earthscan, 2008) pp 222–27.
3 O Nesheim, F Frederick Fishel and M Mossler, “Toxicity of pesticides” (2019) EDIS 2005 (8). <https://journals.flvc.org/edis/article/view/114995> (last accessed 19 March 2021).
4 See, eg, VL Zikankuba, G Mwanyika, JE Ntwenya and A James, “Pesticide regulations and their malpractice implications on food and environment safety” (2019) 5 Cogent Food & Agriculture 1601544.
5 T Hardy, S Bopp, M Egsmose, H Fontier, L Mohimont, H Steinkellner and F Streissl, “Risk assessment of plant protection products” (2012) 10(10) EFSA Journal S1010.
6 Regulation (EC) No 1107/2009 of the European Parliament and of the Council of 21 October 2009 concerning the placing of plant protection products on the market and repealing Council Directives 79/117/EEC and 91/414/EEC. OJ L 309; Regulation (EU) No 528/2012 of the European Parliament and of the Council of 22 May 2012 concerning the making available on the market and use of biocidal products. OJ L 167.
7 GW Suter II (ed.), Ecological Risk Assessment (Boca Raton, FL, Lewis Publishers 1993).
8 Renn, supra, note 2, pp 239–41.
9 SCS Clahsen, I van Kemp, BC Hakkert, TG Vermeire, AH Piersma and E Lebret, “Why do countries regulate environmental health risks differently? A theoretical perspective” (2019) 39(2) Risk Analysis 439.
10 BB Johnson and B Swedlow, “Cultural theory’s contributions to risk analysis: a thematic review with directions and resources for further research” (2021) 41(3) Risk Analysis 429.
11 RL Anjum and E Rocca, “From ideal to real risk: philosophy of causation meets risk analysis” (2019) 39(3) Risk Analysis 729.
12 European Commission, “Pesticides” <https://ec.europa.eu/food/plant/pesticides_en> (last accessed 19 March 2021).
13 European Commission, “Biocides” <https://ec.europa.eu/health/biocides/overview_en> (last accessed 19 March 2021).
14 D Wintermantel, JF Odoux, A Decourtye, M Henry, F Allier and V Bretagnolle, “Neonicotinoid-induced mortality risk for bees foraging on oilseed rape nectar persists despite EU moratorium” (2020) 704 Science of the Total Environment 135400.
15 OECD, “Guidance Document on the Planning and Implementation of Joint Reviews of Pesticides” (2011) Series on Pesticides No. 60 ENV/JM/MONO (2011)11. Organisation for Economic Co-Operation and Development. Environment Directorate, Joint Meeting of the Chemicals Committee and The Working Party on Chemicals, Pesticides and Biotechnology, 54 pp.
16 S Koch, A Epp, M Lohmann & G-F Böl, “Pesticide residues in food: attitudes, beliefs, and misconceptions among conventional and organic consumers” (2017) 80(12) Journal of Food Protection 2083.
17 A Mie, C Rudén and P Grandjean, “Safety of safety evaluation of pesticides: developmental neurotoxicity of chlorpyrifos and chlorpyrifos-methyl” (2018) 17 Environmental Health 77.
18 European Commission, “Farm to Fork Strategy for a fair, healthy and environmentally-friendly food system” (2020) <https://ec.europa.eu/food/sites/food/files/safety/docs/f2f_action-plan_2020_strategy-info_en.pdf> (last accessed 19 March 2021).
19 H Deluyker, A Rodrıguez Pena, M Scannell, J Tarazona and B Url, “What does the future hold for assessment science?” (2016) 14(S1) EFSA Journal S0501.
20 Food and Veterinary Office of the European Commission (2020) <https://ec.europa.eu/food/audits-analysis/audit_reports/index.cfm> (last accessed 19 March 2021).
21 S Kuhlmann and J Bogumil, “Performance measurement and benchmarking as ‘reflexive institutions’ for local governments: Germany, Sweden and England compared” (2018) 31(4) International Journal of Public Sector Management 543; C Luederitz, N Schapke, A Wiek, DJ Lang, M Bergmann, JJ Bos et al, “Learning through evaluation – a tentative evaluative scheme for sustainability transition experiments” (2017) 169 Journal of Cleaner Production 61.
22 The Decree on Mandate, Authorization and Representation by the Ctgb, 3 March 2011, Government Gazette No 4789, the Netherlands, 18 March 2011.
23 SI Donaldson and M Scriven (eds.), Evaluating Social Programmes and Problems. Visions for the New Millennium. The Stauffer Symposium on Applied Psychology at the Claremont Colleges (Mahwah, NJ, Lawrence Erlbaum Associates Publishers, 2003).
24 IF Shaw, JC Greene and MM Mark (eds.), Handbook of Evaluation: Policies, Programmes and Practices (Thousand Oaks, CA, SAGE Publications 2006).
25 DW Compton and ML Baizerman, “The evolution of a philosophy and practice of evaluation advice” (2012) 136 New Directions for Evaluation 67.
26 H Koëter, S Autio, U Banasiak, M Lynch, V Silano and A Cuvillier, Report on the international visitation of the Board for the authorization of plant protection products and biocides (Ctgb) in the Netherlands addressing the scientific process, the scientific output and the decision-making process (Lucca, Orange House Partnership, 2013); H Koëter, S Autio, M Carretero, T Hardy and A Mantovani, Report of the second visitation of the Netherlands Board for the Authorization of Plant Protection Products and Biocides (Ctgb) addressing the scientific process, the scientific output and the decision-making process. (Lucca, Orange House Partnership, 2018).
27 E Desmedt, D Morin, V Pattyn and M Brans, “Impact of performance audit on the administration: a Belgian study (2005–2010)” (2017) 32(3) Managerial Auditing Journal 251.
28 J Lyytimäki, T Assmuth and M Hildén, “Communicating chemical risks for social learning: findings from an expert opinion survey” (2009) 8(3–4) Applied Environmental Education & Communication 174.
29 S Eckhard and V Jankauskas, “The politics of evaluation in international organizations: a comparative study of stakeholder influence potential” (2019) 25(1) Evaluation 62.
30 C Donovan, “State of the art in assessing research impact: introduction to a special issue” (2011) 20(3) Research Evaluation 175.
31 R Lahey and S Nielsen, “Rethinking the relationship among monitoring, evaluation, and results-based management: observations from Canada” (2013) 137 New Directions for Evaluation 45.
32 B Wolf, T Lindenthal, M Szerencsits, JB Holbrook and J Heß, “Evaluating research beyond scientific impact. How to include criteria for productive interactions and impact on practice and society” (2013) 22(2) GAIA 104.
33 See, eg, American Evaluation Association, AEA Guiding Principles. Public Statement on Cultural Competence in Evaluation (Washington, DC, American Evaluation Association 2011).
34 S Bell, B Shaw and A Boaz, “Real-world approaches to assessing the impact of environmental research on policy” (2011) 20(3) Research Evaluation 227.
35 H Selck, PB Adamsen, T Backhaus, GT Banta, PKH Bruce, G Allen Burton Jr et al, “Assessing and managing multiple risks in a changing world – the Roskilde recommendations” (2017) 36 Environmental Toxicology and Chemistry 7.
36 J Fowler and CL Galli, “EUROTOX’s view regarding the role and training of certified European registered toxicologists (ERT)” (2007) 168 Toxicology Letters 192.
37 Supra, note 6.
38 European Food Safety Authority, “Risk Assessment vs. Risk Management: What’s the Difference?” (Food Safety News, 23 April 2014) <https://www.foodsafetynews.com/2014/04/risk-assessment-vs-risk-management-whats-the-difference/> (last accessed 19 March 2021).
39 Commission Regulation (EU) No. 546/2011 implementing Regulation (EC) No 1107/2009 of the European Parliament and of the Council as regards uniform principles for evaluation and authorization of plant protection products. [2011] OJ L 155/127.
40 See, eg, E Ingre-Khans, M Agerstrand, A Beronius and C Ruden, “Transparency of chemical risk assessment data under REACH” (2016) 18 Environmental Science. Processes & Impacts 1508.
41 European Commission, Consumers, Health, Agriculture and Food Executive Agency <https://ec.europa.eu/chafea/about/who-we-are_en.htm> (last accessed 19 March 2021).
42 Regulation (EC) No 1107/2009, supra, note 6.
43 Regulation (EU) No 528/2012, supra, note 6.
44 European Parliament resolution of 16 January 2019 on the Union’s authorization procedure for pesticides (2018/2153(INI)).
45 European Commission, “Guidance document on zonal evaluation and mutual recognition under Regulation (EC) No 1107/2009.” SANCO/13169/2010 rev. 11 (January 2021).
46 M Stenrød, M Almvik, OM Eklo, AL Gimsing, R Holten, K Künnis-Beres et al, “Pesticide regulatory risk assessment, monitoring, and fate studies in the northern zone: recommendations from a Nordic–Baltic workshop” (2016) 23 Environmental Science and Pollution Research 15779.
47 See, eg, OECD, “Guidance Document on the Planning and Implementation of Joint Reviews of Pesticides” (2011) Series on Pesticides No. 60 ENV/JM/MONO (2011)11. Organisation for Economic Co-Operation and Development. Environment Directorate, Joint Meeting of the Chemicals Committee and The Working Party on Chemicals, Pesticides and Biotechnology, 54 pp.
48 A Smith, L Parrino, D Vrbos, G Nicolini, M Bucchi, M Carr et al, “Communicating and engaging with the public in regulatory science” (2019) 17(S1) EFSA Journal e170717.
49 Regulation (EU) 2019/1381 of the European Parliament and of the Council of 20 June 2019 on the transparency and sustainability of the EU risk assessment in the food chain and amending Regulations (EC) No 178/2002, (EC) No 1829/2003, (EC) No 1831/2003, (EC) No 2065/2003, (EC) No 1935/2004, (EC) No 1331/2008, (EC) No 1107/2009, (EU) 2015/2283 and Directive 2001/18/EC (text with EEA relevance). [2019] OJ L 231.
50 European Chemicals Agency, “Review of the Policy of avoiding potential conflicts of interest”. 51st Meeting of the Management Board 0-21 September 2018. MB/40/2018 final, 21.9.2018. Public, 5 pp. <https://echa.europa.eu/documents/10162/3430273/FINAL_MB_40_2018_%281%29_Revision_CoI_policy_MB51.pdf/4a14e991-89ab-52ec-67b6-e9e93181e738> (last accessed 19 March 2021); European Food Safety Authority, “EFSA’s policy on independence. How the European Food Safety Authority assures the impartiality of professionals contributing to its operations.” Adopted in Parma on 21 June 2017. mb170621-a2, 9 pp. <https://www.efsa.europa.eu/sites/default/files/corporate_publications/files/policy_independence.pdf> (last accessed 19 March 2021).
51 Regulation (EC) No 178/2002 of the European Parliament and of the Council laying down the general principles and requirements of food law, establishing the European Food Safety Authority and laying down procedures in matters of food safety of 28 January 2002, OJ L 031.
52 M Blastland, ALJ Freeman, S van der Linden, TM Marteau and D Spiegelhalter, “Five rules for evidence communication” (2020) 587 Nature 362.
53 Directive (EU) 2016/943 of the European Parliament and of the Council on the Protection of Undisclosed Know-How and Business Information (Trade Secrets) Against Their Unlawful Acquisition, Use and Disclosure. [2016] OJ L 157/1.
54 See, eg, C Robinson, CJ Portier, A Čavoški, R Mesnage, A Roger, P Clausing et al, “Achieving a high level of protection from pesticides in Europe: problems with the current risk assessment procedure and solutions” (2020) 11 European Journal of Risk Regulation 450.
55 P McGrath, “Politics meets Science: The case of neonicotinoid insecticides in Europe” (2014) 7(1) S.A.P.I.EN.S Surveys and Perspectives Integrating Environment and Society <http://journals.openedition.org/sapiens/1648> (last accessed 19 March 2021).
56 The United Nations Economic Commission for Europe (UNECE) Convention on Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters (The Aarhus Convention) [1998] <https://ec.europa.eu/environment/aarhus/index.htm> (last accessed 19 March 2021).
57 General Court of the European Union, “EFSA’s decisions refusing access to the toxicity and carcinogenicity studies on the active substance glyphosate are annulled.” PRESS RELEASE No 25/19, Luxembourg, 7 March 2019 <https://curia.europa.eu/jcms/upload/docs/application/pdf/2019-03/cp190025en.pdf> (last accessed 19 March 2021).
58 Y Schreuder, “The Polder Model in Dutch economic and environmental planning” (2001) 21(4) Bulletin of Science, Technology & Society 237.
59 W Ulrich, “Reflective practice in the civil society: the contribution of critically systemic thinking” (2000) 1(2) Reflective Practice 247.
60 I Devos, KC Elliott and A Hardy (eds.), “Proceedings of the 3rd EFSA International Conference 2018: Science, Food, Society” (2019). 17 EFSA Journal Special Issue S1 <https://efsa.onlinelibrary.wiley.com/toc/18314732/2019/17/S1> (last accessed 19 March 2021).
61 UNECE, “United Nations Economic Commission for Europe Evaluation Policy” (2014) <http://www.unece.org/fileadmin/DAM/OPEN_UNECE/03_Evaluation_and_Audit/UNECE_Evaluation_Policy_October_2014.pdf> (last accessed 19 March 2021).
62 MM Mark and GT Henry, “The mechanisms and outcomes of evaluation influence” (2004) 10(1) Evaluation 35.
63 Food and Veterinary Office of the European Commission <https://ec.europa.eu/food/audits-analysis/audit_reports/index.cfm> (last accessed 19 March 2021).
64 Koëter et al., Report of the second visitation, supra, note 26, Annex 7, pp 98–103.