Skip to main content Accessibility help
×
Home
Hostname: page-component-544b6db54f-vq995 Total loading time: 0.226 Render date: 2021-10-20T19:16:45.412Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Assessing assessors: proposal for a guidance for evaluating the scientific performance of a pesticide regulatory authority

Published online by Cambridge University Press:  16 April 2021

Sari AUTIO
Affiliation:
Safety and Chemicals Agency Tukes, Helsinki, Finland; email: sari.autio@tukes.fi.
Herman KOËTER
Affiliation:
Orange House Partnership, Brussels, Belgium.
Mar CARRETERO
Affiliation:
Regional Public Health Laboratory, Madrid, Spain.
Anthony HARDY
Affiliation:
Grayshott, UK.
Alberto MANTOVANI
Affiliation:
Istituto Superiore di Sanità, Rome, Italy.

Abstract

External evaluations by an international committee of the scientific risk assessment and decision-making processes of the Netherlands Competent Authority for the authorisation of plant protection products and biocides (Ctgb) were conducted in 2013 and 2018. Based on the experience drawn from them, a general guidance for such visitations is suggested. An international visitation committee examined the structure and management of the Authority, its human resources and staff policy, the scientific processes and output, the documentation and communication of its decisions and the mechanisms for keeping up to date with international scientific developments. Attention was paid to the degree of openness and transparency throughout the organisation and in particular when dealing with confidential information. From the experience gained it can be concluded that visitations not aiming at finding mistakes and omissions but instead focusing on recommendations and constructive suggestions will result in cooperation, mutual trust and acceptance of the recommendations made. A follow-up visitation after a few years can be effective in maintaining a traceable, high-level scientific output. In view of the strong drive towards the European Union-wide harmonisation of the regulatory practices of hazardous chemicals, a voluntary evaluation of regulatory authorities’ scientific performance is recommended as a means for organisational learning.

Type
Articles
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The international visitation committee acknowledge the support and assistance offered by all Board for the Authorisation of Plant Protection Products and Biocides (Ctgb) staff members and would like to thank them all for their patience and willingness to spend their precious time with us for interviews and technical help. In particular, we would like to thank the chairman and the members of the board, as well as the director of the Ctgb, for their trust in the capacity, capability and expertise of the international visitation committee membership. We are also grateful to the project manager for providing us with all necessary documents and information. He has been our beacon, always patient and tireless, when some of us got lost in the sea of documents. Finally, we thank the unknown peer reviewer for suggesting significant improvements to this article. The authors declare that they have no competing interests.

References

1 T Tweedale, A Lysimachou and H Muilerman, “Missed & dismissed, pesticide regulators ignore the legal obligation to use independent science for deriving safe exposure levels” (2014) Report published by Pesticide Action Network Europe and Generation Futures, 29 pp.

2 O Renn, Risk Governance. Coping with Uncertainty in a Complex World (London, Earthscan, 2008) pp 222–27.

3 O Nesheim, F Frederick Fishel and M Mossler, “Toxicity of pesticides” (2019) EDIS 2005 (8). <https://journals.flvc.org/edis/article/view/114995> (last accessed 19 March 2021).

4 See, eg, VL Zikankuba, G Mwanyika, JE Ntwenya and A James, “Pesticide regulations and their malpractice implications on food and environment safety” (2019) 5 Cogent Food & Agriculture 1601544.

5 T Hardy, S Bopp, M Egsmose, H Fontier, L Mohimont, H Steinkellner and F Streissl, “Risk assessment of plant protection products” (2012) 10(10) EFSA Journal S1010.

6 Regulation (EC) No 1107/2009 of the European Parliament and of the Council of 21 October 2009 concerning the placing of plant protection products on the market and repealing Council Directives 79/117/EEC and 91/414/EEC. OJ L 309; Regulation (EU) No 528/2012 of the European Parliament and of the Council of 22 May 2012 concerning the making available on the market and use of biocidal products. OJ L 167.

7 GW Suter II (ed.), Ecological Risk Assessment (Boca Raton, FL, Lewis Publishers 1993).

8 Renn, supra, note 2, pp 239–41.

9 SCS Clahsen, I van Kemp, BC Hakkert, TG Vermeire, AH Piersma and E Lebret, “Why do countries regulate environmental health risks differently? A theoretical perspective” (2019) 39(2) Risk Analysis 439.

10 BB Johnson and B Swedlow, “Cultural theory’s contributions to risk analysis: a thematic review with directions and resources for further research” (2021) 41(3) Risk Analysis 429.

11 RL Anjum and E Rocca, “From ideal to real risk: philosophy of causation meets risk analysis” (2019) 39(3) Risk Analysis 729.

12 European Commission, “Pesticides” <https://ec.europa.eu/food/plant/pesticides_en> (last accessed 19 March 2021).

13 European Commission, “Biocides” <https://ec.europa.eu/health/biocides/overview_en> (last accessed 19 March 2021).

14 D Wintermantel, JF Odoux, A Decourtye, M Henry, F Allier and V Bretagnolle, “Neonicotinoid-induced mortality risk for bees foraging on oilseed rape nectar persists despite EU moratorium” (2020) 704 Science of the Total Environment 135400.

15 OECD, “Guidance Document on the Planning and Implementation of Joint Reviews of Pesticides” (2011) Series on Pesticides No. 60 ENV/JM/MONO (2011)11. Organisation for Economic Co-Operation and Development. Environment Directorate, Joint Meeting of the Chemicals Committee and The Working Party on Chemicals, Pesticides and Biotechnology, 54 pp.

16 S Koch, A Epp, M Lohmann & G-F Böl, “Pesticide residues in food: attitudes, beliefs, and misconceptions among conventional and organic consumers” (2017) 80(12) Journal of Food Protection 2083.

17 A Mie, C Rudén and P Grandjean, “Safety of safety evaluation of pesticides: developmental neurotoxicity of chlorpyrifos and chlorpyrifos-methyl” (2018) 17 Environmental Health 77.

18 European Commission, “Farm to Fork Strategy for a fair, healthy and environmentally-friendly food system” (2020) <https://ec.europa.eu/food/sites/food/files/safety/docs/f2f_action-plan_2020_strategy-info_en.pdf> (last accessed 19 March 2021).

19 H Deluyker, A Rodrıguez Pena, M Scannell, J Tarazona and B Url, “What does the future hold for assessment science?” (2016) 14(S1) EFSA Journal S0501.

20 Food and Veterinary Office of the European Commission (2020) <https://ec.europa.eu/food/audits-analysis/audit_reports/index.cfm> (last accessed 19 March 2021).

21 S Kuhlmann and J Bogumil, “Performance measurement and benchmarking as ‘reflexive institutions’ for local governments: Germany, Sweden and England compared” (2018) 31(4) International Journal of Public Sector Management 543; C Luederitz, N Schapke, A Wiek, DJ Lang, M Bergmann, JJ Bos et al, “Learning through evaluation – a tentative evaluative scheme for sustainability transition experiments” (2017) 169 Journal of Cleaner Production 61.

22 The Decree on Mandate, Authorization and Representation by the Ctgb, 3 March 2011, Government Gazette No 4789, the Netherlands, 18 March 2011.

23 SI Donaldson and M Scriven (eds.), Evaluating Social Programmes and Problems. Visions for the New Millennium. The Stauffer Symposium on Applied Psychology at the Claremont Colleges (Mahwah, NJ, Lawrence Erlbaum Associates Publishers, 2003).

24 IF Shaw, JC Greene and MM Mark (eds.), Handbook of Evaluation: Policies, Programmes and Practices (Thousand Oaks, CA, SAGE Publications 2006).

25 DW Compton and ML Baizerman, “The evolution of a philosophy and practice of evaluation advice” (2012) 136 New Directions for Evaluation 67.

26 H Koëter, S Autio, U Banasiak, M Lynch, V Silano and A Cuvillier, Report on the international visitation of the Board for the authorization of plant protection products and biocides (Ctgb) in the Netherlands addressing the scientific process, the scientific output and the decision-making process (Lucca, Orange House Partnership, 2013); H Koëter, S Autio, M Carretero, T Hardy and A Mantovani, Report of the second visitation of the Netherlands Board for the Authorization of Plant Protection Products and Biocides (Ctgb) addressing the scientific process, the scientific output and the decision-making process. (Lucca, Orange House Partnership, 2018).

27 E Desmedt, D Morin, V Pattyn and M Brans, “Impact of performance audit on the administration: a Belgian study (2005–2010)” (2017) 32(3) Managerial Auditing Journal 251.

28 J Lyytimäki, T Assmuth and M Hildén, “Communicating chemical risks for social learning: findings from an expert opinion survey” (2009) 8(3–4) Applied Environmental Education & Communication 174.

29 S Eckhard and V Jankauskas, “The politics of evaluation in international organizations: a comparative study of stakeholder influence potential” (2019) 25(1) Evaluation 62.

30 C Donovan, “State of the art in assessing research impact: introduction to a special issue” (2011) 20(3) Research Evaluation 175.

31 R Lahey and S Nielsen, “Rethinking the relationship among monitoring, evaluation, and results-based management: observations from Canada” (2013) 137 New Directions for Evaluation 45.

32 B Wolf, T Lindenthal, M Szerencsits, JB Holbrook and J Heß, “Evaluating research beyond scientific impact. How to include criteria for productive interactions and impact on practice and society” (2013) 22(2) GAIA 104.

33 See, eg, American Evaluation Association, AEA Guiding Principles. Public Statement on Cultural Competence in Evaluation (Washington, DC, American Evaluation Association 2011).

34 S Bell, B Shaw and A Boaz, “Real-world approaches to assessing the impact of environmental research on policy” (2011) 20(3) Research Evaluation 227.

35 H Selck, PB Adamsen, T Backhaus, GT Banta, PKH Bruce, G Allen Burton Jr et al, “Assessing and managing multiple risks in a changing world – the Roskilde recommendations” (2017) 36 Environmental Toxicology and Chemistry 7.

36 J Fowler and CL Galli, “EUROTOX’s view regarding the role and training of certified European registered toxicologists (ERT)” (2007) 168 Toxicology Letters 192.

37 Supra, note 6.

38 European Food Safety Authority, “Risk Assessment vs. Risk Management: What’s the Difference?” (Food Safety News, 23 April 2014) <https://www.foodsafetynews.com/2014/04/risk-assessment-vs-risk-management-whats-the-difference/> (last accessed 19 March 2021).

39 Commission Regulation (EU) No. 546/2011 implementing Regulation (EC) No 1107/2009 of the European Parliament and of the Council as regards uniform principles for evaluation and authorization of plant protection products. [2011] OJ L 155/127.

40 See, eg, E Ingre-Khans, M Agerstrand, A Beronius and C Ruden, “Transparency of chemical risk assessment data under REACH” (2016) 18 Environmental Science. Processes & Impacts 1508.

41 European Commission, Consumers, Health, Agriculture and Food Executive Agency <https://ec.europa.eu/chafea/about/who-we-are_en.htm> (last accessed 19 March 2021).

42 Regulation (EC) No 1107/2009, supra, note 6.

43 Regulation (EU) No 528/2012, supra, note 6.

44 European Parliament resolution of 16 January 2019 on the Union’s authorization procedure for pesticides (2018/2153(INI)).

45 European Commission, “Guidance document on zonal evaluation and mutual recognition under Regulation (EC) No 1107/2009.” SANCO/13169/2010 rev. 11 (January 2021).

46 M Stenrød, M Almvik, OM Eklo, AL Gimsing, R Holten, K Künnis-Beres et al, “Pesticide regulatory risk assessment, monitoring, and fate studies in the northern zone: recommendations from a Nordic–Baltic workshop” (2016) 23 Environmental Science and Pollution Research 15779.

47 See, eg, OECD, “Guidance Document on the Planning and Implementation of Joint Reviews of Pesticides” (2011) Series on Pesticides No. 60 ENV/JM/MONO (2011)11. Organisation for Economic Co-Operation and Development. Environment Directorate, Joint Meeting of the Chemicals Committee and The Working Party on Chemicals, Pesticides and Biotechnology, 54 pp.

48 A Smith, L Parrino, D Vrbos, G Nicolini, M Bucchi, M Carr et al, “Communicating and engaging with the public in regulatory science” (2019) 17(S1) EFSA Journal e170717.

49 Regulation (EU) 2019/1381 of the European Parliament and of the Council of 20 June 2019 on the transparency and sustainability of the EU risk assessment in the food chain and amending Regulations (EC) No 178/2002, (EC) No 1829/2003, (EC) No 1831/2003, (EC) No 2065/2003, (EC) No 1935/2004, (EC) No 1331/2008, (EC) No 1107/2009, (EU) 2015/2283 and Directive 2001/18/EC (text with EEA relevance). [2019] OJ L 231.

50 European Chemicals Agency, “Review of the Policy of avoiding potential conflicts of interest”. 51st Meeting of the Management Board 0-21 September 2018. MB/40/2018 final, 21.9.2018. Public, 5 pp. <https://echa.europa.eu/documents/10162/3430273/FINAL_MB_40_2018_%281%29_Revision_CoI_policy_MB51.pdf/4a14e991-89ab-52ec-67b6-e9e93181e738> (last accessed 19 March 2021); European Food Safety Authority, “EFSA’s policy on independence. How the European Food Safety Authority assures the impartiality of professionals contributing to its operations.” Adopted in Parma on 21 June 2017. mb170621-a2, 9 pp. <https://www.efsa.europa.eu/sites/default/files/corporate_publications/files/policy_independence.pdf> (last accessed 19 March 2021).

51 Regulation (EC) No 178/2002 of the European Parliament and of the Council laying down the general principles and requirements of food law, establishing the European Food Safety Authority and laying down procedures in matters of food safety of 28 January 2002, OJ L 031.

52 M Blastland, ALJ Freeman, S van der Linden, TM Marteau and D Spiegelhalter, “Five rules for evidence communication” (2020) 587 Nature 362.

53 Directive (EU) 2016/943 of the European Parliament and of the Council on the Protection of Undisclosed Know-How and Business Information (Trade Secrets) Against Their Unlawful Acquisition, Use and Disclosure. [2016] OJ L 157/1.

54 See, eg, C Robinson, CJ Portier, A Čavoški, R Mesnage, A Roger, P Clausing et al, “Achieving a high level of protection from pesticides in Europe: problems with the current risk assessment procedure and solutions” (2020) 11 European Journal of Risk Regulation 450.

55 P McGrath, “Politics meets Science: The case of neonicotinoid insecticides in Europe” (2014) 7(1) S.A.P.I.EN.S Surveys and Perspectives Integrating Environment and Society <http://journals.openedition.org/sapiens/1648> (last accessed 19 March 2021).

56 The United Nations Economic Commission for Europe (UNECE) Convention on Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters (The Aarhus Convention) [1998] <https://ec.europa.eu/environment/aarhus/index.htm> (last accessed 19 March 2021).

57 General Court of the European Union, “EFSA’s decisions refusing access to the toxicity and carcinogenicity studies on the active substance glyphosate are annulled.” PRESS RELEASE No 25/19, Luxembourg, 7 March 2019 <https://curia.europa.eu/jcms/upload/docs/application/pdf/2019-03/cp190025en.pdf> (last accessed 19 March 2021).

58 Y Schreuder, “The Polder Model in Dutch economic and environmental planning” (2001) 21(4) Bulletin of Science, Technology & Society 237.

59 W Ulrich, “Reflective practice in the civil society: the contribution of critically systemic thinking” (2000) 1(2) Reflective Practice 247.

60 I Devos, KC Elliott and A Hardy (eds.), “Proceedings of the 3rd EFSA International Conference 2018: Science, Food, Society” (2019). 17 EFSA Journal Special Issue S1 <https://efsa.onlinelibrary.wiley.com/toc/18314732/2019/17/S1> (last accessed 19 March 2021).

61 UNECE, “United Nations Economic Commission for Europe Evaluation Policy” (2014) <http://www.unece.org/fileadmin/DAM/OPEN_UNECE/03_Evaluation_and_Audit/UNECE_Evaluation_Policy_October_2014.pdf> (last accessed 19 March 2021).

62 MM Mark and GT Henry, “The mechanisms and outcomes of evaluation influence” (2004) 10(1) Evaluation 35.

63 Food and Veterinary Office of the European Commission <https://ec.europa.eu/food/audits-analysis/audit_reports/index.cfm> (last accessed 19 March 2021).

64 Koëter et al., Report of the second visitation, supra, note 26, Annex 7, pp 98–103.

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Assessing assessors: proposal for a guidance for evaluating the scientific performance of a pesticide regulatory authority
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Assessing assessors: proposal for a guidance for evaluating the scientific performance of a pesticide regulatory authority
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Assessing assessors: proposal for a guidance for evaluating the scientific performance of a pesticide regulatory authority
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *