Hostname: page-component-848d4c4894-mwx4w Total loading time: 0 Render date: 2024-06-29T18:19:25.594Z Has data issue: false hasContentIssue false
  • English
  • Français

Evaluation in the European Commission: For Accountability or Learning?

Published online by Cambridge University Press:  20 January 2017

Steven Højlund
Steven Højlund*
Affiliation:
Copenhagen Business School – Department of Business and Politics
Get access Rights & Permissions [Opens in a new window]

Abstract

The article accounts for the development of the European Commission's evaluation system. The article shows how internal and external developments shape an evaluation system aiming for both accountability and policy learning. In 58 interviews, several justifications for the evaluation system emerge, including four types of accountability as well as an evaluation system constructed to facilitate learning from past experience. In the system's commencement, financial and legal accountability overshadow the focus on policy learning that was perceived to be in contradiction with the two former. However, the article also demonstrates that accountability and policy learning are not necessarily opposed to each other.

Type
Symposium on Policy Evaluation in the EU
Information
European Journal of Risk Regulation , Volume 6 , Issue 1 , March 2015 , pp. 35 - 46
Copyright
Copyright © Cambridge University Press 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1 Scriven, M, ‘Beyond Formative and Summative Evaluation’ in McLauglin M and Philips DC (eds), Evaluation and Education: At Quarter Century (3 edn, University of Chicago Press 1991)Google Scholar

2 Gornitzka Å and Sverdrup U, ‘Access of experts: information and EU decision–making’ (2011) 34 West European Politics 1452. Smith AS, ‘How the European Commission's Policies Are Made: Problematization, Instrumentation and Legitimation’ [2013] Journal of European Integration

3 Stern E, Evaluation policy in the European Union and its institutions, vol 123 (Trochim WMK, Mark MM and Cooksy LJ eds, 2009)

4 Furubo J–E, Rist RC and Sandahl R, International Atlas of Evaluation (Furubo J–E, Rist RC and Sandahl R eds, Transaction Publishers. 2002)

5 Hoerner J and Stephenson P, ‘Theoretical Perspectives on Approaches to Policy Evaluation in the EU: The Case of Cohesion Policy’ (2012) 90 Public Administration 699

6 Leeuw FL and Furubo J–E, ‘Evaluation Systems : What Are They and Why Study Them?’ (2008) 14 Evaluation 157

7 Jacobsen Sven, “Evalueringsaktiviteter & –resultater på tværs af Europa–Kommissions tjenestegrene”, 2007

8 Communication to the Commission from MS GryBauskaite in agreement with the President. Responding to Strategic Needs: Reinforcing the use of evaluation, SEC(2007)213, at p. 17.

9 Scriven M, ‘Beyond Formative and Summative Evaluation’ in McLauglin M and Philips DC (eds), Evaluation and Education: At Quarter Century (3 edn, University of Chicago Press 1991). Albæk E, ‘Between Knowledge and Power: Utilization of Social Science in Public Policy Making’ (1995) 28 Policy Sciences 79.

10 M, Power, ‘The Theory of Audit Explosion’ in Ferlie E, Lynn LE and Pollitt C (eds), Oxford Handbook of Public Management (Oxford University Press 2005)Google Scholar. M, Boven, ‘Public Accountability’ in E, Ferlie, LE, Lynn and C, Pollitt (eds), The Oxford Handbook of Public Management (Oxford University Press 2005). M, Dubnick, ‘Accountability and the promise of performance – In search of the Mechanisms’ (2005)Google Scholar 28 Public Performance & Management Review 376.

11 Boswell C, ‘The political functions of expert knowledge: knowledge and legitimation in European Union immigration policy’ (2008) 15 Journal of European Public Policy 471. Böhling K, ‘Sidelined Member States: Commission learning from Experts in the Face of Comitology’ [2013] Journal of European Integration. Mendez C and Bachtler J, ‘Administrative reform and unintended consequences: an assessment of the EU Cohesion policy ‘audit explosion'’ 18 Journal of European Public Policy 746

12 Boven, Public Accountability, supra note 10.

13 M, Scriven, ‘The methodology of evaluation’ in Stake RE (ed), Curriculum evaluation (Rand McNully 1967)Google Scholar. Scriven, Beyond Formative and Summative Evaluation, supra note 8. Chen HT, ‘A comprehensive typology for program evaluation’ (1996) 17 Evaluation Practice 121.

14 Torres RT and Preskill H, ‘Evaluation and Organizational Learning: Past, Present, and Future’ (2001) 22 American Journal of Evaluation 387. Weiss CH, ‘Have We Learned Anything New About the Use of Evaluation’ (1998) 19 American Journal of Evaluation 21. Balthasar A, ‘Institutional Design and Utilization of Evaluation – A Construction to a Theory of Evaluation Influence Based on Swiss Experience’ (2009) 33 Evaluation Review 226. Chen, A comprehensive typology.

15 McDavid JC, Huse I and Hawthorn LR, Program Evaluation and Performance Measurement (Sage Publications 2013). Scriven, The methodology of evaluation, supra note 13. Scriven, Beyond Formative and Summative Evaluation, supra note 8.

16 Dahler–Larsen P, The Evaluation Society(Stanford Press 2012). Højlund S, ‘Evaluation use in the organisational context – Changing focus to improve theory’ 20 Evaluation 26.

17 Meyer JW and Rowan B, ‘Institutionalized organizations: Formal structure as Myth and Ceremony’ 83 American Sociological Review 340.

18 Shulha LM and Cousins B, ‘Evaluation Use: Theory, Research and Practice Since 1986’ (1997) 18 Evaluation Practice 195. Preskill H and Boyle S, ‘A multidisciplinary model of evaluation capacity building. ‘ (2008) 29 American Journal of Evaluation 443.

19 See for example Balthasar A and Rieder S, ‘Learning from Evaluations: Effects of the Evaluation of hte Swiss Energy 2000 Programme’ 6 Evaluation 245. Weiss CH, ‘Have We Learned Anything New About the Use of Evaluation’ (1998) 19 American Journal of Evaluation 21. MQ, Patton, Utilization–focused evaluation: The new century text (3 edn, Sage 1997).Google Scholar Alkin MC and Stecher B, ‘Evaluation in Context: Information Use in Elementary School Decision Making’ (1983) 9 Studies in Educational Evaluation 23. Finne H, Levine M and Nilssen T, ‘Trailing research: A model for useful program evaluation’ (1995) 1 Evaluation 11.

20 Alkin M and Taut S, ‘Unbundling Evaluation Use’ (2003) 29 Studies in Educational Evaluation 1

21 Schofield J, ‘A model of learned implementation’ (2004) 82 Public Administration 283. Bennett CJ and Howlett M, ‘The lessons of learning: Reconciling theories of policy learning and policy change’ (1992) 25 Policy Sciences 275.

22 Radaelli CM and Dunlop CA, ‘Learning in the European Union: theoretical lenses and meta–theory’ (2013) 20 Journal of European Public Policy 923.

23 E, Vedung, Public Policy and Program Evaluation (NJ: Transaction Publishers 1997), at p. 3.Google Scholar

24 Boven, Public Accountability, supra note 10.

25 C, Harlow, Accountability in the European Union (Oxford University Press 2002).Google Scholar

26 Laffan B, ‘Auditing and accountability in the European Union’ (2003) 10 Journal of European Public Policy 762.

27 Wille A, ‘Political–Bureaucratic Accountability in the EU Commission: Modernising the Executive’ (2010) 33 West European Politics 1093.

28 For similar approach to studies of the Commission see for example Wonka A, ‘Decision–making dynamics in the European Commission: partisan, national or sectoral?’ (2008) 15 Journal of European Public Policy 1145. Schout A, ‘Organizational learning in the EU's multilevel–governance system’ (2009) 16 Journal of European Public Policy 1124. Smith, How the European Commission’s Policies Are Made, supra note 2. Böhling, Sidelined Member States, supra note 11. Burns C, ‘How and When Did We Get Here? An Historical Institutionalist Analysis of EU Biotechnology Policy’ (2012) 34 Journal of European Integration 341.

29 P, Bazeley, Qualitative Data Analysis with NVIVO (Sage Publications Ltd. 2013).Google Scholar

30 M, Schreier, Qualitative Content Analysis in Practice (Sage Publications Ltd. 2012)Google Scholar. P, Mayring, ‘Qualitative Content Analysis’ in Flick U, Von Kardorff E and Steinke I (eds), A Companion to Qualitative Research (SAGE 2004).Google Scholar Kohlbacher F, ‘The Use of Qualitative Content Analysis in Case Study Research’ (2006) 7 Forum: Qualitative Social Research.

31 H, Summa and J, Toulemonde, ‘Evaluation in the European Union: addressing complexity and ambiguity’ in Furubo J–E, Sandahl R and Rist R (eds), International Atlas of Evaluation (Transaction Publishers 2002), at p. 409.Google Scholar

32 Christiansen T, ‘Tensions of European governance: politicized bureaucracy and multiple accountability in the European Commission’ (1997) 4 Journal of European Public Policy 73. Ellinas A and Suleiman E, ‘Reforming the Commission: between modernization and bureaucratization’ (2008) 15 Journal of European Public Policy 708.

33 Summa, Evaluation in the European Union, supra note 31, at p. 412. Laffan B, ‘From policy entrepreneur to policy manager: the challenge facing the European Commission’ (1997) 4 Journal of European Public Policy 422.

34 Eser TW and Nussmueller E, ‘Mid–term Evaluations of Community Initiatives under European Union Structural Funds: A Process between Accounting and Common Learning’ (2006) 40 Regional Studies 249.

35 C, Pollitt and G, Bouckaert, Public Management Reform: A Comparative Analysis (2nd edn, 2004)Google Scholar. Ellinas, Reforming the Commission, Supra note 32.

36 The Commission's Communication 1996 ‘Concrete steps towards best practice across the Commission’. Communication to the Commission by Mr Liikanen and Mrs Gradin, in agreement with the President, SEC(1996)659.

37 Commission, Concrete steps, supra note 36, at p. 1.

38 Council Regulation (EC, Euratom, ECSC) No 2335/95 of 18 September 1995.

39 Commission, Concrete steps, supra note 36. B, Laffan, The Finances of the European Union (Macmillan 1997)Google Scholar. Laffan. From policy entrepreneur to policy manager, supra note 33.

40 Commission, Concrete steps, supra note 36.

41 Williams K, Laat Bd and Stern E, The Use of Evaluation in the Commission Services(2002).

42 Commission, Concrete steps, supra note 36.

43 Summa, Evaluation in the European Union, supra note 31, at p. 420.

44 Eser, Mid–term Evaluations, supra note 34.

45 See also Højlund S, ‘Evaluation use in evaluation systems – the case of the European Commission’ (2014) 20 Evaluation 428.

46 Ex ante evaluation is conducted about two years ahead of programme start. Midterm evaluation is conducted three years into programme implementation, final evaluation is conducted at programme termination, and ex post evaluation is conducted two years after programme termination. Effectively, the evaluation cycle is therefore not seven years but eleven years and overlapping each other. Including ex ante/IA, there are effectively 2–3 years between evaluations of a programme.

47 The Commission's Communication 1999, “Spending more wisely: Implementation of the Commission's evaluation policy”, SEC(1999)69/4.

48 See Laffan, The Finances of the European Union, supra note 39.

49 J, Toulemonde, H, Summa and N, Usher, ‘Assessing EU Evaluations’ in Schwartz R and Mayne J (eds), Quality matters: seeking confidence in evaluation, auditing and performance reporting (Transaction Publishers 2005), at p. 82.Google Scholar

50 Spence D, ‘Plus ca change, plus c'est la meme chose? Attempting to reform the European Commission’ (2000) 7 Journal of European Public Policy 1.

51 Commission, Reforming the Commission: A White Paper (2000). Kassim H, ‘'Mission impossible', but mission accomplished: the Kinnock reforms and the European Commission’ (2008) 15 Journal of European Public Policy 648. Schön-Quinlivan E, ‘Implementing organizational change – the case of the Kinnock reforms’ (2008) 15 Journal of European Public Policy 726.

52 Bauer M, ‘Introduction: Organizational change, management reform and EU policy-making’ (2008) 15 Journal of European Public Policy. Mendez, Administrative reform and unintended consequences, supra note 11.

53 Communication to the Commission from Mrs. Schreyer in agreement with Mr. Kinnock and the President, “Focus on results: Strengthening evaluation of Commission activities”, SEC(2000)1051.

54 Toulemonde, Assessing EU Evaluations, supra note 49.

55 Commission, Responding to Strategic Needs, Supra note 8.Commission, Revision of the Internal Control Standards and Underlying Framework – Strengthening Contol Effectiveness (European Commission 2007).

56 Toulemonde, Assessing EU Evaluations, supra note 49, at p. 78.

57 Ellinas, Reforming the Commission, Supra note 32. Mendez, Administrative reform and unintended consequences, supra note 11.

58 Toulemonde J, ‘Evaluation culture(s) in Europe: differences and convergence between national practices’ (2000) 69 Vierteljahrshefte zur Wirtschaftsforschung 350. Batterbury SCE, ‘Principles and purposes of European Union policy evaluation’ (2006) 40 Regional Studies 179. Eser, Mid–term Evaluations, supra note 34.

59 Williams, The Use of Evaluation in the Commission Services, supra note 41.

60 See executive summary in Laat Bd, Study on the Use of Evaluation Results in the Commission (2005).

61 Ecorys and COWI, Study on the state of implementation of Activity Based Management in the European Commission (2008).

62 Kassim, Mission impossible, supra note 51. OECD, Regulatory policies in OECD Counctires: From Interventionism to Regulatory Governance (2002). Francesco FD, Radaelli CM and Troeger VE, ‘Implementing regulatory innovations in Europe: the case of impact assessment’ (2011) 19 Journal of European Public Policy 491.

63 Radaelli CM and Meuwese ACM, ‘Hard Questions, Hard Solutions: Proceduralistion through Impact Assessment in the EU’ (2010) 20 West European Politics 923.

64 Communication from the Commission to the European Parliament, the Council, the European economic and social Committee and the Committee of the regions Commission, Smart Regulation in the European Union, COM(2010)543.

65 Commission, Responding to Strategic Needs, Supra note 8.

66 Commission, Responding to Strategic Needs, Supra note 8.

67 Commission, Responding to Strategic Needs, Supra note 8.

68 Laat, Study on the Use of Evaluation Results in the Commission, supra note 60. Williams, The Use of Evaluation in the Commission Services, supra note 41. European Court of Auditors, Presidential Letter (2005).

69 European Commission, Public Consultation on Commission Guidelines for Evaluation (European Commission 2013).