The current rapid decline in biodiversity in human-dominated agricultural landscapes, both in Europe and worldwide, impacts on the provision of environmental services essential to human well-being. There is, therefore, a pressing need to develop and implement incentive-based conservation policies to counteract the ongoing loss of biodiversity. This paper presents results of a regionally-scaled conservation procurement auction, a type of incentive-based payments for environmental services (PES), targeted at the conservation of arable plant diversity. By matching arable fields that were participating in the PES scheme to control fields that were not enrolled in the PES scheme, two critical key characteristics were addressed, namely additionality and bid prices. Additionality was addressed by evaluating whether fields for which PES were issued had significantly higher arable plant diversity than the matched control fields. The cost-effectiveness of a conservation auction increases if payments compensate just farmers’ opportunity costs (in terms of forgone production); bid prices of participating farmers were thus also evaluated to determine whether they were related to their individual opportunity costs. The PES scheme proved to be highly effective in ensuring environmental services delivery through enhanced arable plant diversity on participating fields. In contrast, the potential of the proposed conservation auction design to raise cost-effectiveness has to be questioned, because bid prices submitted in this scheme substantially exceeded individual farmers’ opportunity costs. Therefore, bid prices were most likely influenced by socioeconomic factors other than opportunity costs. This case study illustrates potentials and pitfalls associated with the implementation of a PES scheme and, by evaluating the effectiveness of the scheme, contributes to an improved understanding of incentive-based mechanisms for both policymakers and practitioners involved in PES scheme design and implementation.