13.2.1 Double Standards on the Export of Hazardous Products

The Western democratic states restricted the reach of the regulation of chemicals, pharmaceuticals, pesticides and dangerous technical goods to their territory, paving the way for their industries to export products to the rest of the world, although their use was prohibited or severely restricted in the home country. The phenomenon became known worldwide as the policy of ‘double standards’ and triggered political awareness around the globe, in the exporting and importing states, in international organizations and in what could be ambitiously called an emerging global society.Footnote ¹¹ Communication costs, however, determined the search for political solutions. It has to be recalled that until the 1980s telephone costs were prohibitive, fax did not yet exist and the only way to engage in serious exchange was to meet physically. The decrease in transportation costs rendered the international gathering possible. The level of action to deal with ‘double standards’ was first and foremost political.

The subject related international organizations, WHO with regard to pharmaceuticals, UNEP and FAO with regard to chemicals, pesticides, waste and the later abolished UN-CTC with regard to dangerous technical goods invested into the elaboration of international standards on what is meant to be ‘hazardous’ and equally pushed for international solutions tying the export of double-standard products to the ‘informed’ consent of the recipient states. Within the international organizations, the United States dominated the discussions and negotiations. That is why each and every search for a solution was guided by the attempt to seek the support of the United States, whose president was no longer Jimmy Carter but Ronald Reagan. At the time, the European Union (EU) was near to non-existent in the international sphere, as it had not yet gained the competence to act on behalf of the Member States or jointly with the Member States. The Member States were speaking for themselves, built around two camps: the hard-core free-trade apologists and the softer group of states that were ready to join forces with voices from what is called today the Global South, seeking a balance between free trade and labour, consumer and environmental protection. Typically, the controversies ended in soft law solutions, recommendations adopted by the international organizations if not unanimously but at the minimum with the United States abstaining.

There is a long way from the recommendations adopted in the mid-1980s and the Rotterdam Convention on the export of hazardous chemicals and pesticides adopted in 1998, which entered into force in 2004.Footnote ¹² On the bright side, there is definitely the simple fact that multilateralism was still regarded as the major and appropriate tool for what was recognized as a universal problem, calling for universal solutions. However, there is also a dark side to be taken into consideration. The UN organizations channelled the political debate on double standards, which was originally much more ambitious. NGOs, environmental and consumer organizations, and civil society activists were putting political pressure on the exporting countries to abolish the policy of double standards. The highly conflictual question then was and still is, ‘Is there a responsibility of the exporting state for the health and safety of the citizens of the recipient countries?’ Is there even a constitutional obligation of nation-states to exercise some sort of control over the activities of ‘their’ companies, who are operating from their Western Homebase in the rest of the world? How far does the responsibility/obligation reach? If double standards are legitimate, are nation-states at least constitutionally bound to elaborate and to ensure respect for internationally recognized standards on the safety of products, of health and safety at work, as well as environmental protection?

The adoption of the Rotterdam Convention suffocated the constitutional debate and shifted the focus towards its ratification. The juridification of a highly political conflict on double standards ends in a de-politicization. The attention shifted from the public political fora to the legal fora. The Member States of the EU and the EU ratified the Convention through EU Regulation 304/2003, later 698/2008, today 649/2012.Footnote ¹³ The United States signed the Convention but never ratified it. In order to be able to assess the potential impact of the Rotterdam Convention or more narrowly the role and function of the implementing EU Regulation on the European Member States, one has to dive deep into the activities of the European Chemical Agency, where all the information from the Member States is coming together.Footnote ¹⁴ When comparing the roaring public debate on double standards with the non-existent public interest in its bureaucratic handling, one may wonder to what extent ‘informed consent’ has improved the position of the citizens in the recipient state. The problem of double standards has not vanished at all.Footnote ¹⁵

13.2.2 Double Standards on Industrial Plants

The public attention seems to focus ever stronger on catastrophes which shatter the global world order – from time to time, but with a certain regularity. The level of action is not necessarily political or administrative; it is judicial. The eyes of the victims but also of NGOs, civil society organizations, consumer and environmental organizations were and are directed towards the role and function of courts. Dworkin published his book on Law’s empire, where he relied on the ‘Hercules judge’ in 1986, exactly at a time, where even in the transnational arena national courts and national judges turned into key actors and had to carry the hopes of all those who were fighting against double standards. This type of litigation can be easily associated with Baldwin’s distinction. The decline of transportation costs allowed Western-based multinationals to build subsidiaries around the world. Due to the economies of scale, it was cheaper for the multinationals to get the products manufactured in the subsidiaries and ship them back to the Western world to get them assembled. Typically, the subsidiaries were owned by the mother company, having its business seat in the United States or in Europe, either fully or at least up to a 51 per cent majority.

Again, the story to tell is not new, but it is paradigmatic for the 1980s. In 1984, a US-owned chemical plant in Bhopal India exploded. Thousands of people died. The victims argued that the plant did not even respect the rather low Indian standards of health and safety at work and Indian environmental standards. They were seeking compensation from Union Carbide Corporation, the mother company, and launched tort action claims in the United States.Footnote ¹⁶ The catastrophe mobilized NGOs and civil society organizations, along with class-action lawyers in the United States who combined the high expectations of the victims with their self-interest in bringing the case before US courts. The catastrophe laid bare the range of legal conflicts which arise in North-South civil litigation. Is there a responsibility of US companies which are operating outside the US territory to respect the high standards of the export state or international minimum standards, if they exist? Or does it suffice to comply with the lower standards of the recipient state? Is the American mother company legally liable for the harm produced through its subsidiary to the Indian workers, the Indian citizens affected in the community and the Indian environment? Which is the competent jurisdiction, the one of the US or the one of India, and what is the applicable law, US tort and class action law with its high compensation schemes or the tort law of the recipient state? The litigation fell into a period where socio-legal research played a key role in the United States and where legal scholars heavily engaged in the litigation providing legal support to the victims. There was a heated debate even between scholars sympathizing with the victims of whether it would be better for India to instrumentalise Bhopal so as to develop the Indian judiciary through the litigation in India in accepting the risk that Indian courts could provide carte blanche to the American mother companies or whether the rights of the victims should be preserved through the much more effective and generous US law before US courts. One of the key figures was Marc Galanter from Wisconsin, who left the material collected over decades on the litigation in the United States and in India, background information on the Indian judiciary, and the role and function of American authorities to the Wisconsin public library.Footnote ¹⁷ It remains to be added that in 1986 the US district court declined jurisdiction of American courts as forum non conveniens and that the victims who had to refile their case before Indian courts were never adequately compensated – until today. There are variations of the Bhopal type of litigation; the last one so far which equally gained public prominence is Kiobel.Footnote ¹⁸

The political and legal debate on double standards which dominated the public and legal fora in the 1980s differs in two ways from the one we have today on the responsibility of private parties in the digital economy and society: first and foremost, the primary addressees of the call for action were the Western democratic states as well as international organizations. They were sought to find appropriate solutions for what could not be solved otherwise. There are few examples of existing case law on double standards. Bhopal, though mirroring the problem of double standards, is different due to the dimension of the catastrophe and to the sheer number of victims which were identifiable. It is still noteworthy though that the international community left the search for solutions in the hands of the American respectively the Indian judiciary and that there was no serious political attempt neither of the two states nor of the international community to seek extra-judicial compensation schemes. The American court delegated the problem of double standards back to the Indian state and the Indian society alone. Second, in the 1980s, human rights were not yet or at least to a much lesser extent invoked in the search for political as well as for judicial solutions. There was less emphasis on the ‘rights’ rhetoric, on consumer rights as human rights or the right to safety as a human right.Footnote ¹⁹ Health, safety and the environment were treated as policy objectives that had to be implemented by the states, either nationally or internationally. The 1980s still breathe a different spirit, the belief in and the hope for an internationally agreeable legal framework that could provide a sound compromise between export and import states or, put differently, between the free-trade ideology and the need for some sort of internationally agreeable minimum standards of protection.

13.3.1 International Approaches on GVCs

In June 2011 the United Nations Human Rights Council unanimously adopted the Guiding Principles on Business and Human Rights (UNGPs). This was a major step towards the protection of Human Rights and the evolution of the concept of Social Corporate Responsibility. The adoption of the UNGPs was the result of thirteen years of negotiations. The year 2008 marked another step in the work of the Human Rights Council with the adoption of the framework ‘Protect, Respect and Remedy: A Framework for Business and Human Rights’.Footnote ²⁵ The framework laid down three fundamental pillars: the duty of the state to protect against human rights violations by third parties, including companies; the responsibility of companies to respect human rights; and better access by victims to effective remedies, both judicial and non-judicial. The Guiding Principles, which are seen as the implementation of the Protect, Respect and Remedy Framework, further detail how the three pillars are to be developed. The Guiding Principles are based on the recognition of

The Guiding Principles not only cover state behaviours but introduce a corporate responsibility to respect human rights as well as access to remedies for those affected by corporate behaviour or activities. Despite its non-binding nature, the UN initiative proves the intention to engage corporations in preventing negative impacts of their activities on human rights and in making good the damage they would nevertheless cause.

Here is not the place to give a detailed account of the initiative taken at the international level, but it is relevant to stop on the case of the OECD. The OECD worked closely with the UN Human Rights Council in elaborating the OECD Guidelines for Multinational Enterprises.Footnote ²⁷ The guidelines especially introduced an international grievance mechanism. The governments that adhere to the guidelines are required to establish a National Contact Point (NPC) which has the task of promoting the OECD guidelines and handling complaints against companies that have allegedly failed to adhere to the Guidelines’ standards. The NCP usually acts as a mediator or conciliator in case of disputes and helps the parties reach an agreement.Footnote ²⁸

13.3.2 National Approaches to Regulate GVCs

Not least through the international impact and the changing global environment, national legislators are becoming more willing to address the issue of the responsibility of corporations for their actions abroad from a GVC perspective. They focus explicitly or implicitly on a lead company which has to be held responsible. None have taken the network effects of GVS seriously. In 2010, California passed the Transparency in Supply Chains Act,Footnote ²⁹ the same year the United Kingdom adopted the UK Bribery Act, which creates a duty for undertakings carrying an economic activity in Britain to verify there is no corruption in the supply chain.Footnote ³⁰ The Bribery Act was then complemented by the UK Modern Slavery Act 2015, which focuses on human trafficking and exploitation in GVCs.Footnote ³¹ In the same line, the Netherlands adopted a law on the duty of care in relation to child labour, covering international production chains.Footnote ³² Complemented by EU instruments, such legislation is useful and constitutes a step forward, particularly at the political and legislative levels. Nevertheless, their focus on a sector, a product or certain rights does not enable the body of initiative to be mutually reinforcing. There is a crucial need for a holistic network-related approach to the regulation of GVCs.

Legislation on the responsibility of multinationals for human rights, environment or other harms is being designed in different countries. Germany and Finland have announced being in the process of drafting due diligence legislation.Footnote ³³ Switzerland had been working on a proposal, led by NGOs and left parties. The initiative was put to the votation in the last days of November 2020. A total of 47 per cent of the population participated, of which 50.73 per cent voted ‘Yes’.Footnote ³⁴ The project was rejected at the level of the cantons. Therefore this initiative will not go forward. At the time of writing, it seems to be a lighter initiative that will be discussed – one where responsibility is not imposed along the supply chain but for Swiss companies in third countries. The votation is nevertheless a performance in terms of the willingness to carry out such a project, participation and in terms of result. The result of the vote of the cantons can be partly explained by the lobby strategies multinationals have conducted from the beginning of the initiative.

The French duty of vigilance law was adopted in 2017 and introduced in the Code of Commerce among the provisions on public limited companies in the sub-part on shareholders assemblies.Footnote ³⁵ They require shareholders of large public limited companies with subsidiaries abroad to establish a vigilance plan. A vigilance plan introduces vigilance measures that identify the risks and measures to prevent serious harm to human rights, health, security or environmental harm resulting from the activities of the mother company but also of the company it controls, its subcontractors and its suppliers. The text provides for two enforcement mechanisms. First, a formal notice (mise en demeure) can be addressed to the company that does not establish a vigilance plan or establishes an incomplete one. The company has three months to comply with its obligations. Second, there could be an action in responsibility (action en responsabilité) against the company. Here the company must repair the prejudice the compliance with its obligations would have avoided. French multinationals have already received letters of formal notice. This is the case of EDF and its subsidiary EDF Energies Nouvelles for human rights violations in Mexico.Footnote ³⁶ The first case was heard in January 2020. It was brought by French and Ugandan NGOs against Total. The NGOs argue that the vigilance plan designed and put in place by Total is not in compliance with the law on due diligence and that the measures adopted to mitigate the risks are insufficient or do not exist at all.

13.3.3 The Existing Body of EU Approaches on GVCs and the Recent European Parliament Initiative

Sector-specific or product-specific rules imposed on GVCs have been adopted at the EU level and introduced due diligence obligations. The Conflict Minerals RegulationFootnote ³⁷ and the Regulation of timber productsFootnote ³⁸ impose obligations along the supply chain; the importer at the start of the GVC bears the obligations. The Directive on the Disclosure of Non-Financial and Diversity Information obliges large capital market-oriented companies to include in their non-financial statement information on the effects of the supply chain and the supply chain concepts they pursue.Footnote ³⁹ The Market Surveillance Regulation extends the circle of obligated economic operators in the EU to include participants in GVCs, thus already regulating extraterritorially.Footnote ⁴⁰ The Directive on unfair trading practices in the global food chain regulates trading practices in supply chains through unfair competition and contract law.Footnote ⁴¹ Although these bits and pieces of legislation introduce a form of due diligence along the supply chain, they remain product- or sector-specific, which prevents an overall legal approach to due diligence across sectors for all products. This concern is addressed by the latest Recommendation of the European Parliament.

Most recently, in September 2020, the JURI Committee of the European Parliament published a draft report on corporate due diligence and corporate accountability which includes recommendations for drawing up a Directive.Footnote ⁴² Although the European Parliament’s project has to undergo a number of procedures and discussions among the European institutions and is unlikely to be adopted in its current form, a few aspects are relevant for our discussion. Article 3 defines due diligence as follows:

Following the model of the UN Guiding Principles, the scope of the draft legislation goes beyond human rights to cover social and labour rights, the environment, climate change and governance. Article 4 details that undertakings are to identify and assess risks and publish a risk assessment. This risk-based approach is based on the second pillar of the UN Guiding Principles; it is also followed in the French due diligence law. In case risks are identified, a due diligence strategy is to be established whereby an undertaking designs measures to stop, mitigate or prevent such risks. The firm is to disclose reliable information about its GVC, namely, names, locations and other relevant information concerning subsidiaries, suppliers and business partners.Footnote ⁴³ The due diligence strategy is to be integrated in the undertaking’s business strategy, particularly in the choice of commercial partners. The undertaking is to contractually bind its commercial partners to comply with the company’s due diligence strategy.

13.3.4 Litigation before National Courts

Civil society, NGOs and trade unions are key players in making accountable multinationals for their actions abroad and along the GVC. They have supported legal actions for human rights violations beyond national territories. Such an involvement of the civil society is considerably facilitated through digitalization, through the use of the platforms and through the greater transparency in GVCs.Footnote ⁴⁴ Courts face cases where they have to assess violations of human rights in third countries by multinationals and their subsidiaries and construct extraterritorial responsibility. There is a considerable evolution from the 1980s in that the rights rhetoric goes beyond human rights so as to cover labour law, environmental law, misleading advertising or corporate law. Although the rights rhetoric recognizes the moral responsibility of private companies and accounts for their gravity, the challenges before and during trials to turn a moral responsibility into a legal liability are numerous.

In France, three textile NGOs brought a complaint arguing that Auchan’s communication strategy regarding its commitment to social and environmental standards in the supply chain constituted misleading advertising, since Auchan’s products were found in the Rana Plaza factory in Bangladesh, a factory well-known for its poor working and safety conditions. The case was dismissed at the stage of the investigation. In another case, Gabonese employees of COMILOG were victims of a train accident while at work, which led to financial difficulties for the company. They were dismissed and promised compensation, which they never received. With the support of NGOs, they brought the case to a French employment tribunal, claiming that COMILOG was owned by a French company. Their claim was dismissed but successful on appeal, where the court held COMILOG France and COMILOG international responsible for their own conduct and for the conduct of their subsidiaries abroad. On the merits, the court found that COMILOG had to compensate the workers. On appeal, the Court de Cassation annulled this finding, arguing that there was no sufficient evidence for the legally required strong link with the mother company in France.Footnote ⁴⁵ There is a considerable number of cases with similar constellations, where courts struggle in finding a coherent approach to these legal issues.

In Total, the NGOs pretended that the vigilance plan is incomplete and does not offer appropriate mitigating measures or failing to adopt them. The court did not rule on the merits, as the competence lies with the commercial court, since the law on due diligence is part of the Commercial Code. Nevertheless, the court made a distinction between the formal notice procedure which is targeted at the vigilance plan and its implementation and the action in responsibility.Footnote ⁴⁶ It is unclear whether the court suggested a twofold jurisdiction, a commercial one for due diligence strategies and another one for actions in responsibility. The case triggers fundamental questions as to what a satisfactory vigilance plan is and what appropriate mitigating measures are. It also requires clarifications about the relevant field of law applicable, the relevant procedure and the competent jurisdiction.

Even if there is an evolution as to the substance, today’s cases carry the heritage of those from the 1980s. Before ruling on the merits, courts engage in complex procedural issues, just like in the context of the Bhopal litigation or Kiobel. Such legal questions have not yet been settled at the national level, and they are still examined on a case-by-case basis. This lack of consistency renders the outcome of litigation uncertain. The first barrier is procedural; it concerns the jurisdiction of the national court on corporate action beyond the scope of its territorial jurisdiction. The second relates to the responsibility of the mother companies for their subsidiaries. In the two Shell cases brought up in the UKFootnote ⁴⁷ and in the Netherlands,Footnote ⁴⁸ Nigerian citizens had suffered from environmental damages which affected their territory, water, livelihood and health. Here the jurisdiction of the national courts was not an issue, but the differentiation between the mother company and its subsidiary remained controversial.Footnote ⁴⁹

The tour d’horizon indicates how fragile the belief in judicial activism still is. The adoption of due diligence legislation has not changed the level playing field. Courts are to design the contours and requirements of due diligence. Two methodological questions are at the heart of the ongoing discussions of the private responsibilities of companies in the GVCs. Who is competent? Who is responsible? Such are the challenges of the multilevel internationalized and digitalized environment where law finds itself unequipped to address the relevant legal challenges.

13.3.5 Business Approaches to GVCs within and beyond the Law

Recent initiatives suggest a different approach, one where legal obligations are placed on companies, not only to comply with their own obligations but to make them responsible for the respect of due diligence strategies along the GVC. The role and function of Corporate Social Responsibility and Corporate Digital Responsibility are in the political limelight.Footnote ⁵⁰ Thereby firms have the potential to exercise impact over the GVC. This is particularly true in case a lead company can easily be identified. If the upstream lead company decides to require its downstream partners to comply with its due diligence strategy, the lead company might be able to ensure compliance.Footnote ⁵¹ In GVCs, contracts are turned into a regulatory tool to the benefit of the lead company and perhaps to the benefit of public policy goals. There are two major problems: the first results from the exercise of economic power, which might be for good, but the opposite is also true. The second relates to the organization of the GVC, which more often than not is lacking a lead company but is composed out of a complex network of big, small and medium-sized companies. Designing responsibilities in networks is one of the yet still unsolved legal issues.

A consortium of French NGOs has drafted a report on the first year of application of the law on due diligence, where they have examined eighty vigilance plans published by French corporations falling under the scope of the due diligence law.Footnote ⁵² The report is entitled ‘Companies Must Do Better’ and sheds light on questions we have raised before. As regards the publication and content of the due diligence plans, not all companies have published their vigilance plans, some have incomplete ones, some have a lack of transparency and others seem to ignore the idea behind the due diligence plan. The report writes, ‘The majority of plans are still focusing on the risks for the company rather than those of third parties or the environment.’Footnote ⁵³ Along the different criteria of the vigilance plan analysed by the consortium of NGOs, it becomes clear that few companies have developed methodologies and appropriate responses in designing their due diligence strategy, identifying and mitigating risks. It is also noted that companies have re-used some previous policies and collected them to constitute due diligence. The lack of seriousness does not only make the vigilance plans unreadable; it denies any due diligence strategy of the firm. If multinationals do not take legal obligations seriously at the level of the GVC leading company, are they likely to produce positive spillover effects along the chain? It is too early to condemn the regulatory approach and the French multinationals. Once similar obligations will be adopted in most countries, at least in the EU, we might see a generalization and good practices emerge. Over the long term, we might witness competition arise between firms on the ground of their due diligence strategy.

Externally from the GVC, compliance can also be carried out by actors such as Trade Unions and NGOs. They have long been active in litigation and were consulted in the process of designing legislation. The European Parliament’s Recommendation suggests their involvement in the establishment of the undertaking’s due diligence strategies, similar to French law.Footnote ⁵⁴ Further, due diligence strategies are to be made public. In France, few companies have made public NGOs or stakeholders contributing to the design of the strategy. If there is no constructive cooperation between multinationals and NGOs yet, NGOs have access to grievance mechanisms under the European Parliament’s Recommendation, which resembles the letter of formal notice under the French law.Footnote ⁵⁵ Stakeholders which are not limited to NGOs could thereby voice concerns as to the existence of risks which the undertakings would have to answer to and be transparent about through publication.

NGOs have a unique capacity for gathering information abroad on the ground. The European Parliament’s text explicitly refers to the National Contact Point under the OECD framework. National Contact Points are not only entrusted with the promotion of the OECD guidelines; they offer a non-judicial platform for grievance mechanisms.Footnote ⁵⁶ The OECD conducts an in-depth analysis of the facts and publishes a statement as to the conflict and what it can offer to mediate it. Although such proceedings are non-binding, they do offer the possibility for an exchange between the parties and the case files are often relied on in front of courts. It seems that NGOs and other stakeholders have a role to play in compliance with the due diligence principles. They are given the possibility to penetrate the network and work with it from the inside. There are equally mechanisms that allow for external review of the GVC’s behaviour.

14.2.1 Consumers, Data Subjects and Redress

Consumers may think they are protected against adverse consequences of AI under privacy regulations and personal data protection regulatory regimes. However, it remains to be seen whether personal data protection extends to AI. Privacy policies are not designed to protect consumers against adverse consequences of data generated through AI. In that sense, there is a significant conceptual difference between policies and GTC: privacy policies are unilateral statements for compliance purposes. The policies do not leave room for negotiation. Moreover, privacy policies contain fairly moot purpose limitations. The purpose limitations are formulated de facto as processing rights. The private consumers/data subjects consider their consent implied to data processing, whatever tech is employed. Hence, the traders might be apt to apply their policies to consumers who are subjected to AI and machine learning. The General Data Protection Regulation (GDPR) contains one qualification in the realm of AI:Footnote ²³ a data subject has the right to object at any time against ADM including profiling. This obligation for data controllers is set off by the provision that controllers may employ ADM, provided they demonstrate compelling legitimate grounds for the processing which override the interests, rights and freedoms of the data subject.

Most of the traders’ machine learning is fed by aggregated, large batches of pseudonymised or anonymised non-personal data.Footnote ²⁴ There is no built-in yes/no button to express consent to be subjected to AI, and there is no such regulation on the horizon.Footnote ²⁵ The data policies are less tailored than GTC to defining consumer rights for complex AI systems. Besides, it is likely that most private individuals do not read the digital privacy policies – nor the general contract terms and conditions (GTC) for that matter – prior to responding to AI output.Footnote ²⁶ The provided questions reveal important private law concerns: ‘What are my rights?’ relates to justified questions as regards access rights and vested consumer rights, the right to take note of and save/print the conditions; void unfair user terms; and termination rights. Traders usually refer to the GTC that can be found on the site. There is no meaningful choice. That is even more the case in the continental tradition, where acceptance of GTC is explicit. In Anglo-American jurisdictions, the private individual is confronted with a pop-up window which must be scrolled through and accepted. Declining means aborting the transaction.

‘How can I enforce my rights against the trader?’ requires that the consumer who wishes to enforce her rights must be able to address the trader, either on the platform or through online dispute resolution mechanisms. Voidance or nullification are remedies when an agreement came about through settled European private law principles, such as coercion, error or deceit. Hence the consumer needs to know there is a remedy if the AI process contained errors or was faulty.Footnote ²⁷

14.2.2 Principles of Contract Law

In the algorithmic society, consumers still should have at least some recourse to a counterparty, whom they can ask for information during the consideration process. They must have redress when they do not understand or agree with transactional output that affects their contractual position without explanation. The right to correct steps in contract formation is moot, where the process is cast in stone. Once the consumers have succeeded in identifying the formal counterparty, they can apply remedies. Where does that leave them if the response to these remedies is also automated as a result of the trader’s use of profiling and decision-making tools? This reiterates the question of whether human dignity is at stake, when the counterpart is not a human but a machine. The consumer becomes a string of codes and loses her feeling of uniqueness.Footnote ²⁸ Furthermore, when distributed ledger technology is used, the chain of contracts is extended. There is the possibility that an earlier contractual link will be ‘lost’. For example, there is a gap in the formation on the digital platform, because the contract formation requirements either were not fully met or were waived. Another example is where the consumer wants to partially rescind the transaction but the system does not cater for a partial breach. The impact of a broken upstream contractual link on a downstream contract in an AI-enabled transactional system is likely to raise novel contract law questions, too. An agreement may lack contractual force if there is uncertainty or if a downstream contractual link in the chain is dependent on the performance of anterior upstream agreements. An almost limitless range of possibilities will need to be addressed in software terms, in order to execute the platform transaction validly. When the formation steps are using automated decision-making processes that are not covered in the GTC governing the status of AI output, then this begs the question of how AI using distributed ledger technology could react to non-standard events or conditions, and if and how the chain of transactions is part of the consideration. The consumer could wind up in a vicious cycle, and her fundamental rights of a high consumer protection level could be at stake, more than was the case in the information society. Whereas e-Commerce, Distant Selling and, later, Services Directives imposed information duties on traders, the normative framework for the algorithmic society is based on rather different principles. Theories such as freedom of contract – which entails the exclusion of coercion – and error, when AI output contains flaws or defects may be unenforceable in practice. For the consumer to invoke lack of will theories, she needs to be able to establish where and how in the system the flaws or mistakes occurred.

14.2.3 Data Strategy

Does the data strategy stand in the way of consumer protection against AI? The focus of the EU’s data strategy is on stimulating the potential of data for business, research and innovation purposes.Footnote ²⁹ The old regulatory dilemma on how to balance a fair and competitive business environment with a high level of consumer rights is revived. In 2019–2020, the Commission announced various initiatives, including rules on (1) securing free flow of data within the Union,Footnote ³⁰ (2) provisions on data access and transfer,Footnote ³¹ and (3) and enhanced data portability.Footnote ³² Prima facie, these topics exhibit different approaches to achieve a balance between business and consumer interests. More importantly, how does the political desire for trustworthy technology match with such diverse regulations? The answer is that it does not. The Free Flow of Non-Personal Data Regulation lays down data localization requirements, the availability of data to competent authorities and data porting for professional users.Footnote ³³ It does not cover AI use. The Modernization of Consumer Protection Directive alludes to the requirement for traders to inform consumers about the default main parameters determining the ranking of offers presented to the consumer as a result of the search query and their relative importance as opposed to other parameters only.Footnote ³⁴ The proviso contains a reference to ‘processes, specific signals incorporated into algorithms or other adjustment or demotion mechanisms used in connection with the ranking are not required to disclose the detailed functioning of their ranking mechanisms, including algorithms’.Footnote ³⁵ It does not appear that the Modernization of Consumer Protection Directive is going to protect consumers against adverse consequences of AI output. It also seems that the Trade Secrets Directive stands somewhat in the way of algorithmic transparency.

The provisions on data porting revert to information duties. Codes of Conduct must detail the information on data porting conditions (including technical and operational requirements) that traders should make available to their private individuals in a sufficiently detailed, clear and transparent manner before a contract is concluded.Footnote ³⁶ In light of the limited scope of data portability regulation, there can be some doubt as to whether the high-level European data strategy is going to contribute to a human-centric development of AI.

14.2.4 Trustworthiness and Transparency

The next question is what regulatory requirements could emerge when AI will become ubiquitous in mutual transactions.Footnote ³⁷ The Ethical Guidelines on AI in 2019 allude to seven key requirements for ‘Trustworthy AI’: (1) human agency and oversight; (2) technical robustness and safety; (3) privacy and data governance; (4) transparency, (5) diversity, non-discrimination and fairness; (6) environmental and societal well-being; and (7) accountability.Footnote ³⁸ These non-binding guidelines address different topics, some of which fall outside the scope of private law principles. In this chapter, the focus is on transparency, accountability and other norms, notably traceability, contestability and veracity.Footnote ³⁹ These notions are covered in the following discussion. First, it is established that opaqueness on technology use and lack of accountability could be perceived as being potentially harmful to consumers.Footnote ⁴⁰ There are voices that claim that technology trustworthiness is essential for citizens and businesses that interact.Footnote ⁴¹ Is it up to Constitutional States to warrant and monitor technology trustworthiness, or should this be left to businesses? Does warranting technology trustworthiness not revive complex economic questions, such as how to deal with the possibility of adverse impact on competition or the stifling of innovation, when governments impose standardized technology norms to achieve a common level of technology trustworthiness – in the EU only? What if trust in AI is broken?

A possible denominator for trustworthiness may be transparency. Transparency is a key principle in different areas of EU law. A brief exploration of existing regulation reveals different tools to regulate transparency. Recent examples in 2019–2020 range from the Modernization of Consumer Protection Directive to the Online Intermediary Services Regulation, the Ethical Guidelines on AI, the Open Data Directive and the 2020 White Paper on Artificial Intelligence.Footnote ⁴² All these instruments at least allude to the need for transparency in the algorithmic society. The Modernization of Consumer Protection Directive provides that more transparency requirements should be introduced. Would it be necessary to redefine transparency as a principle of private law in the algorithmic society? One could take this a step further: to achieve technology trustworthiness, should there be more focus on regulating transparency of AI and machine learning?Footnote ⁴³ The Ethics Guidelines 2019 point at permission systems, fairness and explicability. From a private law perspective, especially permission systems could be considered to establish and safeguard trust. But reference is also made to the factual problem that consumers often do not take note of the provisions that drive the permission.

Explicability is not enshrined as a guiding principle. Nevertheless, transparency notions could be a stepping stone to obtaining explicability.Footnote ⁴⁴ Accuracy may be a given. What matters is whether the consumer has the right and is enabled to contest an outcome that is presented as accurate.

14.3.1 Regulating AI through General Terms and Conditions

There are two aspects regarding GTC that must be considered. First, contrary to permission systems, the general rule in private law remains that explicit acceptance of GTC by the consumer is not required, as long as the trader has made the terms available prior to or at the moment the contract is concluded. Contrary to jurisdictions that require parties to scroll through the terms, the European approach of accepting implied acceptance in practice leads to consumers’ passiveness. Indeed, the system of implicit permission encourages consumers to not read GTC. Traders on digital platforms need to provide information on what technologies they use and how they are applied. Given the sheer importance of fundamental rights of human dignity and consumer rights when AI is applied, the question is whether consumers should be asked for explicit consent when the trader applies AI. It would be very simple for traders to implement consent buttons applying varied decision trees. But what is the use when humans must click through to complete a transaction? Take, for example, the system for obtaining cookies consent on digital platforms.Footnote ⁴⁵ On the one hand, the traders (must) provide transparency on which technologies they employ. On the other hand, cookie walls prevent the consumer from making an informed decision, as they are coerced to accept the cookies. A recognizable issue with cookies in comparison with AI is that, often, it is the consumers who are unable to understand what the different technologies could mean for them personally. In the event the AI output matches their expectations or requirements, consumers are unlikely to protest prior consent given. Hence the real question is whether consumers should be offered a menu of choice beforehand, plus an option to accept or reject AI output or ADM. This example will be covered in the following discussion.

Second, where there is no negotiation or modification of the GTC, the consumer still will be protected by her right to either void or rescind black-, blue or grey-list contract provisions. Additionally, the EU Unfair Contract Terms Directive contains a blue list with voidable terms and conditions.Footnote ⁴⁶ However, the black, grey and blue lists do not count for much. Rather, the GTC should contain clauses that oblige the trader to observe norms and principles such as traceability, contestability, transparency and veracity of the AI process. This begs the question of whether ethics guidelines and new principles could be translated into binding, positively formulated obligations or AI use. Rather than unilateral statements on data use, GTC could be subjected to comply with general principles and obligations.

The key for prospective regulation does not lie in art. 6 (1) Modernization of Consumer Protection Directive. Although this clause contains no less than twenty-one provisions on information requirements, including two new requirements on technical aspects, none of the requirements apply to providing the consumer information on the use of AI and ADM, let alone the contestability of the consumer transaction based thereon. Granted, there is an obligation for the trader to provide information on the scope of the services, but not on the specific use of AI technology. It is a very big step from the general information requirements to providing specific information on the application of AI and ADM in mutual transactions. When a consumer is subjected to AI processes, she should be advised in advance, not informed after the fact. A commentary to art. 6 clarifies that the traders must provide the information mentioned therein prior to the consumer accepting the contract terms (GTC).Footnote ⁴⁷ The underlying thought is not new – to protect consumers, as weaker contractual parties, from concluding contracts that may be detrimental to them, and as a result of not having all the necessary information. Absent any relevant information, the consumer lags behind, especially in terms of not being informed adequately (1) that, (2) how and (3) for which purposes AI and machine learning is applied by the trader. The commentators generally feel that providing consumers with the relevant information prior to the conclusion of the contract is essential. Knowing that the trader uses such technologies could be of utmost importance to the consumer. Even if she cannot oversee what the technological possibilities are, she should still get advance notice of the application of AI. Advance notice means a stand-still period during which she can make an informed decision. Going back to the cookie policy example, it is not onerous on the trader to offer the consumer a menu for choice beforehand. This would be especially relevant for the most used application of AI and ADM: profiling. The consumer should have the right to reject a profile scan that contains parameters she does not find relevant or which she perceives as being onerous on her. Granted, the trader will warn the consumer that she will not benefit from the best outcome, but that should be her decision. The consumer should have a say in this important and unpredictable process. She should be entitled to anticipating adverse consequences of AI for her.

The consumer must be able to trace and contest the AI output and ADM. The justification for such rights is discrimination, and lack of information on the essentials underlying the contract terms that come about through the private law principle of offer and acceptance. Granted, art. 9 Modernization of Consumer Protection Directive contains the generic right of withdrawal.Footnote ⁴⁸ Contesting a consumer transaction based on AI is not necessary. The consumer can simply fill in a form to rescind the agreement. Regardless, the point of a consumer approach to AI use is not meant for the consumer to walk away. The consumer must have the right to know what procedures were used, what kind of outcome they produced, what is meant for the transaction and what she can do against it. As said, the consumer also must have a form of redress, not just against the trader but also against the developer of the AI software, the creator of the process, the third-party instructing the algorithms and/or the intermediary or supplier of the trader.

14.3.2 Consumer Law Principles

Which consumer law principles could be reignited in GTC that enable consumers to require the traders to be accountable for unfair processes or non-transparent output? This goes back to the main theorem. Transactions on digital platforms are governed by mutually agreed contract terms. It is still common practice that these are contained in GTC. Is there a regulatory gap that requires for Constitutional States to formulate new or bend existing conditions for traders using AI? The Bureau Européen des Unions de ConsommateursFootnote ⁴⁹ proposes ‘a set of transparency obligations to make sure consumers are informed when using AI-based products and services, particularly about the functioning of the algorithms involved and rights to object automated decisions’. The Modernization of Consumer Protection Directive is open for adjustment of consumer rights ‘in the context of continuous development of digital tools’. The Directive makes a clear-cut case for consumers catering for the adverse consequences of AI.Footnote ⁵⁰ But it contains little concrete wording on AI use and consumers.Footnote ⁵¹ Embedding legal obligations for the trader in GTC could, potentially, be a very effective measure. There is one caveat, in that GTC often contain negatively formulated obligations.Footnote ⁵² Positively phrased obligations, such as the obligation to inform consumers that the trader employs AI, require further conceptual thinking. Another positively phrased obligation could be for the traders to explain the AI process and explain and justify the AI output.

14.3.3 Further Harmonization of Consumer Law in the Context of AI

It should be considered whether the Unfair Commercial Practices Directive could be updated with terms that regulate AI.Footnote ⁵⁸ At the high level, this Directive introduced the notion of ‘good faith’ to prevent imbalances in the rights and obligations of consumers on the one hand and sellers and suppliers on the other hand.Footnote ⁵⁹ It should be borne in mind that consumer protection will become an even more important factor when the chain of consumer agreements with a trader becomes extended. Granted, the question of whether and how to apply AI requires further thinking on what types of AI and data use could constitute unfair contract terms. A case could be made of an earlier agreement voiding follow-up transactions, for example, because the initial contract formation requirements were not met as after AI deployment. But the impact of a voidable upstream contractual link on a downstream agreement in an AI-enabled or contract system is likely to raise different novel contract law questions, for instance, regarding third party liability.

In order to ensure that Member State authorities can impose effective, proportionate and dissuasive penalties in relation to widespread infringements of consumer law and to widespread infringements with an EU dimension that are subject to coordinated investigation and enforcement,Footnote ⁶⁰ special fines could be introduced for the unfair application of AI.Footnote ⁶¹ Contractual remedies, including claims as a result of damages suffered from incorrect ADM, could be considered.

Prima facie, the Modernization of Consumer Protection Directive provides for the inclusion of transparency norms related to the parameters of ranking of prices and persons on digital platforms. However, the Directive does not contain an obligation to inform the consumer about the relative importance of ranking parameters and the reasons why and through what human process, if any, the input criteria were determined. This approach bodes well for the data strategy, but consumers could end up unhappy, for instance, if information about the underlying algorithms is not included in the transparency standard.

By way of an example, the Modernization of Consumer Protection Directive provides for a modest price transparency obligation at the retail level. It proposes a specific information requirement to inform consumers clearly when the price of a product or service presented to them is personalized on the basis of ADM. The purpose of this clause is to ensure that consumers can take into account the potential price risks in their purchasing decision.Footnote ⁶² But the proviso does not go as far as to determine how the consumer should identify these risks. Digital platforms are notoriously silent on price comparisons. Lacking guidance on risk identification results in a limited practical application of pricing transparency. What does not really help is that the Modernization of Consumer Protection Directive provides traders with a legal – if flimsy – basis for profiling and ADM.Footnote ⁶³ This legal basis is, unfortunately, not supplemented by consumer rights that go beyond them receiving certain, non-specific information from the trader. The Modernization of Consumer Protection Directive, as it stands now, does not pass the test of a satisfactorily high threshold for consumer protection on AI-driven platforms.

15.1.1 A Preliminary Distinction: Algorithms and Soft Computing

According to computer science, algorithms are automated decision-making processes to be followed in calculations or other problem-solving operations, especially by a computer.Footnote ² Thus an algorithm is a detailed and numerically finite series of instructions which can be processed through a combination of software and hardware tools: Algorithms start from an initial input and reach a prescribed output, which is based on the subsequent set of commands that can involve several activities, such as calculation, data processing, and automated reasoning. The achievement of the solution depends upon the correct execution of the instructions.Footnote ³ However, it is important to note that, contrary to the common perception, algorithms are neither always efficient nor always effective.

Under the efficiency perspective, algorithms must be able to execute the instructions without exploiting an excessive amount of time and space. Although technological progress allowed for the development of increasingly more powerful computers, provided with more processors and a better memory ability, when algorithms execute instructions that produce great numbers which exceed the space available in memory of a computer, the ability of the algorithm itself to sort the problems is questioned.

As a consequence, under the effectiveness perspective, algorithms may not always reach the exact solution or the best possible solution, as they may include a level of approximation which may range from a second-best solution,Footnote ⁴ to a very low level of accuracy. In this case, computer scientists use the definition of ‘soft computing’ (i.e., the use of algorithms that are tolerant of imprecision, uncertainty, partial truth, and approximation), due to the fact that the problems that they are addressing may not be solved or may be solved only through an excessive time-consuming process.Footnote ⁵

Accordingly, the use of these types of algorithms involves the possibility to provide solutions to hard problems, though these solutions, depending on the type of problems, may not always be the optimal ones. Given the ubiquitous use of algorithms processing our data and consequently affecting our personal decisions, it is important to understand in which occasions we may (or should) not fully trust the algorithm and add a human in the loop.Footnote ⁶

15.1.2 The Power of Algorithms

According to Neyland,Footnote ⁷ we may distinguish between two types of power: one exercised by algorithms, and one exercised across algorithms. The first one is the traditional one, based on the ability of algorithms to influence and steer particular effects. The second one is based on the fact that ‘algorithms are caught up within a set of relations through which power is exercised’.Footnote ⁸ In this sense, it is possible to affirm the groups of individuals that at different stages play a role in the definition of the algorithm share a portion of power.

In practice, one may distinguish between two levels of analysis. Under the first one, for instance when we digit a query over a search engine, the search algorithm activates and identifies the best results related to the keywords inserted, providing a ranked list of results. These results are based on a set of variables that are dependent on the context of the keywords, but also on the trust of the source,Footnote ⁹ on the previous history of searches of the individual, and so forth. The list of results available will then steer the decisions of the individual and affect his/her interpretation of the information searched for. Such power should not be underestimated, because the algorithm has the power to restrict the options available (i.e., avoiding some content because evaluated as untruthful or irrelevant) or to make it more likely to select a specific option. If this can be qualified as the added value of algorithms able to improve the flaws of human reasoning, which include myopia, framing, loss aversion, and overconfidence,Footnote ¹⁰ then it also shows the power of the algorithm over individual decision-making.Footnote ¹¹

Under the second level of analysis, one may widen the view taking into account the criteria that are used to identify the search results, the online information that is indexed, the computer scientist that set those variables, the company that distributes the algorithm, the public or private company that uses the algorithm, and the individuals that may steer the selection of content. All these elements have intertwining relationships that show a more distributed allocation of power – and, as a consequence, a subsequent quest for a shared type of accountability and liability systems.

15.1.3 The Use of Algorithms in Content Moderation

In this chapter, the analysis will focus on those algorithms that are used for content detection and control over user-generated platforms, the so-called content moderation. Big Internet companies have always used filtering algorithms to detect and classify the enormous quantity of uploaded data daily. Automated content filtering is not a new concept on the Internet. Since the first years of Internet development, many tools have been deployed to analyse and filter content, and among them the most common and known are those adopted for spam detection or hash matching. For instance, spam detection tools identify content received in one’s email address, distinguishing between clean emails and unwanted content on the basis of certain sharply defined criteria derived from previously observed keywords, patterns, or metadata.Footnote ¹²

Nowadays, algorithms that are used for content moderation are widely diffuse, having the advantage of scalability. Such systems promise to make the process much easier, quicker, and cheaper than would be the case when using human labour.Footnote ¹³

For instance, the LinkedIn network published the update of the algorithms used to select the best matches between employers and potential employees.Footnote ¹⁴ The first steps of the content moderation are worth describing: at the first step, the algorithms check and verify the compliance of the content published with the platform rules (leading to a potential downgrade of the visibility or complete ban in case of incompliance). Then, the algorithms evaluate the interactions that were triggered by the content posted (such as sharing, commenting, or reporting by other users). Finally, the algorithms weigh such interactions, deciding whether the post will be demoted for low quality (low interaction level) or disseminated further for its high quality.Footnote ¹⁵

As the example of the LinkedIn algorithm clearly shows, the effectiveness of the algorithm depends on its ability to accurately analyse and classify content in its context and potential interactions. The capability to parse the meaning of a text is highly relevant for making important distinctions in ambiguous cases (e.g., when differentiating between contemptuous speech and irony).

For this task, the industry has now increasingly turned to machine learning to train their programmes to become more context sensitive. Although there are high expectations regarding the ability of content moderation tools, one should not underestimate the risks of overbroad censorship,Footnote ¹⁶ violation of the freedom of speech principle, as well as biased decision-making against minorities and non-English speakers.Footnote ¹⁷ The risks are even more problematic in the case of hate speech, an area where the recent interventions of European institutions are pushing for more human and technological investments of IT companies, as detailed in the next section.

15.2.1 A Definition of Hate Speech

Hate speech is generally defined as speech ‘designed to promote hatred on the basis of race, religion, ethnicity, national origin’ or other specific group characteristics.Footnote ²² Although several international treaties and agreements do include hate speech regulation,Footnote ²³ at the European level, such an agreed-upon framework is still lacking. The point of reference available until now is the Council Framework Decision 2008/913/JHA on Combatting Certain Forms and Expressions of Racism and Xenophobia by Means of Criminal Law.Footnote ²⁴ As emerges from the title, the focus of the decision is the approximation of Member States’ laws regarding certain offences involving xenophobia and racism, whereas it does not include any references to other types of motivation, such as gender or sexual orientation.

The Framework Decision 2008/913/JHA should have been implemented by Member States by November 2010. However, the implementation was less effective than expected: not all the Member States have adapted their legal framework to the European provisions.Footnote ²⁵ Moreover, in the countries where the implementation occurred, the legislative intervention followed different approaches than the national approaches to hate speech, either through the inclusion of the offence within the criminal code or through the adoption of special legislation on the issue. The choice is not without effects, as the procedural provisions applicable to special legislation may be different to those applicable to offences included in the criminal code.

Given the limited effect of the hard law approach, the EU institutions moved to a soft law approach regarding hate speech (and, more generally, also illegal content).Footnote ²⁶ Namely, EU institutions moved toward the use of forms of co-regulation where the Commission negotiates a set of rules with the private companies, under the assumption that the latter will have more incentives to comply with agreed-upon rules.Footnote ²⁷

As a matter of fact, on 31 May 2016, the Commission adopted a Code of Conduct on countering illegal hate speech online, signed by the biggest players in the online market: Facebook, Google, Microsoft, and Twitter.Footnote ²⁸ The Code of Conduct requires that the IT company signatories to the code adapt their internal procedures to guarantee that ‘they review the majority of valid notifications for removal of illegal hate speech in less than 24 hours and remove or disable access to such content, if necessary’.Footnote ²⁹ Moreover, according to the Code of Conduct, the IT companies should provide for a removal notification system which allows them to review the removal requests ‘against their rules and community guidelines and, where necessary, national laws transposing the Framework Decision 2008/913/JHA’.

As is evident, the approach taken by the European Commission is more focused on the timely removal of the allegedly hate speech than on the procedural guarantees that such private enforcement mechanism should adopt in order not to unreasonably limit the freedom of speech of users. The most recent evaluation of the effects of the Code of conduct on hate speech shows an increased number of notifications that have been evaluated and eventually led to the removal of hate speech content within an ever-reduced time frame.Footnote ³⁰

In order to achieve such results, the signatory companies adopted a set of technological tools assessing and evaluating the content uploaded on their platforms. In particular, they finetuned their algorithms in order to detect potentially harmful content.Footnote ³¹ According to the figures provided by the IT companies regarding the flagged content, human labour alone may not achieve such task.Footnote ³² However, such algorithms may only flag content based on certain keywords, which are continuously updated, but they always lag behind the evolution of the language. And, most importantly, they may still misinterpret context-dependent wording.Footnote ³³ Hate speech is a type of language that is highly context sensitive, as the same word may radically change its meaning if used at different places over time. Moreover, algorithms may be improved and trained in one language, but not in other languages which are less prominent in online communication. As a result, an algorithm that works only through the classifications of certain keywords cannot attain the level of complexity of human language and runs the risk of producing unexpected false positives and negatives in the absence of context.Footnote ³⁴

15.2.2 The Human Intervention in Hate Speech Detection and Removal

One of the strategies able to reduce the risk of structural over-blocking is the inclusion of some human involvement in the identification and analysis of potential hate speech content.Footnote ³⁵ Such human involvement can take different forms, either internal content checking or external content checking.Footnote ³⁶

In the first case, IT companies allocate to teams of employees the task of verifying the sensitive cases, where the algorithm was not able to single out if the content is contrary to community standards or not.Footnote ³⁷ Given the high number of doubtful cases, the employees are subject to a stressful situation.Footnote ³⁸ They are asked to evaluate in a very short time frame the potentially harmful content, in order to provide a decision regarding the opportunity to take the content down. This will then provide additional feedback to the algorithm, which will learn the lesson. In this framework, the algorithms automatically identify pieces of potentially harmful content, and the people tasked with confirming this barely have time to make a meaningful decision.Footnote ³⁹

The external content checking instead involves the ‘trusted flaggers’ – that is, an individual or entity which is considered to have particular expertise and responsibilities for the purposes of tackling hate speech. Examples for such notifiers can range from individual or organised networks of private organisations, civil society organisations, and semi-public bodies, to public authorities.Footnote ⁴⁰

For instance, YouTube defines trusted flaggers as individual users, government agencies, and NGOs that have identified expertise, (already) flag content frequently with a high rate of accuracy, and are able to establish a direct connection with the platform. It is interesting to note that YouTube does not fully delegate the content detection to trusted notifiers but rather affirms that ‘content flagged by Trusted Flaggers is not automatically removed or subject to any differential policy treatment – the same standards apply for flags received from other users. However, because of their high degree of accuracy, flags from Trusted Flaggers are prioritized for review by our teams’.Footnote ⁴¹

15.3.1 Legal Rules versus Community Standards

As hinted previously, both algorithms and humans involved in content detection and removal of hate speech evaluate content vis-à-vis the community standards adopted by each platform. Such distinction is clearly affirmed also in the YouTube trusted flaggers programme, where it is affirmed that ‘the Trusted Flagger program exists exclusively for the reporting of possible Community Guideline violations. It is not a flow for reporting content that may violate local law. Requests based on local law can be filed through our content removal form’.

These standards, however, do not fully overlap with the legal definition provided by EU law, pursuant to the Framework Decision 2008/913/JHA.

Table 15.1 shows that the definitions provided by the IT companies widen the scope of the prohibition on hate speech to sex, gender, sexual orientation, disability or disease, age, veteran status, and so forth. This may be interpreted as the achievement of a higher level of protection. However, the width of the definition is not always coupled with a subsequent detailed definition of the selected grounds. For instance, the YouTube community standards list the previously mentioned set of attributes, providing some examples of hateful content. But the standard only sets two clusters of cases: encouragement towards violence against individuals or groups based on the attributes, such as threats, and the dehumanisation of individuals or groups (for instance, calling them subhuman, comparing them to animals, insects, pests, disease, or any other non-human entity).Footnote ⁴⁵ The Facebook Community policy provides for a better example, as it includes a more detailed description of the increasing levels of severity attached to three tiers of hate speech content.Footnote ⁴⁶ In each tier, keywords are provided to show the type of content that will be identified (by the algorithms) as potentially harmful.

As a result, the inclusion of such wide hate speech definitions within the Community Guidelines or Standards become de facto rules of behaviour for users of such services.Footnote ⁴⁷ The IT companies are allowed to evaluate a wide range of potentially harmful content published on their platforms, though this content may not be illegal according to the Framework Decision 2008/914/JHA.

This has two consequences. First, there is an extended privatisation of enforcement as regards those conducts that are not covered by legal provisions with the risk of an excessive interference with the right to freedom of expression of users.Footnote ⁴⁸ Algorithms deployed by IT companies will then have the power to draw the often-thin line between legitimate exercise of the right to free speech and hate speech.Footnote ⁴⁹

Second, the extended notion of harmful content provided by community rules imposes a wide obligation on platforms regarding the flow of communication. This may conflict with the liability regime adopted pursuant relevant EU law, namely the e-Commerce Directive, which imposes a three-tier distinction across intermediary liability and, most importantly, prohibits any general monitoring obligation over ISP pursuant art. 15.Footnote ⁵⁰ As it will be addressed later, in the section on liability, striking the balance between sufficient incentives to block harmful content and over-blocking effects is crucial to safeguard the freedom of expression of users.

15.3.2 Due Process Guarantees

As a consequence of the previous analysis, the issue of procedural guarantees of users emerges.Footnote ⁵¹ A first question is related to the availability of internal mechanisms that allow users to be notified about potentially harmful content, to be heard, and to review or appeal against the decisions of IT companies. Although the strongest position safeguarding freedom of expression and fair trial principle would suggest that any restriction (i.e., any removal of potentially harmful content) should be subject to judicial intervention,Footnote ⁵² the number of decisions adopted on a daily basis by IT companies does not allow either the intervention of potential victims and offenders, or the judicial system. It should be noted that the Code of Conduct does not provide for any specific requirement in terms of judicial procedures, nor through alternative dispute resolution mechanisms, thus it is left to the IT companies to introduce an appeal mechanism.

Safeguards to limit the risk of removal of legal content are provided instead in the Commission Recommendation on Tackling Illegal Content Online,Footnote ⁵³ which includes within the wider definition of illegal content also hate speech.Footnote ⁵⁴ The Recommendation points to automated content detection and removal and underlines the need for counter-notice in case of removal of legal content. The procedures involve the exchange between the user and the platform, which should provide a reply: in case of evidence provided by the user that the content may not be qualified as illegal, the platform should restore the content that was removed without undue delay or allow for a re-upload by the user; whereas, in case of a negative decision, the platform should include reasons for said decision.

Among the solutions, the signatories to the Code of Conduct proposed Google provides for a review mechanism, allowing users to present an appeal against the decision to take down any uploaded content.Footnote ⁵⁵ Then, the evaluation of the justifications provided by the user is processed internally and the final decision is sent afterward to the user, with limited or no explanation.

A different approach is adopted by Facebook. In September 2019, the social network announced the creation of an ‘Oversight Board’.Footnote ⁵⁶ The Board has the task of providing the appeals for selected cases that address potentially harmful content. Although the detailed regulation concerning the activities of the board is still to be drafted, it is clear that it will not be able to review all the content under appeal.Footnote ⁵⁷ Although this approach has been praised by scholars, several questions remain open: the transparency in the selection of the people entrusted with the role of adjudication, the type of explanation for the decision taken, the risk of capture (in particular for the oversight board), and so on. And, at the moment, these questions are still unanswered.

15.3.3 Selection of Trusted Flaggers

As mentioned previously in Section 15.2.2., the intervention of trusted flaggers in content detection and removal became a crucial element in order to improve the results of said process. The selection process to identify and recruit trusted flaggers, however, is not always clear.

According to the Commission Recommendation, the platforms should ‘publish clear and objective conditions’ for determining which individuals or entities they consider as trusted flaggers. These conditions include expertise and trustworthiness, and also ‘respect for the values on which the Union is founded as set out in Article 2 of the Treaty on European Union’.Footnote ⁵⁸

Such a level of transparency does not match with the practice: although the Commission Monitoring exercise provides for data regarding at least four IT companies, with a percentage of notifications received by users vis-à-vis trusted flaggers as regards hate speech,Footnote ⁵⁹ apart from the previously noted YouTube programme, none of the other companies provide a procedure for becoming a trusted flagger. Nor is any guidance provided on whether the selection of trusted notifiers is a one-time accreditation process or rather an iterative process whether the privilege is monitored and can be withdrawn.Footnote ⁶⁰

This issue should not be underestimated, as the risk of rubberstamping the decisions of trusted flaggers may lead to over-compliance and excessive content takedown.Footnote ⁶¹

15.3.4 Liability Regime

When IT companies deploy algorithms and recruit trusted flaggers in order to proactively detect and remove potentially harmful content, they may run the risk of losing their exemption of liability according to the e-Commerce Directive.Footnote ⁶² According to art. 14 of the Directive, hosting providers are exempted from liability when they meet the following conditions:

1. – Service providers provide only for the storage of information at the request of third parties;

2. – Service providers do not play an active role of such a kind as to give it knowledge of, or control over, that information.

According to the decision of the CJEU in L’Oréal v. eBay,Footnote ⁶³ the Court of Justice clarified that whenever an online platform provides for the storage of content (in the specific case offers for sale), sets the terms of the service, and receives revenues from such service, this does not change the position of the hosting provider denying the exemptions from liability. In contrast, this may happen when the hosting provider ‘has provided assistance which entail, in particular optimising the presentation of the offers for sale in question or promoting those offers’.

This indicates that the active role of the hosting provider is only to be found when it intervenes directly in user-generated content.Footnote ⁶⁴ If the hosting provider adopts technical measures to detect and remove hate speech, does it fail its neutral position vis-à-vis the content?

The liability exemption may still apply only if two other conditions set by art. 14 e-Commerce Directive apply. Namely,

1. – hosting providers do not have actual knowledge of the illegal activity or information and, as regards claims for damages, are not aware of facts or circumstances from which the illegal activity or information is apparent; or

2. – upon obtaining such knowledge or awareness, they act expeditiously to remove or to disable access to the information.

It follows that proactive measures taken by the hosting provider may result in that platform obtaining knowledge or awareness of illegal activities or illegal information, which could thus lead to the loss of the liability exemption. However, if the hosting provider acts expeditiously to remove or to disable access to content upon obtaining such knowledge or awareness, it will continue to benefit from the liability exemption.

From a different perspective, it is possible that the development of technological tools may lead to a reverse effect as regards monitoring obligations applied over IT companies. According to art. 15 of the e-Commerce Directive, no general monitoring obligation may be imposed on hosting providers as regards illegal content. But in practice, algorithms may already deploy such tasks. Would this indirectly legitimise monitoring obligations applied by national authorities?

This is the question posed by an Austrian court to the CJEU as regards hate speech content published on the social platform Facebook.Footnote ⁶⁵ The preliminary reference addressed the following case: in 2016, the former leader of the Austrian Green Party, Eva Glawischnig-Piesczek was the subject of a set of posts published on Facebook by a fake account. The posts included rude comments, in German, about the politician, along with her image.Footnote ⁶⁶

Although Facebook complied with the injunction of the First Instance court across the Austrian country, blocking access to the original image and comments, the social platform appealed against the decision. After the appeal decision, the case achieved the Oberste Gerichtshof (Austrian Supreme Court). Upon analysing the case, the Austrian Supreme Court affirmed that Facebook can be considered as an abettor to the unlawful comments; thus it may be required to take steps so as to repeat the publication of identical or similar wording. However, in this case, the injunction regarding such a pro-active role for Facebook could indirectly impose a monitoring role, which is in conflict not only with art. 15 of the e-Commerce Directive but also with the previous jurisprudence of the CJEU. Therefore, the Supreme Court decided to stay the proceedings and present a preliminary reference to the CJEU. The Court asked, in particular, whether art. 15(1) of the e-Commerce Directive precludes the national court to make an order requiring a hosting provider, who has failed to expeditiously remove illegal information, not only to remove the specific information but also other information that is identical in wording.Footnote ⁶⁷

The CJEU decided the case in October 2019. The decision argued that as Facebook was aware of the existence of illegal content on its platform, it could not benefit from the exemption of liability applicable pursuant to art. 14 of the e-Commerce Directive. In this sense, the Court affirmed that, according to recital 45 of the e-Commerce Directive, national courts cannot be prevented from requiring a host provider to stop or prevent an infringement. The Court then followed the interpretation of the AG in the case,Footnote ⁶⁸ affirming that no violation of the prohibition of monitoring obligation provided in art. 15(1) of the e-Commerce Directive occurs if a national court orders a platform to stop and prevent illegal activity if there is a genuine risk that the information deemed to be illegal can be easily reproduced. In these circumstances, it was legitimate for a Court to prevent the publication of ‘information with an equivalent meaning’; otherwise the injunction would be simply circumvented.Footnote ⁶⁹

Regarding the scope of the monitoring activity allocated to the hosting provider, the CJEU acknowledged that the injunction cannot impose excessive obligations on an intermediary and cannot require an intermediary to carry out an independent assessment of equivalent content deemed illegal, so automated technologies could be exploited in order to automatically detect, select, and take down equivalent content.

The CJEU decision tries as much as possible to provide a balance between freedom of expression and freedom to conduct a business, but the wide interpretation of art. 15 of the e-Commerce Directive can have indirect negative effects, in particular when looking at the opportunity for social networks to monitor through technological tools the upload of identical or equivalent information.Footnote ⁷⁰ This approach safeguards the incentives for hosting providers to verify the availability of harmful content without incurring additional levels of liability. However, the use of technical tools may pave the way to additional cases of false positives, as they may remove or block content that is lawfully used, such as journalistic reporting on a defamatory post – thus opening up again the problem of over-blocking.