Child obesity is an important public health problem due to its high and growing frequency and serious health consequences(1). Obese children have a higher short-term risk of type 2 diabetes mellitus, dyslipidaemia, high blood pressure, and respiratory, orthopaedic and psychosocial disorders(Reference Reilly and Wilson2). They also have a greater risk of obesity in adulthood(Reference Whitaker, Wright and Pepe3).
It has been suggested that television (TV) viewing is contributing to the current epidemic of child obesity(Reference Livingstone and Helsper4–Reference Taveras, Sandora and Shih7). This is thought to occur via two mechanisms: (i) an increase in sedentariness and food consumption while watching TV; and (ii) the adoption of poorer eating habits due to food advertising(Reference Livingstone and Helsper4), in particular greater consumption of high-energy nutrient-poor foods and beverages(Reference Dixon, Scully and Wakefield8–Reference Lobstein and Dibb11). Due to children’s influence on family purchases, their own purchasing capacity and their role as future consumers, food and beverage companies consider children a valuable target(Reference Aktas10), so advertising aimed at children has increased. In Spain, the proportion of TV food advertising in the children’s time slot increased from 48 % in 2005 to 56 % in 2007(12), and it is estimated that Spanish children currently watch an average of twenty-two food and beverage advertisements per day(13).
Worldwide there are three types of regulation of food and beverage advertising aimed at children: statutory regulations, non-statutory government guidelines and self-regulations(Reference Hawkes14). Self-regulation codes are currently the predominant type, especially in TV advertising, and have been developed primarily by the advertising and food industries(15).
Spain has the PAOS Code (Publicidad, Actividad, Obesidad y Salud) for self-regulation of food advertising that targets minors(16). This code establishes the ethical principles and standards for the design and dissemination of advertising messages from the companies that voluntarily agreed it, in any media or support that will be broadcasted, as well as the control mechanisms (supervision and sanctions) that will apply to them. It is a pioneering code in European countries launched in 2005 within the framework of the National Strategy for Control of Obesity (NAOS – Nutrición, Actividad, Obesidad y Salud), promoted by the Spanish Ministry of Health(17) in light of the alarming rate of childhood overweight in Spain, one of the highest in primary-school children (31 % compared with 13 % in Germany, 15 % in Serbia and Montenegro)(Reference Branca, Nikogosian and Lobstein18). The Spanish prevalence of obesity in 2006 was 8·9 % in children aged 2–17 years, with 9·1 % in boys and 8·7 % in girls(19), twice the figures seen two decades earlier(Reference Aranceta-Bartrina, Serra-Majem and Foz-Sala20).
Supervision of compliance with the PAOS Code is the responsibility of a private organization, the Association for the Self-Regulation of Commercial Communication (Autocontrol)(21). Autocontrol carries out two types of supervision(16): (i) a priori (by its Technical Cabinet), by confidential examination (‘copy advice’) of all food and beverage advertisements in TV aimed at children under 12 during children’s ‘reinforced protection’ viewing time (08.00–09.00 hours and 17.00–20.00 hours on Monday through Friday, and 09.00–12.00 hours on Saturday and Sunday)(22); and (ii) a posteriori (by its Advertising Jury), by resolving complaints on non-compliance with the PAOS Code in the advertising of companies that agreed to abide by the Code.
The PAOS Code establishes sanctions for infractions of the Code, which range from fines to temporary or permanent exclusion of the company from the code agreement. The company′s exclusion is considered in cases of repeated commission of infractions categorized as ‘serious’ or ‘very serious’ or when the company does not respect a resolution emitted by the Advertising Jury. The reasons for the company’s exclusion are made public. The Code also establishes a Monitoring Commission, created to evaluate how the Code performs and whether it is being implemented effectively. Among the Commission’s functions are periodic reviews of reports presented by Autocontrol on code compliance and filing of complaints to the Advertising Jury when considered appropriate.
Many initiatives are currently being developed under WHO sponsorship to develop an international code to regulate food and beverage advertising and to promote national regulatory actions that help protect children from the adverse dietary effects of exposure to this type of marketing(23). At the same time, there is debate in many countries as to whether food advertising should be more tightly regulated. Although most European countries have opted for self-regulation, some governments have warned that if these measures fail they will implement statutory regulations like those that are beginning to be applied in some European countries(Reference Hawkes24).
The current study evaluates the degree to which Spanish TV advertising complies with the PAOS Code. We have studied TV advertisements only, because in Spain TV has the highest level of penetration (88·9 %) and children spend a long time watching TV (an average of 218 min/d)(Reference González Díaz25). TV also is at the top of the ranking of communication media according to total marketing expenditure(26), in particular for food marketing. The results of this evaluation can provide important new information on the effectiveness and usefulness of self-regulation that may help public health authorities to make decisions about regulatory measures to be adopted in the near future.
Methods
Study setting and design
We estimated that 198 advertisements was the minimum number that needed to be watched for estimating the percentage of non-compliance with the PAOS Code with a precision of ±7 % and 95 % confidence. The final study sample consisted of food and beverage TV advertisements aimed at children that were broadcast during 80 h of programming by four of the ten channels that can be watched free-of-charge in Spain. Two of them were national channels (TVE1 and Antena 3) and two were regional ones (Canal Sur and TeleMadrid). Among regional channels, we selected Canal Sur and TeleMadrid because they corresponded to regions with one of the highest (Andalusia) and lowest (Madrid) prevalence of obesity. Among the national channels, we selected Antena 3 (private) and TVE1 (public) because they broadcast five and two, respectively, of the seven most popular programmes for children in Spain(Reference Núñez Ladevéze, Gómez Amigo and Vázquez Barrio27).
The advertisements were recorded during the following viewing periods: 08.00–11.00 hours, 13.00–15.00 hours and 17.00–22.00 hours, so as to include both the so-called ‘reinforced protection’ time slot(22) for the regulation of advertising aimed at children and other time slots that are less protected but still have a large child audience(Reference Núñez Ladevéze, Gómez Amigo and Vázquez Barrio27). Recording did not take place during school vacation periods, holidays, or the eve of a holiday or of the first day of vacation when it was a school day, to maintain the balance between the advertisements broadcast in school days and in weekend days (represented by the Saturday); such balance fairly represents children’s exposure to TV over most of the year.
The following broadcasts were recorded during 2008: Friday 9 May and Saturday 24 May (TVE1); Thursday 5 June and Saturday 24 May (Antena 3); Thursday 22 May and Saturday 24 May (TeleMadrid); Friday 23 May and Saturday 7 June (Canal Sur). For each network we selected one school day (Thursday or Friday) and one weekend day (Saturday) to analyse possible differences between them.
Definition of advertisements targeting children
According to the PAOS Code, a food or beverage advertisement is considered to be aimed at children when it meets any of the following criteria(16).
1. For the type of food product promoted: Advertisements that objectively promote a food product (measured by Dym Panel, a company of market research, or Homescan™, a consumers’ panel developed by Nielsen) aimed primarily at children under 12.
2. For the design of the advertising message: Advertising designed in such a way that the content, language and/or images are objectively appropriate primarily for children under 12.
3. For the way in which the advertising message is disseminated: Advertising disseminated either in a medium or support aimed objectively (measured by Sofres, a company specialized in measuring TV audiences) and primarily to children under 12, or in a general communications medium (TV channels that broadcast programmes aimed both to adults and children, in contrast to those aimed exclusively to children) when inserted in time slots, programming blocks, sections or spaces aimed at children under 12 or with an audience consisting primarily of children under 12.
In cases of doubt, we applied the criteria used by Hawkes in her WHO report Marketing Food to Children: The Global Regulatory Environment (Reference Hawkes14), which considers that advertising is aimed at children when the answer to any of the following questions is affirmative.
1. ‘The type of product or service being marketed: Is it intended exclusively for children? Or is it very interesting to them?
2. The manner in which the marketing is presented: Does it use colours, voices, images, music or sounds of the type that captivate children? Does it involve activities, such as collecting or drawing, which are likely to be popular with children? Does it involve characters with whom children are likely to identify?
3. The place and time of the marketing campaign: Is it broadcast at a time when children are likely to be watching?’
The recordings were examined by a single evaluator (M.M.R.-F.) who analysed all the advertisements and consulted with a second evaluator (M.A.R.-B.) in cases of doubt.
PAOS Code
The PAOS Code has been agreed by thirty-six companies representing over 75 % of food and beverage marketing in Spain. The list of companies is public and is available at the website of the Spanish Ministry of Health(28).
The Code consists of twenty-five ethical standards grouped into the following sections(16): principle of legality (advertising should adhere to the current legislation, independently of its content, the medium of broadcasting or the form that can adopt – Standard 1); principle of loyalty (advertising should adhere to the demands of good faith and good commercial practice – Standard 2); product presentation (should not give misleading information about product characteristics, and should not exploit children’s credulity – Standards 3, 4, 5, 6 and 7); product information (appropriate in form and content for a child audience – Standard 8); sales pressure (should not urge children to obtain the product – Standards 9, 10, 11 and 12); promotion by persons and programmes (avoid using famous persons who are popular with children – Standards 13, 13·1, 13·2 and 14); identification of the advertisement (in the case of promotional offers, guarantee appropriate information for children – Standard 15); comparative presentations (advertising where the characteristics of a branded product are compared against those of a competitor product – Standard 16); promotions, prize drawings, competitions and children’s clubs (establishes the conditions that the advertisements must meet – Standards 17, 18, 19 and 20); safety (Standards 21 and 22); and, finally, nutritional education and information (should not promote unhealthy dietary habits or lifestyles – Standards 23, 24 and 25).
Evaluation of compliance with the PAOS Code
The degree of compliance with the PAOS Code was evaluated by classifying the advertisements into three possible categories for each standard: ‘compliant’, ‘non-compliant’ or ‘uncertain compliance’. The ‘uncertain’ category was established because of the difficulty of objectively evaluating compliance with a standard in some cases.
We did not evaluate compliance with standards that refer to the principles of legality and loyalty, which are required in all TV advertising. The standards regarding conditions for additional product information and promotional offerings were evaluated for all advertisements with the appropriate characteristics (thirty and seventy, respectively). Finally, there were no advertisements that mentioned the price of the product, that made comparative presentations or that referred to children’s clubs; therefore we did not evaluate these standards.
In evaluating overall compliance with the PAOS Code, an advertisement was considered to be compliant if it met all the standards, non-compliant when it contravened one or more standards, and of uncertain compliance in all other cases. The results were expressed as the percentage of advertisements in each compliance category for each standard, together with the 95 % confidence intervals. Comparison of the percentage of compliance with the PAOS Code between different types of foods, advertising characteristics and manufacturing companies was made using the χ 2 test. Food categories were adapted from other studies(Reference González Díaz25), according to the number of advertisements in each category, and reserving a category of ‘others’ for food groups with only a few advertisements. Statistical significance was set at P < 0·05. Statistical analyses were performed with the STATA statistical software package version 10 (Stata Corporation, College Station, TX, USA)(Reference Hills and De Stavola29).
Results
During the 80 h of TV recording, 264 food and beverage advertisements aimed at children were broadcast; this represented 52·9 % of the food advertisements and 22·2 % of all advertisements broadcast. Of the 264 advertisements, 203 (76·9 %) were from companies that agreed to abide by the PAOS Code, and the rest were from companies that did not.
Table 1 shows the characteristics of the food and beverage advertisements aimed at children. Some 75·9 % of the advertisements from companies that agreed to the PAOS Code were broadcast on a Saturday. The most widely advertised products were dairy products, followed by cereals and pastries/bakery goods. Most advertisements were for products manufactured by French or US companies. Some 52·2 % of the advertisements lasted for 10 s or less. Some 61·6 % were presented in the form of a spot. Most of the advertisements from companies that agreed to the PAOS Code were broadcast in the morning (28·6 %) and early afternoon (‘access-afternoon’; 21·2 %). Finally, 60·6 % of the advertisements were broadcast outside the ‘reinforced protection’ viewing time.
Table 1 Characteristics of food and beverage advertisementsFootnote * aimed at children by companies that agreed to the PAOS CodeFootnote †
* Advertisements during 80 h of programming by four Spanish television networks in 2008; see Methods section for details.
† The PAOS Code (Publicidad, Actividad, Obesidad y Salud) establishes standards for the self-regulation of food marketing aimed at minors in Spain.
‡ P for heterogeneity among proportions in the same sample.
§ Rest of Europe: Great Britain, Switzerland, Germany, Italy, The Netherlands and Austria.
Table 2 shows overall compliance of advertisements from companies that agreed to the PAOS Code. In 49·3 % of cases the advertisements did not comply with the Code, and in 20·7 % compliance was uncertain. Among companies that did not agree to the Code, non-compliance was 50·8 %, with 32·8 % of advertisements of uncertain compliance (P = 0·045; data not shown in table).
Table 2 Overall compliance with the PAOS CodeFootnote * by advertisementsFootnote † of companies that agreed to the Code
* The PAOS Code (Publicidad, Actividad, Obesidad y Salud) establishes standards for the self-regulation of food marketing aimed at minors in Spain.
† Advertisements during 80 h of programming by four Spanish television networks in 2008; see Methods section for details.
‡ P for comparison of proportions (χ 2 test).
§ Rest of Europe: Great Britain, Switzerland, Germany, Italy, The Netherlands and Austria.
Non-compliance with the PAOS Code was most frequent on Saturdays, on Canal Sur and in the advertisements of US and French food manufacturers. With regard to product groups, the advertisements that most frequently did not comply with the Code were for dairy products and meat/meat products. Non-compliance was also higher in advertisements that presented some kind of promotion, those that lasted more than 20 s, and those that broadcast in access prime-time and early morning, and outside the reinforced protection time slot (Table 2).
Table 3 presents the advertisements from companies that agreed to the PAOS Code according to each compliance category. Of the PAOS Code standards that could be evaluated for all advertisements (each standard is briefly described in a footnote to Table 3), those with the lowest level of compliance were the following: Standard 23, which refers to avoiding promotion of unhealthy eating habits or lifestyle (83·7 %); Standard 9, which refers to not directly calling on or urging children to ask for the product, and to avoid suggesting that adults who buy the product are better persons (88·7 %); Standard 6, consisting of not exploiting children’s credulity (91·6 %); and Standard 13, which indicates not exploiting children’s special trust in adults, parents and well-known individuals (91·6 %). All eligible advertisements (n 30) did not comply with Standard 8, on conditions for additional product information; and 88·6 % of eligible advertisements (n 70) did not comply with Standard 18, on presentation of essential conditions for a promotional offer.
Table 3 Compliance with individual standards of the PAOS CodeFootnote * by advertisementsFootnote † of companies that agreed to the code
* The PAOS Code (Publicidad, Actividad, Obesidad y Salud) establishes standards for the self-regulation of food marketing aimed at minors in Spain.
† Advertisements during 80 h of programming by four Spanish television networks in 2008; see Methods section for details.
‡ Brief description of standards: Standard 3, not to give misleading information about product characteristics; Standard 4, not to give misleading information about the benefits of a product; Standard 5, not to attribute particular characteristics to a product when these characteristics are inherent to all similar products; Standard 6, not to exploit children’s credulity; Standard 7, avoid presentations that may frighten children; Standard 8, conditions to be met for additional product information; Standard 9, not to directly call on or urge children to ask for the product and not to suggest that adults who buy the product are better; Standard 10, no reward for obtaining the product; Standard 11, not to convey the impression that buying the product means greater acceptance among friends or confers prestige, skills or other special qualities of the persons in the advertisement; Standard 12, conditions to be met if price is mentioned; Standard 13, not to exploit children’s special trust in adults, parents, well-known persons; Standard 13·1, on the participation of famous persons or characters who are particularly well known to children; Standard 13·2, on the appearance of well-known or famous persons; Standard 14, on the existence of telepromotions of foods and beverages in programmes aimed at children; Standard 15, clear separation between advertising and programmes; Standard 16, conditions of comparative presentations; Standard 17, in cases of promotion, separation of the product advertised and the one promoted; Standard 18, presentation of essential conditions in the case of promotional offers; Standard 19, conditions to be met in the case of commercial prize draws; Standard 20, conditions to be met in the case of children’s clubs; Standard 21, not to encourage dangerous or inappropriate use of the product; Standard 22, not to urge children to go with strangers or into unknown places; Standard 23, not to promote or present unhealthy eating habits or lifestyles; Standard 24, not to present the product as a substitute for a meal; Standard 25, not to underestimate the importance of healthy lifestyle habits.
§ Evaluated for a total of thirty advertisements that presented additional information.
|| Evaluated for a total of seventy advertisements that presented some form of promotion.
The results of companies that did not agree to the Code were generally similar to those that did. The most notable differences were the lower percentage of compliance with respect to Standard 6, which refers to not exploit children’s credulity (83·6 % v. 91·6 %); Standard 13·1, which refers to avoid the participation of famous persons or characters who are particularly well known to children (90·2 % v. 99·0 %); and Standard 13·2, which refers to avoid the appearance of well-known or famous persons (80·3 % v. 94·6 %); and the 100 % compliance with Standard 9, which refers to not directly call on or urge children to ask for the product and not to suggest that adults who buy the product are better v. 88·7 % of the advertisements from the companies that agreed to the Code (data not shown in the table).
Discussion
In our study, almost one-quarter of the food and beverage advertisements aimed at children were for products from companies that did not agree to the PAOS Code. Moreover, non-compliance with the Code was very high, and was similar for companies that did and did not agree to the Code. In particular, non-compliance was very frequent with the standards that could be quantified most objectively, such as the ones that regulate the conditions for promotion and additional product information and, to a lesser extent, the one that prohibits well-known persons from appearing in advertisements.
In the reports made by Autocontrol until 2008, the percentage of advertisements in which it was advised they not be broadcast and in which previous modifications were recommended were, respectively, 5 % and 22 % in 2006(30) and 6 % and 21 % in 2007(31). About the last of these reports Autocontrol states that all advertisements for which some modification was recommended were corrected before they were broadcast(31). However, our study showed there was also a high level of non-compliance with the PAOS Code for advertisements in the reinforced protection time slot, which are subject to previous review before broadcasting. With regard to the ten post-broadcast complaints presented up to 2008, only three were considered completely valid by Autocontrol(32). It has been recently remarked that some of the criteria for self-regulation systems rely on vague or subjective notions that make it less likely that a consumer will take the trouble to complain, and harder to obtain a breach finding in the case of a complaint(Reference Handsley, Mehta and Coveney33). We do not deal with the evaluation of these complaints in our study; however, analysis of a recent complaint in New Zealand about an advertisement for chicken nuggets, which detected the use of rhetorical strategies designed to undermine the complaint and the complainant, illustrates those points, throwing serious doubts on the level of protection that self-regulation offers to consumers(Reference Hoek34).
The use of famous persons or of texts that skirt the PAOS Code standards has been observed with other self-regulation codes in Spain, such as the code for tobacco advertising, which preceded the current total ban on this kind of advertising(Reference Martín, Quiles and López35), and the advertising codes for beer and spirits(36). In countries with a long tradition of self-regulation of food advertising, like Australia, the tendency to evade advertising code standards has also been observed(Reference Harker37). In countries with more restrictive regulations, like Canada and Norway, advertisements have been modified to evade the standard that prohibits direct targeting of children; for example, a lawyer for marketing companies in Quebec recommended advertisers to direct their messages to both parents and children(Reference Hawkes14). It was argued that the advertisement was addressed to parents or to the whole family, even though the type of product, content and timing of the broadcast clearly indicated that the advertisement was aimed at children; this situation occurred with an advertisement for ‘Smacks’® breakfast cereals broadcast in Norway(Reference Hawkes14). More recently, the Spanish Organization of Consumers and Users alerted on the use of famous persons to promote unhealthy foods during children’s viewing time in April 2008(13).
As opposed to countries with statutory regulations or non-statutory government guidelines for food advertising aimed at children like the UK, which applies partial restrictions (for energy-dense, nutrient-poor foods)(38), or Sweden, Quebec and Norway, which prohibit all such advertising(Reference Kelly, King and Bauman39), self-regulation is the most widely used system for regulating food advertising in the world(Reference Hawkes40). In this context, the PAOS Code is among the most restrictive. However, almost one-quarter of the food and beverage advertisements aimed at children were from companies that did not agree to the Code, and half of the advertisements were from companies that did not comply with its standards. Moreover, non-compliance was higher during Saturdays and in the early morning, early afternoon and prime-time slots, when children are more likely to be exposed to the advertisements(Reference Núñez Ladevéze, Gómez Amigo and Vázquez Barrio27); non-compliance was also higher for dairy products, which are widely consumed by children. For all of these reasons, and because non-compliance was similar regardless of whether or not companies agreed to the Code, the usefulness of the PAOS Code is questionable. It should also be added that the Code is unable to prevent the dissemination of unhealthy food habits among children, since it does not regulate either the volume of advertising or the type of products advertised. Furthermore, children are also influenced by advertisements targeted to adults and even the statement of self-regulation codes, claiming that children are more influenced by advertising that is aimed at them, is not supported by the research(Reference Handsley, Mehta and Coveney33). As has repeatedly been pointed out, self-regulation codes will not manage to substantially reduce the high volume and strong impact of obesogenic food and beverage marketing on children, even when compliance is strict(Reference Hawkes14, Reference Hawkes41). The International Obesity Taskforce has established the ‘Sydney Principles’ on marketing practices that target children(42). One of these principles proposes statutory regulations as the only way to adequately protect children against food marketing. However, the food industry has disputed the need for statutory regulations, arguing that self-regulation systems are working well(Reference Hawkes24). Our findings contradict that argument.
In addition to its other limitations, the PAOS Code does not regulate either the number of times an advertisement can be broadcast per day or the cumulative effect of similar products or messages. Moreover, when marketing industries are self-regulated(Reference Branca, Nikogosian and Lobstein18), acting as both prosecution and defence, judge and jury, the probability of important sanctions against their own voluntary members is low. Finally, resolution of complaints after an advertisement has been broadcast is usually so slow that it may not occur until after the marketing campaign is over, thus weakening the effect of the sanctions. Furthermore, while the complaint is being resolved, inappropriate or erroneous nutritional messages are broadcast and may be believed by children(Reference Branca, Nikogosian and Lobstein18, Reference Hoek34).
In our study, the advertisements of companies from countries with more restrictive marketing regulations, such as the UK and others in central and northern Europe, were more compliant with the PAOS Code than those of companies from less restrictive countries like France and the USA(Reference Hawkes40). This suggests that the regulatory environment in each country has a positive influence of on the marketing techniques of their companies.
Finally, food and beverage advertising aimed at children on Canal Sur (Andalusia) exerted less marketing pressure than TeleMadrid (Madrid). However, non-compliance with the PAOS Code was higher in Canal Sur than in TeleMadrid. It is difficult to evaluate whether this could have some influence on the different prevalence of child obesity in these two regions(19).
Our study has some limitations. The two most important are the relatively small number of advertisements and hours of broadcasting, and the inevitable subjectivity in the evaluation of compliance with some standards. It is not likely that a larger sample size would change our conclusions, as the size increase would probably result from the same advertisements repeated several times. Neither is it likely the conclusions would change with the selection of other TV channels, because both the number and type of advertisements are similar across channels. To minimize the subjectivity, the advertisements were reviewed by a second evaluator in case of doubt. Nevertheless, with the exception of the use of famous persons, the PAOS Code standards with more objective evaluation criteria, such as those relative to the promotional offers and additional product information (Standards 8 and 18), had the highest levels of non-compliance, suggesting that the inherent subjectivity in the evaluation of compliance in some standards could, in any case, infra-estimate non-compliance. To check for reliability of assessment, we asked eight physicians attending a master of public health programme to independently assess a sample of six advertisements, which we previously had deemed non-compliant with the PAOS Code. All of them also considered the six advertisements to be non-compliant, and noted that some advertisements failed to comply with even more standards than those we did.
In conclusion, compliance with the PAOS Code is low, and is similar in companies that do and do not agree to abide by it. These results suggest the limited effectiveness of the Code and its oversight system. It thus seems sensible to propose the adoption of statutory regulations that reduce the negative impact of advertising on children’s diets, as repeatedly demanded by public health experts and consumer organizations. Restrictions on commercial advertising of energy-rich nutrient-poor foods and beverages, similar to those recently implemented in the UK, are a positive though limited step, because they are based on the proportion of children in the audience rather than in the absolute number(43, Reference Gibson and Smithers44).
Acknowledgements
The authors declare that they have not received any type of funding for conducting the study and there is no conflict of interest. M.M.R.-F. took part in the design of the study, bibliographical research, interpretation of results, editing every part of the manuscript and checking it. She did the fieldwork collecting the data and analysed the data in collaboration with M.A.R.-B., being the first evaluator. M.A.R.-B. took part in the design of the study, bibliographical research, interpretation of results, editing every part of the manuscript and checking it. He analysed the data in collaboration with M.M.R.-F. and he was the second evaluator in cases of doubt. F.R.-A. took part in the design of the study, bibliographical research, interpretation of results, manuscript editing and checking it.
