Introduction: Many health technologies exhibit some form of learning effect, and this represents a barrier to rigorous assessment. It has been shown that the statistical methods used are relatively crude. Methods to describe learning curves in fields outside medicine, for example, psychology and engineering, may be better.

Methods: To systematically search non–health technology assessment literature (for example, PsycLit and Econlit databases) to identify novel statistical techniques applied to learning curves.

Results: The search retrieved 9,431 abstracts for assessment, of which 18 used a statistical technique for analyzing learning effects that had not previously been identified in the clinical literature. The newly identified methods were combined with those previously used in health technology assessment, and categorized into four groups of increasing complexity: a) exploratory data analysis; b) simple data analysis; c) complex data analysis; and d) generic methods. All the complex structured data techniques for analyzing learning effects were identified in the nonclinical literature, and these emphasized the importance of estimating intra- and interindividual learning effects.

Conclusion: A good dividend of more sophisticated methods was obtained by searching in nonclinical fields. These methods now require formal testing on health technology data sets.

By the mid-1960s, American writer Kay Boyle was in possession of a three-book contract from Doubleday publishers in New York. The cornerstone of this deal was to be a history of Germany, a manuscript she began in the late 1950s. Boyle encountered difficulties completing this work, and after lobbying successfully to write a history of German women instead, she eventually abandoned the project altogether. To help her meet her professional obligations, Boyle hoped that Doubleday would accept a new plan to republish Three Short Novels, a work that had appeared under the Beacon imprint in 1958. That publisher still had four thousand copies of the book in its warehouse, however, and Doubleday editor Ken McCormick was unable to agree to Boyle’s proposal. McCormick suggested instead that she undertake work revising Robert McAlmon’s 1938 autobiography, Being Geniuses Together. Indeed, in the years following his death in 1956, Boyle had been unsuccessful in locating an American publisher for her friend’s book, so when Doubleday brought forward an edition of the work in 1968, it contained alternate chapters written by Kay Boyle, herself. McAlmon’s original text is approximately one hundred and ten thousand words in length; Boyle’s edition is one hundred and sixty thousand words, only seventy thousand of which were written by Robert McAlmon. ‘‘This present book is his,’’ Boyle wrote of McAlmon’s achievement in her 1984 afterword (333), and while one might argue that this is the case, no one can question the fact that his book was altered substantially from its original form.

Objective: We reviewed and appraised the methods by which the issue of the learning curve has been addressed during health technology assessment in the past.

Method: We performed a systematic review of papers in clinical databases (BIOSIS, CINAHL, Cochrane Library, EMBASE, HealthSTAR, MEDLINE, Science Citation Index, and Social Science Citation Index) using the search term “learning curve.”

Results: The clinical search retrieved 4,571 abstracts for assessment, of which 559 (12%) published articles were eligible for review. Of these, 272 were judged to have formally assessed a learning curve. The procedures assessed were minimal access (51%), other surgical (41%), and diagnostic (8%). The majority of the studies were case series (95%). Some 47% of studies addressed only individual operator performance and 52% addressed institutional performance. The data were collected prospectively in 40%, retrospectively in 26%, and the method was unclear for 31%. The statistical methods used were simple graphs (44%), splitting the data chronologically and performing a t test or chi-squared test (60%), curve fitting (12%), and other model fitting (5%).

Conclusions: Learning curves are rarely considered formally in health technology assessment. Where they are, the reporting of the studies and the statistical methods used are weak. As a minimum, reporting of learning should include the number and experience of the operators and a detailed description of data collection. Improved statistical methods would enhance the assessment of health technologies that require learning.