From Memory to Mastery explores the development of commercial numeracy and accounting in America and the English-speaking Atlantic world between 1750 and 1880. Most histories of accounting begin in the factories of England and New England, largely ignoring slave economies. I analyze both traditional sites of innovation, including textile mills and iron forges, and also southern and West Indian plantations. Along several dimensions, the calculative practices of slave owners advanced ahead of northern merchants and manufacturers, and many recorded and analyzed the productivity of their human capital with cruel precision. Following threads from Jamaica and Barbados to the American South, I show how plantation power relations stimulated the development of new accounting practices. The control of planters over their slaves made data easier to collect and more profitable to use. Commercial recordkeeping also expanded in free factories, but in different ways than on southern plantations. The mobility of labor made accounting necessary for keeping track of wages but relatively futile for detailed productivity analysis.