‘My father was a white man,’ wrote Douglass. ‘He was admitted to be such by all I ever heard speak of my parentage.’ It was rumoured that his master was his father, but he had no way of telling. In any case, he was not named after Captain Aaron Anthony, Colonel Lloyd's chief overseer at the Great House Farm and a slaveholder himself, but after his mother, Harriet Bailey, though he barely knew her. Her duties on Anthony's own farm took her away from her children, and Frederick was brought up by his grandmother Betsy Bailey and her husband Isaac in their cabin not far away, but at the age of six the boy was sent to the Lloyd plantation twelve miles distant.

Frederick Augustus Washington Bailey – ‘Washington’ was a common patriotic flourish, while ‘Augustus’ may have honoured an uncle who died shortly before he was born – dispensed with his two middle names early in life. When he escaped North, he adopted several other surnames, including Stanley, designed to throw any pursuers off the track, and arrived in New Bedford, Massachusetts as Johnson. But the name Johnson was very common in that town, and was even the name of the family who took him in. It was time for another change. Recalling the occasion in his Narrative seven years later, Douglass wrote:

We don't know for sure why the surname of Walter Scott's James Douglas acquired an extra ‘s’. Douglass's biographer William McFeely has suggested that this was the way ‘prominent black families in Baltimore and Philadelphia spelled it’, and so he was merely conforming to a standard practice. But it was not just peculiar to those cities. The Federal Census of 1840 shows that ‘Douglass's was three times more common a surname in the United States than ‘Douglas’.