While legal papers and case decisions have been the traditional focus of judicial biography, the family papers of Justice John Marshall Harlan the Elder demonstrate the importance for understanding a judge's conception of the polity of shifting our sights to the household. Historians of the 19th century have overestimated the distance between the private and the public spheres. The memoirs of Harlan's wife Malvina offer us unparalleled, and hitherto neglected, testimony. Her depiction of the antebellum Harlan household shows its two hierarchies based on assumptions of fundamental differences—those of gender and of race—and both positing a benevolent white male paternalist at their apex. Malvina Harlan's memoirs indicate the lifelong persistence of this paternalism in her own relationship with Justice Harlan and in his relationship with a black servant. These patterns of hierachy, separation, and mutual devotion were essential to Harlan's understanding of his family identity and personal duty. His famous dissents in favor of black civil rights protections and his lapses from his color-blind rule have their roots in this paternalism even as Harlan came to embrace the racial egalitarianism of the Civil War amendments.