This article explores the removal or exclusion in the late 1940s of people in interracial marriages from two corners of the newly formed Commonwealth of Nations, Australia and Britain's southern African colonies. The stories of Ruth and Sereste Khama, exiled from colonial Botswana, and those of Chinese refugees threatened with deportation and separation from their white Australian wives, reveal how legal rearticulations in the immediate postwar era created new, if quixotic, points of opposition for ordinary people to make their voices heard. As the British Empire became the Commonwealth, codifying the freedoms of the imperial subject, and ideas of universal human rights “irrespective of race, color, or creed” slowly emerged, and claims of rights long denied seemed to take on a renewed meaning. The sanctity of marriage and family, which played central metaphorical and practical roles for both the British Empire and the United Nations, was a primary motor of contention in both cases, and was mobilized in both metaphorical and practical ways to press for change. Striking similarities between our chosen case studies reveal how ideals of imperial domesticity and loyalty, and the universalism of the new global “family of man,” were simultaneously invoked to undermine discourses of racial purity. Our analysis makes a significant contribution to studies of gender and empire, as well as the history of human rights, an ideal which in the late 1940s was being vernacularized alongside existing forms of claim-making and political organization in local contexts across the world.