Although the exact date is not known, Robert Browne was born about the year 1550 at Tolethorpe Hall, which is the Manor House of Little Casterfon, a Rutlandshire village a few miles from Stamford. The house appears to have been owned by the Browne family since 1377, when it was bought by John Browne, alderman of Stamford (1374–6, -7) and “wool-merchant engaged in the staple of Calais.” A son of this original owner was named John, and also served as alderman. So, in turn, did his son John, who held he office three times. This third John Browne began the much needed work of restoring All Saints Church in Stamford, a fine building of Norman type which had suffered severe damage in the War of the Roses. The restoration was carried on and completed by his two sons, John and William, who also continued the family tradition of public service: John was alderman of Stamford in 1448, 1453, 1462; William was several times alderman as well as sheriff of Rutland and Lincoln counties, and founder in 1484 of Browne Hospital, the Bede House of Stamford, which still stands opposite the Corn Market.