The Order of Carmelites (the Whitefriars) originated as a band of hermits living on Mount Carmel near Haifa in Israel, possibly from as early as the 1160s. Christian forces secured the area by 1192. Within a short time, the hermits appealed to the patriarch of Jerusalem, Albert Avogadro, who provided them with a formula vitae, a set of guidelines for them to establish a more formalised community. The Carmelites now lived in silence and poverty, remaining in their separate cells apart from a single daily celebration of mass in their church. They were governed by a prior, a ‘first amongst equals’. Between 1226 and 1229, three papal bulls gave approval of their way of life and their community grew. However, increasing attacks by neighbouring Muslim kingdoms made the hermits’ position precarious and prompted them to found new sites elsewhere: Cyprus in 1235, then Messina (Sicily) and Les Aygalades (France) by c.1238.

Shortly afterwards, Richard of Cornwall led a group of soldiers as part of the Barons’ Crusade, arriving in Acre in 1241. Tradition has it that when two of his noblemen visited Mt. Carmel they discovered a fellow Northumbrian, Ralph Fresburn, an ex-crusader who had been living as a hermit there for twenty years. They ‘besought the Prior to allow them to take him home with them and found a Carmelite house in Northumberland’ and so a small colony of Carmelites led by Ivo the Breton left for England with the Earl of Cornwall's retinue. Under the patronage of some of Cornwall's knights, the Carmelites, beginning in 1242, built their first houses at Hulne (Northumberland), Aylesford (Kent) and Lossenham (or Losenham, Kent). They also established a house at Bradmer, in Burnham Norton on the north Norfolk coast, founded between 1242 and 1247 by Sir William de Calthorp (probably lord of Burnham Thorpe manor) and Sir Ralph de Hemenhale (holder of part of Polstead Hall, another local manor). Calthorp and Hemenhale were probably also part of Cornwall's forces, which included many unidentified knights. It is likely that two early members of the Burnham Norton house, Peter and Reginald Folsham, had been hermits on Mt. Carmel, rather than being men who later joined the group after it arrived in England. However, their surname suggests a Norfolk connection: Foulsham (Domesday, ‘Folsham’) being a village c.28km (17.6 miles) south-east of Burnham Norton.