Retiring from Government service did not mean the end of Milner's imperial activities. As the last of the original trustees still in place, he was now able to devote more time to the Rhodes Trust, the only non-official position he had not resigned when he joined the War Cabinet five years before. Amery later commented about Milner's work at the Trust that if the ‘vision was Rhodes’, it was Milner who over some twenty years laid securely the foundations of a system whose power in shaping the outlook and spiritual kinship of an ever-growing body of men throughout the English-speaking world it would be difficult to exaggerate.’

In June 1921 Geoffrey Dawson (who had left The Times after a falling out with Northcliffe and lately had been editor of the Round Table) was appointed Secretary of the Trust. He wrote to Milner, ‘I am really most grateful & delighted to settle down to the Rhodes Trust as a definite piece of work & to try to make a success of it … There is a good deal to be done.’ Dawson remained Secretary for more than a year, until Northcliffe's premature death allowed him to return (with Milner's blessing and guidance) to the editor's chair at The Times under the new Astor ownership. That summer Milner arranged a final contribution of £2500 from the Rhodes Trust to keep the Round Table afloat. Reflecting the new post-war reality, since the March 1919 issue the journal was no longer subtitled ‘A Quarterly Review of the Politics of the Empire’, but instead of the ‘Commonwealth’. Milner continued as a father-figure, but by this time the movement had lost much of its vigour, diminished by a lack of new leadership, by the war (which had only stimulated dominion nationalism), and the new emphasis on international cooperation embodied by the League of Nations. Curtis, who founded the Institute of International Affairs in London, Kerr and Brand all followed the last path, much to Milner's regret. In his view they ‘dropped the substance for the shadow’.