Death came for James Madison in his eighty-fifth year, only six days before the anniversary of American Independence in 1836. The day before he died, Madison refused stimulants offered by his doctors, who thought they might allow him to live until the Fourth of July, the day that other former Presidents – Jefferson, Adams, and Monroe – had died. Soon after six in the morning, Madison characteristically slipped off “quietly,” unceremoniously.

His life ended with a brief, ambiguous comment that exemplified the controversy surrounding his final years. As recounted by his slave Paul, the former president took breakfast as usual, in his room, but had difficulty swallowing. He was asked, “What is the matter?” Madison seemed to abandon the effort to start the meal and the day as well, explaining before losing consciousness and his life, that he was experiencing “[n]othing more than a change of mind.”

Madison spent much of his last years asserting that he had not changed his mind about some things – such as the Union, the federal Constitution, and that in America the people were the sovereign. Retirement from the presidency nineteen years earlier started peacefully enough when he moved from the White House to Montpelier, his estate near the foothills of Virginia's Blue Ridge Mountains. It was a time, his neighbor Thomas Jefferson assured him, to finally be “release[d] from incessant labors, corroding anxieties, active enemies and interested friends.” Retirement promised a “return to your books and farm, to tranquility and independence.”