For nine years after its founding on 8 August 1967, the Association of Southeast Asian Nations (ASEAN) did not have a central secretariat, although there were occasional suggestions for the establishment of one. To ASEAN at that time, the important thing was for leaders, ministers and officials to get to know one another, to form friendships with one another, and to build confidence among themselves. Working informally and pragmatically was the only way to ensure that they would regard one another with less suspicion and greater comfort.

It was not until 24 February 1976, on the occasion of ASEAN's first summit meeting, in Bali, that the ASEAN foreign ministers signed an agreement establishing the ASEAN Secretariat. (It was also the occasion when ASEAN concluded its first binding agreement, the Treaty of Amity and Cooperation in Southeast Asia.) The Secretariat was to be in Jakarta, with the Indonesian Government, in 1981, turning over to the association the land and building where it is now located.

The Secretariat at that time was rudimentary at best. Its officers were seconded from their respective governments. The secretary-general was nominated by members in the alphabetical order of the countries' English names and for a term of two years. In order to make clear that the secretary-general would not perform any supranational functions, he was called the “Secretary-General of the ASEAN Secretariat”. The quip went around at that time that he was more secretary than general.

The 1976 agreement went into great detail, specifying that the Secretariat would have three bureau directors–one each for economic, science and technology, and social and cultural affairs. The agreement also stipulated that there would be a foreign trade and economic relations officer, an administrative officer, an information officer, and an assistant to the secretary-general. The ASEAN Standing Committee, the association's governing body of national officials, would appoint these officers for terms of three years, renewable once for another three years. The Secretariat was allowed to hire locally recruited personnel.