We carried out single-walled carbon nanotube (SWCNT) growth using a Rh catalyst on Al2O3 buffer layers that were prepared by three different methods based on electron beam (EB) evaporation: native oxidation of Al layer deposited by EB ([EB(Al)+NO]-Al2O3 layer); thermal oxidation of Al layer deposited by EB ([EB(Al)+TO]-Al2O3 layer); EB deposition of Al2O3 layer ([EB(Al2O3)]-Al2O3 layer). SWCNT yield was the largest for the [EB(Al2O3)]-Al2O3 layer, while SWCNTs were not grown on the [EB(Al)+NO]- Al2O3 layer. Transmission electron spectroscopy showed that most of Rh particle sizes were distributed between 1.0 and 2.6 nm on the [EB(Al)+NO]- Al2O3 and [EB(Al2O3)]- Al2O3 layers, while they were distributed between 1.8 and 4.2 nm on the [EB(Al)+TO]- Al2O3 layer. This result indicates that surface migration of Rh catalysts was suppressed on the [EB(Al2O3)]- Al2O3 layer, resulting in the largest SWCNT yield. On the other hand, enlargement of Rh catalyst particles occurred on the [EB(Al)+TO]- Al2O3 layer, leading to the reduction of SWCNT yield. Taking into account our previous study, inward diffusion of Rh catalysts into the Al2O3 buffer layer inhibited SWCNT growth on the [EB(Al)+NO]- Al2O3 layer, although enlargement of Rh particle size was suppressed. We also carried out ultra-violet photoemission measurements for Rh catalysts on the [EB(Al)+TO]- Al2O3 and [EB(Al2O3)]- Al2O3 layers and investigated the electronic states of Rh catalysts on them.