We have studied the effects of Al0.1Ga0.9N(150 nm)/AlN Composite Nucleation Layers (CNLs) having different thicknesses of AlN ranging from 20 to 41 nm on the growth characteristics of GaN/Si(111) epitaxy. The surface morphologies of the GaN epitaxial layers which were grown on Al0.1Ga0.9N(150nm)/AlN CNLs showed that the number of thermal etch pits and cracks was abruptly decreased with the increase of AlN thickness from 20 to 35 nm. However, the morphology of GaN epitaxy which was grown on Al0.1Ga0.9N(150 nm)/AlN CNL having AlN of 41 nm thick above 35 nm showed that the number of them was increased again. So, the GaN/Si(111) epitaxy which was grown using Al0.1Ga0.9N(150 nm)/AlN(35 nm) CNL showed the highest crystallinity having the FWHM of 1157 arcsec for the (0002) diffraction. Photoluminescence spectrum at room temperature for GaN/Si(111) epitaxy grown using Al0.1Ga0.9N(150 nm)/AlN(35 nm) CNL showed a sharp band edge emission at 364 nm, which especially doesn't have yellow luminescence related to various defects such as vacancy and dislocation. Meanwhile, the spectra at room temperature for the others showed yellow luminescence at around 580 nm except each band edge emission. Moreover, the FWHM of main exitonic peak at 10 K for the GaN/Si(111) epitaxy which was grown using Al0.1Ga0.9N(150 nm)/AlN(35 nm) CNL is the lowest value of 12.81 meV among them. It is obvious that the Al0.1Ga0.9N(150 nm)/AlN CNL having suitable thickness of AlN plays an important role in improving the crystallinity and optical properties of GaN/Si(111) heteroepitaxy without any defects such as pits and cracks over the surface by reducing the mismatch of thermal expansion coefficient and lattice constant between GaN and Si(111) comparing with AlxGa1-xN or AlN nucleation layer alone.