Effects of impurity (P and B) doping on the photoluminescence (PL) properties of Si nanocrystals (nc-Si) in SiO2 thin films are studied. It is shown that with increasing P concentration, PL intensity first increases and then decreases. In the P concentration range where PL intensity increases, quenching of the defect-related PL is observed, suggesting that dangling-bond defects are passivated by P doping. On the other hand, in the range where PL intensity decreases, optical absorptiondue to the intravalley transitions of free electrons generated by P doping appears. The generation of free electrons andthe resultant three-body Auger recombination of electron-hole pairs is considered to be responsible for theobserved PL quenching. In the case of B doping, the behavior is much different. With increasing B concentration, PL intensity decreases monotonously. By combining the results obtained for P and B doped samples, theeffects of donor and acceptor impurities on the PL properties of nc-Si are discussed.