This work investigates how knock-on displacements influence fluctuation electron microscopy (FEM) experiments. FEM experiments were conducted on amorphous silicon, formed by self-ion implantation, in a transmission electron microscope at 300 kV and 60 kV at various electron doses, two different binnings and with two different cameras, a CCD and a CMOS one. Furthermore, energy filtering has been utilized in one case. Energy filtering greatly enhances the FEM data by removing the inelastic background intensity, leading to an improved speckle contrast. The CMOS camera yields a slightly larger normalized variance than the CCD at an identical electron dose and appears more prone to noise at low electron counts. Beam-induced atomic displacements affect the 300 kV FEM data, leading to a continuous suppression of the normalized variance with increasing electron dose. Such displacements are considerably reduced for 60 kV experiments since the primary electron's maximum energy transfer to an atom is less than the displacement threshold energy of amorphous silicon. The results show that the variance suppression due to knock-on displacements can be controlled in two ways: Either by minimizing the electron dose to the sample or by conducting the experiment at a lower acceleration voltage.