In acoustic devices such as film bulk acoustic resonators (FBAR), it is most essential to accurately determine the thin-film sound velocities in situ. In this work, we analyzed the microstructure properties of the zirconia thin films deposited by RF magnetron reactive sputtering with various oxygen partial pressures, and measured the longitudinal film velocity with picosecond ultrasonic technique. The picosecond ultrasonic waves were produced by irradiating the testing samples with an ultrafast laser pulse generated by a self-made mode-locked Ti: Sapphire laser, and detected by a delayed probe laser pulse. The acoustic velocities of the thin films were next determined from the echo times of the ultrasonic waves. To derive more accurate and reliable velocity, three different reflective layers were employed so that the echo shapes and intensities of ultrasonic wave can be compared. It was found in this work that the thin film velocities we measured were less than the bulk value, which can be calculated from Young's modulus and the density. Meanwhile, with the measurement results, it is also found that the measured acoustic velocity and the microstructure of films have strong dependence on the growth conditions. Consequently, accurate thin film velocity will be obtained for an SMR designer through better controlling on deposition conditions during manufacturing process.