The paper concentrates on in situ transmission electron microscopy of nano-sized Mo and Nb clusters. In particular, this contribution presents challenges to control the microstructure in nano-structured materials via a relatively new approach, i.e. using a so-called nanocluster source. An important aspect is that the cluster size distribution is monodisperse and that the kinetic energy of the clusters during deposition can be varied. The deposited Mo clusters with sizes 5 nm or larger show a body-centered crystal (bcc) structure. The cubic clusters are self-assembled from smaller ones and forming distorted cubes of typical size 7.8 nm or larger. With reducing cluster size to ≤3 nm, the face centered crystal (fcc) structure appears due to dominance of surface energy minimization, while self-assembly into large cubes with sizes up to 20 nm is still observed. In situ TEM annealing leads to cluster coalescence at temperatures ∼800 °C, with the crystal habit changing to rhombic dodecahedron for isolated clusters, while large cubes change to faceted polyhedra. In situ TEM annealing studies on Nb clusters showed that cluster coalescence events were not observed even at rather elevated temperatures because of the formation of oxides.