We report on follow-up observations of 20 short-duration gamma-ray bursts (T90 < 2s) performed in g′r′i′z′JHKs with the Gamma-Ray Burst Optical Near-Infrared Detector (GROND) between mid-2007 and the end of 2010. This is the most homogeneous and comprehensive data set on GRB afterglow observations of short bursts. In three cases, GROND was on target within less than 10 min after the trigger, leading to the discovery of the afterglow of GRB 081226A and its faint underlying host galaxy. In addition, GROND was able to image the optical afterglow and follow the light curve evolution in five further cases: GRBs 090305, 090426, 090510, 090927, and 100117A. Three of the aforementioned six bursts with optical light curves show a break: GRBs 090426 and 090510 as well as GRB 090305. For GRB 090927, no break is seen in the optical/X-ray light curve until about 150 ks/600 ks after the burst. A decay slope of the optical afterglow of GRB 100117A could be measured. Using these data supplemented by about ten events taken from the literature, we compare the jet half-opening angles of long and short bursts. We find a tentative evidence that short bursts have wider opening angles than long bursts. However, the statistics are still very poor and follow-up observations of these events are therefore very important to gain as much observational data as possible.