We consider the performance of the delta hedging strategy obtained from a local volatility model when using as input the physical prices instead of the model price process. This hedging strategy is called robust if it yields a superhedge as soon as the local volatility model overestimates the market volatility. We show that robustness holds for a standard Black-Scholes model whenever we hedge a path-dependent derivative with a convex payoff function. In a genuine local volatility model the situation is shown to be less stable: robustness can break down for many relevant convex payoffs including average-strike Asian options, lookback puts, floating-strike forward starts, and their aggregated cliquets. Furthermore, we prove that a sufficient condition for the robustness in every local volatility model is the directional convexity of the payoff function.