The rule-following paradox
I shall begin by misdescribing the moral of Wittgenstein's rule-following paradox, because I take the misdescription to be a helpful one, more of a ladder than a stumbling-block (although it should be borne in mind that it is always possible to trip on a ladder).
The moral of the rule-following paradox, then, is that what rule one is following when one acts is (radically) underdetermined by anything in either one's actions or one's thoughts. Underdetermined by anything in one's actions, since every actual sequence of behaviour is finite, and so is capable of being extended in an infinite variety of ways, each one corresponding to a different rule, all such rules being equally consistent with the behaviour thus far exhibited. (Worse yet, even an infinite sequence of behaviour, were one possible, would not settle the matter, since such a sequence would be consistent with both a flawless execution of rule A and a bungled attempt to apply rule B.) And underdetermined by anything in one's thoughts, because no matter what the agent may have before her mind's eye, as it were, it does not count as such-and-such a rule unless the agent reliably applies it in such-and-such a way:
I cannot know what he's planning in his heart. But suppose he always wrote out his plans; of what importance would they be? If, for example, he never acted on them. … Perhaps someone will say: Well, then they really aren't plans. But then neither would they be plans if they were inside him, and looking into him would do us no good. […]