Abstract The question whether or not the construct “rule” is essential for cognition is one of the main divisions between connectionist and rival approaches in cognitive science. In this chapter, we consider the philosophical significance of this division, and its implications for cognitive research, in the light of several possible interpretations of Wittgenstein’s paradox of following a rule. The conclusion is that the rejection of rules by connectionism makes it philosophically incompatible with the symbolic rule-governed approach; nevertheless, the rejection of rules does not necessarily lead, on its own, to a single way of conceptualizing the mind and its place in nature. Wittgenstein’s notions of “form of life” and “language games” are used as an aid for understanding the philosophical foundations of connectionism.