In various agent-based models, the stylized facts of financial markets (unit roots, fat tails, and volatility clustering) have been shown to emerge from the interactions of agents. However, the complexity of these models often limits their analytical accessibility. In this paper we show that even a very simple model of a financial market with heterogeneous interacting agents is capable of reproducing these ubiquitous statistical properties. The simplicity of our approach permits us to derive some analytical insights using concepts from statistical mechanics. In our model, traders are divided into two groups, fundamentalists and chartists, and their interactions are based on a variant of the herding mechanism introduced by A. Kirman (Ants, rationality, and recruitment, Quarterly Journal of Economics 108, 137–156, 1993). The statistical analysis of simulated data points toward long-term dependence in the autocorrelations of squared and absolute returns and hyperbolic decay in the tail of the distribution of raw returns, both with estimated decay parameters in the same range as those of empirical data. Theoretical analysis, however, excludes the possibility of “true” scaling behavior because of the Markovian nature of the underlying process and the boundedness of returns. The model, therefore, only mimics power law behavior. Similarly to the phenomenological volatility models analyzed by LeBaron (Stochastic volatility as a simple generator of apparent financial power laws and long memory, Quantitative Finance 1, 621–631, 2001), the usual statistical tests are not able to distinguish between true and pseudo-scaling laws in the dynamics of our artificial market.