Current methods in credibility theory often rely on parametric models, e.g., a linear function of past experience. During the last decade, the existence of high speed computers and statistical software packages allowed the introduction of more sophisticated and flexible modelling strategies. In recent years, some of these techniques, which made use of the Markov Chain Monte Carlo (MCMC) approach to modelling, have been incorporated in credibility theory. However, very few of these methods made use of additional covariate information related to risk, or collection of risks; and at the same time account for the correlated structure in the data. In this paper, we consider a Bayesian nonparametric approach to the problem of risk modelling. The model incorporates past and present observations related to risk, as well as relevant covariate information. This Bayesian modelling is carried out by sampling from a multivariate Gaussian prior, where the covariance structure is based on a thin-plate spline. The model uses the MCMC technique to compute the predictive distribution of future claims based on available data.