It seems I'm thinking of a prediction algorithm and being discussed is a genetic algorithm, for which I can understand the necessity of a training set and a test set.

Either way, my knowledge on machine learning is lacking, time to hit up Google.