Gollum here is a simplified neural network: he searches memory for similar words and gives the most likely answer. Real models are more complex, but they make similar mistakes.