Test-time compute has recently been popularised with LLMs (e.g. o1/o3, DeepSeek-R1 and other reasoning models) as its application has allowed LLMs to perform significantly better on complex reasoning problems (e.g. mathematics, reasoning). This article explores why it works, why it’s not new, and how it’s been employed across different AI paradigms.

As a researcher, it’s common to have way more ideas than you have time to experiment with. This is especially true in the world of language modeling when you consider the cost of running such experiments. In this post I touch on some of the methods that I’ve seen to run experiments with language models without breaking the bank.

DeepSeek’s R1 had it’s time in the spotlight as a strong reasoning model that came ‘out of nowhere’. One of the highlights of the model was that it was released publicly, including both the training process and weights. However, one thing lacking from the paper was an overview of the pipeline. Unsurprisingly, there are a few steps involved to produce such great results.

Wouldn’t it be great if you could communicate directly with your dog? If you could ask him why he bit your furniture, or just understand what he’s barking about? While research has tried address this in the past, the problem is still far from solved, and potentially unsolvable. Let’s see why.

When training multilingual models a common problem is language mixing or code switching, in which models may respond in multiple languages when we would expect them to use just one. This can also happen in reasoning models, such as DeepSeek-R1. In their paper, they found that

Sometimes when reading books it can be hard to get a grasp for a problem in practice. This video popped up on IG showcasing one of the important principles of multi-agent systems.