Ever since reading “How could I have thought that faster?” this year, I have been trying to put it into practice. Working with AI models, I found there are plenty of opportunities. One can spend hours on some buggy code just to find the bug they fixed wasn’t the real problem after all, or that someone had already solved it five years ago on Stack Overflow. One can invests hours into modelling to realise that there was a far simpler approach if you just thought about it from another angle…

Test-time compute has recently been popularised with LLMs (e.g. o1/o3, DeepSeek-R1 and other reasoning models) as its application has allowed LLMs to perform significantly better on complex reasoning problems (e.g. mathematics, reasoning). This article explores why it works, why it’s not new, and how it’s been employed across different AI paradigms.

As a researcher, it’s common to have way more ideas than you have time to experiment with. This is especially true in the world of language modeling when you consider the cost of running such experiments. In this post I touch on some of the methods that I’ve seen to run experiments with language models without breaking the bank.

DeepSeek’s R1 had it’s time in the spotlight as a strong reasoning model that came ‘out of nowhere’. One of the highlights of the model was that it was released publicly, including both the training process and weights. However, one thing lacking from the paper was an overview of the pipeline. Unsurprisingly, there are a few steps involved to produce such great results.

Wouldn’t it be great if you could communicate directly with your dog? If you could ask him why he bit your furniture, or just understand what he’s barking about? While research has tried address this in the past, the problem is still far from solved, and potentially unsolvable. Let’s see why.

When training multilingual models a common problem is language mixing or code switching, in which models may respond in multiple languages when we would expect them to use just one. This can also happen in reasoning models, such as DeepSeek-R1. In their paper, they found that