Optimisation is at the core of AI research. We spawn instances of massive models with trillions of parameters and then try to optimise their parameters towards some goal, typically represented by a loss function. We’ve become really good at this type of optimisation because it has a key property: we can calculate the gradient. Packages such as PyTorch automatically calculate the expected changes in our loss function if we were to tweak parameters (the gradients) which allows us to make meaningful progress towards the goal. But what if you don’t have gradients?

• we’ve all heard of recurrent neural networks (RNN)
• but there are many ways to ‘recur’ in a model
• autoregressive models, e.g. transformers, as an example of recurrence
• taxonomy of recurrence: sequence length, depth, spatial dimensions (e.g. GNN), others
• All of these can be seen as: hk+1=fθ(hk,x) where the k is over something different (e.g. time, layer, spatial)
• some of the key models
• why recurrence is important
• key issues with recurrence
• next steps for recurrence

ARC-AGI-3 is the latest challenge in Francois Chollet’s ARC Prize. While currently still under development, the authors released a sample of the challenges and are running a small competition. In this post I’d like to discuss my attempt at ‘hand-writing’ some solutions and what it told me about a real solution. If you’d like to know more about the ARC-AGI-3 challenge, I previously wrote about it here.

When we write code we typically leverage something like Git or Mercurial to track changes we make to files. These systems make it easy to see what has changed at a glance without picking through every line of code. Personally, I use VSCode’s Git integration. But who writes code themselves these days? When producing code with LLMs we still want to be able to track diffs easily.

I’ve been a long-time user of Notion for knowledge management. For reference, I have around 1000 pages covering various topics from physics, to sports, to dreams and aspirations. I manage my to-do lists there, travel plans and goal tracking. It’s a terrific visual tool and readily accessible everywhere. But I want more.

Recently I ran an experiment in which I tried to get LLMs to output explicit world models in the form of Python code which could be used as the core of a MCTS policy for control in basic gymnasium environments. This is different to typical world model learning which would be something like a parameterised neural network. The reason why we would care about something like this is for on-the-fly reasoning, such as in ARC-AGI-3 where agents must act in a completely new environment.