In this podcast episode James Wilson chats with Karsten Nohl about his research into using local LLMs to replace cloud AI in security code reviews.

In essence, Karsten created a hybrid code reviewing system where both cloud and local models are used to orchestrate, triage outputs, and write reports. In this system, only the local LLMs have source code access, with the cloud models used to manage the local models.

In this “source-local” review technique, the source code never leaves the local endpoint, which is a requirement for some reviews. But funnily enough, Karsten was able to use this system to generate findings that were as impressive as when using frontier models directly.

In a nutshell, Karsten proved it’s possible to use locally-hosted, open-weight models running on commodity hardware to produce findings comparable to those discovered by frontier cloud models.