LLM Debugger is a VSCode extension that demonstrates the use of large language models (LLMs) for active debugging of programs. This project is a proof of concept developed as a research experiment and will not be actively maintained or further developed.
final.mov
Traditional LLM-based debugging approaches analyze only static source code. With LLM Debugger, the LLM is provided with real-time runtime context including:
- Runtime Variable Values: Observe actual variable states as the program executes.
- Function Behavior: Track how functions are called, what values they return, and how they interact.
- Branch Decisions: Understand which code paths are taken during execution.
This enriched context allows the LLM to diagnose bugs faster and more accurately. The extension also has the capability to generate synthetic data by running code and capturing execution details beyond the static source, offering unique insights into program behavior.
subgraph “VSCode Editor”
User[User] –> Editor[VSCode Editor]
Editor –> DebugSession((Debug Session))
end
subgraph “LLM Debugger Extension”
DebugSession –> DebugAdapter[Debug Adapter Tracker]:::extensionComponent
DebugAdapter –> DebugSession
DebugAdapter — Debug State –> LLMClient[LLM Client]:::extensionComponent
LLMClient — Function Calls –> DebugAdapter
end
subgraph “External Services”
LLMClient — LLM[Large Language Model]
end
DebugSession — Debug Protocol –> NodeDebugAdapter[Node.js Debug Adapter]
NodeDebugAdapter — Executes –> NodeApp[Node.js Application]
NodeApp — Runtime Events –> NodeDebugAdapter
” dir=”auto”>
graph TB
subgraph "VSCode Editor"
User[User] --> Editor[VSCode Editor]
Editor --> DebugSession((Debug Session))
end
subgraph "LLM Debugger Extension"
DebugSession --> DebugAdapter[Debug Adapter Tracker]:::extensionComponent
DebugAdapter --> DebugSession
DebugAdapter -- Debug State --> LLMClient[LLM Client]:::extensionComponent
LLMClient -- Function Calls --> DebugAdapter
end
subgraph "External Services"
LLMClient --- LLM[Large Language Model]
end
DebugSession -- Debug Protocol --> NodeDebugAdapter[Node.js Debug Adapter]
NodeDebugAdapter -- Executes --> NodeApp[Node.js Application]
NodeApp -- Runtime Events --> NodeDebugAdapter
Loading
- Active Debugging: Integrates live debugging information (variables, stack traces, breakpoints) into the LLM’s context.
- Automated Breakpoint Management: Automatically sets initial breakpoints based on code analysis and LLM recommendations.
- Runtime Inspection: Monitors events like exceptions and thread stops, gathering detailed runtime state to guide debugging.
- Debug Operations: Supports common debugging actions such as stepping over (
next), stepping into (stepIn), stepping out (stepOut), and continuing execution. - Synthetic Data Generation: Captures interesting execution details to generate data that extends beyond static code analysis.
- Integrated UI: Features a sidebar panel within the Run and Debug view that lets you toggle AI debugging and view live LLM suggestions and results.
-
Start LLM Debug Session:
- Command:
llm-debugger.startLLMDebug - Description: Launches an AI-assisted debugging session
- Command:
10 Comments
jasonjmcghee
Nice! I recently had and built the same idea using MCP (in order to be client / LLM agnostic) and VS Code (DAP would be even better, but haven't tried tackling it).
https://github.com/jasonjmcghee/claude-debugs-for-you
codenote
Interesting experiment! This feels like it could really expand the potential applications of LLMs. Exciting to see how AI can assist in debugging with live runtime context!
stuaxo
Nice, I did this manually once with ipdb, just cutting and pasting the text to LLM and having it tell me which variables to inspect and what to press.
mohsen1
To people who do this sort of thing:
Does generating synthetic data using this make sense for RL of models to be be really good at debugging code? Currently all LLMs inhaled all of code in the world but the data is only text of the code (maybe plus changes that fixed bugs etc) but the amount of insight that can be generated by actually running the code and getting the runtime values, step-by-step is almost infinite.
Is this sort of data useful for training LLMs?
bravura
A time-traveling debugger for Python + LLM would be amazing.
jdright
also see: https://github.com/plasma-umass/ChatDBG
melvinroest
This would be amazing with Smalltalk/Pharo or a similar language where the concept of debugging is a first class citizen (I guess it's the same for certain Lisp languages?)
crest
I like the first paragraph of the README clearly stating that this is your research project instead of making a lot of grandiose claims.
ericb
Very cool concept! There's a lot of potential in reducing the try-debug–fix cycle for LLMs.
On a related note, here's a Ruby gem I wrote that captures variable state from the moment an Exception is raised. It gets you non-interactive text-based debugging for exceptions.
https://rubygems.org/gems/enhanced_errors
K0IN
Hei this is lovely,
i created a extension to help me debug a while back [0], and i thought of this (ai integration) for a long time, but did not have the time to tackle this.
Thank you so much for sharing!
I might need to add this approach to my extension as well!
[0] https://github.com/K0IN/debug-graph