Anthropic Research on Learning Coding with AI →
This research came out right as I’ve been thinking about how to improve my engineering skills. Pre-LLM I’ve relied primarly on personal projects to randomly give me usefull skills as I work through making my projects a reality. It's been quite successfull.
In 2025 however, I noticed my learning mindset slowly disappearing as AI tools can skip thinking straight to a final result. It's been easy to feel like I'm falling behind or getting rusty. Considering that all of my side projects last year extensively used agentic development, there may be some painful truth to that.
Anthropic tested junior coders with a example python project on a unknown to them library. Here’s what really caught my eye in the research:
On average, participants in the AI group finished about two minutes faster, although the difference was not statistically significant. There was, however, a significant difference in test scores: the AI group averaged 50% on the quiz, compared to 67% in the hand-coding group
Low-scoring interaction patterns:
- AI delegation (n=4): Participants in this group wholly relied on AI to write code and complete the task. They completed the task the fastest and encountered few or no errors in the process.
High-scoring interaction patterns:
- Generation-then-comprehension(n=2): Participants in this group first generated code and then manually copied or pasted the code into their work. After their code was generated, they asked the AI assistant follow-up questions to improve understanding. These participants were not particularly fast when using AI, but showed a higher level of understanding on the quiz. Interestingly, this approach looked nearly the same as that of the AI delegation group, except for the fact that they used AI to check their own understanding.
They talk about several different types of patterns then just these two, but these stuck out to me.
Takeaways for my own learning:
- My biggest weakness that I've identified is “system design”. Therefore, I should not be letting AI decide my project architecture for me like I've been doing lately.
- When code is generated that I don’t understand, make an effort to understand it right there or ask follow up questions. Not understanding = Not learning.
- Outside of my core compenticies, it's likely just as fast to take more mental ownership over my work then floundering with prompt tweaking until X feature works.
Putting these into practice is easier said then done!