Since early 2025, there has been a lot of talk about vibe coding, as popularised by Andrej Karpathy, a co-founder of OpenAI and former AI leader at Tesla.

His engaging blog post told how he had created his MenuGen app by vibe coding. He started with an idea for an app and then described in natural language what he wanted his app to do and through use of AI LLMs such as Claude 3.7 and dev tools such as Cursor, the AI built the app for him. For some of the advanced features of the app, it was plugged into various APIs – such as OpenAI API, to enable OCR of menu items, and the Replicate API, to generate images. Through a series of iterations he was then able to review and refine what had been produced and run it locally on his dev machine.

That sounded like the ‘easy’ part, as he also detailed how he managed to actually deploy the app, so that it wasn’t just available locally, and this had its own learning curve, as he had to patch his app into things like a deployment platform (Vercel), authentication platform (Clerk) and payments platform (Stripe). He also said he’d leave areas such as database and work queues for a future iteration.

From reading his posts, other related posts and then further research via Google Gemini, I have managed to boil it down to the following:

Vibe Coding – How It Works

  • vibe coder outlines/describes the desired functionality and features in natural language 
  • AI generates the code
  • vibe coder runs the app, reviews and refines the code and provides feedback to the AI
  • AI modifies the code based on feedback
  • this iterative process continues until the vibe coder is satisfied with the product

Vibe Coding – Upsides

  • faster development process than a human coder could produce
  • enables those with limited coding knowledge to build applications
  • allows focus on design and strategy of product rather than the detail of the coding

Vibe Coding – Downsides

  • code quality; code may often contain errors or vulnerabilities, such as security issues 
  • vibe coder will likely need to understand the code that has been produced in order to ensure its effectiveness, efficiency and robustness 
  • probably best used for simpler/less-complex projects such as creating a prototype
  • usually need the paid versions of the tools used to build the software, for example, ChatGPT, Gemini, Claude etc 

While researching vibe coding, I read somewhere that the best use of vibe coding maybe as a quick introduction for a developer, when they need to learn or use new languages and technologies. It also seems ideally suited for creating a prototype, when the seed of an app idea forms and may want to try actually building it.

Does vibe coding require testing?

The good news for anyone in the software testing profession is that, yes – vibe coding does require testing – and it’s known as vibe testing. Further research into vibe testing shows that it’s a well-established concept as part of the vibe coding process.

As vibe coding relies heavily on AI-generated code, there are risks such as:

  • lack of understanding of the code, which may lead to potential errors or vulnerabilities
  • traditional QA methods may not suffice, as the code is not manually crafted and may not follow expected patterns

Therefore, vibe testing is used to complement vibe coding by ensuring that the application functions as intended.

It focuses on:

  • Exploratory Testing: Users interact with the application like real users, to identify issues.
  • Prompt-Driven Testing: Test cases are generated based on natural language prompts, similar to vibe coding.
  • Intent-First Approach: It checks if the application meets the intended functionality, rather than just verifying existing code.

Vibe testing is crucial for applications created through vibe coding. It helps validate the functionality and reliability of AI-generated code, ensuring that the software meets user expectations and maintains quality standards.

My thoughts on vibe coding and vibe testing

  • I would say that a very small vibe coding project, perhaps to produce a simple prototype, might be performed by one person doing both the vibe coding and the vibe testing.
  • However, in cases where the vibe coding project was a little more complex – such as being plugged into various APIs, utilising a UI, requiring user input and user interaction, then a software tester could become involved in the project.
  • This would give the project more emphasis on quality, in both the user experience and the functionality the project performed.
  • As ever, the earlier the software tester was involved in the project, the more effective their involvement would be, especially if they could help craft the natural language supplied to the AI LLM when generating the code.

Research

During the research detailed below, several key phrases and concepts were used, which may (or may not) be part of how vibe testing will evolve, and I’ve included some links where relevant.

These include:

Research used (that you can read and listen to)

In a kind of meta way, I used AI tools to research vibe coding and vibe testing.

I used Google Gemini AI to perform deep research on vibe coding and the very useful output is available here via this shared link:

https://docs.google.com/document/d/1l-Fyrh8ebU2gdoIA2pSFGmgjRLKsVWORbw31NWlQFzY/edit?usp=sharing

I also used it to perform deep research on the role of testing in vibe coding and, again, the very useful output is available here via this shared link:

https://docs.google.com/document/d/1L3HlDC-MfJihDw-AkoAYilG-ZrnpdInhNeASVoErxWY/edit?usp=sharing

And going deeper into the AI world, I managed to get Google Gemini AI to produce an Audio Overview of the role of testing in vibe coding, that you can listen to via this shared link:

https://drive.google.com/file/d/1e2nSlT5hHF_BhhPqBJ8cy3VHjKjHnrx1/view?usp=sharing

I also asked Perplexity AI – What do I need to get started in vibe coding?

  • 1. Pick an AI coding platform
    • Cursor – a desktop app similar to VS Code but with built-in AI
    • Replit – browser-based
    • Or tools like ChatGPT, Claude or GitHub Copilot Agent
    • Some tools offer free trials, offers may require sign up or subscription for advanced features
  • 2. Have a clear project idea, which you can describe in natural language
  • 3. Create first project.
    • Use the AI tool to start a new project, describing in natural language what you want e.g. Build a to-do list app with categories.
    • The tool will generate code which you can further guide by asking for fixes of new features
  • 4. Iterate and learn.
    • Review what the AI produces, test it and refine your prompts as needed. Ask the AI to explain the code and add comments so you gradually understand the logic.

And then I asked Perplexity AI – What are the common challenges in vibe coding workflows?

  • Code quality and maintainability Messy architecture. Scalability issues.
  • Debugging and Testing difficulties – Opaque logic. Developers may struggle to understand code they didn’t write themselves. This obscurity slows down debugging and troubleshooting, especially with unexpected bugs or edge cases
  • Security concerns. Vulnerabilities: AI-generated code can introduce insecure patterns, depend on outdated libraries, or omit crucial checks like input validation, making applications susceptible to attacks such as SQL injection or XSS.
  • Context & Complexity Limitations
  • Documentation & Knowledge Transfer
  • Workflow & Collaboration friction
  • Subtle Bugs & Functional Gaps

Useful Links

Wikipedia on ‘vide coding’ – https://en.wikipedia.org/wiki/Vibe_coding

Coding without code – https://innovatingwithai.com/coding-without-code

Google Cloud – AI Code Generation – https://cloud.google.com/use-cases/ai-code-generation

Some ‘vibe coding’ tools
Cursor – The AI Code Editor https://cursor.com/

Replit – Turn your ideas into apps https://replit.com/

NoCode-X – Launch your idea in seconds without risking security breaches https://www.nocode-x.com/

Vibe Testing – Scaling quality with human expertise and AI Intelligence https://testkube.io/blog/what-is-vibe-testing

Note: The featured image used in this post was created via Bing Image Creator 

About my testing services: iOS App Testing / Android App Testing / Website Testing / AI Chatbot Testing

Your next step – Contact Me about Vibe Testing services or any other services I provide

Some YouTube Vibe Coding videos