Vibe Coding & AI-Generated Debt

"Vibe coding" is the practice of coding with AI assistants—accepting suggestions from Copilot, ChatGPT, or Claude without fully understanding every line. It's fast, it's productive, and it's creating a new category of technical debt.

The Rise of AI-Assisted Development

AI coding assistants are now mainstream:

• GitHub Copilot: 1.3M+ paid subscribers
• ChatGPT/Claude: Used daily by millions of developers
• Cursor, Windsurf, Replit: AI-native IDEs growing fast

The productivity gains are real. But so are the problems.

---

The Hidden Costs of Vibe Coding

1. Dead Code Accumulation

AI suggests complete solutions. You use 60% of it:

# You asked: "function to parse JSON config"
# AI gave you:

import json
import yaml  # only need JSON
import toml  # only needed JSON
from pathlib import Path
from typing import Dict, Any, Optional, List  # List unused

def parse_config(path: str) -> Dict[str, Any]:
    file_path = Path(path)
    
    if file_path.suffix == '.json':
        with open(file_path) as f:
            return json.load(f)
    elif file_path.suffix in ['.yml', '.yaml']:  # never use YAML
        with open(file_path) as f:
            return yaml.safe_load(f)
    elif file_path.suffix == '.toml':  # never use TOML
        with open(file_path) as f:
            return toml.load(f)
    else:
        raise ValueError(f"Unsupported format: {file_path.suffix}")

def validate_config(config: Dict[str, Any]) -> bool:  # never called this
    required_keys = ['database', 'api_key', 'debug']
    return all(key in config for key in required_keys)

You wanted 10 lines. You got 25. Half is dead.

2. Security Vulnerabilities

AI doesn't understand your security context:

# You asked: "function to run shell command"
# AI gave you:

import subprocess

def run_command(cmd: str) -> str:
    result = subprocess.run(
        cmd, 
        shell=True,  # CRITICAL: Command injection vulnerability!
        capture_output=True,
        text=True
    )
    return result.stdout

# AI didn't ask: "Is cmd from user input?"
# AI didn't know: This is called from a web endpoint

Result: Exploitable vulnerability shipped to production.

3. Complexity Creep

AI writes verbose, "safe" code that handles cases you don't have:

# You asked: "sum a list of numbers"
# You expected: sum(numbers)
# AI gave you:

from typing import List, Union, Optional
from decimal import Decimal
import logging

logger = logging.getLogger(__name__)

def sum_numbers(
    numbers: List[Union[int, float, Decimal]], 
    initial: Optional[Union[int, float, Decimal]] = None,
    ignore_none: bool = True,
    round_result: Optional[int] = None
) -> Union[int, float, Decimal]:
   
    if not numbers:
        logger.warning("Empty list provided to sum_numbers")
        return initial or 0
        
    total = initial or 0
    for num in numbers:
        if num is None and ignore_none:
            continue
        if num is None:
            raise ValueError("None value in list")
        total += num
    
    if round_result is not None:
        total = round(total, round_result)
        
    return total

30 lines instead of 1. Complexity: 6 instead of 1.

4. Orphaned Functions

AI creates helper functions you never call:

5. Copy-Paste Propagation

You accept AI suggestions across multiple files. Same pattern, same bugs:

# file1.py - AI wrote this
data = request.json()  # No validation
db.execute(f"INSERT INTO t VALUES ({data['id']})")

# file2.py - You asked AI for similar code
data = request.json()  # Same vulnerability copied
db.execute(f"INSERT INTO other VALUES ({data['id']})")

# file3.py - Pattern continues...

---

The Vibe Coding Cycle

The problem isn't AI—it's accepting without reviewing.

---

Why Traditional Tools Miss This

Linters Don't Catch Logic Issues

# Passes all linters!
def process(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"  # SQL injection
    return db.execute(query)

Tests Only Cover What You Test

AI-generated code often has:

• Untested error branches
• Unused parameters (no test exercises them)
• Dead functions (no test calls them)

Code Review Doesn't Scale

Average PR size with AI assistance: 2.5x larger
Code review time: Fixed
Result: Less scrutiny per line

---

How Skylos Fixes Vibe Coding Debt

1. Dead Code Detection

Find the AI bloat:

skylos . --confidence 70

Unused imports: 23
Unused functions: 8
Unused parameters: 12

2. Security Scanning

Catch what AI didn't consider:

skylos . --danger

SKY-D210 CRITICAL: SQL injection in api/users.py:45
SKY-D212 CRITICAL: Command injection in utils/shell.py:12

3. Complexity Alerts

Flag AI's over-engineering:

skylos . --quality

SKY-Q301: Function 'sum_numbers' has complexity 6 (threshold: 10)
SKY-Q304: Function 'parse_config' has 5 parameters (threshold: 5)

4. AI Defect Checks

Catch evidence-backed AI-code failure modes:

skylos . --ai-defects

SKY-L012: Phantom security helper call
SKY-D224: Real package called with invented API
SKY-A101: Test assertion weakened in this diff

5. Quality Gate

Prevent AI debt from shipping:

skylos . --danger --quality --ai-defects --gate

Block PRs that introduce vulnerabilities or dead code.

---

Best Practices

1. Run Skylos before committing

# Pre-commit hook
skylos . --danger --quality --ai-defects || exit 1

Catches AI-introduced issues immediately.

2. Delete AI's unused suggestions

AI gives you 5 functions. You need 2. Delete the other 3.

skylos . -i  # Interactive mode

3. Question AI's security assumptions

AI doesn't know:

• Where your data comes from
• Who can call this function
• What's sensitive in your context

Ask: "Is this input trusted?"

4. Simplify AI's over-engineering

# AI gave you 30 lines for summing
# You need:
total = sum(numbers)

Use Skylos complexity warnings as a signal.

5. Set up CI gates

Don't let AI debt accumulate:

- run: skylos . --danger --quality --ai-defects --gate

---

In-Loop Verification

skylos verify is the agent-loop path for vibe coding defects. Instead of running a full project report after a feature is done, agents and editors can ask Skylos for a narrow verdict on the changed file or range:

skylos verify . --file src/app.py --range 40:75 --project-context

The verifier returns versioned JSON with only AI-code trust findings. Each finding includes rule_id, vibe_category, ai_likelihood, range, message, optional suggested_fix, confidence, severity, and category. AI-defect findings use category: "ai_defect" and full scans group them under the ai_defects output bucket.

The same schema is available through MCP as verify_change, which lets MCP-compatible coding agents verify a change after each edit. The VS Code extension also routes idle changed-function analysis through this verifier path.

For unsaved editor buffers, pass a JSON manifest through stdin:

printf '{"file":"src/app.py","code":"def handler(): pass\n","range":"1:1"}' \
  | skylos verify . --stdin --no-fail

Local Defect Signals

Skylos can record local structural signals when a developer accepts, dismisses, or trains on an AI-defect finding through the agent service. Signal capture is off by default and controlled by [tool.skylos.contribution].

[tool.skylos.contribution]
collect_local_signals = true
contribute_public_corpus = false
structural_signatures_only = true
include_source = false

Local events are written to .skylos/contribution/events.json. They contain rule/category/severity metadata, file extension, line bucket, and hashes. Raw source is not captured.

AI-Specific Detection Rules

Skylos includes rules specifically designed to catch evidence-backed failure modes common in AI-generated code. AI-defect checks (SKY-A101 to SKY-A105, SKY-L012, SKY-L023, SKY-D222, SKY-D224, and SKY-D225) report under the ai_defects output bucket; broader AI-prone quality and security rules keep their existing buckets. See AI Defect Verification for the exact grouping method and why some AI-defect rules keep historical SKY-L or SKY-D IDs.

Rule	What It Catches	Why AI Does This	Vibe Category
SKY-A101	Assertion weakening — exact assertions replaced with truthiness/null checks, skips, or removed exception assertions	LLMs sometimes make tests pass by weakening the test instead of preserving behavior	assertion_weakening
SKY-A102	High-risk change without tests — auth, billing, validation, tenant, webhook, or similar code changed with no test file changed	AI-generated PRs can touch behavior-sensitive code without adding reviewable test evidence	test_impact_gap
SKY-A103	CI permission expansion — GitHub Actions diff adds write permissions or privileged workflow triggers	AI-generated CI edits can accidentally broaden repository token privileges	ci_permission_expansion
SKY-A104	Public CLI surface drift — public CLI flag removed from argparse, Click, or Typer code	AI-generated refactors can break documented commands or automation without noticing compatibility impact	public_api_surface_drift
SKY-A105	Contract route guard missing — a route lacks a decorator required by .skylos/ai-contract.yml	LLMs add endpoints without applying repo-specific auth or tenant guardrails	missing_contract_guardrail
SKY-L012	Phantom function calls — sanitize_input() that doesn't exist	LLMs hallucinate security functions that were never defined or imported	hallucinated_reference
SKY-L023	Phantom decorators — @require_auth that doesn't exist	LLMs hallucinate security decorators, leaving functions unprotected	hallucinated_reference
SKY-L026	Unfinished generation — function body is only pass or ...	LLMs generate stubs and move on, leaving empty functions in production	incomplete_generation
SKY-L016	Undefined config — os.getenv("ENABLE_CACHING") never set	LLMs invent feature flags that reference config that doesn't exist	ghost_config
SKY-L024	Stale mocks — mock.patch("mod.old_func") targets deleted function	After refactoring, AI-generated tests still patch old function names	stale_reference
SKY-L013	Insecure randomness — random.randint() for tokens	LLMs reach for random instead of secrets for security values	—
SKY-L014	Hardcoded credentials — password="admin123"	LLMs generate working examples with real-looking credentials	—
SKY-L010	Security TODO markers — # TODO: add auth check	LLMs leave placeholder comments as "future work" that never happens	—
SKY-L011	Disabled security controls — verify=False	LLMs disable TLS/CSRF to make examples "just work"	—
SKY-L017	Error info disclosure — return str(e)	LLMs return raw exceptions for "debugging" instead of safe error messages	—
SKY-L020	Broad file permissions — os.chmod(f, 0o777)	LLMs use maximally permissive modes to avoid permission errors	—
SKY-D222	Hallucinated dependencies — importing packages that don't exist on PyPI	LLMs invent package names that sound right but aren't real	dependency_hallucination
SKY-D224	API signature hallucinations — real package, invented method or keyword	LLMs confidently call APIs that look plausible but are not in the installed package	api_signature_hallucination
SKY-D225	Dependency version hallucinations — npm or Go versions that were never published	LLMs invent semver pins that make manifests look complete but fail in builds	dependency_hallucination
SKY-D260	Prompt injection — hidden instructions in comments, YAML, Markdown	Hostile content targeting AI agents processing your codebase	—

# Scan for AI-generated code issues
skylos . --ai-defects

---

The Numbers

Metric	Without Skylos	With Skylos
Dead code from AI	~30% of AI-generated code	< 5%
Security vulns shipped	1 in 10 PRs	Blocked at CI
Average function complexity	12	7
Time to onboard new dev	3 weeks	2 weeks

---

Next Steps

Getting Started

Install Skylos and scan your codebase

CI/CD Integration

Set up quality gates to prevent AI debt