---
name: fake-driven-testing
description: This skill should be used when writing tests, fixing bugs, adding features,
---

# fake-driven-testing

This skill should be used when writing tests, fixing bugs, adding features,

# fake-driven-testingby dagster-io
This skill should be used when writing tests, fixing bugs, adding features,

`npx skills add https://github.com/dagster-io/fake-driven-testing --skill fake-driven-testing`Download ZIPGitHub

## Fake-Driven Testing Architecture for Python

Use this skill when: Writing tests, fixing bugs, adding features, or modifying gateway layers in Python projects.

No prerequisites. This skill is self-contained. It focuses on testing architecture, not language-specific style.

## Overview

This skill provides a defense-in-depth testing strategy with five layers for Python applications:

```
`┌─────────────────────────────────────────────────┐
│ Layer 5 "smoke": Business Logic Integration Tests (5%) │ ← Smoke tests over real system
├─────────────────────────────────────────────────┤
│ Layer 4 "logic": Business Logic Tests (70%) │ ← Tests over fakes (MOST TESTS)
├─────────────────────────────────────────────────┤
│ Layer 3 "pure": Pure Unit Tests (10%) │ ← Zero dependencies, isolated testing
├─────────────────────────────────────────────────┤
│ Layer 2 "real-sanity": Integration Sanity Tests (10%) │ ← Fast validation with mocking
├─────────────────────────────────────────────────┤
│ Layer 1 "fake-check": Fake Infrastructure Tests (5%) │ ← Verify test doubles work
└─────────────────────────────────────────────────┘
`
```

Philosophy: Test business logic extensively over fast in-memory fakes. Use real implementations sparingly for integration validation.

Terminology note: The "gateway layer" (also called adapters/providers) refers to thin wrappers around heavyweight external APIs (databases, filesystems, HTTP APIs, message queues, etc.). The pattern matters more than the name.

## Quick Decision: What Should I Read?

Adding a feature or fixing a bug?
→ Read `quick-reference.md` first, then `workflows.md#adding-a-new-feature`

Need to understand where to put a test?
→ Read `testing-strategy.md`

Working with Python-specific patterns?
→ Read `python-specific.md`

Adding/changing a gateway interface?
→ Read `gateway-architecture.md`, then `workflows.md#adding-a-gateway-method`

Wondering where the DI boundary is?
→ Read `gateway-architecture.md#the-di-boundary-only-fake-gateways` — only gateways get fakes

Need to understand non-ideal states vs exceptions?
→ Read `non-ideal-states.md`

Found tests using unittest.mock that should use fakes?
→ Read `mock-to-fake-conversion.md`

Need to implement a specific pattern (CliRunner, builders, etc.)?
→ Read `patterns.md`

Want to extend the gateway system (e.g., dry-run preview)?
→ Read `advanced-extensions.md`

Not sure if I'm doing it right?
→ Read `anti-patterns.md`

Just need a quick lookup?
→ Read `quick-reference.md`

## When to Read Each Reference Document

### 📖 `gateway-architecture.md`

Read when:

* Adding or changing gateway/ABC interfaces

* Understanding the ABC/Real/Fake pattern

* Need examples of gateway implementations

* Want to understand what gateways are (and why they're thin)

* Creating a backend (higher-level abstraction that composes gateways)

Contents:

* What are gateway classes? (naming: gateways/adapters/providers)

* The three core implementations (ABC, Real, Fake)

* Code examples for each

* When to add/change gateway methods

* Design principles (keep gateways thin)

* Common gateway types (Database, API, FileSystem, MessageQueue)

* The DI boundary — only gateways get fakes

### 📖 `non-ideal-states.md`

Read when:

* Designing return types for gateway operations that can fail

* Deciding between exceptions and discriminated unions

* Implementing error injection in fakes

* Understanding error boundaries (where try/except belongs)

Contents:

* Non-ideal states vs exceptions (the core distinction)

* Decision framework for choosing between them

* How this shapes gateway signatures and fake design

* Error boundaries (try/except only in Real implementations)

* Three test categories per discriminated union operation

* The tracking-on-error decision

* isinstance() for type narrowing (never truthiness)

### 📖 `mock-to-fake-conversion.md`

Read when:

* Tests use `unittest.mock.patch` or `@patch` decorators

* An agent wrote tests with mocks instead of gateway fakes

* Converting existing mock-based tests to the gateway pattern

Contents:

* Step-by-step conversion workflow (audit, find/create gateway, inject, rewrite)

* "subprocess.run is never the right gateway boundary"

* Monkeypatch decision tree (when it's still OK)

* Common pitfalls (wrong abstraction level, subprocess-level gateways)

### 📖 `testing-strategy.md`

Read when:

* Deciding where to put a test

* Understanding the five testing layers

* Need test distribution guidance (5/70/10/10/5 rule)

* Want to know which layer tests what

Contents:

* Layer 1 "fake-check": Unit tests of fakes (verify test infrastructure)

* Layer 2 "real-sanity": Integration sanity tests with mocking (quick validation)

* Layer 3 "pure": Pure unit tests (zero dependencies, isolated testing)

* Layer 4 "logic": Business logic over fakes (majority of tests)

* Layer 5 "smoke": Business logic integration tests (smoke tests over real systems)

* Decision tree: where should my test go?

* Test distribution examples

### 📖 `python-specific.md`

Read when:

* Working with pytest fixtures

* Need Python mocking patterns

* Testing Flask/FastAPI/Django applications

* Understanding Python testing tools

* Need Python-specific commands

Contents:

* pytest fixtures and parametrization

* Mocking with unittest.mock and pytest-mock

* Testing web frameworks (Flask, FastAPI, Django)

* Python testing commands

* Type hints in tests

* Python packaging for test utilities

### 📖 `workflows.md`

Read when:

* Adding a new feature (step-by-step)

* Fixing a bug (step-by-step)

* Adding a gateway method (complete checklist)

* Changing an interface (what to update)

* Managing dry-run features

Contents:

* Adding a new feature (TDD workflow)

* Fixing a bug (reproduce → fix → regression test)

* Adding a gateway method (8-step checklist with examples)

* Changing an interface (update all layers)

* Managing dry-run features (wrapping pattern)

* Testing with builder patterns

### 📖 `patterns.md`

Read when:

* Implementing constructor injection for fakes

* Adding mutation tracking to fakes

* Using CliRunner for CLI tests

* Building complex test scenarios with builders

* Testing dry-run behavior

* Need code examples of specific patterns

Contents:

* Constructor injection (how and why)

* Mutation tracking properties (read-only access)

* Using CliRunner (not subprocess)

* Builder patterns for complex scenarios

* Simulated environment pattern

* Error injection pattern

* Dry-run testing pattern

### 📖 `anti-patterns.md`

Read when:

* Unsure if your approach is correct

* Want to avoid common mistakes

* Reviewing code for bad patterns

* Debugging why tests are slow/brittle

Contents:

* ❌ Testing speculative features

* ❌ Hardcoded paths in tests (catastrophic)

* ❌ Not updating all layers

* ❌ Using subprocess in unit tests

* ❌ Complex logic in gateway classes

* ❌ Fakes with I/O operations

* ❌ Testing implementation details

* ❌ Incomplete test coverage for gateways

### 📖 `quick-reference.md`

Read when:

* Quick lookup for file locations

* Finding example tests to reference

* Looking up common fixtures

* Need command reference

* Want test distribution guidelines

Contents:

* Decision tree (where to add test)

* File location map (source + tests)

* Common fixtures (tmp_path, CliRunner, etc.)

* Common test patterns (code snippets)

* Example tests to reference

* Useful commands (pytest, ty, etc.)

* Quick checklist for adding gateway methods

## Quick Navigation by Task

### I'm adding a new feature

* Quick start: `quick-reference.md` → Decision tree

* Step-by-step: `workflows.md#adding-a-new-feature`

* Patterns: `patterns.md` (CliRunner, builders)

* Avoid: `anti-patterns.md` (speculative tests, hardcoded paths)

### I'm fixing a bug

* Step-by-step: `workflows.md#fixing-a-bug`

* Patterns: `patterns.md#constructor-injection-for-fakes`

* Examples: `quick-reference.md#example-tests-to-reference`

### I'm adding/changing a gateway method

* Understanding: `gateway-architecture.md`

* Step-by-step: `workflows.md#adding-a-gateway-method`

* Checklist: `quick-reference.md#quick-checklist-adding-a-new-gateway-method`

* Avoid: `anti-patterns.md#not-updating-all-layers`

### I don't know where my test should go

* Decision tree: `quick-reference.md#decision-tree`

* Detailed guide: `testing-strategy.md`

* Examples: `quick-reference.md#example-tests-to-reference`

### I need to implement a pattern

* All patterns: `patterns.md`

* Examples: `quick-reference.md#common-test-patterns`

### I think I'm doing something wrong

* Anti-patterns: `anti-patterns.md`

* Correct approach: `workflows.md`

## Visual Layer Guide

```
`┌──────────────────────────────────────────────────────────────┐
│ Layer 5 "smoke": Business Logic Integration Tests (5%) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Real database, filesystem, APIs, actual subprocess │ │
│ │ Purpose: Smoke tests, catch integration issues │ │
│ │ When: Sparingly, for critical workflows │ │
│ │ Speed: Seconds per test │ │
│ │ Location: tests/e2e/ or tests/integration/ │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 4 "logic": Business Logic Tests (70%) ← MOST TESTS │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ FakeDatabase, FakeApiClient, FakeFileSystem │ │
│ │ Purpose: Test features and business logic extensively │ │
│ │ When: For EVERY feature and bug fix │ │
│ │ Speed: Milliseconds per test │ │
│ │ Location: tests/unit/, tests/services/, tests/commands/ │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 3 "pure": Pure Unit Tests (10%) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Zero dependencies, no fakes, no mocks │ │
│ │ Purpose: Test isolated utilities and helpers │ │
│ │ When: For pure functions, data structures, parsers │ │
│ │ Speed: Milliseconds per test │ │
│ │ Location: tests/unit/ │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 2 "real-sanity": Integration Sanity Tests (10%) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ RealDatabase with mocked connections │ │
│ │ Purpose: Quick validation, catch syntax errors │ │
│ │ When: When adding/changing real implementation │ │
│ │ Speed: Fast (mocked) │ │
│ │ Location: tests/integration/test_real_*.py │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 1 "fake-check": Fake Infrastructure Tests (5%) │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Test FakeDatabase itself │ │
│ │ Purpose: Verify test infrastructure is reliable │ │
│ │ When: When adding/changing fake implementation │ │
│ │ Speed: Milliseconds per test │ │
│ │ Location: tests/unit/fakes/test_fake_*.py │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
`
```

## Key Principles

* Thin gateway layer: Wrap external state, push complexity to business logic

* Fast tests over fakes: 70% of tests should use in-memory fakes

* Defense in depth: Fakes → sanity tests → pure unit → business logic → integration

* Test what you're building: No speculative tests, only active work

* Update all layers: When changing interfaces, update ABC/Real/Fake

* The DI boundary: Only gateways get fakes; everything above them is tested with real logic and fake gateways

## Layer Selection Guide

Distinguishing Layer 3 "pure" from Layer 4 "logic":

*
Layer 3 "pure" (Pure Unit Tests): ZERO dependencies - no fakes, no mocks, no external state

* Testing string utilities: `sanitize_branch_name("feat/FOO")` → `"feat-foo"`

* Testing parsers: `parse_git_status("## main")` → `{"branch": "main"}`

* Testing data structures: `LinkedList.append()` without any external dependencies

*
Layer 4 "logic" (Business Logic Tests): Uses fakes for external dependencies

* Testing commands: `create_worktree(fake_git, name="feature")`

* Testing workflows: `submit_pr(fake_gh, fake_git, ...)`

* Testing business logic that coordinates multiple integrations

If your test imports a Fake*, it belongs in Layer 4 "logic", not Layer 3 "pure".

## Default Testing Strategy

When in doubt:

* Write test over fakes (Layer 4 "logic") for business logic

* Write pure unit test (Layer 3 "pure") for utilities/helpers with no dependencies

* Use `pytest` with fixtures

* Use `tmp_path` fixture (not hardcoded paths)

* Follow examples in `quick-reference.md`

## Summary

For quick tasks: Start with `quick-reference.md`

For understanding: Start with `testing-strategy.md` or `gateway-architecture.md`

For step-by-step guidance: Use `workflows.md`

For implementation details: Use `patterns.md`

For error handling design: Check `non-ideal-states.md`

For converting mocks to fakes: Check `mock-to-fake-conversion.md`

For validation: Check `anti-patterns.md`

For Python specifics: Check `python-specific.md`

## More skills from dagster-io
erk-execby dagster-ioBefore running any erk exec command, check syntax with -h :fake-driven-testingby dagster-ioThis skill should be used when writing tests, fixing bugs, adding features,dagster-best-practicesby dagster-ioExpert guidance for Dagster data orchestration including assets, resources, automation, testing,dagster-conventionsby dagster-ioExpert guidance for Dagster data orchestration including assets, resources, schedules, sensors,dagster-expertby dagster-ioExpert guidance for working with Dagster and the dg CLI. ALWAYS use before doing any task that requiresdagster-integrationsby dagster-ioSkill that helps users discover and understand Dagster integration libraries. Used when users have requests related todgby dagster-ioDagster CLI for project creation, scaffolding components, launching assets, listing definitions,dignified-pythonby dagster-ioOpinionated production Python standards with automatic version detection (3.10-3.13). Use when

---

**Source**: https://github.com/dagster-io/fake-driven-testing/tree/HEAD/skills/fake-driven-testing
**Author**: dagster-io
**Discovered via**: mcpservers.org