Frontrun Documentation¶
Deterministic concurrency testing for Python.
Race conditions are hard to test because they depend on timing. A test that passes 95% of the time is worse than a test that always fails, because it breeds false confidence. Frontrun replaces timing-dependent thread interleaving with deterministic scheduling, so race conditions either always happen or never happen. Free-threaded Python (3.13t, 3.14t) raises the stakes: races the GIL kept narrow become real, and frontrun — which supports free-threaded builds — exists to enumerate those interleavings and prove which one is buggy.
Four approaches, in order of decreasing interpretability:
DPOR — systematic exploration of every meaningfully different interleaving, with causal conflict analysis.
Bytecode exploration — random opcode-level schedules that often find races very efficiently, including races invisible to DPOR.
Marker schedule exploration — exhaustive exploration of all interleavings at the
# frontrun:marker level. Much smaller search space than bytecode exploration, with completeness guarantees.Trace markers — comment-based synchronization points for reproducing a known race window.
The DPOR engine also drives cross-process exploration: the same
frontrun.explore(...) interface applied to separate Python processes that
contend on shared external state (SQL and Redis).
Time itself can be scheduled: with clock="virtual" sleeps cost zero wall
time and TTL expiry becomes reachable, and with clock="explored" timer
firings are explored against other operations like any other interleaving
choice — see Virtual clock: timeout, retry, and TTL races.
Contents:
- Installation
- Quick Start
- Design Principles
- How Frontrun Works
- Case Studies: Races in Real, Unmodified Libraries
- DPOR in Practice
- What DPOR does
- What it can and cannot find
- Basic usage
- Interpreting results
- Example: verifying that a lock fixes a race
- Example: multiple shared objects
- C-level I/O interception via
LD_PRELOAD - ORM helpers:
django_dporandsqlalchemy_dpor - Redis key-level detection
- Prefer
assert_holds()over manual asserts - Workers in separate processes
- DPOR: Dynamic Partial Order Reduction
- Vector Clocks and Happens-Before
- Search Page
- Examples
- SQLAlchemy Lost-Update Race Condition
- SQL Conflict Detection – Technical Details
- How a SQL Statement Becomes a DPOR Conflict
- The Refinement Hierarchy
- Transaction Grouping
- Concrete Example: SELECT-then-UPDATE Lost Update
- Anomaly Classification
- PostgreSQL Isolation Levels
- Formal Verification with TLA+
- Why Not z3/SMT for Row-Level Conflicts
- Wire Protocol Parsing (C-Level Drivers)
- Indexical INSERT Resource IDs
- References
- Redis Conflict Detection – Technical Details
- Cross-Process Exploration
- Virtual clock: timeout, retry, and TTL races
- Trace Filtering
- How It Works Under the Hood
- Python bytecode and the interleaving problem
sys.settrace: line-level and opcode-level tracingsys.setprofile: detecting C-level calls- Monkey-patching: cooperative primitives and I/O detection
LD_PRELOAD: C-level I/O interception- Putting the layers together
- What each layer can and cannot see
- Deadlock detection
- The DPOR algorithm in detail
- API Reference
Getting Started¶
The front door is frontrun.explore() — point it at shared state, some
workers, and an invariant, and DPOR explores every meaningfully different
interleaving, no annotations needed:
import frontrun
class Counter:
def __init__(self):
self.value = 0
def increment(self):
temp = self.value
self.value = temp + 1
def test_increment_is_atomic():
result = frontrun.explore(
setup=Counter,
workers=Counter.increment,
count=2,
invariant=lambda c: c.value == 2,
)
result.assert_holds() # fails with the exact racy interleaving
Start with Quick Start, then DPOR in Practice for systematic race finding. For races frontrun has found in real, unmodified PyPI packages, see Case Studies: Races in Real, Unmodified Libraries.
Indices and Tables¶
search