Threads, Shared State, and Data Races

The handle() loop from Module 1 has processed every update on one thread all course long. Now the firm adds a second exchange feed and the update rate doubles. One loop cannot drain the queue anymore. The fix looks obvious: the vector is sitting right there, so split it in half, give each half to its own thread, and let a shared counter track how many updates were handled. Ten million updates go in. The counter comes back reading 5,027,405. You run it again and get 5,034,595. No crash, no exception, no warning from the compiler. Just a different wrong number every run.

That wrong number is the subject of this whole module. A second thread buys real throughput, and it quietly breaks rules you did not know you were relying on. This article builds the model of what broke. The rest of Module 4 builds the tools that fix it.

What a thread actually shares

Everything this course has run so far was one thread: a single stream of instructions marching through your code in order. std::thread starts a second stream inside the same process. You hand it any callable, a lambda works fine, and from that moment the operating system decides which thread runs on which core at which instant.

Each thread gets its own stack, so its locals are private, and it keeps its own position in the code. Everything else is shared. The heap is shared. Globals are shared. Anything reachable through a pointer or a reference is shared, which after Module 2 you know is most things worth having. Sharing is the point: two threads can only combine their work through memory both can reach. Sharing is also the entire problem.

long long updates_handled = 0;   // reachable by both threads

void run_half(const std::vector<MarketUpdate>& updates,
              std::size_t begin, std::size_t end) {
    for (std::size_t i = begin; i < end; ++i) {
        apply_update(updates[i]);
        ++updates_handled;       // both threads execute this line
    }
}

void handle_parallel(const std::vector<MarketUpdate>& updates) {
    std::size_t mid = updates.size() / 2;
    std::thread t1([&] { run_half(updates, 0, mid); });
    std::thread t2([&] { run_half(updates, mid, updates.size()); });
    t1.join();                   // join() blocks until the thread finishes
    t2.join();
}

One more ingredient arms the trap: you do not choose when threads run. The scheduler, the part of the operating system that hands out CPU time, pauses and resumes threads whenever it wants. It can pause a thread between any two instructions, mid-loop, mid-expression, halfway through a ++. What a pause costs is a Module 6 story. What matters today is that it can happen anywhere, at any moment, and differently on every run.

An increment is three steps

++updates_handled reads like one operation. The CPU runs three: (1) load the current value from memory into a register, one of the handful of on-chip slots the CPU does its arithmetic in, (2) add one inside the register, (3) store the result back to memory. The thread can be paused between any two of those steps, and the other thread can run all three of its own steps in the gap.

Say the counter holds 41 and each thread increments once. Thread A loads 41 into its register and gets paused. Thread B loads 41, adds, stores 42. Thread A resumes exactly where it stopped, with 41 still sitting in its register. It adds, and it stores 42. Both increments executed completely. The counter moved from 41 to 42, not 43. Thread B's update was overwritten by a store computed from a stale read, and nothing anywhere recorded that it happened.

Two clerks keeping one tally on a whiteboard make the same mistake. Each reads the number, works out the new total in their head, and walks back to write it. If both read 41 before either writes, both write 42. Neither clerk did the arithmetic wrong. The schedule did.

Animated lost-update interleaving. A shared counter cell holding 41 sits between two thread panels, each with an empty register slot. The value 41 physically flies from the cell into thread A's register, then into thread B's register. B adds one and its 42 flies back to the cell, which turns green and reads 42. Then A, still holding the stale 41, adds one and its 42 flies back to the same cell, which pulses red and still reads 42. The final frame holds on the message that two increments ran but the counter moved by one.

Both threads load 41, both add, both store 42. Two increments run and the counter moves by one.

The program below stages exactly this collision, five million increments per thread, and prints how many survive. Run it a few times. The lost count changes from run to run because the interleaving changes from run to run. The std::atomic version at the end always lands exactly on ten million; what atomic does to earn that is a later article in this module.

A data race is undefined behavior

The C++ standard has a name and a verdict for what you just ran. When two threads access the same memory location, at least one of the accesses writes, and nothing synchronizes them, that is a data race, and the standard declares the whole program's behavior undefined. That is the same word Module 2 used for reading through a dangling pointer, and it carries the same weight here. It does not mean the value may be stale. It means the rules no longer constrain what the program does.

The verdict is that harsh because the compiler optimizes on the assumption that no data race exists. The benchmarking article showed the compiler deleting work whose result nothing used; the same license applies here. If a loop increments a plain variable and no synchronization is in sight, the compiler may keep the value in a register for the whole loop and store it once at the end, because no single-threaded program could tell the difference. The demo above marks its increment function noinline for exactly that reason; without it, each thread would store once and the collision you watched would be optimized out of existence. Racy code does not just risk a stale number. It risks being transformed into code you never wrote.

There is a practical detector. Build with -fsanitize=thread and ThreadSanitizer reports data races as they happen, with the two stack traces involved, instead of letting them corrupt results quietly. Racy code can pass a thousand runs and fail on the thousand and first, so a tool that catches the race itself, not the symptom, earns its overhead in tests.

One distinction sharpens all of this. A data race is the mechanical fault just defined. A race condition is a logic fault where the outcome depends on the order of events, and it can exist with zero data races. Picture a risk check: two threads each run "if the position is under the limit, send the order." Make every read and write of the position perfectly synchronized, and both threads can still pass the check before either records its order. Every access was clean, and the combined position breached the limit anyway. The counter bug is the reverse case: nothing wrong with the logic, everything wrong with the access. Fixing a data race is mechanical, you synchronize the access. Fixing a race condition means redesigning the logic so the check and the act become one step.

"What is a data race, and is it the same as a race condition?" is one of the most common concurrency warm-ups in quant interviews, and the two-sided answer above, access fault versus ordering fault, is the shape they want. The formal definition lives in the C++ memory model if you want the standard's wording.

Checkpoint

Two threads each execute ++counter exactly once, and counter starts at 41. In the load, add, store model, which final values are possible?

Correctness first, then speed

In a trading system the counter is never just a counter. It is a position, an order count against a risk limit, a sequence number that decides whether the book is intact. If it can be wrong by one, it can be wrong by five thousand, and everything downstream of it is wrong at full speed. That is why this module's title is a method: handle threads without lying to yourself. A parallel program that has not made its sharing safe is not fast. It is broken, with good latency numbers.

The rest of the module runs in that order. First make shared access correct with a mutex, and learn what the lock really costs. Then see when an atomic makes correct access cheaper. Then design most of the sharing away entirely with a queue built for exactly one producer and one consumer. Speed tuning starts after the number is right.

In interviews

"What is a data race?" → "Two threads access the same memory location, at least one writes, and nothing synchronizes them. In C++ that is undefined behavior, not just a stale read."
"Why did your two-thread counter lose updates?" → "An increment is load, add, store. Both threads can load the same value, and then the second store overwrites the first. Two increments run, the count moves once."
"Is a race condition the same as a data race?" → "No. A data race is unsynchronized access, undefined behavior by definition. A race condition is order-dependent logic, like check-then-act, and survives even perfect synchronization."
"How do you find data races?" → "ThreadSanitizer in tests, plus a design review of what is shared and who writes it. Racy code can pass a thousand runs and fail on the thousand and first."

Next comes the fix: the mutex, which puts a door around those three steps so only one thread at a time can be inside them.