std::vector in HFT: Why push_back slows Your Trading Engine

std::vector in HFT: Why push_back Is Silently Slowing Down Your Trading Engine

5-minute read · C++ · Low-latency trading · Memory management

Here is a question that has bitten more than one trading engineer:

Your order-handling loop is fast. The profiler shows nothing suspicious. The code review passed. So why does your system occasionally freeze for milliseconds, right when the market is busiest?

Hold that thought. We will write the kind of code most teams ship to production, measure it, and see if the answer shows up. Spoiler: at first, it doesn't.

The baseline: filling a std::vector with 1 million orders

An HFT execution layer constantly collects things: orders, fills, market data ticks. The default C++ container for that job is std::vector, and the default way to fill it is push_back.

#include <chrono>
#include <cstdint>
#include <iostream>
#include <vector>

struct Order {
    uint64_t id;
    double   price;
    uint32_t qty;
    char     side;
};

int main() {
    constexpr size_t N = 1'000'000;
    std::vector<Order> orders;               // no reserve

    auto start = std::chrono::steady_clock::now();
    for (size_t i = 0; i < N; ++i) {
        orders.push_back({i, 100.0 + (i % 100) * 0.01, 100, (i & 1) ? 'B' : 'S'});
    }
    auto end = std::chrono::steady_clock::now();

    std::cout << std::chrono::duration<double, std::milli>(end - start).count() << " ms\n";
}

On my test machine this takes about 38 ms for a million orders, roughly 38 nanoseconds per order. It is correct, idiomatic and fast. Most developers would stop here, and honestly, this code works reasonably well in plenty of production systems.

That is exactly why it is dangerous. Keep this number. It is our baseline.

The question the average can't answer

An average hides the thing that hurts in trading: the worst case. So let's time every single push_back and keep the slowest one.

long long worst_ns = 0;
for (size_t i = 0; i < N; ++i) {
    auto s = std::chrono::steady_clock::now();
    orders.push_back({i, 100.0 + (i % 100) * 0.01, 100, (i & 1) ? 'B' : 'S'});
    auto e = std::chrono::steady_clock::now();
    worst_ns = std::max<long long>(worst_ns,
        std::chrono::duration_cast<std::chrono::nanoseconds>(e - s).count());
}

The result: the average insert costs about 38 ns, but the slowest single push_back took about 8 milliseconds. That is more than 200,000 times slower than the average, from one innocent line.

In a market where competitors react in microseconds, 8 ms is an eternity. It is your order sitting still while the price moves away. And nothing in our code looks slow. So where did it come from?

The culprit: hidden dynamic memory allocation

A std::vector keeps its elements in one contiguous block of heap memory. When push_back is called and that block is full, the vector has to:

1. Ask the allocator for a new, bigger block (GCC roughly doubles the capacity each time)
2. Copy or move every existing element into the new block
3. Free the old block
4. Only then insert your new element

In our loop, that happened 21 times on the way to one million orders. The last of those reallocations alone moves around 12 MB (524,288 orders × 24 bytes) into fresh memory that the operating system has to hand over page by page. A large share of that 8 ms stall is this one moment.

The nasty part is that none of it is visible in the source. There is no new, no malloc, no warning. The cost hides inside a method that looks like a constant-time append. Every reallocation also invalidates pointers and iterators to existing elements and leaves the CPU cache cold. That is the answer to our burning question: the spikes are not random, they are reallocations.

The fix: one line with reserve

If you know roughly how many elements are coming, tell the vector up front:

std::vector<Order> orders;
orders.reserve(N);   // one allocation, before the hot loop starts

Nothing else changes. Same loop, same data. Here is what I measured (median of 11 runs):

Metric (1M orders)	Without reserve	With reserve
Total loop time	~38 ms	~11–12 ms
Slowest single push_back	~8,100 µs	~100 µs
Reallocations	21	0

That is roughly 3x faster overall and about 80x better in the worst case. The worst case is the number that matters, because that is the one that decides whether you get filled.

Test setup: g++ 13.3, -O2, 24-byte Order struct, shared single-core cloud sandbox. Absolute numbers will differ on your hardware, especially on a tuned, core-pinned trading box. The ratio and the shape of the result are the point. Per-call timing also adds its own small overhead.

Notice the remaining ~100 µs is not zero. Fresh memory still has to be touched for the first time, which causes page faults. Pre-faulting memory and using custom allocators are the next level of the same idea, and a good topic for a follow-up post.

Why memory management is a latency problem in HFT

In most software, a few extra milliseconds go unnoticed. In HFT, latency is the product. Two ideas follow from that:

• Averages lie, tail latency decides. A system with a 38 ns average and an 8 ms spike will lose to a system with a steady 100 ns every time. Your P99 and P99.9 latency are what your strategy actually experiences during a volatile open.
• The hot path should never ask the OS for anything. Memory allocation can take locks, trigger page faults and even enter the kernel. None of those have a bounded, predictable cost.

That is why experienced low-latency C++ developers follow a simple rule: allocate at startup, reuse on the hot path. In practice that means:

• Call reserve() with a safe upper bound before the trading loop starts
• Use clear() between sessions instead of destroying and recreating containers, since clear() keeps the capacity
• Prefer pre-allocated pools, ring buffers or fixed-size arrays for order and market data objects
• Never let a container silently grow while orders are flowing

Key takeaways

• std::vector::push_back is amortized O(1), but individual calls can spike when the vector reallocates
• Average latency can look perfect while tail latency is terrible
• One reserve() call removed all 21 reallocations and cut the worst-case insert by roughly 80x in this test
• In HFT execution layers, predictable memory behavior matters as much as raw speed

FAQ

Is std::vector a good choice for high-frequency trading? Yes, its contiguous layout is cache-friendly, which is great for speed. The key is to control its memory: reserve capacity up front so it never reallocates on the hot path.

What is the difference between size() and capacity()? size() is how many elements are stored. capacity() is how many fit before the vector must reallocate. reserve(n) raises the capacity without changing the size.

Is reserve() enough to guarantee low latency? It removes reallocation spikes, which is the biggest win, but not every source of jitter. Page faults on first touch, allocator behavior and cache effects still matter, and those are what custom allocators and memory pools address.

What happens if I push more elements than I reserved? The vector still works, but it reallocates like before. That is why you should reserve a safe upper bound, not a tight guess.

What's next

This was the first layer of memory control in a low-latency system. Next, we will look at the remaining ~100 µs: page faults, pre-touching memory and custom allocators. If you work on or want to learn HFT systems, follow along and tell me in the comments: what is the worst latency spike you have ever tracked down?