Large Dataset Performance Optimization

Where you'd use it

• Trading screens, monitoring dashboards, logs and live tables with frequent updates.
• Admin lists that grew from hundreds to tens of thousands of rows.
• Any screen where profiling shows long renders or a laggy input.

How we built it

The board's data, filter, sort and selection are ordinary React state, because the techniques here are React rendering techniques. The measurements are not: a module-level store records numbers and the panel reads it four times a second, so measuring never re-renders the board. Render time runs from the event that caused the update (a keystroke, click, scroll or feed update) to the first layout effect after the commit, which includes filtering, rendering and DOM updates. React's <Profiler> would be the usual tool, but its callback doesn't run in production builds unless you ship the profiling build (next build --profile). Each row render bumps a counter (“rows re-rendered”); typing delay is keydown to the next painted frame; a PerformanceObserver counts long tasks (over 50 ms) and their blocking time; a requestAnimationFrame loop counts frames.

Windowing: rows are 36 px, so the rows to render are floor(scrollTop / 36) plus what fits in the viewport, with 8 extra on each side, positioned inside a spacer as tall as the whole list. Fixed heights make this constant-time; practice 7 handles rows of different heights. Memoized rows: rows are wrapped in memo, receive the instrument object (whose identity changes only when it's updated), a selected boolean and a useCallback handler; feed updates replace only the changed objects. With it off, rows are plain components given a fresh arrow function each render, the usual naive code. Cached filter and sort: useMemo over data, filter and sort; note that it can't help when the data itself changed, so with a sort active every feed update still re-sorts. Deferred filtering: useDeferredValue(query) feeds the board, so a keystroke first renders with the old query (the memoized board skips) and the new filter renders in the background. Indexed updates: a feed update names instruments by id; a Map from id to position finds each in O(1), where findIndex scans the list for every changed row.

Two guards keep the naive version usable: turning windowing off above 20,000 rows asks first (with the playground's shared confirm dialog), and any update slower than one second pauses the feed. The benchmark replays the same seeded scenario through the same handlers as you, so its numbers compare configurations, not luck.

Best practices

• Do: Profile first (React DevTools Profiler, the Performance panel), then fix the biggest cost.

  Avoid: Sprinkling memo and useMemo everywhere: they cost memory and comparisons and often don't help.

• Do: Window long lists, so render cost depends on the viewport, not the data size.

  Avoid: Rendering thousands of rows “because it's only text”: each row is several DOM nodes and a component render.

• Do: Update immutably and pass stable props (useCallback, primitives) to memoized rows.

  Avoid: onSelect={() => select(row.id)} or style={{...}} on a memo row: a new prop every render, so it always re-renders.

• Do: Keep typing urgent with useDeferredValue or startTransition, and memoize the expensive subtree.

  Avoid: Deferring the value but rendering the heavy list anyway in the urgent render: nothing gets faster.

• Do: Index by id (Map) when applying many updates to a large list.

  Avoid: list.find / findIndex inside a loop over updates: quadratic, invisible at 100 rows, painful at 50,000.

• Do: Measure in a production build, on a mid-range device, several runs.

  Avoid: Judging performance in development mode: React does extra work there and numbers are several times higher.

Try it with the demo controls

Naive / Optimized

With 20,000 rows, Naive re-renders all 20,000 rows on every update, blocks the main thread and drops frames; Optimized re-renders only the few visible rows that changed.

Windowing

“Rows in the page” drops from every matching row to about 25. Turning it off at 50,000 rows asks first.

Memoized rows

With windowing on, “rows re-rendered” per update goes from about 25 to the few visible rows that changed; clicking a row re-renders 2 rows instead of all.

Cached filter & sort

Sort by Change % and pause the feed, then click rows: “Filter + sort” stays at its old value with the cache, and re-runs on every click without it.

Deferred filtering

Turn it off with windowing off and type: the input itself lags (typing delay in the hundreds of ms). With it on, letters appear at once and “updating…” shows while the board catches up.

Indexed updates

Set 2,000 rows per update at 50,000 instruments: “Apply update” goes from under 1 ms to hundreds of ms without the index.

Updates / Rows per update

More updates per second multiply every per-update cost; find where each configuration starts to drop frames.

Pause feed

Stops live updates (also paused automatically while the tab is hidden or after a render over 1 s).

Run benchmark

Runs the same 10-second scenario; run Naive and Optimized with the same dataset and compare the rows. The best value in each column is marked.

Trade-offs and limits

• Windowing breaks find-in-page and printing of off-screen rows, and needs care for screen readers (row counts and positions are exposed here).
• memo adds a prop comparison per row and keeps previous props in memory; for cheap rows that change every time it can cost more than it saves.
• Deferring keeps the input responsive but shows stale results for a moment; the board is dimmed while it catches up.
• All work stays on the main thread. Beyond this scale, move filtering and sorting to a Web Worker or the server (practice 13), or draw with canvas.
• The browser downloads the whole dataset (7 MB of JSON at 50,000 rows); real apps page or stream it.
• Page-side timing covers render and commit but not the browser's own layout and paint; frame rate and long tasks show that part.