Technical deep-dive · Reality Engine

Tracking we build, not rent

The Reality Engine is our in-house tracking stack — body, face, iris, ear, and wrist — running in the browser with no app install. Standardized TensorFlow models underneath, our own solvers and API on top, rendered in three.js.

Why we built it

Platforms rent you tracking. We spent five years inside one — supporting the developers, agencies, and brands shipping production WebAR at 8th Wall — and learned exactly where rented stacks crack under a launch deadline. So we built our own: every layer, from the perception models to the pixels, under our hands.

Client projects start from that working infrastructure — improved continuously, versioned, and documented publicly — with no vendor roadmap between an idea and shipping it.

Reality Engine

Four layers, one pair of hands

01 — Perception

Standardized TensorFlow models — the same open, battle-tested networks the industry benchmarks on. We don't ship mystery models; we engineer on proven ones.

02 — Solve

Our layer: a metric 3D body solve (33 camera-space joints), 6DoF chest, hip, head, and wrist anchors, dedicated iris and ear solvers, filtering and visibility gating tuned per effect.

03 — Render

three.js through our own API: person occlusion, room-light estimation, and a real refraction/reflection material system. One pipeline from camera to pixels.

04 — Delivery

A URL. iOS Safari and Android Chrome, no app store, no install — self-hostable on your infrastructure.

Tracking, by anatomy

Body

33 metric 3D joints, chest and hip 6DoF anchors, real perspective from a single camera. Digital costumes, AR mirrors, gesture games, AR-activated apparel.

More

Face, iris & ear

A 478-point expression-true mesh for filters, makeup, eyewear, and headwear — plus the two anchors most stacks don't have: iris tracking, and true ear anchors (lobe, canal, helix) for earrings and earbuds.

More

Wrist

6DoF wrist anchoring for watches, bracelets, and jewelry — the newest anchors in the engine, shipping through try-on pilots.

More

World

SLAM world tracking, sky segmentation, and image targets — deployed through 8th Wall's open-source stack, one of the solutions we build on alongside the engine.

What it powers

Virtual try-on

Makeup, eyewear, earrings, watches — anchored to real anatomy on the shopper's own phone, one tap from the product page.

More

Semi-digital apparel

Garments designed with a digital layer: body-tracked effects anchored to physical pieces, a woven tag as the key. Uncharted is the living example.

See Uncharted

The engine isn't for sale. The experiences it powers are.

Start from working infrastructure

Your project begins on a tracking stack that already runs — not on rebuilding the camera pipeline against your budget.

evan@shapereality.com