Spatial UI Design

Spatial UI design is the practice of creating user interfaces that exist in three-dimensional space rather than on a flat screen. In XR, menus, panels, buttons, labels, and controls must be positioned, scaled, and laid out within a volumetric environment — and they must respond to inputs that have no direct equivalent in 2D: gaze, hand gestures, head orientation, voice, and the user's physical position in the room.

The Break from Flat UI

Conventional screen UI design operates on a 2D canvas with well-established conventions: fixed pixel dimensions, z-ordering for depth simulation, pointer-click interaction, and the assumption that the user is seated at a fixed distance. None of these assumptions hold in XR.⁴

In a headset, the display follows the user's head. UI elements that are too close cause vergence strain; too far and they lose legibility. Elements positioned directly in the user's line of sight can obscure important real-world content; elements placed peripherally may be missed. There is no global "desktop" — UI competes spatially with the environment and with virtual objects placed in it.

The two dominant layout metaphors are world-locked UI (panels anchored to the physical space, staying fixed as the user moves) and head-locked UI (panels that follow the user's gaze, always in the same relative position). Head-locked UI is convenient but fatiguing — the user cannot look away from the interface, and the constant motion is visually intrusive. World-locked UI requires users to turn or walk to interact with panels that may end up behind them or too far away. Most systems use body-locked UI as a middle ground: panels that follow the user's body orientation with some lag, remaining roughly in front without strictly tracking head movement.

Depth and Scale

UI elements in XR must respect depth perception. A panel rendered at arm's reach (60–80 cm) and one rendered at 3 metres require different text sizes and hit-target dimensions to appear equally readable and selectable. Apple's visionOS Human Interface Guidelines specify a baseline reading distance of 1.5 metres for spatial windows, with font sizes adjusted to maintain consistent angular subtense.²

The billboard effect — rendering a flat UI panel that always faces the camera — is a common technique that solves the legibility problem for labels and signage but breaks physical plausibility when overused. Panels that are permanently face-on to the viewer cannot be walked around naturally, which is perceptible and reduces spatial coherence.

Curved and Cylindrical Layouts

A flat panel viewed peripherally subtends an increasingly oblique angle as the user looks away, causing geometric distortion. Cylindrical UI — bending panels along a cylinder centred on the user — keeps every point of the surface at a constant angle to the user regardless of left-right head rotation. Apple Vision Pro's system windows default to this cylindrical layout, and media players frequently use it to allow a wide field of content without corner distortion.²

Hit Testing and Interaction Volumes

In 2D, a button is clicked when a pointer enters its bounding box. In 3D, interaction must account for a beam (ray casting from the hand or gaze), a volume (physical hand reaching into the button's depth), or proximity (the hand approaching a surface). Hit testing in spatial UI requires defining interaction volumes for each element — typically a thin slab for far-field panels, and a sphere or capsule for near-field objects — and resolving conflicts when multiple elements overlap.³

Microsoft's Mixed Reality Toolkit (MRTK) codified a set of interaction patterns for HoloLens that became influential across the industry: the hand ray (a ray extending from the user's palm for far-field pointing), the direct touch (fingertip physically contacting a surface for near-field), and the grab (hand closing around an object to manipulate it).¹ These three interaction types — far ray, near touch, grab — cover the interaction vocabulary needed for most spatial UI scenarios.

Legibility and Typography

Typography in XR is substantially more constrained than on screen. Rendering resolution in current headsets is lower than a laptop display, anti-aliasing is more limited, and the display is physically in front of the eye rather than emitting light toward it from a distance. Text at small sizes becomes illegible before it would on a screen. The practical minimum is approximately 0.5–1 degree of visual angle per character — at a reading distance of 1.5 metres, this corresponds to roughly 20pt type at 72 DPI. System fonts with generous x-height and high contrast outperform decorative typefaces substantially.²

References

App patterns — Windows Mixed Reality Design, Microsoft(accessed May 1, 2026)
Eyes — visionOS Human Interface Guidelines, Apple(accessed May 1, 2026)
Interaction SDK Overview — Meta Quest Developer Docs(accessed May 1, 2026)
3D User Interfaces: Theory and Practice, 2nd ed. — LaViola, Kruijff et al., Addison-Wesley, 2017(accessed May 1, 2026)
A Survey of Augmented Reality — Billinghurst, Clark & Lee, Foundations and Trends in HCI, 2015