The closing article of the series treats the camera as a linear operator in projective space and recovers each previous article’s projection mode as a restricted case of the modern graphics-processing-unit rendering pipeline. The pipeline expresses the forward map as a composition of three matrices that the engine applies in sequence to each world vertex,

The model matrix $M$ transforms model-local coordinates into world coordinates. The view matrix $V$ transforms world coordinates into camera-frame coordinates. The projection matrix $P$ transforms camera-frame coordinates into clip-space coordinates that the graphics hardware maps to the screen.

The perspective division follows the matrix multiplication and converts the four-component clip-space coordinate into a three-component normalised-device coordinate through division by the fourth component. The division is the operational meaning of perspective foreshortening and is the new mathematical content that distinguishes the projective synthesis from the affine projection modes of the previous articles.

The article frames each previous projection mode as a specific choice of $P$, $V$, and $M$ that produces the visual signature of the mode. Top-down projection uses an orthographic $P$. Side-scrolling, belt-scroll, oblique, and axonometric projections all use orthographic $P$ with different model or view transforms. Mode 7 uses an orthographic $P$ with a per-scanline view transform. Sprite scaling uses a per-sprite model transform that approximates the perspective division. Raycasting uses a per-column ray that the perspective division produces directly. Each restricted case falls out of the projective framework as a limit, a constraint, or a per-fragment approximation.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is the projective matrix pipeline that the article describes. The delivery mechanism is the modern graphics-processing-unit pipeline that the article credits as the modern realisation of the projective framework.

A Brief History of the Projective Synthesis

The projective interpretation of perspective drawing dates to the Renaissance. Filippo Brunelleschi’s perspective experiments in 1413 Florence demonstrated the use of a fixed vanishing point and the linear convergence of parallel lines. Leon Battista Alberti’s 1435 treatise On Painting formalised the technique as a mathematical procedure that the painter could follow to construct geometrically-correct perspective drawings.

The projective geometry of lines and planes emerged in the seventeenth century through Girard Desargues’s work on conics and Blaise Pascal’s hexagon theorem. The nineteenth century saw the development of projective geometry as a formal mathematical discipline distinct from Euclidean geometry, with Jean-Victor Poncelet’s 1822 treatise On the Projective Properties of Figures establishing the field.

Computer graphics adopts the projective framework in the mid-twentieth century through the work of Larry Roberts in 1963 on hidden-surface removal and James Blinn and Martin Newell in the 1970s on transformation matrices for shading and texturing. The OpenGL and Direct3D standards of the 1990s codified the matrix pipeline that modern graphics-processing-unit hardware accelerates.

The video-game adoption of true projective perspective rendering arrives in the early 1990s through arcade hardware that supported polygon rendering and through home consoles with cartridge coprocessors that accelerated the matrix arithmetic.

Star Fox from Nintendo and Argonaut Software in 1993 on the Super Nintendo Entertainment System used the Super FX chip to render polygons in true perspective projection. The game ran at low frame rates on the SNES central processing unit but demonstrated three-dimensional polygon gameplay that the pre-coprocessor SNES hardware could not have produced.

Doom from id Software in 1993 on the IBM PC used a binary-space-partition rendering technique that the previous article on raycasting identified as outside the raycasting framework. Doom’s rendering is sometimes described as two-and-a-half-dimensional because the camera could not tilt or roll and the levels did not contain rooms above rooms.

Quake from id Software in 1996 on the IBM PC brought true projective perspective rendering to the mass-market personal computer. The Quake engine implemented the full three-dimensional pipeline including arbitrary camera placement, fully three-dimensional geometry, and the perspective division per pixel that the modern pipeline requires.

The modern era from the late 1990s onward runs the projective pipeline on graphics-processing-unit hardware that handles the matrix arithmetic and the perspective division at the hardware-rasterisation level. The article treats the modern pipeline as the universal projection framework that subsumes the affine and pseudo-three-dimensional modes of the previous articles.

The PVM Pipeline

The world coordinate of a vertex is a three-dimensional position $\mathbf{p}_{\text{world}} = (w_x, w_y, w_z)$ in the $y$-down convention of the previous articles. The homogeneous augmentation adds a fourth component that the projective framework requires,

The fourth component distinguishes points that have a finite location in space from directions that extend to infinity. A point at finite location has $w = 1$. A direction vector has $w = 0$.

The pipeline applies three matrices to the homogeneous world coordinate.

The model matrix $M$ is a four-by-four matrix that transforms model-local coordinates into world coordinates. For a single object, $M$ encodes the object’s position, orientation, and scale in the world frame,

The view matrix $V$ is a four-by-four matrix that transforms world coordinates into camera-frame coordinates,

The view matrix encodes the camera’s position and orientation in the world frame. A camera at the origin looking along $+w_z$ into the screen has $V$ as the identity matrix under the series’s depth-into-screen convention.

The projection matrix $P$ is a four-by-four matrix that transforms camera-frame coordinates into clip-space coordinates,

The projection matrix encodes the camera’s lens parameters including the field of view, the aspect ratio, and the near and far clipping planes.

The full pipeline is

When the article treats the world coordinate as the model-local input without an explicit model transform, the pipeline simplifies to

The series typically uses this simplified form for projection-mode comparisons.

The Perspective Division

The clip-space coordinate is a four-component vector $\tilde{\mathbf{p}}_{\text{clip}} = (x_c, y_c, z_c, w_c)$. The perspective division converts the clip-space coordinate into a three-component normalised-device coordinate through division by the fourth component,

The division is the operational meaning of perspective foreshortening. A vertex further from the camera has a larger $w_c$ and a correspondingly smaller normalised-device coordinate in the screen plane. The smaller coordinate maps to a smaller screen-space position that the viewer reads as further away.

The viewport transformation converts the normalised-device coordinate into screen pixel coordinates,

where $W$ and $H$ are the screen width and height in pixels. The $y$ sign convention matches the screen $y$-down convention that the series uses throughout, where positive NDC $y$ corresponds to a position below the screen centre.

A projection matrix with the fourth row $(0, 0, 0, 1)$ produces $w_c = 1$ for every input vertex. The perspective division becomes a division by one, which is the identity. The projection collapses to an affine map without perspective foreshortening. The article calls this case the orthographic projection.

A projection matrix with the fourth row $(0, 0, 1, 0)$ produces $w_c = z_v$ where $z_v$ is the view-space depth. The perspective division divides by the depth, producing the inverse-distance scaling that the projective framework requires. The article calls this case the perspective projection.

The canonical perspective projection matrix takes the form

where $f = 1 / \tan(\phi/2)$ is the focal length factor for vertical field of view $\phi$, $a = W/H$ is the aspect ratio, and $A$ and $B$ are constants that map the view-space depth to the normalised-device-coordinate depth within the visible range.

The canonical orthographic projection matrix takes the form

where $r$ and $t$ are the half-widths of the viewing volume along the lateral and vertical axes and $z_n$ and $z_f$ are the near and far clipping distances. The article uses $z_n$ and $z_f$ for the clipping planes to avoid the symbol clash with the focal length factor $f$ in the perspective matrix above.

Reduction to Affine Special Cases

Each projection mode of the previous clusters emerges from the projective framework as a specific choice of $P$, $V$, and $M$.

The top-down floor case of the first cluster uses an orthographic $P$ with the view matrix oriented to look straight down at the world. The forward map of the top-down article follows from the projection matrix multiplication without the perspective division because $P_{\text{ortho}}$ produces $w_c = 1$.

The decoupled-vertical-axis case uses the same orthographic $P$ with an additional model transform that translates the object by its height above the ground. The shadow-drop forward map of the decoupled-vertical-axis article follows from the model transform combined with the orthographic projection.

The side-scrolling case uses an orthographic $P$ with the view matrix oriented to look horizontally at the world. The view matrix encodes the lateral camera position that the side-scrolling article treated as the canonical right-bias camera.

The parallax variant uses multiple model matrices each with a different scale factor on the camera position. The per-layer scroll factor of the parallax article is the per-layer scaling that the model matrix encodes.

The belt-scroll case uses an orthographic $P$ with a shear in the view matrix that adds the depth coordinate to the screen-y direction. The depth-mixing slope of the belt-scroll article is the shear coefficient in the view matrix.

The oblique projection of the second cluster uses an orthographic $P$ with a two-axis shear in the view matrix that the oblique article treated.

The axonometric projection uses an orthographic $P$ with a rotation in the view matrix that aligns the camera direction with the body-diagonal direction in the case of strict isometric.

The Mode 7 case uses an orthographic $P$ with a per-scanline view matrix that the engine updates through horizontal-blank scroll register modulation. The per-scanline scaling approximates the perspective division that the projective framework provides directly.

The sprite-scaling case uses a perspective $P$ applied per sprite anchor combined with a per-sprite scaling factor in the model matrix. The scaling factor $s = f/d$ of the sprite-scaling article is the inverse-depth scaling that the perspective division produces for the sprite anchor.

The raycasting case uses a perspective $P$ applied to a degenerate world model where the world is a two-dimensional grid extruded vertically by the wall height. The per-column ray cast of the raycasting article is the perspective division applied per screen column.

The Mode 7, sprite-scaling, and raycasting modes all use forms of inverse-distance scaling that emerge from the same perspective division. For a vertex at view-space depth $d$, the perspective division produces an inverse-distance scaling factor

The Mode 7 per-scanline scaling factor $z(s_y) = h/(s_y - s_y^{\text{horizon}})$, the sprite-scaling factor $s = f/d$, and the raycasting wall-slice height $h_{\text{slice}} = f/d_{\text{perp}}$ all specialise this unifying inverse-distance factor to their respective projection contexts.

The piecewise projections of the stylised-hybrid article combine multiple choices of $P$, $V$, and $M$ in a single scene based on world region or game state. The Mother hybrid uses an orthographic $P$ with a top-down view matrix for the ground tiles and an orthographic $P$ with a side-elevation view matrix for the building sprites. The two projections share the same screen output through the artistic convention that the previous article described.

The painter’s algorithm and Y-sort framework of the draw-order article are independent of the choice of $P$, $V$, and $M$ and apply to any forward map that produces a back-to-front sort key.

The picking framework of the previous article uses the inverse of the same projection matrix that the forward map applied. For an affine $P$, the inverse is the standard matrix inverse. For a perspective $P$, the inverse map includes the inverse of the perspective division which the article calls unprojection.

The Inverse Projective Map and Picking

The inverse of the projective pipeline takes a screen pixel and a depth value back to a world coordinate. The screen pixel inverts through the viewport transformation to a normalised-device coordinate,

The depth value either comes from the depth buffer that the graphics-processing-unit pipeline maintains or from a gameplay-context assumption such as the ground-plane assumption of the earlier articles.

The clip-space coordinate follows from the inverse perspective division,

where $w_c$ is the clip-space fourth component that the depth and the projection matrix together determine. The world-coordinate inverse follows from applying the inverses of $P$, $V$, and $M$ in reverse order,

The full inverse-projective pipeline gives the world coordinate of the click pixel at the given depth. The picking framework of the previous article applies the inverse pipeline followed by the per-projection-mode disambiguation strategies to identify the gameplay object.

For two-dimensional games that use only an orthographic $P$, the inverse projective map reduces to the affine inverse that the projection-mode articles introduced.

For three-dimensional games that use a perspective $P$, the inverse projective map requires the depth value to identify a unique world position. The depth buffer of the graphics-processing-unit pipeline provides this value through hardware-accelerated per-pixel depth storage.

A Worked Example

Consider a single world vertex at world position $\mathbf{p}_{\text{world}} = (10, 5, 50)$ in the $y$-down convention. The screen is 800 pixels wide and 600 pixels tall with aspect ratio $a = 800/600 = 4/3$.

The view matrix places the camera at the origin looking along $+w_z$ axis,

The orthographic projection matrix uses lateral half-width $r = 20$, vertical half-height $t = 15$, near plane $z_n = 1$, and far plane $z_f = 100$,

The clip-space coordinate through the orthographic pipeline is

The perspective division divides by $w_c = 1$ and produces the normalised-device coordinate

The viewport transformation gives the screen pixel coordinate

The vertex projects to screen $(600, 400)$ under the orthographic projection.

The perspective projection matrix uses focal length factor $f = 1$ for a 90-degree vertical field of view,

where $A$ and $B$ map the view-space depth to normalised-device-coordinate depth.

The clip-space coordinate through the perspective pipeline is

The perspective division divides by $w_c = 50$. The normalised-device coordinate is

The viewport transformation gives the screen pixel coordinate

The vertex projects to screen $(460, 330)$ under the perspective projection.

The two projections produce different screen positions for the same world point. The orthographic projection maps the lateral and vertical world coordinates directly to screen pixels without any depth dependence. The perspective projection applies the inverse-depth scaling that the perspective division produces to map the same world coordinates through the foreshortening factor that depth introduces.

A second vertex at world position $(10, 5, 25)$ is half as far from the camera as the first vertex. Under the perspective projection, the second vertex’s clip-space $w_c = 25$ gives a normalised-device coordinate twice as far from the screen centre as the first vertex,

The screen position is $(520, 360)$, further from the screen centre than the first vertex at $(460, 330)$. The closer vertex appears further from the centre on screen, matching the perspective expansion that the projective framework produces.

Under the orthographic projection, both vertices at $(10, 5, 50)$ and $(10, 5, 25)$ project to the same screen pixel $(600, 400)$ because the orthographic projection discards the depth coordinate from the screen-position computation.

The round-trip identity holds within each projection,

where $\varepsilon$ is the floating-point precision of the engine. For the perspective projection, the round-trip requires the depth value that the depth buffer or gameplay context provides.

Variations Within the Mode

The projective framework admits several variations that engines have explored.

A perspective-camera variant uses the canonical perspective projection matrix with adjustable field of view, aspect ratio, and near and far clipping planes. The variant is the default in three-dimensional graphics-processing-unit pipelines.

An orthographic-camera variant uses the canonical orthographic projection matrix without the perspective division. The variant is the default in two-dimensional games and in technical drawing.

An off-axis perspective variant shifts the projection’s principal point away from the screen centre to produce asymmetric viewing volumes. The variant appears in stereo rendering and in head-tracked virtual reality where the two eyes have slightly different projection matrices.

A frustum-symmetric variant forces the projection volume to be symmetric around the camera’s principal axis. The variant simplifies the matrix arithmetic at the cost of flexibility.

A reverse-Z variant maps the near plane to clip-space $z = 1$ and the far plane to clip-space $z = 0$ rather than the conventional opposite mapping. The variant improves depth-buffer precision for distant objects.

A linear-Z variant maps the view-space depth linearly into clip-space depth rather than through the standard non-linear mapping. The variant simplifies depth-buffer arithmetic at the cost of standard pipeline compatibility.

A non-pinhole-camera variant applies a non-rectangular projection such as a fisheye or panoramic mapping through a per-pixel shader that replaces the matrix-and-division pipeline. The variant produces non-rectilinear visual effects including dramatic curvature distortions that the standard projection cannot reproduce.

A motion-blur variant samples the projection at multiple time points within a single frame and accumulates the results to produce temporal anti-aliasing or motion-blur effects. The variant is rendering-pipeline territory beyond the projection math.

Delivery Mechanisms

The projective pipeline permits five distinct delivery mechanisms on period and modern hardware.

The first is software vertex-and-pixel processing on a general-purpose central processing unit. The engine computes the matrix multiplication and the perspective division in software and writes the resulting pixels to the frame buffer. The mechanism was the universal delivery on personal computers through the mid-1990s including Quake and the early three-dimensional first-person shooters.

The second is fixed-function vertex hardware on early graphics-processing-unit accelerator cards. The 3dfx Voodoo, the NVIDIA RIVA TNT, and similar mid-1990s hardware implemented the projective pipeline in fixed-function silicon that the engine drove through a standard API. The mechanism removed the central-processing-unit cost of matrix arithmetic and the perspective division.

The third is programmable shader hardware on modern graphics processing units. A vertex shader computes the projective matrix multiplication per vertex. A fragment shader computes the per-pixel colour including any non-linear projection effects. The mechanism is the dominant delivery on contemporary graphics-processing-unit hardware.

The fourth is cartridge coprocessor hardware on home consoles of the early 1990s. The Super FX chip in the Star Fox cartridge provided polygon-rendering computations that the standard SNES hardware could not produce. The mechanism brought three-dimensional rendering to the home console before the dedicated three-dimensional graphics-processing-unit era.

The fifth is dedicated three-dimensional rendering hardware on arcade systems of the early and mid-1990s. The Sega Model 1 board in 1992 and the Sega Model 2 board in 1993 demonstrated polygon rendering at frame rates and visual quality that home-console hardware could not match.

All five mechanisms implement the same projective pipeline. The choice trades hardware availability, implementation complexity, the maximum polygon throughput, and the achievable resolution and frame rate.

Where the Framing Breaks Down

The projective framework is insufficient when any of the following conditions hold.

When the rendering must produce non-photorealistic stylised output that the projective camera does not produce, the engine augments the projective pipeline with post-processing shaders that transform the rendered image into the desired style. The Limbo and Inside aesthetic that the previous article on hybrid projections treated is an example of this case.

When the world contains genuinely infinite or unbounded content that cannot fit within a finite far plane, the projective framework’s far-plane parameter limits the visible distance. The engine must either extend the far plane to a large but finite value or augment the projective pipeline with techniques such as logarithmic depth buffers or cascaded depth representations.

When the rendering must depict relativistic effects such as the apparent contraction of objects at high speed or the aberration of light, the standard projective pipeline is insufficient. The engine must implement a per-vertex aberration computation that the pipeline does not provide natively.

When the gameplay requires camera positions or orientations that the standard pinhole camera cannot produce, the engine must extend the pipeline with custom projection mathematics. The 360-degree panoramic camera of virtual-reality applications is the canonical example.

When the rendering must scale to extremely large or extremely small object dimensions that exceed the depth buffer’s precision, the engine must implement specialised depth-precision techniques such as the reverse-Z variant or the cascaded shadow-map techniques that the article does not treat in detail.

When the gameplay requires arbitrary non-linear visual effects that the matrix-based pipeline cannot produce, the engine must implement the effects through pixel-shader programs that operate after the projection or instead of it.

The Canon

The following games established the projective synthesis in the consumer video-game tradition. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Star Fox on the Super Nintendo Entertainment System in 1993 brought polygon-based three-dimensional rendering to the home console through the Super FX cartridge coprocessor. The on-rails space shooter established that home console hardware could deliver real-time polygon rendering with the assistance of cartridge silicon.

Doom on the IBM PC in 1993 brought a binary-space-partition rendering technique to the mass-market personal computer that produced visually three-dimensional gameplay without the full projective pipeline. The article treats Doom’s rendering as a precursor to the full projective synthesis that subsequent titles delivered.

Quake on the IBM PC in 1996 brought true three-dimensional polygon rendering with the full projective pipeline to the mass-market personal computer. The Quake engine established the visual template for the first-person shooter genre and for the consumer three-dimensional engines that the late 1990s saw.

Half-Life on Microsoft Windows in 1998 extended the Quake engine with scripted-sequence gameplay and a more cohesive single-player campaign that the polygon-based engine could deliver.

The subsequent decades produced an essentially uncountable list of three-dimensional games that the projective pipeline rendered. The article treats the modern era as the universal projective synthesis that subsumes all earlier projection modes.

Each game in the canon exercises the projective pipeline with different choices of $P$, $V$, and $M$. The differences lie in the gameplay mechanics, the art style, and the choice of delivery hardware appropriate to the target platform.

Out of Scope

The article does not cover the following.

The full graphics-processing-unit shader programming framework including the vertex and fragment shader stages, the texture-sampling rules, the depth and stencil buffer operations, and the blending and compositing rules is rendering-pipeline territory adjacent to but distinct from the projection math.

The non-Euclidean geometry that some experimental games employ including the hyperbolic geometry of HyperRogue in 2011 and the spherical geometry of Antichamber in 2013 is outside the standard projective framework that the article treats.

The animation pipeline including skeletal animation, morph targets, and procedural animation applies before the projective pipeline and is rendering-systems territory that the article does not cover.

The lighting and shading models including the Phong, Blinn-Phong, physically-based, and image-based rendering models operate after the projective pipeline on the rendered pixels and are outside the projection math.

The full ray-tracing framework that real-time ray tracing hardware of the late 2010s and 2020s enabled is an alternative rendering approach that bypasses the projective rasterisation pipeline and uses per-pixel ray-mesh intersection that the article does not treat.

The general history of the personal-computer and console industries across the late 1990s and 2000s is outside the article’s scope. The article treats the modern era through its projective-pipeline lens without surveying the broader industry history.

Conclusion

The projective synthesis gathers the projection-mode articles of the series into a single matrix-based framework that the modern graphics-processing-unit pipeline implements. The forward map applies three matrices to each world vertex,

followed by the perspective division that produces the normalised-device coordinate and the viewport transformation that produces the screen pixel. Each previous projection mode emerges as a specific choice of the projection matrix $P$, the view matrix $V$, and the model matrix $M$. Top-down, side-scrolling, decoupled-vertical-axis, belt-scroll, oblique, axonometric, parallax, Mode 7, sprite-scaling, raycasting, and stylised-hybrid projections all reduce to restricted cases of the projective framework. The draw-order, picking, and hit-test articles provide cross-cutting concerns that apply to any forward map that the framework produces. The series began with the floor-case top-down projection of a two-dimensional world and closes with the projective synthesis that subsumes all the modes the cluster articles introduced. The mathematics runs from the simplest affine map of the floor case to the full projective pipeline of modern three-dimensional games, and the journey illustrates how the camera as linear operator encodes everything from the simplest two-dimensional view to the rich three-dimensional rendering that contemporary games produce.

References

• Reference, Doom
• Reference, Half-Life
• Reference, Quake
• Reference, Star Fox

The second article of the cross-cutting cluster treats picking and hit testing across the projection modes that the previous clusters covered. Picking is the operation that converts a screen pixel into a world position or a gameplay object identifier. The screen pixel comes from a mouse click, a touchscreen tap, a light-gun trigger pull, or any other input event that names a single screen location. The engine applies the inverse of the forward projection to recover the world coordinates, then identifies which gameplay object the click corresponds to.

The previous articles introduced inverse maps for each projection mode and three disambiguation strategies for the cases where the inverse is underdetermined. The decoupled-vertical-axis article introduced the ground-plane assumption, the known-depth assumption, and the screen-space sprite hit test. The belt-scroll, oblique, and axonometric articles extended these to their respective projection modes. The draw-order article introduced the topmost-in-draw-order convention for resolving overlapping sprites. The article here gathers the full framework, treats the numerical stability of the inverse through the condition number, gives the area-scaling interpretation of the determinant, treats the canonical sprite-scale-and-rotate hit test that Battle Clash and Metal Combat used for the Super Scope light-gun gameplay, and discusses light-gun picking as the historical realisation of the abstract picking operation.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is the inverse-map equation that converts the screen pixel into a world position. The delivery mechanism chooses the input device that produces the screen pixel and the algorithm that identifies the gameplay object.

A Brief History of Picking

Picking in computer graphics dates to the 1960s as part of the interactive systems that the early graphics laboratories built. The video-game adoption arrives with the home and arcade systems that supported direct screen input.

The light-gun lineage begins with the Magnavox Odyssey light gun in 1972 which provided an analogue photodiode that fired when pointed at a bright cathode-ray-tube screen pixel. The technique became universal in arcade and console games through the late 1970s and 1980s. The NES Zapper shipped with the Nintendo Entertainment System in 1985 and powered Duck Hunt along with several subsequent titles through the standard photodiode mechanism.

Operation Wolf from Taito in 1987 brought light-gun shooting to the arcade with a mounted machine-gun controller that the player aimed at on-screen enemies. Operation Thunderbolt from Taito in 1988 followed with a similar mounted-gun cooperative gameplay format.

The Super Scope released as a separately-purchased accessory for the Super Nintendo Entertainment System in 1992 and powered the Super Scope shooter subgenre. Battle Clash from Nintendo and Intelligent Systems in 1992 brought the canonical sprite-scale-and-rotate hit-test case to the home console. Metal Combat, Falcon’s Revenge from Nintendo and Intelligent Systems in 1993 extended the format with additional enemy mecha and gameplay scenarios.

The arcade light-gun tradition continued through the 1990s with The House of the Dead from Sega in 1996 which combined three-dimensional polygon rendering with light-gun targeting in the survival-horror format.

The mouse-driven picking tradition runs in parallel with point-and-click adventure games of the mid-1980s that the article treats as a parallel lineage to the light-gun games. The modern era combines mouse, touchscreen, and motion-controller input through a common abstract picking framework that the article describes.

The Inverse Map Framework

The forward map of a projection mode takes a world coordinate $\mathbf{p}{\text{world}}$ to a screen coordinate $\mathbf{p}{\text{screen}}$. The inverse map takes a screen coordinate to a world coordinate or to a candidate set when the inverse is underdetermined,

The candidate set has a single element when the forward map is a bijection on the relevant world region, as in the floor-case top-down and the side-scrolling modes. The candidate set has multiple elements when the inverse is underdetermined, as in the decoupled-vertical-axis, belt-scroll, oblique, and axonometric modes.

The three disambiguation strategies that the previous articles introduced reduce the candidate set to a single element through a contextual assumption.

The ground-plane assumption sets the vertical world coordinate to the ground level $w_y = w_y^{\text{ground}}$ and recovers the remaining coordinates through the projection mode’s inverse,

The convention is appropriate for movement commands where the click identifies a ground destination.

The known-depth assumption sets the depth coordinate to a value $w_z^{\text{known}}$ that the gameplay context provides and recovers the remaining coordinates,

The convention is appropriate when the engine knows the gameplay-relevant depth through the object list and the object’s current state.

The screen-space sprite hit test ignores the world coordinates entirely and tests the clicked screen pixel against the screen-space bounding rectangles of every visible sprite,

where $B_i^{\text{screen}}$ is the screen-space bounding box of sprite $i$. The convention is appropriate for direct sprite selection including light-gun targeting and cursor-driven gameplay.

The topmost-in-draw-order convention of the draw-order article extends the screen-space hit test by iterating the visible sprites in front-to-back order,

where $\text{depth}_i$ is the depth from camera of sprite $i$. The closest sprite that the click hits is returned.

For piecewise projections of the stylised-hybrid article, the inverse map is itself piecewise. The engine first identifies which region or game state the click belongs to based on the screen position or on the current game state, then applies the corresponding region’s inverse.

The full picking framework combines these elements into a per-click algorithm that the engine executes on each input event. The algorithm identifies the projection mode active in the click region, applies the appropriate inverse map, runs the appropriate disambiguation strategy, and returns the world position or the gameplay object identifier.

Condition Number and Numerical Stability

The inverse map for projection modes where the matrix is two-by-two and invertible involves solving a linear system $A \mathbf{x} = \mathbf{b}$ for $\mathbf{x}$ given the matrix $A$ and the right-hand side $\mathbf{b}$. The numerical stability of the solution depends on the condition number of $A$.

The condition number is the ratio of the largest singular value to the smallest,

An equivalent expression uses the operator two-norm,

The condition number bounds how input perturbations propagate to output perturbations. If $\mathbf{b}$ is perturbed by $\Delta \mathbf{b}$, the resulting perturbation $\Delta \mathbf{x}$ in the solution satisfies

A condition number near $1$ indicates a well-conditioned inverse where small input perturbations produce small output perturbations. A large condition number indicates an ill-conditioned inverse where small input perturbations produce large output perturbations and the inverse becomes numerically unreliable.

The condition number values for the projection modes of the previous clusters are typically small. The sprite scale-and-rotate matrix $A = s\, R(\theta)$ has $\kappa(A) = 1$ because rotation and uniform scaling preserve angles. A rotation alone or a uniform scale alone has $\kappa = 1$.

The game-iso ground-plane matrix

has singular values $z\sqrt{2}$ and $z/\sqrt{2}$ with $\kappa(A_{\text{game-iso}}) = 2$ independent of the zoom factor.

The belt-scroll ground-plane matrix with depth-mixing slope $\beta$ has

singular values $z$ and $\beta z$, and condition number $\kappa = 1/\beta$. For typical $\beta = 1/2$, $\kappa = 2$.

The Mode 7 per-scanline matrix $M(s_y) = z(s_y) R(\theta)$ has $\kappa(M(s_y)) = 1$ at every scanline because rotation and uniform scaling remain conformal under the per-scanline scaling factor.

The two-dimensional projection modes of the series all produce well-conditioned inverses with $\kappa$ on the order of unity. Numerical issues do not arise in standard gameplay because the floating-point precision of modern engines is far below the condition-number threshold that would degrade the inverse.

The Determinant as Area Scale

The determinant of the forward map’s two-by-two matrix gives the screen-space area that a unit world-space rectangle covers under the projection. For a forward map $\mathbf{p}{\text{screen}} = A \mathbf{p}{\text{world}} + \mathbf{o}$, the screen-space area $A_{\text{screen}}$ covered by a world-space rectangle of area $A_{\text{world}}$ is

The sprite scale-and-rotate matrix has $\det(s R(\theta)) = s^2$ because $\det(R) = 1$ and the scaling factor enters quadratically. A sprite at twice the distance from the camera covers a quarter of the screen area of the same sprite at unit distance, matching the quadratic area law of the sprite-scaling article.

The game-iso ground-plane matrix has $\det(A_{\text{game-iso}}) = z^2 (1 \cdot 1/2 - (-1)(1/2)) = z^2$, so a unit ground-tile area covers $z^2$ screen pixels. For $z = 32$, each unit tile covers $32^2 = 1024$ screen pixels, matching the diamond tile area of game-iso forward map.

The Mode 7 per-scanline matrix has $\det(M(s_y)) = z(s_y)^2$, which varies per scanline. The variation produces the apparent shrinkage of texture content at distant scanlines and the apparent magnification of texture content at near scanlines. The article on Mode 7 treats the per-scanline scaling in detail.

The belt-scroll ground-plane matrix has $\det(A_{\text{belt-scroll}}) = -\beta z^2$ with $|\det| = \beta z^2$. A unit ground-tile area covers $\beta z^2$ screen pixels, which is smaller than the top-down case by the factor $\beta$. The reduction reflects the foreshortening of the depth axis in the belt-scroll projection.

The determinant also appears in the picking density. The world-space area sampled by a screen-space area $\Delta A_{\text{screen}}$ is the inverse of the area-scaling relationship,

A larger determinant gives finer picking precision through smaller world-space area per screen pixel at the cost of larger screen-space coverage per world unit.

A vanishing determinant indicates that the forward map is singular and the inverse does not exist. The picking ray then samples a one-dimensional curve in the world space rather than a two-dimensional area. The case arises at extreme camera tilt or at degenerate viewing angles where the projection collapses one of the world axes onto the screen plane.

Sprite Scale-and-Rotate Hit Test

The canonical sprite-scale-and-rotate hit test of the Battle Clash and Metal Combat Super Scope shooters inverts the sprite-scaling forward map of the sprite-scaling article to recover the sprite-local pixel position from the screen click.

The forward map for a sprite at screen anchor position $\mathbf{c}$ with scaling factor $s$ and orientation angle $\theta$ maps a sprite-local pixel $\mathbf{p}{\text{local}}$ to a screen pixel $\mathbf{p}{\text{screen}} = \mathbf{c} + s R(\theta) \mathbf{p}_{\text{local}}$.

The inverse map recovers the sprite-local position from the screen click,

The hit test then checks the sprite-local position against the sprite’s local bounding rectangle,

A refined hit test samples the sprite’s alpha value at the local pixel position to distinguish opaque from transparent sprite regions,

where $\alpha(\mathbf{p}{\text{local}})$ is the sprite’s alpha at the rounded local pixel coordinates and $\alpha{\text{threshold}}$ is the engine’s transparency threshold. The refined test rejects clicks on the sprite’s transparent border and accepts only clicks on the sprite’s drawn content.

For Battle Clash specifically, the giant enemy mecha renders through Mode 7 background hardware treated as a single rotating-scaling sprite. The cartridge software maintains the current Mode 7 matrix entries that determine the scale and rotation of the mecha sprite. The Super Scope hit reads as a screen pixel, and the cartridge software inverts the Mode 7 matrix to recover the sprite-local position on the mecha sprite. The sprite-local position identifies which part of the mecha was hit, which drives the damage modelling and visual effects.

When multiple sprites are visible, the hit test iterates the sprites in front-to-back depth order through the topmost-in-draw-order convention of the previous article. The first sprite whose inverse-mapped click pixel falls within the sprite’s local rectangle and on an opaque pixel is returned. Sprites behind the hit sprite are not tested further.

Light-Gun Picking

The light-gun input device makes the picking operation literal. The player physically aims a gun-shaped controller at the cathode-ray-tube display and pulls the trigger. A photodiode in the gun fires when the cathode ray sweeps across the gun’s pointed pixel.

The console synchronises the photodiode reading with the cathode-ray-tube raster scan to determine which screen pixel the gun was pointed at when the trigger fired. The standard implementation uses a two-frame approach.

In the first frame, the console draws the entire scene normally. The player aims the gun and pulls the trigger.

In the second frame, the console flashes a bright field at each potential hit target’s screen position. The photodiode in the gun fires when the gun is pointed at one of these bright fields, and the console records the timing of the photodiode pulse.

The pulse timing identifies which screen scanline and which horizontal pixel the gun was pointed at, because the cathode-ray-tube raster sweeps the screen in a known pattern. The console maps the timing to the screen pixel coordinates $(s_x, s_y)$.

The screen pixel coordinates then enter the inverse map framework to identify the world position or gameplay object. For Battle Clash, the inverse map applies the Mode 7 sprite-scale-and-rotate hit test to determine which part of the mecha was hit. For Duck Hunt, the inverse map applies the screen-space hit test to determine which duck was hit.

The light-gun framework is a special case of the general picking framework where the input device delivers the screen pixel through hardware rather than through software cursor tracking. The downstream inverse map and hit test follow the same algorithm as cursor-driven or touchscreen-driven picking.

Modern flat-panel displays do not support the cathode-ray-tube light gun because the display update mechanism differs. Modern light-gun-style games use camera-based or accelerometer-based input that the article treats as the modern equivalent of the cathode-ray-tube light gun.

A Worked Example

Consider a Battle Clash-style Super Scope shooter with the following parameters. The screen is 256 pixels wide and 224 pixels tall. The mecha boss sprite is 64 pixels wide and 64 pixels tall in sprite-local coordinates with extent $|u| \leq 32$, $|v| \leq 32$. The sprite’s current screen anchor is $\mathbf{c} = (160, 140)$. The sprite’s current scale factor is $s = 2$. The sprite’s current orientation is $\theta = \pi/6$ matching 30 degrees counterclockwise rotation.

The Super Scope reports a hit at screen pixel $\mathbf{p}_{\text{screen}} = (200, 150)$. The cartridge software computes the sprite-local hit position through the inverse forward map.

The screen-space offset from the sprite anchor is

The inverse rotation by $-\pi/6$ uses $\cos(-\pi/6) = \sqrt{3}/2 \approx 0.866$ and $\sin(-\pi/6) = -1/2$. The rotation matrix is

Applying the rotation to the screen-space offset gives

Dividing by $s = 2$ gives the sprite-local hit position,

The bounding-box test verifies $|19.8| \leq 32$ and $|-5.7| \leq 32$. Both conditions hold, so the click hits the sprite.

The determinant of the forward map gives $\det(s R(\theta)) = 4$, so a unit area in sprite-local coordinates covers four screen pixels. The sprite’s screen area is $64 \cdot 64 \cdot 4 = 16{,}384$ screen pixels.

The condition number of the forward map is $\kappa(s R(\theta)) = 1$ because rotation and uniform scaling preserve angles. The inverse is well-conditioned and the hit-position computation is numerically reliable.

A refined alpha test samples the sprite’s alpha value at sprite-local pixel coordinates $(20, -6)$ after rounding the inverse result to the nearest integer. The sample returns the alpha value from the sprite atlas at that pixel. If the value exceeds the transparency threshold, the hit registers as opaque and the engine reports a hit on the mecha.

The round-trip identity holds within the sprite,

where $\varepsilon$ is the floating-point precision of the engine.

A second sprite at the same screen anchor but with scale $s = 1$ and zero rotation would have its own inverse map applied to the same screen click. The smaller sprite covers less screen area and the click at $(200, 150)$ at offset $(40, 10)$ from the anchor falls outside its local extent $|u| \leq 32$. The smaller sprite does not register a hit.

When both sprites are visible, the topmost-in-draw-order convention returns the closer sprite that the click hits. The larger mecha sprite at scale $s = 2$ returns as the hit.

Variations Within the Framework

The picking framework admits several variations that engines have explored.

A cursor-driven variant uses a mouse pointer or touchscreen tap to deliver the screen pixel to the picking algorithm. The variant applies the same inverse map and hit test as the light-gun variant but with a software-tracked screen pixel rather than a hardware-detected one.

A camera-based variant uses an image sensor and a calibration light source to track the controller’s pointing direction. The Wii Remote pairs a camera in the controller with fixed infrared emitters on a sensor bar near the display. The PlayStation Move pairs a camera near the display with an illuminated orb on the controller. The variant translates the controller’s pointed position to a screen pixel through a calibrated camera-to-screen transform.

A gaze-tracking variant uses an eye-tracking camera to determine where the player is looking on the screen. The pointed position becomes the screen pixel that the picking algorithm processes.

A motion-control variant combines accelerometer and gyroscope data to determine the controller’s orientation relative to a fixed forward direction. The variant projects the controller’s orientation onto the screen to produce the screen pixel.

A pixel-level alpha-tested hit samples the rendered alpha value at the click position to distinguish opaque from transparent sprite regions. The variant gives more precise hit detection at the cost of additional texture-sampling work.

A bounding-volume-hierarchy variant organises the visible sprites into a spatial data structure that accelerates the front-to-back iteration. The variant trades implementation complexity for picking-iteration cost in scenes with many sprites.

A multi-touch variant processes multiple simultaneous touch points through the same picking framework applied per touch. The variant supports touchscreen gameplay with multi-finger gestures.

Delivery Mechanisms

The picking framework permits five distinct delivery mechanisms on period hardware.

The first is the cathode-ray-tube light gun with hardware photodiode and raster-timing detection. The NES Zapper, the Super Scope, and the arcade light-gun controllers all use this mechanism on cathode-ray-tube displays.

The second is mouse-driven cursor picking on the personal computer. The mouse driver reports cursor screen positions that the gameplay engine applies to the picking algorithm. The mechanism is the universal cursor-picking delivery on the IBM PC and on subsequent personal computers.

The third is touchscreen tap picking on handheld devices including the Nintendo DS in 2004 and modern smartphones and tablets. The touchscreen driver reports tap positions that the gameplay engine processes.

The fourth is camera-and-controller-based input through image sensors and calibration light sources that the system uses to track the controller’s pointing. The Wii Remote and the PlayStation Move both use this mechanism with different placements of camera and light source.

The fifth is graphics-processing-unit shader picking on modern hardware where a fragment shader records the object identifier at each screen pixel into a separate buffer that the engine reads to identify the picked object. The mechanism removes the per-object iteration in favour of per-pixel storage.

All five mechanisms deliver a screen pixel to the inverse-map framework. The choice trades hardware availability, input-device cost, implementation complexity, and the achievable precision.

Where the Framing Breaks Down

The picking framework is insufficient when any of the following conditions hold.

When the click falls on a sprite whose visible pixels are partially transparent and the engine cannot afford the pixel-level alpha test, the bounding-box hit may return a false positive. The variant must either restrict the hit detection to clearly-opaque sprite content or accept the false positives as a design trade.

When multiple sprites overlap at the click position with rapid temporal motion, the picking decision may flip between adjacent frames based on the click timing. The variant must either lock the picking to the gameplay tick rate or accept the timing-dependent ambiguity.

When the projection mode applies non-affine transformations beyond the affine inverse that the framework supports, the picking framework must extend to per-pixel ray casting that the article does not treat in detail.

When the picking must respect the depth of the gameplay world at sub-pixel precision, the floating-point precision of the inverse may become a binding constraint. The condition number bounds of the previous section quantify the precision limit.

When the input device delivers screen positions that are themselves imprecise through analogue noise or calibration drift, the picking precision is limited by the input device rather than by the inverse map. The variant must include input-device error modelling that the article does not treat.

The Canon

The following games use specific picking techniques that distinguish their gameplay or that became canonical for the subgenre.

Operation Wolf in the arcade in 1987 gave the medium its canonical mounted-gun light-gun shooter through the Taito arcade hardware and the integrated gun controller.

Operation Thunderbolt in the arcade in 1988 extended the format with cooperative two-player gameplay.

Battle Clash on the Super Nintendo Entertainment System in 1992 brought the canonical sprite-scale-and-rotate hit-test case to the home console through the Super Scope light gun and the Mode 7 background-as-sprite rendering. The hit-detection cartridge code is the canonical implementation of the inverse map framework in the engineering tradition.

Metal Combat, Falcon’s Revenge on the same console in 1993 extended the Battle Clash formula with additional enemy types and gameplay scenarios.

The House of the Dead in the arcade in 1996 combined three-dimensional polygon rendering with light-gun targeting in the survival-horror format. The hit-detection algorithm applied per-polygon ray-mesh intersection that the modern three-dimensional picking framework subsumes.

Each game in the canon exercises a specific picking technique that the projection mode and the gameplay together impose on the engine.

Out of Scope

The article does not cover the following.

The full three-dimensional picking framework through ray-mesh intersection on arbitrary polygon meshes is the subject of the synthesis closer of the series. The article presents picking in its two-dimensional and pseudo-three-dimensional forms that the previous clusters covered.

The full input-device-error modelling that accounts for analogue noise, calibration drift, and player-skill variability is gameplay-systems territory that the article does not treat.

The shader programming techniques for graphics-processing-unit shader picking including the object-identifier buffer and the per-pixel readback are implementation concerns adjacent to but distinct from the projection math.

The full alpha-test framework including premultiplied alpha, alpha-to-coverage, and order-independent transparency is rendering territory that the article treats only at the level of the sprite-local alpha sampling.

The hardware design of the cathode-ray-tube light gun including the photodiode characteristics and the raster-timing synchronisation circuits is hardware-design territory that the article treats only at the level of the delivery-mechanism sidebar.

Conclusion

The picking and hit-testing framework gathers the inverse-map material of the previous projection-mode articles into a single cross-cutting treatment. The framework distinguishes the three disambiguation strategies that the inverse map admits, treats the numerical stability of the inverse through the condition number $\kappa(A)$, gives the area-scaling interpretation of the determinant $|\det(A)|$, treats the canonical sprite-scale-and-rotate hit test $\mathbf{p}{\text{local}} = (1/s) R(-\theta) (\mathbf{p}{\text{screen}} - \mathbf{c})$ that Battle Clash and Metal Combat used for the Super Scope light-gun gameplay, and discusses light-gun picking as the historical realisation of the abstract picking operation. The remaining article in the series treats the camera as a linear operator in the synthesis closer that subsumes the two-dimensional projection modes of the previous clusters into the projective generalisation of the three-dimensional rendering pipeline.

References

• Reference, Battle Clash
• Reference, Metal Combat, Falcon’s Revenge
• Reference, NES Zapper
• Reference, Operation Thunderbolt
• Reference, Operation Wolf
• Reference, The House of the Dead

The first article of the cross-cutting cluster treats draw order management across the projection modes that the previous clusters covered. Draw order matters whenever two world objects project to overlapping screen regions. The renderer must decide which object appears in front and which appears behind. A wrong decision produces a wall in front of the character when the character should be in front of the wall, a treetop behind a building when the building should occlude the tree, or a flickering ordering that flips between adjacent frames.

The article treats four related techniques. The painter’s algorithm provides the general framework of back-to-front sorting that every two-dimensional renderer applies. Y-sort uses the screen-y position of each object’s anchor as the sort key. Z-sort uses the world depth coordinate as the sort key. The hybrid Y-then-Z sort combines the two for projection modes where neither alone produces a fully-correct order.

The previous articles in the series introduced Y-sort in the belt-scroll, oblique, and axonometric articles through the operational comparison on the ground-projection screen-y. The article here treats the full framework, including tie-breaking rules, the Z-sort alternative, floating-point conditioning, and the hybrid sort that production engines use. The pass-through bug that classic top-down role-playing games sometimes exhibit is the article’s running case study of what goes wrong when Y-sort is misapplied.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a sort criterion that the engine applies to a list of renderable objects. The delivery mechanism chooses the sort algorithm and the data structure that the engine uses to maintain the sorted order across frames.

A Brief History of Draw Order

The painter’s algorithm appears in computer graphics literature in Newell, Newell, and Sancha’s 1972 paper on hidden-surface removal for three-dimensional polygon rendering. The technique sorts polygons by their depth from the camera and renders them in back-to-front order, relying on the later-rendered polygons to overwrite the earlier-rendered ones in the frame buffer.

The two-dimensional game adoption of the painter’s algorithm emerges with the multi-sprite scenes of the late 1970s and early 1980s arcade games. The Nintendo Entertainment System sprite hardware maintained a 64-entry object attribute memory that the engine wrote in the desired draw order, with lower-index sprites rendered in front of higher-index ones. Console sprite hardware of the era provided per-sprite priority bits or layer fields that the engine wrote per frame to control the visual front-to-back layering.

The Y-sort technique emerges with the isometric and oblique-projected games of the late 1980s and the 1990s. Knight Lore in 1984 used Y-sort for the small isometric rooms that the player navigated. The Y-sort convention became universal in axonometric role-playing games through the 1990s including Diablo II from Blizzard in 2000 which is widely cited as the canonical Y-sort case in the game-development tradition.

The Z-sort alternative emerges in the same era as the natural choice for games that track an explicit world depth coordinate for each object. Tactical role-playing games with discrete height tiers including Disgaea, Hour of Darkness from Nippon Ichi Software in 2003 use Z-sort to layer characters and obstacles across the multi-level battlefield.

The hybrid Y-then-Z sort emerges in production engines that need to handle both ground-aligned objects and stacked-height objects in the same scene. The combination catches cases that either sort alone fails to resolve correctly.

The pass-through bug that classic top-down role-playing and adventure games sometimes exhibit is documented in many period titles where an NPC sprite appears to overlap the wrong side of a wall sprite due to a sort-key misconfiguration. The bug is the canonical Y-sort failure case in the engineering tradition.

The modern era uses graphics-processing-unit depth buffers that displace the painter’s algorithm for true three-dimensional rendering. The two-dimensional game continues to use Y-sort and its variants because the depth buffer requires per-pixel depth values that two-dimensional sprites do not naturally provide.

The Painter’s Algorithm

The painter’s algorithm sorts a list of renderable objects by a per-object sort key $k_i$ and renders them in decreasing order of $k_i$. The first-rendered object sits at the back of the scene and the last-rendered object sits at the front. Each subsequent render overwrites the frame buffer at the pixels where the new object covers existing content, giving the last-rendered objects visual priority over the earlier ones.

The sort criterion is the comparison

The sort key $k_i$ encodes the back-to-front depth that the projection mode produces. For top-down projection, the natural sort key is the world-y coordinate or equivalently the screen-y of the ground anchor. For belt-scroll, oblique, and axonometric projection, the sort key is the ground-projection screen-y that the cluster articles introduced. For sprite scaling, the sort key is the world depth from the camera.

The algorithmic complexity is dominated by the sort. A comparison-based sort runs in $O(n \log n)$ time for $n$ visible objects. The rendering pass runs in $O(n)$ time plus the per-pixel cost of drawing each sprite. The total per-frame cost is

where $A_i$ is the screen-space area of object $i$.

The painter’s algorithm assumes that the sort key is well-defined for each object and that the sorted order produces a correct visible result. The assumption fails when two objects’ sort keys are equal or when the sort key does not correctly capture the depth relationship between objects with overlapping screen extents. The tie-breaking and conditioning rules of the following sections address these failures.

The Y-Sort Criterion

The Y-sort criterion uses each object’s anchor screen-y as the primary sort key,

The comparison selects the smaller anchor screen-y for earlier rendering,

For ground-aligned objects in any of the projection modes of the previous clusters, the smaller anchor screen-y corresponds to a position further back in the world from the camera. The Y-sort criterion produces the correct back-to-front rendering order.

The anchor convention matters for Y-sort correctness. The canonical convention places the anchor at the object’s foot or at the bottom-centre of the sprite, which is the world position where the object touches the ground. Two characters whose feet are at different world positions sort correctly by foot-anchor screen-y.

When two objects have equal anchor screen-y, the primary Y-sort criterion is ambiguous. The engine selects a tie-breaker from several common choices.

The first tie-breaker uses the world depth coordinate $w_z$ of each object,

The deeper object draws first. The Y-then-Z hybrid sort below formalises this tie-breaker.

The second tie-breaker uses the object’s creation order through a stable sort algorithm,

The older object draws first. The stable-sort approach prevents flicker between frames because the order does not change unless the gameplay state changes.

The third tie-breaker uses a unique object identifier,

The identifier provides a deterministic order that does not depend on gameplay state. The convention is rare in practice because the order does not correspond to any meaningful spatial relationship.

The pass-through bug arises when the Y-sort anchor convention does not match the visual occlusion expectation. A wall sprite anchored at its base and a character sprite anchored at its feet sort correctly when the character is in front of the wall. The bug appears when the character’s feet anchor screen-y exceeds the wall’s base anchor screen-y by a small amount that does not match the visual depth relationship. The character renders on top of the wall because of the Y-sort criterion, giving the visual impression of walking through the wall. The fix either changes the wall’s anchor convention or adds a secondary sort key that distinguishes the wall from the character explicitly.

The Z-Sort Criterion

The Z-sort criterion uses the world depth coordinate as the sort key directly,

The comparison selects the larger world depth for earlier rendering,

The criterion is more direct than Y-sort because the world depth does not depend on the projection. A change of projection mode does not change the world depth coordinate of the object.

The criterion requires that the engine track each object’s world depth in addition to its world position. For projection modes where the world depth is one of the world coordinates, the requirement is automatic. For projection modes where the depth is computed from the world coordinates through the projection matrix, the engine computes the depth value once per object per frame.

Floating-point conditioning becomes a concern when two objects have nearly equal world depths. The comparison

can flip between adjacent frames if the depth difference is smaller than the floating-point precision $\varepsilon$,

The flip produces visible flicker where the rendered order changes between frames even though the world configuration is approximately stable.

The conditioning fix uses an epsilon-tolerance comparison that treats nearly-equal depths as tied and falls back to a secondary criterion,

A typical secondary criterion uses the object identifier or the creation order to break the near-tie deterministically. A quantised-depth alternative rounds the world depth values to a fixed-point grid that eliminates the near-tie cases entirely,

where $\Delta$ is the quantisation step. The quantisation introduces a maximum sort-key error of $\Delta/2$ in exchange for stable cross-frame ordering.

The Hybrid Y-Then-Z Sort

The hybrid Y-then-Z sort combines the Y-sort and Z-sort criteria through a case-based comparison that uses the anchor screen-y as the primary key in increasing order and the world depth as the secondary key in decreasing order,

The tertiary criterion typically uses object identifier or creation order to break the doubly-tied case.

The hybrid sort handles both the canonical Y-sort cases where objects are spread across the ground plane and the stacked-height cases where objects share the same ground anchor but have different vertical positions. The hybrid is the production-engine default in two-dimensional games that combine ground characters with airborne projectiles or stacked vertical structures.

The Disgaea series uses a variant of the hybrid sort where the primary key is the Z-axis layer that the tactical battlefield assigns and the secondary key is the Y position within the layer. The variant inverts the Y-then-Z order to layer-by-layer rendering that matches the discrete-height tactical map.

A Worked Example

Consider a Diablo-II-style axonometric action role-playing game with the following parameters. The screen is 800 pixels wide and 600 pixels tall. The forward map uses game-iso with zoom factor $z = 32$ and screen offset $\mathbf{o} = (400, 300)$. The camera position is $\mathbf{c} = (0, 0, 0)$. The forward-map matrix is

Three world objects populate the scene on the ground plane at $w_y = 0$. The first is a tree at world position $(5, 0, 3)$. The second is a sign at world position $(4, 0, 4)$. The third is the player character at world position $(6, 0, 5)$.

The anchor screen positions follow from applying $M$ to each world position.

The tree projects to

The sign projects to

The player projects to

The Y-sort criterion applied to the three anchor screen-y values gives

The tree and the sign tie at $s_y = 428$. The player has $s_y = 476$ and renders after both tied objects.

The tie-break on world depth $w_z$ gives $w_z^{\text{tree}} = 3$ and $w_z^{\text{sign}} = 4$. The sign has the larger depth and renders before the tree under the Z-sort criterion that draws larger-depth objects first.

The hybrid Y-then-Z sort produces the draw order

The sign renders first at the back. The tree renders second over any sign pixels it overlaps. The player renders last at the front on top of any sign or tree pixels it overlaps.

A flying projectile above the player at world position $(6, -2, 5)$ shares the player’s lateral and depth coordinates but has $w_y = -2$ indicating a position two world tiles above the ground. The projectile’s anchor screen position is

The projectile renders at screen $(432, 412)$, 64 pixels above the player’s screen anchor. The Y-sort criterion produces the projectile’s anchor screen-y as 412, smaller than the player’s 476. The projectile would render before the player and appear behind it under the Y-sort criterion alone.

The hybrid Y-then-Z sort fixes this case by using the player’s shadow screen-y or by including the world height $w_y$ in the sort key. A Y-sort variant that uses the shadow screen-y sets the projectile’s effective Y-sort key to the screen-y of its ground projection,

The projectile’s shadow Y-sort key matches the player’s anchor Y-sort key, and the Z-sort tie-breaker distinguishes them through the world $w_y$ comparison.

The round-trip identity of the sort is that the sort order is stable across frames when the world configuration is stable,

where $\sigma_n$ is the sorted order at frame $n$. The stable-sort guarantee prevents cross-frame flicker when the gameplay state has not changed the sort keys between adjacent frames.

Variations Within the Mode

The draw-order framework admits several variations that engines have explored.

A static-priority variant assigns each object a fixed priority value that does not change with the object’s world position. Background objects use low priority and foreground objects use high priority. The variant is mathematically the simplest and works well for games where each object has a clear front-or-back role that the gameplay does not change.

A layered-sort variant divides objects into discrete layers and sorts within each layer independently. The layers render bottom-to-top. The criterion is a case-based comparison on the layer index $\ell_i$ as the primary key and the within-layer criterion as the secondary key,

The within-layer criterion is typically Y-sort, Z-sort, or the hybrid Y-then-Z sort. The variant is common in tile-based games where background tiles, mid-ground objects, sprites, and overlays each form a layer. The layers do not interact across boundaries.

A camera-projected-depth variant uses the camera-frame depth of each object as the sort key,

where $R$ is the camera rotation matrix. The variant generalises Z-sort to projection modes where the world depth axis is not directly $w_z$.

A bounding-box-overlap-resolving variant detects pairs of objects whose screen-space bounding boxes overlap and applies the sort criterion only to overlapping pairs. The variant reduces the cost of unnecessary sort comparisons in scenes with many widely-separated objects.

A topological-sort variant allows the engine to declare explicit ordering relationships between specific object pairs. The engine performs a topological sort of the resulting directed acyclic graph. The variant handles cases where neither Y-sort, Z-sort, nor the hybrid captures the correct order.

A hand-authored-order variant permits the level designer to manually specify the draw order for each scene. The variant is common in cinematic scenes where the designer’s artistic intent overrides any algorithmic sort.

A depth-buffer alternative replaces the per-object sort with per-pixel depth comparison. The graphics processing unit maintains a depth buffer that stores the closest depth value at each screen pixel,

The depth buffer removes the need for a per-object sort and handles per-pixel occlusion correctly including the cases where two objects’ screen rectangles intersect. Modern two-and-three-dimensional engines use the depth buffer when graphics processing unit support permits.

Delivery Mechanisms

The draw-order management permits five distinct delivery mechanisms on period hardware.

The first is hardware sprite priority bits on console picture-processing-unit hardware. The Nintendo Entertainment System sprite list included a per-sprite priority bit that placed each sprite in front of or behind the background. The Super Nintendo Entertainment System extended this to four priority levels. The Sega Genesis sprite list used per-sprite link fields that determined the rendering order. The mechanism provided hardware-supported draw order at the cost of limited sprite count and limited priority levels.

The second is per-frame engine-software sort on a general-purpose central processing unit. The engine maintains a list of visible objects and sorts them once per frame through a software sort algorithm that the engine chooses. The mechanism scales with the central-processing-unit budget and is the dominant delivery on the personal computer and on modern console hardware that exposes a general-purpose programming model.

The third is incremental sort maintenance through a stable insertion-sorted list that the engine updates as objects move. The variant exploits the temporal coherence of object positions across adjacent frames. The sort cost amortises to $O(n)$ per frame when most objects do not change relative position. The variant is rare in retro games but common in modern engines.

The fourth is graphics-processing-unit shader sort through compute shaders that perform parallel sort algorithms across the visible object list. The variant trades programming complexity for the parallel throughput of modern graphics processing units. The variant is dominant in modern engines that use shader-based pipelines.

The fifth is depth-buffer-based per-pixel resolution that the previous section described. The depth buffer displaces the sort for projection modes where per-object sorting is insufficient.

All five mechanisms implement back-to-front rendering with different trade-offs. The choice trades hardware requirements, implementation complexity, the maximum supported object count, the depth-buffer memory budget, and the achievable frame rate.

Where the Framing Breaks Down

The painter’s algorithm and its sort criteria are insufficient when any of the following conditions hold.

When the visible scene contains two objects whose screen rectangles intersect such that each contains some pixels in front of the other, no per-object sort can produce a correct rendering. The classic case is two crossing rectangles in three dimensions where one half is in front of the other and the other half is behind. The depth-buffer alternative resolves the case correctly.

When the scene contains transparent objects that should partially blend with the background, the painter’s algorithm must render the transparent objects after the opaque background in a separate pass. The depth-buffer alternative handles the case through careful sort and blending rules that the article does not treat in detail.

When the scene contains shadows that two objects cast on each other, the sort criterion must respect both the object’s depth from the camera and the shadow’s projection onto the receiving surface. Period games typically simplified the shadow rendering to avoid the complexity.

When the camera moves rapidly between frames, the temporal coherence that incremental sort maintenance relies on breaks down. The engine must fall back to a full sort that the variant cost saves only during periods of camera stability.

When the gameplay introduces new sort-affecting objects mid-frame, the sort must update before the rendering pass to include the new objects. The engine must handle the synchronisation between gameplay state updates and the sort.

The Canon

The following games use specific draw-order techniques that distinguish their visual style or that became canonical for the subgenre.

Diablo II on Microsoft Windows in 2000 uses Y-sort as the canonical axonometric action-role-playing-game convention. The Diablo II sprite stack includes the player character, multiple monster sprites, projectile effects, and environmental objects all sorted by ground-anchor screen-y per frame.

Disgaea, Hour of Darkness on the PlayStation 2 in 2003 uses Z-sort on the discrete-height tactical battlefield where characters and obstacles occupy explicit elevation tiers. The discrete-Z layered sort is a special case of the Z-sort criterion that suits the tactical-grid gameplay.

Classic top-down role-playing and adventure games of the Famicom and Super Nintendo Entertainment System era exhibited the pass-through bug at corner cases of the Y-sort criterion. The bugs are the canonical illustrations of what goes wrong when the anchor convention or the tie-breaker is insufficient.

Each game in the canon exercises a specific draw-order technique and reveals the trade-offs that the projection mode and the gameplay requirements impose on the engine.

Out of Scope

The article does not cover the following.

The full painter’s algorithm for three-dimensional polygon rendering including the polygon-clipping cases that the original Newell paper introduced is outside the article’s two-dimensional scope. The graphics-processing-unit depth-buffer displaces the polygon-painter’s algorithm on modern hardware.

The full pick-disambiguation framework that builds on the draw order to identify the clicked object is the subject of the next cross-cutting article. The article presents draw order without the corresponding picking treatment.

True three-dimensional rendering with arbitrary camera placement and per-pixel depth resolution is the subject of the synthesis closer of the series. The painter’s algorithm is the two-dimensional approximation that pre-Z-buffer hardware required.

The internal data structures that production engines use for incremental sort maintenance are implementation details that the article treats only at the level of the delivery-mechanism sidebar.

The shader programming techniques for parallel sort on the graphics processing unit are implementation details adjacent to but distinct from the sort criterion math.

Conclusion

Draw order management is a cross-cutting concern that the projection-mode articles each touched briefly through their Y-sort sections. The article gathers the full framework here. The painter’s algorithm sorts a list of renderable objects by a depth-related sort key and renders them in back-to-front order. Y-sort uses the anchor screen-y as the sort key. Z-sort uses the world depth coordinate. The hybrid Y-then-Z sort combines the two through a lexicographic comparison that handles both ground-aligned and stacked-height cases. Floating-point conditioning prevents cross-frame flicker through epsilon tolerance or depth quantisation. The depth-buffer alternative displaces the per-object sort on hardware that supports per-pixel depth resolution. Production engines typically use the hybrid sort for retro-style two-dimensional games and the depth buffer for three-dimensional and two-and-a-half-dimensional games. The next article in the series treats the cross-cutting picking framework that uses the draw-order structure to identify the clicked object.

References

• Reference, Diablo II
• Reference, Disgaea, Hour of Darkness
• Reference, Painter’s Algorithm
• Reference, Z-Buffering

The closing article of the affine-and-projective cluster treats stylised and hybrid projections that combine multiple projection modes within a single game. A game that uses a piecewise projection applies one projection mode in one part of the world or one game state and a different projection mode in another part. The result is a visual inconsistency that the player accepts as a stylistic convention even though the rendered scene does not represent a single physical camera view.

The previous articles in the series treated each projection mode in isolation as if a game used only that mode. Real games often combine modes. A role-playing game may render the overworld in a top-down view, the towns in a top-down view at a higher zoom, the battle scene in a side view, and the menus as a flat overlay. A platformer may use a side-scrolling view for gameplay and a top-down view for the world map. A stylised independent release may combine a side-scrolling projection with atmospheric post-processing that the projection math alone does not produce.

The article frames piecewise projection as a forward map that depends on the world region or on the game state. The Mother hybrid where the ground renders in top-down view and buildings render in side elevation within the same scene is the canonical example of intra-scene inconsistency. The Final Fantasy lineage where the overworld, town, and battle each render through their own projection is the canonical example of mode-based switching. Asteroids and its successors use a torus-wrap world topology that makes the inverse map multi-valued without changing the projection itself. The modern stylised independent games including Limbo and Inside apply post-processing effects to a side-scrolling projection to produce a distinctive visual signature beyond what the projection math contains.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a piecewise forward map defined by region or by game state. The delivery mechanism chooses how the engine switches between the pieces and how the rendered output combines into a single frame.

A Brief History of the Mode

The earliest commercial use of hybrid projection is debated. Asteroids from Atari in 1979 provided a flat-projection world with a torus-wrap topology that the player’s ship exited at one edge and reentered at the opposite edge. The wrap was not strictly a hybrid projection but a topology that the inverse map had to accommodate. Pac-Man and other early arcade games used similar wrap topologies for the tunnel transitions on the side of the maze.

Final Fantasy from Square in 1987 on the Famicom and Nintendo Entertainment System established the per-mode projection switching that defined the Japanese role-playing game across the late 1980s and 1990s. The overworld used a top-down view of a continent-scale map. Towns used a top-down view at a closer zoom with detailed tile graphics. Dungeons used the same top-down view in a constrained corridor format. The battle scene used a side-view with the player party on one side and the enemy formation on the other side. The menu and inventory used a flat overlay that ignored the world geometry entirely. Mode switches between gameplay scenes transitioned the projection through brief animations or scene fades that hid the inconsistency from the player, while the menu overlay appeared instantly on button press without an intervening transition.

Mother from Ape and Pax Softnica in 1989 in Japan, released internationally as EarthBound Beginnings in 2015, introduced the intra-scene hybrid projection that the series carried forward. The overworld ground tiles rendered in a top-down view but the buildings, trees, and obstacles rendered in side-elevation view as if the camera were looking horizontally at the upright objects while looking straight down at the ground. The visual inconsistency gave the Mother world its distinctive flat-but-occupied feel that subsequent Mother games inherited.

EarthBound from Ape and HAL Laboratory in 1994 on the Super Nintendo Entertainment System extended the Mother hybrid projection into the sixteen-bit era with richer tile graphics and more visually complex buildings. The hybrid projection remained the visual signature of the Mother series through the post-Famicom releases.

Mother 3 from HAL Laboratory and Brownie Brown in 2006 on the Game Boy Advance continued the lineage into the handheld portable era with the same intra-scene hybrid projection and a refined art style that pushed the technique toward an emotionally-resonant aesthetic that the series became known for.

The modern stylised independent game emerges with Limbo from Playdead in 2010 on the Xbox 360. The side-scrolling puzzle-platformer applies a monochrome silhouette aesthetic with atmospheric depth-of-field and particle effects that distinguish the scene from a literal side view. The projection math is the side-scrolling forward map of an earlier cluster article, but the post-processing produces a visual style that the projection alone does not.

Inside from Playdead in 2016 on Microsoft Windows extended the Limbo formula with three-dimensional rendering projected through a side-scrolling camera that produces visually two-and-a-half-dimensional gameplay. The hybrid here is between three-dimensional world geometry and two-dimensional gameplay constraints, which the article treats as a special case of mode-based projection switching.

The Forward Map

The world has a partition into regions $R_1, R_2, \dots, R_n$ that the engine selects from based on the world point’s location or on the current game state. Each region $R_i$ carries its own projection forward map $F_i$ that the previous articles defined. The piecewise forward map is

where $i(\mathbf{p}_{\text{world}})$ identifies the region that the world point falls into. The function $i$ may depend on world coordinates, on game state, or on both.

The Mother hybrid partitions the world by the vertical world coordinate $w_y$ in the $y$-down convention of the previous articles. The ground plane at $w_y = w_y^{\text{ground}}$ uses the top-down forward map of the first cluster article,

The objects above the ground at $w_y < w_y^{\text{ground}}$ use a hybrid map that combines the top-down lateral and depth axes with a side-elevation vertical extent through the height-above-ground value $h = w_y^{\text{ground}} - w_y$,

The forward map for objects reproduces the decoupled-vertical-axis forward map of an earlier cluster article. The hybrid arises not from the equation but from the artistic rendering convention where the object sprites depict the side-elevation view of the object even though the world is otherwise rendered from above. The viewer sees the building’s walls, windows, and roof, while the ground underneath the building is the same top-down tile texture as elsewhere on the map.

The per-scene switching variant partitions the world by game state $\sigma$ rather than by world coordinate. The Final Fantasy overworld state uses a top-down forward map at zoom $z_{\text{overworld}}$. The town state uses a top-down forward map at zoom $z_{\text{town}} > z_{\text{overworld}}$. The battle state uses a side-view forward map with the world axes reassigned to match the battle layout. The menu state uses a flat overlay that the article treats as zero-zoom orthographic where the world coordinates do not contribute to the screen position,

The forward map for the menu state ignores the world position entirely and uses a fixed screen position that the menu layout designer chose.

The state-switching function chooses among the four forward maps based on the current game state,

The transitions between states typically run through a brief animation that interpolates between the source and target projections or simply masks the transition with a fade or wipe. The interpolating-transition variant applies a time-dependent convex combination of the source and target forward maps,

where $\beta(t) \in [0, 1]$ is the transition progress curve that the engine animates from $0$ at the start to $1$ at the end. A linear $\beta(t)$ produces a uniform-speed transition. A smoothstep or sigmoid $\beta(t)$ produces ease-in and ease-out at the boundaries. The interpolating variant is rare in retro games that prefer fade or wipe masking but appears in modern releases that emphasise visual continuity.

The torus-wrap topology of Asteroids applies a modulo operation to the world coordinates before the forward map,

where $\mathbf{W}{\text{world}} = (W{\text{world}, x}, W_{\text{world}, z})$ is the world dimensions along the two wrapping axes. The forward map applies to the effective position,

The base forward map $F_{\text{base}}$ is the floor-case top-down map of the first cluster article without modification. The wrap topology affects only the world-coordinate input.

The stylised post-processing variant of Limbo and Inside applies a side-scrolling forward map of an earlier cluster article followed by a post-rendering pass that modifies the rendered image through monochrome filtering, depth-of-field blurring, or particle compositing,

where $P$ is the post-processing pipeline that the engine applies after the projection has placed pixels on the screen. The post-processing does not change the projection math but contributes to the stylised visual identity that the player associates with the game.

The Inverse Map

The inverse map of a piecewise projection is itself piecewise. Given a screen pixel $\mathbf{p}_{\text{screen}}$, the engine identifies which region or game state the click refers to and applies the corresponding inverse,

The function $i(\mathbf{p}_{\text{screen}})$ that identifies the region from the screen pixel is the dual of the world-region partition function. For the Mother hybrid, the engine knows whether each screen pixel belongs to the ground or to a building sprite from the per-sprite draw order and the sprite’s screen-space bounding rectangle. Clicks on the ground tiles use the top-down inverse. Clicks on building sprites use the building-sprite hit-test convention of the decoupled-vertical-axis article.

For the per-scene switching variant, the current game state $\sigma$ fully determines the inverse,

The engine knows the current state and applies the appropriate inverse without ambiguity.

The torus-wrap topology makes the inverse multi-valued when the visible world contains multiple wraps,

where $\mathbf{p}_{\text{world}}^{\text{base}}$ is the base inverse before wrap consideration and $(k_x, k_z)$ runs over integer shift indices that produce the equivalent wrapped positions. The engine resolves the multi-valuedness by choosing the closest wrap representative to the camera or to the gameplay-relevant position.

The post-processing variant does not affect the inverse map of the underlying projection. The engine inverts the side-scrolling map to recover the world position and ignores the post-processing when answering picking queries.

A Worked Example

Consider an EarthBound-style Super Nintendo Entertainment System role-playing game with the following parameters. The screen is 256 pixels wide and 224 pixels tall. The zoom factor is $z = 16$ pixels per world tile, giving 16-by-16-pixel ground tiles. The screen-centre offset is $\mathbf{o} = (128, 112)$. The camera position is $\mathbf{c} = (5, 5)$ world tiles. The ground vertical world coordinate is $w_y^{\text{ground}} = 0$.

A ground tile at world position $\mathbf{p}_{\text{world}} = (5, 0, 5)$ projects through the top-down forward map to

The tile renders at the screen centre.

A building anchored at world position $\mathbf{p}{\text{world}} = (5, 0, 6)$ with height $h{\text{building}} = 3$ world tiles has its base at the screen position

The building sprite extends upward on the screen from the base by $z \cdot h_{\text{building}} = 16 \cdot 3 = 48$ pixels. The building top edge appears at

The building sprite extends from screen $(128, 80)$ at the top to $(128, 128)$ at the base. The Mother-style art renders the building as a side-elevation view of the structure within this screen-space extent.

A click at screen pixel $(128, 100)$ falls inside the building sprite’s vertical extent. The inverse map identifies the click as a building-sprite hit through the per-sprite bounding rectangle of the building. The local height of the click above the ground follows from the screen-y position relative to the building base,

The recovered world coordinates combine the building’s ground anchor position with the local height,

The negative vertical world coordinate indicates a position above the ground in the $y$-down convention where the ground sits at $w_y^{\text{ground}} = 0$.

A click at screen pixel $(140, 130)$ falls below the building sprite on a ground tile. The inverse map identifies the click as a ground-tile hit and applies the top-down inverse,

The click corresponds to a position slightly to the right and slightly forward of the building’s base.

A transition to battle mode switches the projection state from overworld to battle. The battle screen uses a side-view forward map with the party on the right and the enemies on the left. The world coordinates of the overworld do not carry into the battle scene. The battle scene uses its own world frame with the party and enemies positioned along the battle’s lateral axis.

The round-trip identity holds within each region or state,

where $\sigma$ is the active state or region and $\varepsilon$ is the floating-point precision of the engine. The round-trip does not hold across regions or states.

Variations Within the Mode

The piecewise projection framework admits several variations that engines have explored.

A spatial-partition variant divides the world by world-coordinate region. The Mother ground-and-building hybrid is the canonical case. A multi-storey building might use one projection for ground-level floors and a different projection for an upper floor’s interior view.

A state-based variant divides the projection by game state without considering world coordinates. The Final Fantasy mode-switching is the canonical case. A modern role-playing game might have an exploration mode, a combat mode, a cutscene mode, and a menu mode each with its own projection.

A topological-wrap variant applies modulo arithmetic to the world coordinates to create a wrapping world topology. Asteroids and Pac-Man both use topological wrap. The single-axis wrap of Pac-Man’s tunnel transitions is mathematically simpler than Asteroids’ two-axis torus but otherwise follows the same framework.

A post-processing variant applies effects to the rendered image after the projection to produce a stylised visual identity. Limbo and Inside both use post-processing variants. The variant does not affect the projection math but contributes to the visual style that the player associates with the game.

A multi-projection-layer variant renders the same world position through multiple projections simultaneously, combining the results into a single screen image. A game might render the gameplay world through one projection and overlay a minimap that shows the same world through a different projection. The combination is hybrid even though each layer is consistent.

A cinematic-camera variant switches the projection during scripted cutscenes to produce camera angles that the gameplay projection cannot achieve. A top-down strategy game might switch to a cinematic third-person view during cutscenes, then return to top-down for gameplay.

A user-selectable projection variant permits the player or the engine to choose between projection modes based on player preference or based on accessibility settings. The variant is rare in older games and increasingly common in modern releases.

Delivery Mechanisms

The piecewise projection permits five distinct delivery mechanisms on period hardware.

The first is multiple sprite layers with different projection conventions within a single hardware rendering pipeline. The Mother games on the Nintendo Entertainment System used the standard PPU sprite hardware for both ground tiles and building sprites, relying on artistic convention to produce the hybrid visual style rather than on different rendering paths.

The second is mode-switched rendering pipelines. The Final Fantasy games on the Famicom and Nintendo Entertainment System ran different rendering code paths for the overworld, town, dungeon, battle, and menu modes. The mode transitions triggered the rendering pipeline change at well-defined game-state boundaries.

The third is world-coordinate wrap arithmetic in the gameplay simulation. Asteroids on the Atari arcade hardware performed the modulo operation in the gameplay code before the rendering pipeline applied the standard top-down projection. The wrap arithmetic ran on the central processing unit without requiring special graphics hardware support.

The fourth is post-processing image filters applied after the projection-based rendering. Limbo and Inside on the modern personal computer apply the monochrome filter and depth-of-field through graphics-processing-unit shaders that run after the side-scrolling projection has placed the pixels. The mechanism trades the additional shader passes for the distinctive visual identity.

The fifth is shader-based per-fragment projection variation on modern hardware. A modern engine can implement a piecewise projection through a fragment shader that selects the projection map based on the fragment’s world coordinates or the current game state. The mechanism gives the engine fine-grained control over the projection per screen pixel.

All five mechanisms support the piecewise projection abstraction. The choice trades hardware requirements, the implementation complexity of mode transitions, and the achievable visual style.

Where the Framing Breaks Down

The piecewise projection framing is insufficient when any of the following conditions hold.

When the game expects the player to read a consistent three-dimensional world geometry from the rendered scene, the inconsistency of a piecewise projection disrupts the spatial intuition. The Mother hybrid works because the player accepts the convention that the ground is top-down and the buildings are side-elevation as a stylistic choice. A game that needed accurate spatial reasoning could not use the same hybrid.

When the gameplay requires unique inverse mapping for picking, the topological-wrap variant introduces ambiguity that the engine must resolve through additional logic. The torus-wrap inverse returns an infinite candidate set that the engine must reduce to a single relevant world position.

When the gameplay physics require a consistent world frame for collision detection and motion simulation, the per-scene switching variant introduces seams between scenes that the engine must handle through scene-transition logic. The seams are usually invisible to the player because the transitions are scripted.

When the visual style must be consistent across all scenes for narrative or thematic reasons, the mode-switching variant disrupts the consistency. A game that wanted a uniformly grim visual style through all gameplay scenes could not switch projections freely.

When the rendering must scale to support arbitrary new modes without engine modification, the static piecewise projection of the article is insufficient. A modern engine might use a data-driven projection system where new modes can be added through configuration without code changes.

The Canon

The following games use piecewise or stylised projections as defining visual features. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Asteroids in the arcade in 1979 gave the medium its first commercially-released torus-wrap world topology. The flat-projection space with edges that wrapped to opposite edges became the visual template for several arcade space-shooter contemporaries.

Final Fantasy on the Famicom and Nintendo Entertainment System in 1987 established the per-mode projection switching between overworld, town, dungeon, battle, and menu modes that the Japanese role-playing game subgenre inherited.

Mother on the Famicom in 1989 introduced the intra-scene hybrid projection where ground tiles render in top-down view and buildings render in side elevation. The international release as EarthBound Beginnings in 2015 brought the original Famicom title to a wider audience.

EarthBound on the Super Nintendo Entertainment System in 1994 extended the Mother hybrid projection into the sixteen-bit era and became the most widely-played Mother-series game in the West through its post-release cult following.

Mother 3 on the Game Boy Advance in 2006 continued the hybrid projection lineage into the handheld portable era.

Limbo on the Xbox 360 in 2010 established the modern stylised side-scrolling through a monochrome silhouette aesthetic with atmospheric post-processing.

Inside on Microsoft Windows in 2016 extended the Limbo formula with three-dimensional rendering projected through a side-scrolling camera.

Each game in the canon combines projection math with a stylistic convention that the player accepts even though the rendered scene does not represent a single physical camera view.

Out of Scope

The article does not cover the following.

True three-dimensional rendering with arbitrary camera placement through a graphics-processing-unit pipeline is the modern technique that the synthesis closer of the series treats. The piecewise projections of this article are stylised choices within the affine and projective projection families that the cluster has covered.

The full Y-sort framework and the full pick-disambiguation framework are the subjects of the following cross-cutting articles. The article presents picking in its simplest piecewise form and defers the full treatment.

The internal architecture of state transitions between game modes including save state management, asset loading, and gameplay state preservation is gameplay-systems territory adjacent to but distinct from the projection math.

The choice of when to apply a piecewise projection versus when to use a consistent projection is a design question that the series does not adjudicate. Each visual style has games for which it is the right choice and games for which it is the wrong choice.

The shader programming techniques that modern engines use for post-processing effects are implementation concerns that the article treats only at the level of the delivery-mechanism sidebar.

Conclusion

Stylised and hybrid projections combine multiple projection modes within a single game to produce a visual identity that no single mode achieves alone. The piecewise forward map $F(\mathbf{p}{\text{world}}) = F{i(\mathbf{p}{\text{world}})}(\mathbf{p}{\text{world}})$ applies different projection modes to different regions of the world or different game states. The Mother hybrid where the ground renders in top-down view and buildings render in side elevation is the canonical example of intra-scene inconsistency. The Final Fantasy lineage where the overworld, town, and battle each render through their own projection is the canonical example of mode-based switching. Asteroids and its successors use a torus-wrap world topology that makes the inverse map multi-valued. The modern stylised independent games including Limbo and Inside apply post-processing effects to a side-scrolling projection to produce a distinctive visual signature. The article closes the affine-and-projective cluster with the recognition that real games mix projection modes freely in service of artistic and gameplay goals. The remaining articles in the series treat cross-cutting concerns that span the projection modes of the previous clusters, beginning with draw-order management and the Y-sort algorithm in the next article.

References

• Reference, Asteroids
• Reference, EarthBound
• Reference, Final Fantasy
• Reference, Inside
• Reference, Limbo
• Reference, Mother
• Reference, Mother 3

The third article of the affine-and-projective cluster treats raycasting as a pseudo-three-dimensional rendering technique that operates on a two-dimensional grid map. Raycasting casts one ray per screen column from the camera through the visible portion of the world and computes the distance to the first wall the ray hits. The wall is rendered as a vertical strip with height inversely proportional to the distance. The continuous variation of slice heights across screen columns gives the player the impression of looking forward at a three-dimensional corridor even though the world is a flat two-dimensional grid of wall and floor cells.

The technique pre-dates true three-dimensional polygon rendering on consumer hardware and shares the late-1980s and early-1990s era with the sprite-scaling and Mode 7 techniques of the previous articles. Raycasting differs from both in that the depth illusion arises from per-column ray geometry rather than from per-pixel affine scaling or per-scanline matrix interpolation. The result is a genuine first-person view of a tile-based world, which the canonical first-person shooters of the early 1990s established as the visual signature of the subgenre.

The article frames raycasting as a per-column projection technique where the projection math maps each screen column to a single ray and computes the slice height from the perpendicular distance to the wall. The forward map produces the wall slice geometry from the world configuration. The inverse map recovers the wall hit point from a screen pixel for picking and gameplay interaction purposes.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a per-column inverse-distance scaling combined with a fisheye correction that the article derives. The delivery mechanism chooses how the engine casts the rays and how the renderer draws the wall slices that the rays produce.

A Brief History of the Mode

Ray casting in computer graphics dates to the 1960s as a rendering technique for displaying solid geometry through ray-surface intersection. The video-game adoption arrives in the early 1990s with the commercial first-person shooters of id Software.

Hovertank 3D from id Software in 1991 gave the medium its first commercially-released raycasting game. The flat-shaded vehicle simulator rendered walls without textures and demonstrated the per-column raycasting algorithm on personal-computer hardware of the era.

Catacomb 3-D from id Software later in 1991 added texture mapping to the wall slices through the per-column ray-grid intersection that the texture-coordinate calculation requires. The textured walls gave the genre its first taste of visually-rich tile-based environments.

Wolfenstein 3D from id Software in 1992 brought raycasting to the mainstream personal-computer audience. The first-person shooter gameplay across a labyrinth of Nazi-themed levels became the canonical raycasting visual style and the template for the subsequent first-person-shooter genre. Wolfenstein 3D’s commercial success established the raycasting renderer as a viable technique for the early-1990s personal computer.

Ken’s Labyrinth from Ken Silverman and Epic MegaGames in 1993 extended the raycasting framework with textured floors and ceilings and additional gameplay features beyond the wall-only Wolfenstein engine.

Rise of the Triad from Apogee Software in 1994 extended the Wolfenstein engine with additional gameplay mechanics including jumping and crouching that pushed the raycasting framework’s limits.

The technique recedes from the canon after the mid-1990s as the polygon-based rendering of Doom in 1993 and the true three-dimensional engines of the late 1990s displaced raycasting for first-person games. Doom used binary-space-partition rendering, and the Build engine of Duke Nukem 3D in 1996 extended the technique with portal and sector rendering that the article treats as outside the raycasting framework.

The technique persists in modern independent retro releases and in the educational tradition where raycasting serves as an accessible introduction to pseudo-three-dimensional rendering on hardware without three-dimensional acceleration.

The Forward Map

The world is a two-dimensional grid of cells in the $(w_x, w_z)$ plane, where the lateral axis $w_x$ and the depth axis $w_z$ both correspond to the floor plane under the $y$-down convention of the previous articles. The vertical world axis $w_y$ is gravity-aligned downward. Each grid cell is either an open cell that the player can occupy or a wall cell that blocks the player and the ray. Walls extend from the floor at $w_y = w_y^{\text{floor}}$ to the ceiling at $w_y = w_y^{\text{ceiling}}$ by the wall height $h_{\text{wall}} = w_y^{\text{floor}} - w_y^{\text{ceiling}}$ under the convention that the ceiling is above the floor and therefore has smaller $w_y$ in the $y$-down frame.

The camera position is a two-dimensional world coordinate $\mathbf{c} = (c_x, c_z)$ on the floor plane. The camera’s vertical position is fixed at the midpoint between floor and ceiling, $c_y = (w_y^{\text{floor}} + w_y^{\text{ceiling}})/2$. The camera looks forward along a unit-length direction vector $\mathbf{d} = (d_x, d_z)$ with $|\mathbf{d}| = 1$.

The camera plane vector $\mathbf{p}$ is perpendicular to $\mathbf{d}$ with magnitude set by the horizontal field of view,

For a 90-degree field of view, $|\mathbf{p}| = 1$. For a 66-degree field of view common in early raycasters, $|\mathbf{p}| \approx 0.66$.

The per-column ray cast emits one ray per screen column $s_x \in {0, 1, \dots, W-1}$. The normalised camera-x coordinate runs from $-1$ at the left screen edge to $+1$ at the right edge,

The ray direction for column $s_x$ is

The ray is not unit-length in general. The central column has $\mathrm{camx} = 0$ and ray direction $\mathbf{d}$. The screen edges have $\mathrm{camx} = \pm 1$ and rays that deviate from forward by the field-of-view angle.

The engine casts each ray through the grid map from the camera position $\mathbf{c}$ along the direction $\mathbf{r}$. The standard algorithm is the digital differential analyser, which iterates through grid cells along the ray until a wall cell is encountered. The article treats the algorithm as a primitive without deriving the iteration details and focuses instead on the projection math that converts the wall hit into the slice geometry.

The wall hit point is the world position $\mathbf{h} = \mathbf{c} + t\, \mathbf{r}$ where $t > 0$ is the smallest ray parameter at which the ray crosses a wall cell boundary. The perpendicular distance from the camera plane to the wall is the projection of the displacement onto the forward direction,

The simplification to $t$ holds because $\mathbf{d}$ is unit-length and $\mathbf{p}$ is perpendicular to $\mathbf{d}$, so $\mathbf{r} \cdot \mathbf{d} = (\mathbf{d} + \mathrm{camx} \cdot \mathbf{p}) \cdot \mathbf{d} = 1$. The perpendicular distance equals the ray parameter $t$ at the hit without requiring an explicit trigonometric correction.

The wall slice height in pixels follows from the inverse-distance scaling that perspective projection introduces,

where $f$ is the vertical focal length in pixels per world unit at unit distance. For the canonical normalisation where a wall of height $h_{\text{wall}} = 1$ fills the screen vertically when at distance $d_{\text{perp}} = f/H$, the simplified formula is

A wall at distance $d_{\text{perp}} = f$ has slice height $h_{\text{slice}} = 1$ pixel, and a wall at distance $d_{\text{perp}} = f/H$ fills the screen with slice height $H$ pixels.

The wall slice is centred vertically around the horizon screen-y $s_y^{\text{horizon}} = H/2$ under the assumption that the camera vertical position sits at the wall midpoint. The slice spans the vertical screen range

The renderer writes the wall texture into the screen frame buffer at each pixel within this vertical range at horizontal position $s_x$. Above the slice the renderer draws the ceiling. Below the slice the renderer draws the floor.

The texture coordinate within the wall slice follows from the wall hit position. For a vertical wall at constant $w_z$, the horizontal texture coordinate is the fractional part of the hit’s $w_x$,

For a wall at constant $w_x$, the texture coordinate uses the $w_z$ fractional part instead. The vertical texture coordinate maps the slice screen-y to the wall’s local vertical,

The vertical texture coordinate ranges from $0$ at the top of the slice to $1$ at the bottom.

The Fisheye Correction

The article uses the perpendicular distance $d_{\text{perp}}$ in the slice-height formula. An engineer new to raycasting might use the Euclidean distance instead,

The Euclidean distance and the perpendicular distance are related by

where $\alpha$ is the angle between the ray and the forward direction,

Using $d_{\text{euc}}$ in the slice-height formula gives slices at the screen edges that are smaller than they should be, because $d_{\text{euc}} > d_{\text{perp}}$ at any column other than the central one. A flat wall perpendicular to the forward direction would render with center slice tall and edge slices short, making the wall appear convex toward the viewer. The community calls the artifact the fisheye effect.

The fisheye correction replaces $d_{\text{euc}}$ with $d_{\text{perp}}$ in the slice-height formula,

The corrected slice rendering preserves flat walls as flat across the screen.

The simplification of the previous section in which $d_{\text{perp}} = t$ when $\mathbf{d}$ is unit-length and $\mathbf{p}$ is perpendicular to $\mathbf{d}$ gives the fisheye correction for free. The digital differential analyser computes $t$ at the hit and the engine uses $t$ directly as $d_{\text{perp}}$ without needing to compute $\cos\alpha$ separately.

The Inverse Map

The forward map is per-column, so the inverse map is per-pixel. A click at screen pixel $(s_x, s_y)$ inverts in two steps.

The first step identifies the column $s_x$ and looks up the ray that the column corresponds to. The engine retrieves the wall hit point $\mathbf{h}$ and the perpendicular distance $d_{\text{perp}}$ that the forward map already computed for that column.

The second step distinguishes wall, ceiling, and floor pixels based on the vertical position $s_y$ relative to the wall slice extent.

For a wall pixel, the inverse is the wall hit point combined with the texture coordinate at the click,

The vertical world coordinate recovers from the wall slice position through the texture-coordinate formula above.

For a floor pixel below the slice, the inverse follows the ray to the floor plane $w_y = w_y^{\text{floor}}$. The floor distance from the camera along the camera-forward axis is

with the same inverse-screen-y form as the Mode 7 ground-plane depth of the previous cluster article. The floor world position at pixel $(s_x, s_y)$ follows from advancing along the per-column ray by the floor distance,

where $r_x(s_x)$ and $r_z(s_x)$ are the horizontal components of the per-column ray direction. The ray parameter at the floor hit equals the perpendicular distance because $\mathbf{r}(s_x) \cdot \mathbf{d} = 1$.

For a ceiling pixel above the slice, the inverse follows the same form with $w_y^{\text{ceiling}}$ replacing $w_y^{\text{floor}}$ and $s_y - H/2$ replaced by $H/2 - s_y$ to keep the distance positive.

The floor and ceiling inverses use only the per-column ray $\mathbf{r}(s_x)$ of the forward map together with the $s_y$-dependent distance. The raycasting framework does not require an explicit per-pixel ray construction because the floor and ceiling planes are at known constant $w_y$ and the depth from the camera follows from the screen-y by inverse projection.

The visible region of the world that raycasting renders is the set of world points that the rays can reach. The visible region is bounded by walls that the rays terminate at and by the maximum ray-cast distance that the engine permits. For a closed map with no openings beyond the ray-cast distance, the visible region is the union of cones extending from the camera through each of the visible open cells.

Picking against gameplay objects positioned as sprites in the world follows the depth-sorted sprite rendering of the previous article. The engine computes each sprite’s screen-space position through the perspective projection of the previous article and Y-sorts the sprites against the wall slices through the per-column $d_{\text{perp}}$ values. The click identification iterates the visible sprites in front-to-back depth order and tests the screen-space bounding rectangle of each. The first sprite whose rectangle contains the click is returned.

A Worked Example

Consider a Wolfenstein-3D-style first-person shooter with the following parameters. The screen is 320 pixels wide and 200 pixels tall, matching the personal-computer Mode 13h display that early raycasters targeted. The horizontal field of view is 66 degrees, so the camera plane magnitude is $|\mathbf{p}| = \tan(33°) \approx 0.65$. The vertical focal length is $f = 200$ pixels per world unit at unit distance, chosen so that a wall of height 1 at distance 1 fills the screen vertically.

The camera position is $\mathbf{c} = (5.5, 5.5)$ world units. The camera looks forward along the $+w_x$ axis, so $\mathbf{d} = (1, 0)$. The camera plane vector, perpendicular to $\mathbf{d}$ with magnitude $0.65$, is $\mathbf{p} = (0, 0.65)$. A wall cell occupies the grid column at $w_x = 10$, so all walls at $w_x \in [10, 11]$ are solid.

The central screen column $s_x = 160$ has $\mathrm{camx} = 2 \cdot 160 / 320 - 1 = 0$ and ray direction $\mathbf{r}(160) = (1, 0) + 0 \cdot (0, 0.65) = (1, 0)$. The ray hits the wall at world position $\mathbf{h} = (10, 5.5)$ with ray parameter $t = 4.5$.

The perpendicular distance is

The wall slice height is

The slice spans screen-y range $[100 - 22, 100 + 22] = [78, 122]$. The renderer writes the wall texture into the column $s_x = 160$ at rows 78 through 122.

The leftmost screen column $s_x = 0$ has $\mathrm{camx} = -1$ and ray direction $\mathbf{r}(0) = (1, 0) + (-1)(0, 0.65) = (1, -0.65)$. The ray hits the same wall at $w_x = 10$ at ray parameter $t = 4.5$ because the wall is perpendicular to the forward direction and $t$ is the projection of the displacement onto $\mathbf{d}$. The hit point is

The perpendicular distance is

The wall slice at column $s_x = 0$ has the same height as the central column,

The flat wall renders as a flat constant-height strip across the screen, which is the correct visual behaviour for a wall perpendicular to the view direction.

To illustrate the fisheye effect, consider what would happen without the fisheye correction. The Euclidean distance for the edge column is

The uncorrected slice height would be

which is shorter than the central column’s 44 pixels. A flat wall would render with the centre at full height and the edges shortened, giving the wall a bowed convex appearance. The fisheye correction restores the flat wall to a constant slice height across the screen.

A click at screen pixel $(160, 100)$ falls in the central column’s wall slice. The inverse map identifies the column $s_x = 160$, retrieves the wall hit $\mathbf{h} = (10, 5.5)$, and recovers the vertical texture coordinate through

The click corresponds to the wall at world position $(10, w_y^{\text{ceiling}} + 0.5 \cdot h_{\text{wall}}, 5.5)$, which is the wall midpoint vertically.

The round-trip identity holds per column,

where $\varepsilon$ is the floating-point precision of the engine.

Variations Within the Mode

The raycasting framework admits several variations that engines have explored.

A textured-floor-and-ceiling variant extends the wall raycasting with per-pixel floor and ceiling sampling. For each pixel below the wall slice, the engine computes the ray-floor intersection and samples the floor texture at the intersection. The variant requires per-pixel arithmetic that exceeds the per-column cost of wall-only raycasting. Ken’s Labyrinth used textured floors and ceilings through this variant.

A variable-height-wall variant permits walls to have heights that vary by grid cell. The wall slice formula extends to non-unit wall heights through the multiplication by $h_{\text{wall}}$ that the slice-height formula admits. Rise of the Triad and later raycasters used variable-height walls to suggest more complex environments than the uniform-height Wolfenstein labyrinth.

A jumping-and-crouching variant permits the camera vertical position to vary from the wall midpoint. The wall slice positions vertically according to the camera height,

Rise of the Triad used the jumping-and-crouching variant.

A look-up-and-down variant permits the camera pitch to vary from straight ahead. The wall slice positions vertically according to the pitch offset,

The variant is mathematically an approximation to true three-dimensional rotation that the raycasting framework can support within its per-column projection model.

A textured-wall variant applies a texture map to each wall slice indexed by the wall hit’s horizontal texture coordinate and the slice screen-y’s vertical texture coordinate. The variant is the default in all post-Hovertank raycasters.

A multi-level-grid variant extends the two-dimensional grid to a stack of grids that the engine treats as multiple floors. The variant approximates multi-storey environments that pure raycasting cannot represent. The Build engine of Duke Nukem 3D extended this approach further through portal rendering that the article does not treat.

A transparent-wall variant permits the ray to continue past a wall if the wall texture has alpha-transparent regions. The variant requires the engine to sample the wall texture during the ray cast and continue if the sample is transparent. The transparent-wall variant permits doorways, windows, and decorative grilles within the otherwise-opaque wall framework.

Delivery Mechanisms

The raycasting forward map permits five distinct delivery mechanisms on period hardware.

The first is software raycasting on a general-purpose central processing unit. The engine casts each ray through the grid map on the central processing unit and writes the resulting wall slice into the frame buffer. The mechanism was the universal delivery on the IBM PC running the Microsoft Disk Operating System through the early 1990s. Wolfenstein 3D, Catacomb 3-D, Hovertank 3D, Ken’s Labyrinth, and Rise of the Triad all used software raycasting.

The second is fixed-point arithmetic within software raycasting to avoid floating-point operations on processors without hardware floating-point support. The Intel 80386 and earlier central processing units that the early-1990s personal computers used either lacked hardware floating-point or had it as an optional coprocessor. Wolfenstein 3D used fixed-point arithmetic for the ray-grid intersection and the slice-height calculation to achieve real-time frame rates on the era’s hardware.

The third is precomputed lookup tables for trigonometric functions and reciprocals. The wall slice height involves a division by the perpendicular distance, and the per-column ray angle involves a cosine for the fisheye correction. The engine precomputes both as tables indexed by the angle or by the distance, trading memory for computation time.

The fourth is hardware texture mapping on later-1990s personal-computer hardware that included three-dimensional accelerator cards. The engine casts the rays in software but draws the wall slices through hardware texture-mapped quads. The variant transitions raycasting to a hybrid software-and-hardware pipeline that the late-1990s personal computer supported.

The fifth is graphics-processing-unit shader raycasting on modern hardware. A fragment shader casts the rays in parallel across screen columns and computes the wall slice geometry per fragment. The mechanism takes the raycasting algorithm to its computational extreme through massive parallelism. Modern indie retro-style raycasters typically use graphics-processing-unit shader raycasting to achieve high resolutions that software raycasting on the central processing unit could not match.

All five mechanisms compute the same forward map and produce the same visible result. The choice trades implementation complexity, the central-processing-unit budget, the lookup-table memory budget, and the achievable resolution and frame rate.

Where the Framing Breaks Down

The raycasting framing is insufficient when any of the following conditions hold.

When the world contains non-axis-aligned walls that cannot be represented as full or empty grid cells, the raycasting framework is insufficient. The portal-and-sector rendering that the Build engine introduced addresses this case through sector polygons that the article does not treat.

When the world contains multiple floors at the same horizontal position, the single-grid model of pure raycasting is insufficient. The multi-level-grid variant above extends the framework partially, and the portal-rendering frameworks extend it further.

When the gameplay requires true three-dimensional geometry with arbitrary mesh structures, the raycasting framework is insufficient. The polygon-based rendering of Doom and its successors addresses this case through binary-space-partition trees and explicit polygon rasterisation that the article treats as outside its scope.

When the camera must pitch significantly or roll around the forward axis, the per-column projection of raycasting loses its perpendicular-distance simplification. The look-up-and-down variant extends the framework with an approximation, but significant pitch warps the wall slices in a way that purer perspective projection avoids.

When the wall textures must distort with viewing angle to suggest non-flat surfaces, the per-column constant texture-x mapping of raycasting is insufficient. A genuinely curved wall requires per-pixel ray-surface intersection that the article does not treat.

The Canon

The following games use raycasting as their primary rendering technique. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Hovertank 3D on the IBM PC in 1991 gave the medium its first commercially-released raycasting game. The flat-shaded walls demonstrated the algorithm without the texturing complexity that subsequent games added.

Catacomb 3-D on the same hardware later in 1991 added wall texture mapping to the raycasting framework and brought visually-rich tile-based environments to the genre.

Wolfenstein 3D on the same hardware in 1992 brought raycasting to the mainstream personal-computer audience. The first-person shooter became the canonical raycasting game and the template for the first-person-shooter genre that the polygon-based successors inherited.

Ken’s Labyrinth on the same hardware in 1993 extended the raycasting framework with textured floors and ceilings, demonstrating that the per-column algorithm could carry additional visual fidelity beyond Wolfenstein’s wall-only treatment.

Rise of the Triad on the same hardware in 1994 extended the Wolfenstein engine with jumping, crouching, and variable-height walls that pushed the raycasting framework to its visual upper bound.

Each game in the canon uses the per-column raycasting forward map with a different combination of variation choices appropriate to the gameplay it supports.

Out of Scope

The article does not cover the following.

The binary-space-partition rendering of Doom in 1993 and its successors is outside the raycasting framework. Doom used a precomputed binary-space-partition tree to determine wall visibility and rasterised polygons through a different algorithmic path than per-column raycasting.

The portal-and-sector rendering of the Build engine in Duke Nukem 3D and its contemporaries extends raycasting toward sector polygons that the article does not treat in detail.

True three-dimensional polygon rendering through hardware-accelerated rasterisation is the modern technique that the synthesis closer of the series treats. Raycasting is a per-column projection technique that the synthesis closer subsumes as a special case.

Voxel-based pseudo-three-dimensional rendering of the Comanche series and other voxel terrain renderers is a separate technique that produces superficially similar visuals through height-map-traversal rather than wall-grid raycasting.

The full picking framework including the sprite-scale-and-rotate hit test of the previous article is the subject of a later cross-cutting article. The article presents picking in its simplest raycasting form and defers the full treatment.

Conclusion

Raycasting renders a two-dimensional grid map as a three-dimensional first-person view through per-column ray casting. The forward map per column emits a ray from the camera in the direction $\mathbf{r}(s_x) = \mathbf{d} + \mathrm{camx}(s_x) \cdot \mathbf{p}$, finds the wall the ray hits, and renders a vertical wall slice with height $h_{\text{slice}} = f/d_{\text{perp}}$ where $d_{\text{perp}}$ is the perpendicular distance from the camera plane. The fisheye correction uses the perpendicular distance rather than the Euclidean distance to preserve flat walls as flat across the screen. The algorithm runs in real time on early-1990s personal-computer hardware through fixed-point arithmetic and precomputed lookup tables. The mode defined the first-person shooter genre on the IBM PC through Wolfenstein 3D and its contemporaries before polygon-based rendering displaced it in the mid-to-late 1990s. The next article in the cluster treats stylised and hybrid projections that combine multiple projection modes in a single game.

References

• Reference, Catacomb 3-D
• Reference, Hovertank 3D
• Reference, Ken’s Labyrinth
• Reference, Rise of the Triad
• Reference, Wolfenstein 3D

The second article of the affine-and-projective cluster treats sprite scaling as a pseudo-three-dimensional rendering technique. A sprite is a two-dimensional bitmap that the engine renders at a scaled size on the screen. Sprite scaling multiplies the bitmap dimensions by a per-sprite factor that the engine computes from the sprite’s depth from the camera. A sprite further from the camera appears smaller. A sprite closer to the camera appears larger. The continuous depth-to-size relationship produces a perspective illusion that hardware-supported scaling makes affordable at real-time frame rates.

The technique pre-dates the Mode 7 affine background of the previous article by nearly a decade. The Sega Super Scaler arcade boards of the mid-1980s demonstrated sprite scaling through Space Harrier, Out Run, After Burner, and Galaxy Force II. The home console adoption arrives with the Super Nintendo Entertainment System Mode 7 background hardware repurposed as a single rotating-and-scaling sprite, which Battle Clash and Metal Combat used for the giant enemy mecha in their Super Scope light-gun gameplay. The arcade lineage ran in parallel with and informed the development of the home console hardware that the previous article treated.

The article frames sprite scaling as a per-sprite affine transformation that combines a scaling factor with a sprite-plane rotation. The forward map takes a sprite-local pixel coordinate to a screen pixel coordinate through the combined transform. The inverse map returns to the sprite-local frame for picking and hit-test purposes. The inverse map is the canonical case for the light-gun targeting in the Battle Clash and Metal Combat lineage that the cross-cutting picking article treats in detail later in the series.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a per-sprite affine transform parameterised by a scaling factor and a rotation angle. The delivery mechanism chooses how the engine computes the per-sprite scaling factor and how the hardware applies the scaling and rotation during sprite rendering.

A Brief History of the Mode

The earliest commercial sprite-scaling game is widely cited as Pole Position from Namco in 1982. The arcade game rendered roadside billboards, opposing race cars, and the road itself through pre-scaled sprite frames that the hardware swapped based on the car’s distance from the player. The pre-scaled approach used a sprite atlas at multiple discrete sizes rather than continuous scaling, but the visual effect was the canonical depth-by-size pseudo-three-dimensional rendering that subsequent games inherited.

The Sega Super Scaler arcade boards of the mid-1980s brought hardware-supported continuous sprite scaling to commercial games. Space Harrier from Sega in 1985 rendered hundreds of enemies across a flying-through-a-world setting through hardware sprite scaling that the Super Scaler boards provided. The technical achievement was the simultaneous rendering of many sprites at independent scaling factors that the arcade hardware computed in real time.

Out Run from Sega in 1986 applied Super Scaler sprite scaling to a driving game across a continuous landscape. The hilly roads and roadside scenery all rendered through sprite scaling that the hardware updated per-sprite per-frame. Out Run became the visual template for arcade driving games through the late 1980s.

After Burner from Sega in 1987 applied the same hardware to a flight simulator where the player piloted a jet fighter through formation enemies that the Super Scaler boards rendered. Galaxy Force II from Sega in 1988 pushed the Super Scaler hardware further through cinematic terrain and large boss enemies that demonstrated the upper limit of pre-three-dimensional arcade rendering.

The home console era adopted sprite scaling through the Super Nintendo Entertainment System Mode 7 background hardware of the previous article, repurposed as a single large rotating-and-scaling sprite. Battle Clash from Nintendo and Intelligent Systems in 1992 rendered each giant enemy mecha as a Mode 7 background, treating the affine-background hardware as a single sprite under software control. The cartridge software computed the Mode 7 matrix entries per frame based on the gameplay state. The result was a Super Scope light-gun shooter with rotating, scaling, and translating enemy sprites that the player aimed at and shot.

Metal Combat, Falcon’s Revenge from Nintendo and Intelligent Systems in 1993 extended the Battle Clash formula with additional enemy types and gameplay scenarios through the same Mode 7 background-as-sprite technique.

Yoshi’s Safari from Nintendo in 1993 combined a Mode 7 ground-and-environment background with standard sprite hardware to render a first-person Super Scope shooter across pre-rendered cartoon environments. The Mode 7 background advanced toward the player on rails while sprite enemies at various pre-rendered sizes appeared at corresponding depths.

The technique continues in modern indie and retro releases that emulate the arcade aesthetic through software sprite scaling or graphics-processing-unit shader scaling that produces the same visual effect through different mathematics.

The Forward Map

A sprite is a two-dimensional bitmap of width and height $w_{\text{sprite}}$ and $h_{\text{sprite}}$ in sprite-local pixel coordinates. The sprite-local coordinate system has its origin at the sprite’s anchor point, which is typically the centre of the sprite or the bottom-centre for ground-aligned characters. A pixel of the sprite has the local-frame position

with $u$ and $v$ ranging over the sprite’s pixel extent.

The sprite has a world position $\mathbf{p}{\text{world}}^{\text{sprite}} = (w_x, w_y, w_z)$ in the $y$-down convention of the previous articles. The camera is at world position $\mathbf{c}{\text{camera}} = (c_x, c_y, c_z)$ and looks forward along the world $w_z$ axis. The depth from the camera to the sprite is

The focal length $f$ gives the per-world-unit-of-depth screen scaling at the camera’s unit-distance plane, as in the previous article.

The sprite scaling factor is the inverse-depth ratio of focal length to depth,

A sprite at depth $f$ renders at unit scale with one sprite-local pixel covering one screen pixel. A sprite at depth $2f$ renders at half scale with one sprite-local pixel covering half a screen pixel. A sprite at depth $f/2$ renders at double scale.

The sprite’s screen-space anchor position follows from the perspective projection of the previous article,

The point $\mathbf{c}$ is the screen position of the sprite’s anchor pixel under the perspective camera.

The sprite has an orientation angle $\theta$ in the sprite-local plane, which the engine updates each frame to track the sprite’s gameplay rotation. The sprite-plane rotation matrix is

The forward map takes a sprite-local pixel coordinate to a screen pixel coordinate through the scaling and rotation,

The forward map applies the rotation to the local-frame coordinate, scales the rotated coordinate by $s$, and translates by the sprite anchor’s screen position $\mathbf{c}$. The composition of scaling and rotation is a two-by-two matrix that the engine writes once per sprite per frame.

The Jacobian of the forward map with respect to the sprite-local coordinates is the matrix

with determinant

The screen-space area that the sprite covers follows from the determinant as

A sprite at twice the unit distance covers a quarter of the screen-space area of the same sprite at unit distance, because $s$ halves and $s^2$ quarters. The quadratic area law is the canonical perspective foreshortening applied to a finite sprite.

The factorisation pattern from the opener extends to the sprite-scaling case as a composition of scaling, rotation, and translation,

The composition acts on the sprite-local coordinate to produce the screen coordinate.

The Inverse Map

The forward map’s matrix $sR(\theta)$ is a uniform scaling followed by rotation, and is invertible whenever $s > 0$. The inverse map returns to the sprite-local frame as

The inverse undoes the translation, the rotation, and the scaling in reverse order. The inverse is single-valued within a single sprite because the per-sprite forward map is bijective on the sprite’s screen-space region.

Hit-testing for a click against a sprite is the dominant use of the inverse map. The engine takes the clicked screen pixel, applies the per-sprite inverse, and tests whether the recovered sprite-local pixel falls within the sprite’s local pixel rectangle,

The test succeeds if the click hits the sprite, and fails otherwise. The bounding-box test is the simplest sprite hit test and uses no information beyond the sprite’s local extent. A more refined test samples the sprite’s alpha or palette index at $(u, v)$ to test against the sprite’s transparent regions.

Picking a sprite from a stack of overlapping sprites follows the topmost-in-draw-order convention of the belt-scroll and oblique articles. The engine iterates the visible sprites in front-to-back draw order and returns the first sprite whose inverse-mapped click pixel falls within the sprite’s local rectangle. For sprites at distinct depths, the front-to-back order matches the increasing scaling factor $s$, so the engine effectively returns the sprite at the largest $s$ that the click intersects.

The Battle Clash and Metal Combat light-gun targeting is the canonical use of the sprite inverse map. The player aims the Super Scope light gun at a screen pixel that the SNES reports to the cartridge software. The cartridge code iterates the visible enemies, applies the per-sprite inverse map that the current Mode 7 matrix entries define, and identifies which enemy the player hit. The cross-cutting picking article later in the series treats the full Battle Clash hit-test framework including damage modelling and multi-sprite enemy decomposition.

The visible region that a sprite occupies on screen is the image of the sprite’s local rectangle under the forward map. The region is a rotated rectangle with side lengths $s\, w_{\text{sprite}}$ and $s\, h_{\text{sprite}}$ at orientation $\theta$. For an axis-aligned sprite at $\theta = 0$, the screen-space rectangle simplifies to

The renderer clips world content to the visible region when culling for rendering.

A Worked Example

Consider a Super Nintendo Entertainment System Super Scope light-gun shooter with the following parameters. The screen is 256 pixels wide and 224 pixels tall. The focal length is $f = 100$ pixels. The horizon sits at $s_y^{\text{horizon}} = 112$, the screen vertical centre. The camera position is $\mathbf{c}_{\text{camera}} = (0, 0, 0)$, looking forward along the world $w_z$ axis.

A sprite is a 64-by-32-pixel enemy mecha at sprite-local pixel extent $|u| \le 32$, $|v| \le 16$. The mecha’s world position is $\mathbf{p}_{\text{world}}^{\text{sprite}} = (15, 0, 50)$, 15 units to the right of the camera on the horizon line at depth 50 units.

The depth from camera is $d = 50$. The scaling factor is

The sprite anchor on screen is

The sprite has orientation $\theta = \pi/6$, matching a 30-degree clockwise rotation. $\cos(\pi/6) = \sqrt{3}/2 \approx 0.866$, $\sin(\pi/6) = 1/2$.

The sprite’s top-right corner at sprite-local position $\mathbf{p}_{\text{local}} = (32, -16)$ maps to screen position

Computing the matrix-vector product,

Scaling by $s = 2$ gives $(71.4,\ 4.2)$. The top-right corner’s screen position is

A second mecha sprite at depth 100 has $s = 100/100 = 1$. At the same world horizontal offset, its screen anchor is at $(128 + 15, 112) = (143, 112)$. Its sprites cover half the screen area of the first mecha because the squared scaling factor ratio is $1/4$.

A click at screen pixel $(229, 116)$ falls within the first mecha’s screen-space region. The inverse map applied at $\theta = \pi/6$ and $s = 2$ gives

Computing,

and dividing by 2 gives $(31.8,\ -16)$.

The inverse-mapped click is at sprite-local position $(31.8, -16)$, which lies within the sprite’s local rectangle $|u| \le 32$, $|v| \le 16$. The bounding-box test succeeds and the engine reports a hit on the first mecha.

The round-trip identity holds within a sprite,

where $\varepsilon$ is the floating-point precision of the engine.

The Y-sort criterion of the belt-scroll and oblique articles extends to sprite scaling through the back-to-front comparison on the sprite’s depth from the camera. The criterion writes as a comparison inequality on the depths and equivalently on the scaling factors,

A sprite at smaller world depth $w_z$ has a larger scaling factor $s$ and renders in front of sprites at larger depth. The Y-sort under sprite scaling matches the depth-based sort that perspective projection introduces.

Variations Within the Mode

The sprite-scaling framework admits several variations that engines have explored.

A discrete-scale variant quantises the scaling factor $s$ to a small set of values, matching the available sprite-atlas sizes that the hardware can store. The variant approximates continuous scaling through a small set of pre-rendered sprite frames. Pole Position used the discrete-scale variant because the early-1980s arcade hardware could not support continuous scaling.

A continuous-scale variant permits arbitrary $s$ through hardware-supported scaling or through software scaling that interpolates the sprite bitmap at the target screen size. The Sega Super Scaler boards supported continuous scaling.

A discrete-rotation variant quantises the rotation angle $\theta$ to a small set of values, typically eight or sixteen evenly-spaced angles, and uses a pre-rendered sprite atlas for each angle. The variant trades sprite memory for runtime rotation cost.

A continuous-rotation variant permits arbitrary $\theta$ through hardware-supported rotation or through software bitmap rotation. The Mode 7 background hardware in Battle Clash supported continuous rotation of the giant enemy mecha sprite through per-frame matrix update that the cartridge software computed.

A pseudo-three-dimensional-orientation variant extends the rotation to three axes including pitch and roll in addition to the screen-plane yaw $\theta$. The variant requires pre-rendered sprite atlases covering the three-axis rotation space or software bitmap warping that the engine performs at render time. After Burner and Galaxy Force II used pre-rendered atlases for pitch and roll variations of enemy aircraft.

A non-uniform-scale variant permits independent scaling factors along the sprite-local $u$ and $v$ axes,

The variant produces squashed or stretched sprites that the designer might use for visual effects such as compression on impact or directional stretch during fast motion.

A perspective-tilted-sprite variant applies a full perspective transformation to a flat sprite quad so the sprite appears as a tilted rectangle in three-dimensional space. The variant departs from the affine framework of this article into the projective generalisation that the synthesis closer of the series treats. Modern graphics-processing-unit pipelines implement this variant through textured-quad rendering with perspective camera transforms.

A row-by-row sprite distortion variant modifies the sprite scaling factor per row of the sprite to simulate three-dimensional sprite geometry such as cylinders or spheres that the bitmap alone cannot represent. Period games rarely used the row-by-row variant because the central-processing-unit cost exceeded what most games could afford.

Delivery Mechanisms

The sprite-scaling forward map permits five distinct delivery mechanisms on period hardware.

The first is dedicated arcade sprite-scaling hardware. The Sega Super Scaler boards of the mid-1980s provided hardware-supported continuous sprite scaling through dedicated graphics chips that handled hundreds of sprites at independent scaling factors per frame. Space Harrier, Out Run, After Burner, and Galaxy Force II all ran on Super Scaler hardware. The mechanism trades dedicated hardware cost for the highest sprite throughput of any technique in the period.

The second is cartridge coprocessor chips on home console cartridges. The Capcom CX4 chip in Mega Man X2 in 1994 and Mega Man X3 in 1995 performed runtime scaling and rotation arithmetic that the standard SNES sprite hardware could not compute on its own. The CX4 returned per-sprite screen positions and orientation parameters that the Super Nintendo Entertainment System Picture Processing Unit applied during sprite rendering. The Super FX chip in Star Fox in 1993 and subsequent titles provided polygon-rendering computations that included sprite scaling for various game elements. The mechanism trades cartridge cost for capabilities exceeding the standard console hardware.

The third is software sprite scaling on a general-purpose central processing unit. The engine computes the scaling factor $s$ and the rotation angle $\theta$ per sprite per frame and applies the affine transform to the sprite bitmap through software pixel-by-pixel interpolation. The mechanism was the universal delivery on the IBM PC running the Microsoft Disk Operating System through the early 1990s and on the early home computers that lacked dedicated sprite-scaling hardware. Modern independent-game engines that emulate the classic sprite-scaling aesthetic typically use software scaling through the central-processing-unit or through graphics-processing-unit shader passes.

The fourth is background-layer-as-sprite using affine background hardware to render a single large sprite. The Super Nintendo Entertainment System Mode 7 background of the previous article could be repurposed as a single large rotating-and-scaling sprite for cinematic effects. Final Fantasy VI used the technique for the airship sequences that combined a Mode 7 ground plane with a Mode-7-rendered airship sprite that the engine treated as a single large sprite rather than as a ground texture. The mechanism extends the Mode 7 capability to objects above or below the ground plane.

The fifth is pre-rendered sprite atlases at multiple discrete sizes and orientations. The engine pre-renders the sprite bitmap at each combination of scaling factor and rotation angle that the gameplay might require and stores the result as a sprite atlas indexed by the gameplay parameters. The runtime renderer selects the appropriate atlas frame without computing the forward map per pixel because the sprite frame already encodes the scaling and rotation. Pole Position used pre-rendered atlases across multiple discrete sizes. After Burner used pre-rendered atlases across multiple rotation angles in addition. The mechanism trades sprite-atlas storage for runtime computational cost.

All five mechanisms compute the same forward map and produce the same visible result. The choice trades hardware cost, sprite-atlas storage, central-processing-unit budget, the maximum sprite count per frame, and the achievable continuous-scale resolution.

Where the Framing Breaks Down

The sprite-scaling framing is insufficient when any of the following conditions hold.

When the sprite represents a genuinely three-dimensional object with visible internal structure that varies with viewing angle, the two-dimensional sprite bitmap is insufficient. Pre-rendered atlases at multiple angles cover the case partially. The synthesis closer of the series treats the full projective rendering that handles three-dimensional geometry without atlas pre-rendering.

When the sprite must depict true perspective foreshortening within its own extent, the affine scaling of this article is insufficient. A long object such as a road sign should foreshorten near-end-large to far-end-small within the sprite, which the uniform scaling factor $s$ cannot produce. The perspective-tilted-sprite variant above addresses this case through full projective rendering.

When many sprites at distinct depths must render simultaneously, the per-sprite forward map imposes a per-sprite cost that the hardware budget may not afford. The Sega Super Scaler boards addressed this through dedicated hardware that could support hundreds of sprites at once. Hardware without comparable support must reduce the visible sprite count or accept a lower frame rate.

When the gameplay requires the player to interact with sprite-internal geometry beyond the rectangular bounding box, the affine sprite-scaling framework provides only the inverse map for the bounding rectangle. The cross-cutting picking article later in the series treats the inverse for sprite-local pixel-level hit testing that respects transparency and per-pixel sprite shape.

When the sprite must occlude or be occluded by a Mode 7 ground plane or a tilemap background, the depth-buffer-aware sort that modern hardware provides displaces the manual Y-sort that period sprite-scaling games used.

The Canon

The following games use sprite scaling as their primary pseudo-three-dimensional technique. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Pole Position in the arcade in 1982 gave the medium its first commercially-released sprite-scaling pseudo-three-dimensional racing game. The pre-rendered scaled-sprite atlases became the visual template for arcade racing for the next decade.

Space Harrier in the arcade in 1985 gave the medium its first hardware-continuous-scaling action shooter through the Sega Super Scaler boards. The high sprite throughput demonstrated what dedicated arcade hardware could deliver in the era.

Out Run in the arcade in 1986 applied Super Scaler hardware to a driving game across a continuous landscape that became the visual standard for arcade driving through the late 1980s.

After Burner in the arcade in 1987 combined sprite scaling with pre-rendered pitch and roll atlases for a flight simulator that the Super Scaler hardware could drive.

Galaxy Force II in the arcade in 1988 demonstrated the upper limit of pre-three-dimensional arcade rendering through cinematic terrain and large boss enemies.

Battle Clash on the Super Nintendo Entertainment System in 1992 brought the Mode 7 background-as-sprite technique to a Super Scope light-gun shooter. The rotating-and-scaling enemy mecha became the canonical case for the sprite inverse map that the cross-cutting picking article treats later in the series.

Metal Combat, Falcon’s Revenge on the same console in 1993 extended the Battle Clash formula with additional enemy types and gameplay scenarios through the same Mode 7 background-as-sprite technique.

Yoshi’s Safari on the same console in 1993 combined a Mode 7 ground-and-environment background with standard pre-rendered sprite atlases for the enemies in a first-person Super Scope shooter across pre-rendered cartoon environments.

Each game in the canon uses sprite scaling with a different combination of variation choices appropriate to the gameplay it supports.

Out of Scope

The article does not cover the following.

True three-dimensional polygon rendering of the kind that Star Fox and later Super FX titles introduced is the subject of the synthesis closer of the series. Sprite scaling is the affine pseudo-three-dimensional approximation that the projective synthesis subsumes.

Raycasting through a two-dimensional grid to produce three-dimensional-looking walls is the subject of the next article in the cluster. Raycasting and sprite scaling both produced pseudo-three-dimensional visuals on hardware of the late 1980s and early 1990s through different mathematics.

The full Y-sort framework including tie-breaking rules and the Z-sort alternative is the subject of a later cross-cutting article. The article treats the Y-sort criterion in its simplest sprite-scaling form.

The full pick-disambiguation framework including the sprite-scale-and-rotate hit-test for the Battle Clash and Metal Combat lineage is the subject of a later cross-cutting article. The article presents the inverse map in its simplest form and defers the full treatment.

The internal architecture of the Sega Super Scaler boards and the SNES cartridge coprocessor chips including CX4 and Super FX is a hardware-design subject that the article treats only at the level of the delivery-mechanism sidebar.

Conclusion

Sprite scaling extends the affine projection family of the previous articles with a per-sprite scaling factor that depends on the sprite’s depth from the camera. The forward map combines a scaling factor $s = f/d$ with a sprite-plane rotation $R(\theta)$ to map the sprite-local pixel coordinate to the screen pixel coordinate. The inverse map returns to the sprite-local frame for hit-testing against the sprite’s bounding rectangle. The technique pre-dates the Mode 7 affine background of the previous article by nearly a decade and provided the visual signature of arcade pseudo-three-dimensional games across the mid-1980s and into the early 1990s. The Battle Clash and Metal Combat lineage brought the technique to home consoles through the Mode 7 background hardware repurposed as a single sprite, and established the canonical inverse-map case for light-gun picking that the cross-cutting picking article treats later in the series. The next article in the cluster treats raycasting as the alternative pseudo-three-dimensional technique that contemporary first-person shooters demonstrated through entirely different mathematics.

References

• Reference, After Burner
• Reference, Battle Clash
• Reference, Galaxy Force II
• Reference, Metal Combat, Falcon’s Revenge
• Reference, Out Run
• Reference, Pole Position
• Reference, Space Harrier
• Reference, Yoshi’s Safari

The opening article of the affine-and-projective cluster treats the Mode 7 graphics feature of the Super Nintendo Entertainment System. Mode 7 provides hardware-accelerated affine transformation of a single tile-based background layer. The hardware applies a two-by-two matrix per rendered scanline, with the matrix updated during the horizontal-blank interval that separates one scanline from the next. By updating the matrix per scanline, the engine produces the appearance of a flat ground plane viewed in perspective from above. The Mode 7 ground plane became the canonical pseudo-three-dimensional rendering technique on sixteen-bit hardware and the visual signature of the racing, flight, and cinematic genres on the Super Nintendo Entertainment System.

The trick is not true three-dimensional rendering. Mode 7 transforms a single two-dimensional texture that the engine treats as a ground plane, rather than projecting genuinely three-dimensional geometry. The illusion of depth arises from the per-scanline scaling that the affine matrix encodes. Distant scanlines toward the horizon use a small scale that makes the texture appear far. Near scanlines toward the bottom of the screen use a large scale that makes the texture appear close. The continuous scale gradient across the rendered frame gives the player the impression of looking forward at a ground plane that recedes into the distance.

The article frames Mode 7 as a perspective-approximating affine projection where the affine matrix is modulated per scanline to encode the inverse-distance scaling that true perspective projection would produce. The Mode 7 hardware permits arbitrary affine transforms, including rotation and shear, in addition to scaling. The classical Mode 7 ground plane combines the per-scanline scaling with a rotation that the player or the engine can update to turn the apparent view. The article works through the per-scanline matrix in detail and walks a concrete F-Zero-like example.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a per-scanline two-by-two affine matrix applied as a backward sampling map from screen coordinates to texture coordinates. The delivery mechanism is the Super Nintendo Entertainment System hardware that the SNES designers built specifically for the technique, the horizontal-blank interrupt that the engine uses to update the matrix per scanline, and the software emulation that other contemporary systems used to approximate the same effect through central-processing-unit cycles.

A Brief History of the Mode

The Super Nintendo Entertainment System launched in Japan in November 1990 and in North America in August 1991. The hardware shipped with eight background-rendering modes numbered zero through seven, of which Mode 7 was the only one to support per-scanline affine transformation of the single background layer. The mode trades the multiple-background-layer capability of modes zero through six for hardware-accelerated transformation of one background layer.

The Mode 7 launch titles demonstrated the technique across multiple game genres. F-Zero from Nintendo in 1990 gave the medium its canonical Mode 7 racing game. The player piloted a futuristic anti-gravity vehicle at high speed around a banked track rendered as a Mode 7 ground plane that rotated with the player’s steering input. The Mode 7 ground plane gave the track its apparent depth and the rotation gave the player’s view its responsiveness.

Pilotwings from Nintendo in 1990 gave the medium its canonical Mode 7 flight simulator. The player flew a hang glider, a small aeroplane, or a parachuting figure across a Mode 7 landscape that tilted and rotated as the player manoeuvred. Pilotwings demonstrated the full Mode 7 affine capability including tilt simulation through per-scanline matrix variation that exceeded the canonical ground-plane formulation.

Super Castlevania IV from Konami in 1991 applied Mode 7 to dramatic environmental rotation effects within an otherwise side-scrolling platformer. The rotating room sequences displayed the Mode 7 hardware’s general affine capability beyond the ground-plane formulation of the racing and flight genres.

ActRaiser from Quintet and Enix in 1990 combined a side-scrolling action mode with a top-down strategy mode that used Mode 7 rotation to convey movement across the world map. The strategy sections demonstrated Mode 7 as a navigational aid rather than only as a perspective effect.

Super Mario Kart from Nintendo in 1992 extended the F-Zero racing formula into the kart racing subgenre that the title established. The Mode 7 track combined per-scanline scaling with track-aligned scrolling and sprite-based opponents. The Super Mario Kart visual style defined the kart racing subgenre for two decades.

Final Fantasy VI from Square in 1994 used Mode 7 for the airship cinematic sequences that book-ended the major plot beats of the game. The airship sequences combined Mode 7 ground plane with sprite-based cloud and structure overlays to give the player a sense of cinematic movement. The airship visual style became the template for cinematic interludes in subsequent Square role-playing games.

The technique persists in modern indie and retro releases through software emulation of the Mode 7 algorithm on hardware that lacks the original chip support. Modern graphics-processing-unit pipelines render the equivalent effect through true perspective projection of a textured ground plane, which produces a visually similar result through different mathematics.

The Forward Map

The Mode 7 hardware works through backward sampling rather than through a forward map. The renderer iterates over screen pixels and computes the texture coordinate to sample at each pixel. The article therefore treats the backward map as the primary equation and derives the forward map as the inverse where needed.

The world model is a single two-dimensional ground plane at vertical world coordinate $w_y = w_y^{\text{ground}}$ in the $y$-down convention of the previous articles. The ground plane carries a texture that the engine treats as the world content. World positions on the ground are described by the two coordinates $(w_x, w_z)$ giving the lateral and depth coordinates respectively. The camera position is a three-dimensional world coordinate $\mathbf{c} = (c_x, c_y, c_z)$ with the camera height above the ground given by

The camera looks forward at the horizon along the world $w_z$ axis with a yaw rotation angle $\theta$ that the player or the engine can update. The screen coordinate is a two-dimensional pixel position $\mathbf{p}_{\text{screen}} = (s_x, s_y)$ with $W$ and $H$ the screen width and height in pixels. The focal length $f$ is a positive scalar in pixels that the engine chooses to set the field of view. The horizontal field of view subtended by the screen follows from the focal length and the screen width as

A small focal length gives a wide field of view and an aggressive perspective. A large focal length gives a narrow field of view and a flatter perspective.

The horizon screen-y $s_y^{\text{horizon}}$ is the screen row at which the camera ray points horizontally, parallel to the ground plane. A pixel above the horizon $s_y < s_y^{\text{horizon}}$ does not intersect the ground plane and the renderer leaves it untextured. A pixel below the horizon $s_y > s_y^{\text{horizon}}$ intersects the ground at some finite distance from the camera.

The per-scanline scaling factor captures the inverse-distance relationship that perspective projection introduces. For a pixel at screen row $s_y$ below the horizon, the world-space depth from the camera along the camera’s forward axis is

The scaling factor that the per-scanline matrix uses is the world distance per screen pixel at the scanline’s depth,

The factor is proportional to $d(s_y)$ and inversely proportional to the scanline offset from the horizon. Distant scanlines near the horizon have large $z(s_y)$, so one screen pixel covers many world units and the texture content there appears compressed. Near scanlines toward the bottom of the screen have small $z(s_y)$, so one screen pixel covers few world units and the texture content there appears stretched. The continuous variation across screen rows produces the perspective illusion.

The per-scanline gradient of the scale factor varies hyperbolically with the scanline offset,

The gradient is large near the horizon where the scale factor changes rapidly between adjacent scanlines and small toward the bottom of the screen where the scale factor changes slowly. The engine precomputes the per-scanline matrix entries in a lookup table that the horizontal-blank-direct-memory-access transfer writes to the Mode 7 registers between scanlines.

The per-scanline affine matrix that the Super Nintendo Entertainment System applies combines the per-scanline scaling with the camera-yaw rotation,

The matrix has four entries that the SNES designers named $A(s_y) = D(s_y)$ for the scaled cosine, and $B(s_y) = -C(s_y)$ for the scaled sine, matching the rotation-and-scale structure. The engine updates these four entries once per scanline through the horizontal-blank interrupt that fires between consecutive scanlines.

The backward map from screen pixel to world ground position is

Written out by component,

The backward map is the equation the SNES renderer evaluates at every screen pixel below the horizon. The result is the world ground position whose texture content the renderer samples and writes to the frame buffer.

The forward map from world ground position to screen pixel inverts the backward map through the perspective relationship that the backward map encodes. Writing $(\tilde{w}_x, \tilde{w}_z)$ for the camera-frame ground coordinates,

the forward map writes as

The forward map contains a division by $\tilde{w}_z$ that the affine framework of the previous articles does not include. The division is the perspective denominator that the synthesis closer of the series treats in full generality. Mode 7 implements the perspective denominator for the ground plane through the per-scanline scaling. The depth from camera is constant across a scanline, so the per-scanline matrix captures the exact perspective division for any point on the ground plane at that scanline without requiring per-pixel division. The technique is exact for the ground plane and approximate only in the sense that it cannot represent above-ground or below-ground geometry.

The Mode 7 forward map holds only for world points on the ground plane $w_y = w_y^{\text{ground}}$. World content above or below the ground plane falls outside the Mode 7 hardware’s capability. Period games typically rendered above-ground content through the sprite hardware overlaid on top of the Mode 7 background.

The Inverse Map

The backward map of the previous section is the operational inverse map that the Mode 7 hardware computes. Every screen pixel below the horizon recovers a unique ground-plane world position. Above the horizon the backward map returns no ground intersection and the renderer treats the pixel as sky.

The ambiguity that earlier articles treated through known-depth or ground-plane assumptions does not arise in Mode 7 because the world model is two-dimensional rather than three-dimensional. The single ground plane gives a unique world position per pixel without requiring a height assumption.

Picking against the Mode 7 ground plane uses the backward map directly. The clicked screen pixel inverts to a unique world position that the engine compares against gameplay objects positioned in world coordinates.

The visible region of the world on the ground plane is the image of the screen rectangle below the horizon under the backward map. The region is a trapezoid in the $(w_x, w_z)$ world plane, narrow at the back near the horizon and wide at the front near the bottom of the screen. The trapezoid widens linearly with the inverse of $(s_y - s_y^{\text{horizon}})$, which is the canonical perspective foreshortening that gives Mode 7 its visual style.

A Worked Example

Consider an F-Zero-style Super Nintendo Entertainment System racing game with the following parameters. The screen is 256 pixels wide and 224 pixels tall, matching the Super Nintendo Entertainment System resolution. The horizon sits at screen row $s_y^{\text{horizon}} = 80$. The camera height above the ground is $h = 16$ world units. The focal length is $f = 96$ pixels, giving a horizontal field of view $\mathrm{FOV} = 2 \arctan(128/96) \approx 106$ degrees across the screen width. The camera yaw is $\theta = 0$, looking straight ahead along the world $w_z$ axis. The camera position is $\mathbf{c} = (0, w_y^{\text{ground}} - 16, 0)$, placing the camera at the origin of the lateral and depth axes with the camera height $h = 16$ above the ground.

The per-scanline scaling factor at screen row $s_y$ below the horizon is

At screen row $s_y = 100$, just below the horizon, $z(100) = 16/20 = 0.8$, giving a far-field scale where one screen pixel covers $0.8$ world units.

At the bottom of the screen $s_y = 223$, $z(223) = 16/143 \approx 0.112$, giving a near-field scale where one screen pixel covers about a tenth of a world unit.

The depth from camera at each scanline is

At $s_y = 100$, $d = 1536 / 20 = 76.8$ world units. At $s_y = 200$, $d = 1536 / 120 = 12.8$ world units. At $s_y = 223$, $d = 1536 / 143 \approx 10.74$ world units.

The backward map at screen pixel $(128, 100)$ with $\theta = 0$, $\cos\theta = 1$, $\sin\theta = 0$, gives

The pixel at the centre of the screen just below the horizon samples the world position $(0, 76.8)$, which is $76.8$ world units forward of the camera and matches the scanline depth $d(100) = 76.8$.

At the screen edge $(0, 100)$ with the same $\theta = 0$,

The pixel at the upper-left of the visible ground samples world position $(-102.4, 76.8)$, which is $102.4$ units to the left of the camera and $76.8$ units forward.

A rotation of $\theta = \pi/2$ turns the camera 90 degrees clockwise as viewed from above. $\cos\theta = 0$, $\sin\theta = 1$, and the per-scanline matrix becomes

The pixel at the centre of the screen just below the horizon now samples the world position

The same pixel now samples the world position $(-76.8, 0)$, $76.8$ units to the left of the camera along the original lateral axis. The rotation has reoriented the apparent forward direction from the original $+w_z$ axis to the $-w_x$ axis.

The forward map verifies the round trip. A world point at $(0, w_y^{\text{ground}}, 76.8)$ with $\theta = 0$ gives $\tilde{w}_x = 0$, $\tilde{w}_z = 76.8$, and the forward map yields

The forward map returns the screen pixel $(128, 100)$ that the backward map sampled the world position from, confirming the round-trip identity,

where $\varepsilon$ is the floating-point precision of the engine. The identity is exact within the affine ground-plane model.

Variations Within the Mode

The Mode 7 framework admits several variations that engines have explored.

A camera-yaw variant updates $\theta$ each frame based on player input or based on a scripted camera path. F-Zero and Super Mario Kart both use the camera-yaw variant to allow the player to steer the track view. The per-scanline matrix recomputes on every $\theta$ change.

A camera-pitch variant introduces an additional rotation around the lateral axis to allow the camera to tilt forward and backward. The variant modifies the per-scanline scaling factor to depend on the pitch angle in addition to the screen row. Pilotwings uses the camera-pitch variant to simulate the pitching of an aeroplane.

A camera-roll variant introduces a rotation around the forward axis to allow the camera to bank left and right. The variant modifies the per-scanline matrix to include a roll component. F-Zero and Super Mario Kart use camera roll during sharp turns and during track banking to give the player a feeling of vehicle dynamics.

A horizon-translation variant moves the horizon screen row $s_y^{\text{horizon}}$ during play to simulate the camera looking up or down. The variant is mathematically equivalent to the camera-pitch variant but is sometimes implemented as a separate parameter for ease of authoring.

A non-uniform-scale variant allows the per-scanline scaling factor along the lateral and depth axes to differ from each other,

The variant produces stylised perspective effects where horizontal foreshortening differs from depth foreshortening. Period games rarely used the non-uniform variant because the visual asymmetry is unusual to players accustomed to natural perspective.

A texture-scrolling variant animates the ground texture by scrolling the texture content in addition to the per-scanline matrix update. F-Zero uses the texture-scrolling variant to suggest forward motion when the player’s forward velocity exceeds what the camera position change alone would convey.

A ground-tile-deformation variant modifies the per-scanline matrix to include a banking effect that simulates the road tilting left or right without rotating the player’s vehicle. Super Mario Kart uses ground-tile deformation during banked turns to give the appearance of the kart tilting into the curve.

A pure-rotation variant sets the scaling factor to a constant value and varies only the rotation, producing a flat top-down view that rotates with the player. The variant is used in ActRaiser and Super Castlevania IV for navigation and dramatic effect respectively.

Delivery Mechanisms

The Mode 7 forward map permits four distinct delivery mechanisms on period and modern hardware.

The first is the Super Nintendo Entertainment System Mode 7 hardware that the SNES Picture Processing Unit provided natively. The hardware applies the four scanline matrix entries $A, B, C, D$ read from registers during the rendering of each scanline. The engine updates the registers through horizontal-blank-direct-memory-access transfers that fire between scanlines without requiring central-processing-unit involvement during the active rendering period. The mechanism allows real-time per-scanline matrix update at the full frame rate without consuming central-processing-unit budget.

The second is software emulation on contemporary sixteen-bit hardware that lacked the SNES’s affine background hardware. The Sega Genesis, the NEC PC Engine, and other contemporary consoles emulated Mode 7 effects through software that computed the per-scanline matrix on the central processing unit and wrote pre-transformed tile data into the frame buffer. The mechanism produced visually similar results at significant central-processing-unit cost and at reduced frame rates compared to the SNES’s native hardware support.

The third is software composition on a general-purpose central processing unit on an IBM PC running the Microsoft Disk Operating System. The personal-computer ports of SNES Mode 7 games typically implemented the technique through software that computed each screen pixel’s backward-mapped world coordinates on the central processing unit. The mechanism scaled to higher resolutions at proportional central-processing-unit cost.

The fourth is graphics-processing-unit-accelerated perspective projection on modern hardware. The modern engine renders the ground plane as a textured polygon in three-dimensional world space with a perspective camera transform that produces the Mode 7 effect through true perspective projection rather than per-scanline affine approximation. The graphics processing unit handles the perspective division per pixel through its built-in fragment shader. The result is visually indistinguishable from authentic Mode 7 but uses fundamentally different mathematics.

All four mechanisms produce the appearance of a Mode 7 ground plane. The choice trades hardware availability, central-processing-unit budget, implementation complexity, and the achievable resolution and frame rate.

Where the Framing Breaks Down

The Mode 7 framing is insufficient when any of the following conditions hold.

When the world contains significant above-ground or below-ground content that the renderer must depict in true perspective, the single ground plane of Mode 7 is insufficient. Period games handled above-ground content through the sprite hardware overlaid on top of the Mode 7 background. Modern games handle the case through full three-dimensional rendering.

When the ground itself bends or has elevation that varies smoothly, the Mode 7 ground plane formulation is insufficient because the affine matrix encodes a flat ground plane only. The sprite-scaling article later in the series treats the related case of pseudo-three-dimensional sprite scaling on hardware without Mode 7 support.

When the gameplay requires camera positions and orientations beyond the canonical above-ground forward-looking setup, the Mode 7 backward map loses its perspective accuracy. Pilotwings already pushed the boundaries of what Mode 7 could handle through its tilt and roll variants. Games beyond Pilotwings’s flexibility typically migrated to true three-dimensional rendering.

When the player must interact with three-dimensional geometry beyond the ground plane through gameplay actions such as flying obstacles, the Mode 7 framework is insufficient. The sprite-based overlay system that Period Mode 7 games used adds limited three-dimensional gameplay but does not extend the Mode 7 math.

When the rendering must scale to a resolution beyond what the SNES hardware supports, the modern graphics-processing-unit pipeline displaces the classic Mode 7 mechanism. The graphics-processing-unit-accelerated perspective projection mechanism above bridges classic Mode 7 to modern three-dimensional rendering.

The Canon

The following games use Mode 7 as a defining visual feature. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

F-Zero on the Super Nintendo Entertainment System in 1990 gave the medium its canonical Mode 7 racing game. The high-speed banked-track gameplay demonstrated the Mode 7 ground plane at its visual peak.

Pilotwings on the same console in 1990 gave the medium its canonical Mode 7 flight simulator. The full range of Mode 7 variations including tilt and roll appeared in this title.

ActRaiser on the same console in 1990 gave the medium a Mode 7 strategic-map view that combined the action-platformer mode with a top-down strategy mode.

Super Castlevania IV on the same console in 1991 applied Mode 7 to dramatic environmental rotation effects within an otherwise side-scrolling platformer.

Super Mario Kart on the same console in 1992 extended the F-Zero racing formula into the kart racing subgenre. The Super Mario Kart visual style defined the kart racing subgenre for two decades.

Final Fantasy VI on the same console in 1994 used Mode 7 for the airship cinematic sequences that book-ended the major plot beats of the game.

Each game in the canon uses Mode 7 with a different combination of variations appropriate to the gameplay it supports. The forward map and the backward map are the same equations across the canon.

Out of Scope

The article does not cover the following.

True three-dimensional rendering with a perspective camera through a graphics-processing-unit pipeline is the canonical modern technique that the synthesis closer of the series treats. Mode 7 is the affine approximation to perspective projection that the SNES hardware made affordable in 1990.

Sprite scaling along the depth axis to suggest distance-dependent object size is the subject of the sprite-scaling article later in the series. The Mode 7 hardware does not handle sprite scaling and the sprite overlay system of period Mode 7 games used separate hardware support for sprite size variation.

Raycasting through a height-mapped two-dimensional grid to produce three-dimensional-looking walls is the subject of the raycasting article later in the series. Raycasting and Mode 7 produced superficially similar effects on different hardware through entirely different mathematics.

The full perspective generalisation that recovers the perspective denominator for arbitrary three-dimensional geometry is the subject of the synthesis closer. Mode 7 is a per-scanline approximation to the projective synthesis.

The horizontal-blank-direct-memory-access programming required to update the Mode 7 registers between scanlines is a hardware implementation detail that the article treats only at the level of the delivery-mechanism sidebar.

Conclusion

Mode 7 is the Super Nintendo Entertainment System hardware feature that provides per-scanline affine transformation of a single background layer. The per-scanline matrix combines a scaling factor proportional to $h/(s_y - s_y^{\text{horizon}})$ with a yaw rotation $R(\theta)$ to produce the appearance of a flat ground plane viewed in perspective. The hardware updates the matrix entries through horizontal-blank-direct-memory-access transfers that the engine schedules between scanlines. The Mode 7 backward map samples the ground texture at the world position that the scanline matrix recovers from each screen pixel. The forward map contains a perspective denominator that the per-scanline scaling approximates within the affine projection framework. The mode defined the racing, flight, and cinematic genres on the Super Nintendo Entertainment System through F-Zero, Pilotwings, Super Mario Kart, and Final Fantasy VI. The next article in the cluster treats sprite scaling as the pre-Mode 7 lineage of pseudo-three-dimensional rendering that arcade hardware demonstrated in the early 1980s.

References

• Reference, ActRaiser
• Reference, F-Zero
• Reference, Final Fantasy VI
• Reference, Mode 7
• Reference, Pilotwings
• Reference, Super Castlevania IV
• Reference, Super Mario Kart

The second article of the oblique-and-axonometric cluster generalises the previous article’s oblique projection by rotating the world before parallel projection rather than tilting the depth axis on the screen. The result is an axonometric projection in which the three world axes take three distinct screen-space directions at angles that the rotation determines. The article distinguishes three classical variants by the equality of the foreshortening factors on the three projected world axes. Isometric projection has all three foreshortening factors equal and produces the symmetric three-axis view that engineering drawing schools formalised in the nineteenth century. Dimetric projection has two foreshortening factors equal and one different. Trimetric projection has all three different.

The article also distinguishes the strict mathematical isometric from the game-industry isometric that classic action role-playing games used. The strict isometric has foreshortening factor $\sqrt{2/3}$ on each axis and 120-degree angles between projected axes, which requires the height axis to project upward on screen and is incompatible with the $y$-down convention used throughout the series. The article presents an equal-foreshortening isometric variant under the $y$-down convention as the closest practical equivalent. The game-industry isometric has tiles drawn at a two-to-one width-to-height pixel ratio that mathematically classifies as dimetric. The article uses the term game-iso for the latter to avoid the ambiguity that the inherited industry usage produces.

The mode is the canonical visual style of the strategy game, the action role-playing game, and the tactical role-playing game through the 1990s and into the 2000s. The player views a tilted top-down world with diamond-shaped ground tiles and pre-rendered character sprites that face the camera at all times. Buildings, trees, and obstacles rise above the ground along a vertical world axis that projects to a screen vertical direction. The visual style permits a rich environment while keeping the rendering math within the affine-projection family.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a three-dimensional-to-two-dimensional affine map composed of a world-axis rotation, an orthogonal projection, and a non-uniform scale. The delivery mechanism chooses how the engine renders the rotated sprites and resolves the picking ambiguity that the underdetermined inverse map admits.

A Brief History of the Mode

Axonometric projection in technical drawing dates to the early nineteenth century as a generalisation of the older cabinet and cavalier oblique projections. The German engineer William Farish formalised the term in 1822 in a treatise on isometric drawing for engineering education. The technique entered architectural drawing, machine design, and military mapping across the nineteenth and early twentieth centuries.

The video-game adoption of axonometric projection begins in the arcade. Zaxxon from Sega in 1982 is widely cited as the first commercially-released arcade game with an axonometric world. The player piloted a fighter aircraft across an isometric three-dimensional landscape of vertically-stacked enemy installations. The Zaxxon visual style demonstrated the depth conveyed by axonometric in a way that single-axis side-scrolling could not.

Knight Lore from Ultimate Play the Game in 1984 brought axonometric projection to the home-computer adventure game. The original ZX Spectrum release rendered a series of small isometric rooms that the player explored in search of items and exits. Knight Lore established the visual template for British home-computer isometric adventures through the mid 1980s.

The strategy game adopts axonometric in the early 1990s. SimCity 2000 from Maxis in 1993 followed the top-down original SimCity with an isometric city-builder that conveyed the vertical extent of buildings, elevations of terrain, and underground water and transportation networks. SimCity 2000 became the visual template for subsequent city-builder games.

X-COM, UFO Defense from MicroProse in 1994 brought axonometric to the tactical strategy game. The player commanded a squad of soldiers across an isometric battlefield that exposed multi-level buildings, indoor and outdoor environments, and the elevation differences that ranged-combat gameplay required. The X-COM series ran for a decade in its original form and was rebooted with the same axonometric approach in the 2010s.

The mid-1990s see a convergence of the role-playing game and the axonometric visual style. Diablo from Blizzard in 1996 established the axonometric action role-playing game. The player controlled a single hero exploring procedurally-generated dungeons across an isometric world of tile-based corridors and rooms. The Diablo visual style became the dominant convention for the action-role-playing-game subgenre for two decades.

Civilization II from MicroProse in 1996 brought axonometric to the turn-based strategy game. The top-down original Civilization yielded to an isometric tile-based world map that gave the strategy player a richer visual sense of the geography being managed.

Fallout from Interplay in 1997 applied axonometric to the post-apocalyptic role-playing game with detailed pre-rendered backgrounds and small character sprites walking across a wasteland of isometric tile-based ruins.

Age of Empires II from Ensemble Studios in 1999 applied axonometric to the real-time strategy game with large-scale armies moving across an isometric battlefield of mixed-elevation terrain. The Age of Empires series continued in the same visual style through its sequels and remastered re-releases.

The mode persists through the modern independent role-playing game. Titles such as Pillars of Eternity in 2015 and Disco Elysium in 2019 continue the axonometric tradition on modern hardware through graphics-processing-unit-accelerated rendering that emulates the classic pre-rendered-tile visual style.

The Forward Map

The world coordinate is a three-dimensional position $\mathbf{p}{\text{world}} = (w_x, w_y, w_z)$, with the $y$-down convention of the previous articles. The two horizontal world axes are $w_x$ and $w_z$, which together describe positions on the ground plane. The vertical world axis is $w_y$, gravity-aligned downward, matching the screen-down convention of the earlier articles. The screen coordinate is a two-dimensional pixel position $\mathbf{p}{\text{screen}} = (s_x, s_y)$. The camera position is a three-dimensional world coordinate $\mathbf{c} = (c_x, c_y, c_z)$.

The forward map factors as a world-axis rotation $R$, an orthogonal projection $P$, a non-uniform scale $S$, and a screen-centre translation,

where the rotation $R$ orients the world so that the view axis of the camera points along the projection direction, the orthogonal projection $P$ drops the projection-axis coordinate, and the scale $S$ applies the per-axis foreshortening factors that the chosen axonometric variant requires.

In practice the forward map writes as a single two-by-three matrix $M$ acting on the camera-relative world coordinate,

where the columns of $M$ are the screen-space directions of the three world-axis unit vectors,

The first column $(m_{xx}, m_{yx})$ is the screen-space image of the world $x$ unit vector. The second column $(m_{xy}, m_{yy})$ is the screen-space image of the world $y$ unit vector. The third column $(m_{xz}, m_{yz})$ is the screen-space image of the world $z$ unit vector.

The per-axis foreshortening factor is the screen-space magnitude of the corresponding column,

The three variants of axonometric projection correspond to three patterns of equality among the foreshortening factors. Isometric projection sets $f_x = f_y = f_z$. Dimetric projection sets two of the three factors equal and the third different, typically $f_x = f_z \neq f_y$. Trimetric projection sets all three factors different.

In homogeneous form the forward map writes as

The matrix is the three-by-four affine matrix that the homogeneous form requires. The third row carries the homogeneous-coordinate constant.

The equal-foreshortening isometric variant sets all three column magnitudes to a common value $z$,

A unit ground-plane tile projects to a diamond of width $\sqrt{3}\, z$ pixels and height $z$ pixels, giving a $\sqrt{3}!:!1$ tile aspect. A unit cube in world space projects to a regular hexagon with all six sides equal. The equal-foreshortening variant satisfies the isometric property in the foreshortening sense of equal screen-space lengths for the three world unit vectors. Strict mathematical isometric also requires the three projected world-axis directions to lie at 120-degree intervals on the screen, which the $y$-down convention used throughout the series does not permit because the height axis must project downward to match the gravity direction. The article presents the equal-foreshortening variant as the closest practical equivalent of strict mathematical isometric under the series convention.

The game-industry isometric variant, which the article calls game-iso, uses a tile width-to-height pixel ratio of two to one,

with $z$ giving the half-tile-width in pixels. A unit cell on the ground plane projects to a diamond tile of width $2z$ pixels and height $z$ pixels. The horizontal-axis foreshortening factors are $f_x = f_z = z\sqrt{5}/2$. The vertical-axis foreshortening factor is $f_y = z$. The variant is dimetric, with the two horizontal axes sharing a foreshortening factor that differs from the vertical-axis factor by a ratio of $\sqrt{5}/2$.

The general dimetric forward map relaxes the two-to-one tile aspect to an arbitrary aspect $r$,

Setting $r = 2$ recovers game-iso. Setting $r = \sqrt{3}$ recovers a tile aspect close to strict isometric but with the two horizontal axes treated symmetrically with each other rather than with the vertical axis.

The general trimetric forward map allows three independent column scalings,

where $z_x$, $z_y$, $z_z$ are the per-axis foreshortening factors and $\alpha$, $\gamma$ are the angles of the horizontal axes below the screen horizontal. The trimetric variant provides three independent scalings and two independent angles for a total of five free parameters. Period games rarely used the full trimetric flexibility.

The factorisation of the axonometric forward map into rotation, projection, and scale generalises the previous article’s factorisation of the oblique forward map by replacing the depth-axis shear with a full world-axis rotation. The oblique forward map is a special case of axonometric in which the rotation leaves the world $x$ and $y$ axes parallel to the screen and only the $w_z$ axis takes a screen-space angle. Setting the rotation to identity in axonometric recovers the side-scrolling forward map where the depth axis is the projection direction.

When a character jumps, the height-above-ground convention of the decoupled-vertical-axis article applies unchanged. Writing $h \geq 0$ for the character’s height above the ground and $w_y^{\text{ground}}$ for the ground-plane vertical world coordinate, the airborne character’s $w_y$ is

The sprite-shadow decomposition writes the airborne-sprite screen position as the shadow screen position shifted upward by the height-driven offset,

In game-iso the column for world $y$ points straight downward on the screen, so the shift reduces to

The shadow-drop convention remains identical in form to the belt-scroll and decoupled-vertical-axis cases.

The Inverse Map

The forward map’s matrix takes three world inputs $(w_x, w_y, w_z)$ and produces two screen outputs $(s_x, s_y)$. The system is underdetermined, as in the oblique and belt-scroll articles.

Solving the forward map for the world coordinates in terms of the screen coordinates yields two linear combinations $m_{xx} w_x + m_{xy} w_y + m_{xz} w_z$ and $m_{yx} w_x + m_{yy} w_y + m_{yz} w_z$ that the screen pixel determines. The three world coordinates individually are not determined. The pre-image of a screen pixel is a line in three-dimensional world space, the picking ray of axonometric projection.

The engine resolves the ambiguity through one of three conventions that the previous articles introduced.

The first is the ground-plane assumption. The engine treats the click as referring to the ground at $w_y = w_y^{\text{ground}}$. Substituting and solving yields a system of two equations in the two unknowns $w_x$ and $w_z$,

where $m_{\text{col-}y} = (m_{xy}, m_{yy})^T$. The two-by-two matrix on the left is invertible as long as the projected $x$ and $z$ axes are not collinear on the screen. For game-iso and the equal-foreshortening isometric variant the two axes are non-collinear by construction and the inverse exists.

The second is the known-vertical-coordinate assumption. The engine treats the click as referring to an object at a known $w_y = w_y^{\text{known}}$. The same two-by-two inverse applies with $w_y^{\text{ground}}$ replaced by $w_y^{\text{known}}$.

The third is the screen-space sprite hit test. The engine ignores the world coordinates entirely and tests the clicked screen pixel against the screen-space bounding rectangles of every visible sprite.

For game-iso the ground-plane inverse simplifies because the world $y$ column is $(0, z)$ and the ground-plane equation reduces to a simpler form. Substituting and solving yields

The inverse is fully closed-form under the ground-plane assumption. The engine evaluates the inverse per click to identify the ground tile that the player selected.

The visible region of the world on the ground plane is a parallelogram in the $(w_x, w_z)$ plane, the image of the screen rectangle under the per-pixel ground-plane inverse map. The shape of the parallelogram depends on the angles $\alpha$ and $\gamma$ between the projected axes and the screen horizontal. For game-iso the parallelogram is a rhombus because the two projected horizontal axes have equal magnitude and symmetric angles about the vertical screen axis.

A Worked Example

Consider a game-iso strategy game with the following parameters. The screen is 800 pixels wide and 600 pixels tall. The zoom factor is $z = 32$, giving a 64-by-32-pixel diamond tile. The ground-plane vertical world coordinate is $w_y^{\text{ground}} = 0$. The camera position is $\mathbf{c} = (0, 0, 0)$. The screen-centre offset is $\mathbf{o} = (400, 300)$.

The forward-map matrix is

A tile at world position $(0, 0, 0)$ projects to screen $(0, 0)$ plus the screen-centre offset, giving screen $(400, 300)$.

A tile at world position $(1, 0, 0)$, one unit east of the origin, projects to

A tile at world position $(0, 0, 1)$, one unit south of the origin, projects to

A tile at world position $(1, 0, 1)$ projects to

The four tiles $(0,0,0)$, $(1,0,0)$, $(1,0,1)$, $(0,0,1)$ project to the diamond $(400, 300)$, $(432, 316)$, $(400, 332)$, $(368, 316)$, which has width 64 pixels and height 32 pixels, matching the $2!:!1$ game-iso aspect.

A building at world position $(0, -2, 0)$, two units above the ground at the origin, projects to

The building base coincides with the tile origin’s screen position $(400, 300)$, and the building roof at world $y = -2$ renders 64 pixels above the base at screen $(400, 236)$.

A jumping character at world position $(0, -1, 0)$, one unit above the ground at the origin, projects to

The character renders 32 pixels above the ground-tile centre, matching the height value $h = 1$ through the height-to-screen offset $z h = 32$.

Click-picking at screen $(432, 316)$ under the ground-plane assumption with $w_y^{\text{ground}} = 0$ returns

The ground-plane inverse returns world tile $(1, 0, 0)$, matching the tile that the click targeted.

The round-trip identity holds along the ground plane,

where $\varepsilon$ is the floating-point precision of the engine.

The Y-sort criterion of the belt-scroll and oblique articles extends to axonometric projection through the same back-to-front ground-projection screen-y comparison. For an axonometric projection with matrix $M$, the ground-projection screen-y is a linear combination of the two horizontal world coordinates plus a constant that does not affect ordering. The variable part of the sort key is

The criterion writes as the comparison inequality

For game-iso, the equality $m_{yx} = m_{yz} = z/2$ reduces the sort key to $\frac{z}{2}(w_x + w_z)$, proportional to the diagonal coordinate $w_x + w_z$. For the equal-foreshortening isometric variant, the same equality holds and gives the same diagonal sort key. For general dimetric and trimetric variants, the sort key uses the matrix coefficients appropriate to the chosen forward map.

Variations Within the Mode

The axonometric framework admits several variations that engines have explored.

An equal-foreshortening isometric variant uses the matrix with $\sqrt{3}!:!1$ tile aspect and equal column magnitudes. The variant is mathematically clean but produces a tile aspect that does not align cleanly with integer pixel grids. Engineering drawings use the strict variant of mathematical isometric for measurement accuracy and accept the constraint that the height axis projects upward, which game graphics typically invert to the $y$-down convention used here.

A game-iso variant uses a tile aspect of $2!:!1$ that aligns cleanly with the integer pixel grid. The variant is dimetric mathematically but is universally called isometric in the game industry. The vast majority of period role-playing games and strategy games used game-iso.

A general dimetric variant chooses an arbitrary tile aspect $r$ to produce a stylised view that the designer judges visually appealing. Period games rarely used arbitrary $r$ because the $2!:!1$ aspect of game-iso proved a strong attractor.

A trimetric variant provides three independent foreshortening factors and two independent axis angles. The variant gives the designer full control over the screen-space rotation of the world. Period games rarely used trimetric because the asymmetric appearance is unfamiliar to players accustomed to isometric.

A free-rotation variant permits the player or the engine to rotate the world around the vertical axis during gameplay. The variant requires the engine to recompute the forward map matrix $M$ at each camera-rotation event. Sprites must be pre-rendered at multiple rotation angles for the rotation to look consistent. The Sims from Maxis in 2000 used a four-step rotation of 90-degree increments around the vertical world axis to let the player see different sides of a building.

A free-zoom variant permits the player or the engine to change the zoom factor $z$ during gameplay. The variant requires the engine to recompute the forward map matrix $M$ at each zoom event. Sprites either pre-rendered at multiple sizes or scaled at render time to match the new zoom.

A multi-tier-elevation variant permits the ground plane $w_y^{\text{ground}}$ to vary across the world as a function of $(w_x, w_z)$. Hills, valleys, and stair-stepped buildings all use the multi-tier variant. The forward map is unchanged and the engine simply uses the local $w_y^{\text{ground}}$ when rendering each tile. SimCity 2000, X-COM, and Age of Empires II all use multi-tier elevation.

A pseudo-three-dimensional sprite variant pre-renders character sprites at multiple facing angles that match the axonometric viewing angles. A character facing north-east in the world displays the north-east-facing sprite without any runtime sprite rotation. The variant trades sprite memory for runtime performance.

A two-and-a-half-dimensional building variant renders buildings as two-dimensional sprites positioned at the building’s ground location and Y-sorted with the other objects. The variant gives the appearance of three-dimensional buildings without requiring true three-dimensional rendering.

Delivery Mechanisms

The axonometric forward map permits five distinct delivery mechanisms on period hardware.

The first is pre-rendered tile graphics. The engine pre-renders each ground tile already in its axonometric-projected form and stores the result as a tile atlas. The runtime renderer positions the pre-rendered tile at the camera-relative screen position without computing the forward map per pixel. The mechanism is the dominant delivery for classic role-playing games and strategy games including Diablo, Fallout, X-COM, and SimCity 2000.

The second is sprite-on-tile-background with per-sprite Y-sort. The engine renders moving objects as object-attribute-memory sprites at engine-computed axonometric screen positions above the pre-rendered tile background. The sprite priority is set per sprite to match the Y-sort order that the axonometric world requires.

The third is software composition on a general-purpose central processing unit. The engine maintains a frame buffer and writes each visible object into the frame buffer in Y-sort order. The mechanism was the universal delivery on the IBM PC running the Microsoft Disk Operating System through the late 1980s and the 1990s, and remains the dominant mechanism on modern independent-game engines that render to a software frame buffer.

The fourth is pre-rendered sprite atlases at multiple facing angles. The engine pre-renders character animations at four or eight rotation angles around the vertical world axis and selects the appropriate atlas frame based on the character’s current facing. The variant supports the pseudo-three-dimensional sprite style without runtime sprite rotation. Period role-playing games and tactical games commonly used eight-direction sprite atlases.

The fifth is graphics-processing-unit-accelerated quad rendering. Modern game engines such as Unity, Godot, and Unreal render the axonometric world as textured quads in three-dimensional world space with a camera transform that produces the desired axonometric projection. The graphics processing unit performs the depth sort through its built-in depth buffer and produces the screen image. The depth buffer enforces back-to-front rendering at the rasterisation level without requiring the engine to maintain a per-frame sprite sort list.

All five mechanisms compute the same forward map and produce the same visible result. The choice trades implementation complexity, the tile-atlas storage budget, the central-processing-unit budget for runtime sort, the sprite-atlas storage budget, and the achievable frame rate.

Where the Framing Breaks Down

The axonometric framing is insufficient when any of the following conditions hold.

When the depth axis must show true perspective foreshortening proportional to inverse distance, the affine projection of this article is insufficient. The synthesis closer of the series treats the full projective generalisation. The axonometric projection preserves the world parallelism that perspective projection breaks for visual realism.

When the camera position must move to follow the player along the depth axis in addition to the lateral axes, the rotation $R$ must update with the camera. The article treats $R$ as constant for a given scene and defers the dynamic-camera case to the synthesis closer.

When the world has non-flat ground with significant elevation variation, the multi-tier-elevation variant above extends the axonometric projection to support per-tile ground elevation. The forward map remains affine but the rendering pipeline must sort tiles by their per-tile elevation in addition to lateral position.

When the gameplay requires camera rotation during play, the engine must recompute the forward map matrix $M$ at each rotation event, and sprites must be pre-rendered at multiple rotation angles for visual consistency. The free-rotation variant above treats this case.

When the gameplay requires sprite scaling with depth to suggest distance-from-camera proportional to size, the affine projection of this article is insufficient. The sprite-scaling article later in the series treats the depth-dependent sprite scaling case.

When the world is genuinely three-dimensional with arbitrary mesh geometry rather than flat tiles with vertical extents, the modern three-dimensional rendering pipeline displaces the classic axonometric mechanism. The graphics-processing-unit-accelerated quad rendering mechanism above bridges classic axonometric to modern three-dimensional rendering.

The Canon

The following games use axonometric projection as their primary projection mode. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Zaxxon in the arcade in 1982 gave the medium its first commercially-released axonometric arcade game through a vertical-scrolling shooter over an isometric three-dimensional landscape.

Knight Lore on the ZX Spectrum in 1984 brought axonometric projection to the home-computer adventure game through small isometric rooms and a series of progress-blocking puzzles.

SimCity 2000 on the IBM PC and Apple Macintosh in 1993 gave the city-builder genre its canonical isometric template through deep terrain elevation, underground networks, and a wide repertoire of buildings.

X-COM, UFO Defense on the IBM PC in 1994 brought axonometric to the tactical strategy game through multi-level buildings, indoor and outdoor environments, and the elevation differences that ranged-combat gameplay required.

Diablo on Microsoft Windows in 1996 established the axonometric action role-playing game through procedurally-generated dungeons and tile-based corridors. The Diablo visual style became the dominant convention for the action role-playing subgenre for two decades.

Civilization II on the IBM PC in 1996 brought axonometric to the turn-based strategy game through an isometric tile-based world map that the prior top-down Civilization had not used.

Fallout on Microsoft Windows in 1997 applied axonometric to the post-apocalyptic role-playing game through detailed pre-rendered backgrounds and small character sprites walking across the wasteland.

Age of Empires II on Microsoft Windows in 1999 applied axonometric to the real-time strategy game through large-scale armies on mixed-elevation isometric terrain.

Each game in the canon uses some variant of the axonometric forward map. Most use game-iso with $2!:!1$ tile aspect. The differences lie in the foreshortening factor $z$, the per-tile elevation handling, the camera policy, and the choice of delivery mechanism appropriate to the target hardware.

Out of Scope

The article does not cover the following.

True perspective projection with a perspective denominator is the subject of the synthesis closer of the series. Axonometric projection is a parallel projection without perspective foreshortening.

Sprite scaling along the depth axis is the subject of the sprite-scaling article later in the series. The classic axonometric projection does not scale sprites with depth.

The full Y-sort framework including tie-breaking rules and the Z-sort alternative is the subject of a later cross-cutting article. The article treats the Y-sort criterion in its simplest axonometric form.

The full pick-disambiguation framework including the sprite-scale-and-rotate hit-test is the subject of a later cross-cutting article. The article presents the three baseline conventions parallel to the previous articles’ treatments.

The dynamic-camera case where the rotation $R$ and the camera $\mathbf{c}$ update during play is treated as a special case in the variations above but is not the article’s primary concern.

The choice between axonometric and true three-dimensional rendering for a given game is a design question the series does not adjudicate. Each visual style has games for which it is the right choice and games for which it is the wrong choice.

Conclusion

Axonometric projection generalises the previous article’s oblique projection by rotating the world before parallel projection rather than tilting only the depth axis on the screen. The forward map factors as a world-axis rotation, an orthogonal projection, and a non-uniform scale. The three classical variants are isometric, dimetric, and trimetric, distinguished by the equality of the foreshortening factors. The game-industry isometric style, universally called isometric in industry usage, is mathematically dimetric with a $2!:!1$ tile aspect. The equal-foreshortening isometric variant uses a $\sqrt{3}!:!1$ tile aspect under the $y$-down convention. Strict mathematical isometric with 120-degree intervals between projected axes requires the height axis to project upward on screen and is incompatible with the gravity-aligned $y$-down convention that the series uses throughout. The mode reached its visual peak on the personal-computer role-playing games and strategy games of the 1990s through pre-rendered tile graphics and pre-rendered sprite atlases. The next article in the series opens the affine-and-projective cluster with the Mode 7 affine ground plane that the Super Nintendo Entertainment System pioneered.

References

• Reference, Age of Empires II
• Reference, Civilization II
• Reference, Diablo
• Reference, Fallout
• Reference, Knight Lore
• Reference, SimCity 2000
• Reference, The Sims
• Reference, X-COM, UFO Defense
• Reference, Zaxxon

The opening article of the second cluster moves from the Cartesian projections of the previous cluster to the oblique projection that tilts the depth axis at an angle of the engine’s choosing. The world remains a three-dimensional grid of objects at world positions $(w_x, w_y, w_z)$. The screen remains a two-dimensional pixel grid. The forward map gains a depth-mixing direction controlled by an angle $\phi$ and a foreshortening factor $\alpha$ that the engine chooses to produce the visual style the game requires. Belt-scroll from the previous article is the limit case where the depth-mixing direction points straight up the screen. Oblique projection in this article permits the depth-mixing direction to point in any direction on the screen, producing a richer family of projection styles and the foundation for axonometric projection that the next article treats.

The quarter view of the article title refers to the visual style that historic engineering drawing calls oblique projection and that game designers have used since the early arcade era to render a small world strip with apparent depth. The quarter view shows three faces of a cuboid object simultaneously, the front face directly, the top face foreshortened upward, and the side face foreshortened to the side. The viewer reads the depth axis as the offset between the front face and the corresponding edges of the top and side faces. The depth-mixing direction is the screen-space direction of that offset.

The article distinguishes two classical variants, cavalier projection that preserves the depth lines at their full world length and cabinet projection that halves the depth lines. Cavalier preserves measurement accuracy at the cost of an artificially deep appearance. Cabinet sacrifices measurement accuracy for a more naturalistic appearance. The Worked Example treats both variants in turn.

The framing the series carries from the opener distinguishes the projection math from the delivery mechanism. The projection math is a three-dimensional-to-two-dimensional affine map with an oblique-shear factor. The delivery mechanism chooses how the engine renders the depth-shifted sprites and resolves the picking ambiguity that the underdetermined inverse map admits.

A Brief History of the Mode

Oblique projection in technical drawing dates to the seventeenth century as a convention for depicting fortifications and machinery. The eighteenth-century engineering schools formalised the cabinet and cavalier variants under those names. The technique entered industrial design and architectural drawing in the nineteenth and twentieth centuries as a parallel-projection alternative to perspective drawing.

The video-game adoption of oblique projection arrives with the personal-computer era. Habitat on the Commodore 64 in 1986 gave the mass-market personal computer its first networked persistent world with an oblique-projection visual style. Avatars and rooms rendered in a cabinet-style oblique view that conveyed the depth of each room without departing from the limited graphics capabilities of the Commodore 64.

Populous from Bullfrog in 1989 gave the genre of strategy games its canonical cabinet-oblique terrain. The player viewed a small section of a deformable terrain from a fixed cabinet angle that tilted the depth axis into the upper-left of the screen. The Populous visual style defined the god-game subgenre that the studio later extended through Powermonger and the Bullfrog catalogue.

Ultima VII, The Black Gate from Origin in 1992 brought oblique projection to the mass-market personal-computer role-playing game. The Avatar walked across a continuous oblique-projected world where each tile was an oblique cuboid and where building walls and trees rendered with cabinet-style depth shifts that the player read as depth. The Ultima VII engine became the visual template for several Origin titles across the early 1990s.

The technique recedes from the canon after the mid-1990s as the personal-computer industry moves to hardware-accelerated three-dimensional rendering. Games that retained a tilted-view aesthetic moved predominantly to the axonometric and isometric styles of the next article in the cluster. The axonometric style that the next article treats continued to flourish through the late nineties in strategy and role-playing games that found the rotational symmetry of axonometric more useful than oblique.

The Forward Map

The world coordinate is a three-dimensional position $\mathbf{p}{\text{world}} = (w_x, w_y, w_z)$, with the $y$-down convention of the previous articles. The screen coordinate is a two-dimensional pixel position $\mathbf{p}{\text{screen}} = (s_x, s_y)$. The camera position is a three-dimensional world coordinate $\mathbf{c} = (c_x, c_y, c_z)$. The zoom factor is a positive scalar $z$. The depth-mixing direction is parameterised by an angle $\phi$ in the open interval $(0, \pi)$, giving the screen-space direction in which the depth axis tilts. The foreshortening factor is a positive scalar $\alpha$ giving the screen-space length of one unit of depth along the depth-mixing direction.

The forward map is

where $W$ and $H$ are the screen width and height in pixels.

Writing $\mathbf{d}(\phi) = (\cos\phi,\ -\sin\phi)$ for the screen-space depth-direction unit vector, the forward map writes in vector form as

The first term is the side-scrolling forward map of the second cluster article applied to the camera-relative lateral and vertical coordinates. The second term is the depth-driven screen-space shift along the direction $\mathbf{d}(\phi)$ with magnitude $\alpha z |w_z - c_z|$.

The sign convention matches the $y$-down screen convention. The negative sign on the $\sin\phi$ term ensures that increasing depth $w_z$ shifts the screen position upward on the screen, which the visual reader expects of objects further back in the world.

The screen-space separation between two world objects at the same lateral and vertical coordinates but different depths follows from the linearity of the forward map,

The separation is parallel to $\mathbf{d}(\phi)$, has magnitude $\alpha z |w_{z,1} - w_{z,2}|$, and is independent of the lateral and vertical coordinates. The depth-direction screen-shift is the operational reading of the foreshortening factor $\alpha$ and the depth angle $\phi$.

In homogeneous form the forward map writes as

The matrix is a three-by-four affine matrix acting on the four-component augmented world coordinate. The matrix is the parallel projection from the three-dimensional world onto the two-dimensional screen with the oblique-projection angle that the parameters $\alpha$ and $\phi$ encode.

The factorisation pattern from the opener extends to the oblique case as a composition of a two-axis depth-mixing shear, an orthogonal projection from three dimensions to two, a uniform scale, and a screen-centre translation,

where $K(\alpha, \phi)$ is the three-by-three two-axis shear matrix that adds the depth-driven shifts to the screen-x and screen-y components,

and $P_z$ is the two-by-three orthogonal projection that drops the depth component,

The two-axis shear generalises the single-axis shear of the previous article. The belt-scroll case is the limit $\phi = \pi/2$ where the depth-mixing direction points straight up the screen and $\cos\phi = 0$ zeroes the screen-x contribution of the depth axis. Setting $\alpha = \beta$ and $\phi = \pi/2$ recovers the belt-scroll forward map of the previous article exactly.

The cavalier and cabinet variants correspond to specific choices of $\alpha$. The cavalier variant sets $\alpha = 1$ and preserves the world length of depth lines in the screen image. A unit depth shift appears as a unit pixel shift along the depth-mixing direction. The depth axis appears stretched compared to natural perspective since true perspective foreshortens distant objects. The cavalier variant is appropriate for engineering drawings where measurement accuracy matters and for game scenes where the designer wants the depth axis to feel pronounced.

The cabinet variant sets $\alpha = 1/2$ and halves the screen-space length of depth lines relative to their world length. A unit depth shift appears as a half-pixel shift along the depth-mixing direction. The depth axis appears foreshortened in a way that approximates true perspective more closely than cavalier does without abandoning the affine-projection family. The cabinet variant is appropriate for game scenes where the designer wants a more naturalistic depth appearance.

Other choices of $\alpha$ are uncommon but mathematically valid. A value of $\alpha = 2/3$ sits between cavalier and cabinet and is sometimes used when the designer wants slightly more depth pronouncement than cabinet provides. The article treats the cavalier and cabinet cases as the canonical variants.

The angle $\phi$ typically takes values of $\pi/6$, $\pi/4$, or $\pi/3$ matching 30, 45, or 60 degrees from the screen-horizontal axis. The most common quarter-view angle in classic games is $\pi/4$, producing a depth axis that tilts at 45 degrees on the screen.

When a character jumps, the height-above-ground convention of the decoupled-vertical-axis article applies unchanged. Writing $h \geq 0$ for the character’s height above the ground and $w_y^{\text{ground}}$ for the ground-plane vertical world coordinate, the airborne character’s $w_y$ is

The sprite-shadow decomposition writes the airborne-sprite screen position as the shadow screen position shifted upward by the height-driven vertical offset,

with the magnitude of the vertical screen-space separation

The depth-mixing direction $\mathbf{d}(\phi)$ plays no role in the height shift because the height axis stays parallel to the gravity-aligned screen-y axis regardless of the depth angle. The shadow-drop convention is identical in form to the belt-scroll and decoupled-vertical-axis cases.

The Inverse Map

The forward map’s matrix takes three world inputs $(w_x, w_y, w_z)$ and produces two screen outputs $(s_x, s_y)$. The system is underdetermined, as in the belt-scroll article.

Solving the forward map for the world coordinates in terms of the screen coordinates yields

The two linear combinations $w_x + \alpha w_z \cos\phi$ and $w_y - \alpha w_z \sin\phi$ are uniquely determined. The three world coordinates $(w_x, w_y, w_z)$ individually are not determined. The pre-image of a screen pixel is a line in the three-dimensional world, the picking ray of oblique projection.

The candidate set is therefore

where $u_x = (s_x - W/2)/z + c_x + \alpha c_z \cos\phi$ and $u_y = (s_y - H/2)/z + c_y - \alpha c_z \sin\phi$ are the determined linear combinations. The line is parameterised by the depth $w_z$ along the picking ray.

The engine resolves the ambiguity through one of three conventions parallel to the conventions of the previous article.

The first is the ground-plane assumption. The engine treats the click as referring to the ground at $w_y = w_y^{\text{ground}}$ and solves for $w_z$,

The recovered $w_x$ follows from the first equation,

The convention is appropriate when the click is interpreted as a movement command to walk to a ground position.

The second is the known-depth assumption. The engine treats the click as referring to an object at a known depth $w_z^{\text{known}}$. The recovered $w_x$ and $w_y$ follow from the inverse-map equations with $w_z$ replaced by $w_z^{\text{known}}$.

The third is the screen-space sprite hit test. The engine ignores the world coordinates entirely and tests the clicked screen pixel against the screen-space bounding rectangles of every visible sprite.

The visible region of the world on the ground plane $w_y = w_y^{\text{ground}}$ is the image of the screen rectangle under the per-pixel ground-plane inverse map. The region is a parallelogram in the $(w_x, w_z)$ plane. The depth-axis edge of the parallelogram runs along the world $w_z$ axis with half-extent $H/(2\alpha\, z\, \sin\phi)$ in world units. The lateral edge runs along the world $w_x$ axis with half-extent $W/(2z)$ in world units, shifted at each depth $w_z$ by the depth-dependent screen-space x offset of the ground-plane inverse map. The centre of the parallelogram is the world point that the screen centre $(W/2, H/2)$ inverts to,

The engine clips world content to the visible parallelogram when culling for rendering.

A Worked Example

Consider an oblique-projection role-playing game with the following parameters. The screen is 320 pixels wide and 200 pixels tall, matching a typical late-1980s personal-computer resolution. The zoom factor is $z = 1$, one world unit per pixel. The depth-mixing angle is $\phi = \pi/4$, a 45-degree tilt. The foreshortening factor is $\alpha = 1/2$, the cabinet variant. The ground-plane vertical world coordinate is $w_y^{\text{ground}} = 100$. The camera position is $\mathbf{c} = (200, 80, 0)$. The screen-centre offset is $\mathbf{o} = (160, 100)$.

For $\phi = \pi/4$, $\cos\phi = \sin\phi = \sqrt{2}/2 \approx 0.7071$. The depth-mixing direction tilts equally into screen-x and negative screen-y at the 45-degree angle. A unit of world depth shifts the screen position by $\alpha \cos\phi = 0.5 \cdot 0.7071 \approx 0.3536$ pixels along screen-x and by $-\alpha \sin\phi \approx -0.3536$ pixels along screen-y under cabinet projection.

A character on the ground at world position $(200, 100, 0)$ projects to

The character renders at screen $(160, 120)$.

A character at the back of the playfield at world position $(200, 100, 40)$ projects to

The back character renders at approximately screen $(174, 106)$. The screen position has shifted both to the right and upward compared to the player at world depth zero, matching the 45-degree quarter-view tilt.

A character at the front of the playfield at world position $(200, 100, -20)$ projects to

The front character renders at approximately $(153, 127)$, shifted to the left and downward relative to the player.

The cavalier variant with $\alpha = 1$ doubles the per-unit screen shift of the cabinet variant. The back character at world $(200, 100, 40)$ under cavalier projection would render at approximately $(160 + 28.28,\ 120 - 28.28) = (188, 92)$ instead of $(174, 106)$. The cavalier variant produces a more pronounced depth shift on the screen for the same world depth offset.

The round-trip identity holds along the ground plane under either variant,

where $\varepsilon$ is the floating-point precision of the engine.

The Y-sort criterion of the previous article extends to oblique projection through the same back-to-front ground-projection screen-y comparison. A character at larger $w_z$ has a smaller ground-projection $s_y$ and is drawn earlier in the frame. The criterion writes as

which is the same comparison the belt-scroll article introduced. Oblique projection introduces no new sort criterion beyond what belt-scroll already required.

Variations Within the Mode

The oblique-projection framework admits several variations that engines have explored.

A cavalier-variant scene sets $\alpha = 1$ everywhere. The depth axis appears pronounced in the screen image. The variant is rare in games because the visual stretch is unattractive. Engineering drawings use cavalier where measurement accuracy is required.

A cabinet-variant scene sets $\alpha = 1/2$ everywhere. The depth axis appears foreshortened. The variant is the dominant choice in oblique-projection games.

A mixed-angle scene varies $\phi$ between scenes to reflect changes in the world geometry or in the camera orientation. The variant is rare in classic games because the discontinuity in depth-mixing direction disorients the player.

A per-object foreshortening variant allows each object to use its own $\alpha$ that the engine chooses based on the object class. Small foreground objects might use $\alpha = 1$ to emphasise their position while large background structures use $\alpha = 1/2$ to recede into the scene. The variant breaks the projection invariant and is mathematically inconsistent but appears in stylised games that prioritise visual hierarchy over geometric accuracy.

A two-axis foreshortening variant allows different foreshortening factors along the $w_z$ axis and along an additional rotated axis. The variant approaches axonometric projection where the world axes themselves take rotated screen-space angles. The next article in the cluster treats the axonometric generalisation.

A non-zero $w_y$ contribution from the depth axis appears when the camera tilts to look down at the scene in addition to looking obliquely. The variant combines oblique projection with a tilted camera frame and produces an effect midway between cabinet oblique and isometric. The article treats it as a generalised oblique with two independent shear angles.

A wrapping world boundary applies the modulo operation to the world coordinates before the forward map, producing an infinite oblique terrain from a small content tile. Strategy games such as Populous use the wrapping variant to render a much larger apparent world than the world data holds.

Delivery Mechanisms

The oblique-projection forward map permits five distinct delivery mechanisms on period hardware.

The first is software composition on a general-purpose central processing unit. The engine computes each sprite’s screen position through the forward map and writes the sprites to the frame buffer in Y-sort order. The mechanism was the universal delivery on the IBM PC running the Microsoft Disk Operating System through the late 1980s and early 1990s and on the personal-computer titles of the era including Populous and Ultima VII.

The second is sprite-on-tile-background with pre-tilted tile graphics. The engine paints each background tile already in its oblique-projected form into a tile-grid background layer that the hardware scrolls without further per-tile transformation. The sprites for moving objects sit above the background at engine-computed oblique screen positions. The mechanism saves runtime computation at the cost of tile-set storage for the pre-tilted tiles. Period console oblique games used this technique.

The third is line-by-line oblique scroll modulation. The engine updates the horizontal scroll register during the horizontal-blank interrupt to produce a per-scanline horizontal shift that simulates the oblique projection for a single-layer background. The technique appears on the Nintendo Entertainment System in games that wanted oblique effects but had only the single-background-layer hardware. The mechanism is rare in pure oblique games and more common in stylised hybrids.

The fourth is pre-rendered oblique sprite atlases. The engine pre-renders each sprite at the oblique angle and stores the result as a sprite atlas. The runtime renderer draws the appropriate atlas frame without computing the forward map per frame because the sprite frame already encodes the oblique transformation. The mechanism is common in console role-playing games of the early 1990s where the central-processing-unit budget could not support runtime sprite tilting.

The fifth is graphics-processing-unit-accelerated quad rendering. Modern game engines such as Unity, Godot, and Unreal render the oblique world as textured quads in three-dimensional world space with a camera transform that produces the desired oblique projection. The graphics processing unit performs the depth sort through its built-in depth buffer and produces the screen image. The depth buffer enforces the back-to-front sort at the rasterisation level without requiring the engine to maintain a per-frame sprite sort list.

All five mechanisms compute the same forward map and produce the same visible result. The choice trades implementation complexity, the tile-set storage budget, the central-processing-unit budget for runtime transformation, the sprite-atlas storage budget, and the achievable frame rate.

Where the Framing Breaks Down

The oblique-projection framing is insufficient when any of the following conditions hold.

When the world axes themselves take rotated screen-space angles rather than remaining parallel to the screen, the projection becomes axonometric and the next article in the cluster treats the case. Oblique projection keeps the world’s $x$ and $y$ axes parallel to the screen. Axonometric projection rotates the world axes so that all three world axes take different screen-space angles.

When the depth axis must show true perspective foreshortening proportional to inverse distance, the affine projection of this article is insufficient. The sprite-scaling article later in the series treats the depth-dependent sprite scaling case. The synthesis closer of the series treats the full projective generalisation.

When the camera tilts so steeply that the floor case becomes a top-down view, the foreshortening factor $\alpha$ approaches $1$ and the depth axis collapses into screen y. The article treats this as the belt-scroll limit already covered in the previous article.

When the camera rotates around the depth axis, the depth-mixing direction $\phi$ rotates with it, which the article treats as a per-frame change in the forward map’s $\phi$ parameter. The variant is uncommon but mathematically straightforward.

When the world has a non-flat ground, the oblique projection still applies to each point but the visible terrain deviates from the simple ground rectangle of the worked example. The engine renders the terrain through a triangulated mesh that the forward map projects on a per-vertex basis.

The Canon

The following games use oblique projection as their primary projection mode. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Habitat on the Commodore 64 in 1986 gave the personal computer its first networked-persistent-world social game through a cabinet-oblique room-and-avatar rendering style. The world was a set of rooms each rendered as a cabinet-oblique cuboid that the avatars walked around in under the constraints of the Commodore 64’s graphics.

Populous from Bullfrog in 1989 gave the strategy genre its canonical cabinet-oblique terrain and the god-game subgenre. The deformable terrain rendered as a small portion of a wrapping world map through a cabinet projection that tilted the depth axis into the upper-left of the screen.

Ultima VII, The Black Gate from Origin in 1992 brought oblique projection to the mass-market personal-computer role-playing game through a continuous oblique-projected world of tile-based oblique cuboids, trees and buildings with cabinet-style depth shifts, and an interactive object system that became the visual template for several subsequent Origin titles.

Each game in the canon uses an oblique projection with a different choice of $\phi$ and $\alpha$, a different camera policy, and a different choice of delivery mechanism appropriate to the target hardware.

Out of Scope

The article does not cover the following.

Axonometric projection where the world axes themselves take rotated screen-space angles is the subject of the next article in the cluster. Isometric, dimetric, and trimetric variants all fall under the axonometric framework that the article does not treat here.

True perspective projection with a perspective denominator is the subject of the synthesis closer of the series. Oblique projection is a parallel projection without perspective foreshortening.

Sprite scaling along the depth axis is the subject of the sprite-scaling article later in the series. The classic oblique projection does not scale sprites with depth.

The full Y-sort framework including tie-breaking rules and the Z-sort alternative is the subject of a later cross-cutting article. The article treats the Y-sort criterion in its simplest form matching the belt-scroll article’s treatment.

The full pick-disambiguation framework including the sprite-scale-and-rotate hit-test is the subject of a later cross-cutting article. The article presents the three baseline conventions parallel to the belt-scroll article’s treatment.

The choice between oblique and isometric for a given game is a design question the series does not adjudicate. Each projection has games for which it is the right choice and games for which it is the wrong choice.

Conclusion

Oblique projection extends the belt-scroll forward map of the previous article by allowing the depth-mixing direction to take any screen-space angle. The forward map gains a two-axis shear factor controlled by an angle $\phi$ and a foreshortening factor $\alpha$. The cavalier and cabinet variants correspond to $\alpha = 1$ and $\alpha = 1/2$ respectively, trading measurement accuracy for naturalistic appearance. The inverse map is underdetermined and admits the three disambiguation conventions that the belt-scroll article introduced. The Y-sort criterion applies unchanged because both belt-scroll and oblique projection share the same depth-to-screen-y relationship. The mode reached its visual peak on the personal-computer titles of the early 1990s before the industry moved to hardware-accelerated three-dimensional rendering. The next article in the cluster generalises the oblique-shear matrix to a full axonometric rotation in which all three world axes take distinct screen-space angles. Isometric, dimetric, and trimetric projection are the cases the next article distinguishes.

References

• Reference, Habitat
• Reference, Populous
• Reference, Ultima VII, The Black Gate

The third article in the Cartesian cluster adds an explicit depth axis to the side-scrolling view. The world becomes a strip of finite depth along which the player walks toward or away from the camera while also moving laterally across the strip. The screen shows the strip from a slightly elevated angle that maps the depth axis onto a vertical screen offset. A character at the back of the strip appears higher on the screen. A character at the front of the strip appears lower on the screen. The forward map gains a depth-to-screen-y term controlled by a slope coefficient that quantifies how steeply the depth axis tilts into the rendered image. The inverse map ceases to return a unique world point and instead returns a family of candidates indexed by the depth-or-height assumption that the engine commits to. Picking ambiguity arises when multiple sprites stack along the depth axis at the same lateral position. The Y-sort criterion that the cross-cutting article on draw order treats later in the series gets its first introduction here.

The mode is the foundation of the beat-em-up genre. The player walks on a flat field viewed from a slightly tilted camera. Enemies approach from both sides and from both depth directions. Punches, kicks, and weapon swings must connect with enemies at the same depth as the player since hits across depth are not possible without a depth-discriminating collision system. The depth axis is the new dimension that gameplay must respect and that the projection map must encode.

The framing the series carries from the opener distinguishes the projection math from the gameplay rules. The projection math is a three-dimensional-to-two-dimensional affine map with a depth-mixing term. The gameplay rules choose how to handle collision across depth, how to render shadows and impacts, and how to sort overlapping sprites on the screen. The article treats each component in turn, walks a concrete worked example, introduces the Y-sort criterion in its simplest form, covers the variations that classic and modern belt-scrolls have explored, enumerates the delivery mechanisms period hardware used, and closes with the canonical games that defined the mode.

A Brief History of the Mode

The earliest belt-scroll beat-em-up is widely cited as Renegade from Technōs Japan in 1986. The game introduced the explicit depth axis into the side-scrolling beat-em-up format that earlier titles such as Kung-Fu Master from Irem in 1984 had constrained to a single axis. The protagonist walks across a flat field toward and away from the camera while fighting enemies at multiple depths along the strip.

Double Dragon from Technōs in 1987 refined the formula with two-player cooperative gameplay, a wider depth-mixing slope, and a wider repertoire of attacks. The arcade success of Double Dragon established the belt-scroll beat-em-up as a viable arcade subgenre.

Final Fight from Capcom in 1989 codified the genre. The game ran on the Capcom CPS-1 arcade system and combined large character sprites with a deep playfield strip and a depth-mixing slope that produced a strong oblique-projection feel. Final Fight became the template that subsequent belt-scrolls inherited across the late 1980s and the early 1990s.

Streets of Rage on the Sega Genesis in 1991 brought the genre to home consoles with a Genesis-quality belt-scroll that approached arcade visual density. The Streets of Rage series ran across three Genesis entries and reached a fourth entry on modern hardware in 2020.

Battletoads on the Nintendo Entertainment System in 1991 extended the belt-scroll formula beyond pure beat-em-up gameplay through level variety that included vehicle stages, auto-scrolling sequences, and platform-style stages. The Battletoads belt-scroll segments demonstrated the formula’s viability on the eight-bit hardware of the previous console generation.

The mode persists through modern independent and revival releases. Streets of Rage 4 on Microsoft Windows in 2020 brought the genre into a hand-drawn modern art style while preserving the underlying projection math of the earlier entries.

The Forward Map

The world coordinate is a three-dimensional position $\mathbf{p}{\text{world}} = (w_x, w_y, w_z)$, where $w_x$ is the lateral position, $w_y$ is the vertical position in the gravity-aligned screen-down convention of the previous articles, and $w_z$ is the depth into the screen. The screen coordinate is a two-dimensional pixel position $\mathbf{p}{\text{screen}} = (s_x, s_y)$. The camera position is a three-dimensional world coordinate $\mathbf{c} = (c_x, c_y, c_z)$. The zoom factor is a positive scalar $z$ giving the number of screen pixels that correspond to one world unit along the $w_x$ and $w_y$ axes. The depth-mixing slope is a non-negative scalar $\beta$ giving the screen-y shift per unit of depth offset from the camera.

The forward map is

where $W$ and $H$ are the screen width and height in pixels.

The lateral term matches the side-scrolling forward map of the previous articles. The vertical term combines the gravity-aligned vertical world coordinate with the depth coordinate through the depth-mixing slope. A character at the back of the playfield has a larger $w_z$ and a smaller $s_y$, appearing higher on the screen. A character at the front has a smaller $w_z$ and a larger $s_y$, appearing lower on the screen.

The depth-mixing slope $\beta$ is the design parameter that distinguishes belt-scroll from its degenerate special cases. Setting $\beta = 0$ recovers the side-scrolling forward map of the previous articles, in which the depth axis collapses and the world is effectively two-dimensional. Setting $\beta = 1$ and dropping the gravity-aligned $w_y$ contribution recovers a top-down view in which the depth axis maps directly onto the screen vertical axis, which is the floor-case forward map of the first cluster article. Belt-scroll uses a slope $\beta$ in the open interval $(0, 1)$, with classic arcade and console belt-scrolls choosing values around one half, to combine both axes into a single screen-y coordinate.

In homogeneous form the forward map writes as

The matrix is a three-by-four affine matrix acting on the four-component augmented world coordinate and producing the three-component augmented screen coordinate. The matrix is the parallel projection from a three-dimensional world onto a two-dimensional screen with the oblique-projection angle that the slope $\beta$ encodes. The synthesis closer of the series treats the more general projective projection of which the belt-scroll forward map is an affine specialisation.

The factorisation pattern from the opener extends to the belt-scroll case as a composition of a depth-mixing shear, an orthogonal projection from three dimensions to two, a uniform scale, and a screen-centre translation,

where $K_z(-\beta)$ is the three-by-three shear matrix that adds $-\beta\, w_z$ to the second component of the camera-relative world coordinate,

and $P_z$ is the two-by-three orthogonal projection that drops the depth component,

The shear and the orthogonal projection are the new factors that belt-scroll introduces above and beyond the previous articles’ factorisations. Subsequent articles in the series generalise the shear matrix into rotation matrices and the orthogonal projection into perspective projection.

A character on the ground at the camera’s depth $w_z = c_z$ projects to screen y

which is the side-scrolling forward map of the previous article without any depth contribution. The article treats the depth term as a screen-y offset that the engine adds to the side-scrolling screen-y for characters not at the camera’s depth.

When a character jumps, the height-above-ground convention of the decoupled-vertical-axis article applies unchanged. Writing $h \geq 0$ for the character’s height above the ground and $w_y^{\text{ground}}$ for the ground-plane vertical world coordinate, the airborne character’s $w_y$ is

with the subtraction reflecting the $y$-down convention in which higher airborne objects sit at smaller $w_y$. The shadow renders at $w_y = w_y^{\text{ground}}$ for any $h$, and the airborne sprite renders at $w_y = w_y^{\text{ground}} - h$. The sprite-shadow decomposition writes the airborne-sprite screen position as the shadow screen position shifted upward by the height-driven vertical offset,

with the magnitude of the vertical screen-space separation

The shadow-drop convention of the decoupled-vertical-axis article extends to belt-scroll without modification.

The Inverse Map

The forward map’s matrix takes three world inputs $(w_x, w_y, w_z)$ and produces two screen outputs $(s_x, s_y)$. The system is underdetermined, as in the decoupled-vertical-axis article. No closed-form inverse exists that recovers all three world inputs from the two screen outputs alone.

Solving the forward map for the world coordinates in terms of the screen coordinates yields

The horizontal world coordinate $w_x$ is uniquely determined. The vertical and depth world coordinates appear only through their linear combination $w_y - \beta\, w_z$. The combination is determined. The individual coordinates are not.

The candidate set is a one-parameter family along a line in the $(w_y, w_z)$ plane parameterised by either $w_y$ or $w_z$,

The line is the pre-image of the screen pixel in the three-dimensional world, also called the picking ray of belt-scroll.

The engine resolves the ambiguity through one of three conventions.

The first is the ground-plane assumption. The engine treats the click as referring to the ground and sets $w_y = w_y^{\text{ground}}$. The recovered depth is

The convention is appropriate when the click is interpreted as a movement command to walk to a ground position.

The second is the known-depth assumption. The engine treats the click as referring to an object at a known depth $w_z^{\text{known}}$. The recovered vertical world coordinate is

The convention is appropriate when the engine has a list of airborne objects and their current depths, and the picking step needs to match a click against one of the candidates.

The third is the screen-space sprite hit test. The engine ignores the world coordinates entirely and tests the clicked screen pixel against the screen-space bounding rectangles of every visible sprite. The strategy is identical to the screen-space hit test introduced in the decoupled-vertical-axis article.

The visible region of the world on the ground plane $w_y = w_y^{\text{ground}}$ is the pre-image of the screen rectangle restricted to that plane. Substituting $w_y = w_y^{\text{ground}}$ into the inverse-map system gives

The visible depth range is finite and centred on the depth at which the camera’s centre-of-screen ray intersects the ground plane. The engine clips world content to the visible ground rectangle when culling for rendering.

A Worked Example

Consider a Capcom CPS-1 belt-scroll with the following parameters. The screen is 384 pixels wide and 224 pixels tall, matching the CPS-1 output resolution. The zoom factor is $z = 1$, one world unit per pixel. The depth-mixing slope is $\beta = 1/2$, a moderate oblique tilt. The ground-plane vertical world coordinate is $w_y^{\text{ground}} = 200$. The camera position is $\mathbf{c} = (500, 100, 0)$. The vertical camera coordinate $c_y = 100$ places the camera 100 world units above the ground at $w_y^{\text{ground}} = 200$ in the $y$-down convention. The depth coordinate $c_z = 0$ is the origin of the depth axis. The choice produces a tilted view where the ground sits in the bottom portion of the screen. The screen-centre offset is $\mathbf{o} = (192, 112)$.

A player character on the ground at world position $\mathbf{p}_{\text{world}} = (500, 200, 0)$ projects to

The player sprite sits at screen $(192, 212)$, near the bottom of the screen where the camera’s ground level renders.

An enemy at the back of the playfield at world position $\mathbf{p}_{\text{world}} = (500, 200, 80)$ projects to

The enemy renders at screen $(192, 172)$, 40 pixels higher on the screen than the player, matching the depth-mixing slope $\beta = 1/2$ applied to the 80-unit depth offset.

An enemy in front of the player at world position $\mathbf{p}_{\text{world}} = (500, 200, -40)$ projects to

The enemy renders at screen $(192, 232)$, 20 pixels lower on the screen than the player, matching the negative depth offset projected through the slope.

Three characters share the same lateral and ground-plane vertical coordinates $(w_x = 500, w_y = 200)$ but differ only in depth. They project to three distinct screen rows separated by depth.

Now consider a jumping player at world position $\mathbf{p}_{\text{world}} = (500, 200 - 25, 0) = (500, 175, 0)$, a height of $h = 25$ above the ground at the camera’s depth. The forward map gives

The jumping player renders at screen $(192, 187)$. The shadow at $w_y = 200$ renders at $s_y = 212$ as in the rest-position case. The vertical separation between the jumping player and the shadow is $\Delta s_y = 25$ pixels, matching $z\, h = 1 \cdot 25 = 25$.

Click-picking at screen pixel $(192, 172)$, the back-enemy’s screen position, returns a candidate line in the $(w_y, w_z)$ plane. The ground-plane assumption with $w_y^{\text{ground}} = 200$ returns

The ground-plane inverse returns $w_z = 80$, matching the back-enemy’s world depth. The picking step then queries the world’s object list for an object at $(500, 200, 80)$ and returns the back enemy.

The round-trip identity holds along the ground plane, which is the same identity pattern the earlier articles introduced,

where $\varepsilon$ is the floating-point precision of the engine.

The Y-Sort Criterion

The depth axis introduces a draw-order question that the previous articles’ single-plane treatments did not need to answer. Two world objects at different depths may project to overlapping screen rectangles. The renderer must decide which object’s sprite draws on top.

The Y-sort criterion sorts objects by their ground-projection screen y, which is the screen y at which the object’s shadow renders. For two ground-level objects at world positions $(w_{x,1}, w_y^{\text{ground}}, w_{z,1})$ and $(w_{x,2}, w_y^{\text{ground}}, w_{z,2})$, the ground-projection screen-y values are

The object with the larger ground-projection $s_y$ is closer to the camera in depth, because the depth-mixing term $-\beta z w_z$ makes deeper objects render at smaller $s_y$. The draw-order criterion writes as a comparison inequality on the ground-projection screen-y values and equivalently on the world depths,

The renderer therefore draws objects in ascending order of ground-projection $s_y$, or equivalently in descending order of world depth $w_z$, from back to front.

The Y-sort criterion generalises to airborne objects by sorting on the shadow’s screen y rather than the sprite’s screen y. An airborne object at world $(w_x, w_y^{\text{ground}} - h, w_z)$ has a sprite at $s_y = z(w_y^{\text{ground}} - h - c_y) - \beta z(w_z - c_z) + H/2$ and a shadow at $s_y^{\text{shadow}} = z(w_y^{\text{ground}} - c_y) - \beta z(w_z - c_z) + H/2$. The shadow’s screen y is independent of $h$ and determines the draw-order position correctly across both ground and airborne objects.

The cross-cutting article on draw order later in the series treats the full Y-sort framework including tie-breaking rules for objects at the same depth, the Z-sort alternative for engines that track depth explicitly, and the hybrid Y-then-Z sort that combines the two.

Pick Disambiguation for Stacked Sprites

Multiple sprites at different depths may project to overlapping screen rectangles. A click at a screen pixel can fall inside more than one sprite’s screen-space bounding box. The engine must decide which sprite the click selects.

Three pick-disambiguation conventions appear in practice.

The first is topmost-in-draw-order. The engine iterates the visible sprites in front-to-back draw order, which is the reverse Y-sort order, and returns the first sprite whose screen-space bounding box contains the click. The closest-to-camera sprite is returned. This is the dominant convention for cursor-driven gameplay and for the editor environments in which most belt-scroll picking occurs.

The second is smallest-screen-area. The engine iterates all visible sprites whose screen-space bounding boxes contain the click and returns the sprite with the smallest screen-space area. The convention favours small sprites that might otherwise be occluded by larger sprites stacked behind them. The convention is rare in pure belt-scroll but appears in level-editor tools that need to select hard-to-reach objects.

The third is world-space depth priority. The engine inverts the click under the ground-plane assumption, gets a candidate depth $w_z^{\text{ground}}$, and returns the sprite whose world-space depth $w_z$ is closest to $w_z^{\text{ground}}$. The convention is appropriate when the click is interpreted as a movement command to a ground position, and the engine wants to identify the object at that ground position rather than the topmost-drawn object.

The cross-cutting article on picking later in the series treats the full pick-disambiguation framework including the Battle Clash and Metal Combat sprite-scale-and-rotate hit-test case and the light-gun picking generalisation.

Variations Within the Mode

The belt-scroll framework admits several variations that engines have explored.

A fixed-depth-camera variant locks the camera at a single $c_z$ for the duration of the level. The player walks on the strip between $c_z + d_{\text{front}}$ and $c_z + d_{\text{back}}$ for fixed front and back boundaries. The camera tracks the player only laterally, matching the canonical side-scrolling camera policy of the previous article applied to belt-scroll.

A two-axis-depth-camera variant also follows the player in the depth axis, so the camera $c_z$ tracks the player’s $w_z$ with a smoothing or dead-zone policy. The variant is rare in classic belt-scrolls and more common in modern entries that emphasise depth navigation as a gameplay feature.

A discrete depth variant restricts the player to a small set of discrete depth values, typically three or four lanes. The player’s $w_z$ changes only when the player explicitly switches lanes through gameplay input. The variant simplifies collision detection across depth and is common in classic NES and Master System belt-scrolls where the central processing unit budget could not support continuous depth.

A continuous depth variant permits any $w_z$ in the playfield strip. The player moves smoothly along the depth axis under directional pad input. Most CPS-1 and Genesis belt-scrolls use the continuous variant.

A scrolling-depth variant moves the back boundary of the playfield strip forward as the player advances laterally, producing the illusion that the world deepens as the player moves to the right. The variant is rare and is mentioned for completeness.

A per-row scroll factor variant introduces per-scanline horizontal scroll modulation that simulates a per-row depth value, which produces a stronger perspective approximation than the constant $\beta$ allows. The per-row scroll factor anticipates the affine ground-plane treatment of the Mode 7 article later in the series.

A sprite-scaling variant scales the sprite size in proportion to the inverse of the depth,

where $d_{\text{ref}}$ is a reference depth and $s_{\text{sprite}}^{(0)}$ is the sprite size at the reference depth. The variant departs from the affine projection of this article into the sprite-scaling territory of a later article in the series. Pure belt-scroll does not scale sprites with depth.

A foreground-occlusion variant allows foreground sprites at $w_z < c_z - d_{\text{occlude}}$ to occlude the playfield strip from the player’s view. The occluding sprites render through the same forward map but at a depth in front of the play strip. The variant produces the effect of trees, signs, or fences that the player walks behind.

Delivery Mechanisms

The belt-scroll forward map permits five distinct delivery mechanisms on period hardware.

The first is software composition on a general-purpose central processing unit. The engine computes each sprite’s screen position through the forward map and writes the sprites to the frame buffer in Y-sort order. The mechanism was the universal delivery on the IBM PC running the Microsoft Disk Operating System through the early 1990s, and remains the dominant mechanism on modern independent-game engines that render to a software frame buffer.

The second is sprite-on-tile-background with per-sprite Y-sort. The CPS-1 arcade hardware, the Sega Genesis, and the Super Nintendo Entertainment System all provide object-attribute-memory sprite hardware that the engine can drive with per-sprite screen positions computed through the belt-scroll forward map. The background tile layer renders the playfield strip through the standard hardware scroll register. The sprite priority is set per sprite to match the Y-sort order.

The third is hardware-sprite-drawing-order Y-sort. The Sega Genesis sprite list uses per-sprite link fields that determine the traversal order during rendering. The engine writes the link sequence in Y-sort order during the per-frame sprite list construction. The hardware then renders sprites in the link-traversal order automatically. The mechanism trades engine flexibility for hardware throughput.

The fourth is pre-baked depth-sorted sprite atlases. The engine pre-renders the sprite frames at each discrete depth in the play-strip range and selects the appropriate frame at render time based on the sprite’s current $w_z$. The mechanism removes the per-frame depth-sort cost at the cost of larger sprite memory. The technique appears in late-period sixteen-bit games that needed to render many sprites at high frame rates.

The fifth is graphics-processing-unit-accelerated quad rendering. Modern game engines such as Unity, Godot, and Unreal render the belt-scroll world as textured quads in three-dimensional world space with a camera transform that produces the desired oblique projection. The graphics processing unit performs the depth sort through its built-in depth buffer and produces the screen image. The modern engine does not need to enforce Y-sort manually because the depth buffer enforces it at the hardware-rasterisation level.

All five mechanisms compute the same forward map and produce the same visible result. The choice trades implementation complexity, the sprite-budget pressure, the central-processing-unit budget for the per-frame sort, the sprite-memory budget, and the achievable frame rate.

Where the Framing Breaks Down

The belt-scroll framing is insufficient when any of the following conditions hold.

When the camera angle is steep enough that perspective scaling becomes visually significant, the affine projection of this article is insufficient. The sprite-scaling article later in the series treats the case where the engine scales each sprite by an inverse-depth factor. The synthesis closer of the series treats the full projective generalisation.

When the world has a non-flat ground, the constant depth-mixing slope $\beta$ fails to track the visible ground level. The Mode 7 article later in the series treats the per-scanline affine matrix that permits the ground plane to bend under a more general projection.

When the playfield exposes truly three-dimensional geometry rather than a flat strip with depth, the oblique projection of this article is insufficient. The synthesis closer of the series treats the projective generalisation that handles three-dimensional geometry through the perspective denominator.

When the player can rotate the camera, the depth axis ceases to point into the screen, and the forward map gains a rotation matrix. The axonometric and isometric articles in the next cluster treat the rotated-camera case.

When the gameplay treats depth as a discrete lane assignment rather than a continuous coordinate, the picking ambiguity collapses because each lane has a discrete $w_z$ and the candidate set has cardinality at most equal to the lane count. The discrete-depth variant above is the relevant case.

When sprite scaling is needed to convey the perspective accurately, the affine projection of this article is insufficient. The classic belt-scroll does not scale sprites and instead lets the depth-mixing slope carry the visual depth cue alone. Modern belt-scrolls sometimes scale sprites lightly for visual interest, crossing into the sprite-scaling territory.

The Canon

The following games use belt-scroll as their primary projection mode. The list is selective rather than exhaustive and emphasises the games that defined the mode at a given moment.

Renegade in the arcade in 1986 gave the medium its first belt-scroll beat-em-up with an explicit depth axis that the side-scrolling beat-em-ups before it did not provide.

Double Dragon in the arcade in 1987 refined the formula with cooperative two-player gameplay and a wider repertoire of attacks.

Final Fight in the arcade in 1989 codified the genre through the Capcom CPS-1 hardware’s sprite throughput and the large-character art style that subsequent belt-scrolls inherited.

Streets of Rage on the Sega Genesis in 1991 brought arcade-quality belt-scroll to the home console with a soundtrack and visual style that the home-console audience embraced.

Battletoads on the Nintendo Entertainment System in 1991 demonstrated the belt-scroll formula on eight-bit hardware through level variety that mixed beat-em-up belt-scroll segments with auto-scrolling vehicle stages and platform-style stages.

Streets of Rage 4 on Microsoft Windows in 2020 brought the genre into modern hardware with a hand-drawn art style that preserves the underlying projection math of the earlier entries.

Each game in the canon uses the belt-scroll forward map with a different depth-mixing slope $\beta$, a different camera policy along the depth axis, and a different choice of delivery mechanism appropriate to the target hardware.

Out of Scope

The article does not cover the following.

Sprite scaling along the depth axis is treated in the sprite-scaling article later in the series. The classic belt-scroll uses uniform sprite size without depth-dependent scaling.

The full Y-sort framework including tie-breaking and Z-sort variants is the subject of a later cross-cutting article. The article introduces the Y-sort criterion in its simplest form and defers the full treatment.

The full pick-disambiguation framework including the sprite-scale-and-rotate hit-test of Battle Clash and Metal Combat is the subject of a later cross-cutting article. The article presents the three baseline conventions and defers the full treatment.

Collision detection across depth is gameplay-physics territory that the projection math does not depend on. The article treats projection only.

True three-dimensional rendering of the belt-scroll world through a graphics-processing-unit pipeline with a perspective camera is the modern delivery mechanism that the article treats only briefly. The mechanism produces the belt-scroll forward map as a special case of perspective projection but introduces depth buffering and shader programming that the article does not depend on.

The axonometric and isometric viewing angles that permit camera rotation are treated in the next cluster of the series.

Conclusion

Belt-scroll side-scrolling with explicit depth extends the side-scrolling view of the previous article with a depth axis that the camera projects through a depth-mixing slope $\beta$. The forward map is a three-dimensional-to-two-dimensional affine projection with a shear factor on the depth coordinate. The inverse map is underdetermined and admits three disambiguation conventions parallel to the decoupled-vertical-axis article’s conventions. The Y-sort criterion makes its first appearance in the series as the sort-by-ground-projection-screen-y rule that the renderer must enforce to draw stacked sprites in the correct order. The pick disambiguation for stacked sprites admits three conventions that the engine selects based on the gameplay action being requested. The math is the simplest non-trivial three-dimensional projection in the Cartesian cluster, and the delivery admits five distinct mechanisms on period hardware and modern engines. The next article in the series swaps the camera orientation from looking at the world along a primary axis to looking at the world along a tilted axis, which is the oblique view of the quarter-view cluster.

References

• Reference, Battletoads
• Reference, Double Dragon
• Reference, Final Fight
• Reference, Kung-Fu Master
• Reference, Renegade
• Reference, Streets of Rage
• Reference, Streets of Rage 4