This post is the first in a series on a group of related tricks we can file under the category of “interpolation” or “control systems.” This can seem very abstract, but there’s a reason: it’s applicable in so many different areas, as this first post will attempt to show. In this post, I want to discuss some general principles of signals, systems, and interpolation, and give practical examples of where these things come up in everyday video game programming.
Many of the data we deal with in video games can be viewed as a “signal,” or to use an equivalent term, a “function.” Quite often we deal with signals in the time domain, or more simply, we have some data that changes over time. Here are some examples:
- The player’s input values. (Mouse movement rates, whether or not keys are depressed at that moment, etc)
- The player character’s velocity.
- The health of a character.
- The visual size of the character’s health meter.
- The positions and orientations of player characters and other objects in the world.
- The current lock-on target.
- Which context-sensitive camera mode is currently active? For example, a close-up up on a taunt, a hint showing a door that opened as a result of the lever you just pulled, an adjustment to zoom to highlight an acrobatic element that can be interacted with, and so forth.
- The “ideal” camera position, given the current state affairs in the game world: the selected camera mode, character positions, obstacles that must be avoided, etc.
- The player’s current selection from some menu.
- The screen-space position oi the visual cursor showing which menu element is highlighted.
- A target’s position that an AI entity is trying to aim at
- The authoritative position of an object, received periodically from the game server, when we are a client.
- The proper position a character should be so that his animation lines up with some other object or character he is participating with.
- The raw cursor position from the Wiimote.
- The position of the cursor on the screen.
- The data we get from Kinect libraries telling us where it thinks the player’s arms and legs are.
- The actual positions and orientation of the bones used to animate the on-screen representation of the player.
- …and I’m sure you can think of many more examples.
Continuous and discrete signals
You will notice that some of the examples signals take on values from a continuous domain, such as the positions of characters and other objects in the world, or mouse movement rates. Those are the signals that we store using floating point data types, whether they be 1D scalar values or 2D or 3D vectors. Other signals may only assume values from a discrete set, for example, menu selection, discrete camera modes, lock-on targets, whether or not the ‘W’ key is currently depressed, and so forth. We use bools, ints, and enums for these.
Just because a signal comes from a continuous domain does not mean it is continuous function. For example, a character’s health may be constant and then take a sudden jump up or down if they get shot or pick up a potion, etc. An AI entity may suddenly switch targets, so that even if each individual target moves around in a continuous fashion, the signal representing “the position that the AI wants to be aiming at” may exhibit discontinuities. Characters can warp and spawn. A client depending upon a game server for the authoritative value of some continuous variable is only going to receive that variable in bursts. One of the main reasons to understand control systems theory is to know how best to mask these discontinuities, making a smooth output signal out of a jumpy input one.
Of course, since we are working in a digital computer, in reality all of our signals are not truly continuous in time, because we only know (or care about) its value for particular time values, for example, perhaps at 30 different times per second. But we still consider the underlying signal to be continuous, even if we are only sampling an input signal or updating an output signal once per frame.
Must every signal be a function of time?
All of the examples we’ll look at are time-domain signals, but before moving one, it’s worth mentioning that there are certainly signals from other domains. For example, a visual image can be thought of as a function of the continuous screen space coordinates, defined at any (x,y) coordinates, not just at the pixel “centers.” (We could use raytracing to determine the value.) Practical images are represented as a discrete grid of pixel values (a bitmap); how to best sample the continuous signal to determine the pixel colors is a very important topic, one that can only be fully understood using the tools of signal processing theory. (It is NOT simply the value of the continuous signal at the “center” of the pixel!)
Video games contain lots of code that, when viewed from the standpoint of signal processing theory, is a system. A system is basically any process, function, algorithm — or block of video game code — that takes one or more input signals, and produces one or more output signals. The list of example signals given previously was hopefully highly suggestive, and you already thought of these examples of systems:
- We typically do not map the player’s movement directly from the input. For example, when the player presses the ‘W’ key to move forward, he doesn’t snap into motion; likewise, when he releases it, he doesn’t instantly stop. There is a some logic that decides how to nicely ramp his velocity in and out. A system is taking the raw input signal of the current status of the `W’ key and mapping it to the actual player velocity.
- Console controllers are notorious for being “noisy”, if we just scaled the raw value as our velocity, you couldn’t stand still.
- We might want the chracter’s health bar to visually animate, for an extra level of polish, even if the underlying health value jumps.
- Camera control is full of control systems. Often, there is one system that takes as input the current character position, lock on target, special interaction, etc, and produces an “ideal” camera position. But this position cannot be used directly, it is extremely jerky. The camera would snap suddenly when you switched lock-on targets. The character movement might need to be “jerky” for gameplay reasons (be able to stop on a dime), but we might want the camera to not have this rigid, mechanical feeling. So another system takes the raw ideal position and smooths it out, to provide the transitions between different modes and smooth out jerky movement.
- The AI cannot just snap to aim at his desired target. The assassin’s arm, the gun, the sentry turret, or whatever, would snap with infinite velocity; this is not realistic. And if the AI happens to be shooting at you, it probably won’t seem fair or fun, either.
- We might want the visual highlight over the user’s selection to animate smoothly, rather than just snapping into place.
- The raw data from the Wiimote and the Kinect camera are noisy. We must filter it or else the pointer and avatar will be jitterly
As you can see, many of these examples can be put into the category of “making things feel right.” Different types of systems will “feel” differently and have different performance characteristics. It’s very helpful to have a library of systems to choose from.
In the next post, I will consider some very special signals that are able to capture common behaviour in signals: they are the impulse, step, ramp, and sinosoid. By analyzing how our system responds to these special signals, we will learn a lot about how our system behaves in general.