Chapter 2

Vectors

Yellow fever vaccine should be administered
10 to 12 days before exposure to the vector.

— The United States Dispensatory (1978)

Vectors are the formal mathematical entities we use to do 2D and 3D math. The word vector has two distinct but related meanings. Mathematics books, especially those on linear algebra, tend to focus on a rather abstract definition, caring about the numbers in a vector but not necessarily about the context or actual meaning of those numbers. Physics books, on the other hand, tend towards an interpretation that treats a vector as a geometric entity to the extent that they avoid any mention of the coordinates used to measure the vector, when possible. It's no wonder that you can sometimes find people from these two disciplines correcting one another on the finer points of “how vectors really work.” Of course the reality is that they are both right,1 and to be proficient with 3D math, we need to understand both interpretations of vectors and how the two interpretations are related.

This chapter introduces the concept of vectors. It is divided into the following sections.

Section 2.1 covers some of the basic mathematical properties of vectors.
Section 2.2 gives a high-level introduction to the geometric properties of vectors.
Section 2.3 connects the mathematical definition with the geometric one and discusses how vectors work within the framework of Cartesian coordinates.
Section 2.4 discusses the often confusing relationship between points and vectors and considers the rather philosophical question of why it is so difficult to make absolute measurements.
Sections 2.5–2.12 discuss the fundamental calculations we can perform with vectors, considering both the algebra and geometric interpretations of each operation.
Section 2.13 presents a list of helpful vector algebra laws.

2.1Mathematical Definition of Vector,
and Other Boring Stuff

To mathematicians, a vector is a list of numbers. Programmers will recognize the synonymous term array. Notice that the STL template array class in C++ is named vector, and the basic Java array container class is java.util.Vector. So mathematically, a vector is nothing more than an array of numbers.

Yawn… If this abstract definition of a vector doesn't inspire you, don't worry. Like many mathematical subjects, we must first introduce some terminology and notation before we can get to the “fun stuff.”

Mathematicians distinguish between vector and scalar (pronounced “SKAY-lur”) quantities. You're already an expert on scalars—scalar is the technical term for an ordinary number. We use this term specifically when we wish to emphasize that a particular quantity is not a vector quantity. For example, as we will discuss shortly, “velocity” and “displacement” are vector quantities, whereas “speed” and “distance” are scalar quantities.

The dimension of a vector tells how many numbers the vector contains. Vectors may be of any positive dimension, including one. In fact, a scalar can be considered a 1D vector. In this book, we primarily are interested in 2D, 3D, and (later) 4D vectors.

When writing a vector, mathematicians list the numbers surrounded by square brackets, for example, $[1, 2, 3]$ . When we write a vector inline in a paragraph, we usually put commas between the numbers. When we write it out in an equation, the commas are often omitted. In either case, a vector written horizontally is called a row vector. Vectors are also frequently written vertically:

A 3D column vector

[\begin{matrix} 1 \\ 2 \\ 3 \end{matrix}] .

A vector written vertically is known as a column vector. This book uses both notations. For now, the distinction between row and column vectors won't matter. However, in Section 4.1.7 we discuss why in certain circumstances the distinction is critical.

When we wish to refer to the individual components in a vector, we use subscript notation. In math literature, integer indices are used to access the elements. For example $v_{1}$ refers to the first element in $v$ . However, we are specifically interested in 2D, 3D, and 4D vectors rather than vectors of arbitrary dimension $n$ , so we rarely use this notation. Instead, we use $x$ and $y$ to refer to the elements in a 2D vector; $x$ , $y$ , and $z$ to refer to the elements in a 3D vector; and $x$ , $y$ , $z$ , and $w$ to refer to the elements in a 4D vector. This notation is shown in Equation (2.1).

Vector subscript notation

\begin{matrix} (2.1) & \begin{array}{ccl} a = [\begin{matrix} 1 \\ 2 \end{matrix}] & \begin{array}{ll} a_{1} = a_{x} = 1 \\ a_{2} = a_{y} = 2 \end{array} \\ b = [\begin{matrix} 3 \\ 4 \\ 5 \end{matrix}] & \begin{array}{ll} b_{1} = b_{x} = 3 \\ b_{2} = b_{y} = 4 \\ b_{3} = b_{z} = 5 \end{array} \\ c = [\begin{matrix} 6 \\ 7 \\ 8 \\ 9 \end{matrix}] & \begin{array}{ll} c_{1} = c_{x} = 6 \\ c_{2} = c_{y} = 7 \\ c_{3} = c_{z} = 8 \\ c_{4} = c_{w} = 9 \end{array} \end{array} \end{matrix}

Notice that the components of a 4D vector are not in alphabetical order. The fourth value is $w$ . (Hey, they ran out of letters in the alphabet!)

Now let's talk about some important typeface conventions that are used in this book. As you know, variables are placeholder symbols used to stand for unknown quantities. In 3D math, we work with scalar, vector, and (later) matrix quantities. In the same way that it's important in a C++ or Java program to specify what type of data is stored by a variable, it is important when working with vectors to be clear what type of data is represented by a particular variable. In this book, we use different fonts for variables of different types:

Scalar variables are represented by lowercase Roman or Greek letters in italics: $a$ , $b$ , $x$ , $y$ , $z$ , $θ$ , $α$ , $ω$ , $γ$ .
Vector variables of any dimension are represented by lowercase letters in boldface: $a$ , $b$ , $u$ , $v$ , $q$ , $r$ .
Matrix variables are represented using uppercase letters in boldface: $A$ , $B$ , $M$ , $R$ .

Note that other authors may use different conventions. One common convention used frequently when writing vectors by hand, is to draw an arrow over the vector, for example,

\vec{a}

Before we go any further, a bit of context is in order concerning the perspective that we are adopting about vectors. The branch of mathematics that deals primarily with vectors and matrices is called linear algebra, a subject that assumes the abstract definition given previously: a vector is an array of numbers. This highly generalized approach allows for the exploration of a large set of mathematical problems. In linear algebra, vectors and matrices of dimension $n$ are used to solve a system of $n$ linear equations for $n$ unknowns, without knowing or caring what physical significance, if any, is attached to any of the numbers. This is certainly a fascinating and highly practical study, but it is not of primary interest to our investigation of 3D math. For 3D math, we are mostly concerned with the geometric interpretations of vectors and vector operations.

Our focus is geometric, so we omit many details and concepts of linear algebra that do not further our understanding of 2D or 3D geometry. Even though we occasionally discuss properties or operations for vectors of an arbitrary dimension $n$ , we will usually focus on 2D, 3D, and (later) 4D vectors and matrices. Even when the numbers in a vector do not have any physical significance, the ability to visualize the linear algebra operations is of some utility, so learning how to interpret the operations geometrically is useful even in nonphysical applications. Some more context about how the topics in this book fit into the bigger picture of linear algebra can be found in Section 4.3.

2.2Geometric Definition of Vector

Now that we have discussed what a vector is mathematically, let's look at a more geometric interpretation of vectors. Geometrically speaking, a vector is a directed line segment that has magnitude and direction.

The magnitude of a vector is the length of the vector. A vector may have any nonnegative length.
The direction of a vector describes which way the vector is pointing in space. Note that “direction” is not exactly the same as “orientation,” a distinction we will reexamine in Section 8.1.

Figure 2.1A 2D vector

Let's look at a vector. Figure 2.1 shows an illustration of a vector in 2D.

It looks like an arrow, right? This is the standard way to represent a vector graphically, since the two defining characteristics of a vector are captured: its magnitude and direction.

We sometimes refer to the head and tail of a vector. As shown in Figure 2.2, the head is the end of the vector with the arrowhead on it (where the vector “ends”), and the tail is the other end (where the vector “starts”).

Figure 2.2A vector has a head and a tail

Where is this vector? Actually, that is not an appropriate question. Vectors do not have position, only magnitude and direction. This may sound impossible, but many quantities we deal with on a daily basis have magnitude and direction, but no position. Consider how the two statements below could make sense, regardless of the location where they are applied.

Displacement. “Take three steps forward.” This sentence seems to be all about positions, but the actual quantity used in the sentence is a relative displacement and does not have an absolute position. This relative displacement consists of a magnitude (3 steps) and a direction (forward), so it could be represented by a vector.
Velocity. “I am traveling northeast at 50 mph.” This sentence describes a quantity that has magnitude (50 mph) and direction (northeast), but no position. The concept of “northeast at 50 mph” can be represented by a vector.

Notice that displacement and velocity are technically different from the terms distance and speed. Displacement and velocity are vector quantities and therefore entail a direction, whereas distance and speed are scalar quantities that do not specify a direction. More specifically, the scalar quantity distance is the magnitude of the vector quantity displacement, and the scalar quantity speed is the magnitude of the vector quantity velocity.

Because vectors are used to express displacements and relative differences between things, they can describe relative positions. (“My house is 3 blocks east of here.”) However, you should not think of a vector as having an absolute position itself, instead, remember that it is describing the displacement from one position to another, in this case from “here” to “my house.” (More on relative versus absolute position in Section 2.4.1.) To help enforce this, when you imagine a vector, picture an arrow. Remember that the length and direction of this arrow are significant, but not the position.

Since vectors do not have a position, we can represent them on a diagram anywhere we choose, provided that the length and direction of the vector are represented correctly. We often use this fact to our advantage by sliding the vector around into a meaningful location on a diagram.

Now that we have the big picture about vectors from a mathematical and geometric perspective, let's learn how to work with vectors in the Cartesian coordinate system.

2.3Specifying Vectors with Cartesian Coordinates

When we use Cartesian coordinates to describe vectors, each coordinate measures a signed displacement in the corresponding dimension. For example, in 2D, we list the displacement parallel to the $x$ -axis, and the displacement parallel to the $y$ -axis, as illustrated in Figure 2.3.

Figure 2.3 Vectors are specified by giving the signed displacement in each dimension.

Figure 2.4 shows several 2D vectors and their values. Notice that the position of each vector on the diagram is irrelevant. (The axes are conspicuously absent to emphasize this fact, although we do assume the standard convention of $+ x$ pointing to the right and $+ y$ pointing up.) For example, two of the vectors in Figure 2.4 have the value $[1.5, 1]$ , but they are not in the same place on the diagram.

Figure 2.4 Examples of 2D vectors and their values

3D vectors are a simple extension of 2D vectors. A 3D vector contains three numbers, which measure the signed displacements in the $x$ , $y$ , and $z$ directions, just as you'd expect.

We are focusing on Cartesian coordinates for now, but they are not the only way to describe vectors mathematically. Polar coordinates are also common, especially in physics textbooks. Polar coordinates are the subject of Chapter 7.

2.3.1Vector as a Sequence of Displacements

One helpful way to think about the displacement described by a vector is to break out the vector into its axially aligned components. When these axially aligned displacements are combined, they cumulatively define the displacement defined by the vector as a whole.

Figure 2.5 Interpreting a vector as a sequence of displacements

For example, the 3D vector $[1, - 3, 4]$ represents a single displacement, but we can visualize this displacement as moving 1 unit to the right, 3 units down, and then 4 units forward. (Assume our convention that $+ x$ , $+ y$ , and +z point right, up, and forward, respectively. Also note that we do not “turn” between steps, so “forward” is always parallel to $+ z$ .) This displacement is illustrated in Figure 2.5.

The order in which we perform the steps is not important; we could move 4 units forward, 3 units down, and then 1 unit to the right, and we would have displaced by the same total amount. The different orderings correspond to different routes along the axially aligned bounding box containing the vector. Section 2.7.2 mathematically verifies this geometric intuition.

2.3.2The Zero Vector

For any given vector dimension, there is a special vector, known as the zero vector, that has zeroes in every position. For example, the 3D zero vector is $[0, 0, 0]$ . We denote a zero vector of any dimension using a boldface zero: $0$ . In other words,

The zero vector

0 = [\begin{matrix} 0 \\ 0 \\ ⋮ \\ 0 \end{matrix}] .

The zero vector is special because it is the only vector with a magnitude of zero. All other vectors have a positive magnitude. The zero vector is also unique because it is the only vector that does not have a direction.

Since the zero vector doesn't have a direction or length, we don't draw it as an arrow like we do for other vectors. Instead, we depict the zero vector as a dot. But don't let this make you think of the zero vector as a “point” because a vector does not define a location. Instead, think of the zero vector as a way to express the concept of “no displacement,” much as the scalar zero stands for the concept of “no quantity.”

Like the scalar zero you know, the zero vector of a given dimension is the additive identity for the set of vectors of that dimension. Try to take yourself back to your algebra class, and retrieve from the depths of your memory the concept of the additive identity: for any set of elements, the additive identity of the set is the element $x$ such that for all $y$ in the set, $y + x = y$ .2 In other words, when we add the zero vector to any other vector, we get that vector: $0 + a = a$ . Section 2.7 deals with vector addition.

2.4Vectors versus Points

Recall that a “point” has a location but no real size or thickness. In this chapter, we have learned how a “vector” has magnitude and direction, but no position. So “points” and “vectors” have different purposes, conceptually: a “point” specifies a position, and a “vector” specifies a displacement.

But now examine Figure 2.6, which compares an illustration from Chapter 1 (Figure 1.8), showing how 2D points are located, with a figure from earlier in this chapter (Figure 2.3), showing how 2D vectors are specified. It seems that there is a strong relationship between points and vectors. This section examines this important relationship.

Figure 2.6Locating points versus specifying vectors

2.4.1Relative Position

Section 2.2 discussed the fact that because vectors can describe displacements, they can describe relative positions. The idea of a relative position is fairly straightforward: the position of something is specified by describing where it is in relation to some other, known location.

This begs the questions: Where are these “known” locations? What is an “absolute” position? It is surprising to realize that there is no such thing! Every attempt to describe a position requires that we describe it relative to something else. Any description of a position is meaningful only in the context of some (typically “larger”) reference frame. Theoretically, we could establish a reference frame encompassing everything in existence and select a point to be the “origin” of this space, thus defining the “absolute” coordinate space. However, even if such an absolute coordinate space were possible, it would not be practical. Luckily for us, absolute positions in the universe aren't important. Do you know your precise position in the universe right now? We don't know ours, either.3

2.4.2The Relationship between Points and Vectors

Vectors are used to describe displacements, and therefore they can describe relative positions. Points are used to specify positions. But we have just established in Section 2.4.1 that any method of specifying a position must be relative. Therefore, we must conclude that points are relative as well—they are relative to the origin of the coordinate system used to specify their coordinates. This leads us to the relationship between points and vectors.

Figure 2.7 illustrates how the point $(x, y)$ is related to the vector $[x, y]$ , given arbitrary values for $x$ and $y$ .

Figure 2.7 The relationship between points and vectors

As you can see, if we start at the origin and move by the amount specified by the vector $[x, y]$ , we will end up at the location described by the point $(x, y)$ . Another way of saying this is that the vector $[x, y]$ gives the displacement from the origin to the point $(x, y)$ .

This may seem obvious, but it is important to understand that points and vectors are conceptually distinct, but mathematically equivalent. This confusion between “points” and “vectors” can be a stumbling block for beginners, but it needn't be a problem for you. When you think of a location, think of a point and visualize a dot. When you think of a displacement, think of a vector and visualize an arrow.

In many cases, displacements are from the origin, and so the distinction between points and vectors will be a fine one. However, we often deal with quantities that are not relative to the origin, or any other point for that matter. In these cases, it is important to visualize these quantities as an arrow rather than a point.

The math we develop in the following sections operates on “vectors” rather than “points.” Keep in mind that any point can be represented as a vector from the origin.

Actually, now would be a good time to warn you that a lot of people take a much firmer stance on this issue and would not approve of our cavalier attitude in treating vectors and points as mathematical equals.4 Such hard-liners will tell you, for example, that while you can add two vectors (yielding a third vector), and you can add a vector and a point (yielding a point), you cannot add two points together. We admit that there is some value in understanding these distinctions in certain circumstances. However, we have found that, especially when writing code that operates on points and vectors, adherence to these ethics results in programs that are almost always longer and never faster.5 Whether it makes the code cleaner or easier to understand is a highly subjective matter. Although this book does not use different notations for points and vectors, in general it will be clear whether a quantity is a point or a vector. We have tried to avoid presenting results with vectors and points mixed inappropriately, but for all the intermediate steps, we might not have been quite as scrupulous.

2.4.3It's All Relative

Before we move on to the vector operations, let's take a brief philosophical intermission. Spatial position is not the only aspect of our world for which we have difficulty establishing an “absolute” reference, and so we use relative measurements. There are also temperature, loudness, and velocity.

Temperature. One of the first attempts to make a standard temperature scale occurred about AD 170, when Galen proposed a standard “neutral” temperature made up of equal quantities of boiling water and ice. On either side of this temperature were four degrees of “hotter” and four degrees of “colder.” Sounds fairly primitive, right? In 1724, Gabriel Fahrenheit suggested a bit more precise system. He suggested that mercury be used as the liquid in a thermometer, and calibrated his scale using two reference points: the freezing point of water, and the temperature of a healthy human being. He called his scale the Fahrenheit scale, and measurements were in °F. In 1745, Carolus Linnaeus of Uppsala, Sweden, suggested that things would be simpler if we made the scale range from 0 (at the freezing point of water) to 100 (water's boiling point), and called this scale the centigrade scale. (This scale was later abandoned in favor of the Celsius scale, which is technically different from centigrade in subtle ways that are not important here.) Notice that all of these scales are relative—they are based on the freezing point of water, which is an arbitrary (but highly practical) reference point. A temperature reading of $x^{o}$ C basically means “ $x$ degrees hotter than the temperature at which water freezes.” It wasn't until 1848, with the invention of the Kelvin scale by Lord Kelvin, that mankind finally had an absolute temperature scale. 0 K is the coldest possible temperature, corresponding to $- 273^{o}$ C.

Loudness. Loudness is usually measured in decibels (abbreviated dB). To be more precise, decibels are used to measure the ratio of two power levels. If we have two power levels $P_{1}$ and $P_{2}$ , then the difference in decibels between the two power levels is

10 \log_{10} (P_{2} / P_{1}) d B .

So, if $P_{2}$ is about twice the level of $P_{1}$ , then the difference is about 3 dB. Notice that this is a relative system, providing a precise way to measure the relative strength of two power levels, but not a way to assign a number to one power level. In other words, we haven't established any sort of absolute reference point. (It's also a logarithmic scale, but that isn't important here.) You may have used a mixer board, volume control knob, or digital audio program that measures volume in dB. Normally, there's an arbitrary point marked 0 dB, and then most of the readings have negative values. In other words, 0 dB is the loudest volume, and all other volume settings are softer.

None of these values are absolute—but how could they be? How could your digital audio program know the absolute loudness you will experience, which depends not only on the audio data, but also the volume setting on your computer, the volume knob on your amplifier, the power supplied by the amplifier to your speakers, the distance you are from the speakers, and so on.

Sometimes people describe how loud something is in terms of an absolute dB number. Following in the footsteps of Gabriel Fahrenheit, this scale uses a reference point based on the human body. “Absolute” dB numbers are actually relative to the threshold of hearing for a normal human.6 Because of this, it's actually possible to have an “absolute” dB reading that is negative. This simply means that the intensity is below the threshold where most people are able to hear it.

At this point, we should probably mention that there is a way to devise an absolute scale for loudness, by measuring a physical quantity such as pressure, energy, or power, all of which have an absolute minimum value of zero. The point is that these absolute systems aren't used in many cases—the relative system is the one that's the most useful.

Velocity. How fast are you moving right now? Perhaps you're sitting in a comfy chair, so you'd say that your speed was zero. Maybe you're in a car and so you might say something like 65 mph. (Hopefully someone else is driving!) Actually, you are hurtling through space at almost 30 km per second! That's about the speed that Earth travels in order to make the 939-million-km trek around the sun each year. Of course, even this velocity is relative to the sun. Our solar system is moving around within the Milky Way galaxy. So then how fast are we actually moving, in absolute terms? Galileo told us back in the 17th century that this question doesn't have an answer—all velocity is relative.

Our difficulty in establishing absolute velocity is similar to the difficulty in establishing position. After all, velocity is displacement (difference between positions) over time. To establish an absolute velocity, we'd need to have some reference location that would “stay still” so that we could measure our displacement from that location. Unfortunately, everything in our universe seems to be orbiting something else.

2.5Negating a Vector

The previous sections have presented a high-level overview of vectors. The remainder of this chapter looks at specific mathematical operations we perform on vectors. For each operation, we first define the mathematical rules for performing the operation and then describe the geometric interpretations of the operation and give some practical uses for the operation.

The first operation we'd like to consider is that of vector negation. When discussing the zero vector, we asked you to recall from group theory the idea of the additive identity. Please go back to wherever it was in your brain that you found the additive identity, perhaps between the metaphorical couch cushions, or at the bottom of a box full of decade-old tax forms. Nearby, you will probably find a similarly discarded obvious-to-the-point-of-useless concept: the additive inverse. Let's dust it off. For any group, the additive inverse of $x$ , denoted by $- x$ , is the element that yields the additive identity (zero) when added to $x$ . Put simply, $x + (- x) = 0$ . Another way of saying this is that elements in the group can be negated.

The negation operation can be applied to vectors. Every vector $v$ has an additive inverse $- v$ of the same dimension as $v$ such that $v + (- v) = 0$ . (We will learn how to add vectors in Section 2.7.)

2.5.1Official Linear Algebra Rules

Figure 2.8 Examples of vectors and their negatives. Notice that a vector and its negative are parallel and have the same magnitude, but point in opposite directions.

To negate a vector of any dimension, we simply negate each component of the vector. Stated formally,

Negating a vector

- [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] = [\begin{matrix} - a_{1} \\ - a_{2} \\ ⋮ \\ - a_{n - 1} \\ - a_{n} \end{matrix}] .

Applying this to the specific cases of 2D, 3D, and 4D vectors, we have

Negating 2D, 3D, and 4D vectors

\begin{aligned} - [\begin{array}{c} x & y \end{array}] & = [\begin{array}{c} - x & - y \end{array}], \\ - [\begin{array}{c} x & y & z \end{array}] & = [\begin{array}{c} - x & - y & - z \end{array}], \\ - [\begin{array}{c} x & y & z & w \end{array}] & = [\begin{array}{c} - x & - y & - z & - w \end{array}] . \end{aligned}

A few examples are

\begin{aligned} - [\begin{array}{c} 4 & - 5 \end{array}] & = [\begin{array}{c} - 4 & 5 \end{array}], \\ - [\begin{array}{c} - 1 & 0 & \sqrt{3} \end{array}] & = [\begin{array}{c} 1 & 0 & - \sqrt{3} \end{array}], \\ - [\begin{array}{c} 1.34 & - 3 / 4 & - 5 & π \end{array}] & = [\begin{array}{c} - 1.34 & 3 / 4 & 5 & - π \end{array}] . \end{aligned}

2.5.2Geometric Interpretation

Negating a vector results in a vector of the same magnitude but opposite direction, as shown in Figure 2.8.

Remember, the position of a vector on a diagram is irrelevant—only the magnitude and direction are important.

2.6Vector Multiplication by a Scalar

Although we cannot add a vector and a scalar, we can multiply a vector by a scalar. The result is a vector that is parallel to the original vector, with a different length and possibly opposite direction.

2.6.1Official Linear Algebra Rules

Vector-times-scalar multiplication is straightforward; we simply multiply each component of the vector by the scalar. Stated formally,

Multiplying a vector by a scalar

k [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] = [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] k = [\begin{matrix} k a_{1} \\ k a_{2} \\ ⋮ \\ k a_{n - 1} \\ k a_{n} \end{matrix}] .

Applying this rule to 3D vectors, as an example, we get

Multiplying a 3D vector by a scalar

k [\begin{matrix} x \\ y \\ z \end{matrix}] = [\begin{matrix} x \\ y \\ z \end{matrix}] k = [\begin{matrix} k x \\ k y \\ k z \end{matrix}] .

Although the scalar and vector may be written in either order, most people choose to put the scalar on the left, preferring $k v$ to $v k$ .

A vector may also be divided by a nonzero scalar. This is equivalent to multiplying by the reciprocal of the scalar:

Dividing a 3D vector by a scalar

\frac{v}{k} = (\frac{1}{k}) v = [\begin{matrix} v_{x} / k \\ v_{y} / k \\ v_{z} / k \end{matrix}] for 3D vector v and nonzero scalar k .

Figure 2.9A 2D vector multiplied by various scalars

Some examples are

\begin{aligned} 2 [\begin{array}{c} 1 & 2 & 3 \end{array}] & = [\begin{array}{c} 2 & 4 & 6 \end{array}], \\ - 3 [\begin{array}{c} - 5.4 & 0 \end{array}] & = [\begin{array}{c} 16.2 & 0 \end{array}], \\ [\begin{array}{c} 4.7 & - 6 & 8 \end{array}] / 2 & = [\begin{array}{c} 2.35 & - 3 & 4 \end{array}] . \end{aligned}

Here are a few things to notice about multiplication of a vector by a scalar:

When we multiply a vector and a scalar, we do not use any multiplication symbol. The multiplication is signified by placing the two quantities side-by-side (usually with the vector on the right).
Scalar-times-vector multiplication and division both occur before any addition and subtraction. For example $3 a + b$ is the same as $(3 a) + b$ , not $3 (a + b)$ .
A scalar may not be divided by a vector, and a vector may not be divided by another vector.
Vector negation can be viewed as the special case of multiplying a vector by the scalar $- 1$ .

2.6.2Geometric Interpretation

Geometrically, multiplying a vector by a scalar $k$ has the effect of scaling the length by a factor of $| k |$ . For example, to double the length of a vector we would multiply the vector by 2. If $k < 0$ , then the direction of the vector is flipped. Figure 2.9 illustrates a vector multiplied by several different scalars.

2.7Vector Addition and Subtraction

We can add and subtract two vectors, provided they are of the same dimension. The result is a vector quantity of the same dimension as the vector operands. We use the same notation for vector addition and subtraction as is used for addition and subtraction of scalars.

2.7.1Official Linear Algebra Rules

The linear algebra rules for vector addition are simple: to add two vectors, we add the corresponding components:

Adding two vectors

[\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] + [\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{n - 1} \\ b_{n} \end{matrix}] = [\begin{matrix} a_{1} + b_{1} \\ a_{2} + b_{2} \\ ⋮ \\ a_{n - 1} + b_{n - 1} \\ a_{n} + b_{n} \end{matrix}] .

Subtraction can be interpreted as adding the negative, so $a - b = a + (- b)$ :

Subtracting two vectors

[\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] - [\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{n - 1} \\ b_{n} \end{matrix}] = [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] + (- [\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{n - 1} \\ b_{n} \end{matrix}]) = [\begin{matrix} a_{1} - b_{1} \\ a_{2} - b_{2} \\ ⋮ \\ a_{n - 1} - b_{n - 1} \\ a_{n} - b_{n} \end{matrix}] .

For example, given

a = [\begin{matrix} 1 \\ 2 \\ 3 \end{matrix}], b = [\begin{matrix} 4 \\ 5 \\ 6 \end{matrix}], c = [\begin{matrix} 7 \\ - 3 \\ 0 \end{matrix}],

then

\begin{aligned} a + b & = [\begin{array}{c} 1 \\ 2 \\ 3 \end{array}] + [\begin{array}{c} 4 \\ 5 \\ 6 \end{array}] = [\begin{array}{c} 1 + 4 \\ 2 + 5 \\ 3 + 6 \end{array}] = [\begin{array}{c} 5 \\ 7 \\ 9 \end{array}], \\ a - b & = [\begin{array}{c} 1 \\ 2 \\ 3 \end{array}] - [\begin{array}{c} 4 \\ 5 \\ 6 \end{array}] = [\begin{array}{c} 1 - 4 \\ 2 - 5 \\ 3 - 6 \end{array}] = [\begin{array}{c} - 3 \\ - 3 \\ - 3 \end{array}], \\ b + c - a & = [\begin{array}{c} 4 \\ 5 \\ 6 \end{array}] + [\begin{array}{c} 7 \\ - 3 \\ 0 \end{array}] - [\begin{array}{c} 1 \\ 2 \\ 3 \end{array}] = [\begin{array}{c} 4 + 7 - 1 \\ 5 + (- 3) - 2 \\ 6 + 0 - 3 \end{array}] = [\begin{array}{c} 10 \\ 0 \\ 3 \end{array}] . \end{aligned}

A vector cannot be added or subtracted with a scalar, or with a vector of a different dimension. Also, just like addition and subtraction of scalars, vector addition is commutative,

a + b = b + a,

whereas vector subtraction is anticommutative,

a - b = - (b - a) .

2.7.2Geometric Interpretation

Figure 2.102D vector addition and subtraction using the triangle rule.

We can add vectors $a$ and $b$ geometrically by positioning the vectors so that the head of $a$ touches the tail of $b$ and then drawing a vector from the tail of $a$ to the head of $b$ . In other words, if we start at a point and apply the displacements specified by $a$ and then $b$ , it's the same as if we had applied the single displacement $a + b$ . This is known as the triangle rule of vector addition. It also works for vector subtraction, as shown in Figure 2.10.

Figure 2.10 provides geometric evidence that vector addition is commutative but vector subtraction is not. Notice that the vector labeled $a + b$ is identical to the vector labeled $b + a$ , but the vectors $d - c$ and $c - d$ point in opposite directions because $d - c = - (c - d)$ .

Figure 2.11Extending the triangle rule to more than two vectors

The triangle rule can be extended to more than two vectors. Figure 2.11 shows how the triangle rule verifies something we stated in Section 2.3.1: a vector can be interpreted as a sequence of axially aligned displacements.

Figure 2.12 is a reproduction of Figure 2.5, which shows how the vector $[1, - 3, 4]$ may be interpreted as a displacement of 1 unit to the right, 3 units down, and then 4 units forward, and can be verified mathematically by using vector addition:

[\begin{matrix} 1 \\ - 3 \\ 4 \end{matrix}] = [\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}] + [\begin{matrix} 0 \\ - 3 \\ 0 \end{matrix}] + [\begin{matrix} 0 \\ 0 \\ 4 \end{matrix}] .

Figure 2.12Interpreting a vector as a sequence of displacements

This seems obvious, but this is a very powerful concept. We will use a similar technique in Section 4.2 to transform vectors from one coordinate space to another.

2.7.3Displacement Vector from One Point to Another

It is very common that we will need to compute the displacement from one point to another. In this case, we can use the triangle rule and vector subtraction. Figure 2.13 shows how the displacement vector from $a$ to $b$ can be computed by subtracting $a$ from $b$ .

Figure 2.13 Using 2D vector subtraction to compute the vector from point

a

to point

b

As Figure 2.13 shows, to compute the vector from $a$ to $b$ , we interpret the points $a$ and $b$ as vectors from the origin, and then use the triangle rule. In fact, this is how vectors are defined in some texts: the subtraction of two points.

Notice that the vector subtraction $b - a$ yields a vector from $a$ to $b$ . It doesn't make any sense to simply find the vector “between two points,” since the language in this sentence does not specify a direction. We must always form a vector that goes from one point to another point.

2.8Vector Magnitude (Length)

As we have discussed, vectors have magnitude and direction. However, you might have noticed that neither the magnitude nor the direction is expressed explicitly in the vector (at least not when we use Cartesian coordinates). For example, the magnitude of the 2D vector $[3, 4]$ is neither 3 nor 4; it's 5. Since the magnitude of the vector is not expressed explicitly, we must compute it. The magnitude of a vector is also known as the length or norm of the vector.

2.8.1Official Linear Algebra Rules

In linear algebra, the magnitude of a vector is denoted by using double vertical bars surrounding the vector. This is similar to the single vertical bar notation used for the absolute value operation for scalars. This notation and the equation for computing the magnitude of a vector of arbitrary dimension $n$ are shown in Equation (2.2):

Magnitude of a vector of arbitrary dimension

\begin{matrix} (2.2) & ∥ v ∥ = \sqrt{\sum_{i = 1}^{n} {v_{i}}^{2}} = \sqrt{{v_{1}}^{2} + {v_{2}}^{2} + \dots + {v_{n - 1}}^{2} + {v_{n}}^{2}} . \end{matrix}

Thus, the magnitude of a vector is the square root of the sum of the squares of the components of the vector. This sounds complicated, but the magnitude equations for 2D and 3D vectors are actually very simple:

Vector magnitude for 2D and 3D vectors

\begin{aligned} (2.3) & ∥ v ∥ & = \sqrt{{v_{x}}^{2} + {v_{y}}^{2}} & (for a 2D vector\ v), \\ ∥ v ∥ & = \sqrt{{v_{x}}^{2} + {v_{y}}^{2} + {v_{z}}^{2}} & (for a 3D vector\ v) . \end{aligned}

The magnitude of a vector is a nonnegative scalar quantity. An example of how to compute the magnitude of a 3D vector is

\begin{aligned} \begin{aligned} ∥ [\begin{array}{c} 5 & - 4 & 7 \end{array}] ∥ & = \sqrt{5^{2} + {(- 4)}^{2} + 7^{2}} = \sqrt{25 + 16 + 49} = \sqrt{90} \\ \approx 9.4868 . \end{aligned} \end{aligned}

Some books use a single bar notation to indicate vector magnitude:

| v |

One quick note to satisfy all you sticklers who already know about vector norms and at this moment are pointing your web browser to gamemath.com, looking for the email address for errata. The term norm actually has a very general definition, and basically any equation that meets a certain set of criteria can call itself a norm. So to describe Equation (2.2) as the equation for the vector norm is slightly misleading. To be more accurate, we should say that Equation (2.2) is the equation for the 2-norm, which is one specific way to calculate a norm. The 2-norm belongs to a class of norms known as the $p$ -norms, and the $p$ -norm is not the only way to define a norm. Still, omitting this level of generality isn't too harmful of a delusion; because the 2-norm measures Euclidian distance, it is by far the most commonly used norm in geometric applications. It is also widely used in situations even where a geometric interpretation is not directly applicable. Readers interested in such exotica should check out Exercise 15.

2.8.2Geometric Interpretation

Figure 2.14Geometric interpretation of the magnitude equation

Let's try to get a better understanding of why Equation (2.3) works. For any vector $v$ in 2D, we can form a right triangle with $v$ as the hypotenuse, as shown in Figure 2.14.

Notice that to be precise we had to put absolute value signs around the components $v_{x}$ and $v_{y}$ . The components of the vector may be negative, since they are signed displacements, but length is always positive.

The Pythagorean theorem states that for any right triangle, the square of the length of the hypotenuse is equal to the sum of the squares of the lengths of the other two sides. Applying this theorem to Figure 2.14, we have

{∥ v ∥}^{2} = {| v_{x} |}^{2} + {| v_{y} |}^{2} .

Since ${| x |}^{2} = x^{2}$ , we can omit the absolute value symbols:

{∥ v ∥}^{2} = {v_{x}}^{2} + {v_{y}}^{2} .

Then, by taking the square root of both sides and simplifying, we get

\begin{aligned} \sqrt{{∥ v ∥}^{2}} & = \sqrt{{v_{x}}^{2} + {v_{y}}^{2}}, \\ ∥ v ∥ & = \sqrt{{v_{x}}^{2} + {v_{y}}^{2}}, \end{aligned}

which is the same as Equation (2.3). The proof of the magnitude equation in 3D is only slightly more complicated.

For any positive magnitude $m$ , there are an infinite number of vectors of magnitude $m$ . Since these vectors all have the same length but different directions, they form a circle when the tails are placed at the origin, as shown in Figure 2.15.

Figure 2.15 For any positive magnitude, there are an infinite number of vectors with that magnitude

2.9Unit Vectors

For many vector quantities, we are concerned only with direction and not magnitude: “Which way am I facing?” “Which way is the surface oriented?” In these cases, it is often convenient to use unit vectors. A unit vector is a vector that has a magnitude of one. Unit vectors are also known as normalized vectors.

Unit vectors are also sometimes simply called normals; however, a warning is in order concerning terminology. The word “normal” carries with it the connotation of “perpendicular.” When most people speak of a “normal” vector, they are usually referring to a vector that is perpendicular to something. For example, a surface normal at a given point on an object is a vector that is perpendicular to the surface at that location. However, since the concept of perpendicular is related only to the direction of a vector and not its magnitude, in most cases you will find that unit vectors are used for normals instead of a vector of arbitrary length. When this book refers to a vector as a “normal,” it means “a unit vector perpendicular to something else.” This is common usage, but be warned that the word “normal” primarily means “perpendicular” and not “unit length.” Since it is so common for normals to be unit vectors, we will take care to call out any situation where a “normal” vector does not have unit length.

In summary, a “normalized” vector always has unit length, but a “normal” vector is a vector that is perpendicular to something and by convention usually has unit length.

2.9.1Official Linear Algebra Rules

Figure 2.16Normalizing vectors in 2D

For any nonzero vector $v$ , we can compute a unit vector that points in the same direction as $v$ . This process is known as normalizing the vector. In this book we use a common notation of putting a hat symbol over unit vectors; for example, $\hat{v}$ (pronounced “v hat”). To normalize a vector, we divide the vector by its magnitude:

Normalizing a vector

\hat{v} = \frac{v}{∥ v ∥} for any nonzero vector v .

For example, to normalize the 2D vector $[12, - 5]$ ,

\begin{aligned} \begin{aligned} \frac{[\begin{array}{c} 12 & - 5 \end{array}]}{∥ [\begin{array}{c} 12 & - 5 \end{array}] ∥} & = \frac{[\begin{array}{c} 12 & - 5 \end{array}]}{\sqrt{12^{2} + 5^{2}}} = \frac{[\begin{array}{c} 12 & - 5 \end{array}]}{\sqrt{169}} = \frac{[\begin{array}{c} 12 & - 5 \end{array}]}{13} = [\begin{array}{c} \frac{12}{13} & \frac{- 5}{13} \end{array}] \\ \approx [\begin{array}{c} 0.923 & - 0.385 \end{array}] . \end{aligned} \end{aligned}

The zero vector cannot be normalized. Mathematically, this is not allowed because it would result in division by zero. Geometrically, it makes sense because the zero vector does not define a direction—if we normalized the zero vector, in what direction should the resulting vector point?

2.9.2Geometric Interpretation

In 2D, if we draw a unit vector with the tail at the origin, the head of the vector will touch a unit circle centered at the origin. (A unit circle has a radius of 1.) In 3D, unit vectors touch the surface of a unit sphere. Figure 2.16 shows several 2D vectors of arbitrary length in gray, beneath their normalized counterparts in black.

Notice that normalizing a vector makes some vectors shorter (if their length was greater than 1) and some vectors longer (if their length was less than 1).

2.10The Distance Formula

We are now prepared to derive one of the oldest and most fundamental formulas in computational geometry: the distance formula. This formula is used to compute the distance between two points.

First, let's define distance as the length of the line segment between the two points. Since a vector is a directed line segment, geometrically it makes sense that the distance between the two points would be equal to the length of a vector from one point to the other. Let's derive the distance formula in 3D. First, we will compute the vector $d$ from $a$ to $b$ . We learned how to do this in 2D in Section 2.7.3. In 3D, we use

d = b - a = [\begin{matrix} b_{x} - a_{x} \\ b_{y} - a_{y} \\ b_{z} - a_{z} \end{matrix}] .

The distance between $a$ and $b$ is equal to the length of the vector $d$ , which we computed in Section 2.8:

d i s t a n c e (a, b) = ∥ d ∥ = \sqrt{{d_{x}}^{2} + {d_{y}}^{2} + {d_{z}}^{2}} .

Substituting for $d$ , we get

The 3D distance formula

d i s t a n c e (a, b) = ∥ b - a ∥ = \sqrt{(b_{x} - a_{x})^{2} + (b_{y} - a_{y})^{2} + (b_{z} - a_{z})^{2}} .

Thus, we have derived the distance formula in 3D. The 2D equation is even simpler:

The 2D distance formula

d i s t a n c e (a, b) = ∥ b - a ∥ = \sqrt{(b_{x} - a_{x})^{2} + (b_{y} - a_{y})^{2}} .

Let's look at an example in 2D:

\begin{aligned} \begin{aligned} d i s t a n c e ([\begin{array}{c} 5 & 0 \end{array}], [\begin{array}{c} - 1 & 8 \end{array}]) & = \sqrt{(- 1 - 5)^{2} + (8 - 0)^{2}} \\ = \sqrt{(- 6)^{2} + 8^{2}} = \sqrt{100} = 10. \end{aligned} \end{aligned}

Notice that it doesn't matter which point we call $a$ and which point we call $b$ . If we define $d$ to be the vector from $b$ to $a$ instead of from $a$ to $b$ , we will derive a slightly different, but mathematically equivalent, equation.

2.11Vector Dot Product

Section 2.6 showed how to multiply a vector by a scalar. We can also multiply two vectors together. There are two types of vector products. The first vector product is the dot product (also known as the inner product), the subject of this section. We talk about the other vector product, the cross product, in Section 2.12.

The dot product is ubiquitous in video game programming, useful in everything from graphics, to simulation, to AI. Following the pattern we used for the operations, we first discuss the algebraic rules for computing dot products in Section 2.11.1, followed by some geometric interpretations in Section 2.11.2.

The dot product formula is one of the few formulas in this book worth memorizing. First of all, it's really easy to memorize. Also, if you understand what the dot product does, the formula makes sense. Furthermore, the dot product has important relationships to many other operations, such as matrix multiplication, convolution of signals, statistical correlations, and Fourier transforms. Understanding the formula will make these relationships more apparent.

Even more important than memorizing a formula is to get an intuitive grasp for what the dot product does. If there is only enough space in your brain for either the formula or the geometric definition, then we recommend internalizing the geometry, and getting the formula tattooed on your hand. You need to understand the geometric definition in order to use the dot product. When programming in computer languages such as C++, HLSL, or even Matlab and Maple, you won't need to know the formula anyway, since you will usually tell the computer to do a dot product calculation not by typing in the formula, but by invoking a high-level function or overloaded operator. Furthermore, the geometric definition of the dot product does not assume any particular coordinate frame or even the use of Cartesian coordinates.

2.11.1Official Linear Algebra Rules

The name “dot product” comes from the dot symbol used in the notation: $a \cdot b$ . Just like scalar-times-vector multiplication, the vector dot product is performed before addition and subtraction, unless parentheses are used to override this default order of operations. Note that although we usually omit the multiplication symbol when multiplying two scalars or a scalar and a vector, we must not omit the dot symbol when performing a vector dot product. If you ever see two vectors placed side-by-side with no symbol in between, interpret this according to the rules of matrix multiplication, which we discuss in Chapter 4.7

The dot product of two vectors is the sum of the products of corresponding components, resulting in a scalar:

Vector dot product

[\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n - 1} \\ a_{n} \end{matrix}] \cdot [\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{n - 1} \\ b_{n} \end{matrix}] = a_{1} b_{1} + a_{2} b_{2} + \dots + a_{n - 1} b_{n - 1} + a_{n} b_{n} .

This can be expressed succinctly by using the summation notation

Dot product using summation notation

a \cdot b = \sum_{i = 1}^{n} a_{i} b_{i} .

Applying these rules to the 2D and 3D cases yields

2D and 3D dot products

\begin{aligned} a \cdot b & = a_{x} b_{x} + a_{y} b_{y} & (a and b are 2D vectors), \\ a \cdot b & = a_{x} b_{x} + a_{y} b_{y} + a_{z} b_{z} & (a and b are 3D vectors) . \end{aligned}

Examples of the dot product in 2D and 3D are

\begin{aligned} [\begin{array}{c} 4 & 6 \end{array}] \cdot [\begin{array}{c} - 3 & 7 \end{array}] & = (4) (- 3) + (6) (7) = 30, \\ [\begin{array}{c} 3 \\ - 2 \\ 7 \end{array}] \cdot [\begin{array}{c} 0 \\ 4 \\ - 1 \end{array}] & = (3) (0) + (- 2) (4) + (7) (- 1) = - 15. \end{aligned}

It is obvious from inspection of the equations that vector dot product is commutative: $a \cdot b = b \cdot a$ . More vector algebra laws concerning the dot product are given in Section 2.13.

2.11.2Geometric Interpretation

Now let's discuss the more important aspect of the dot product: what it means geometrically. It would be difficult to make too big of a deal out of the dot product, as it is fundamental to almost every aspect of 3D math. Because of its supreme importance, we're going to dwell on it a bit. We'll discuss two slightly different ways of thinking about this operation geometrically; since they are really equivalent, you may or may not think one interpretation or the other is “more fundamental,” or perhaps you may think we are being redundant and wasting your time. You might especially think this if you already have some exposure to the dot product, but please indulge us.

The first geometric definition to present is perhaps the less common of the two, but in agreement with the advice of Dray and Manogue [1], we believe it's actually the more useful. The interpretation we first consider is that of the dot product performing a projection.

Assume for the moment that $\hat{a}$ is a unit vector, and $b$ is a vector of any length. Now take $b$ and project it onto a line parallel to $\hat{a}$ , as in Figure 2.17.

Figure 2.17The dot product as a projection

(Remember that vectors are displacements and do not have a fixed position, so we are free to move them around on a diagram anywhere we wish.) We can define the dot product $\hat{a} \cdot b$ as the signed length of the projection of $b$ onto this line. The term “projection” has a few different technical meanings (see Section 5.3) and we won't bother attempting a formal definition here.8 You can think of the projection of $b$ onto $\hat{a}$ as the “shadow” that $b$ casts on $\hat{a}$ when the rays of light are perpendicular to $\hat{a}$ .

We have drawn the projections as arrows, but remember that the result of a dot product is a scalar, not a vector. Still, when you first learned about negative numbers, your teacher probably depicted numbers as arrows on a number line, to emphasize their sign, just as we have. After all, a scalar is a perfectly valid one-dimensional vector.

What does it mean for the dot product to measure a signed length? It means the value will be negative when the projection of $b$ points in the opposite direction from $\hat{a}$ , and the projection has zero length (it is a single point) when $\hat{a}$ and $b$ are perpendicular. These cases are illustrated in Figure 2.18.

Figure 2.18Sign of the dot product

In other words, the sign of the dot product can give us a rough classification of the relative directions of the two vectors. Imagine a line (in 2D) or plane (in 3D) perpendicular to the vector $\hat{a}$ . The sign of the dot product $\hat{a} \cdot b$ tells us which half-space $b$ lies in. This is illustrated in Figure 2.19.

Figure 2.19 The sign of the dot product gives a rough classification of the relative orientation of two vectors.

Next, consider what happens when we scale $b$ by some factor $k$ . As shown in Figure 2.20, the length of the projection (and thus the value of the dot product) increases by the same factor. The two triangles have equal interior angles and thus are similar. Since the hypotenuse on the right is longer than the hypotenuse on the left by a factor of $k$ , by the properties of similar triangles, the base on the right is also longer by a factor of $k$ .

Figure 2.20Scaling one operand of the dot product

Let's state this fact algebraically and prove it by using the formula:

Dot product is associative with multiplication by a scalar

\begin{aligned} \begin{aligned} \hat{a} \cdot (k b) & = a_{x} (k b_{x}) + a_{y} (k b_{y}) + a_{z} (k b_{z}) \\ = k (a_{x} b_{x} + a_{y} b_{y} + a_{z} b_{z}) \\ = k (\hat{a} \cdot b) . \end{aligned} \end{aligned}

The expanded scalar math in the middle uses three dimensions as our example, but the vector notation at either end of the equation applies for vectors of any dimension.

We've seen what happens when we scale $b$ : the length of its projection onto $\hat{a}$ increases along with the value of the dot product. What if we scale $a$ ? The algebraic argument we just made can be used to show that the value of the dot product scales with the length of $a$ , just like it does when we scale $b$ . In other words,

Dot product is associative with multiplication by a scalar for either vector

(k a) \cdot b = k (a \cdot b) = a \cdot (k b) .

So scaling $a$ scales the numeric value of the dot product. However, this scale has no affect geometrically on the length of the projection of $b$ onto $a$ . Now that we know what happens if we scale either $a$ or $b$ , we can write our geometric definition without any assumptions about the length of the vectors.

Dot Product as Projection

The dot product

a \cdot b

is equal to the signed length of the projection of

b

onto any line parallel to

a

, multiplied by the length of

a

As we continue to examine the properties of the dot product, some will be easiest to illustrate geometrically when either $a$ , or both $a$ and $b$ , are unit vectors. Because we have shown that scaling either $a$ or $b$ directly scales the value of the dot product, it will be easy to generalize our results after we have obtained them. Furthermore, in the algebraic arguments that accompany each geometric argument, unit vector assumptions won't be necessary. Remember that we put hats on top of vectors that are assumed to have unit length.

You may well wonder why the dot product measures the projection of the second operand onto the first, and not the other way around. When the two vectors $\hat{a}$ and $\hat{b}$ are unit vectors, we can easily make a geometric argument that the projection of $\hat{a}$ onto $\hat{b}$ has the same length as the projection of $\hat{b}$ onto $\hat{a}$ . Consider Figure 2.21. The two triangles have equal interior angles and thus are similar. Since $\hat{a}$ and $\hat{b}$ are corresponding sides and have the same length, the two triangles are reflections of each other.

Figure 2.21Dot product is commutative

We've already shown how scaling either vector will scale the dot product proportionally, so this result applies for $a$ and $b$ with arbitrary length. Furthermore, this geometric fact is also trivially verified by using the formula, which does not depend on the assumption that the vectors have equal length. Using two dimensions as our example this time,

a \cdot b = a_{x} b_{x} + a_{y} b_{y} = b_{x} a_{x} + b_{y} a_{y} = b \cdot a .

Dot product is commutative

The next important property of the dot product is that it distributes over addition and subtraction, just like scalar multiplication. This time let's do the algebra before the geometry. When we say that the dot product “distributes,” that means that if one of the operands to the dot product is a sum, then we can take the dot product of the pieces individually, and then take their sum. Switching back to three dimensions for our example,

Dot product distributes over addition and subtraction

\begin{aligned} a \cdot (b + c) & = [\begin{array}{c} a_{x} \\ a_{y} \\ a_{z} \end{array}] \cdot [\begin{array}{c} b_{x} + c_{x} \\ b_{y} + c_{y} \\ b_{z} + c_{z} \end{array}] \\ = a_{x} (b_{x} + c_{x}) + a_{y} (b_{y} + c_{y}) + a_{z} (b_{z} + c_{z}) \\ = a_{x} b_{x} + a_{x} c_{x} + a_{y} b_{y} + a_{y} c_{y} + a_{z} b_{z} + a_{z} c_{z} \\ = (a_{x} b_{x} + a_{y} b_{y} + a_{z} b_{z}) + (a_{x} c_{x} + a_{y} c_{y} + a_{z} c_{z}) \\ = a \cdot b + a \cdot c . \end{aligned}

By replacing $c$ with $- c$ , it's clear that the dot product distributes over vector subtraction just as it does for vector addition. Figure 2.22 shows how the dot product distributes over addition.

Figure 2.22The dot product distributes over addition.

Now let's look at a special situation in which one of the vectors is the unit vector pointing in the $+ x$ direction, which we'll denote as $\hat{x}$ . As shown in Figure 2.23, the signed length of the projection is simply the $x$ -coordinate of the original vector. In other words, taking the dot product of a vector with a cardinal axis “sifts” out the coordinate for that axis.

If we combine this “sifting” property of the dot product with the fact that it distributes over addition, which we have been able to show in purely geometric terms, we can see why the formula has to be what it is.

Figure 2.23 Taking the dot product with a cardinal axis sifts out the corresponding coordinate.

Because the dot product measures the length of a projection, it has an interesting relationship to the vector magnitude calculation. Remember that the vector magnitude is a scalar measuring the amount of displacement (the length) of the vector. The dot product also measures the amount of displacement, but only the displacement in a particular direction is counted; perpendicular displacement is discarded by the projecting process. But what if we measure the displacement in the same direction that the vector is pointing? In this case, all of the vector's displacement is in the direction being measured, so if we project a vector onto itself, the length of that projection is simply the magnitude of the vector. But remember that $a \cdot b$ is equal to the length of the projection of $b$ onto $a$ , scaled by $∥ a ∥$ . If we dot a vector with itself, such as $v \cdot v$ , we get the length of the projection, which is $∥ v ∥$ , times the length of the vector we are projecting onto, which is also $∥ v ∥$ . In other words,

Relationship between vector magnitude and the dot product

\begin{aligned} v \cdot v & = {∥ v ∥}^{2}, & ∥ v ∥ & = \sqrt{v \cdot v} . \end{aligned}

Before we switch to the second interpretation of the dot product, let's check out one more very common use of the dot product as a projection. Assume once more that $\hat{a}$ is a unit vector and $b$ has arbitrary length. Using the dot product, it's possible to separate $b$ into two values, $b_{∥}$ and $b_{⊥}$ (read “ $b$ parallel” and “ $b$ perp”), which are parallel and perpendicular to $\hat{a}$ , respectively, such that $b = b_{∥} + b_{⊥}$ . Figure 2.24 illustrates the geometry involved.

Figure 2.24Projecting one vector onto another

We've already established that the length of $b_{∥}$ will be equal to $\hat{a} \cdot b$ . But the dot product yields a scalar, and $b_{∥}$ is a vector, so we'll take the direction specified by the unit vector $\hat{a}$ and scale it up:

b_{∥} = (\hat{a} \cdot b) \hat{a} .

Once we know $b_{∥}$ , we can easily solve for $b_{⊥}$ :

\begin{aligned} b_{⊥} + b_{∥} & = b, \\ b_{⊥} & = b - b_{∥}, \\ b_{⊥} & = b - (\hat{a} \cdot b) \hat{a} . \end{aligned}

It's not too difficult to generalize these results to the case where $a$ is not a unit vector.

In the rest of this book, we make use of these equations several times to separate a vector into components that are parallel and perpendicular to another vector.

Now let's examine the dot product through the lens of trigonometry. This is the more common geometric interpretation of the dot product, which places a bit more emphasis on the angle between the vectors. We've been thinking in terms of projections, so we haven't had much need for this angle. Less experienced and conscientious authors [2] might give you just one of the two important viewpoints, which is probably sufficient to interpret an equation that contains the dot product. However, a more valuable skill is to recognize situations for which the dot product is the correct tool for the job; sometimes it helps to have other interpretations pointed out, even if they are “obviously” equivalent to each other.

Figure 2.25 Interpreting the dot product by using the trigonometry of the right triangle

Consider the right triangle on the right-hand side of Figure 2.25. As the figure shows, the length of the hypotenuse is 1 (since $\hat{b}$ is a unit vector) and the length of the base is equal to the dot product $\hat{a} \cdot \hat{b}$ . From elementary trig (which was reviewed in Section 1.4.4), remember that the cosine of an angle is the ratio of the length of the adjacent leg divided by the length of the hypotenuse. Plugging in the values from Figure 2.25, we have

\begin{aligned} \cos θ & = \frac{adjacent}{hypotenuse} = \frac{\hat{a} \cdot \hat{b}}{1} = \hat{a} \cdot \hat{b} . \end{aligned}

In other words, the dot product of two unit vectors is equal to the cosine of the angle between them. This statement is true even if the right triangle in Figure 2.25 cannot be formed, when $\hat{a} \cdot \hat{b} \leq 0$ and $θ > 90^{o}$ . Remember that the dot product of any vector with the vector $\hat{x} = [1, 0, 0]$ will simply extract the $x$ -coordinate of the vector. In fact, the $x$ -coordinate of a unit vector that has been rotated by an angle of $θ$ from standard position is one way to define the value of $\cos θ$ . Review Section 1.4.4 if this isn't fresh in your memory.

By combining these ideas with the previous observation that scaling either vector scales the dot product by the same factor, we arrive at the general relationship between the dot product and the cosine.

Dot Product Relation to Intercepted Angle

The dot product of two vectors

a

and

b

is equal to the cosine of the angle

θ

between the vectors, multiplied by the lengths of the vectors (see Figure 2.26). Stated formally,

\begin{matrix} (2.4) & a \cdot b = ∥ a ∥ ∥ b ∥ \cos θ . \end{matrix}

Figure 2.26The dot product is related to the angle between two vectors.

What does it mean to measure the angle between two vectors in 3D? Any two vectors will always lie in a common plane (place them tail to tail to see this), and so we measure the angle in the plane that contains both vectors. If the vectors are parallel, the plane is not unique, but the angle is either $0^{o}$ or $\pm 180^{o}$ , and it doesn't matter which plane we choose.

The dot product provides a way for us to compute the angle between two vectors. Solving Equation (2.4) for $θ$ ,

Using the dot product to compute the angle between two vectors

\begin{aligned} (2.5) & θ & = \arccos (\frac{a \cdot b}{∥ a ∥ ∥ b ∥}) . \end{aligned}

We can avoid the division in Equation (2.5) if we know that $a$ and $b$ are unit vectors. In this very common case, the denominator of Equation (2.5) is trivially 1, and we are left with

Computing the angle between two unit vectors

\begin{aligned} θ & = \arccos (\hat{a} \cdot \hat{b}) & (assume \hat{a} and \hat{b} are unit vectors) . \end{aligned}

If we do not need the exact value of $θ$ , and need only a classification of the relative orientation of $a$ and $b$ , then we need only the sign of the dot product. This is the same idea illustrated in Figure 2.18, only now we can relate it to the angle $θ$ , as shown in Table 2.1.

$a \cdot b$	$θ$	Angle is	$a$ and $b$ are
$> 0$	$0^{o} \leq θ < 90^{o}$	acute	pointing mostly in the same direction
$0$	$θ = 90^{o}$	right	perpendicular
$< 0$	$90^{o} < θ \leq 180^{o}$	obtuse	pointing mostly in the opposite direction

Table 2.1 The sign of the dot product can be used as a rough classification of the angle between two vectors.

Since the magnitude of the vectors does not affect the sign of the dot product, Table 2.1 applies regardless of the lengths of $a$ and $b$ . However, notice that if either $a$ or $b$ is the zero vector, then $a \cdot b = 0$ . Thus, when we use the dot product to classify the relationship between two vectors, the dot product acts as if the zero vector is perpendicular to any other vector. As it turns out, the cross product behaves differently.

Let's summarize the dot product's geometric properties.

The dot product $a \cdot b$ measures the length of the projection of $b$ onto $a$ , multiplied by the length of $a$ .
The dot product can be used to measure displacement in a particular direction.
The projection operation is closely related to the cosine function. The dot product $a \cdot b$ also is equal to $∥ a ∥ ∥ b ∥ \cos θ$ , where $θ$ is the angle between the vectors.

We review the commutative and distributive properties of the dot product at the end of this chapter along with other algebraic properties of vector operations.

2.12Vector Cross Product

The other vector product, known as the cross product, can be applied only in 3D. Unlike the dot product, which yields a scalar and is commutative, the vector cross product yields a 3D vector and is not commutative.

2.12.1Official Linear Algebra Rules

Similar to the dot product, the term “cross” product comes from the symbol used in the notation $a \times b$ . We always write the cross symbol, rather than omitting it as we do with scalar multiplication. The equation for the cross product is

Cross product

[\begin{matrix} x_{1} \\ y_{1} \\ z_{1} \end{matrix}] \times [\begin{matrix} x_{2} \\ y_{2} \\ z_{2} \end{matrix}] = [\begin{matrix} y_{1} z_{2} - z_{1} y_{2} \\ z_{1} x_{2} - x_{1} z_{2} \\ x_{1} y_{2} - y_{1} x_{2} \end{matrix}] .

For example,

[\begin{matrix} 1 \\ 3 \\ 4 \end{matrix}] \times [\begin{matrix} 2 \\ - 5 \\ 8 \end{matrix}] = [\begin{matrix} (3) (8) - (4) (- 5) \\ (4) (2) - (1) (8) \\ (1) (- 5) - (3) (2) \end{matrix}] = [\begin{matrix} 24 - (- 20) \\ 8 - 8 \\ - 5 - 6 \end{matrix}] = [\begin{matrix} 44 \\ 0 \\ - 11 \end{matrix}] .

The cross product enjoys the same level of operator precedence as the dot product: multiplication occurs before addition and subtraction. When dot product and cross product are used together, the cross product takes precedence: $a \cdot b \times c = a \cdot (b \times c)$ . Luckily, there's an easy way to remember this: it's the only way it could work. The dot product returns a scalar, and so $(a \cdot b) \times c$ is undefined, since you cannot take the cross product of a scalar and a vector. The operation $a \cdot (b \times c)$ is known as the triple product. We present some special properties of this computation in Section 6.1.

As mentioned earlier, the vector cross product is not commutative. In fact, it is anticommutative: $a \times b = - (b \times a)$ . The cross product is not associative, either. In general, $(a \times b) \times c \neq a \times (b \times c)$ . More vector algebra laws concerning the cross product are given in Section 2.13.

2.12.2Geometric Interpretation

The cross product yields a vector that is perpendicular to the original two vectors, as illustrated in Figure 2.27.

Figure 2.27Vector cross product

The length of $a \times b$ is equal to the product of the magnitudes of $a$ and $b$ and the sine of the angle between $a$ and $b$ :

The magnitude of the cross product is related to the sine of the angle between the vectors

∥ a \times b ∥ = ∥ a ∥ ∥ b ∥ \sin θ .

As it turns out, this is also equal to the area of the parallelogram formed with two sides $a$ and $b$ . Let's see if we can verify why this is true by using Figure 2.28.

Figure 2.28A parallelogram with sides

a

and

b

Figure 2.29Area of a parallelogram

First, from planar geometry, we know that the area of the parallelogram is $b h$ , the product of the base and the height. (In Figure 2.28, the base is $b = ∥ b ∥$ .) We can verify this rule by “clipping” off a triangle from one end and moving it to the other end, forming a rectangle, as shown in Figure 2.29.

The area of a rectangle is given by its length and width. In this case, this area is the product $b h$ . Since the area of the rectangle is equal to the area of the parallelogram, the area of the parallelogram must also be $b h$ .

Returning to Figure 2.28, let $a$ and $b$ be the lengths of $a$ and $b$ , respectively, and note that $\sin θ = h / a$ . Then

\begin{aligned} A & = b h \\ = b (a \sin θ) \\ = ∥ a ∥ ∥ b ∥ \sin θ \\ = ∥ a \times b ∥ . \end{aligned}

If $a$ and $b$ are parallel, or if $a$ or $b$ is the zero vector, then $a \times b = 0$ . So the cross product interprets the zero vector as being parallel to every other vector. Notice that this is different from the dot product, which interprets the zero vector as being perpendicular to every other vector. (Of course, it is ill-defined to describe the zero vector as being perpendicular or parallel to any vector, since the zero vector has no direction.)

We have stated that $a \times b$ is perpendicular to $a$ and $b$ . But there are two directions that are perpendicular to $a$ and $b$ —which of these two directions does $a \times b$ point? We can determine the direction of $a \times b$ by placing the tail of $b$ at the head of $a$ , and examining whether we make a clockwise or counterclockwise turn from $a$ to $b$ . In a left-handed coordinate system, $a \times b$ points towards you if the vectors $a$ and $b$ make a clockwise turn from your viewpoint, and away from you if $a$ and $b$ make a counterclockwise turn. In a right-handed coordinate system, the exact opposite occurs: if $a$ and $b$ make a counterclockwise turn, $a \times b$ points towards you, and if $a$ and $b$ make a clockwise turn, $a \times b$ points away from you.

Clockwise turn	Counterclockwise turn



In a left-handed coordinate system, $a \times b$ (not shown) points towards you. In a right-handed coordinate system, $a \times b$ points away from you.	In a left-handed coordinate system, $a \times b$ (not shown) points away from you. In a right-handed coordinate system, $a \times b$ points towards you.

Figure 2.30Determining clockwise versus counterclockwise turns

Figure 2.30 shows clockwise and counterclockwise turns. Notice that to make the clockwise or counterclockwise determination, we must align the

head of $a$ with the tail of $b$ . Compare this to Figure 2.26, where the tails are touching. The tail-to-tail alignment shown in Figure 2.26 is the correct way to position the vectors to measure the angle between them, but to judge whether the turn is clockwise or counterclockwise, the vectors should be aligned head-to-tail, as shown in Figure 2.30.

Let's apply this general rule to the specific case of the cardinal axes. Let $\hat{x}$ , $\hat{y}$ , and $\hat{z}$ be unit vectors that point in the $+ x$ , $+ y$ , and $+ z$ directions, respectively. The results of taking the cross product of each pair of axes are

Cross product of the cardinal axes

\begin{array}{rclrcl} \hat{x} & \hat{y} & \hat{z}, & \hat{y} & \hat{x} & - \hat{z}, \\ \hat{y} & \hat{z} & \hat{x}, & \hat{z} & \hat{y} & - \hat{x}, \\ \hat{z} & \hat{x} & \hat{y}, & \hat{x} & \hat{z} & - \hat{y} . \end{array}

You can also remember which way the cross product points by using your hand, similar to the way we distinguished between left-handed and right-handed coordinate spaces in Section 1.3.3. Since we're using a left-handed coordinate space in this book, we'll show how it's done using your left hand. Let's say you have two vectors, $a$ and $b$ , and you want to figure out which direction $a \times b$ points. Point your thumb in the direction of $a$ , and your index finger (approximately) in the direction of $b$ . If $a$ and $b$ are pointing in nearly the opposite direction, this may be difficult. Just make sure that if your thumb points exactly in the direction of $a$ ; then your index finger is on the same side of $a$ as the vector $b$ is. With your fingers in this position, extend your third finger to be perpendicular to your thumb and index finger, similar to what we did in Section 1.3.3. Your third finger now points in the direction of $a \times b$ .

Of course, a similar trick works with your right hand for right-handed coordinate spaces.

One of the most important uses of the cross product is to create a vector that is perpendicular to a plane (see Section 9.5), triangle (Section 9.6), or polygon (Section 9.7).

2.13Linear Algebra Identities

The Greek philosopher Arcesilaus reportedly said, “Where you find the laws most numerous, there you will find also the greatest injustice.” Well, nobody said vector algebra was fair. Table 2.2 lists some vector algebra laws that are occasionally useful but should not be memorized. Several identities are obvious and are listed for the sake of completeness; all of them can be derived from the definitions given in earlier sections.

Identity	Comments
$a + b = b + a$	Commutative property of vector addition
$a - b = a + (- b)$	Definition of vector subtraction
$(a + b) + c = a + (b + c)$	Associative property of vector addition
$s (t a) = (s t) a$	Associative property of scalar multiplication
$k (a + b) = k a + k b$	Scalar multiplication distributes over vector addition
$∥ k a ∥ = \| k \| ∥ a ∥$	Multiplying a vector by a scalar scales the magnitude by a factor equal to the absolute value of the scalar
$∥ a ∥ \geq 0$	The magnitude of a vector is nonnegative
${∥ a ∥}^{2} + {∥ b ∥}^{2} = {∥ a + b ∥}^{2}$	The Pythagorean theorem applied to vector addition.
$∥ a ∥ + ∥ b ∥ \geq ∥ a + b ∥$	Triangle rule of vector addition. (No side can be longer than the sum of the other two sides.)
$a \cdot b = b \cdot a$	Commutative property of dot product
$∥ a ∥ = \sqrt{a \cdot a}$	Vector magnitude defined using dot product
$k (a \cdot b) = (k a) \cdot b = a \cdot (k b)$	Associative property of scalar multiplication with dot product
$a \cdot (b + c) = a \cdot b + a \cdot c$	Dot product distributes over vector addition and subtraction
$a \times a = 0$	The cross product of any vector with itself is the zero vector. (Because any vector is parallel with itself.)
$a \times b = - (b \times a)$	Cross product is anticommutative.
$a \times b = (- a) \times (- b)$	Negating both operands to the cross product results in the same vector.
$k (a \times b) = (k a) \times b = a \times (k b)$	Associative property of scalar multiplication with cross product.
$a \times (b + c) = a \times b + a \times c$	Cross product distributes over vector addition and subtraction.

Table 2.2Table of vector algebra identities

Exercises

Let
$a = [\begin{matrix} - 3 & 8 \end{matrix}], b = [\begin{matrix} 4 \\ 0 \\ 5 \end{matrix}], c = [\begin{matrix} 16 \\ - 1 \\ 4 \\ 6 \end{matrix}] .$
1. Identify $a$ , $b$ , and $c$ , as row or column vectors, and give the dimension of each vector.
2. Compute $b_{y} + c_{w} + a_{x} + b_{z}$ .
Identify the quantities in each of the following sentences as scalar or vector. For vector quantities, give the magnitude and direction. (Note: some directions may be implicit.)
1. How much do you weigh?
2. Do you have any idea how fast you were going?
3. It's two blocks north of here.
4. We're cruising from Los Angeles to New York at 600 mph, at an altitude of 33,000 ft.
Give the values of the following vectors. The darker grid lines represent one unit.
Identify the following statements as true or false. If the statement is false, explain why.
1. The size of a vector in a diagram doesn't matter; we just need to draw it in the right place.
2. The displacement expressed by a vector can be visualized as a sequence of axially aligned displacements.
3. These axially aligned displacements from the previous question must occur in order.
4. The vector $[x, y]$ gives the displacement from the point $(x, y)$ to the origin.
Evaluate the following vector expressions:
1. $- [\begin{matrix} 3 & 7 \end{matrix}]$
2. $∥ [\begin{matrix} - 12 & 5 \end{matrix}] ∥$
3. $∥ [\begin{matrix} 8 & - 3 & 1 / 2 \end{matrix}] ∥$
4. $3 [\begin{matrix} 4 & - 7 & 0 \end{matrix}]$
5. $[\begin{matrix} 4 & 5 \end{matrix}] / 2$
Normalize the following vectors:
1. $[\begin{matrix} 12 & 5 \end{matrix}]$
2. $[\begin{matrix} 0 & 743.632 \end{matrix}]$
3. $[\begin{matrix} 8 & - 3 & 1 / 2 \end{matrix}]$
4. $[\begin{matrix} - 12 & 3 & - 4 \end{matrix}]$
5. $[\begin{matrix} 1 & 1 & 1 & 1 \end{matrix}]$
Evaluate the following vector expressions:
1. $[\begin{matrix} 7 & - 2 & - 3 \end{matrix}] + [\begin{matrix} 6 & 6 & - 4 \end{matrix}]$
2. $[\begin{matrix} 2 & 9 & - 1 \end{matrix}] + [\begin{matrix} - 2 & - 9 & 1 \end{matrix}]$
3. $[\begin{matrix} 3 \\ 10 \\ 7 \end{matrix}] - [\begin{matrix} 8 \\ - 7 \\ 4 \end{matrix}]$
4. $[\begin{matrix} 4 \\ 5 \\ - 11 \end{matrix}] - [\begin{matrix} - 4 \\ - 5 \\ 11 \end{matrix}]$
5. $3 [\begin{matrix} a \\ b \\ c \end{matrix}] - 4 [\begin{matrix} 2 \\ 10 \\ - 6 \end{matrix}]$
Compute the distance between the following pairs of points:
1. $[\begin{matrix} 10 \\ 6 \end{matrix}], [\begin{matrix} - 14 \\ 30 \end{matrix}]$
2. $[\begin{matrix} 0 \\ 0 \end{matrix}], [\begin{matrix} - 12 \\ 5 \end{matrix}]$
3. $[\begin{matrix} 3 \\ 10 \\ 7 \end{matrix}], [\begin{matrix} 8 \\ - 7 \\ 4 \end{matrix}]$
4. $[\begin{matrix} - 2 \\ - 4 \\ 9 \end{matrix}], [\begin{matrix} 6 \\ - 7 \\ 9.5 \end{matrix}]$
5. $[\begin{matrix} 4 \\ - 4 \\ - 4 \\ 4 \end{matrix}], [\begin{matrix} - 6 \\ 6 \\ 6 \\ - 6 \end{matrix}]$
Evaluate the following vector expressions:
1. $[\begin{matrix} 2 \\ 6 \end{matrix}] \cdot [\begin{matrix} - 3 \\ 8 \end{matrix}]$
2. $- 7 [\begin{matrix} 1 & 2 \end{matrix}] \cdot [\begin{matrix} 11 & - 4 \end{matrix}]$
3. $10 + [\begin{matrix} - 5 \\ 1 \\ 3 \end{matrix}] \cdot [\begin{matrix} 4 \\ - 13 \\ 9 \end{matrix}]$
4. $3 [\begin{matrix} - 2 \\ 0 \\ 4 \end{matrix}] \cdot ([\begin{matrix} 8 \\ - 2 \\ 3 / 2 \end{matrix}] + [\begin{matrix} 0 \\ 9 \\ 7 \end{matrix}])$
Given the two vectors
$v = [\begin{matrix} 4 \\ 3 \\ - 1 \end{matrix}], \hat{n} = [\begin{matrix} \sqrt{2} / 2 \\ \sqrt{2} / 2 \\ 0 \end{matrix}],$
separate $v$ into components that are perpendicular and parallel to $\hat{n}$ . (As the notation implies, $\hat{n}$ is a unit vector.)
Use the geometric definition of the dot product
$\begin{array}{rcl} a \cdot b = ∥ a ∥ ∥ b ∥ \cos θ \end{array}$
to prove the law of cosines.
Use trigonometric identities and the algebraic definition of the dot product in 2D
$\begin{array}{rcl} a \cdot b = a_{x} b_{x} + a_{y} b_{y} \end{array}$
to prove the geometric interpretation of the dot product in 2D. (Hint: draw a diagram of the vectors and all angles involved.)
Calculate $a \times b$ and $b \times a$ for the following vectors:
1. $a = [\begin{matrix} 0 & - 1 & 0 \end{matrix}], b = [\begin{matrix} 0 & 0 & 1 \end{matrix}]$
2. $a = [\begin{matrix} - 2 & 4 & 1 \end{matrix}], b = [\begin{matrix} 1 & - 2 & - 1 \end{matrix}]$
3. $a = [\begin{matrix} 3 & 10 & 7 \end{matrix}], b = [\begin{matrix} 8 & - 7 & 4 \end{matrix}]$
Prove the equation for the magnitude of the cross product
$\begin{array}{rcl} ∥ a \times b ∥ = ∥ a ∥ ∥ b ∥ \sin θ . \end{array}$
(Hint: make use of the geometric interpretation of the dot product and try to show how the left and right sides of the equation are equivalent, rather than trying to derive one side from the other.)
Section 2.8 introduced the norm of a vector, namely, a scalar value associated with a given vector. However, the definition of the norm given in that section is not the only definition of a norm for a vector. In general, the $p$ -norm of an $n$ -dimensional vector is defined as
${∥ x ∥}_{p} \equiv {(\sum_{i = 1}^{n} {| x_{i} |}^{p})}^{1 / p} .$
Some of the more common $p$ -norms include:
- The $L^{1}$ norm, a.k.a. Taxicab norm ( $p = 1$ ):
  ${∥ x ∥}_{1} \equiv \sum_{i = 1}^{n} | x_{i} | .$
- The $L^{2}$ norm, a.k.a. Euclidean norm ( $p = 2$ ). This is the most common and familiar norm, since it measures geometric length:
  ${∥ x ∥}_{2} \equiv \sqrt{\sum_{i = 1}^{n} x_{i}^{2}} .$
- The infinity norm, a.k.a. Chebyshev norm ( $p = \infty$ ):
  ${∥ x ∥}_{\infty} \equiv max (| x_{1} |, \dots, | x_{n} |) .$
Each of these norms can be thought of as a way to assigning a length or size to a vector. The Euclidean norm was discussed in Section 2.8. The Taxicab norm gets its name from how a taxicab would measure distance driving the streets of a city laid out in a grid (e.g., Cartesia from Section 1.2.1). For example, a taxicab that drives 1 block east and 1 block north drives a total distance of 2 blocks, whereas a bird flying “as the crow flies” can fly in a straight line from start to finish and travel only $\sqrt{2}$ blocks (Euclidean norm). The Chebyshev norm is simply the absolute value of the vector component with the largest absolute value. An example of how this norm can be used is to consider the number of moves required to move a king in a game of chess from one square to another. The immediately surrounding squares require 1 move, the squares surrounding those require 2 moves, and so on.
1. (a)For each of the following find ${∥ x ∥}_{1}$ , ${∥ x ∥}_{2}$ , ${∥ x ∥}_{3}$ , and ${∥ x ∥}_{\infty}$ :
  1. (1) $[\begin{matrix} 3 & 4 \end{matrix}]$
  2. (2) $[\begin{matrix} 5 & - 12 \end{matrix}]$
  3. (3) $[\begin{matrix} - 2 & 10 & - 7 \end{matrix}]$
  4. (4) $[\begin{matrix} 6 & 1 & - 9 \end{matrix}]$
  5. (5) $[\begin{matrix} - 2 & - 2 & - 2 & - 2 \end{matrix}]$
2. *(b)Draw the unit circle (i.e., the set of all vectors with ${∥ x ∥}_{p} = 1$ ) centered at the origin for the $L^{1}$ norm, $L^{2}$ norm, and infinity norm.
A man is boarding a plane. The airline has a rule that no carry-on item may be more than two feet long, two feet wide, or two feet tall. He has a very valuable sword that is three feet long, yet he is able to carry the sword on board with him.9 How is he able to do this? What is the longest possible item that he could carry on?
Verify Figure 2.11 numerically.
Is the coordinate system used in Figure 2.27 a left-handed or right-handed coordinate system?
One common way of defining a bounding box for a 2D object is to specify a center point $c$ and a radius vector $r$ , where each component of $r$ is half the length of the side of the bounding box along the corresponding axis.
1. Describe the four corners $p_{UpperLeft}$ , $p_{UpperRight}$ , $p_{LowerLeft}$ , and $p_{LowerRight}$ .
2. Describe the eight corners of a bounding cube, extending this idea into 3D.
A nonplayer character (NPC) is standing at location $p$ with a forward direction of $v$ .
1. How can the dot product be used to determine whether the point $x$ is in front of or behind the NPC?
2. Let $p = [\begin{matrix} - 3 & 4 \end{matrix}]$ and $v = [\begin{matrix} 5 & - 2 \end{matrix}]$ . For each of the following points $x$ determine whether $x$ is in front of or behind the NPC:
  1. (1) $x = [\begin{matrix} 0 & 0 \end{matrix}]$
  2. (2) $x = [\begin{matrix} 1 & 6 \end{matrix}]$
  3. (3) $x = [\begin{matrix} - 6 & 0 \end{matrix}]$
  4. (4) $x = [\begin{matrix} - 4 & 7 \end{matrix}]$
  5. (5) $x = [\begin{matrix} 5 & 5 \end{matrix}]$
  6. (6) $x = [\begin{matrix} - 3 & 0 \end{matrix}]$
  7. (7) $x = [\begin{matrix} - 6 & - 3.5 \end{matrix}]$
Extending the concept from Exercise 20, consider the case where the NPC has a limited field of view (FOV). If the total FOV angle is $ϕ$ , then the NPC can see to the left or right of its forward direction by a maximum angle of $ϕ / 2$ .
1. How can the dot product be used to determine whether the point $x$ is visible to the NPC?
2. For each of the points $x$ in Exercise 20 determine whether $x$ is visible to the NPC if its FOV is $90^{\circ}$ .
3. Suppose that the NPC's viewing distance is also limited to a maximum distance of 7 units. Which points are visible to the NPC then?
Consider three points labeled $a$ , $b$ , and $c$ in the $x z$ plane of our left-handed coordinate system, which represent waypoints on an NPC's path.
1. How can the cross product be used to determine whether, when moving from $a$ to $b$ to $c$ , the NPC makes a clockwise or counterclockwise turn at $b$ , when viewing the path from above?
2. For each of the following sets of three points, determine whether the NPC is turning clockwise or counterclockwise when moving from $a$ to $b$ to $c$ :
  1. (1) $a = [\begin{matrix} 2 & 0 & 3 \end{matrix}]$ , $b = [\begin{matrix} - 1 & 0 & 5 \end{matrix}]$ , $c = [\begin{matrix} - 4 & 0 & 1 \end{matrix}]$
  2. (2) $a = [\begin{matrix} - 3 & 0 & - 5 \end{matrix}]$ , $b = [\begin{matrix} 4 & 0 & 0 \end{matrix}]$ , $c = [\begin{matrix} 3 & 0 & 3 \end{matrix}]$
  3. (3) $a = [\begin{matrix} 1 & 0 & 4 \end{matrix}]$ , $b = [\begin{matrix} 7 & 0 & - 1 \end{matrix}]$ , $c = [\begin{matrix} - 5 & 0 & - 6 \end{matrix}]$
  4. (4) $a = [\begin{matrix} - 2 & 0 & 1 \end{matrix}]$ , $b = [\begin{matrix} 1 & 0 & 2 \end{matrix}]$ , $c = [\begin{matrix} 4 & 0 & 4 \end{matrix}]$
In the derivation of a matrix to scale along an arbitrary axis, we reach a step where we have the vector expression
$p^{'} = p + (k - 1) (p \cdot n) n,$
where $n$ is an arbitrary vector $[n_{x}, n_{y}, n_{z}]$ and $k$ is an arbitrary scalar, but $p$ is one of the cardinal axes. Plug in the value $p = [1, 0, 0]$ and simplify the resulting expression for $p^{'}$ . The answer is not a vector expression, but a single vector, where the scalar expressions for each coordinate have been simplified.
A similar problem arises with the derivation of a matrix to rotate about an arbitrary axis. Given an arbitrary scalar $θ$ and a vector $n$ , substitute $p = [1, 0, 0]$ and simplify the value of $p^{'}$ in the expression
$p^{'} = \cos θ (p - (p \cdot n) n) + \sin θ (n \times p) + (p \cdot n) n .$

What's our vector, Victor?

— Captain Oveur in Airplane! (1980)

References

[1] Tevian Dray and Corinne A. Manogue. “The Geometry of the Dot and Cross Products.” Journal of Online Mathematics and Its Applications 6. http://mathdl.maa.org/images/upload_library/4/vol6/Dray2/Dray.pdf.

[2] Fletcher Dunn and Ian Parberry. 3D Math Primer for Graphics and Game Development, First edition. Plano, TX: Wordware Publishing, 2002.

But the perspective taken by physics textbooks is probably the one that's more appropriate for video game programming, at least in the beginning.
The typeface used here is not intended to limit the discussion to the set of scalars. We are talking about elements in any set. Also, we request leniency from the abstract algebra sticklers for our use of the word “set,” when we should use “group.” But the latter term is not as widely understood, and we could only afford this footnote to dwell on the distinction.
But we do know our position relative to the nearest Taco Bell.
If you are one of those people, then this is a warning of a slightly different sort!
Indeed, sometimes slower, depending on your compiler.
About 20 micropascals. However, this number varies with frequency. It also increases with age. One author remembers that when he was young, his father would never turn the radio in the car completely off, but rather would turn the volume down below the (father's) threshold of hearing. The son's threshold of hearing was just low enough for this to be irritating. Today the son owns his own car and car radio, and has realized, with some degree of embarrassment, that he also often turns the radio volume down without turning it off. However, he offers in his defense that he turns it all the way down, below everyone's threshold of hearing. (The other author wishes to suggest that clearly even the term “normal human” is relative.)
One notation you will probably bump up against is treating the dot product as an ordinary matrix multiplication, denoted by $a^{T} b$ if $a$ and $b$ are interpreted as column vectors, or $a b^{T}$ for row vectors. If none of this makes sense, don't worry, we will repeat it after we learn about matrix multiplication and row and column vectors in Chapter 4.
Thus shirking our traditional duties as mathematics authors to make intuitive concepts sound much more complicated than they are.
Please ignore the fact that nowadays this could never happen for security reasons. You can think of this exercise as taking place in a Quentin Tarantino movie.

2.1Mathematical Definition of Vector,and Other Boring Stuff