linindep.xml

<?xml version="1.0" encoding="UTF-8"?>

<!--********************************************************************
Copyright 2017 Georgia Institute of Technology

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation.  A copy of
the license is included in gfdl.xml.
*********************************************************************-->

<section xml:id="linear-independence">
  <title>Linear Independence</title>

  <objectives>
    <ol>
      <li>Understand the concept of linear independence.</li>
      <li>Learn two criteria for linear independence.</li>
      <li>Understand the relationship between linear independence and pivot columns / free variables.</li>
      <li><em>Recipe:</em> test if a set of vectors is linearly independent / find an equation of linear dependence.</li>
      <li><em>Picture:</em> whether a set of vectors in <m>\R^2</m> or <m>\R^3</m> is linearly independent or not.</li>
      <li><em>Vocabulary words:</em> <term>linear dependence relation</term> / <term>equation of linear dependence</term>.</li>
      <li><em>Essential vocabulary words:</em> <term>linearly independent</term>, <term>linearly dependent</term>.</li>
    </ol>
  </objectives>

  <introduction>
    <p>
      Sometimes the span of a set of vectors is <q>smaller</q> than you expect from the
number of vectors, as in the picture below.  This means that (at least) one of the vectors is redundant: it can be removed without affecting the span.  In the present section, we formalize this idea in the notion of <em>linear independence</em>.
    </p>
    <figure xml:id="linindep-intro-pic">
      <caption>Pictures of sets of vectors that are linearly dependent.  Note that in each case, one vector is in the span of the others<mdash/>so it doesn<rsq/>t make the span bigger.</caption>
      <image>
        <latex-image-code>
          <![CDATA[
\begin{tikzpicture}[thin border nodes, scale=.7, baseline=.5cm]
  \draw[grid lines, light gray] (-3,-3) grid (4, 4);
  \path[clip] (-3,-3) rectangle (4, 4);
  \node[seq4] at (.5,2.5) {$\Span\{v,w\}$};
  \draw[thin, seq4] ($-2*(2,2)$) -- ($2*(2,2)$);
  \draw[vector, seq1] (0,0)
    -- node[midway,above left,text=seq1] {$v$} (2,2);
  \draw[vector, seq2] (0,0)
    -- node[pos=.4,above left,text=seq2] {$w$} (-1,-1);
  \point at (0,0);
\end{tikzpicture}
\qquad
\begin{tikzpicture}[myxyz, thin border nodes, baseline=0cm]
  \path[clip, resetxy] (-4,-3) rectangle (4,3);

  \def\v{(-1,2,1)}
  \def\w{(1,2,.3)}
  \def\u{(0,2,.65)}

  \node[coordinate] (X) at \v {};
  \node[coordinate] (Y) at \w {};

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-1.5, 5) -- (1.5, -5) -- (1.5, -7) -- (-1.5, -7) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4, very thin, opacity=.3] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \begin{scope}[transformxy]
    \fill[white, nearly opaque] (-2, -2) rectangle (3, 3);
    \draw[grid lines] (-2, -2) grid (3, 3);
  \end{scope}

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-1.5, 5) -- (1.5, -5) -- (1.5, 7) -- (-1.5, 7) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4, very thin, opacity=.3] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \node[seq4] at (-1cm, 2cm) {$\Span\{u,v,w\}$};

  \draw[vector, seq1] (0,0,0) --
    node [midway, above] {$v$} \v;
  %\draw[thin, densely dotted] \v -- \projxy\v;
  \draw[vector, seq2] (0,0,0) --
    node [midway, below] {\strut$w$} \w;
  %\draw[thin, densely dotted] \w -- \projxy\w;
  \draw[vector, seq3] (0,0,0) --
    \u node [right] {$u$};
  %\draw[thin, densely dotted] \u -- \projxy\u;

  \draw[thin, densely dotted] \v -- \w;

  \point at (0,0,0);

\end{tikzpicture}
          ]]>
        </latex-image-code>
      </image>
    </figure>
  </introduction>

  <subsection>
    <title>The Definition of Linear Independence</title>

    <essential xml:id="linindep-defn-linindep">
      <idx><h>Linear independence</h><h>definition of</h></idx>
      <idx><h>Linear dependence</h><see>Linear independence</see></idx>
      <statement>
        <p>
          A set of vectors <m>\{v_1,v_2,\ldots,v_k\}</m> is <term>linearly independent</term> if the vector equation
          <me>x_1v_1 + x_2v_2 + \cdots + x_kv_k = 0</me>
          has only the trivial solution <m>x_1=x_2=\cdots=x_k=0</m>.  The set <m>\{v_1,v_2,\ldots,v_k\}</m> is <term>linearly dependent</term> otherwise.
        </p>
      </statement>
    </essential>

    <p>
      <idx><h>Linear Independence</h><h>linear dependence relation</h></idx>
      <idx><h>Linear Independence</h><h>equation of linear dependence</h></idx>
      <idx><h>Equation of linear dependence</h><see>Linear independence</see></idx>
      <idx><h>Linear dependence relation</h><see>Linear independence</see></idx>
      In other words, <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly dependent if there exist numbers <m>x_1,x_2,\ldots,x_k</m>, not all equal to zero, such that
      <me>x_1v_1 + x_2v_2 + \cdots + x_kv_k = 0.</me>
      This is called a <term>linear dependence relation</term> or <term>equation of linear dependence</term>.
    </p>

    <bluebox>
      <p>
        Note that linear dependence and linear independence are notions that apply to a <em>collection of vectors</em>.  It does not make sense to say things like <q>this vector is linearly dependent on these other vectors,</q> or <q>this matrix is linearly independent.</q>
      </p>
    </bluebox>

    <example>
      <title>Checking linear dependence</title>
      <statement>
        <p>
          Is the set
          <me>\left\{ \vec{1 1 1},\, \vec{1 -1 2},\, \vec{3 1 4} \right\}</me>
          linearly independent?
        </p>
      </statement>
      <solution>
        <p>
          Equivalently, we are asking if the homogeneous vector equation
          <me>x\vec{1 1 1} + y\vec{1 -1 2} + z\vec{3 1 4} = \vec{0 0 0}</me>
          has a nontrivial solution.  We solve this by forming a matrix and row reducing (we do not augment because of this <xref ref="solnsets-no-augment"/>):
          <me>\mat{1 1 3; 1 -1 1; 1 2 4}
          \quad\xrightarrow{\text{row reduce}}\quad
          \mat{1 0 2; 0 1 1; 0 0 0}</me>
          This says <m>x = -2z</m> and <m>y = -z</m>.  So there exist nontrivial solutions: for instance, taking <m>z=1</m> gives this equation of linear dependence:
          <me>-2\vec{1 1 1} - \vec{1 -1 2} + \vec{3 1 4} = \vec{0 0 0}.</me>
        </p>
        <figure>
          <caption>Move the sliders to solve the homogeneous vector equation in this example.  Do you see why the vectors need to be coplanar in order for there to exist a nontrivial solution?</caption>
          <mathbox source="demos/spans.html?v1=1,1,1&amp;v2=1,-1,2&amp;v3=3,1,4&amp;target=0,0,0" height="500px"/>
        </figure>
      </solution>
    </example>

    <example>
      <title>Checking linear independence</title>
      <statement>
        <p>
          Is the set
          <me>\left\{ \vec{1 1 -2},\, \vec{1 -1 2},\, \vec{3 1 4} \right\}</me>
          linearly independent?
        </p>
      </statement>
      <solution>
        <p>
          Equivalently, we are asking if the homogeneous vector equation
          <me>x\vec{1 1 -2} + y\vec{1 -1 2} + z\vec{3 1 4} = \vec{0 0 0}</me>
          has a nontrivial solution.  We solve this by forming a matrix and row reducing (we do not augment because of this <xref ref="solnsets-no-augment"/>):
          <me>\mat{1 1 3; 1 -1 1; -2 2 4}
          \quad\xrightarrow{\text{row reduce}}\quad
          \mat{1 0 0; 0 1 0; 0 0 1}</me>
          This says <m>x = y = z = 0</m>, i.e., the only solution is the trivial solution.  We conclude that the set is linearly independent.
        </p>
        <figure>
          <caption>Move the sliders to solve the homogeneous vector equation in this example.  Do you see why the vectors would need to be coplanar in order for there to exist a nontrivial solution?</caption>
          <mathbox source="demos/spans.html?v1=1,1,-2&amp;v2=1,-1,2&amp;v3=3,1,4&amp;target=0,0,0&amp;range=4" height="500px"/>
        </figure>
      </solution>
    </example>

    <example xml:id="param-vect-form-lin-ind">
      <title>Vector parametric form</title>
      <p>An important observation is that the vectors coming from the parametric vector form of the solution of a matrix equation <m>Ax=0</m> are linearly independent.  In this <xref ref="solnsets-eg-plane"/> we saw that the solution set of <m>Ax=0</m> for 
        <me>A = \mat{1 -1 2; -2 2 -4}?</me>
is
        <me>x = \vec{x_1 x_2 x_3} = x_2\vec{1 1 0} + x_3\vec{-2 0 1}.</me>
Let's explain why the vectors <m>(1,1,0)</m> and <m>(-2,0,1)</m> are linearly independent.  Suppose that
<me>\vec{0 0 0} = x_2\vec{1 1 0} + x_3\vec{-2 0 1} = \vec{x_2-2x_3 x_2 x_3}.</me>
          Comparing the second and third coordinates, we see that <m>x_2=x_3=0</m>.  This reasoning will work in any example, since the entries corresponding to the free variables are all equal to 1 or 0, and are only equal to 1 in one of the vectors.  This observation forms part of this <xref ref="dimension-basis-nulspace"/>.
	</p>
    </example>

    <p>
      The above examples lead to the following recipe.
    </p>

    <bluebox xml:id="linindep-matrix-cols">
      <title>Recipe: Checking linear independence</title>
      <idx><h>Linear Independence</h><h>verifying</h></idx>
    <p>
      A set of vectors <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly independent if and only if the vector equation
      <me>x_1v_1 + x_2v_2 + \cdots + x_kv_k = 0</me>
      has only the trivial solution, if and only if the matrix equation
      <m>Ax=0</m>
      has only the trivial solution, where <m>A</m> is the matrix with columns <m>v_1,v_2,\ldots,v_k</m>:
      <me>A = \mat{| |, , |; v_1 v_2 \cdots, v_k; | |, , |}.</me>
      This is true if and only if <m>A</m> has a <xref ref="defn-pivot-pos" text="title">pivot position</xref> in every column.
    </p>
    <p>
      Solving the matrix equatiion <m>Ax=0</m> will either verify that the columns <m>v_1,v_2,\ldots,v_k</m> are linearly independent, or will produce a linear dependence relation by substituting any nonzero values for the free variables.
    </p>
    </bluebox>

    <p>
      (Recall that <m>Ax=0</m> has a nontrivial solution if and only if <m>A</m> has a column without a pivot: see this <xref ref="solnsets-nontrivial"/>.)
    </p>

    <p>
      Suppose that <m>A</m> has more columns than rows.  Then <m>A</m> cannot have a pivot in every column (it has at most one pivot per row), so its columns are automatically linearly dependent.
    </p>

    <bluebox xml:id="linindep-wide-matrix">
      <idx><h>Linear Independence</h><h>wide matrices</h></idx>
      <p>A wide matrix (a matrix with more columns than rows) has linearly dependent columns.</p>
    </bluebox>

      <p>For example, four vectors in <m>\R^3</m> are automatically linearly dependent.  Note that a tall matrix may or may not have linearly independent columns.</p>

    <fact hide-type="true" xml:id="linindep-facts">
      <title>Facts about linear independence</title>
      <idx><h>Linear Independence</h><h>basic facts</h></idx>
      <statement>
        <p>
          <ol>
            <li>Two vectors are linearly dependent if and only if they are collinear, i.e., one is a scalar multiple of the other.</li>
            <li>Any set containing the zero vector is linearly dependent.</li>
            <li>If a subset of <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly dependent, then <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly dependent as well.</li>
          </ol>
        </p>
      </statement>
      <proof>
        <p>
          <ol>
            <li>
              If <m>v_1 = cv_2</m> then <m>v_1-cv_2=0</m>, so <m>\{v_1,v_2\}</m> is linearly dependent.  In the other direction, if <m>x_1v_1+x_2v_2=0</m> with <m>x_1\neq0</m> (say), then <m>v_1 = -\frac{x_2}{x_1}v_2</m>.
            </li>
            <li>
              It is easy to produce a linear dependence relation if one vector is the zero vector: for instance, if <m>v_1=0</m> then
              <me>1\cdot v_1 + 0\cdot v_2 + \cdots + 0\cdot v_k = 0.</me>
            </li>
            <li>
              After reordering, we may suppose that <m>\{v_1,v_2,\ldots,v_r\}</m> is linearly dependent, with <m>r &lt; p</m>.  This means that there is an equation of linear dependence
              <me>x_1v_1 + x_2v_2 + \cdots + x_rv_r = 0</me>,
              with at least one of <m>x_1,x_2,\ldots,x_r</m> nonzero.  This is also an equation of linear dependence among <m>\{v_1,v_2,\ldots,v_k\}</m>, since we can take the coefficients of <m>v_{r+1},\ldots,v_k</m> to all be zero.
            </li>
          </ol>
        </p>
      </proof>
    </fact>

    <p>
      With regard to the first fact, note that the zero vector is a multiple of any vector, so it is collinear with any other vector.  Hence facts 1 and 2 are consistent with each other.
    </p>

  </subsection>

  <subsection>
    <title>Criteria for Linear Independence</title>

    <p>
      In this subsection we give two criteria for a set of vectors to be linearly independent.  Keep in mind, however, that the actual <xref ref="linindep-defn-linindep"/> is above.
    </p>

    <theorem xml:id="linindep-criterion-inspan">
      <statement>
        <p>A set of vectors <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly dependent if and only if one of the vectors is in the span of the other ones.</p>
        <p>Any such vector may be removed without affecting the span.</p>
      </statement>
      <proof>
        <p>
          Suppose, for instance, that <m>v_3</m> is in <m>\Span\{v_1,v_2,v_4\}</m>, so we have an equation like
          <me>v_3 = 2v_1 - \frac 12v_2 + 6v_4.</me>
          We can subract <m>v_3</m> from both sides of the equation to get
          <me>0 = 2v_1 - \frac 12v_2 - v_3 + 6v_4.</me>
          This is a linear dependence relation.
        </p>
        <p>
          In this case, any linear combination of <m>v_1,v_2,v_3,v_4</m> is already a linear combination of <m>v_1,v_2,v_4</m>:
          <md>
            <mrow>
              x_1v_1 + x_2v_2 + x_3v_3 + x_4v_4 &amp;=
              x_1v_1 + x_2v_2 + x_3\left(2v_1-\frac 12v_2 + 6v_4\right) + x_4v_4
            </mrow>
            <mrow>
              &amp;= (x_1+2x_3)v_1 + \left(x_2-\frac 12x_3\right)v_2 + (x_4+6)v_4.
            </mrow>
          </md>
          Therefore, <m>\Span\{v_1,v_2,v_3,v_4\}</m> is contained in <m>\Span\{v_1,v_2,v_4\}</m>.  Any linear combination of <m>v_1,v_2,v_4</m> is also a linear combination of <m>v_1,v_2,v_3,v_4</m> (with the <m>v_3</m>-coefficient equal to zero), so <m>\Span\{v_1,v_2,v_4\}</m> is also contained in <m>\Span\{v_1,v_2,v_3,v_4\}</m>, and thus they are equal.
        </p>
        <p>
          In the other direction, if we have a linear dependence relation like
          <me>0 = 2v_1 - \frac 12v_2 + v_3 - 6v_4,</me>
          then we can move any nonzero term to the left side of the equation and divide by its coefficient:
          <me>v_1 = \frac 12\left(\frac 12v_2 - v_3 + 6v_4\right).</me>
          This shows that <m>v_1</m> is in <m>\Span\{v_2,v_3,v_4\}</m>.
        </p>
        <p>
          We leave it to the reader to generalize this proof for any set of vectors.
        </p>
      </proof>
    </theorem>

    <bluebox type-name="Warning" hide-type="false" xml:id="linindep-not-any"><p>
      In a linearly dependent set <m>\{v_1,v_2,\ldots,v_k\}</m>, it is not generally true that <em>any</em> vector <m>v_j</m> is in the span of the others, only that <em>at least one</em> of them is.
    </p></bluebox>

    <p>
      For example, the set <m>\bigl\{{1\choose 0},\,{2\choose 0},\,{0\choose 1}\bigr\}</m> is linearly dependent, but <m>{0\choose 1}</m> is not in the span of the other two vectors.  Also see this <xref ref="linindep-3d-4vecs"/> below.
    </p>

    <p>
      The previous <xref ref="linindep-criterion-inspan"/> makes precise in what sense a set of linearly dependent vectors is redundant.
    </p>

    <theorem xml:id="linindep-increasing-span">
      <title>Increasing Span Criterion</title>
      <idx><h>Linear Independence</h><h>increasing span criterion</h></idx>
      <idx><h>Increasing span criterion</h><see>Linear independence</see></idx>
      <statement>
        <p>A set of vectors <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly independent if and only if, for every <m>j</m>, the vector <m>v_j</m> is not in <m>\Span\{v_1,v_2,\ldots,v_{j-1}\}</m>.</p>
      </statement>
      <proof>
        <p>
          It is equivalent to show that <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly dependent if and only if <m>v_j</m> is in <m>\Span\{v_1,v_2,\ldots,v_{j-1}\}</m> for some <m>j</m>.  The <q>if</q> implication is an immediate consequence of the previous <xref ref="linindep-criterion-inspan"/>.  Suppose then that <m>\{v_1,v_2,\ldots,v_k\}</m> is linearly dependent.  This means that some <m>v_j</m> is in the span of the others.  Choose the largest such <m>j</m>.  We claim that this <m>v_j</m> is in <m>\Span\{v_1,v_2,\ldots,v_{j-1}\}</m>.  If not, then
          <me>v_j = x_1v_1 + x_2v_2 + \cdots + x_{j-1}v_{j-1} + x_{j+1}v_{j+1} + \cdots + x_kv_k</me>
          with not all of <m>x_{j+1},\ldots,x_k</m> equal to zero.  Suppose for simplicity that <m>x_k\neq 0</m>.  Then we can rearrange:
          <me>
            v_k = -\frac 1{x_k}\bigl( x_1v_1 + x_2v_2 + \cdots + x_{j-1}v_{j-1}
            - v_j + x_{j+1}v_{j+1} + \cdots + x_{p-1}v_{p-1}
            \bigr).
          </me>
          This says that <m>v_k</m> is in the span of <m>\{v_1,v_2,\ldots,v_{p-1}\}</m>, which contradicts our assumption that <m>v_j</m> is the last vector in the span of the others.
        </p>
      </proof>
    </theorem>

    <p>
      We can rephrase this as follows:
    </p>

    <bluebox>
      <p>If you make a set of vectors by adding one vector at a time, and if the span got bigger every time you added a vector, then your set is linearly independent.</p>
    </bluebox>

  </subsection>


  <subsection>
    <title>Pictures of Linear Independence</title>
      <idx><h>Linear Independence</h><h>pictures of</h></idx>

    <p>A set containg one vector <m>\{v\}</m> is linearly independent when <m>v\neq 0</m>, since <m>xv = 0</m> implies <m>x=0</m>.
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[thin border nodes, scale=1.5]
  \path[use as bounding box] (-3,-3) rectangle (3,3);

  \path[clip] (-3,-3) rectangle (3,3);
  \draw[seq4!30] (-4,-2) -- (4,2);
  \path (-2,-1) node[below right, seq4] {$\Span\{v\}$};
  \draw[vector, seq1] (0,0) --
    node[midway,above left]{$v$} (2,1);
  \point at (0,0);
  \draw[thin] (-3,-3) rectangle (3,3);
\end{tikzpicture}
          ]]>
    </latex-code>
    A set of two noncollinear vectors <m>\{v,w\}</m> is linearly independent:
    <ul>
      <li>Neither is in the span of the other, so we can apply the first <xref ref="linindep-criterion-inspan">criterion</xref>.</li>
      <li>The span got bigger when we added <m>w</m>, so we can apply the <xref ref="linindep-increasing-span">increasing span criterion</xref>.</li>
    </ul>
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[thin border nodes, scale=1.5]
  \path[use as bounding box] (-3,-3) rectangle (3,3);

  \path[clip] (-3,-3) rectangle (3,3);
  \draw[seq4!30] (-4,-2) -- (4,2);
  \draw[seq4!30] (-3,3) -- (3,-3);
  \path (-2,-1) node[below right, seq4] {$\Span\{v\}$};
  \path (-2,2) node[above right, seq4] {$\Span\{w\}$};
  \draw[vector, seq1] (0,0) --
    node[midway,above left]{$v$} (2,1);
  \draw[vector, seq2] (0,0) --
    node[midway,below left]{$w$} (-1,1);
  \point at (0,0);
  \draw[thin] (-3,-3) rectangle (3,3);
\end{tikzpicture}
          ]]>
        </latex-code>
    The set of three vectors <m>\{v,w,u\}</m> below is linearly dependent:
    <ul>
      <li><m>u</m> is in <m>\Span\{v,w\}</m>, so we can apply the first <xref ref="linindep-criterion-inspan">criterion</xref>.</li>
      <li>The span did not increase when we added <m>u</m>, so we can apply the <xref ref="linindep-increasing-span">increasing span criterion</xref>.</li>
    </ul>
    In the picture below, note that <m>v</m> is in <m>\Span\{u,w\}</m>, and <m>w</m> is in <m>\Span\{u,v\}</m>, so we can remove any of the three vectors without shrinking the span.
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[scale=1.5, thin border nodes]
  \path[use as bounding box] (-3,-3) rectangle (3,3);

  \path[clip] (-3,-3) rectangle (3,3);
  \fill[seq4!10] (-3,-3) rectangle (3,3);
  \draw[seq4!30] (-4,-2) -- (4,2);
  \draw[seq4!30] (-3,3) -- (3,-3);
  \path (-2,-1) node[below right, seq4] {$\Span\{v\}$};
  \path (-2,2) node[above right, seq4] {$\Span\{w\}$};
  \node[seq4] at (1.2,1.8) {$\Span\{v,w\}$};
  \draw[vector, seq1] (0,0) --
    node[midway,above left]{$v$} (2,1);
  \draw[vector, seq2] (0,0) --
    node[midway,below left]{$w$} (-1,1);
  \draw[vector, seq3] (0,0) --
    node[midway,right]{$u$} (0,-1.5);
  \point at (0,0);
  \draw[thin] (-3,-3) rectangle (3,3);
\end{tikzpicture}
          ]]>
    </latex-code>
    Two collinear vectors are always linearly dependent:
      <ul>
        <li><m>w</m> is in <m>\Span\{v\}</m>, so we can apply the first <xref ref="linindep-criterion-inspan">criterion</xref>.</li>
        <li>The span did not increase when we added <m>w</m>, so we can apply the <xref ref="linindep-increasing-span">increasing span criterion</xref>.</li>
      </ul>
      <latex-code>
          <![CDATA[
\begin{tikzpicture}[scale=1.5, thin border nodes]
  \path[use as bounding box] (-3,-3) rectangle (3,3);

  \path[clip] (-3,-3) rectangle (3,3);
  \draw[seq4!30] (-4,-2) -- (4,2);
  \path (-2,-1) node[below right, seq4] {$\Span\{v\}$};
  \draw[vector, seq1] (0,0) --
    node[midway,above left]{$v$} (2,1);
  \draw[vector, seq2] (0,0) --
    node[pos=.4,above left]{$w$} (-1,-.5);
  \point at (0,0);
  \draw[thin] (-3,-3) rectangle (3,3);
\end{tikzpicture}
          ]]>
        </latex-code>
    These three vectors <m>\{v,w,u\}</m> are linearly dependent: indeed, <m>\{v,w\}</m> is already linearly dependent, so we can use the third <xref ref="linindep-facts"/>.
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[scale=1.5, thin border nodes]
  \path[use as bounding box] (-3,-3) rectangle (3,3);

  \path[clip] (-3,-3) rectangle (3,3);
  \draw[seq4!30] (-4,-2) -- (4,2);
  \path (-2,-1) node[below right, seq4] {$\Span\{v\}$};
  \draw[vector, seq1] (0,0) --
    node[midway,above left]{$v$} (2,1);
  \draw[vector, seq2] (0,0) --
    node[pos=.4,above left]{$w$} (-1,-.5);
  \draw[vector, seq3] (0,0) --
    node[midway,right]{$u$} (0,-1.5);
  \point at (0,0);
  \draw[thin] (-3,-3) rectangle (3,3);
\end{tikzpicture}
          ]]>
    </latex-code>
  </p>

    <example hide-type="true">
      <title>Interactive: Linear independence of two vectors in <m>\R^2</m></title>
      <figure>
        <caption>Move the vector heads and the demo will tell you if they are linearly independent and show you their span.</caption>
        <mathbox source="demos/spans.html?captions=indep&amp;v1=2,1&amp;v2=-1,-.5&amp;labels=v,w&amp;range=5&amp;showPlane=true&amp;closed" height="500px"/>
      </figure>
    </example>

    <example hide-type="true">
      <title>Interactive: Linear dependence of three vectors in <m>\R^2</m></title>
      <figure>
        <caption>Move the vector heads and the demo will tell you that they are linearly dependent and show you their span.</caption>
        <mathbox source="demos/spans.html?captions=indep&amp;v1=2,1&amp;v2=-1,-.5&amp;v3=0,-1.5&amp;labels=v,w,u&amp;range=5&amp;showPlane=true&amp;closed" height="500px"/>
      </figure>
    </example>

    <p>The two vectors <m>\{v,w\}</m> below are linearly independent because they are not collinear.
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[myxyz, thin border nodes, scale=1.25]
  \path[clip, resetxy] (-4,-4) rectangle (4,4);

  \def\v{(-1,2,1)}
  \def\w{(0,1,-1)}
  \def\u{(-.5,.5,1)}
  \def\uu{(2,0,2)}

  \node[coordinate] (X) at \v {};
  \node[coordinate] (Y) at \w {};

  \draw[seq4!30] ($-3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[transformxy]
    \fill[white, semitransparent] (-2, -2) rectangle (3, 3);
    \draw[help lines, light gray] (-2, -2) grid (3, 3);
  \end{scope}

  \draw[seq4!30] ($3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($-5*(0,1,-1)$) -- (0,0,0);

  \draw[vector, seq1] (0,0,0) --
    node [midway, below right] {$v$} \v;
  \draw[thin, densely dotted] \v -- \projxy\v;

  \draw[vector, seq2!50!white] (0,0,0) --
    node [midway, below left] {$w$} \w;
  \draw[thin, densely dotted, black!50!white] \w -- \projxy\w;

  \path[seq4] ($.9*(1.5,-3,-1.5)$) node[below right] {$\Span\{v\}$};
  \path[seq4] (0,-2,2) node[below left] {$\Span\{w\}$};

  \point at (0,0);
  \draw[thin,resetxy] (-4,-4) rectangle (4,4);

\end{tikzpicture}
          ]]>
        </latex-code>
    The three vectors <m>\{v,w,u\}</m> below are linearly independent: the span got bigger when we added <m>w</m>, then again when we added <m>u</m>, so we can apply the <xref ref="linindep-increasing-span">increasing span criterion</xref>.
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[myxyz, scale=1.25, thin border nodes]
  \path[clip, resetxy] (-4,-4) rectangle (4,4);

  \def\v{(-1,2,1)}
  \def\w{(0,1,-1)}
  \def\u{(-.5,.5,1)}
  \def\uu{(2,0,2)}

  \node[coordinate] (X) at \v {};
  \node[coordinate] (Y) at \w {};

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, -5) -- (5, 5) -- (5, 10) -- (-5, 10) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4!30, very thin] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \draw[seq4!30] ($-3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[transformxy]
    \fill[white, semitransparent] (-2, -2) rectangle (3, 3);
    \draw[help lines, light gray] (-2, -2) grid (3, 3);
  \end{scope}

  \draw[seq4!30] ($3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($-5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, -5) -- (5, 5) -- (5, -10) -- (-5, -10) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4!30, very thin] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \draw[vector, seq1] (0,0,0) --
    node [midway, below right] {$v$} \v;
  \draw[thin, densely dotted] \v -- \projxy\v;

  \draw[vector, seq2!50!white] (0,0,0) --
    node [midway, below left] {$w$} \w;
  \draw[thin, densely dotted, black!50!white] \w -- \projxy\w;

  \draw[vector, seq3] (0,0,0) --
    node [midway, below left] {$u$} \uu;
  \draw[thin, densely dotted] \uu -- \projxy\uu;

  \path[seq4] ($.9*(1.5,-3,-1.5)$) node[below right] {$\Span\{v\}$};
  \path[seq4] (0,-2,2) node[below left] {$\Span\{w\}$};
  \node[seq4] at (.8cm, 2.8cm) {$\Span\{v,w\}$};

  \point at (0,0);
  \draw[thin,resetxy] (-4,-4) rectangle (4,4);

\end{tikzpicture}
          ]]>
    </latex-code>
    The three coplanar vectors <m>\{v,w,u\}</m> below are linearly dependent:
    <ul>
      <li><m>u</m> is in <m>\Span\{v,w\}</m>, so we can apply the first <xref ref="linindep-criterion-inspan">criterion</xref>.</li>
      <li>The span did not increase when we added <m>u</m>, so we can apply the <xref ref="linindep-increasing-span">increasing span criterion</xref>.</li>
    </ul>
    <latex-code>
          <![CDATA[
\begin{tikzpicture}[myxyz, scale=1.25, thin border nodes]
  \path[clip, resetxy] (-4,-4) rectangle (4,4);

  \def\v{(-1,2,1)}
  \def\w{(0,1,-1)}
  \def\u{(-.5,.5,1)}
  \def\uu{(2,0,2)}

  \node[coordinate] (X) at \v {};
  \node[coordinate] (Y) at \w {};

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, -5) -- (5, 5) -- (5, 10) -- (-5, 10) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4!30, very thin] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \draw[seq4!30] ($-3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[transformxy]
    \fill[white, semitransparent] (-2, -2) rectangle (3, 3);
    \draw[help lines, light gray] (-2, -2) grid (3, 3);
  \end{scope}

  \draw[seq4!30] ($3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($-5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, -5) -- (5, 5) -- (5, -10) -- (-5, -10) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4!30, very thin] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \draw[vector, seq1] (0,0,0) --
    node [midway, below right] {$v$} \v;
  \draw[thin, densely dotted] \v -- \projxy\v;

  \draw[vector, seq2!50!white] (0,0,0) --
    node [midway, below left] {$w$} \w;
  \draw[thin, densely dotted, black!50!white] \w -- \projxy\w;

  \draw[vector, seq3] (0,0,0) --
    node [midway, above left] {$u$} \u;
  \draw[thin, densely dotted] \u -- \projxy\u;

  \path[seq4] ($.9*(1.5,-3,-1.5)$) node[below right] {$\Span\{v\}$};
  \path[seq4] (0,-2,2) node[below left] {$\Span\{w\}$};
  \node[seq4] at (.8cm, 2.8cm) {$\Span\{v,w\}$};

  \point at (0,0);
  \draw[thin,resetxy] (-4,-4) rectangle (4,4);

\end{tikzpicture}
          ]]>
    </latex-code>
    Note that three vectors are linearly dependent if and only if they are <em>coplanar</em>.  Indeed, <m>\{v,w,u\}</m> is linearly dependent if and only if one vector is in the span of the other two, which is a plane (or a line) (or <m>\{0\}</m>).
    </p>

    <p>The four vectors <m>\{v,w,u,x\}</m> below are linearly dependent: they are the columns of a <xref ref="linindep-wide-matrix">wide matrix</xref>.  Note however that <m>u</m> is not contained in <m>\Span\{v,w,x\}</m>.  See this <xref ref="linindep-not-any"/>.</p>

    <figure xml:id="linindep-3d-4vecs">
      <caption>The vectors <m>\{v,w,u,x\}</m> are linearly dependent, but <m>u</m> is not contained in <m>\Span\{v,w,x\}</m>.</caption>
      <image>
        <latex-image-code>
          <![CDATA[
\begin{tikzpicture}[myxyz, scale=1.25, thin border nodes]
  \path[clip, resetxy] (-4,-4) rectangle (4,4);

  \def\v{(-1,2,1)}
  \def\w{(0,1,-1)}
  \def\u{(-.5,.5,1)}
  \def\uu{(2,0,2)}

  \node[coordinate] (X) at \v {};
  \node[coordinate] (Y) at \w {};

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, -5) -- (5, 5) -- (5, 10) -- (-5, 10) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4!30, very thin] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \draw[seq4!30] ($-3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[transformxy]
    \fill[white, semitransparent] (-2, -2) rectangle (3, 3);
    \draw[help lines, light gray] (-2, -2) grid (3, 3);
  \end{scope}

  \draw[seq4!30] ($3*(-1,2,1)$) -- (0,0,0);
  \draw[seq4!30] ($-5*(0,1,-1)$) -- (0,0,0);

  \begin{scope}[x=(X), y=(Y), transformxy]
    \path[clip] (-5, -5) -- (5, 5) -- (5, -10) -- (-5, -10) -- cycle;
    \fill[seq4!10, nearly opaque] (-1.5,-1) rectangle (1.5,2);
    \draw[step=.5cm, seq4!30, very thin] (-1.5,-1) grid (1.5,2);
  \end{scope}

  \draw[vector, seq1] (0,0,0) --
    node [midway, below right] {$v$} \v;
  \draw[thin, densely dotted] \v -- \projxy\v;

  \draw[vector, seq2!50!white] (0,0,0) --
    node [midway, below left] {$w$} \w;
  \draw[thin, densely dotted, black!50!white] \w -- \projxy\w;

  \draw[vector, seq3] (0,0,0) --
    node [midway, below left] {$u$} \uu;
  \draw[thin, densely dotted] \uu -- \projxy\uu;

  \draw[vector, seq5] (0,0,0) --
    node [midway, above left] {$x$} \u;
  \draw[thin, densely dotted] \u -- \projxy\u;

  \path[seq4] ($.9*(1.5,-3,-1.5)$) node[below right] {$\Span\{v\}$};
  \path[seq4] (0,-2,2) node[below left] {$\Span\{w\}$};
  \node[seq4] at (.8cm, 2.8cm) {$\Span\{v,w\}$};

  \point at (0,0);
  \draw[thin,resetxy] (-4,-4) rectangle (4,4);

\end{tikzpicture}
          ]]>
        </latex-image-code>
      </image>
    </figure>

    <example hide-type="true">
      <title>Interactive: Linear independence of two vectors in <m>\R^3</m></title>
      <figure>
        <caption>Move the vector heads and the demo will tell you if they are linearly independent and show you their span.</caption>
        <mathbox source="demos/spans.html?captions=indep&amp;v1=2,-1,1&amp;v2=1,0,-1&amp;labels=v,w&amp;range=5&amp;closed" height="500px"/>
      </figure>
    </example>

    <example hide-type="true">
      <title>Interactive: Linear independence of three vectors in <m>\R^3</m></title>
      <figure>
        <caption>Move the vector heads and the demo will tell you if they are linearly independent and show you their span.</caption>
        <mathbox source="demos/spans.html?captions=indep&amp;v1=2,-1,1&amp;v2=1,0,-1&amp;v3=.5,-.5,1&amp;labels=v,w,u&amp;range=5&amp;closed" height="500px"/>
      </figure>
    </example>

  </subsection>

  <subsection>
    <title>Linear Dependence and Free Variables</title>

    <p>
      In light of this <xref ref="linindep-matrix-cols"/> and this <xref ref="linindep-criterion-inspan">criterion</xref>, it is natural to ask which columns of a matrix are redundant, i.e., which we can remove without affecting the column span.
    </p>

    <theorem xml:id="linindep-pivot-cols">
      <statement>
        <p>
          Let <m>v_1,v_2,\ldots,v_k</m> be vectors in <m>\R^n</m>, and consider the matrix
          <me>A = \mat{| |, , |; v_1 v_2 \cdots, v_k; | |, , |}.</me>
          Then we can delete the columns of <m>A</m> <em>without</em> pivots (the columns corresponding to the free variables), without changing <m>\Span\{v_1,v_2,\ldots,v_k\}</m>.
        </p>

        <p>The pivot columns are linearly independent, so we cannot delete any more columns without changing the span.</p>
      </statement>

      <proof>
        <p>
          If the matrix is in reduced row echelon form:
          <me>A = \mat{1 0 2 0; 0 1 3 0; 0 0 0 1}</me>
          then the column without a pivot is visibly in the span of the pivot columns:
          <me>\vec{2 3 0} = 2\vec{1 0 0} + 3\vec{0 1 0} + 0\vec{0 0 1},</me>
          and the pivot columns are linearly independent:
          <me>
            \vec{0 0 0} = x_1\vec{1 0 0} + x_2\vec{0 1 0} + x_4\vec{0 0 1}
            = \vec{x_1 x_2 x_4} \implies x_1 = x_2 = x_4 = 0.
          </me>
        </p>

        <p>
          If the matrix is not in reduced row echelon form, then we row reduce:
          <me>
            A = \mat{1 7 23 3; 2 4 16 0; -1 -2 -8 4}
            \quad\xrightarrow{\text{RREF}}\quad\mat{1 0 2 0; 0 1 3 0; 0 0 0 1}.
          </me>
          The following two vector equations have the same solution set, as they come from row-equivalent matrices:
          <me>
            \spalignsysdelims..\syseq{
            x_1\vec{1 2 -1} + x_2\vec{7 4 -2} + x_3\vec{23 16 -8} + x_4\vec{3 0 4} = 0;
            x_1\vec{1 0  0} + x_2\vec{0 1  0} + x_3\vec{ 2  3  0} + x_4\vec{0 0 1}
            = 0\rlap.
            }
          </me>
          We conclude that
          <me>\vec{23 16 -8} = 2\vec{1 2 -1} + 3\vec{7 4 -2} + 0\vec{3 0 4}</me>
          and that
          <me>x_1\vec{1 2 -1} + x_2\vec{7 4 -2} + x_4\vec{3 0 4} = 0</me>
          has only the trivial solution.
        </p>
      </proof>
    </theorem>

    <p>
      Note that it is necessary to row reduce <m>A</m> to find which are its <xref ref="defn-pivot-pos" text="title">pivot columns</xref>.  However, the span of the columns of the row reduced matrix is generally <em>not</em> equal to the span of the columns of <m>A</m>: one must use the pivot columns of the <em>original</em> matrix.  See <xref ref="dimension-basis-colspace"/> for a restatement of the above theorem.
    </p>

    <example>
      <p>
        The matrix
        <me>A = \mat[r]{1 2 0 -1; -2 -3 4 5; 2 4 0 -2}</me>
        has reduced row echelon form
        <me>\mat[r]{1 0 -8 -7; 0 1 4 3; 0 0 0 0}.</me>
        Therefore, the first two columns of <m>A</m> are the pivot columns, so we can delete the others without changing the span:
        <me>
          \Span\left\{\vec{1 -2 2},\;\vec{2 -3 4}\right\}
          = \Span\left\{\vec{1 -2 2},\;\vec{2 -3 4},\;\vec{0 4 0},\;\vec{-1 5 -2}\right\}.
        </me>
        Moreover, the first two columns are linearly independent.
      </p>
    </example>

    <bluebox xml:id="pivot-cols-dim">
      <title>Pivot Columns and Dimension</title>
      <p>
        Let <m>d</m> be the number of pivot columns in the matrix
        <me>A = \mat{| |, , |; v_1 v_2 \cdots, v_k; | |, , |}.</me>
        <ul>
          <li>If <m>d=1</m> then <m>\Span\{v_1,v_2,\ldots,v_k\}</m> is a line.</li>
          <li>If <m>d=2</m> then <m>\Span\{v_1,v_2,\ldots,v_k\}</m> is a plane.</li>
          <li>If <m>d=3</m> then <m>\Span\{v_1,v_2,\ldots,v_k\}</m> is a 3-space.</li>
          <li>Et cetera.</li>
        </ul>
	The number <m>d</m> is called the dimension.  We discussed this notion in this <xref ref="soln-sets-dim-hom"/> and this <xref ref="soln-sets-dim-inhom"/>.  We will define this concept rigorously in <xref ref="dimension"/>.
      </p>
    </bluebox>

  </subsection>

  <!--
  <exercises>
    <title>Reading Questions</title>

    <exercise>
      <statement>
        <p>
          Let <m>S</m> be the set of three vectors below.<me>
          S = \setof{ \vec{1 2 -1},\, \vec{3 -4 2},\, \vec{4 -2 1} }
          </me>
          Is <m>S</m> linearly independent or linearly dependent?  Explain why.
        </p>
      </statement>
    </exercise>

    <exercise>
      <statement>
        <p>
          Let <m>S</m> be the set of three vectors below.<me>
          S = \setof{ \vec{1 -1 0},\, \vec{3 2 2},\, \vec{4 3 -4} }
          </me>
          Is <m>S</m> linearly independent or linearly dependent?  Explain why.
        </p>
      </statement>
    </exercise>

    <exercise>
      <statement>
        <p>
          Is the matrix below singular or nonsingular?  Explain your answer using only the final conclusion you reached in the previous question, along with one new theorem.<me>
          \mat{1 3 4; -1 2 3; 0 2 -4}
          </me>
        </p>
      </statement>
    </exercise>
  </exercises>

  <exercises>
    <title>Exercises</title>

    <exercisegroup>
      <introduction>
        <p>Determine if the sets of vectors in the following exercises are linearly independent or linearly dependent.  When the set is linearly dependent, exhibit a nontrivial relation of linear dependence.</p>
      </introduction>

      <exercise xml:id="exercise-li-10">
        <statement>
          <p><m>
            \setof{ \vec{1 -2 1},\, \vec{2 -1 3},\, \vec{1 5 0} }
          </m></p>
        </statement>
        <solution xml:id="solution-li-10">
          <p>With three vectors from <m>\R^3</m>, we can form a square matrix by making these three vectors the columns of a matrix.  We do so, and row-reduce to obtain<me>
          \mat{\leading1 0 0; 0 \leading1 0; 0 0 \leading1},
          </me>the <m>3\times 3</m> identity matrix.  So by <xref provisional="theorem-?"/> the original matrix is nonsingular and its columns are therefore a linearly independent set.</p>
        </solution>
      </exercise>

      <exercise xml:id="exercise-li-11">
        <statement>
          <p><m>
            \setof{ \vec{-1 2 4 2},\, \vec{3 3 -1 3},\, \vec{7 3 -6 4} }
          </m></p>
        </statement>
        <solution xml:id="solution-li-11">
          <p><xref provisional="theorem-?"/> says we can answer this question by putting these vectors into a matrix as columns and row-reducing.  Doing this we obtain<me>
          \mat{\leading1 0 0; 0 \leading1 0; 0 0 \leading1; 0 0 0}.
          </me>  Since every column has a pivot, the theorem says the vectors are linearly independent.</p>
        </solution>
      </exercise>

      <exercise xml:id="exercise-li-12">
        <statement>
          <p><m>
            \setof{ \vec{-2 -1 -1},\, \vec{1 0 -1},\, \vec{3 3 6},\,
            \vec{-5 -4 -6},\, \vec{4 4 7}}
          </m></p>
        </statement>
        <solution xml:id="solution-li-12">
          <p>This is a set of five vectors in <m>\R^3</m>.  <xref provisional="theorem-?"/> says the set is linearly dependent.  Boom.</p>
        </solution>
      </exercise>

      <exercise xml:id="exercise-li-13">
        <statement>
          <p><m>
            \setof{
              \vec{1 -2 2 5 3},\,
              \vec{3 3 1 2 -4},\,
              \vec{2 1 2 -1 1},\,
              \vec{1 0 1 2 2}}
          </m></p>
        </statement>
        <solution xml:id="solution-li-13">
          <p><xref provisional="theorem-?"/> says we can answer this question by putting these vectors into a matrix as columns and row-reducing.  Doing this we obtain<me>
          \mat{\leading1 0 0 0;
              0 \leading1 0 0;
              0 0 \leading1 0;
              0 0 0 \leading1;
              0 0 0 0}.
          </me>Since every column has a pivot, the theorem says the vectors are linearly independent.</p>
        </solution>
      </exercise>

      <exercise xml:id="exercise-li-14">
        <statement>
          <p><m>
            \setof{
              \vec{1 2 -1 0 1},\,
              \vec{3 2 -1 2 2},\,
              \vec{4 4 -2 2 3},\,
              \vec{-1 2 -1 -2 0}
            }
          </m></p>
        </statement>
        <solution xml:id="solution-li-14">
          <p><xref provisional="theorem-?"/> says we can answer this question by putting these vectors into a matrix as columns and row-reducing.  Doing this we obtain<me>
          \mat{\leading1 0 1 2;
              0 \leading1 1 -1;
              0 0 0 0;
              0 0 0 0;
              0 0 0 0}.
          </me>Since the last two columns do not have a pivot, the theorem says the vectors are linearly dependent.</p>
        </solution>
      </exercise>

      <exercise xml:id="exercise-li-15">
        <statement>
          <p><m>
            \setof{\vec{2 1 3 -1 2},\, \vec{4 -2 1 3 2},\, \vec{10 -7 0 10 4}}
          </m></p>
        </statement>
        <solution xml:id="solution-li-15">
          <p><xref provisional="theorem-?"/> says we can answer this question by putting these vectors into a matrix as columns and row-reducing.  Doing this we obtain<me>
          \mat{\leading1 0 -1;
              0 \leading1 3;
              0 0 0;
              0 0 0;
              0 0 0}.
          </me>Since the last two columns do not have a pivot, the theorem says the vectors are linearly dependent.</p>
        </solution>
      </exercise>

    </exercisegroup>

    <exercise xml:id="exercise-li-20">
      <statement>
        <p>
          For the matrix <m>B</m> below, find a set <m>S</m> of linearly independent vectors that spans the null space of <m>B</m>.<me>
          B = \mat{-3 1 -2 7; -1 2 1 4; 1 1 2 -1}
          </me>
        </p>
      </statement>
      <solution xml:id="solution-li-20">
        <p>The requested set is described by <xref provisional="theorem-?"/>.  It is easiest to find by using the procedure of example <xref provisional="example-?"/>.  Begin by row-reducing the matrix, viewing it as the coefficient matrix of a homogeneous system of equations.  We obtain
        <me>
          \mat{\leading1 0 1 -2; 0 \leading1 1 1; 0 0 0 0}.
        </me>
        The free variables are <m>x_3</m> and <m>x_4</m>, due to the locations of the non-pivot columns.  It follows from <xref provisional="theorem-?" /> that the parametric vector form of the solution set of the matrix equation <m>Bx=0</m> is
        <me>\vec{x_1 x_2 x_3 x_4} = x_3\vec{-1 -1 1 0} + x_4\vec{2 -1 0 1}.</me>
        The null space is exactly the solution set of <m>Bx=0</m>;
        the constant vectors in the above expression span the null space, and they are linearly independent.  Thus we can take our answer to be
        <me>
          S = \setof{\vec{-1 -1 1 0},\, \vec{2 -1 0 1}}.
        </me>
        </p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-21">
      <statement>
        <p>
          For the matrix <m>A</m> below, find a set <m>S</m> of linearly independent vectors that spans the null space of <m>A</m>.
          <me>
            B = \mat{-1 -2 2 1 5; 1 2 1 1 5; 3 6 1 2 7; 2 4 0 1 2}
          </me>
        </p>
      </statement>
      <solution xml:id="solution-li-21">
        <p>
          We want to find a linearly independent set <m>S</m> that spans the solution set of the homogeneous system <m>Ax=0</m>.    We begin by row-reducing the matrix, to obtain
          <me>
            \mat{\leading1 2 0 0 3; 0 0 \leading1 0 6; 0 0 0 \leading1 -4; 0 0 0 0 0}.
          </me>
          The free variables correspond to the second and fifth columns.  Using <xref provisional="theorem-?" />, the parametric vector form of the solution set of <m>Ax=0</m> is
          <me>
            \vec{x_1 x_2 x_3 x_4 x_5} = x_2\vec{-2 1 0 0 0} + x_5\vec{-3 0 -6 4 1}.
          </me>
          The constant vectors in the above expression span the null space, and they are linearly independent.  Thus we can take our answer to be
        <me>
          S = \setof{\vec{-2 1 0 0 0},\, \vec{-3 0 -6 4 1}}.
        </me>
        </p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-22">
      <statement>
        <p>
          Find a set <m>T</m> of column vectors such that (1) the span of <m>T</m> is the null space of <m>B</m>, and (2) <m>T</m> is a linearly independent set.
          <me>
            B = \mat{2 1 1 1; -4 -3 1 -7; 1 1 -1 3}
          </me>
        </p>
      </statement>
      <solution xml:id="solution-li-22">
        <p>The conclusion of <xref provisional="theorem-?"/> gives us everything this question asks for.  We need the reduced row-echelon form of the matrix in order to determine the number of vectors in <m>T</m>, and their entries.
        <me>
          \mat{2 1 1 1; -4 -3 1 -7; 1 1 -1 3} \rref
          \mat{\leading1 0 2 -2; 0 \leading1 -3 5; 0 0 0 0}
        </me>
        The third and fourth columns are free, so we can apply <xref provisional="theorem-?"/> and take the set <m>T</m> to have the two vectors
        <me>
          T = \setof{\vec{-2 3 1 0},\,\vec{2 -5 0 1}}.
        </me>
        </p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-23">
      <statement>
        <p>
          Find a set <m>S</m> that is linearly independent and spans the null space of the matrix <m>A</m> below.
          <me>
            A = \mat{2 3 3 1 4; 1 1 -1 -1 -3; 3 2 -8 -1 1}
          </me>
        </p>
      </statement>
      <solution xml:id="solution-li-23">
        <p>
          A direct application of <xref provisional="theorem-?"/> will provide the desired set.  We require the reduced row-echelon form of <m>A</m>:
          <me>
            \mat{2 3 3 1 4; 1 1 -1 -1 -3; 3 2 -8 -1 1} \rref
            \mat{\leading1 0 -6 0 3; 0 \leading1 5 0 -2; 0 0 0 \leading1 4}
          </me>
          The third and fifth columns are free, so we can apply <xref provisional="theorem-?"/> and take the set <m>T</m> to have the two vectors
          <me>
            T = \setof{\vec{6 -5 1 0 0},\,\vec{-3 2 0 -4 1}}.
          </me>
        </p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-24">
      <statement>
        <p>
          For the matrix <m>A</m> below, find a set of vectors <m>S</m> such that (1) <m>S</m> is linearly independent, and (2) the span of <m>S</m> equals the null space of <m>A</m>.
          <me>
            A = \mat{1 1 6 -8; 1 -2 0 1; -2 1 -6 7}
          </me>
        </p>
      </statement>
      <solution xml:id="solution-li-24">
        <p>
          <xref provisional="theorem-?" /> says that if we find the vector form of the solutions to the homogeneous system <m>Ax=0</m>, then the spanning vectors will have the desired properties.  We row-reduce <m>A</m>, viewing it as the augmented matrix of a homogeneous system with an invisible columns of zeros as the last column:
          <me>
            \mat{1 1 6 -8; 1 -2 0 1; -2 1 -6 7} \rref
            \mat{\leading1 0 4 -5; 0 1 2 -3; 0 0 0 0}
          </me>
          Moving to the parametric vector form of the solutions (<xref provisional="theorem-?"/>), with free variables <m>x_3</m> and <m>x_4</m>, the solutions to the consistent system <m>Ax=0</m> (it is consistent: see <xref provisional="theorem-?"/>) can be expressed as
          <me>
            \vec{x_1 x_2 x_3 x_4} = x_3\vec{-4 -2 1 0} + x_4\vec{5 3 0 1}.
          </me>
          Thus we can take <m>S</m> to be
          <me>
            S = \setof{\vec{-4 -2 1 0},\, \vec{5 3 0 1}}.
          </me>
        </p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-30">
      <statement>
        <p>
          Suppose that <m>S = \{v_1,v_2,v_3\}</m> is a set of three vectors in <m>\R^{873}</m>.  Prove that the set
          <me>
            T = \bigl\{2v_1+3v_2+v_3,\, v_1-v_2-2v_3,\, 2v_1+v_2-v_3\bigr\}
          </me>
          is linearly dependent.
        </p>
      </statement>
      <solution xml:id="solution-li-30">
        <p>
          According to Definition <xref provisional="defn-?" />, we need to find scalars <m>a_1,a_2,a_3</m>, not all equal to zero, such that
          <me>
            a_1\bigl(2{v}_1+3{v}_2+{v}_3\bigr) +
            a_2\bigl({v}_1-{v}_2-2{v}_3\bigr) +
            a_3\bigl(2{v}_1+{v}_2-{v}_3\bigr) = 0.
          </me>
          We can rearrange this vector equation to
          <me>
            \bigl(2a_1+a_2+2a_3\bigr){v}_1+
                    \bigl(3a_1-a_2+a_3\bigr){v}_2+
                    \bigl(a_1-2a_2-a_3\bigr){v}_3
                    = 0
          </me>
          We can certainly make this vector equation true if we can determine values for the <m>a</m><rsq/>s such that
          <me>
            \syseq{2a_1 + a_2 + a_3 = 0;
                   3a_1 - a_2 + a_3 = 0;
                   a_1 - 2a_2 - a_3 = 0\rlap.}
          </me>
          This is a homogeneous system of equations, and it has infinitely many solutions.  By the now familiar techniques, you can find one such solution to be <m>a_1=3</m>, <m>a_2=4</m>, <m>a_3=-5</m>, which you can (and should!) check in the original relation of linear dependence on <m>T</m> above.
        </p>

        <p>Note that simply writing down the three scalars, and demonstrating that they provide a nontrivial relation of linear dependence on <m>T</m>, could be considered an ironclad solution.  But it would not have been very informative for you if we had only done just that here.  Compare this solution very carefully with the solution to <xref ref="exercise-li-31" text="type-global"/>.</p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-31">
      <statement>
        <p>
          Suppose that <m>S = \{v_1,v_2,v_3\}</m> is a linearly independent set of three vectors in <m>\R^{873}</m>.  Prove that the set
          <me>
            T = \bigl\{2v_1+3v_2+v_3,\, v_1-v_2+2v_3,\, 2v_1+v_2-v_3\bigr\}
          </me>
          is linearly independent.
        </p>
      </statement>
      <solution xml:id="solution-li-31">
        <p>
          According to Definition <xref provisional="defn-?" />, we need show that if <m>a_1,a_2,a_3</m> are any scalars satisfying
          <me>
            a_1\bigl(2{v}_1+3{v}_2+{v}_3\bigr) +
            a_2\bigl({v}_1-{v}_2+2{v}_3\bigr) +
            a_3\bigl(2{v}_1+{v}_2-{v}_3\bigr) = 0,
          </me>
          then necessarily <m>a_1=a_2=a_3=0</m>.
          We can rearrange this vector equation to
          <me>
            \bigl(2a_1+a_2+2a_3\bigr){v}_1+
                    \bigl(3a_1-a_2+a_3\bigr){v}_2+
                    \bigl(a_1+2a_2-a_3\bigr){v}_3
                    = 0.
          </me>
          Because the set <m>S=\setof{v,v_2,v_3}</m> was assumed to be linearly independent, by <xref provisional="definition-?"/> we <em>must</em> conclude that
          <me>
            \syseq{2a_1 + a_2 + a_3 = 0;
                   3a_1 - a_2 + a_3 = 0;
                   a_1 + 2a_2 - a_3 = 0\rlap.}
          </me>
          By the familiar methods, you can check that this homogeneous system of linear equations admits only the trivial solution <m>a_1=a_2=a_3=0</m>, as desired.
        </p>

        <p>Compare this solution very carefully with <xref ref="exercise-li-30" text="type-global"/>, noting especially how this problem required (and used) the hypothesis that the original set be linearly independent, and how this solution feels more like a proof, while the previous problem could be solved with a fairly simple demonstration of any nontrivial relation of linear dependence.</p>
      </solution>
    </exercise>

    <exercise xml:id="exercise-li-32">
      <statement>
        <p>
          Let <m>W</m> be the span of the set of five vectors in <m>\R^3</m> given below.  Find a set <m>T</m> of three vectors in <m>W</m> such that <m>W=\Span T</m>.
          <me>
            W = \Span\setof{\vec{2 1 1},\,\vec{-1 -1 1},\,\vec{1 2 3},\,
            \vec{3 1 3},\,\vec{0 1 -3}}.
          </me>
        </p>
      </statement>
      <solution xml:id="solution-li-32">
        <p>
          not copied/written yet.
        </p>
      </solution>
    </exercise>


  </exercises>
  -->

</section>