tags/teachingunfolding disastershttp://blog.tremily.us//tags/teaching/unfolding disastersikiwiki2013-09-20T04:04:58ZFactor analysishttp://blog.tremily.us//posts/Factor_analysis/2013-09-20T04:04:58Z2013-03-16T17:04:21Z
<p>I've been trying to wrap my head around <a href="http://en.wikipedia.org/wiki/Factor_analysis">factor analysis</a> as a
theory for designing and understanding test and survey results. This
has turned out to be <a href="http://blog.tremily.us//tags/teaching/../../posts/Gumbel-Fisher-Tippett_distributions/">another</a>
one of those fields where the going has been a bit rough. I think the
key factors in making these older topics difficult are:</p>
<ul>
<li>“Everybody knows this, so we don't need to write up the details.”</li>
<li>“Hey, I can do better than Bob if I just tweak this knob…”</li>
<li>“I'll just publish this seminal paper behind a paywall…”</li>
</ul>
<p>The resulting discussion ends up being overly complicated, and it's
hard for newcomers to decide if people using similar terminology are
in fact talking about the same thing.</p>
<p>Some of the better open sources for background has been <a href="http://www.unc.edu/~rcm/book/factornew.htm">Tucker and
MacCallum's “Exploratory Factor Analysis” manuscript</a> and <a href="http://www.ics.uci.edu/~welling/classnotes/classnotes.html">Max
Welling's notes</a>. I'll use Welling's terminology for this
discussion.</p>
<p>The basic idea of factor analsys is to model <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>d</mi></math> measurable attributes
as generated by <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi><mi><</mi><mi>d</mi></math> common factors and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>d</mi></math> unique factors.
With <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>n</mi><mo>=</mo><mn>4</mn></math> and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi><mo>=</mo><mn>2</mn></math>, you get something like:</p>
<table class="img"><caption>Relationships between factors and measured attributes
(adapted from Tucker and MacCallum's Figure 1.2)</caption><tr><td><a href="http://blog.tremily.us//tags/teaching/../../posts/Factor_analysis/factors.png"><img src="http://blog.tremily.us//tags/teaching/../../posts/Factor_analysis/factors.png" width="520" height="472" alt="Relationships between factors and measured attributes" class="img" /></a></td></tr></table>
<p>Corresponding to the equation (<a href="http://www.ics.uci.edu/~welling/classnotes/papers_class/LinMod.ps.gz">Welling's eq. 1</a>):</p>
<div class="numberedEq"><span>(1)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>+</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>+</mo><mstyle mathvariant="bold"><mi>ν</mi></mstyle></math></div>
<p>The independent random variables <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math> are distributed
according to a Gaussian with zero mean and unit
variance <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>y</mi></mstyle></msub><mo stretchy="false">[</mo><mn>0</mn><mo>,</mo><mstyle mathvariant="bold"><mi>I</mi></mstyle><mo stretchy="false">]</mo></math> (zero mean because
constant offsets are handled by <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle></math>; unit variance because
scaling is handled by <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>). The independent random
variables <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>ν</mi></mstyle></math> are distributed according
to <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>ν</mi></mstyle></msub><mo stretchy="false">[</mo><mn>0</mn><mo>,</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo></math>, with (Welling's
eq. 2):</p>
<div class="numberedEq"><span>(2)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo>≡</mo><mtext>diag</mtext><mo stretchy="false">[</mo><msubsup><mi>σ</mi> <mn>1</mn> <mn>2</mn></msubsup><mo>,</mo><mi>…</mi><mo>,</mo><msubsup><mi>σ</mi> <mi>d</mi> <mn>2</mn></msubsup><mo stretchy="false">]</mo></math></div>
<p>The matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math> (linking common factors with measured
attributes <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mi>is</mi><mi>refered</mi><mi>to</mi><mi>as</mi><mi>the</mi><mo><em></mo><mi>factor</mi><mi>weights</mi><mo></em></mo><mi>or</mi><mo><em></mo><mi>factor</mi><mi>loadings</mi><mo></em></mo><mo>.</mo><mi>Because</mi><mi>the</mi><mi>only</mi><mi>source</mi><mi>of</mi><mi>constant</mi><mi>offset</mi><mi>is</mi></math>\mathbf{\mu}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>,</mo><mi>we</mi><mi>can</mi><mi>calculate</mi><mi>it</mi><mi>by</mi><mi>averaging</mi><mi>out</mi><mi>the</mi><mi>random</mi><mi>noise</mi><mo stretchy="false">(</mo><mi>Welling</mi><mo>′</mo><mi>s</mi><mi>eq</mi><mo>.</mo><mn>6</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>3</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>=</mo><mfrac><mn>1</mn><mi>N</mi></mfrac><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>where</mi></math>N<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>the</mi><mi>number</mi><mi>of</mi><mi>measurements</mi><mo stretchy="false">(</mo><mi>survey</mi><mi>responders</mi><mo stretchy="false">)</mo><mi>and</mi></math>\mathbf{x}<em>n<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>the</mi><mi>response</mi><mi>vector</mi><mi>for</mi><mi>the</mi></math>n^\text{th}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>responder</mi><mo>.</mo><mi>How</mi><mi>do</mi><mi>we</mi><mi>find</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>?</mo><mi>This</mi><mi>is</mi><mi>the</mi><mi>tricky</mi><mi>bit</mi><mo>,</mo><mi>and</mi><mi>there</mi><mi>are</mi><mi>a</mi><mi>number</mi><mi>of</mi><mi>possible</mi><mi>approaches</mi><mo>.</mo><mi>Welling</mi><mi>suggests</mi><mi>using</mi><mi>expectation</mi><mi>maximization</mi><mo stretchy="false">(</mo><mi>EM</mi><mo stretchy="false">)</mo><mo>,</mo><mi>and</mi><mi>there</mi><mo>′</mo><mi>s</mi><mi>an</mi><mi>excellent</mi><mi>example</mi><mi>of</mi><mi>the</mi><mi>procedure</mi><mi>with</mi><mi>a</mi><mi>colorblind</mi><mi>experimenter</mi><mi>drawing</mi><mi>colored</mi><mi>balls</mi><mi>in</mi><mi>his</mi><mo stretchy="false">[</mo><mi>EM</mi><mi>notes</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mi>EM</mi><mo stretchy="false">]</mo><mo stretchy="false">(</mo><mi>to</mi><mi>test</mi><mi>my</mi><mi>understanding</mi><mo>,</mo><mi>I</mi><mi>wrote</mi><merror><mtext>Unknown character</mtext></merror><mi>a</mi><mi>href</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mo>.</mo><mo>.</mo><mo stretchy="false">/</mo><mo>.</mo><mo>.</mo><mo stretchy="false">/</mo><mi>posts</mi><mo stretchy="false">/</mo><msub><mi>Factor</mi> <mi>analysis</mi></msub><mo stretchy="false">/</mo><mi>color</mi><mo>−</mo><mi>ball</mi><mo>.</mo><mi>py</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>color</mi><mo>−</mo><mi>ball</mi><mo>.</mo><mi>py</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>a</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">)</mo><mo>.</mo><mi>To</mi><mi>simplify</mi><mi>calculations</mi><mo>,</mo><mi>Welling</mi><mi>defines</mi><mo stretchy="false">(</mo><mi>before</mi><mi>eq</mi><mo>.</mo><mn>15</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>4</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>′</mo></mtd> <mtd><mo>≡</mo><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">]</mo></mtd></mtr> <mtr><mtd><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo></mtd> <mtd><mo>≡</mo><mo stretchy="false">[</mo><msup><mstyle mathvariant="bold"><mi>y</mi></mstyle> <mi>T</mi></msup><mo>,</mo><mn>1</mn><msup><mo stretchy="false">]</mo> <mi>T</mi></msup></mtd></mtr></mtable></mrow></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>which</mi><mi>reduce</mi><mi>the</mi><mi>model</mi><mi>to</mi><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>5</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>′</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mo lspace="verythinmathspace" rspace="0em">+</mo><mstyle mathvariant="bold"><mi>ν</mi></mstyle></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>After</mi><mi>some</mi><mi>manipulation</mi><mi>Welling</mi><mi>works</mi><mi>out</mi><mi>the</mi><mi>maximizing</mi><mi>updates</mi><mo stretchy="false">(</mo><mi>eq</mi><mo>′</mo><mi>ns</mi><mn>16</mn><mi>and</mi><mn>17</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>6</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mo>′</mo> <mtext>new</mtext></msup></mtd> <mtd><mo>=</mo><mrow><mo>(</mo><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><msup><mo stretchy="false">]</mo> <mi>T</mi></msup><mo>)</mo></mrow><msup><mrow><mo>(</mo><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><msup><mo>′</mo> <mi>T</mi></msup><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo><mo>)</mo></mrow> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup></mtd></mtr> <mtr><mtd><msup><mstyle mathvariant="bold"><mi>Σ</mi></mstyle> <mtext>new</mtext></msup></mtd> <mtd><mo>=</mo><mfrac><mn>1</mn><mi>N</mi></mfrac><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><mtext>diag</mtext><mo stretchy="false">[</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><msubsup><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi> <mi>T</mi></msubsup><mo>−</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mo>′</mo> <mtext>new</mtext></msup><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo><msubsup><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi> <mi>T</mi></msubsup><mo stretchy="false">]</mo></mtd></mtr></mtable></mrow></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>The</mi><mi>expectation</mi><mi>values</mi><mi>used</mi><mi>in</mi><mi>these</mi><mi>updates</mi><mi>are</mi><mi>given</mi><mi>by</mi><mo stretchy="false">(</mo><mi>Welling</mi><mo>′</mo><mi>s</mi><mi>eq</mi><mo>′</mo><mi>ns</mi><mn>12</mn><mi>and</mi><mn>13</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>7</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo></mtd> <mtd><mo>=</mo><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo>+</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><msup><mo stretchy="false">)</mo> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><msup><mstyle mathvariant="bold"><mi>y</mi></mstyle> <mi>T</mi></msup><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo></mtd> <mtd><mo>=</mo><mstyle mathvariant="bold"><mi>I</mi></mstyle><mo>−</mo><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo>+</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><msup><mo stretchy="false">)</mo> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>+</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><msup><mo stretchy="false">]</mo> <mi>T</mi></msup></mtd></mtr></mtable></mrow></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>Survey</mi><mi>analysis</mi><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mi>Enough</mi><mi>abstraction</mi><mo>!</mo><mi>Let</mi><mo>′</mo><mi>s</mi><mi>look</mi><mi>at</mi><mi>an</mi><mi>example</mi><mo>:</mo><mo stretchy="false">[</mo><mi>survey</mi><mi>results</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mi>survey</mi><mo stretchy="false">]</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>import</mi><mi>numpy</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>scores</mi><mo>=</mo><mi>numpy</mi><mo>.</mo><mi>genfromtxt</mi><mo stretchy="false">(</mo><mo>′</mo><msub><mi>Factor</mi> <mi>analysis</mi></msub><mo stretchy="false">/</mo><mi>survey</mi><mo>.</mo><mi>data</mi><mo>′</mo><mo>,</mo><mi>delimiter</mi><mo>=</mo><mo>′</mo><mo lspace="0em" rspace="thinmathspace">t</mo><mo>′</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>scores</mi><mi>array</mi><mo stretchy="false">(</mo><mo stretchy="false">[</mo><mo stretchy="false">[</mo><mn>1</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>1</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo stretchy="false">]</mo><mo stretchy="false">]</mo><mo stretchy="false">)</mo><mo><code></mo><mi>scores</mi><mo stretchy="false">[</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo stretchy="false">]</mo><mo></code></mo><mi>is</mi><mi>the</mi><mi>answer</mi><mi>the</mi><mo><code></mo><mi>i</mi><mo></code></mo><mi>th</mi><mi>respondent</mi><mi>gave</mi><mi>for</mi><mi>the</mi><mo><code></mo><mi>j</mi><mo></code></mo><mi>th</mi><mi>question</mi><mo>.</mo><mi>We</mi><mo>′</mo><mi>re</mi><mi>looking</mi><mi>for</mi><mi>underlying</mi><mi>factors</mi><mi>that</mi><mi>can</mi><mi>explain</mi><mi>covariance</mi><mi>between</mi><mi>the</mi><mi>different</mi><mi>questions</mi><mo>.</mo><mi>Do</mi><mi>the</mi><mi>question</mi><mi>answers</mi><mo stretchy="false">(</mo></math>\mathbf{x}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo stretchy="false">)</mo><mi>represent</mi><mi>some</mi><mi>underlying</mi><mi>factors</mi><mo stretchy="false">(</mo></math>\mathbf{y}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo stretchy="false">)</mo><mo>?</mo><mi>Let</mi><mo>′</mo><mi>s</mi><mi>start</mi><mi>off</mi><mi>by</mi><mi>calculating</mi></math>\mathbf{\mu}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>:</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>def</mi><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>row</mi><mo stretchy="false">)</mo><mo>:</mo><mo>.</mo><mo>.</mo><mo>.</mo><mi>print</mi><mo stretchy="false">(</mo><mo>′</mo><mo>′</mo><mo>.</mo><mi>join</mi><mo stretchy="false">(</mo><mo>′</mo><mrow><mo>:</mo><mn>0.2</mn><mi>f</mi></mrow><mo>′</mo><mo>.</mo><mi>format</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mi>for</mi><mi>x</mi><mi>in</mi><mi>row</mi><mo stretchy="false">)</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>mu</mi><mo>=</mo><mi>scores</mi><mo>.</mo><mi>mean</mi><mo stretchy="false">(</mo><mi>axis</mi><mo>=</mo><mn>0</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>mu</mi><mo stretchy="false">)</mo><mn>2.42</mn><mn>3.58</mn><mn>5.08</mn><mn>5.75</mn><mn>6.08</mn><mn>3.42</mn><mn>3.92</mn><mn>4.25</mn><mi>Next</mi><mi>we</mi><mi>need</mi><mi>priors</mi><mi>for</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><merror><mtext>Unknown character</mtext></merror><mi>span</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>createlink</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>MDP</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mi>has</mi><mi>an</mi><mi>implementation</mi><mi>for</mi><merror><mtext>Unknown character</mtext></merror><mi>a</mi><mi>href</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mo>.</mo><mo>.</mo><mo stretchy="false">/</mo><mo>.</mo><mo>.</mo><mo stretchy="false">/</mo><mi>posts</mi><mo stretchy="false">/</mo><mi>Python</mi><mo stretchy="false">/</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>Python</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>a</mi><merror><mtext>Unknown character</mtext></merror><mo>,</mo><mi>and</mi><mi>their</mi><mo stretchy="false">[</mo><mi>FANode</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mo stretchy="false">]</mo><mi>uses</mi><mi>a</mi><mi>Gaussian</mi><mi>random</mi><mi>matrix</mi><mi>for</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi><mi>the</mi><mi>diagonal</mi><mi>of</mi><mi>the</mi><mi>score</mi><mi>covariance</mi><mi>for</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><mi>They</mi><mi>also</mi><mi>use</mi><mi>the</mi><mi>score</mi><mi>covariance</mi><mi>to</mi><mi>avoid</mi><mi>repeated</mi><mi>summations</mi><mi>over</mi></math>n<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>import</mi><mi>mdp</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>def</mi><msub><mi>print</mi> <mi>matrix</mi></msub><mo stretchy="false">(</mo><mi>matrix</mi><mo stretchy="false">)</mo><mo>:</mo><mo>.</mo><mo>.</mo><mo>.</mo><mi>for</mi><mi>row</mi><mi>in</mi><mi>matrix</mi><mo>:</mo><mo>.</mo><mo>.</mo><mo>.</mo><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>row</mi><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>fa</mi><mo>=</mo><mi>mdp</mi><mo>.</mo><mi>nodes</mi><mo>.</mo><mi>FANode</mi><mo stretchy="false">(</mo><msub><mi>output</mi> <mi>dim</mi></msub><mo>=</mo><mn>3</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>numpy</mi><mo>.</mo><mi>random</mi><mo>.</mo><mi>seed</mi><mo stretchy="false">(</mo><mn>1</mn><mo stretchy="false">)</mo><mo>#</mo><mi>for</mi><mi>consistend</mi><mi>doctest</mi><mi>results</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>responder</mi> <mi>scores</mi></msub><mo>=</mo><mi>fa</mi><mo stretchy="false">(</mo><mi>scores</mi><mo stretchy="false">)</mo><mo>#</mo><mi>common</mi><mi>factors</mi><mi>for</mi><mi>each</mi><mi>responder</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>matrix</mi></msub><mo stretchy="false">(</mo><msub><mi>responder</mi> <mi>scores</mi></msub><mo stretchy="false">)</mo><mo>−</mo><mn>1.92</mn><mo>−</mo><mn>0.45</mn><mn>0.00</mn><mn>0.67</mn><mn>1.97</mn><mn>1.96</mn><mn>0.70</mn><mn>0.03</mn><mo>−</mo><mn>2.00</mn><mn>0.29</mn><mn>0.03</mn><mo>−</mo><mn>0.60</mn><mo>−</mo><mn>1.02</mn><mn>1.79</mn><mo>−</mo><mn>1.43</mn><mn>0.82</mn><mn>0.27</mn><mo>−</mo><mn>0.23</mn><mo>−</mo><mn>0.07</mn><mo>−</mo><mn>0.08</mn><mn>0.82</mn><mo>−</mo><mn>1.38</mn><mo>−</mo><mn>0.27</mn><mn>0.48</mn><mn>0.79</mn><mo>−</mo><mn>1.17</mn><mn>0.50</mn><mn>1.59</mn><mo>−</mo><mn>0.30</mn><mo>−</mo><mn>0.41</mn><mn>0.01</mn><mo>−</mo><mn>0.48</mn><mn>0.73</mn><mo>−</mo><mn>0.46</mn><mo>−</mo><mn>1.34</mn><mn>0.18</mn><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>fa</mi><mo>.</mo><mi>mu</mi><mo>.</mo><mi>flat</mi><mo stretchy="false">)</mo><mn>2.42</mn><mn>3.58</mn><mn>5.08</mn><mn>5.75</mn><mn>6.08</mn><mn>3.42</mn><mn>3.92</mn><mn>4.25</mn><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>fa</mi><mo>.</mo><mi>mu</mi><mo>.</mo><mi>flat</mi><mo>=</mo><mo>=</mo><mi>mu</mi><mo>#</mo><mi>MDP</mi><mi>agrees</mi><mi>with</mi><mi>our</mi><mi>earlier</mi><mi>calculation</mi><mi>array</mi><mo stretchy="false">(</mo><mo stretchy="false">[</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo stretchy="false">]</mo><mo>,</mo><mi>dtype</mi><mo>=</mo><mi>bool</mi><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>matrix</mi></msub><mo stretchy="false">(</mo><mi>fa</mi><mo>.</mo><mi>A</mi><mo stretchy="false">)</mo><mo>#</mo><mi>factor</mi><mi>weights</mi><mi>for</mi><mi>each</mi><mi>question</mi><mn>0.80</mn><mo>−</mo><mn>0.06</mn><mo>−</mo><mn>0.45</mn><mn>0.17</mn><mn>0.30</mn><mo>−</mo><mn>0.65</mn><mn>0.34</mn><mo>−</mo><mn>0.13</mn><mo>−</mo><mn>0.25</mn><mn>0.13</mn><mo>−</mo><mn>0.73</mn><mo>−</mo><mn>0.64</mn><mn>0.02</mn><mo>−</mo><mn>0.32</mn><mo>−</mo><mn>0.70</mn><mn>0.61</mn><mn>0.23</mn><mn>0.86</mn><mn>0.08</mn><mn>0.63</mn><mn>0.59</mn><mo>−</mo><mn>0.09</mn><mn>0.67</mn><mn>0.13</mn><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>fa</mi><mo>.</mo><mi>sigma</mi><mo stretchy="false">)</mo><mo>#</mo><mi>unique</mi><mi>noise</mi><mi>for</mi><mi>each</mi><mi>question</mi><mn>0.04</mn><mn>0.02</mn><mn>0.38</mn><mn>0.55</mn><mn>0.30</mn><mn>0.05</mn><mn>0.48</mn><mn>0.21</mn><mi>Because</mi><mi>the</mi><mi>covariance</mi><mi>is</mi><mi>unaffected</mi><mi>by</mi><mi>the</mi><mi>rotation</mi></math>\mathbf{A}\rightarrow\mathbf{A}\mathbf{R}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>,</mo><mi>the</mi><mi>estimated</mi><mi>weights</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi><mi>responder</mi><mi>scores</mi></math>\mathbf{y}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>can</mi><mi>be</mi><mi>quite</mi><mi>sensitive</mi><mi>to</mi><mi>the</mi><mi>seed</mi><mi>priors</mi><mo>.</mo><mi>The</mi><mi>width</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>of</mi><mi>the</mi><mi>unique</mi><mi>noise</mi></math>\mathbf{\nu}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>more</mi><mi>robust</mi><mo>,</mo><mi>because</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>unaffected</mi><mi>by</mi><mi>rotations</mi><mi>on</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><mi>Related</mi><mi>tidbits</mi><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mi>Communality</mi><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mi>The</mi><mo stretchy="false">[</mo><mi>communality</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mo stretchy="false">]</mo></math>h</em>i^2<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>of</mi><mi>the</mi></math>i^\text{th}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>measured</mi><mi>attribute</mi></math>x_i<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>the</mi><mi>fraction</mi><mi>of</mi><mi>variance</mi><mi>in</mi><mi>the</mi><mi>measured</mi><mi>attribute</mi><mi>which</mi><mi>is</mi><mi>explained</mi><mi>by</mi><mi>the</mi><mi>set</mi><mi>of</mi><mi>common</mi><mi>factors</mi><mo>.</mo><mi>Because</mi><mi>the</mi><mi>common</mi><mi>factors</mi></math>\mathbf{y}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>have</mi><mi>unit</mi><mi>variance</mi><mo>,</mo><mi>the</mi><mi>communality</mi><mi>is</mi><mi>given</mi><mi>by</mi><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>8</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>h</mi> <mi>i</mi></msub><mo>=</mo><mfrac><mrow><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></msubsup><msubsup><mi>A</mi> <mi>ij</mi> <mn>2</mn></msubsup></mrow><mrow><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></msubsup><msubsup><mi>A</mi> <mi>ij</mi> <mn>2</mn></msubsup><mo>+</mo><msubsup><mi>σ</mi> <mn>1</mn> <mn>2</mn></msubsup></mrow></mfrac></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>factor</mi> <mi>variance</mi></msub><mo>=</mo><mi>numpy</mi><mo>.</mo><mi>array</mi><mo stretchy="false">(</mo><mo stretchy="false">[</mo><mi>sum</mi><mo stretchy="false">(</mo><mi>row</mi><mo><em></mo><mo></em></mo><mn>2</mn><mo stretchy="false">)</mo><mi>for</mi><mi>row</mi><mi>in</mi><mi>fa</mi><mo>.</mo><mi>A</mi><mo stretchy="false">]</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>h</mi><mo>=</mo><mi>numpy</mi><mo>.</mo><mi>array</mi><mo stretchy="false">(</mo><mo>.</mo><mo>.</mo><mo>.</mo><mo stretchy="false">[</mo><mi>var</mi><mo stretchy="false">/</mo><mo stretchy="false">(</mo><mi>var</mi><mo>+</mo><mi>sig</mi><mo stretchy="false">)</mo><mi>for</mi><mi>var</mi><mo>,</mo><mi>sig</mi><mi>in</mi><mi>zip</mi><mo stretchy="false">(</mo><msub><mi>factor</mi> <mi>variance</mi></msub><mo>,</mo><mi>fa</mi><mo>.</mo><mi>sigma</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>h</mi><mo stretchy="false">)</mo><mn>0.95</mn><mn>0.97</mn><mn>0.34</mn><mn>0.64</mn><mn>0.66</mn><mn>0.96</mn><mn>0.61</mn><mn>0.69</mn><mi>There</mi><mi>may</mi><mi>be</mi><mi>some</mi><mi>scaling</mi><mi>issues</mi><mi>in</mi><mi>the</mi><mi>communality</mi><mi>due</mi><mi>to</mi><mi>deviations</mi><mi>between</mi><mi>the</mi><mi>estimated</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math> and the
variations contained in the measured scores (why?):</p>
<pre><code>>>> print_row(factor_variance + fa.sigma)
0.89 0.56 0.57 1.51 0.89 1.21 1.23 0.69
>>> print_row(scores.var(axis=0, ddof=1)) # total variance for each question
0.99 0.63 0.63 1.66 0.99 1.36 1.36 0.75
</code></pre>
<p>The <a href="http://sites.stat.psu.edu/~ajw13/stat505/fa06/17_factor/07_factor_commun.html">proportion of total variation</a> explained by the
common factors is given by:</p>
<div class="numberedEq"><span>(9)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mfrac><mrow><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></munderover><msub><mi>h</mi> <mi>i</mi></msub></mrow><mrow></mrow></mfrac></math></div>
<h2>Varimax rotation</h2>
<p>As mentioned earlier, factor analysis generated loadings <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>
that are unique up to an arbitrary rotation <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>R</mi></mstyle></math> (as you'd
expect for a <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi></math>-dimensional Gaussian ball of factors <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math>).
A number of of schemes have been proposed to simplify the initial
loadings by rotating <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math> to reduce off-diagonal terms. One
of the more popular approaches is Henry Kaiser's <em>varimax rotation</em>
(unfortunately, I don't have access to either his <a href="http://oskicat.berkeley.edu/record=b12138762~S1">thesis</a> or
the <a href="http://dx.doi.org/10.1007%2FBF02289233">subsequent paper</a>). I did find (via
<a href="http://en.wikipedia.org/wiki/Varimax_rotation">Wikipedia</a>) <a href="http://www.stat.ufl.edu/~tpark/Research/about_varimax.html">Trevor Park's notes</a> which have been
very useful.</p>
<p>The idea is to iterate rotations to maximize the raw varimax criterion
(Park's eq. 1):</p>
<div class="numberedEq"><span>(10)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mi>V</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></munderover><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msubsup><mi>A</mi> <mi>ij</mi> <mn>4</mn></msubsup><mo>−</mo><msup><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msubsup><mi>A</mi> <mi>ij</mi> <mn>4</mn></msubsup><mo>)</mo></mrow> <mn>2</mn></msup><mo>)</mo></mrow></math></div>
<p>Rather than computing a <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi></math>-dimensional rotation in one sweep, we'll
iterate through 2-dimensional rotations (on successive column pairs)
until convergence. For a particular column pair <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo stretchy="false">(</mo><mi>p</mi><mo>,</mo><mi>q</mi><mo stretchy="false">)</mo></math>, the
rotation matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>R</mi> <mo>*</mo></msup></math> is the usual rotation matrix:</p>
<div class="numberedEq"><span>(11)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><msup><mi>R</mi> <mo>*</mo></msup><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>cos</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd> <mtd><mo lspace="verythinmathspace" rspace="0em">−</mo><mi>sin</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>sin</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd> <mtd><mi>cos</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
<p>where the optimum rotation angle <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>ϕ</mi> <mo>*</mo></msup></math> is (Park's eq. 3):</p>
<div class="numberedEq"><span>(12)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><msup><mi>ϕ</mi> <mo>*</mo></msup><mo>=</mo><mfrac><mn>1</mn><mn>4</mn></mfrac><mo>∠</mo><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msup><mrow><mo>(</mo><msub><mi>A</mi> <mi>jp</mi></msub><mo>+</mo><msub><mi>iA</mi> <mi>jq</mi></msub><mo>)</mo></mrow> <mn>4</mn></msup><mo>−</mo><msup><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msup><mrow><mo>(</mo><msub><mi>A</mi> <mi>jp</mi></msub><mo>+</mo><msub><mi>iA</mi> <mi>jq</mi></msub><mo>)</mo></mrow> <mn>2</mn></msup><mo>)</mo></mrow> <mn>2</mn></msup><mo>)</mo></mrow></math></div>
<p>where <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>i</mi><mo>≡</mo><msqrt><mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msqrt></math>.</p>
<h1>Nomenclature</h1>
<dl>
<dt id="subsripts"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>A</mi> <mi>ij</mi></msub></math></dt>
<dd>The element from the <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>i</mi> <mtext>th</mtext></msup></math> row and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>j</mi> <mtext>th</mtext></msup></math>
column of a matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>. For example here is a 2-by-3
matrix terms of components:
<div class="numberedEq"><span>(13)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><msub><mi>A</mi> <mn>11</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>12</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>13</mn></msub></mtd></mtr> <mtr><mtd><msub><mi>A</mi> <mn>21</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>22</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>23</mn></msub></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
</dd>
<dt id="transpose"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup></math></dt>
<dd>The transpose of a matrix (or vector) <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>.
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msubsup><mi>A</mi> <mi>ij</mi> <mi>T</mi></msubsup><mo>=</mo><msub><mi>A</mi> <mi>ji</mi></msub></math></dd>
<dt id="inverse"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup></math></dt>
<dd>The inverse of a matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>.
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mover><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>˙</mo></mover><mo>=</mo><mn>1</mn></math></dd>
<dt id="diag"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mtext>diag</mtext><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo stretchy="false">]</mo></math></dt>
<dd>A matrix containing only the diagonal elements of
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>, with the off-diagonal values set to zero.</dd>
<dt id="expect"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></math></dt>
<dd>Expectation value for a function <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>f</mi></math> of a random variable
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math>. If the probability density of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> is
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo></math>, then <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>=</mo><mo>∫</mo><mi>d</mi><mstyle mathvariant="bold"><mi>x</mi></mstyle><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo></math>. For example,
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>=</mo><mn>1</mn></math>.</dd>
<dt id="mean"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle></math></dt>
<dd>The mean of a random variable <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> is given by
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">]</mo></math>.</dd>
<dt id="variance"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math></dt>
<dd>The covariance of a random variable <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> is given by
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><msup><mo stretchy="false">)</mo> <mstyle mathvariant="bold"><mi>T</mi></mstyle></msup><mo stretchy="false">]</mo></math>. In the factor analysis
model discussed above, <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math> is restricted to a
diagonal matrix.</dd>
<dt id="gaussian"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>x</mi></mstyle></msub><mo stretchy="false">[</mo><mi>μ</mi><mo>,</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo></math></dd>
<dd>A Gaussian probability density for the random variables
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> with a mean <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle></math> and a covariance
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math>.
<div class="numberedEq"><span>(14)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>x</mi></mstyle></msub><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mo stretchy="false">(</mo><mn>2</mn><mi>π</mi><msup><mo stretchy="false">)</mo> <mfrac><mi>D</mi><mn>2</mn></mfrac></msup><msqrt><mrow><mi>det</mi><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo></mrow></msqrt></mrow></mfrac><msup><mi>e</mi> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><msup><mo stretchy="false">)</mo> <mi>T</mi></msup><msup><mstyle mathvariant="bold"><mi>Σ</mi></mstyle> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">)</mo></mrow></msup></math></div>
</dd>
<dt id="probability-of-y-given-x"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo></math></dt>
<dd>Probability of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math> occurring given that <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math>
occured. This is commonly used in <a
href="http://en.wikipedia.org/wiki/Bayes%27_theorem">Bayesian
statistics</a>.</dd>
<dt id="probability-of-x-and-y"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo></math></dt>
<dd>Probability of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math> and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> occuring
simultaneously (the joint density).
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">∣</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo></math></dd>
<dt id="angle"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>∠</mo><mo stretchy="false">(</mo><mi>z</mi><mo stretchy="false">)</mo></math></dt>
<dd>The angle of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>z</mi></math> in the complex plane.
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>∠</mo><mo stretchy="false">(</mo><msup><mi>re</mi> <mrow><mi>i</mi><mi>θ</mi></mrow></msup><mo stretchy="false">)</mo><mo>=</mo><mi>θ</mi></math>.</dd>
</dl>
<p>Note: if you have trouble viewing some of the more obscure <a href="http://en.wikipedia.org/wiki/Unicode">Unicode</a>
used in this post, you might want to install the <a href="http://www.stixfonts.org/">STIX fonts</a>.</p>
catalysthttp://blog.tremily.us//posts/catalyst/2013-03-09T17:54:16Z2013-03-06T17:50:45Z
<p><span class="infobox">
Available in a <a href="http://blog.tremily.us//tags/teaching/../git/">git</a> repository.<br />
Repository: <a href="git://tremily.us/catalyst-swc.git" rel="vcs-git" title="catalyst-swc repository">catalyst-swc</a><br />
Browsable repository: <a href="http://git.tremily.us/?p=catalyst-swc.git" rel="vcs-git" title="catalyst-swc repository">catalyst-swc</a><br />
Author: W. Trevor King<br />
</span></p>
<p><a href="http://www.gentoo.org/proj/en/releng/catalyst/">Catalyst</a> is a release-building tool for Gentoo. If you use Gentoo
and want to roll your own live CD or bootable USB drive, this is the
way to go. As I've been wrapping my head around catalyst, I've been
<a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2006">pushing</a> <a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=0a99c494d4602ebb92f6d352f86ec519903fdc88">my</a> <a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2107">notes</a>
<a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=b7f2cf0f4c34af8d2dd9dcfa186930c17323e298">upstream</a>. This post builds on those notes to discuss the
construction of <a href="http://lists.software-carpentry.org/pipermail/discuss-software-carpentry.org/2013-March/000210.html">a bootable ISO</a> for <a href="http://software-carpentry.org/">Software Carpentry</a>
boot camps.</p>
<h1>Getting a patched up catalyst</h1>
<p>Catalyst has been around for a while, but the user base has been
fairly small. If you try to do something that <a href="http://www.gentoo.org/proj/en/releng/">Gentoo's Release
Engineering team</a> doesn't do on a regular basis, built in
catalyst support can be spotty. There's been a fair amount of patch
submissions an <a href="http://www.gentoo.org/main/en/lists.xml#doc_chap1_sect2">gentoo-catalyst@</a> recently, but patch acceptance can
be slow. For the SWC ISO, I applied versions of the following patches
(or patch series) to <a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=37540ff5e9db28e65a4f55b71c16cec1e7d294ec">37540ff</a>:</p>
<ul>
<li><a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2085/focus=2093">chmod +x all sh scripts so they can run from the git checkout</a></li>
<li><a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2182">livecdfs-update.sh: Set XSESSION in /etc/env.d/90xsession</a></li>
<li><a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2187/focus=2191">Fix livecdfs-update.sh startx handling</a></li>
</ul>
<h1>Configuring catalyst</h1>
<p>The easiest way to run catalyst from a Git checkout is to setup a
<a href="http://article.gmane.org/gmane.linux.gentoo.catalyst/2111">local config file</a>. I didn't have enough hard drive space on
my local system (~16 GB) for this build, so I set things up in a
temporary directory on an external hard drive:</p>
<pre><code>$ cat catalyst.conf | grep -v '^#\|^$'
digests="md5 sha1 sha512 whirlpool"
contents="auto"
distdir="/usr/portage/distfiles"
envscript="/etc/catalyst/catalystrc"
hash_function="crc32"
options="autoresume kerncache pkgcache seedcache snapcache"
portdir="/usr/portage"
sharedir="/home/wking/src/catalyst"
snapshot_cache="/mnt/d/tmp/catalyst/snapshot_cache"
storedir="/mnt/d/tmp/catalyst"
</code></pre>
<p>I used the default values for everything except <code>sharedir</code>,
<code>snapshot_cache</code>, and <code>storedir</code>. Then I cloned the <code>catalyst-swc</code>
repository into <code>/mnt/d/tmp/catalyst</code>.</p>
<h1>Portage snapshot and a seed stage</h1>
<p>Take a snapshot of the current Portage tree:</p>
<pre><code># catalyst -c catalyst.conf --snapshot 20130208
</code></pre>
<p>Download a seed stage3 from a <a href="http://www.gentoo.org/main/en/mirrors.xml">Gentoo mirror</a>:</p>
<pre><code># wget -O /mnt/d/tmp/catalyst/builds/default/stage3-i686-20121213.tar.bz2 \
> http://distfiles.gentoo.org/releases/x86/current-stage3/stage3-i686-20121213.tar.bz2
</code></pre>
<h1>Building the live CD</h1>
<pre><code># catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-stage1-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-stage2-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-stage3-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-livecd-stage1-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-livecd-stage2-i686-2013.1.spec
</code></pre>
<h1>isohybrid</h1>
<p>To make the ISO bootable from a USB drive, I used <a href="http://www.syslinux.org/wiki/index.php/Doc/isolinux#HYBRID_CD-ROM.2FHARD_DISK_MODE">isohybrid</a>:</p>
<pre><code># cp swc-x86.iso swc-x86-isohybrid.iso
# isohybrid iso-x86-isohybrid.iso
</code></pre>
<p>You can install the resulting ISO on a USB drive with:</p>
<pre><code># dd if=iso-x86-isohybrid.iso of=/dev/sdX
</code></pre>
<p>replacing replacing <code>X</code> with the appropriate drive letter for your USB
drive.</p>
<p>With versions of catalyst after <a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=d1c2ba9bdaf1a69078fcbcd6ec4210e829f99f0a">d1c2ba9</a>, the <code>isohybrid</code> call is
built into catalysts ISO construction.</p>
SymPyhttp://blog.tremily.us//posts/SymPy/2013-02-27T13:47:15Z2013-02-27T12:52:03Z
<p><a href="http://sympy.org/">SymPy</a> is a <a href="http://blog.tremily.us//tags/teaching/../../posts/Python/">Python</a> library for symbolic mathematics. To give
you a feel for how it works, lets extrapolate the extremum location
for <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> given a quadratic model:</p>
<div class="numberedEq"><span>(1)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mi>A</mi><msup><mi>x</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>x</mi><mo>+</mo><mi>C</mi></math></div>
<p>and three known values:</p>
<div class="numberedEq"><span>(2)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></mtd> <mtd><mo>=</mo><mi>A</mi><msup><mi>a</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>a</mi><mo>+</mo><mi>C</mi></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo></mtd> <mtd><mo>=</mo><mi>A</mi><msup><mi>b</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>b</mi><mo>+</mo><mi>C</mi></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mtd> <mtd><mo>=</mo><mi>A</mi><msup><mi>c</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>c</mi><mo>+</mo><mi>C</mi></mtd></mtr></mtable></mrow></math></div>
<p>Rephrase as a matrix equation:</p>
<div class="numberedEq"><span>(3)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><msup><mi>a</mi> <mn>2</mn></msup></mtd> <mtd><mi>a</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>b</mi> <mn>2</mn></msup></mtd> <mtd><mi>b</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>c</mi> <mn>2</mn></msup></mtd> <mtd><mi>c</mi></mtd> <mtd><mn>1</mn></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>⋅</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>A</mi></mtd></mtr> <mtr><mtd><mi>B</mi></mtd></mtr> <mtr><mtd><mi>C</mi></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
<p>So the solutions for <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>A</mi></math>, <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>B</mi></math>, and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>C</mi></math> are:</p>
<div class="numberedEq"><span>(4)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>A</mi></mtd></mtr> <mtr><mtd><mi>B</mi></mtd></mtr> <mtr><mtd><mi>C</mi></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>=</mo><msup><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><msup><mi>a</mi> <mn>2</mn></msup></mtd> <mtd><mi>a</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>b</mi> <mn>2</mn></msup></mtd> <mtd><mi>b</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>c</mi> <mn>2</mn></msup></mtd> <mtd><mi>c</mi></mtd> <mtd><mn>1</mn></mtd></mtr></mtable></mrow><mo>)</mo></mrow> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mo>⋅</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mtext>long</mtext></mtd></mtr> <mtr><mtd><mtext>complicated</mtext></mtd></mtr> <mtr><mtd><mtext>stuff</mtext></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
<p>Now that we've found the model parameters, we need to find the <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>x</mi></math>
coordinate of the extremum.</p>
<div class="numberedEq"><span>(5)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mfrac><mrow><mi mathvariant="normal">d</mi><mi>f</mi></mrow><mrow><mi mathvariant="normal">d</mi><mi>x</mi></mrow></mfrac><mo>=</mo><mn>2</mn><mi>A</mi><mi>x</mi><mo>+</mo><mi>B</mi><mspace width="thickmathspace"></mspace><mo>,</mo></math></div>
<p>which is zero when</p>
<div class="numberedEq"><span>(6)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mn>2</mn><mi>A</mi><mi>x</mi></mtd> <mtd><mo>=</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mi>B</mi></mtd></mtr> <mtr><mtd><mi>x</mi></mtd> <mtd><mo>=</mo><mfrac><mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mi>B</mi></mrow><mrow><mn>2</mn><mi>A</mi></mrow></mfrac></mtd></mtr></mtable></mrow></math></div>
<p>Here's the solution in SymPy:</p>
<pre><code>>>> from sympy import Symbol, Matrix, factor, expand, pprint, preview
>>> a = Symbol('a')
>>> b = Symbol('b')
>>> c = Symbol('c')
>>> fa = Symbol('fa')
>>> fb = Symbol('fb')
>>> fc = Symbol('fc')
>>> M = Matrix([[a**2, a, 1], [b**2, b, 1], [c**2, c, 1]])
>>> F = Matrix([[fa],[fb],[fc]])
>>> ABC = M.inv() * F
>>> A = ABC[0,0]
>>> B = ABC[1,0]
>>> x = -B/(2*A)
>>> x = factor(expand(x))
>>> pprint(x)
2 2 2 2 2 2
a *fb - a *fc - b *fa + b *fc + c *fa - c *fb
---------------------------------------------
2*(a*fb - a*fc - b*fa + b*fc + c*fa - c*fb)
>>> preview(x, viewer='pqiv')
</code></pre>
<p>Where <code>pqiv</code> is the executable for <a href="http://www.pberndt.com/Programme/Linux/pqiv/">pqiv</a>, my preferred image
viewer. With a bit of additional factoring, that is:</p>
<div class="numberedEq"><span>(7)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mi>x</mi><mo>=</mo><mfrac><mrow><msup><mi>a</mi> <mn>2</mn></msup><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><msup><mi>b</mi> <mn>2</mn></msup><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><msup><mi>c</mi> <mn>2</mn></msup><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mn>2</mn><mo>⋅</mo><mo stretchy="false">{</mo><mi>a</mi><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><mi>b</mi><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><mi>c</mi><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo stretchy="false">}</mo></mrow></mfrac></math></div>
Open physics texthttp://blog.tremily.us//posts/Open_physics_text/2013-05-31T19:13:43Z2011-05-04T00:36:24Z
<p>Since I love both teaching and open source development, I suppose it
was only a matter of time before I attempted a survey of open source
text books. Here are my notes on the projects I've come across so
far:</p>
<h1>Light and Matter</h1>
<p>The <a href="http://www.lightandmatter.com/">Light and Matter</a> series is a set of six texts by Benjamin
Crowell at Fullerton College in California. The series is aimed at
the High School and Biology (i.e. low calc) audience. The source is
distributed in <a href="http://blog.tremily.us//tags/teaching/../../posts/LaTeX/">LaTeX</a> and versioned in <a href="http://blog.tremily.us//tags/teaching/../../posts/Git/">Git</a>. I love this guy!</p>
<p>Crowell also runs a book review site <a href="http://theassayer.org/">The Assayer</a>, which
reviews free books.</p>
<h1>Radically Modern Introductory Physics</h1>
<p><a href="http://kestrel.nmt.edu/~raymond/teaching.html">Radically Modern Introductory Physics</a> is David J. Raymond's
modern-physics-based approach to introductory physics. He posts the
<a href="http://blog.tremily.us//tags/teaching/../../posts/LaTeX/">LaTeX</a> source, but it does not seem to be version controlled.</p>
<h1>Calculus Based Physics</h1>
<p><a href="http://www.anselm.edu/internet/physics/cbphysics/">Calculus Based Physics</a>, by Jeffrey W. Schnick at St. Anselm in
New Hampshire. It is under the Creative Commons
Attribution-ShareAlike 3.0 License, and the sources are free to alter.
However, there is no official version control, and the sources are in
MS Word format :(. On the other hand, I wholeheartedly agree with all
the objectives Schnick lists in <a href="http://www.anselm.edu/internet/physics/cbphysics/AboutTheBook/WHY.pdf">his motivational note</a>.</p>
<h1>Textbook Revolution</h1>
<p>Calculus Based Physics' Schnick linked to <a href="http://textbookrevolution.org/index.php/Main_Page">Textbook Revolution</a>,
which immediately gave off good tech vibes with an IRC node
(#textbookrevolution). The site is basically a wiki with a browsable
list of pointers to open textbooks. The list isn't huge, but it does
prominently display copyright information, which makes it easier to
separate the wheat from the chaff.</p>
<h1>College Open Textbooks</h1>
<p><a href="http://www.collegeopentextbooks.org/">College Open Textbooks</a> provides another <a href="http://www.collegeopentextbooks.org/opentextbookcontent/open-textbooks-by-subject/physics.html">registry of open
textbooks</a> with clearly listed license information.
They're funded by The William and Flora Hewlett Foundation (of NPR
underwriting fame).</p>
<h1>MERLOT's Open Textbook Initiative</h1>
<p>The Multimedia Educational Resource for Learning and Online Teaching
(<a href="http://www.merlot.org/">MERLOT</a>) is a California-based project that assembles educational
resources. They have <a href="http://www.merlot.org/merlot/materials.htm?category=2736&&materialType=Open%20Textbook&sort.property=overallRating">a large collection of open
textbooks</a> in a variety of fields. The Light and
Matter series is well represented. Unfortunately, many of the texts
seem to be "free as in beer" not "free as in freedom".</p>
<h1>Open Access Textbooks</h1>
<p><a href="http://www.openaccesstextbooks.org/">The Open Access Textbooks</a> project is run by a number of
Florida-based groups and funded by the U.S. Department of Education.
However, I have grave doubts about any open source project that opens
their <a href="http://www.openaccesstextbooks.org/projectInfo.html">project discussion</a> with</p>
<blockquote>
<p>Numerous issues that impact open textbook implementation (such as
creating sustainable review processes and institutional reward
structures) have yet to be resolved. The ability to financially
sustain a large scale open textbook effort is also in question.</p>
</blockquote>
<p>There are zounds of academics with enough knowledge and invested
interest in developing an open source textbook. The resources
(computers and personal websites) are generally already provided by
academic institutions. Just pick a framework (LaTeX, HTML, ...), put
the whole thing in Git, and start hacking. The community will take it
from there.</p>
<p>Anyhow, everything I've read about this project smells like a bunch of
bureaucrats churning out sound bytes.</p>
<h1>ArXiv</h1>
<p>Finally, there are a number of textbooks on <a href="http://arxiv.org/">arXiv</a>. For example,
Siegel's <a href="http://arxiv.org/abs/hep-th/0107094">Introduction to string field theory</a> and <a href="http://arxiv.org/abs/hep-th/9912205">Fields</a>
are posted source and all. The source will probably be good quality,
but the licensing information may be unclear.</p>
Parallel computinghttp://blog.tremily.us//posts/Parallel_computing/2011-12-15T18:57:28Z2010-10-14T12:59:13Z
<p><span class="infobox">
Available in a <a href="http://blog.tremily.us//tags/teaching/../git/">git</a> repository.<br />
Repository: <a href="git://tremily.us/parallel_computing.git" rel="vcs-git" title="parallel_computing repository">parallel_computing</a><br />
Browsable repository: <a href="http://git.tremily.us/?p=parallel_computing.git" rel="vcs-git" title="parallel_computing repository">parallel_computing</a><br />
Author: W. Trevor King<br />
</span></p>
<p>In contrast to my <a href="http://blog.tremily.us//tags/teaching/../../posts/Course_website/">course website</a> project, which is mostly about
constructing a framework for automatically compiling and installing
<a href="http://blog.tremily.us//tags/teaching/../../posts/LaTeX/">LaTeX</a> problem sets, <a href="http://www.drexel.edu/physics/contact/facultyDirectory/Vallieres%20Michel/">Prof. Vallières'</a> <a href="http://www.physics.drexel.edu/~valliere/PHYS405/">Parallel
Computing</a> course is basically an online textbook with a
large amount of example software. In order to balance between to
Prof. Vallières' original and my own aesthetic, I rolled a new
solution from scratch. See <a href="http://course.tremily.us/2010-fall/phys405/">my version of his Fall 2010 page</a>
for a live example.</p>
<p>Differences from my course website project:</p>
<ul>
<li>No PHP, since there is no dynamic content that cannot be handled
with <a href="http://blog.tremily.us//tags/teaching/../../posts/SSI/">SSI</a>.</li>
<li>Less installation machinery. Only a few build/cleanup scripts to
avoid versioning really tedious bits. The repository is designed to
be dropped into your <code>~/public_html/</code> whole, while the course
website project is designed to <code>rsync</code> the built components up as
they go live.</li>
<li>Less LaTeX, more XHTML. It's easier to edit XHTML than it is to
exit and compile LaTeX, and PDFs are large and annoying. As a
computing class, there are fewer graphics than there are in an
intro-physics class, so the extra power of LaTeX is not as useful.</li>
</ul>
Course websitehttp://blog.tremily.us//posts/Course_website/2012-06-26T18:25:34Z2010-10-14T12:55:17Z
<p><span class="infobox">
Available in a <a href="http://blog.tremily.us//tags/teaching/../git/">git</a> repository.<br />
Repository: <a href="git://tremily.us/course.git" rel="vcs-git" title="course repository">course</a><br />
Browsable repository: <a href="http://git.tremily.us/?p=course.git" rel="vcs-git" title="course repository">course</a><br />
Author: W. Trevor King<br />
</span></p>
<p>Over a few years as a TA for assorted introductory physics classes,
I've assembled a nice website framework with lots of problems using my
<a href="http://blog.tremily.us//tags/teaching/../../posts/LaTeX/">LaTeX</a> <a href="http://blog.tremily.us//tags/teaching/../../posts/problempack/">problempack</a> package, along with some handy <code>Makefiles</code>,
a bit of <span class="createlink">php</span>, and <a href="http://blog.tremily.us//tags/teaching/../../posts/SSI/">SSI</a>.</p>
<p>The result is the <code>course</code> package, which should make it very easy to
whip up a course website, homeworks, etc. for an introductory
mechanics or E&M class (431 problems implemented as of June 2012).
With a bit of work to write up problems, the framework could easily be
extended to other subjects.</p>
<p>The idea is that a course website consists of a small, static HTML
framework, and a bunch of content that is gradually filled in as the
semester/quarter progresses. I've put the HTML framework in the
<code>html/</code> directory, along with some of the write-once-per-course
content (e.g. Prof & TA info). See <code>html/README</code> for more information
on the layout of the HTML.</p>
<p>The rest of the directories contain the code for compiling material
that is deployed as the course progresses. The <code>announcements/</code>
directory contains the atom feed for the course, and possibly a list
of email addresses of people who would like to (or should) be notified
when new announcements are posted. The <code>latex/</code> directory contains
LaTeX source for the course documents for which it is available, and
the <code>pdf/</code> directory contains PDFs for which no other source is
available (e.g. scans, or PDFs sent in by Profs or TAs who neglected
to include their source code).</p>
<p>Note that because this framework assumes the HTML content will be
relatively static, it may not be appropriate for courses with large
amounts of textbook-style content, which will undergo more frequent
revision. It may also be excessive for courses that need less
compiled content. For an example of another framework, see my
<a href="http://blog.tremily.us//tags/teaching/../../posts/Parallel_computing/">branch</a> of <a href="http://drexel.edu/physics/contact/facultyDirectory/Vallieres%20Michel/">Prof. Vallières'</a> <a href="http://www.physics.drexel.edu/~valliere/PHYS405/">Parallel
Computing</a> website.</p>
problempackhttp://blog.tremily.us//posts/problempack/2011-12-15T18:57:28Z2010-10-14T12:50:54Z
<p><span class="infobox">
Available in a <a href="http://blog.tremily.us//tags/teaching/../git/">git</a> repository.<br />
Repository: <a href="git://tremily.us/problempack.git" rel="vcs-git" title="problempack repository">problempack</a><br />
Browsable repository: <a href="http://git.tremily.us/?p=problempack.git" rel="vcs-git" title="problempack repository">problempack</a><br />
Author: W. Trevor King<br />
</span></p>
<p>I've put together a <a href="http://blog.tremily.us//tags/teaching/../../posts/LaTeX/">LaTeX</a> package <code>problempack</code> to make it easier
to write up problem sets with solutions for the classes I TA.</p>
<h2>problempack.sty</h2>
<p>The package takes care of a few details:</p>
<ul>
<li>Make it easy to compile one pdf with only the problems and another
pdf with problems and solutions.</li>
<li>Define nicely typeset environments for automatically or manually
numbered problems.</li>
<li>Save retyping a few of the parameters (course title, class title,
etc), that show up in the note title and also need to go out to
<code>pdftitle</code> and <code>pdfsubject</code>.</li>
<li>Change the page layout to minimize margins (saves paper on
printing).</li>
<li>Set the spacing between problems (e.g. to tweak output to a single
page, versions >= 0.2).</li>
<li>Add section level entries to the table-of-contents and hyperref
bookmarks (versions >= 0.3).</li>
</ul>
<p>The basic idea is to make it easy to write up notes. Just install
<code>problempack.sty</code> in your <code>texmf</code> tree, and then use it like I do in
the example included in the package. The example produces a simple
problem set (<span class="createlink">probs.pdf</span>) and solution notes (<span class="createlink">sols.pdf</span>).</p>
<p>For a real world example, look at my Phys 102 notes <a href="http://course.tremily.us/2008-spring-phys102/notes/rec1_sols.pdf">with</a> and
<a href="http://course.tremily.us/2008-spring-phys102/notes/rec1_probs.pdf">without</a> solutions (<a href="http://course.tremily.us/2008-spring-phys102/latex/notes/rec1/">source</a>). Other notes produced in this
fashion: <a href="http://course.tremily.us/2009-winter-phys201/source/">Phys201 winter 2009</a>, <a href="http://http://course.tremily.us/2009-spring-phys201/source/">Phys201 spring 2009</a>, and
<a href="http://http://course.tremily.us/2009-summer-phys102/source/">Phys102 summer 2009</a>.</p>
<h2>wtk_cmmds.sty</h2>
<p>A related package that defines some useful physics macros (<code>\U</code>, <code>\E</code>,
<code>\dg</code>, <code>\vect</code>, <code>\ihat</code>, ...) is my <code>wtk_cmmds.sty</code>. This used to be a
part of <code>problempack.sty</code>, but the commands are less general, so I
split them out into their own package.</p>
<h2>wtk_format.sty</h2>
<p>The final package in the <code>problempack</code> repository is <code>wtk_format.sty</code>,
which adjusts the default LaTeX margins to pack more content into a
single page.</p>
Mathhttp://blog.tremily.us//posts/Math/2010-11-17T14:23:57Z2008-08-07T13:13:59Z
<p>I've had a few students confused by this sort of "<a href="http://www.abstractmath.org/MM/MMZoomChunk.htm">zooming and
chunking</a>" approach to
analyzing functions specifically, and technical problems in general,
so I'll pass the link on in case you're interested. Curtesy of
Charles Wells.</p>