unfolding disastersunfolding disastershttp://blog.tremily.us//unfolding disastersikiwiki2013-09-20T04:04:58ZNodehttp://blog.tremily.us//posts/Node/2013-09-20T04:04:58Z2013-09-20T04:04:58Z
<p><a href="http://nodejs.org/">Node</a> is a server-side <a href="http://en.wikipedia.org/wiki/JavaScript">JavaScript</a> engine (i.e. it executes
JavaScript without using a browser). This means that JavaScript
developers can now develop tools in their native language, so it's not
a surprise that the <a href="http://getbootstrap.com/">Bootstrap</a> folks use <a href="http://gruntjs.com/">Grunt</a> for their build
system. I'm new to the whole Node ecosystem, so here are my notes on
how it works.</p>
<p>Start off by installing <a href="https://npmjs.org/">npm</a>, the Node package manager. On Gentoo,
that's:</p>
<pre><code># USE=npm emerge -av net-libs/nodejs
</code></pre>
<p>[Configure npm][npm-config] to make <a href="https://npmjs.org/doc/files/npm-folders.html#prefix-Configuration">"global"
installs</a> in your personal space:</p>
<pre><code># npm config set prefix ~/.local/
</code></pre>
<p>Install the <a href="http://gruntjs.com/">Grunt</a> command line interface for building
<a href="http://getbootstrap.com/">Bootstrap</a>:</p>
<pre><code>$ npm install -g grunt-cli
</code></pre>
<p>That installs the libraries under <code>~/.local/lib/node_modules</code> and
drops symlinks to binaries in <code>~/.local/bin</code> (which is <a href="http://git.tremily.us/?p=dotfiles-public.git;a=blob;f=src/.bash_profile;hb=HEAD">already in my
<code>PATH</code></a> thanks to my <a href="http://blog.tremily.us//./posts/dotfiles/">dotfiles</a>).</p>
<p>Clone Boostrap and <a href="https://github.com/twbs/bootstrap/blob/master/README.md#install-grunt">install it's
dependencies</a>:</p>
<pre><code>$ git clone git://github.com/twbs/bootstrap.git
$ cd bootstrap
$ npm install
</code></pre>
<p>This looks in the local [package.json][] to extract a list of
dependencies, and <a href="https://npmjs.org/doc/files/npm-folders.html">installs each of them</a> under
<a href="http://nodejs.org/api/modules.html#modules_loading_from_node_modules_folders">node_modules</a>. Node likes to isolate its packages, so every
dependency for a given package is installed underneath that package.
This leads to some crazy nesting:</p>
<pre><code>$ find node_modules/ -name graceful-fs
node_modules/grunt/node_modules/glob/node_modules/graceful-fs
node_modules/grunt/node_modules/rimraf/node_modules/graceful-fs
node_modules/grunt-contrib-clean/node_modules/rimraf/node_modules/graceful-fs
node_modules/grunt-contrib-qunit/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/node_modules/rimraf/node_modules/graceful-fs
node_modules/grunt-contrib-watch/node_modules/gaze/node_modules/globule/node_modules/glob/node_modules/graceful-fs
</code></pre>
<p>Sometimes the redundancy is due to different version requirements, but
sometimes the redundancy is just redundant :p. Let's look with <a href="https://npmjs.org/doc/cli/npm-ls.html">npm
ls</a>.</p>
<pre><code>$ npm ls graceful-fs
bootstrap@3.0.0 /home/wking/src/bootstrap
├─┬ grunt@0.4.1
│ ├─┬ glob@3.1.21
│ │ └── graceful-fs@1.2.3
│ └─┬ rimraf@2.0.3
│ └── graceful-fs@1.1.14
├─┬ grunt-contrib-clean@0.5.0
│ └─┬ rimraf@2.2.2
│ └── graceful-fs@2.0.1
├─┬ grunt-contrib-qunit@0.2.2
│ └─┬ grunt-lib-phantomjs@0.3.1
│ └─┬ phantomjs@1.9.2-1
│ └─┬ rimraf@2.0.3
│ └── graceful-fs@1.1.14
└─┬ grunt-contrib-watch@0.5.3
└─┬ gaze@0.4.1
└─┬ globule@0.1.0
└─┬ glob@3.1.21
└── graceful-fs@1.2.3
</code></pre>
<p>Regardless of on-disk duplication, Node <a href="http://nodejs.org/api/modules.html#modules_caching">caches modules</a> so a
given module only loads once. If it really bothers you, you can
<a href="https://npmjs.org/doc/files/npm-folders.html#Cycles-Conflicts-and-Folder-Parsimony">avoid some duplicates by installing duplicated packages higher up in
the local tree</a>:</p>
<pre><code>$ rm -rf node_modules
$ npm install graceful-fs@1.1.14
$ npm install
$ npm ls graceful-fs
bootstrap@3.0.0 /home/wking/src/bootstrap
├── graceful-fs@1.1.14 extraneous
├─┬ grunt@0.4.1
│ └─┬ glob@3.1.21
│ └── graceful-fs@1.2.3
├─┬ grunt-contrib-clean@0.5.0
│ └─┬ rimraf@2.2.2
│ └── graceful-fs@2.0.1
└─┬ grunt-contrib-watch@0.5.3
└─┬ gaze@0.4.1
└─┬ globule@0.1.0
└─┬ glob@3.1.21
└── graceful-fs@1.2.3
</code></pre>
<p>This is probably not worth the trouble.</p>
<p>Now that we have Grunt and the Bootstrap dependencies, we can build
the distributed libraries:</p>
<pre><code>$ ~/src/node_modules/.bin/grunt dist
Running "clean:dist" (clean) task
Cleaning dist...OK
Running "recess:bootstrap" (recess) task
File "dist/css/bootstrap.css" created.
Running "recess:min" (recess) task
File "dist/css/bootstrap.min.css" created.
Original: 121876 bytes.
Minified: 99741 bytes.
Running "recess:theme" (recess) task
File "dist/css/bootstrap-theme.css" created.
Running "recess:theme_min" (recess) task
File "dist/css/bootstrap-theme.min.css" created.
Original: 18956 bytes.
Minified: 17003 bytes.
Running "copy:fonts" (copy) task
Copied 4 files
Running "concat:bootstrap" (concat) task
File "dist/js/bootstrap.js" created.
Running "uglify:bootstrap" (uglify) task
File "dist/js/bootstrap.min.js" created.
Original: 58543 bytes.
Minified: 27811 bytes.
Done, without errors.
</code></pre>
<p>Wohoo!</p>
<p>Unfortunately, like all <a href="https://pypi.python.org/pypi">language-specific</a> <a href="http://rubygems.org/">packing</a>
<a href="http://www.cpan.org/">systems</a>, npm has trouble installing packages that aren't
written in its native language. This means you get things like:</p>
<pre><code>$ ~/src/node_modules/.bin/grunt
…
Running "jekyll:docs" (jekyll) task
`jekyll build` was initiated.
Jekyll output:
Warning: Command failed: /bin/sh: jekyll: command not found
Use --force to continue.
Aborted due to warnings.
</code></pre>
<p>Once everybody wises up and starts writing packages for <a href="http://blog.tremily.us//./posts/Gentoo_Prefix_overlay/">Gentoo
Prefix</a>, we can stop worrying about
installation and get back to work developing :p.</p>
Comcasthttp://blog.tremily.us//posts/Comcast/2013-09-20T04:04:58Z2013-09-20T04:04:58Z
<p>A while back I <a href="http://blog.tremily.us//./posts/Comcast_rediculousness/">posted about Comcast blocking outgoing traffic on
port 25</a>. We've spent some time with
Verizon's DSL service, but after our recent move we're back with
Comcast. Luckily, Comcast now explicitly lists <a href="http://customer.comcast.com/help-and-support/internet/list-of-blocked-ports/">the ports they
block</a>. Nothing I care about, except for port 25 (incoming and
outgoing). For incoming mail, I use <a href="http://dyn.com/">Dyn</a> to forward mail to <a href="http://blog.tremily.us//./posts/Postfix/">port
587</a>. For outgoing mail, I had been using <span class="createlink">stunnel</span>
through <code>outgoing.verizon.net</code> for my <a href="http://blog.tremily.us//./posts/SMTP/">SMTP</a> connections. Comcast
takes a <a href="http://customer.comcast.com/help-and-support/internet/email-client-programs-with-xfinity-email/">similar approach</a> forcing outgoing mail through port
465 on <code>smtp.comcast.net</code>.</p>
Relative submoduleshttp://blog.tremily.us//posts/Relative_submodules/2013-07-13T15:03:02Z2013-07-13T15:03:02Z
<p>I like <a href="http://blog.tremily.us//./posts/Git/">Git</a> <a href="http://git-scm.com/docs/git-submodule">submodules</a> quite a bit, but they often get a <a href="http://git-scm.com/book/en/Git-Tools-Submodules#Issues-with-Submodules">bad
rap</a>. Most of the problems involve bad git hygiene (e.g. not
developing in feature branches) or limitations in the current
submodule implementation (e.g. it's hard to move submodules). Other
problems involve <a href="http://thread.gmane.org/gmane.comp.python.ipython.devel/10287">not being able to fetch submodules with <code>git://</code>
URLs</a> (due to restrictive firewalls).</p>
<p>This last case is easily solved by using relative submodule URLs in
<code>.gitmodules</code>. I've been through the relative-vs.-absolute URL
argument <a href="https://github.com/inducer/pycuda/commit/6ba1a47802ec896c6eccf3cebf1f9863989aeb4d">a</a> <a href="http://thread.gmane.org/gmane.comp.python.ipython.devel/10287/focus=10290">few</a> <a href="http://thread.gmane.org/gmane.comp.python.ipython.devel/10304">times</a> now, so I
thought I'd write up my position for future reference. I prefer the
relative URL in:</p>
<pre><code>[submodule "some-name"]
path = some/path
url = ../submod-repo.git
</code></pre>
<p>to the absolute URL in:</p>
<pre><code>[submodule "some-name"]
path = some/path
url = git://example.net/submod-repo.git
</code></pre>
<p>Arguments in favor of relative URLs:</p>
<ul>
<li>Users get submodules over their preferred transport (<code>ssh://</code>,
<code>git://</code>, <code>https://</code>, …). Whatever transport you used to clone the
superproject will be recycled when you use <code>submodule init</code> to set
submodule URLs in your <code>.git/config</code>.</li>
<li>No need to tweak <code>.gitmodules</code> if you mirror (or move) your
superproject Git hosting somewhere else (e.g. from <code>example.net</code> to
<code>elsewhere.com</code>).</li>
<li>As a special case of the mirror/move situation, there's no need to
tweak <code>.gitmodules</code> in long-term forks. If I setup a local version
of the project and host it on my local box, my lab-mates can clone
my local superproject and use my local submodules without my having
to alter <code>.gitmodules</code>. Reducing trivial differences between forks
makes collaboration on substantive changes more likely.</li>
</ul>
<p>The only argument I've heard in favor of absolute URLs is <a href="http://thread.gmane.org/gmane.comp.python.ipython.devel/10304">Brian
Granger's</a> GitHub workflow:</p>
<ul>
<li>If a user forks <code>upstream/repo</code> to <code>username/repo</code> and then clones
their fork for local work, relative submodule URLs will not work
until they also fork the submodules into <code>username/</code>.</li>
</ul>
<p>This workflow needs absolute URLs:</p>
<p><object type="image/svg+xml" data="http://blog.tremily.us//fork.svg"></object></p>
<p>But relative URLs are fine if you also fork the submodule(s):</p>
<p><object type="image/svg+xml" data="http://blog.tremily.us//mirror.svg"></object></p>
<p>Personally, I only create a public repository (<code>username/repo</code>) after
cloning the central repository (<code>upstream/repo</code>). Several projects I
contribute too (such as <a href="https://github.com/gitster/git/blob/master/Documentation/SubmittingPatches#L123">Git itself</a>) prefer changes via
<a href="http://git-scm.com/docs/git-send-email">send-email</a>, in which case there is no need for contributors to
create public repositories at all. Relative URLs are also fine here:</p>
<p><object type="image/svg+xml" data="http://blog.tremily.us//clone.svg"></object></p>
<p>Once you understand the trade-offs, picking absolute/relative is just
a political/engineering decision. I don't see any benefit to the
absolute-URL-only repo relationship, so I favor relative URLs. The
<a href="http://article.gmane.org/gmane.comp.python.ipython.devel/10307">IPython folks</a> felt that too many devs already used the
absolute-URL-only relationship, and that the relative-URL benefits
were not worth the cost of retraining those developers.
`</p>
Open hardware analog I/Ohttp://blog.tremily.us//posts/Open_hardware_analog_IO/2013-06-19T18:40:34Z2013-06-19T13:06:57Z
<p>Over at <a href="http://software-carpentry.org/blog/2013/06/twelve-bar-blues-of-science.html">Software Carpentry</a>, Greg Wilson just posted <a href="http://software-carpentry.org/blog/2013/06/twelve-bar-blues-of-science.html">some
thoughts about a hypothetical open science framework</a>. He uses
Ruby on Rails and similar web frameworks as examples where frameworks
can leverage standards and conventions to take care of most of the
boring boilerplate that has to happen for serving a website. Greg
points out that it would be useful to have a similar open science
framework that small projects could use to get off the ground and
collaborate more easily.</p>
<p>My <a href="http://blog.tremily.us//./posts/Thesis/">thesis</a> is about developing an <a href="http://blog.tremily.us//./posts/Open_source_force_spectroscopy/">open source framework
for single molecule force
spectroscopy</a>, so this is an avenue
I'm very excited about. However, it's difficult to get this working
for experimental labs with a diversity of the underlying hardware. If
different labs have different hardware, it's hard to write a generic
software stack that works for everybody (at least at the lower levels
of the stack). Our lab does analog control and aquisition via an old
National Instruments card. NI no longer sells this card, and
developing <a href="http://blog.tremily.us//./posts/Comedi/">Comedi</a> drivers for new cards is too much work for many
to take on pro bono. This means that new labs that want to use my
software can't get started with off the shelf components; they'll need
to find a second-hand card or rework the lower layers of my stack to
work with a DAQ card that they can source.</p>
<p>I'd be happy to see an inexpensive, microprocessor-based open hardware
project for synchronized, multi-channel, near-MHz analog I/O to serve
as a standard interface between software and the real world, but
that's not the sort of thing I can whip out over a free weekend
(although I have <a href="http://blog.tremily.us//./posts/AVR/">dipped my toe in the water</a>). I think the
missing component is a client-side version of <a href="http://www.libusb.org/">libusb</a>, to allow
folks to write the firmware for the microprocessor without dealing
with the intricacies of the <a href="http://www.usb.org/developers/docs/">USB specs</a>. It would also be nice
to have a standard USB protocol for Comedi commands, so a single
driver could interface with commodity DAQ hardware—much like the
current situation for <a href="http://www.usb.org/developers/devclass_docs#approved">mice, keyboards, webcams, and other approved
classes</a>. Then the software stack could work unchanged on
any hardware, once the firmware supporting the hardware had been
ported to a new microprocessor. There are two existing classes (a
<a href="http://www.usb.org/developers/devclass_docs/pid1_01.pdf">physical interface device class</a> and a <a href="http://www.usb.org/developers/devclass_docs/USBTMC_1_006a.zip">test and measurement
class</a>), but I haven't had time to dig through those with an eye
toward Comedi integration yet. So much to do, so little time…</p>
NOAA chartshttp://blog.tremily.us//posts/NOAA_charts/2013-06-11T00:50:13Z2013-06-11T00:50:13Z
<p>NOAA has a <a href="http://historicalcharts.noaa.gov/">Historical Map & Chart Collection</a> with free
downloads of high resolution scans of all sorts of historical maps and
charts. For example, <a href="http://historicalcharts.noaa.gov/historicals/preview/image/VAN520">George Vancouver's 1791 “Chart Shewing part of
the Coast of N.W. America”</a>.</p>
Package managementhttp://blog.tremily.us//posts/Package_management/2013-05-31T19:13:43Z2013-05-31T19:02:36Z
<p>Lex Nederbragt posted <a href="http://lists.software-carpentry.org/pipermail/discuss-software-carpentry.org/2013-May/000529.html">a question about version control and
provenance</a> on the <a href="http://software-carpentry.org/">Software Carpentry</a> <a href="http://lists.software-carpentry.org/listinfo.cgi/discuss-software-carpentry.org">discussion
list</a>. I responded with my <a href="http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1">Portage</a>-based
<a href="http://lists.software-carpentry.org/pipermail/discuss-software-carpentry.org/2013-May/000533.html">workflow</a>, but C. Titus Brown <a href="http://lists.software-carpentry.org/pipermail/discuss-software-carpentry.org/2013-May/000534.html">pointed out a number of reasons
why this approach isn't more widely used</a>, which seem to boil
down to “that sounds like more trouble than it's worth”. Because
recording the state of a system is important for <a href="http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=4720211">reproducible
research</a>, it is worth doing <em>something</em> to clean up the current
seat-of-the-pants approach.</p>
<p>Figuring out what software you have intalled on your system is
actually a (mostly) solved problem. There is a long history in the
Linux ecosystem for <a href="http://en.wikipedia.org/wiki/Package_management_system">package management systems</a> that track
installed packages and install new software (and any dependencies)
automatically. Unfortunately, there is not a consensus package
manager across distributions, with <a href="http://www.debian.org/">Debian</a>-based distributions
using <a href="http://wiki.debian.org/Apt">apt</a>, <a href="http://fedoraproject.org/">Fedora</a>-based distributions using <a href="http://yum.baseurl.org/">yum</a>, …. If
you are not the system administrator for your computer, you can either
talk your sysadmin into installing the packages you need, or use one
of a number of guest package managers (<a href="http://www.gentoo.org/proj/en/gentoo-alt/prefix/">Gentoo Prefix</a>,
<a href="http://mxcl.github.io/homebrew/">homebrew</a>, …). The guest package managers also work if you're
committed to an OS that doesn't have an existing native package
manager.</p>
<p>Despite the existence of many high quality package managers, I know
many people who continue to install significant amounts of software by
hand. While this is sustainable for a handful of packages, I see no
reason to struggle through manual installations (subsequent upgrades,
dependencies, …) when existing tools can automate the procedure. A
stopgap solution is to use language specific package managers (<a href="http://www.pip-installer.org/en/latest/">pip</a>
for <a href="http://blog.tremily.us//./posts/Python/">Python</a>, <a href="http://rubygems.org/">gem</a> for <a href="http://www.ruby-lang.org/">Ruby</a>, …). This works fairly well, but
once you reach a certain level of complexity (e.g. integrating
<a href="http://en.wikipedia.org/wiki/Fortran">Fortran</a> and <a href="http://blog.tremily.us//./posts/C/">C</a> extensions with <a href="http://blog.tremily.us//./posts/Python/">Python</a> in <a href="http://www.scipy.org/">SciPy</a>), things
<a href="http://www.scipy.org/Installing_SciPy/BuildingGeneral#head-fe5a03a1d9ad7414becca62672c1316dc91b5f88">get difficult</a>. While language-specific packaging
standards ease automation, they are not a substitute for a
language-agnostic package manager.</p>
<p>Many distributions distribute pre-compiled, binary packages, which
give fast, stable installs without the need to have a full build
system on your local machine. When the package you need is in the
official repository (or a third-party repository), this approach works
quite well. There's no need to go through the time or effort of
compiling <a href="http://www.mozilla.org/en-US/firefox/new/">Firefox</a>, <a href="http://blog.tremily.us//./posts/LaTeX/">LaTeX</a>, <a href="http://www.libreoffice.org/">LibreOffice</a>, or other software
that I interact with as a general a user. However, my own packages
(or actively developed libraries that use from my own software) are
rarely available as pre-compiled binaries. If you find yourself in
this situation, it is useful to use a package manager that makes it
easy to write source-based packages (<a href="http://www.gentoo.org/">Gentoo</a>'s <a href="http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1">Portage</a>,
<a href="http://www.exherbo.org/">Exherbo</a>'s <a href="http://paludis.exherbo.org/">Paludis</a>, <a href="http://www.archlinux.org/">Arch</a>'s <a href="https://www.archlinux.org/pacman/">packman</a>, …).</p>
<p>With source-based packaging systems, packaging an existing Python
package is usually a matter of <a href="http://git.tremily.us/?p=wtk-overlay.git;a=blob;f=dev-python/ipythonblocks/ipythonblocks-9999.ebuild;hb=HEAD">listing a bit of
metadata</a>. With <a href="http://layman.sourceforge.net/">layman</a>, integrating your <a href="http://blog.tremily.us//./posts/Gentoo_overlay/">local
packages</a> into your <a href="http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1">Portage</a> tree is extremely
simple. Does your package depend on some other package in another
oddball language? Some wonky build tool? No problem! Just list the
new dependency in your <a href="http://www.gentoo.org/proj/en/qa/pms.xml">ebuild</a> (it probably <a href="http://gpo.zugaina.org/">already
exists</a>). Source-based package managers also make it easy
to stay up to date with ongoing development. <a href="http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1">Portage</a> supports
live ebuilds that build fresh checkouts from a project's version
control repository (use <a href="http://blog.tremily.us//./posts/Git/">Git</a>!). There is no need to dig out your
old installation notes or reread the projects installation
instructions.</p>
<p>Getting back to the goals of reproducible research, I think that
existing package managers are an excellent solution for tracking the
software used to perform experiments or run simulations and analysis.
The main stumbling block is the lack of market penetration ;).
Building a lightweight package manager that can work easily at both
the system-wide and per-user levels across a range of host OSes is
hard work. With the current fractured packaging ecosystem, I doubt
that rolling a new package manager from scratch would be an effective
approach. Existing package managers have mostly satisfied their
users, and the fundamental properties haven't changed much in over a
decade. Writing a system appealing enough to drag these satisfied
users over to your new system is probably not going to happen.</p>
<p><a href="http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1">Portage</a> (and <a href="http://www.gentoo.org/proj/en/gentoo-alt/prefix/">Gentoo Prefix</a>) get you most of the way
there, with the help of well written <a href="http://www.gentoo.org/proj/en/qa/pms.xml">specifications</a> and
<a href="http://devmanual.gentoo.org/quickstart/">documentation</a>. However, compatibility and testing in the
prefix configuration still need some polishing, as does <a href="http://article.gmane.org/gmane.linux.gentoo.devel/84964/">robust binary
packaging support</a>. These issues are less interesting to most
<a href="http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1">Portage</a> developers, as they usually run Portage as the native
package manager and avoid binary packages. If the broader scientific
community is interested in sustainable software, I think effort
channeled into polishing these use-cases would be time well spent.</p>
<p>For those less interested in adopting a full-fledged package manager,
you should at least make <em>some</em> effort to package your software. I
have used software that didn't even have a <code>README</code> with build
instructions, but compiling it was awful. If you're publishing your
software in the hopes that others will find it, use it, and cite you
in their subsequent paper, it behooves you to make the installation as
easy as possible. Until your community coalesces around a single
package management framework, picking a standard build system
(<a href="http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Autotools-Introduction.html">Autotools</a>, <a href="http://docs.python.org/3/distutils/">Distutils</a>, …) will at least make it easier for
folks to install your software by hand.</p>
rss2emailhttp://blog.tremily.us//posts/rss2email/2013-04-14T00:59:48Z2013-04-14T00:44:55Z
<p><span class="infobox">
Available in a <a href="http://blog.tremily.us//./tags/git/">git</a> repository.<br />
Repository: <a href="git://tremily.us/rss2email.git" rel="vcs-git" title="rss2email repository">rss2email</a><br />
Browsable repository: <a href="http://git.tremily.us/?p=rss2email.git" rel="vcs-git" title="rss2email repository">rss2email</a><br />
Author: W. Trevor King<br />
</span></p>
<p>Since November 2012 I've been maintaining <a href="https://github.com/wking/rss2email">rss2email</a>, a package
that converts <a href="http://en.wikipedia.org/wiki/RSS">RSS</a> or <a href="http://en.wikipedia.org/wiki/Atom_%28standard%29">Atom</a> feeds to email so you can follow
them with your mail user agent. Rss2email was created by the late
<a href="http://en.wikipedia.org/wiki/Aaron_Swartz">Aaron Swartz</a> and maintained for several years by <a href="http://www.gerundinganimal.com/">Lindsey
Smith</a>. I've added a <a href="http://dir.gmane.org/gmane.mail.rss2email">mailing list</a> (hosted with <a href="http://blog.tremily.us//./posts/mlmmj/">mlmmj</a>)
and <a href="https://pypi.python.org/pypi/rss2email/">PyPI package</a> and made the GitHub location the homepage.</p>
<p>Overall, setting up the standard project infrastructure has been fun,
and it's nice to see interest in the newly streamlined code picking
up. The timing also works out well, since the <a href="http://googlereader.blogspot.com/2013/03/powering-down-google-reader.html">demise of Google
Reader</a> may push some talented folks in our direction. I'm not
sure how visible rss2email is, especially the fresh development
locations, hence this post ;). If you know anyone who might be
interested in using (or contributing to!) rss2email, please pass the
word.</p>
Factor analysishttp://blog.tremily.us//posts/Factor_analysis/2013-09-20T04:04:58Z2013-03-16T17:04:21Z
<p>I've been trying to wrap my head around <a href="http://en.wikipedia.org/wiki/Factor_analysis">factor analysis</a> as a
theory for designing and understanding test and survey results. This
has turned out to be <a href="http://blog.tremily.us//./posts/Gumbel-Fisher-Tippett_distributions/">another</a>
one of those fields where the going has been a bit rough. I think the
key factors in making these older topics difficult are:</p>
<ul>
<li>“Everybody knows this, so we don't need to write up the details.”</li>
<li>“Hey, I can do better than Bob if I just tweak this knob…”</li>
<li>“I'll just publish this seminal paper behind a paywall…”</li>
</ul>
<p>The resulting discussion ends up being overly complicated, and it's
hard for newcomers to decide if people using similar terminology are
in fact talking about the same thing.</p>
<p>Some of the better open sources for background has been <a href="http://www.unc.edu/~rcm/book/factornew.htm">Tucker and
MacCallum's “Exploratory Factor Analysis” manuscript</a> and <a href="http://www.ics.uci.edu/~welling/classnotes/classnotes.html">Max
Welling's notes</a>. I'll use Welling's terminology for this
discussion.</p>
<p>The basic idea of factor analsys is to model <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>d</mi></math> measurable attributes
as generated by <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi><mi><</mi><mi>d</mi></math> common factors and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>d</mi></math> unique factors.
With <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>n</mi><mo>=</mo><mn>4</mn></math> and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi><mo>=</mo><mn>2</mn></math>, you get something like:</p>
<table class="img"><caption>Relationships between factors and measured attributes
(adapted from Tucker and MacCallum's Figure 1.2)</caption><tr><td><a href="http://blog.tremily.us//./posts/Factor_analysis/factors.png"><img src="http://blog.tremily.us//./posts/Factor_analysis/factors.png" width="520" height="472" alt="Relationships between factors and measured attributes" class="img" /></a></td></tr></table>
<p>Corresponding to the equation (<a href="http://www.ics.uci.edu/~welling/classnotes/papers_class/LinMod.ps.gz">Welling's eq. 1</a>):</p>
<div class="numberedEq"><span>(1)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>+</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>+</mo><mstyle mathvariant="bold"><mi>ν</mi></mstyle></math></div>
<p>The independent random variables <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math> are distributed
according to a Gaussian with zero mean and unit
variance <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>y</mi></mstyle></msub><mo stretchy="false">[</mo><mn>0</mn><mo>,</mo><mstyle mathvariant="bold"><mi>I</mi></mstyle><mo stretchy="false">]</mo></math> (zero mean because
constant offsets are handled by <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle></math>; unit variance because
scaling is handled by <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>). The independent random
variables <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>ν</mi></mstyle></math> are distributed according
to <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>ν</mi></mstyle></msub><mo stretchy="false">[</mo><mn>0</mn><mo>,</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo></math>, with (Welling's
eq. 2):</p>
<div class="numberedEq"><span>(2)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo>≡</mo><mtext>diag</mtext><mo stretchy="false">[</mo><msubsup><mi>σ</mi> <mn>1</mn> <mn>2</mn></msubsup><mo>,</mo><mi>…</mi><mo>,</mo><msubsup><mi>σ</mi> <mi>d</mi> <mn>2</mn></msubsup><mo stretchy="false">]</mo></math></div>
<p>The matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math> (linking common factors with measured
attributes <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mi>is</mi><mi>refered</mi><mi>to</mi><mi>as</mi><mi>the</mi><mo><em></mo><mi>factor</mi><mi>weights</mi><mo></em></mo><mi>or</mi><mo><em></mo><mi>factor</mi><mi>loadings</mi><mo></em></mo><mo>.</mo><mi>Because</mi><mi>the</mi><mi>only</mi><mi>source</mi><mi>of</mi><mi>constant</mi><mi>offset</mi><mi>is</mi></math>\mathbf{\mu}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>,</mo><mi>we</mi><mi>can</mi><mi>calculate</mi><mi>it</mi><mi>by</mi><mi>averaging</mi><mi>out</mi><mi>the</mi><mi>random</mi><mi>noise</mi><mo stretchy="false">(</mo><mi>Welling</mi><mo>′</mo><mi>s</mi><mi>eq</mi><mo>.</mo><mn>6</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>3</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>=</mo><mfrac><mn>1</mn><mi>N</mi></mfrac><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>where</mi></math>N<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>the</mi><mi>number</mi><mi>of</mi><mi>measurements</mi><mo stretchy="false">(</mo><mi>survey</mi><mi>responders</mi><mo stretchy="false">)</mo><mi>and</mi></math>\mathbf{x}<em>n<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>the</mi><mi>response</mi><mi>vector</mi><mi>for</mi><mi>the</mi></math>n^\text{th}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>responder</mi><mo>.</mo><mi>How</mi><mi>do</mi><mi>we</mi><mi>find</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>?</mo><mi>This</mi><mi>is</mi><mi>the</mi><mi>tricky</mi><mi>bit</mi><mo>,</mo><mi>and</mi><mi>there</mi><mi>are</mi><mi>a</mi><mi>number</mi><mi>of</mi><mi>possible</mi><mi>approaches</mi><mo>.</mo><mi>Welling</mi><mi>suggests</mi><mi>using</mi><mi>expectation</mi><mi>maximization</mi><mo stretchy="false">(</mo><mi>EM</mi><mo stretchy="false">)</mo><mo>,</mo><mi>and</mi><mi>there</mi><mo>′</mo><mi>s</mi><mi>an</mi><mi>excellent</mi><mi>example</mi><mi>of</mi><mi>the</mi><mi>procedure</mi><mi>with</mi><mi>a</mi><mi>colorblind</mi><mi>experimenter</mi><mi>drawing</mi><mi>colored</mi><mi>balls</mi><mi>in</mi><mi>his</mi><mo stretchy="false">[</mo><mi>EM</mi><mi>notes</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mi>EM</mi><mo stretchy="false">]</mo><mo stretchy="false">(</mo><mi>to</mi><mi>test</mi><mi>my</mi><mi>understanding</mi><mo>,</mo><mi>I</mi><mi>wrote</mi><merror><mtext>Unknown character</mtext></merror><mi>a</mi><mi>href</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mo>.</mo><mo stretchy="false">/</mo><mi>posts</mi><mo stretchy="false">/</mo><msub><mi>Factor</mi> <mi>analysis</mi></msub><mo stretchy="false">/</mo><mi>color</mi><mo>−</mo><mi>ball</mi><mo>.</mo><mi>py</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>color</mi><mo>−</mo><mi>ball</mi><mo>.</mo><mi>py</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>a</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">)</mo><mo>.</mo><mi>To</mi><mi>simplify</mi><mi>calculations</mi><mo>,</mo><mi>Welling</mi><mi>defines</mi><mo stretchy="false">(</mo><mi>before</mi><mi>eq</mi><mo>.</mo><mn>15</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>4</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>′</mo></mtd> <mtd><mo>≡</mo><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">]</mo></mtd></mtr> <mtr><mtd><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo></mtd> <mtd><mo>≡</mo><mo stretchy="false">[</mo><msup><mstyle mathvariant="bold"><mi>y</mi></mstyle> <mi>T</mi></msup><mo>,</mo><mn>1</mn><msup><mo stretchy="false">]</mo> <mi>T</mi></msup></mtd></mtr></mtable></mrow></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>which</mi><mi>reduce</mi><mi>the</mi><mi>model</mi><mi>to</mi><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>5</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>′</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mo lspace="verythinmathspace" rspace="0em">+</mo><mstyle mathvariant="bold"><mi>ν</mi></mstyle></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>After</mi><mi>some</mi><mi>manipulation</mi><mi>Welling</mi><mi>works</mi><mi>out</mi><mi>the</mi><mi>maximizing</mi><mi>updates</mi><mo stretchy="false">(</mo><mi>eq</mi><mo>′</mo><mi>ns</mi><mn>16</mn><mi>and</mi><mn>17</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>6</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mo>′</mo> <mtext>new</mtext></msup></mtd> <mtd><mo>=</mo><mrow><mo>(</mo><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><msup><mo stretchy="false">]</mo> <mi>T</mi></msup><mo>)</mo></mrow><msup><mrow><mo>(</mo><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><msup><mo>′</mo> <mi>T</mi></msup><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo><mo>)</mo></mrow> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup></mtd></mtr> <mtr><mtd><msup><mstyle mathvariant="bold"><mi>Σ</mi></mstyle> <mtext>new</mtext></msup></mtd> <mtd><mo>=</mo><mfrac><mn>1</mn><mi>N</mi></mfrac><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow> <mi>N</mi></msubsup><mtext>diag</mtext><mo stretchy="false">[</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><msubsup><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi> <mi>T</mi></msubsup><mo>−</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mo>′</mo> <mtext>new</mtext></msup><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo>′</mo><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo><msubsup><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi> <mi>T</mi></msubsup><mo stretchy="false">]</mo></mtd></mtr></mtable></mrow></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>The</mi><mi>expectation</mi><mi>values</mi><mi>used</mi><mi>in</mi><mi>these</mi><mi>updates</mi><mi>are</mi><mi>given</mi><mi>by</mi><mo stretchy="false">(</mo><mi>Welling</mi><mo>′</mo><mi>s</mi><mi>eq</mi><mo>′</mo><mi>ns</mi><mn>12</mn><mi>and</mi><mn>13</mn><mo stretchy="false">)</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>7</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo></mtd> <mtd><mo>=</mo><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo>+</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><msup><mo stretchy="false">)</mo> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><msup><mstyle mathvariant="bold"><mi>y</mi></mstyle> <mi>T</mi></msup><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo></mtd> <mtd><mo>=</mo><mstyle mathvariant="bold"><mi>I</mi></mstyle><mo>−</mo><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup><mo>+</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><msup><mo stretchy="false">)</mo> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>+</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><mo stretchy="false">]</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><msub><mstyle mathvariant="bold"><mi>x</mi></mstyle> <mi>n</mi></msub><msup><mo stretchy="false">]</mo> <mi>T</mi></msup></mtd></mtr></mtable></mrow></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><mi>Survey</mi><mi>analysis</mi><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mi>Enough</mi><mi>abstraction</mi><mo>!</mo><mi>Let</mi><mo>′</mo><mi>s</mi><mi>look</mi><mi>at</mi><mi>an</mi><mi>example</mi><mo>:</mo><mo stretchy="false">[</mo><mi>survey</mi><mi>results</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mi>survey</mi><mo stretchy="false">]</mo><mo>:</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>import</mi><mi>numpy</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>scores</mi><mo>=</mo><mi>numpy</mi><mo>.</mo><mi>genfromtxt</mi><mo stretchy="false">(</mo><mo>′</mo><msub><mi>Factor</mi> <mi>analysis</mi></msub><mo stretchy="false">/</mo><mi>survey</mi><mo>.</mo><mi>data</mi><mo>′</mo><mo>,</mo><mi>delimiter</mi><mo>=</mo><mo>′</mo><mo lspace="0em" rspace="thinmathspace">t</mo><mo>′</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>scores</mi><mi>array</mi><mo stretchy="false">(</mo><mo stretchy="false">[</mo><mo stretchy="false">[</mo><mn>1</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>1</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo>,</mo><mn>4</mn><mo>.</mo><mo stretchy="false">]</mo><mo>,</mo><mo stretchy="false">[</mo><mn>2</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>5</mn><mo>.</mo><mo>,</mo><mn>7</mn><mo>.</mo><mo>,</mo><mn>6</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo>,</mo><mn>3</mn><mo>.</mo><mo stretchy="false">]</mo><mo stretchy="false">]</mo><mo stretchy="false">)</mo><mo><code></mo><mi>scores</mi><mo stretchy="false">[</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo stretchy="false">]</mo><mo></code></mo><mi>is</mi><mi>the</mi><mi>answer</mi><mi>the</mi><mo><code></mo><mi>i</mi><mo></code></mo><mi>th</mi><mi>respondent</mi><mi>gave</mi><mi>for</mi><mi>the</mi><mo><code></mo><mi>j</mi><mo></code></mo><mi>th</mi><mi>question</mi><mo>.</mo><mi>We</mi><mo>′</mo><mi>re</mi><mi>looking</mi><mi>for</mi><mi>underlying</mi><mi>factors</mi><mi>that</mi><mi>can</mi><mi>explain</mi><mi>covariance</mi><mi>between</mi><mi>the</mi><mi>different</mi><mi>questions</mi><mo>.</mo><mi>Do</mi><mi>the</mi><mi>question</mi><mi>answers</mi><mo stretchy="false">(</mo></math>\mathbf{x}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo stretchy="false">)</mo><mi>represent</mi><mi>some</mi><mi>underlying</mi><mi>factors</mi><mo stretchy="false">(</mo></math>\mathbf{y}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo stretchy="false">)</mo><mo>?</mo><mi>Let</mi><mo>′</mo><mi>s</mi><mi>start</mi><mi>off</mi><mi>by</mi><mi>calculating</mi></math>\mathbf{\mu}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>:</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>def</mi><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>row</mi><mo stretchy="false">)</mo><mo>:</mo><mo>.</mo><mo>.</mo><mo>.</mo><mi>print</mi><mo stretchy="false">(</mo><mo>′</mo><mo>′</mo><mo>.</mo><mi>join</mi><mo stretchy="false">(</mo><mo>′</mo><mrow><mo>:</mo><mn>0.2</mn><mi>f</mi></mrow><mo>′</mo><mo>.</mo><mi>format</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mi>for</mi><mi>x</mi><mi>in</mi><mi>row</mi><mo stretchy="false">)</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>mu</mi><mo>=</mo><mi>scores</mi><mo>.</mo><mi>mean</mi><mo stretchy="false">(</mo><mi>axis</mi><mo>=</mo><mn>0</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>mu</mi><mo stretchy="false">)</mo><mn>2.42</mn><mn>3.58</mn><mn>5.08</mn><mn>5.75</mn><mn>6.08</mn><mn>3.42</mn><mn>3.92</mn><mn>4.25</mn><mi>Next</mi><mi>we</mi><mi>need</mi><mi>priors</mi><mi>for</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><merror><mtext>Unknown character</mtext></merror><mi>span</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>createlink</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>MDP</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mi>has</mi><mi>an</mi><mi>implementation</mi><mi>for</mi><merror><mtext>Unknown character</mtext></merror><mi>a</mi><mi>href</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mo>.</mo><mo stretchy="false">/</mo><mi>posts</mi><mo stretchy="false">/</mo><mi>Python</mi><mo stretchy="false">/</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>Python</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>a</mi><merror><mtext>Unknown character</mtext></merror><mo>,</mo><mi>and</mi><mi>their</mi><mo stretchy="false">[</mo><mi>FANode</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mo stretchy="false">]</mo><mi>uses</mi><mi>a</mi><mi>Gaussian</mi><mi>random</mi><mi>matrix</mi><mi>for</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi><mi>the</mi><mi>diagonal</mi><mi>of</mi><mi>the</mi><mi>score</mi><mi>covariance</mi><mi>for</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><mi>They</mi><mi>also</mi><mi>use</mi><mi>the</mi><mi>score</mi><mi>covariance</mi><mi>to</mi><mi>avoid</mi><mi>repeated</mi><mi>summations</mi><mi>over</mi></math>n<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>import</mi><mi>mdp</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>def</mi><msub><mi>print</mi> <mi>matrix</mi></msub><mo stretchy="false">(</mo><mi>matrix</mi><mo stretchy="false">)</mo><mo>:</mo><mo>.</mo><mo>.</mo><mo>.</mo><mi>for</mi><mi>row</mi><mi>in</mi><mi>matrix</mi><mo>:</mo><mo>.</mo><mo>.</mo><mo>.</mo><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>row</mi><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>fa</mi><mo>=</mo><mi>mdp</mi><mo>.</mo><mi>nodes</mi><mo>.</mo><mi>FANode</mi><mo stretchy="false">(</mo><msub><mi>output</mi> <mi>dim</mi></msub><mo>=</mo><mn>3</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>numpy</mi><mo>.</mo><mi>random</mi><mo>.</mo><mi>seed</mi><mo stretchy="false">(</mo><mn>1</mn><mo stretchy="false">)</mo><mo>#</mo><mi>for</mi><mi>consistend</mi><mi>doctest</mi><mi>results</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>responder</mi> <mi>scores</mi></msub><mo>=</mo><mi>fa</mi><mo stretchy="false">(</mo><mi>scores</mi><mo stretchy="false">)</mo><mo>#</mo><mi>common</mi><mi>factors</mi><mi>for</mi><mi>each</mi><mi>responder</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>matrix</mi></msub><mo stretchy="false">(</mo><msub><mi>responder</mi> <mi>scores</mi></msub><mo stretchy="false">)</mo><mo>−</mo><mn>1.92</mn><mo>−</mo><mn>0.45</mn><mn>0.00</mn><mn>0.67</mn><mn>1.97</mn><mn>1.96</mn><mn>0.70</mn><mn>0.03</mn><mo>−</mo><mn>2.00</mn><mn>0.29</mn><mn>0.03</mn><mo>−</mo><mn>0.60</mn><mo>−</mo><mn>1.02</mn><mn>1.79</mn><mo>−</mo><mn>1.43</mn><mn>0.82</mn><mn>0.27</mn><mo>−</mo><mn>0.23</mn><mo>−</mo><mn>0.07</mn><mo>−</mo><mn>0.08</mn><mn>0.82</mn><mo>−</mo><mn>1.38</mn><mo>−</mo><mn>0.27</mn><mn>0.48</mn><mn>0.79</mn><mo>−</mo><mn>1.17</mn><mn>0.50</mn><mn>1.59</mn><mo>−</mo><mn>0.30</mn><mo>−</mo><mn>0.41</mn><mn>0.01</mn><mo>−</mo><mn>0.48</mn><mn>0.73</mn><mo>−</mo><mn>0.46</mn><mo>−</mo><mn>1.34</mn><mn>0.18</mn><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>fa</mi><mo>.</mo><mi>mu</mi><mo>.</mo><mi>flat</mi><mo stretchy="false">)</mo><mn>2.42</mn><mn>3.58</mn><mn>5.08</mn><mn>5.75</mn><mn>6.08</mn><mn>3.42</mn><mn>3.92</mn><mn>4.25</mn><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>fa</mi><mo>.</mo><mi>mu</mi><mo>.</mo><mi>flat</mi><mo>=</mo><mo>=</mo><mi>mu</mi><mo>#</mo><mi>MDP</mi><mi>agrees</mi><mi>with</mi><mi>our</mi><mi>earlier</mi><mi>calculation</mi><mi>array</mi><mo stretchy="false">(</mo><mo stretchy="false">[</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo>,</mo><mi>True</mi><mo stretchy="false">]</mo><mo>,</mo><mi>dtype</mi><mo>=</mo><mi>bool</mi><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>matrix</mi></msub><mo stretchy="false">(</mo><mi>fa</mi><mo>.</mo><mi>A</mi><mo stretchy="false">)</mo><mo>#</mo><mi>factor</mi><mi>weights</mi><mi>for</mi><mi>each</mi><mi>question</mi><mn>0.80</mn><mo>−</mo><mn>0.06</mn><mo>−</mo><mn>0.45</mn><mn>0.17</mn><mn>0.30</mn><mo>−</mo><mn>0.65</mn><mn>0.34</mn><mo>−</mo><mn>0.13</mn><mo>−</mo><mn>0.25</mn><mn>0.13</mn><mo>−</mo><mn>0.73</mn><mo>−</mo><mn>0.64</mn><mn>0.02</mn><mo>−</mo><mn>0.32</mn><mo>−</mo><mn>0.70</mn><mn>0.61</mn><mn>0.23</mn><mn>0.86</mn><mn>0.08</mn><mn>0.63</mn><mn>0.59</mn><mo>−</mo><mn>0.09</mn><mn>0.67</mn><mn>0.13</mn><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>fa</mi><mo>.</mo><mi>sigma</mi><mo stretchy="false">)</mo><mo>#</mo><mi>unique</mi><mi>noise</mi><mi>for</mi><mi>each</mi><mi>question</mi><mn>0.04</mn><mn>0.02</mn><mn>0.38</mn><mn>0.55</mn><mn>0.30</mn><mn>0.05</mn><mn>0.48</mn><mn>0.21</mn><mi>Because</mi><mi>the</mi><mi>covariance</mi><mi>is</mi><mi>unaffected</mi><mi>by</mi><mi>the</mi><mi>rotation</mi></math>\mathbf{A}\rightarrow\mathbf{A}\mathbf{R}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>,</mo><mi>the</mi><mi>estimated</mi><mi>weights</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi><mi>responder</mi><mi>scores</mi></math>\mathbf{y}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>can</mi><mi>be</mi><mi>quite</mi><mi>sensitive</mi><mi>to</mi><mi>the</mi><mi>seed</mi><mi>priors</mi><mo>.</mo><mi>The</mi><mi>width</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>of</mi><mi>the</mi><mi>unique</mi><mi>noise</mi></math>\mathbf{\nu}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>more</mi><mi>robust</mi><mo>,</mo><mi>because</mi></math>\mathbf{\Sigma}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>unaffected</mi><mi>by</mi><mi>rotations</mi><mi>on</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>.</mo><mi>Related</mi><mi>tidbits</mi><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mo>=</mo><mi>Communality</mi><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mo>−</mo><mi>The</mi><mo stretchy="false">[</mo><mi>communality</mi><mo stretchy="false">]</mo><mo stretchy="false">[</mo><mo stretchy="false">]</mo></math>h</em>i^2<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>of</mi><mi>the</mi></math>i^\text{th}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>measured</mi><mi>attribute</mi></math>x_i<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>is</mi><mi>the</mi><mi>fraction</mi><mi>of</mi><mi>variance</mi><mi>in</mi><mi>the</mi><mi>measured</mi><mi>attribute</mi><mi>which</mi><mi>is</mi><mi>explained</mi><mi>by</mi><mi>the</mi><mi>set</mi><mi>of</mi><mi>common</mi><mi>factors</mi><mo>.</mo><mi>Because</mi><mi>the</mi><mi>common</mi><mi>factors</mi></math>\mathbf{y}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>have</mi><mi>unit</mi><mi>variance</mi><mo>,</mo><mi>the</mi><mi>communality</mi><mi>is</mi><mi>given</mi><mi>by</mi><mo>:</mo><merror><mtext>Unknown character</mtext></merror><mi>div</mi><mi>class</mi><mo>=</mo><merror><mtext>Unknown character</mtext></merror><mi>numberedEq</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>span</mi><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">(</mo><mn>8</mn><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>span</mi><merror><mtext>Unknown character</mtext></merror></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>h</mi> <mi>i</mi></msub><mo>=</mo><mfrac><mrow><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></msubsup><msubsup><mi>A</mi> <mi>ij</mi> <mn>2</mn></msubsup></mrow><mrow><msubsup><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></msubsup><msubsup><mi>A</mi> <mi>ij</mi> <mn>2</mn></msubsup><mo>+</mo><msubsup><mi>σ</mi> <mn>1</mn> <mn>2</mn></msubsup></mrow></mfrac></math><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>Unknown character</mtext></merror><mo stretchy="false">/</mo><mi>div</mi><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>factor</mi> <mi>variance</mi></msub><mo>=</mo><mi>numpy</mi><mo>.</mo><mi>array</mi><mo stretchy="false">(</mo><mo stretchy="false">[</mo><mi>sum</mi><mo stretchy="false">(</mo><mi>row</mi><mo><em></mo><mo></em></mo><mn>2</mn><mo stretchy="false">)</mo><mi>for</mi><mi>row</mi><mi>in</mi><mi>fa</mi><mo>.</mo><mi>A</mi><mo stretchy="false">]</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><mi>h</mi><mo>=</mo><mi>numpy</mi><mo>.</mo><mi>array</mi><mo stretchy="false">(</mo><mo>.</mo><mo>.</mo><mo>.</mo><mo stretchy="false">[</mo><mi>var</mi><mo stretchy="false">/</mo><mo stretchy="false">(</mo><mi>var</mi><mo>+</mo><mi>sig</mi><mo stretchy="false">)</mo><mi>for</mi><mi>var</mi><mo>,</mo><mi>sig</mi><mi>in</mi><mi>zip</mi><mo stretchy="false">(</mo><msub><mi>factor</mi> <mi>variance</mi></msub><mo>,</mo><mi>fa</mi><mo>.</mo><mi>sigma</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo stretchy="false">)</mo><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><merror><mtext>Unknown character</mtext></merror><msub><mi>print</mi> <mi>row</mi></msub><mo stretchy="false">(</mo><mi>h</mi><mo stretchy="false">)</mo><mn>0.95</mn><mn>0.97</mn><mn>0.34</mn><mn>0.64</mn><mn>0.66</mn><mn>0.96</mn><mn>0.61</mn><mn>0.69</mn><mi>There</mi><mi>may</mi><mi>be</mi><mi>some</mi><mi>scaling</mi><mi>issues</mi><mi>in</mi><mi>the</mi><mi>communality</mi><mi>due</mi><mi>to</mi><mi>deviations</mi><mi>between</mi><mi>the</mi><mi>estimated</mi></math>\mathbf{A}<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>and</mi><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math> and the
variations contained in the measured scores (why?):</p>
<pre><code>>>> print_row(factor_variance + fa.sigma)
0.89 0.56 0.57 1.51 0.89 1.21 1.23 0.69
>>> print_row(scores.var(axis=0, ddof=1)) # total variance for each question
0.99 0.63 0.63 1.66 0.99 1.36 1.36 0.75
</code></pre>
<p>The <a href="http://sites.stat.psu.edu/~ajw13/stat505/fa06/17_factor/07_factor_commun.html">proportion of total variation</a> explained by the
common factors is given by:</p>
<div class="numberedEq"><span>(9)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mfrac><mrow><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></munderover><msub><mi>h</mi> <mi>i</mi></msub></mrow><mrow></mrow></mfrac></math></div>
<h2>Varimax rotation</h2>
<p>As mentioned earlier, factor analysis generated loadings <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>
that are unique up to an arbitrary rotation <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>R</mi></mstyle></math> (as you'd
expect for a <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi></math>-dimensional Gaussian ball of factors <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math>).
A number of of schemes have been proposed to simplify the initial
loadings by rotating <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math> to reduce off-diagonal terms. One
of the more popular approaches is Henry Kaiser's <em>varimax rotation</em>
(unfortunately, I don't have access to either his <a href="http://oskicat.berkeley.edu/record=b12138762~S1">thesis</a> or
the <a href="http://dx.doi.org/10.1007%2FBF02289233">subsequent paper</a>). I did find (via
<a href="http://en.wikipedia.org/wiki/Varimax_rotation">Wikipedia</a>) <a href="http://www.stat.ufl.edu/~tpark/Research/about_varimax.html">Trevor Park's notes</a> which have been
very useful.</p>
<p>The idea is to iterate rotations to maximize the raw varimax criterion
(Park's eq. 1):</p>
<div class="numberedEq"><span>(10)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mi>V</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>k</mi></munderover><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msubsup><mi>A</mi> <mi>ij</mi> <mn>4</mn></msubsup><mo>−</mo><msup><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msubsup><mi>A</mi> <mi>ij</mi> <mn>4</mn></msubsup><mo>)</mo></mrow> <mn>2</mn></msup><mo>)</mo></mrow></math></div>
<p>Rather than computing a <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>k</mi></math>-dimensional rotation in one sweep, we'll
iterate through 2-dimensional rotations (on successive column pairs)
until convergence. For a particular column pair <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo stretchy="false">(</mo><mi>p</mi><mo>,</mo><mi>q</mi><mo stretchy="false">)</mo></math>, the
rotation matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>R</mi> <mo>*</mo></msup></math> is the usual rotation matrix:</p>
<div class="numberedEq"><span>(11)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><msup><mi>R</mi> <mo>*</mo></msup><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>cos</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd> <mtd><mo lspace="verythinmathspace" rspace="0em">−</mo><mi>sin</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>sin</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd> <mtd><mi>cos</mi><mo stretchy="false">(</mo><msup><mi>ϕ</mi> <mo>*</mo></msup><mo stretchy="false">)</mo></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
<p>where the optimum rotation angle <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>ϕ</mi> <mo>*</mo></msup></math> is (Park's eq. 3):</p>
<div class="numberedEq"><span>(12)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><msup><mi>ϕ</mi> <mo>*</mo></msup><mo>=</mo><mfrac><mn>1</mn><mn>4</mn></mfrac><mo>∠</mo><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msup><mrow><mo>(</mo><msub><mi>A</mi> <mi>jp</mi></msub><mo>+</mo><msub><mi>iA</mi> <mi>jq</mi></msub><mo>)</mo></mrow> <mn>4</mn></msup><mo>−</mo><msup><mrow><mo>(</mo><mfrac><mn>1</mn><mi>d</mi></mfrac><munderover><mo lspace="thinmathspace" rspace="thinmathspace">∑</mo> <mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow> <mi>d</mi></munderover><msup><mrow><mo>(</mo><msub><mi>A</mi> <mi>jp</mi></msub><mo>+</mo><msub><mi>iA</mi> <mi>jq</mi></msub><mo>)</mo></mrow> <mn>2</mn></msup><mo>)</mo></mrow> <mn>2</mn></msup><mo>)</mo></mrow></math></div>
<p>where <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>i</mi><mo>≡</mo><msqrt><mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msqrt></math>.</p>
<h1>Nomenclature</h1>
<dl>
<dt id="subsripts"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>A</mi> <mi>ij</mi></msub></math></dt>
<dd>The element from the <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>i</mi> <mtext>th</mtext></msup></math> row and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mi>j</mi> <mtext>th</mtext></msup></math>
column of a matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>. For example here is a 2-by-3
matrix terms of components:
<div class="numberedEq"><span>(13)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><msub><mi>A</mi> <mn>11</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>12</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>13</mn></msub></mtd></mtr> <mtr><mtd><msub><mi>A</mi> <mn>21</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>22</mn></msub></mtd> <mtd><msub><mi>A</mi> <mn>23</mn></msub></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
</dd>
<dt id="transpose"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mi>T</mi></msup></math></dt>
<dd>The transpose of a matrix (or vector) <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>.
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msubsup><mi>A</mi> <mi>ij</mi> <mi>T</mi></msubsup><mo>=</mo><msub><mi>A</mi> <mi>ji</mi></msub></math></dd>
<dt id="inverse"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup></math></dt>
<dd>The inverse of a matrix <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>.
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msup><mstyle mathvariant="bold"><mi>A</mi></mstyle> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mover><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo>˙</mo></mover><mo>=</mo><mn>1</mn></math></dd>
<dt id="diag"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mtext>diag</mtext><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>A</mi></mstyle><mo stretchy="false">]</mo></math></dt>
<dd>A matrix containing only the diagonal elements of
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>A</mi></mstyle></math>, with the off-diagonal values set to zero.</dd>
<dt id="expect"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></math></dt>
<dd>Expectation value for a function <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>f</mi></math> of a random variable
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math>. If the probability density of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> is
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo></math>, then <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>=</mo><mo>∫</mo><mi>d</mi><mstyle mathvariant="bold"><mi>x</mi></mstyle><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo></math>. For example,
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>=</mo><mn>1</mn></math>.</dd>
<dt id="mean"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle></math></dt>
<dd>The mean of a random variable <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> is given by
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">]</mo></math>.</dd>
<dt id="variance"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math></dt>
<dd>The covariance of a random variable <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> is given by
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo>=</mo><mstyle mathvariant="bold"><mi>E</mi></mstyle><mo stretchy="false">[</mo><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><msup><mo stretchy="false">)</mo> <mstyle mathvariant="bold"><mi>T</mi></mstyle></msup><mo stretchy="false">]</mo></math>. In the factor analysis
model discussed above, <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math> is restricted to a
diagonal matrix.</dd>
<dt id="gaussian"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>x</mi></mstyle></msub><mo stretchy="false">[</mo><mi>μ</mi><mo>,</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo></math></dd>
<dd>A Gaussian probability density for the random variables
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> with a mean <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>μ</mi></mstyle></math> and a covariance
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>Σ</mi></mstyle></math>.
<div class="numberedEq"><span>(14)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><msub><mi>𝒢</mi> <mstyle mathvariant="bold"><mi>x</mi></mstyle></msub><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mo stretchy="false">(</mo><mn>2</mn><mi>π</mi><msup><mo stretchy="false">)</mo> <mfrac><mi>D</mi><mn>2</mn></mfrac></msup><msqrt><mrow><mi>det</mi><mo stretchy="false">[</mo><mstyle mathvariant="bold"><mi>Σ</mi></mstyle><mo stretchy="false">]</mo></mrow></msqrt></mrow></mfrac><msup><mi>e</mi> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><msup><mo stretchy="false">)</mo> <mi>T</mi></msup><msup><mstyle mathvariant="bold"><mi>Σ</mi></mstyle> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold"><mi>μ</mi></mstyle><mo stretchy="false">)</mo></mrow></msup></math></div>
</dd>
<dt id="probability-of-y-given-x"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">∣</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">)</mo></math></dt>
<dd>Probability of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math> occurring given that <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math>
occured. This is commonly used in <a
href="http://en.wikipedia.org/wiki/Bayes%27_theorem">Bayesian
statistics</a>.</dd>
<dt id="probability-of-x-and-y"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo></math></dt>
<dd>Probability of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>y</mi></mstyle></math> and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mstyle mathvariant="bold"><mi>x</mi></mstyle></math> occuring
simultaneously (the joint density).
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo>,</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>x</mi></mstyle><mo stretchy="false">∣</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold"><mi>y</mi></mstyle><mo stretchy="false">)</mo></math></dd>
<dt id="angle"><math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>∠</mo><mo stretchy="false">(</mo><mi>z</mi><mo stretchy="false">)</mo></math></dt>
<dd>The angle of <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>z</mi></math> in the complex plane.
<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mo>∠</mo><mo stretchy="false">(</mo><msup><mi>re</mi> <mrow><mi>i</mi><mi>θ</mi></mrow></msup><mo stretchy="false">)</mo><mo>=</mo><mi>θ</mi></math>.</dd>
</dl>
<p>Note: if you have trouble viewing some of the more obscure <a href="http://en.wikipedia.org/wiki/Unicode">Unicode</a>
used in this post, you might want to install the <a href="http://www.stixfonts.org/">STIX fonts</a>.</p>
catalysthttp://blog.tremily.us//posts/catalyst/2013-03-09T17:54:16Z2013-03-06T17:50:45Z
<p><span class="infobox">
Available in a <a href="http://blog.tremily.us//./tags/git/">git</a> repository.<br />
Repository: <a href="git://tremily.us/catalyst-swc.git" rel="vcs-git" title="catalyst-swc repository">catalyst-swc</a><br />
Browsable repository: <a href="http://git.tremily.us/?p=catalyst-swc.git" rel="vcs-git" title="catalyst-swc repository">catalyst-swc</a><br />
Author: W. Trevor King<br />
</span></p>
<p><a href="http://www.gentoo.org/proj/en/releng/catalyst/">Catalyst</a> is a release-building tool for Gentoo. If you use Gentoo
and want to roll your own live CD or bootable USB drive, this is the
way to go. As I've been wrapping my head around catalyst, I've been
<a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2006">pushing</a> <a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=0a99c494d4602ebb92f6d352f86ec519903fdc88">my</a> <a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2107">notes</a>
<a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=b7f2cf0f4c34af8d2dd9dcfa186930c17323e298">upstream</a>. This post builds on those notes to discuss the
construction of <a href="http://lists.software-carpentry.org/pipermail/discuss-software-carpentry.org/2013-March/000210.html">a bootable ISO</a> for <a href="http://software-carpentry.org/">Software Carpentry</a>
boot camps.</p>
<h1>Getting a patched up catalyst</h1>
<p>Catalyst has been around for a while, but the user base has been
fairly small. If you try to do something that <a href="http://www.gentoo.org/proj/en/releng/">Gentoo's Release
Engineering team</a> doesn't do on a regular basis, built in
catalyst support can be spotty. There's been a fair amount of patch
submissions an <a href="http://www.gentoo.org/main/en/lists.xml#doc_chap1_sect2">gentoo-catalyst@</a> recently, but patch acceptance can
be slow. For the SWC ISO, I applied versions of the following patches
(or patch series) to <a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=37540ff5e9db28e65a4f55b71c16cec1e7d294ec">37540ff</a>:</p>
<ul>
<li><a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2085/focus=2093">chmod +x all sh scripts so they can run from the git checkout</a></li>
<li><a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2182">livecdfs-update.sh: Set XSESSION in /etc/env.d/90xsession</a></li>
<li><a href="http://thread.gmane.org/gmane.linux.gentoo.catalyst/2187/focus=2191">Fix livecdfs-update.sh startx handling</a></li>
</ul>
<h1>Configuring catalyst</h1>
<p>The easiest way to run catalyst from a Git checkout is to setup a
<a href="http://article.gmane.org/gmane.linux.gentoo.catalyst/2111">local config file</a>. I didn't have enough hard drive space on
my local system (~16 GB) for this build, so I set things up in a
temporary directory on an external hard drive:</p>
<pre><code>$ cat catalyst.conf | grep -v '^#\|^$'
digests="md5 sha1 sha512 whirlpool"
contents="auto"
distdir="/usr/portage/distfiles"
envscript="/etc/catalyst/catalystrc"
hash_function="crc32"
options="autoresume kerncache pkgcache seedcache snapcache"
portdir="/usr/portage"
sharedir="/home/wking/src/catalyst"
snapshot_cache="/mnt/d/tmp/catalyst/snapshot_cache"
storedir="/mnt/d/tmp/catalyst"
</code></pre>
<p>I used the default values for everything except <code>sharedir</code>,
<code>snapshot_cache</code>, and <code>storedir</code>. Then I cloned the <code>catalyst-swc</code>
repository into <code>/mnt/d/tmp/catalyst</code>.</p>
<h1>Portage snapshot and a seed stage</h1>
<p>Take a snapshot of the current Portage tree:</p>
<pre><code># catalyst -c catalyst.conf --snapshot 20130208
</code></pre>
<p>Download a seed stage3 from a <a href="http://www.gentoo.org/main/en/mirrors.xml">Gentoo mirror</a>:</p>
<pre><code># wget -O /mnt/d/tmp/catalyst/builds/default/stage3-i686-20121213.tar.bz2 \
> http://distfiles.gentoo.org/releases/x86/current-stage3/stage3-i686-20121213.tar.bz2
</code></pre>
<h1>Building the live CD</h1>
<pre><code># catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-stage1-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-stage2-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-stage3-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-livecd-stage1-i686-2013.1.spec
# catalyst -c catalyst.conf -f /mnt/d/tmp/catalyst/spec/default-livecd-stage2-i686-2013.1.spec
</code></pre>
<h1>isohybrid</h1>
<p>To make the ISO bootable from a USB drive, I used <a href="http://www.syslinux.org/wiki/index.php/Doc/isolinux#HYBRID_CD-ROM.2FHARD_DISK_MODE">isohybrid</a>:</p>
<pre><code># cp swc-x86.iso swc-x86-isohybrid.iso
# isohybrid iso-x86-isohybrid.iso
</code></pre>
<p>You can install the resulting ISO on a USB drive with:</p>
<pre><code># dd if=iso-x86-isohybrid.iso of=/dev/sdX
</code></pre>
<p>replacing replacing <code>X</code> with the appropriate drive letter for your USB
drive.</p>
<p>With versions of catalyst after <a href="http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=commit;h=d1c2ba9bdaf1a69078fcbcd6ec4210e829f99f0a">d1c2ba9</a>, the <code>isohybrid</code> call is
built into catalysts ISO construction.</p>
SymPyhttp://blog.tremily.us//posts/SymPy/2013-02-27T13:47:15Z2013-02-27T12:52:03Z
<p><a href="http://sympy.org/">SymPy</a> is a <a href="http://blog.tremily.us//./posts/Python/">Python</a> library for symbolic mathematics. To give
you a feel for how it works, lets extrapolate the extremum location
for <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math> given a quadratic model:</p>
<div class="numberedEq"><span>(1)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mi>A</mi><msup><mi>x</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>x</mi><mo>+</mo><mi>C</mi></math></div>
<p>and three known values:</p>
<div class="numberedEq"><span>(2)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></mtd> <mtd><mo>=</mo><mi>A</mi><msup><mi>a</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>a</mi><mo>+</mo><mi>C</mi></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo></mtd> <mtd><mo>=</mo><mi>A</mi><msup><mi>b</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>b</mi><mo>+</mo><mi>C</mi></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mtd> <mtd><mo>=</mo><mi>A</mi><msup><mi>c</mi> <mn>2</mn></msup><mo>+</mo><mi>B</mi><mi>c</mi><mo>+</mo><mi>C</mi></mtd></mtr></mtable></mrow></math></div>
<p>Rephrase as a matrix equation:</p>
<div class="numberedEq"><span>(3)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><msup><mi>a</mi> <mn>2</mn></msup></mtd> <mtd><mi>a</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>b</mi> <mn>2</mn></msup></mtd> <mtd><mi>b</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>c</mi> <mn>2</mn></msup></mtd> <mtd><mi>c</mi></mtd> <mtd><mn>1</mn></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>⋅</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>A</mi></mtd></mtr> <mtr><mtd><mi>B</mi></mtd></mtr> <mtr><mtd><mi>C</mi></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
<p>So the solutions for <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>A</mi></math>, <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>B</mi></math>, and <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>C</mi></math> are:</p>
<div class="numberedEq"><span>(4)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>A</mi></mtd></mtr> <mtr><mtd><mi>B</mi></mtd></mtr> <mtr><mtd><mi>C</mi></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>=</mo><msup><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><msup><mi>a</mi> <mn>2</mn></msup></mtd> <mtd><mi>a</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>b</mi> <mn>2</mn></msup></mtd> <mtd><mi>b</mi></mtd> <mtd><mn>1</mn></mtd></mtr> <mtr><mtd><msup><mi>c</mi> <mn>2</mn></msup></mtd> <mtd><mi>c</mi></mtd> <mtd><mn>1</mn></mtd></mtr></mtable></mrow><mo>)</mo></mrow> <mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mn>1</mn></mrow></msup><mo>⋅</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo></mtd></mtr> <mtr><mtd><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></mtd></mtr></mtable></mrow><mo>)</mo></mrow><mo>=</mo><mrow><mo>(</mo><mrow><mtable rowspacing="0.5ex"><mtr><mtd><mtext>long</mtext></mtd></mtr> <mtr><mtd><mtext>complicated</mtext></mtd></mtr> <mtr><mtd><mtext>stuff</mtext></mtd></mtr></mtable></mrow><mo>)</mo></mrow></math></div>
<p>Now that we've found the model parameters, we need to find the <math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><mi>x</mi></math>
coordinate of the extremum.</p>
<div class="numberedEq"><span>(5)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mfrac><mrow><mi mathvariant="normal">d</mi><mi>f</mi></mrow><mrow><mi mathvariant="normal">d</mi><mi>x</mi></mrow></mfrac><mo>=</mo><mn>2</mn><mi>A</mi><mi>x</mi><mo>+</mo><mi>B</mi><mspace width="thickmathspace"></mspace><mo>,</mo></math></div>
<p>which is zero when</p>
<div class="numberedEq"><span>(6)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mrow><mtable columnalign="right left right left right left right left right left" columnspacing="0em"><mtr><mtd><mn>2</mn><mi>A</mi><mi>x</mi></mtd> <mtd><mo>=</mo><mo lspace="verythinmathspace" rspace="0em">−</mo><mi>B</mi></mtd></mtr> <mtr><mtd><mi>x</mi></mtd> <mtd><mo>=</mo><mfrac><mrow><mo lspace="verythinmathspace" rspace="0em">−</mo><mi>B</mi></mrow><mrow><mn>2</mn><mi>A</mi></mrow></mfrac></mtd></mtr></mtable></mrow></math></div>
<p>Here's the solution in SymPy:</p>
<pre><code>>>> from sympy import Symbol, Matrix, factor, expand, pprint, preview
>>> a = Symbol('a')
>>> b = Symbol('b')
>>> c = Symbol('c')
>>> fa = Symbol('fa')
>>> fb = Symbol('fb')
>>> fc = Symbol('fc')
>>> M = Matrix([[a**2, a, 1], [b**2, b, 1], [c**2, c, 1]])
>>> F = Matrix([[fa],[fb],[fc]])
>>> ABC = M.inv() * F
>>> A = ABC[0,0]
>>> B = ABC[1,0]
>>> x = -B/(2*A)
>>> x = factor(expand(x))
>>> pprint(x)
2 2 2 2 2 2
a *fb - a *fc - b *fa + b *fc + c *fa - c *fb
---------------------------------------------
2*(a*fb - a*fc - b*fa + b*fc + c*fa - c*fb)
>>> preview(x, viewer='pqiv')
</code></pre>
<p>Where <code>pqiv</code> is the executable for <a href="http://www.pberndt.com/Programme/Linux/pqiv/">pqiv</a>, my preferred image
viewer. With a bit of additional factoring, that is:</p>
<div class="numberedEq"><span>(7)</span><math xmlns='http://www.w3.org/1998/Math/MathML' display='block'><mi>x</mi><mo>=</mo><mfrac><mrow><msup><mi>a</mi> <mn>2</mn></msup><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><msup><mi>b</mi> <mn>2</mn></msup><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><msup><mi>c</mi> <mn>2</mn></msup><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mn>2</mn><mo>⋅</mo><mo stretchy="false">{</mo><mi>a</mi><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><mi>b</mi><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><mi>c</mi><mo stretchy="false">[</mo><mi>f</mi><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>b</mi><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo stretchy="false">}</mo></mrow></mfrac></math></div>