Recent changes to this wiki:
posts:comcast: Add an update on Comcast's port blocking
And clean up the old Comcast post. The links are likely dead, but I
was able to rescue some of my old Liki notes.
And clean up the old Comcast post. The links are likely dead, but I
was able to rescue some of my old Liki notes.
posts:node: Add a post on Node and npm
beep.py: Fix tab -> spaces indentation
posts:dvd-backup: Fix "you're" -> "your" typo
posts:dvd-backup: Fix 'congfig' -> 'config' typo
posts:relative-submodules: Add a post promoting relative URLs
posts:relative-submodules: Add SVG exports of Dia graphics
posts:relative-submodules: Add Dia source for graphics
posts:open-hardware-analog-io: Add post on open analog I/O
Also update the open source force spectroscopy post to link to my
thesis.
Also update the open source force spectroscopy post to link to my
thesis.
posts:NOAA-charts: Add short post pointing to historicalcharts.noaa.gov
crunch.py: Extract unfolding frequency from unfold-protein curves
posts:open_physics_text: Fix "bureaucrat's" -> "bureaucrats" typo
posts:bugs: Mention when my SCons dir-lookup fix landed
posts:package_management: Add post following SWC discussion
contact:pubkey.txt: Update with a new subkey (1A3646ED)
posts:bibtex: Update with crossref ordering notes
Fool me once, shame on... shame on you. Fool me... you can't get
fooled again.
Fool me once, shame on... shame on you. Fool me... you can't get
fooled again.
posts:entrez: Link to BibTeX post
plotpick.py: Reroll around `picker` API and bump to version 0.3
plotpick.py: Force a redraw after changing the mouse-move crosshairs
There should be a way to just redraw the axes in question, but I'm not
sure what it is.
There should be a way to just redraw the axes in question, but I'm not
sure what it is.
compare-unfold.py: Add --contact-slope and --output options
For more quantitative comparisons and easier automation respectively.
For more quantitative comparisons and easier automation respectively.
gyang_curve.py: Allow auxiliary files in the curve directory
I haven't bothered to trace this down to a particular version of the
LabVIEW VIs, but some of my archived experiments have the auxiliary
files in a 'params' subdirectory and some do not. With this patch,
gyang_curve can successfully read both layouts.
I haven't bothered to trace this down to a particular version of the
LabVIEW VIs, but some of my archived experiments have the auxiliary
files in a 'params' subdirectory and some do not. With this patch,
gyang_curve can successfully read both layouts.
posts:rss2email: Mention the rss2email project to raise visibility
In case anyone actually follows this blog ;).
In case anyone actually follows this blog ;).
posts:one-off-git-daemon: Add an --enable=receive-pack section
Also add headings for serving a single repository and serving a few
repositories, since with --enable=receive-pack the post was getting a
bit long for one section.
Also add headings for serving a single repository and serving a few
repositories, since with --enable=receive-pack the post was getting a
bit long for one section.
posts:nginx: Fix spawn-fcgi.fcgiwrap rc-update invocation
We need a four-space indent (not three) for Markdown <pre> blocks.
We're also using `#` to mark a root shell, so we don't need sudo.
We need a four-space indent (not three) for Markdown <pre> blocks.
We're also using `#` to mark a root shell, so we don't need sudo.
posts:nginx: Export HOME=/ to spawn-fcgi process for gitweb
Since Git's 96b9e0e (config: treat user and xdg config permission
problems as errors, 2012-10-13), `git` dies with:
fatal: unable to access '/root/.config/git/config': Permission denied
when it doesn't have access to the HOME directory where it think's its
config files are stored. Running an strace shows:
access("/root/.config/git/config", R_OK) = -1 EACCES (Permission denied)
Since this error is fatal, we need to set the HOME environment
variable to somewhere where the server process (the `nginx` user on my
box) *does* have access. Regardless of the user, `HOME=/` should be
pretty safe ;).
Since Git's 96b9e0e (config: treat user and xdg config permission
problems as errors, 2012-10-13), `git` dies with:
fatal: unable to access '/root/.config/git/config': Permission denied
when it doesn't have access to the HOME directory where it think's its
config files are stored. Running an strace shows:
access("/root/.config/git/config", R_OK) = -1 EACCES (Permission denied)
Since this error is fatal, we need to set the HOME environment
variable to somewhere where the server process (the `nginx` user on my
box) *does* have access. Regardless of the user, `HOME=/` should be
pretty safe ;).
git-grepall.sh: Add a script for grepping all local branches
Sometimes I have a lot of branches (e.g. per-assignment branches for a
single class, to support sharing work between assignments), and I
forget which feature is in which branch. Now I can run:
$ git-grepall.sh PATTERN
to search all the local branches.
Sometimes I have a lot of branches (e.g. per-assignment branches for a
single class, to support sharing work between assignments), and I
forget which feature is in which branch. Now I can run:
$ git-grepall.sh PATTERN
to search all the local branches.
pca.py: Update Newcastle SPSS link
The internet archive page appears to be gone (which is scary ;), but I
found a URL pointing to the current page, which has all the same
stuff.
The internet archive page appears to be gone (which is scary ;), but I
found a URL pointing to the current page, which has all the same
stuff.
pca.py: Add ability to drop columns (questions) from input data
pca.py: Add principal component analysis script
A good deal of this script is a huge docstring full of tests comparing
my analysis with other examples I found online. You can run the
doctests with:
$ python -m doctest pca.py
References:
Hervé Abdi. Factor Rotations in Factor Analyses. In: Lewis-Beck M.,
Bryman, A., Futing T. (Eds.) (2003). Encyclopedia of Social Sciences
Research Methods. Thousand Oaks (CA): Sage.
Trevor Park. A Note on Terse Coding of Kaiser's Varimax Rotation
Using Complex Number Representation. (March 2003).
Andrew Wiesner's factor analysis notes.
Daniel Denis's factor analysis notes. (2009).
Archived version of the University of Newcastle upon Tyne's "How to
Perform and Interpret Factor Analysis using SPSS". (November 2005).
A good deal of this script is a huge docstring full of tests comparing
my analysis with other examples I found online. You can run the
doctests with:
$ python -m doctest pca.py
References:
Hervé Abdi. Factor Rotations in Factor Analyses. In: Lewis-Beck M.,
Bryman, A., Futing T. (Eds.) (2003). Encyclopedia of Social Sciences
Research Methods. Thousand Oaks (CA): Sage.
Trevor Park. A Note on Terse Coding of Kaiser's Varimax Rotation
Using Complex Number Representation. (March 2003).
Andrew Wiesner's factor analysis notes.
Daniel Denis's factor analysis notes. (2009).
Archived version of the University of Newcastle upon Tyne's "How to
Perform and Interpret Factor Analysis using SPSS". (November 2005).
posts:factor_analysis: Add a post on Factor Analysis
This is the statistical approach used to analyze Software Carpentry
surveys, so I've been figuring out how it works.
This is the statistical approach used to analyze Software Carpentry
surveys, so I've been figuring out how it works.
posts:catalyst: Link to HOWTO commit and fix 'amd64' -> 'x86'
posts:catalyst: Add post about building the SWC bootable ISO
posts:sympy: Add post demonstrating usage
posts:one-off-git-daemon: Add a git://192.168.1.2/ example
Sometimes requiring a project name is not useful (e.g. all students
are sharing the same project).
Sometimes requiring a project name is not useful (e.g. all students
are sharing the same project).
posts:bugs: Mention comments on Python's #9263
This came up while digging into an rss2email report by Dmitry Bogatov
[1].
[1]: http://article.gmane.org/gmane.mail.rss2email/13
http://article.gmane.org/gmane.mail.rss2email/14
This came up while digging into an rss2email report by Dmitry Bogatov
[1].
[1]: http://article.gmane.org/gmane.mail.rss2email/13
http://article.gmane.org/gmane.mail.rss2email/14
posts:bugs: Update URL for my utf_8 iText patch
The original patch page is gone, but the change has been accepted
upstream via the itext-questions@ list.
The original patch page is gone, but the change has been accepted
upstream via the itext-questions@ list.
posts:bugs: Mention my Gnuplot '<&n' file descriptor patch
Now accepted :). Thanks Ethan!
Now accepted :). Thanks Ethan!
posts:One-off_Git_daemon: Add post describing ad-hoc usage
mw2txt.py: Update to version 0.2 (mostly Python 3 conversion)
Also:
* Use logging module instead of `if debug: print(...)`.
* Rework coloring to work with a logger (e.g. don't print debugging
info directly to stdout).
* Upgrade from optparse to argparse.
* Add --verbose argument.
Also:
* Use logging module instead of `if debug: print(...)`.
* Rework coloring to work with a logger (e.g. don't print debugging
info directly to stdout).
* Upgrade from optparse to argparse.
* Add --verbose argument.
pdf-merge.py: Bump to version 0.3
Changes since 0.2:
77a9ba6 pdf-merge.py: Upgrade to Python 3.x
0c3b4c3 pdf-merge.py: Add KEYWORD metavar for --keyword.
8003741 pdf-merge.py: Add -v/--version option
8966242 pdf-merge.py: Fix 'perserving' -> 'preserving' typo in module doctest
5cef5f7 pdfmerge.py: Bump copyright year
71c7be5 pdf-merge.py: Update dump parser for pdftk v1.45
c75bc05 Add `--pdfmarks` option to pdf-merge.py.
abdbff4 Remove `[:-len('�')]` from Unicode parsing of pdftk output in pdf-merge.py.
8efa8f6 Credit Larry Cai <larry.caiyu@gmail.com> for suggesting Unicode handling.
Changes since 0.2:
77a9ba6 pdf-merge.py: Upgrade to Python 3.x
0c3b4c3 pdf-merge.py: Add KEYWORD metavar for --keyword.
8003741 pdf-merge.py: Add -v/--version option
8966242 pdf-merge.py: Fix 'perserving' -> 'preserving' typo in module doctest
5cef5f7 pdfmerge.py: Bump copyright year
71c7be5 pdf-merge.py: Update dump parser for pdftk v1.45
c75bc05 Add `--pdfmarks` option to pdf-merge.py.
abdbff4 Remove `[:-len('�')]` from Unicode parsing of pdftk output in pdf-merge.py.
8efa8f6 Credit Larry Cai <larry.caiyu@gmail.com> for suggesting Unicode handling.
pdf-merge.py: Upgrade to Python 3.x
This probably breaks Python 2.x support. If you can't upgrade your
Python implementation, you'll have to stick to older versions of this
script.
This probably breaks Python 2.x support. If you can't upgrade your
Python implementation, you'll have to stick to older versions of this
script.
pdf-merge.py: Add KEYWORD metavar for --keyword.
This should make the `--keyword KW1 --keyword KW2 ...` usage more
obvious.
This should make the `--keyword KW1 --keyword KW2 ...` usage more
obvious.
pdf-merge.py: Add -v/--version option
pdf-merge.py: Fix 'perserving' -> 'preserving' typo in module doctest
pdfmerge.py: Bump copyright year
pdf-merge.py: Update dump parser for pdftk v1.45
On Tue, Jan 29, 2013 at 08:15:57AM -0800, Tsung-Han Yang wrote:
> I've attached the dumpdata message. I've just installed pdftk on
> windows yesterday. I've attached the diff file with -u, too.
> ...
> It looks like my pdf has some metadata like "InfoBegin" without ":"
> in the line.
The new lines in the dump are due to the addition of m_begin_mark in
pdftk between v1.44.1 and v1.45:
Sid Stewart wrote:
> Added record delimiters to dump_data output to help make parsing
> more reliable.
On Tue, Jan 29, 2013 at 08:15:57AM -0800, Tsung-Han Yang wrote:
> I've attached the dumpdata message. I've just installed pdftk on
> windows yesterday. I've attached the diff file with -u, too.
> ...
> It looks like my pdf has some metadata like "InfoBegin" without ":"
> in the line.
The new lines in the dump are due to the addition of m_begin_mark in
pdftk between v1.44.1 and v1.45:
Sid Stewart wrote:
> Added record delimiters to dump_data output to help make parsing
> more reliable.
posts:mutt-ldap: Add title, README link, and PiPI tag.
posts:mutt-ldap: Mention Wade and Niels and add tags
mutt-ldap.py: Pulled out into its own repository
script-publish.py: Fix double-spacing in <pre> by removing \n
This makes the HTML less readable, but it's not worth the trouble to
make these HTML-readability endlines optional.
This makes the HTML less readable, but it's not worth the trouble to
make these HTML-readability endlines optional.
script-publish.py: Don't print raw strings to the log
Print their repr() instead, since the strings may contain funky
characters and missed control codes.
Print their repr() instead, since the strings may contain funky
characters and missed control codes.
script-publish.py: Add DOCTYPE and <meta charset="..." />
<meta charset="..." /> is new in HTML5.
<meta charset="..." /> is new in HTML5.
script-publish.py: More newlines in the output HTML source
These newlines don't affect the rendered HTML, but they make the HTML
source more readable. The `previous_tail` bit keeps unstyled text
from clobbering newlines that were attached to a preceding <br />.
These newlines don't affect the rendered HTML, but they make the HTML
source more readable. The `previous_tail` bit keeps unstyled text
from clobbering newlines that were attached to a preceding <br />.
script-publish.py: Use -o/--output instead of -t/--target
This seems to be more conventional. As a side benefit, we can now use
-t as the short form of --term.
This seems to be more conventional. As a side benefit, we can now use
-t as the short form of --term.
script-publish.py: Drop empty text chunks
Firefox 10.0.11 was having trouble with:
<span style="..." />
Which, instead of being a no-op, was changing the default styling of
every element after that tag.
Firefox 10.0.11 was having trouble with:
<span style="..." />
Which, instead of being a no-op, was changing the default styling of
every element after that tag.
script-publish.py: Add more control codes (reset1, reset2, ...)
These new codes were from a typescript where I had used the `reset`
command. Xterm doesn't support rs3, but I thought I'd add it for
completeness.
The reset commands are listed in order of decreasing complication
(rs3, rs2, rs1), so that if rs1 is a subset of rs2, we don't match
part of an rs2 entry with our rs1 regexp and leave the rest unmatched.
Ideally, we'll only match the low level escape codes, and compound
escapes like rs2 will be handled automatically by taking care of their
constituent parts. Until we get to that point, we'll need to be
careful about the order of ControlCodes.
These new codes were from a typescript where I had used the `reset`
command. Xterm doesn't support rs3, but I thought I'd add it for
completeness.
The reset commands are listed in order of decreasing complication
(rs3, rs2, rs1), so that if rs1 is a subset of rs2, we don't match
part of an rs2 entry with our rs1 regexp and leave the rest unmatched.
Ideally, we'll only match the low level escape codes, and compound
escapes like rs2 will be handled automatically by taking care of their
constituent parts. Until we get to that point, we'll need to be
careful about the order of ControlCodes.
script-publish.py: Add debugging output for the chunk stream
So you can take a look at the raw stream before we start dumping
stuff. I also made the SGR skipping message more explicit.
So you can take a look at the raw stream before we start dumping
stuff. I also made the SGR skipping message more explicit.
script-publish.py: Also reference the console_codes(4) man page
This page discusses the console escapes and control codes implemented
by the Linux console.
This page discusses the console escapes and control codes implemented
by the Linux console.
script-publish.py: Use Unicode horizontal tabs
Replace the ControlCodeMatch instances with textual tabs, since they
are appropriately handled in `<pre>` blocks.
Replace the ControlCodeMatch instances with textual tabs, since they
are appropriately handled in `<pre>` blocks.
script-publish.py: Allow emulator selection with --term
This means we have to delay ControlCode initialization until the user
has actually selected a terminal. The curses.setupterm() logic also
assumes that the selected terminal doesn't change after the initial
call. For our script, this is a safe assumption. If that assumtion
eventually breaks down, you'll have to cache the last-setup terminal
somewhere, and call setupterm() whenever the current control code's
`.term` doesn't match the cached value.
I also fixed a bug where args.encoding was not being passed to
ScriptServerApp.
This means we have to delay ControlCode initialization until the user
has actually selected a terminal. The curses.setupterm() logic also
assumes that the selected terminal doesn't change after the initial
call. For our script, this is a safe assumption. If that assumtion
eventually breaks down, you'll have to cache the last-setup terminal
somewhere, and call setupterm() whenever the current control code's
`.term` doesn't match the cached value.
I also fixed a bug where args.encoding was not being passed to
ScriptServerApp.
script-publish.py: Add my terminal-escape-to-HTML conversion script
So students can follow along (and easily archive) the instructor's
live terminal session. I convert all the escape sequences that turned
up during some simple messing around (including opening an Emacs
window), but there are surely some escape sequences that I don't catch
yet. Also, the HTML output doesn't understand anything about windows
or cursors, so anything inside a curses application (e.g. Emacs) will
probably come out horribly mangled. Still, things should recover
quickly once you get back to the command line.
If you do want to record curses sessions, you'll probably want to
record scripts with timing information and play them back in an actual
terminal emulator ;).
So students can follow along (and easily archive) the instructor's
live terminal session. I convert all the escape sequences that turned
up during some simple messing around (including opening an Emacs
window), but there are surely some escape sequences that I don't catch
yet. Also, the HTML output doesn't understand anything about windows
or cursors, so anything inside a curses application (e.g. Emacs) will
probably come out horribly mangled. Still, things should recover
quickly once you get back to the command line.
If you do want to record curses sessions, you'll probably want to
record scripts with timing information and play them back in an actual
terminal emulator ;).
font:GoodCityModern-reduced: Update to add a 0x20 (space)
Still compiled with:
$ font-reduce.py -r 0x9f,0x9f -t 0xfc,0xfc -o GoodCityModern-reduced.woff /usr/share/fonts/freefonts/gocmseq_.pfb
but, I just tweaked font-reduce.py, so the new font has a space
character. On Gentoo, gocmseq_.pfb is bundled in the
media-fonts/freefonts package.
Still compiled with:
$ font-reduce.py -r 0x9f,0x9f -t 0xfc,0xfc -o GoodCityModern-reduced.woff /usr/share/fonts/freefonts/gocmseq_.pfb
but, I just tweaked font-reduce.py, so the new font has a space
character. On Gentoo, gocmseq_.pfb is bundled in the
media-fonts/freefonts package.
font:Pecita-reduced: Upgrade to Pecita 4.0.0 alpha and add 0x20 (space)
Upgrade from 3.3.0 to a recently downloaded 4.0.0 alpha and reduce with:
$ font-reduce.py -s 'http://pecita.eu/police-en.php' -l 'OFL (SIL Open Font License)' Pecita.otf
I just tweaked font-reduce.py, so the new font has a space character.
Upgrade from 3.3.0 to a recently downloaded 4.0.0 alpha and reduce with:
$ font-reduce.py -s 'http://pecita.eu/police-en.php' -l 'OFL (SIL Open Font License)' Pecita.otf
I just tweaked font-reduce.py, so the new font has a space character.
font-reduce.py: Add special space handling
For some reason, FontForge doesn't like selecting spaces with
`select(('ranges',),...)`. No idea why, but this works around the
problem.
Reported-by: David Loyall <david.loyall@nebraska.gov>
For some reason, FontForge doesn't like selecting spaces with
`select(('ranges',),...)`. No idea why, but this works around the
problem.
Reported-by: David Loyall <david.loyall@nebraska.gov>
font-reduce.py: Improve ranges/targets error handling and logging
Now the ranges in the fontlog reflect the actual ranges (instead of
being hardcoded to match the defaults). Also raise an explicit error
if the number of ranges does not match the number of targets.
Now the ranges in the fontlog reflect the actual ranges (instead of
being hardcoded to match the defaults). Also raise an explicit error
if the number of ranges does not match the number of targets.
mkogg.py: Upgrade from optparse to argparse
mkogg.py: Prefix imports with underscores for a clean namespace
posts:bugs: mention Hub "pull request warning" pull request
posts:script: write up the terminal-recording utility
emerge-lastsync.py: handle the never-synced case
emerge-lastsync.py: use open() instead of file()
posts:bugs: mention PyCuda relative .gitmodules URLs
posts:bugs: mention distlib depgraph.rst merge into database.rst
posts:git:notes: mention `git remote prune`
emerge-status.sh: cleanup Bash command substitution and testing
emerge-lastsync.py: convert lastsync.pl to Python
I don't always have Perl installed.
I don't always have Perl installed.
emerge-current.sh: strip trailing whitespace
posts:conky:*.sh|*.pl: add emerge scripts for conky.
Scripts are from http://conky.sourceforge.net/screenshots.html:
http://conky.sourceforge.net/emerge-current.sh
http://conky.sourceforge.net/emerge-progress.sh
http://conky.sourceforge.net/emerge-status.sh
http://conky.sourceforge.net/lastsync.pl
Scripts are from http://conky.sourceforge.net/screenshots.html:
http://conky.sourceforge.net/emerge-current.sh
http://conky.sourceforge.net/emerge-progress.sh
http://conky.sourceforge.net/emerge-status.sh
http://conky.sourceforge.net/lastsync.pl
posts:git:notes: show how to tag a file (e.g. junio-gpg-pub)
posts:btrfs: use mkfs.btrfs instead of mkbtrfs
Version 0.19 of btrfs-progs installs the mkfs version.
Version 0.19 of btrfs-progs installs the mkfs version.
posts:bugs: mention html2text initial crowded <pre> patch
posts:mlmmj: use virtual aliases to match mailing list addresses
Also convert tab indents to spaces.
Also convert tab indents to spaces.
name-by-date.sh: change default exif field to 'Date and Time (Original)'
posts:git:notes: add cat-file example
posts:bugs: reference update-copyright pull request for python-kmod.
posts:bugs: add some bugs/patches (including git submodule add --branch).
posts:git:notes: talk about submodule updates and the branch config option.
spectrum.py: add a simple script for plotting power spectra
Sometimes you want more frequency detail than the standard
calibcant-plot.py scripts will give you. This script plots a single
(not chunked and averaged) power spectrum, which increases frequency
resolution at the expense of statistical significance. Example usage
would be something like
$ spectrum.py -f detail.png -t 'Vibration 15' -F 50000 \
calibcant/2012-10-01T16-51-15-calibcant-data.h5 /vibration/15/raw/data
Sometimes you want more frequency detail than the standard
calibcant-plot.py scripts will give you. This script plots a single
(not chunked and averaged) power spectrum, which increases frequency
resolution at the expense of statistical significance. Example usage
would be something like
$ spectrum.py -f detail.png -t 'Vibration 15' -F 50000 \
calibcant/2012-10-01T16-51-15-calibcant-data.h5 /vibration/15/raw/data
compare-unfold.py: add --single option for less crowded plots.
gallery.py: add caption title/alt to images.
gallery.py: escape XML chars (<, &, and >) in caption paragraphs.
gallery.py: adjust CGIGalleryServer._image() to handle no kwargs case.
compare-unfold.py: add bit-level comparison between my and Marisa's programs.
Marisa was using Prof. Yang's LabVIEW unfolder, while I use my
pyafm-based unfolder. The `compare-unfold.py` script allows bit-level
comparison between the two approaches, reading the native data saved
by each approach. Here's an example using local data on my system:
$ ./compare-unfold.py \
~/rsrch/analysis/labview-comparison/gyang/2012-10-01-unfold/ \
~/rsrch/analysis/labview-comparison/wking/unfold/
Marisa was using Prof. Yang's LabVIEW unfolder, while I use my
pyafm-based unfolder. The `compare-unfold.py` script allows bit-level
comparison between the two approaches, reading the native data saved
by each approach. Here's an example using local data on my system:
$ ./compare-unfold.py \
~/rsrch/analysis/labview-comparison/gyang/2012-10-01-unfold/ \
~/rsrch/analysis/labview-comparison/wking/unfold/
git-publish.py: bump to Python 3.
gyang_curve.py: reader for LabVIEW unfolding experiments.
posts:aspire_one: mention sys-kernel/linux-firmware package for brcm firmware.
post:irc-server: Convert Global.NoPAM -> Options.PAM for ngIRCd v18+.
caption: add a captioning script.
scgi-test: update byte/string handling for Python 2/3 compatability.
scgi-test: convert urlparse -> urllib.parse for Python 3.