pypi

Available in a git repository.
Repository: rss2email
Browsable repository: rss2email
Author: W. Trevor King

Since November 2012 I've been maintaining rss2email, a package that converts RSS or Atom feeds to email so you can follow them with your mail user agent. Rss2email was created by the late Aaron Swartz and maintained for several years by Lindsey Smith. I've added a mailing list (hosted with mlmmj) and PyPI package and made the GitHub location the homepage.

Overall, setting up the standard project infrastructure has been fun, and it's nice to see interest in the newly streamlined code picking up. The timing also works out well, since the demise of Google Reader may push some talented folks in our direction. I'm not sure how visible rss2email is, especially the fresh development locations, hence this post ;). If you know anyone who might be interested in using (or contributing to!) rss2email, please pass the word.

Available in a git repository.
Repository: mutt-ldap
Browsable repository: mutt-ldap
Author: W. Trevor King

I wrote this Python script to query an LDAP server for addresses from Mutt. In December 2012, I got some patches from Wade Berrier and Niels de Vos. Anything interesting enough for others to hack on deserves it's own repository, so I pulled it out of my blog repository (linked above, and mirrored on GitHub).

The README is posted on the PyPI page.

Available in a git repository.
Repository: pyassuan
Browsable repository: pyassuan
Author: W. Trevor King

I've been trying to come up with a clean way to verify detached PGP signatures from Python. There are a number of existing approaches to this problem. Many of them call gpg using Python's multiprocessing or subprocess modules, but to verify detached signatures, you need to send the signature in on a separate file descriptor, and handling that in a way safe from deadlocks is difficult. The other approach, taken by PyMe is to wrap GPGME using SWIG, which is great as far as it goes, but development seems to have stalled, and I find the raw GPGME interface excessively complicated.

The GnuPG tools themselves often communicate over sockets using the Assuan protocol, and I'd already written an Assuan server to handle pinentry (originally for my gpg-agent post, not part of pyassuan). I though it would be natural if there was a gpgme-agent which would handle cryptographic tasks over this protocol, which would make the pgp-mime implementation easier. It turns out that there already is such an agent (gpgme-tool), so I turned my pinentry script into the more general pyassuan package. Now using Assuan from Python should be as easy (or easier?) than using it from C via libassuan.

The README is posted on the PyPI page.

Available in a git repository.
Repository: pygrader
Browsable repository: pygrader
Author: W. Trevor King

The last two courses I've TAd at Drexel have been scientific computing courses where the students are writing code to solve homework problems. When they're done, they email the homework to me, and I grade it and email them back their grade and comments. I've played around with developing a few grading frameworks over the years (a few years back, one of the big intro courses kept the grades in an Excel file on a Samba share, and I wrote a script to automatically sync local comma-separated-variable data with that spreadsheet. Yuck :p), so I figured this was my change to polish up some old scripts into a sensible system to help me stay organized. This system is pygrader.

During the polishing phase, I was searching around looking for prior art ;), and found that Alex Heitzmann had already created pygrade, which is the name I under which I had originally developed my own project. While they are both grade databases written in Python, Alex's project focuses on providing a more integrated grading environment.

Pygrader accepts assignment submissions from students through its mailpipe command, which you can run on your email inbox (or from procmail). Students submit assignments with an email subject like

[submit] <assignment name>

mailpipe automatically drops the submissions into a student/assignment/mail mailbox, extracts any MIME attachments into the student/assignment/ directory (without clobbers, with proper timestamps), and leaves you to get to work.

Pygrader also supports multiple graders through the mailpipe command. The other graders can request a student's submission(s) with an email subject like

[get] <student name>, <assignment name>

Then they can grade the submission and mail the grade back with an email subject like

[grade] <student name>, <assignment name>

The grade-altering messages are also stored in the student/assignment/mail mailbox, so you can peruse them later.

Pygrader doesn't spawn editors or GUIs to help you browse through submissions or assigning grades. As far as I am concerned, this is a good thing.

When you're done grading, pygrader can email (email) your grades and comments back to the students, signing or encrypting with pgp-mime if either party has configured a PGP key. It can also email a tab-delimited table of grades to the professors to keep them up to speed. If you're running mailpipe via procmail, responses to grade request are sent automatically.

While you're grading, pygrader can search for ungraded assignments, or for grades that have not yet been sent to students (todo). It can also check for resubmissions, where new submissions come in response to earlier grades.

The README is posted on the PyPI page.

Available in a git repository.
Repository: update-copyright
Browsable repository: update-copyright
Author: W. Trevor King

A few years ago I was getting tired of having missing or out-of-date copyright blurbs in packages that I was involved with (old license text, missing authors, etc.). This is important stuff, but not the kind of thing that is fun to maintain by hand. I wrote a script for bugs everywhere that automated the process, using the version control system to extract lists of authors and dates for each file. The script was great, so I ported it into a few other projects I was involved in.

This month I realized that it would be much easier to just break the script out into its own package, and only maintain a config file in each of the projects that use it. I don't know why this didn't occur to me years ago :p. Anyhow, here it is! Enjoy.

The README, with usage details, is posted on the PyPI page.

Available in a git repository.
Repository: h5config
Browsable repository: h5config
Author: W. Trevor King

Since the number of packages mooching off pypiezo's configuration scheme was growing, I've split it out into it's own package. Now there's a general package for all your HDF5-based configuration needs.

The README is posted on the PyPI page.

Available in a git repository.
Repository: pypid
Browsable repository: pypid
Author: W. Trevor King

I've just finished rewriting my PID temperature control package in pure-Python, and it's now clean enough to go up on PyPI. Features:

Backend-agnostic architecture. I've written a first-order process with dead time (FOPDT) test backend and a pymodbus-based backend for our Melcor MTCA controller, but it should be easy to plug in your own custom backend.
The general PID controller will automatically tune your backend using any of a variety of tuning rules.

The README is posted on the PyPI page.

Available in a git repository.
Repository: insider
Browsable repository: insider
Author: W. Trevor King

Insider is a little Django app I wrote to help my brother, Garrett, track insider trading with a simple, familiar web interface. It's a pretty simple app, partly thanks to Bradley Ayers' django-tables2, which does the table formatting. Just goes to show that a good scripting language and framework make developing simple apps a breeze!

The README is posted on the PyPI page.

Available in a git repository.
Repository: calibcant
Browsable repository: calibcant
Author: W. Trevor King

Here is my Python module for AFM cantilever calibration via the thermal tune method.

The README is posted on the PyPI page, and you might also be interested in the package dependency graph generated with Yu-Jie Lin's PDepGraph.py:

# python PDepGraph.py -o calibcant.dot \
    -D matplotlib,scipy,numpy,pyyaml,h5py,python,eselect-python \
    calibcant
# dot -T svg -o calibcant.svg calibcant.dot

Thermal calibration requires three separate measurements: photodiode sensitivity (via surface bumps), fluid temperature (estimated, or via thermocouple), and thermal vibration (watching the cantilever vibrate in far from the surface). The calibcant package takes repeated measurements (statistics.png) of each of these parameters to allow estimation of statistical uncertainty:

# calibcant-analyze.py calibcant/examples/calibration.h5
...
... variable (units)         : mean +/- std. dev. (relative error)
... cantilever k (N/m)       : 0.0629167 +/- 0.00439057 (0.0697838)
... photo sensitivity (V/m)  : 2.4535e+07 +/- 616119 (0.0251118)
... T (K)                    : 295.15 +/- 0 (0)
... vibration variance (V^2) : 3.89882e-05 +/- 1.88897e-06 (0.0484497)
...

While this cannot account for systematic errors, calibration numbers are fairly meaninless without at least statistical error estimates.

Extracting the photodiode sensitivity and thermal deflection variance from the raw data can vary a suprising amount depending on your cantilever/photodiode linearity and drift and signal/noise ratio in the vibration data. To help deal with this, calibcant provides a choice of models for fitting each measurement type.

The contact region of surface bumps can be fit with either linear or quadratic models. Here is an example of a single surface bump fit with a quadratic model. The green line is the initial guess (before fitting), the red line is the final model (after fitting), and the blue dots are measured data points. The red dots in the bottom panel are the residual, which looks cubic because we've subtracted a quadratic model.

Extracting the thermal vibration variance is also trickier than it might seem. Fitting the data in frequency-space to a Lorentzian (Breit-Wigner?) model helps filter out low frequency drift, as well as white noise from the measurement equipment.

(1)

PSD (x, ω) = \frac{2 k_{B} T β}{π m [(ω_{0}^{2} - ω^{2})^{2} + β^{2} ω^{2}]} + W .

where $β$ and $ω_{0}$ come from the damped-forced harmonic oscillator equation of motion

(2)

\ddot{x} + β \dot{x} + ω_{0}^{2} x = \frac{F (t)}{m},

the cantilever's effective mass is $m$ , and $W$ is an optional white-noise offset.

Here is an example of a one-second thermal vibration fit with the offset Breit-Wigner model. The top panel is the time-series deflection voltage (bits vs sample index). The center pannel is a histogram of the deflection voltage, showing an approximately Gaussian distribution. The bottom panel shows the power spectral density fit (red dots) fit with an offset Breit-Wigner model (blue curve). The horizontal blue line marks the white-noise offset, and the vertical blue line marks the resonant frequency. Points outside the light blue region were not considered during the fitting. This allows us to isolate the cantilever's thermal vibration from other noise sources, which leads to more accurate and reproducible spring constant estimates.

Finally, all data and analysis results are stored in the standard, portable HDF5 file format, so it's easy to reanalyze earlier calibration data with different models if you decide to do so at a later date, or just look back and see exactly what calculations went into your spring constant calibration in the first place.

I tried to build calibcant on top of a chain of packages to make swapping out the hardware interface easier, but Comedi is at the bottom of the current chain, so it may be hard to use this package if you're not running Linux. If you're not running Linux, but are interested in getting calibcant working on your system anyway, send me an email! I'd be happy to help generalize calibcant, but it's hard for me to imagine hardware control from Windows (do people run experiments from Macs?). If you've figured that part out, I can probably graft calibcant onto your system.

Available in a git repository.
Repository: pyrisk
Browsable repository: pyrisk
Author: W. Trevor King

Play Risk (or similar games) over email!

The README is posted on the PyPI page.