unfolding disasters/ tags/ fun

Fun pages! Nothing serious here.

RSS Atom
Ultimo instant and proximo

Looking into old birth, death, and marriage records, I ran across dates such oas "on the 12th instant". Looking this up it turns out to be shorthand for "the current month". Related terms:

  • Ultimo (the previous month, from Latin ultimo mense, abbreviated ult)
  • Instant (the current month, from Latin instante mense, abbreviated inst)
  • Proximo (the next month, from Latin proximo mense, abbreviated prox)
Posted
Tektronix emulation and Gnuplot

I just discovered that gnuplot has an xterm terminal output mode which uses xterm's tektronix emulator (since gnuplot 4.2.6). This is useful if you have logged into a remote machine and want to plot something, but you didn't have the foresight to use ssh -Y to bring along a connection to your X server. I used to use

gnuplot> set term dumb

but often the low resolution available with standard terminal characters left the resulting output almost unintelligible. With

gnuplot> set term xterm

your xterm will pop up a new window in tektronix mode, in which you get very nice monochrome graphics.

Besides the “forgot to tunnel X” use case outlined above, this output would also be useful if you wanted to avoid exposing yourself to X vulnerabilies while logging into an untrusted machine.

Obviously, you'll have to be running xterm for any of this to work ;).

Posted
Compose key

Add the compose key to your Xmodmap file with something like

keycode 134 = Multi_key

The compose key has it's own unicode symbol: ⎄, which I'll use in the following composition tables.

Accents

  • ⎄`a à
  • ⎄'a á
  • ⎄^a â
  • ⎄ca ǎ
  • ⎄ba ă
  • ⎄oa å
  • ⎄!a ạ
  • ⎄.a ȧ
  • ⎄"a ä
  • ⎄~a ã
  • ⎄_a ā
  • ⎄;a ą
  • ⎄,c ç

Weird letters

  • ⎄AE Æ
  • ⎄ae æ
  • ⎄ss ß
  • ⎄TH Þ
  • ⎄th þ

Currency

  • ⎄=C €
  • ⎄|c ¢
  • ⎄-L £

Typography

  • ⎄<< «
  • ⎄>> »
  • ⎄"< “
  • ⎄"> ”
  • ⎄'< ‘
  • ⎄'> ’
  • ⎄.. …
  • ⎄p! ¶
  • ⎄os §
  • ⎄?? ¿
  • ⎄!! ¡
  • ⎄--- — (em dash)
  • ⎄--. – (em dash)

Math

  • ⎄oo °
  • ⎄^0 ⁰
  • ⎄^1 ¹
  • ⎄_1 ₁
  • ⎄12 ½
  • ⎄+- ±
  • ⎄xx ×
  • ⎄:- ÷
  • ⎄/o ø
  • ⎄mu µ

Business

  • ⎄oc ©
  • ⎄or ®
  • ⎄tm ™

Emoticons

  • ⎄:) ☺
  • ⎄:( ☹

As you can imagine, the list goes on and on. The compositing system depends on your application (see the Ubuntu wiki for Gnome notes), but if you're using the X Input Method (XIM), your compose table depends on your locale:

$ echo $LANG
en_US.UTF-8
$ grep $LANG\$ /usr/share/X11/locale/compose.dir
en_US.UTF-8/Compose             en_US.UTF-8
en_US.UTF-8/Compose:            en_US.UTF-8
$ less /usr/share/X11/locale/en_US.UTF-8/Compose

If you want to customize your compose keys, just add your own rules to an ~/.XCompose file:

$ cat ~/.XCompose
include "%L"

<Multi_key> <h> <o> <m> <e> : "http://tremily.us/"  # Home page

Read Compose(5) for more information. You may need to log out and log back in (or use the newgrp trick) for your new ~/.XCompose file to take effect.

Posted
GnuPG maintenance

It's a good idea to periodically replace old PGP encryption keys to minimize the amount of data exposed by cracking the old key.

$ gpg --expert --edit-key F15F5BE8
…
pub  1024D/F15F5BE8  created: 2008-08-09  expires: 2011-08-08  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048g/42407C74  created: 2008-08-09  expired: 2009-08-09  usage: E   
sub  2048g/4DA3FC0B  created: 2009-07-26  expired: 2010-08-08  usage: E   
sub  1024D/EB357E60  created: 2009-07-26  expired: 2010-08-08  usage: S   
[ultimate] (1). William Trevor King <wking@drexel.edu>
[ultimate] (2)  William Trevor King <tvrkng@gmail.com>

The usage characters are:

  • e = encrypt/decrypt
  • s = sign
  • c = certify (sign another key)
  • a = authenticate (e.g. log in to SSH with a PGP key)

See doc/DETAILS in the GnuPG source directory for details on the output format (and the related colon listing format).

If your primary key has expired, you can extend its expiration time with

gpg> expire

Note that my encryption keys have expired. This makes it hard for people to send me encrypted mail. Create a new encryption key with

gpg> addkey

Answering the prompts as you see fit (I usually pick Elgamal for encryption). You can also add signing keys with addkey (I usually pick RSA for signing, since DSA keys are limited to 1024 bits, see ssh-keygen(1)).

There doesn't seem to be much to differentiate Elgamml vs. RSA for encryption. With the --expert mode, you can select

RSA (set your own capabilities)

so that's what I do (since then I only need one subkey for all tasks).

Several gpg operations require a particular subkey to be selected. Use key to select subkeys by index (marked with a *):

gpg> key 1

pub  1024D/F15F5BE8  created: 2008-08-09  expires: 2012-05-24  usage: SC  
                     trust: ultimate      validity: ultimate
sub* 2048g/42407C74  created: 2008-08-09  expired: 2009-08-09  usage: E   
sub  2048g/4DA3FC0B  created: 2009-07-26  expired: 2010-08-08  usage: E   
sub  1024D/EB357E60  created: 2009-07-26  expired: 2010-08-08  usage: S   
sub  2048g/3FB721E8  created: 2011-05-25  expires: 2012-05-24  usage: E   
sub  2048R/9CADC4D9  created: 2011-05-25  expires: 2012-05-24  usage: S   
[ultimate] (1). William Trevor King <wking@drexel.edu>
[ultimate] (2)  William Trevor King <tvrkng@gmail.com>

If you get confused, there's also a help command.

Save and quit when you're done:

gpg> save

Once you've got your key all fixed up, upload the new version to your chosen keyserver:

$ gpg --send-keys F15F5BE8

You probably also want to post your new key somewhere on your website:

$ gpg --export --armor -o ~/.gnupg/pubkey.txt F15F5BE8
$ scp ~/.gnupg/pubkey.txt you@somewhere:public_html/pubkey.txt

Checking signatures

Here are some quick notes on checking signatures:

$ gpg --check-sigs F15F5BE8

will list the status of signatures for which you have the signing key in your keyring. However, if you are missing one of the signing keys, you may get a message like

10 signatures not checked due to missing keys

If you run

$ gpg --list-sigs F15F5BE8

you'll see all the signatures, and you can use the usual gpg --recv-key KEYID to check out the ones you don't have.

Adding user IDs

If you get a new email account, you'll want to add it to your key. 

$ gpg --edit-key F15F5BE8
gpg> adduid
…

Optionally make the new ID your primary ID.

gpg> uid 3
gpg> primary

Finall, save your changes.

gpg> save

Don't worry about the [unknown] trust level next to your new ID. Once you've saved the key, it will change to [ultimate]. I imagine the initial [unknown] listing is because you haven't officially confirmed the new ID's signature by saving your changes.

Posted
pyproj

pyproj is a Python wrapper around PROJ.4. Here's a quick walkthrough.

Initialize a geodetic converter:

>>> from pyproj import Geod
>>> g = Geod(ellps='clrk66')

where ellps='clrk66' selects Clarke's 1866 reference ellipsoid. help(Geod.__new__) gives a list of possible ellipsoids.

Calculate the distance between two points, as well as the local heading, try

>>> lat1,lng1 = (40.7143528, -74.0059731)  # New York, NY
>>> lat2,lng2 = (49.261226, -123.1139268)   # Vancouver, Canada
>>> az12,az21,dist = g.inv(lng1,lat1,lng2,lat2)
>>> az12,az21,dist
(-59.10918706123901, 84.99453463527395, 3914198.2912370963)

which gives forward and back azimuths as well as the geodesic distance in meters. Not that longitude comes before latitude in the these pyproj argument lists.

Calculate the terminus of a geodesic from an initial point, azimuth, and distance with:

>>> lng3,lat3,az3 = g.fwd(lng1,lat1,az12, dist)
>>> lat3,lng3,az3
(49.26122600306212, -123.11392684861474, 84.99453467574762)

Plan your trip with:

>>> pts = g.npts(lng1,lat1,lng2,lat2,npts=5)
>>> pts.insert(0, (lng1, lat1))
>>> pts.append((lng2, lat2))
>>> import numpy
>>> npts = numpy.array(pts)
>>> npts
array([[ -74.0059731 ,   40.7143528 ],
       [ -80.93566289,   43.52686057],
       [ -88.48167748,   45.87969433],
       [ -96.61187851,   47.6930911 ],
       [-105.22271807,   48.89347605],
       [-114.13503215,   49.42510006],
       [-123.1139268 ,   49.261226  ]])

To plot the above New York to Vancouver route on a flat map, we need a Proj instance:

>>> from pyproj import Proj
>>> awips221 = Proj(proj='lcc', R=6371200, lat_1=50, lat_2=50,
...     lon_0=-107, ellps='clrk66')
>>> awips218 = Proj(proj='lcc', R=6371200, lat_1=25, lat_2=25,
...     lon_0=-95, ellps='clrk66')  #x_0=-llcrnrx,y_0=-llcrnry)

#llcrnrlon,llcrnrlat are lon and lat (in degrees) of lower
#    left hand corner of projection region.

where proj='lcc' selects the Lambert conformal conic projection for the x/y points, and ellps='clrk66' selects the reference ellipsoid for the lat/lng coordinates. The other coordinates are LCC parameters that select the AWIPS 221 and AWIPS 226 projections respectively (lat_1 corresponds to Latin1, lat_2 corresponds to Latin2, and lon_0 corresponds to Lov; see this description of the two-standard-parallel LCC and its PROJ.4 parameters).

Convert our lat/lng pairs into grid points:

>>> awips221(lng1, lat1)
(2725283.842678774, 5823260.730665273)
>>> x221,y221 = awips221(npts[:,0], npts[:,1])
>>> # xy221 = numpy.concatenate((a1, a2, ...), axis=0)  # numpy-2.0
>>> xy221 = numpy.ndarray(shape=npts.shape, dtype=npts.dtype)
>>> xy221[:,0] = x221
>>> xy221[:,1] = y221
>>> xy221
array([[ 2725283.84267877,  5823260.73066527],
       [ 2071820.3526011 ,  5892518.49630526],
       [ 1422529.71760395,  5967565.49899035],
       [  775650.03731228,  6046475.43928965],
       [  129946.46495299,  6127609.80532071],
       [ -515306.57275941,  6209785.69230076],
       [-1160447.80254759,  6292455.41884832]])

Finally, you can convert points from one projection to another.

>>> from pyproj import transform
>>> x218,y218 = transform(awips221, awips218, x221, y221)
>>> xy218 = numpy.ndarray(shape=npts.shape, dtype=npts.dtype)
>>> xy218[:,0] = x218
>>> xy218[:,1] = y218
>>> xy218
array([[ 1834251.59591561,  4780900.70184736],
       [ 1197541.13209718,  5028862.9881648 ],
       [  542391.04388716,  5258740.71523961],
       [ -131577.34942316,  5464828.45934687],
       [ -822685.42269932,  5641393.59760613],
       [-1527077.85176048,  5783597.16169582],
       [-2239159.34620498,  5888495.91009021]])

Another useful coordinate system is the Universal Transverse Mercator projection which slices the world into zones.

>>> p = Proj(proj='utm', zone=10, ellps='clrk66')

Putting everything together, here's a route map based on digital lat/lng pairs stored in a text file:

>>> from numpy import array
>>> from pylab import plot, show
>>> from pyproj import Geod, Proj
>>> latlng = array([[float(x) for x in ln.split()]
...                for ln in open('coords', 'r')
...                if not ln.startswith('#')])
>>> g = Geod(ellps='WGS84')
>>> az12s,az21s,dists = g.inv(latlng[:-1,1], latlng[:-1,0],
...                           latlng[1:,1], latlng[1:,0])
>>> print('total distance: %g m' % dists.sum())
total distance: 2078.93 m
>>> mlng = latlng[:,1].mean()
>>> zone = int(round((mlng + 180) / 6.))
>>> p = Proj(proj='utm', zone=zone, ellps='WGS84')
>>> xs,ys = p(latlng[:,1], latlng[:,0])
>>> lines = plot(xs, ys, 'r.-')
>>> show()

I've written up a simple script using this approach: maproute.py. I've also written up a simple script to draw a map with labeled points: maplabel.py.

Note that you can easily get lat/lng pairs using geopy (ebuild in my Gentoo overlay):

>>> import geopy
>>> g = geopy.geocoders.Google()
>>> place1,(lat1,lng1) = g.geocode("New York, NY")
>>> place2,(lat2,lng2) = g.geocode("Vancouver, Canada")
>>> place1,(lat1,lng1)
(u'New York, NY, USA', (40.7143528, -74.0059731))
>>> place2,(lat2,lng2)
(u'Vancouver, BC, Canada', (49.261226, -123.1139268))

If you're looking for a more compact C++ package for geographic conversions, GeographicLib looks promising.

find_duplicates

I've reorganized my music a few times, and it took me a while to get organized enough to want a single directory tree that held everything I owned. Every once and a while I go through the junk drawer and move a few more songs into the "official" tree, checking their metadata and whatnot. I was getting annoyed at finding duplicate songs in several junk drawers, so I wrote up a little script, find duplicates.py which can compare SHA1s for files in two trees and remove any duplicates from the lesser tree. Now there is a lot less junk to merge into my official tree.

Sometimes you want to do something more effective than printing duplicates, but more subtle than removing them. To give you this flexibility, I've added the --one-line option, which you can use along these lines:

$ find_duplicates.py dir_a dir_b --one-line | while read -a DUPS; do mv "${DUPS[1]}" "${DUPS[0]}".dup; done