Eric Neilsen's notebook

Table of Contents

1 Introduction

These are "exports" of my personal notes, which I keep in emacs org-mode, lightly cleaned up (but not exhaustively explained) for wider consumption. Notes directly connected with DES can be found here.

2 Older long notes

3 Older short notes

3.1 GNU Screen

Screen is a great but obscure tool for managing terminals. Some features are a bit superfluous if you are using X windows, but the feature that allows you to get a execute a terminal command in a screen, disconnect it from the physical terminal (say, the node in my office), and then reconnect from a completely different terminal (my laptop, or my desktop at home) without disturbing the processing running in that terminal is still enormously helpful.

There is a User's manual, a guide, a beginner's turorial, and a quick reference card at assorted other places, but not on the official screen home page.

These are the commands I regularly forget:

\^a [ start scrollback
\^a ] end scrollback
. start marking text
> end marking text
\^a ] paste text
\^a S split screen
\^a TAB move between splits
\^a Q unsplit
\^a H start/stop logging

3.2 Quick-and-dirty methods of looking at FITS headers

FITS headers are in ASCII, but need line breaks added. I also want to avoid sending the actual unprocessed data into the terminal. Most linux and other UNIX-like systems have hexdump, which can be used to look at a FITS header:

bash$ hexdump -e '80/1 "%_p" "\n"' | less

If you are on a big-endian machine, hexdump can also be used to look at the data.

3.3 Getting a quick approximation of the Modified Julian Date (MJD)

To get the MJD Now:

bash$ echo "40587+$(date +%s)/86400" | bc -l

To get the MJD at some specific date:

bash$ echo "40587+$(date -d '06/05/2007' +%s)/86400" | bc -l

3.4 Switch between QWERTY and Dvorak keyboard mappings in linux

bash$ setxkbmap us
bash$ setxkbmap dvorak

3.5 Change the default printer used by flpr

flpr is an lpr-like utility used at Fermilab.

bash$ export FLPQUE=WH7W_XEROX7400_D
bash$ export FLPHOST=fnprt

3.6 Something to link to when referencing a book

When referencing a book, my favorite thing to link to is the Library of Congress Online Catalog LCCN permalink for the book, for example: would be the link to include for Steward's Fourier optics book.

An alternative might be to use a Digital Object Identifier's (DOI), but I haven't found a good way to track down DOIs for arbitrary books (and books may be less likely to have DOI entries than LOC entries).

3.7 Fixing raw SDSS FITS files

The SDSS DA system sometimes violates the FITS standard when representing unsigned integers. These images start with

SIMPLE  =                    F / FIT STANDARD

in the fits header.

The easiest way to fix this is to use Dervish, which knows how to read the bad files, but writes files correctly. For example:

bash$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_21/Linux/etc/dervishStartup.tcl:
dervish> hdrReadAsFits [hdrNew]
dervish> regReadAsFits [regNew -type U16]
dervish> mv
dervish> hdrWriteAsFits h0 mdR-00114174.fits
dervish> regWriteAsFits h1 mdR-00114174.fits

The file mdR-00114174.fits is now a standard FITS file.

3.8 Printing PDF overheads, many to a page

A presentation, printed one slide to a page, is a waste of paper and irritating to read. psnup, part of psutils, converts a postscript file so that it will print many to a page. The rotation and ordering of slides on the page can come out badly without the appropriate paramateres. Here is an example of one that works, starting with a postscript file producted by the Adobe Acrobat reader:

bash$ psnup -nup 6 -pletter -l -c

(The result looked strange in gv, but it prints correctly on paper.)

3.9 Links to Astronomy Papers

  1. Bibliographic Identifiers (bibcodes)

    Many sources of on-line astronomy information, including ADS, NED, and Simbad, have adopted a common format for representing references in a way that can be easily incorporated into URLs. Most bibcodes can be constructed thus:

    bash$ year=2007
    bash$ journal=ApJ
    bash$ volume=662
    bash$ qualifier=L
    bash$ firstPage=51
    bash$ lastNameChar=A
    bash$ printf "%d%- 5s% 4d%1s% 4d%s" $year $journal $volume $qualifier $firstPage $lastNameChar | tr ' ' .

    For most papers, there is no qualifier, and the character is a period. An "L" is used for letters.

    When page numbers go over 1000, the qualifier is dropped and the page component expands to 5 places. When the page number goes higher than 10000, strange things happen.

    The official definition is in Information & on-line data in astronomy, and is available online here.

    This page gives a table of journal abbrevations.

    This form lets a user validate a bibcode.

    To see the NASA/ADS abstract and link for a paper, you can plug the bibcode into this template for the URL:

    Where %s represents the bibcode.

  2. Digital Object Identifiers (DOIs)

    An alternative might be to use a Digital Object Identifier (DOI). You can construct a DOI link by appending the DOI to For example, the DOI of the SDSS EDR paper is 10.1086/324741 so the URL would be You can look up a DOI here, or get it from the ADS.

3.10 Checking disk and CPU usage on a linux box

Obviously, start with ps or top. iostat and sar are also useful:

bash$ DEVICE=/dev/sda
bash$ INTERVAL=2
bash$ COUNT=3
bash$ iostat -dkx $DEVICE $INTERVAL $COUNT
Linux 2.6.9-55.0.2.ELsmp (   01/30/2009

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda          0.06   1.22  1.27  0.58   15.62   14.39     7.81     7.20    16.28     0.03   14.87   1.31   0.24

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda          0.00   0.00  0.00  1.51    0.00   12.06     0.00     6.03     8.00     0.00    3.00   1.00   0.15

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda          0.00   0.00  0.00  0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

bash$ iostat -ct $INTERVAL $COUNT
Linux 2.6.9-55.0.2.ELsmp (   01/30/2009

Time: 09:48:38 AM
avg-cpu:  %user   %nice    %sys %iowait   %idle
           2.45    0.00    0.08    0.12   97.35

Time: 09:48:40 AM
avg-cpu:  %user   %nice    %sys %iowait   %idle
           1.50    0.00    0.00    0.00   98.50

Time: 09:48:42 AM
avg-cpu:  %user   %nice    %sys %iowait   %idle
           1.25    0.00    0.00    0.00   98.75

sar does something similar, but with cumulative activity counters.

3.11 Check what processes are using a disk (eg so it can be dismounted)

The two main utilitios are lsof and fuser.

bash$ lsof /media/wd120
bash    30408 neilsen  cwd    DIR   8,17 4096 6602755 /media/wd120/neilsen/devel

The -k option of fuser can be used to automatically kill any processes using a resource.

3.12 Copying disk partition using dd

A bit-by-bit backup of a partition can be made thus:

bash# dd if=/dev/sda1 conv=sync,noerror bs=1k | gzip -c > /var/backup/sda1-backup.gz

Note that you can sometimes do something even simpler:

bash# gzip -c < /dev/sda1 > /var/backup/sda1-backup.gz

but this will not work properly if there are bad blocks (or other errors) on the partition.

This can take a while, so if you have pv, it's nice to use here. Here is the skeleton of a script to make a backup tracking progress:

PART_SIZE=$(df -k $PARTITION | tail -1 | awk '{print $2}')
dd if=$PARTITION conv=sync,noerror bs=1k | pv -s ${PART_SIZE}k | gzip -c > $IMAGE_FILE

The resulting progress bar is close to being correct, but not exactly. I don't understand why.

There exists a GNU tool, ddrescue, which is reported to be similar but handle damaged disks better. I have not tried it yet.

Seb Wills has a page with much more information on low level backup and recovery in linux.

3.13 Installing my labtec camera on a linux box

The driver is provided by, but this seems to have evaporated. It appears to be available from the spca50x sourceforge page, but I haven't tried the version available there.

When if first compiled, it failed. Looking at the source where it fails, there is an ifdef/else pair in which it was apparently choosing the wrong branch. After recompiling it forcing each branch, I found the one that worked.

as root, do:

bash# modprobe videodev
bash# insmod spca50x.o

I then downloaded the mplayer source rpm and rebuilt it, and installed all the RPM's generated from a rebuild of it EXCEPT for mplayer-gui, which lacked dependencies. It seems not to be needed, though.

Once installed, I ran:

bash$ mplayer tv:// -tv  driver=v4l:width=352:height=288:outfmt=yv12:device=/dev/video0

and was able to see the images.

The following also works:

camorama -x 288 -y 352 -d /dev/video0

3.14 Including equations in Scribus

There was a Google Summer of Code project to do this, and relevant pages in the Scribus bug tracker and wiki page suggest that the project was completed successfully, but my version of scribus doesn't have it.

Instead, I need to use one of two methods. First, write the equations in latex or tex, and produce a postscript file. Then, import the postscript into the scribus document (File->Import->Import EPS), select the imported object, ungroup it, select the specific equation in the document, and regroup it (Item->group).

Alternately, I can write the equation in mathML, convert it into SVG using jEuclid, and import the resultant SVG into scribus.

3.15 Generate mathML from LaTeX

TeX4ht includes a utility, mzlatex, which will do the conversion. On my linux distribution this is part of the tetex-tex4ht rpm. I often find I want to run xmllint on the result, so the commands look something like this:

bash$ mzlatex file.tex
bash$ xmllint --format file.xml > file.mml

3.16 pv pipe status viewer examples

Track progress making md5sums:

$ cat * | md5sum
6ef2106208e536dd8256a7801ef53778  -
$ pv * | md5sum
20.1MB 0:00:02 [10.3MB/s] [===>                               ] 14% ETA 0:00:11

Track stages in a pipe, labeling each stage:

pv -cN source *.gz | gunzip | pv -N gunzip | md5sum

It can be used in combination with zenity. This example makes a disk image and shows the progress in an X window:

PART_SIZE=$(df -k $PARTITION | tail -1 | awk '{print $2}')
(dd if=$PARTITION conv=sync,noerror bs=1k |
 pv -n -s ${PART_SIZE}k |
 gzip -c > $IMAGE_FILE) 2>&1 |
zenity --progress --percentage=0 --title="disk dump progress" --text="Making disk dump"

3.17 Die-roll password generation tables

Table 1:

  1 2 3 4 5 6
1 1 2 3 4 5 6
2 7 8 9 0 a b
3 c d e f g h
4 i j k l m n
5 o p q r s t
6 u v w x y z

Table 2:

  1 2 3 4 5 6
1 ' , ; . / =
2 [ ] \ - A B
3 C D E F G H
4 I J K L M N
5 O P Q R S T
6 U V W X Y Z

Table 3:

  1 2 3 4 5 6
1 ! @ # $ % ^
2 & * ( ) { }
3 : " < > ~ ?
4 + ` " SP _ TB

An alternative is to use the Dicewords word list.

3.18 Multicolumn CSS with text that can flow between columns

This will be a CSS3 feature, the draft specification of which is here. Both firefox and safari have implemented it, but each adds its own special prefix. Explorer does not implement it at all. The specifications below will work with either.

This example sets an individual column's width to 40em, and automatically adjusts the number of columns:

body {
  [-moz- | -webkit-]column-width: 40em;

and this one sets the number of columns to 3, and sets their width automatically:

body {
  [-moz- | -webkit-]column-count: 3;

See quirksmode for more examples.

3.19 git

  1. git commands

    Essential git documentation can be found here. The default branch is "master".

    git init create a new repository
    git clone <repo> clone a repository into a new directory
    git pull update local repository with content of repository originally cloned
    git pull /some/dir foo pull changes made in copy /some/dir, branch foo into current copy
    git remote add foo /dir/of/foo establish shorthand for repository at /dir/of/foo
    git fetch foo update remote tracking branch of repo with "foo" shorthand
    git fetch foo; git merge foo/bar merge repo w/ shorthand foo, branch bar into current
    git tag Foo tag the current version "Foo"
    git status show status of files (including a list of untracked files)
    git add <file> start tracking a new file
    git commit -a -m "foo" commit all changes with comment "foo"
    git log show history
    git show XXXXX… show details of commit (using index from git log)
    git log -p show history with diffs
    git log –stat –summary show history with additional info
    git tag v1.0 XXX… tag vesion ofter commit XXX v1.0
    git revert XXX revert to previouse version XXX
    git reset –hard HEAD undo all unchecked-in changes
    git branch list existing branches
    git branch foo create branch foo
    git branch foo v1.2 make foo branch branching at version v1.2
    git checkout foo switch to branch foo
    git merge foo merge branch foo into current branch
    gitk view histore graphically
    git push origin master like cvs commit (copies to master branch of origin repo)

    To access a repo through ssh, use a url like this:

  2. starting a project
    git init
    git add .
    git commit

3.20 Using SVG produced by dia in inkscape

Inkscape doesn't like the SVG produced by dia, in particular it can't handle the way dia specifies text.

The positions in dia don't have units, but inkscape needs them. For example,

<text x="15.2" y="19.9225">Foo</text>

needs to be:

<text x="15.2cm" y="19.9225cm">Foo</text>

A transform to do this:

<?xml version="1.0" ?>
<xsl:stylesheet version="1.0" xmlns:xsl=""
<xsl:output method="xml" doctype-public="-//W3C//DTD SVG 1.0//EN" doctype-system="" />

<xsl:template match="/|node()|@*">
   <xsl:apply-templates select="node()|@*"/>

<xsl:template match="svg:text">
 <xsl:attribute name="x"><xsl:value-of select="@x * 0.8"/>cm</xsl:attribute>
 <xsl:attribute name="y"><xsl:value-of select="@y * 0.8"/>cm</xsl:attribute>
 <xsl:attribute name="font-size">22.7</xsl:attribute>
 <xsl:value-of select="."/>


The 0.8 is empirical; I do not know why this factor is needed, and it isn't exactly right. Other text attributes also get lost, but I haven't yet figured out how to get scribus to read them.

3.21 Spell checking in scribus

Scribus does not come with its own spell checker (although I am told that future releases will). In the mean time, use aspell:

aspell --encoding=utf-8 --mode=sgml --add-sgml-check=ch check foo.sla

To print an A0 to a Letter printer, use pdfposter to convert the A0 pdf into an 18 page letter document.

3.22 Extract a few pages from a PDF file

Use gs:

gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER -dFirstPage=8 -dLastPage=11 -sOutputFile=foo8-11.pdf foo.pdf

Author: Eric H. Neilsen, Jr.

Emacs 24.3.1 (Org mode 8.2.5c)