Dervish is a collection of libraries and corresponding
tcl bindings on
which much of the SDSS software infrastructure is built. This document
demonstrates some of the most basic things that can be done with it
using the tcl interface. I wrote it after I had been away from using
it for a while, and found myself needing reminders. I expect to need to
do this occasionally, and this document is supposed to help with that.
The most useful way to think about Dervish is as a C library with
some tcl bindings. If you think about it that way, its workings are
fairly intuitive. If you think of it as tcl extended to have a
variety of features that astronomers might want, the results may be
frustrating.
These notes assume a previous knowledge of
tcl. See the official
tcl tutorial of you need help there. Note that most of the
resources on the web are designed for tcl version 8.0 or later, while
Dervish is built on tcl 7.4; not everyithing will work the same.
In most cases, you can get help on a specific commend using the
help command, or by using the "-help" argument on
the command itself.
Dervish lets you allocate, use, and deallocate memory for data structures,
not just strings as is usual in tcl. A dervish "handle" allows you to
reference the memory allocated for a structure, and a "schema" describes what
is found there:
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> typedef struct {char name[80]; int x; int y} SAMP;
dervish> genericNew SAMP
h0
dervish> schemaPrint h0
name {char[80]}
x int
y int
dervish> exprPrint -header -noquote h0
h0 SAMP
(h0).name
(h0).x 0
(h0).y 0
dervish> handleSet h0.name Joe
dervish> handleSet h0.x 10
dervish> handleSet h0.y 42
dervish> exprPrint -header -noquote h0
h0 SAMP
(h0).name Joe
(h0).x 10
(h0).y 42
dervish> exprGet h0.x
10
dervish> exprGet h0
{name Joe} {x 10} {y 42}
dervish> handleSet h0 { Fred 2 3}
dervish> exprPrint h0
(h0).name "Fred"
(h0).x 2
(h0).y 3
dervish> handleDel h0
dervish> exit
sdsslnx33$
|
If you want to use a structure defined in a C header file, you can use
makeio instead of
typedef.
You can view and edit one of these structures interactively
using structsEdit.
(Ctrl-x saves
changes to memory, and
Ctrl-c exits.)
A "chain" is a doubly linked list of these structures. You
can interactively look through a chain using
chainPage.
(Ctrl-n
and
Ctrl-p
go to the next and
previous elements of the chain, respectively.) Commands that load and
save data often do such as chains. You can get specific elements of a
chain using chainElementGetByPos.
You can load a Yanny param file into a chain using
param2Chain, and write to one using
chain2Param.
dervish> param2Chain opCamera-52639.par h
h0 h1
dervish> keylget h mjd
52641
dervish> puts $h
{mjd {52641}} {scalef {16.596119193140311}}
|
Note that there are two type of structures in this par file, so two
chains were allocated. You can look through them interactively using
and
You can read a FITS header using hdrReadAsFits:
dervish> set h [hdrReadAsFits [hdrNew] image.fits]
h0
dervish> set ra [hdrGetAsDbl $h RA]
189.34690000000001
dervish> hdrDel $h
|
The "-hdu" option will let you get the header for an extension HDU.
Other header utilities can be found using:
dervish> info command hdr*
|
To load a FITS image into memory first create a handle for the
region using regNew, and then load it using
regReadAsFits. You can then display the image. When
you are done, be sure do deallocate the memory.
For example:
dervish> set im [regReadAsFits [regNew] image.fits]
h0
dervish> saoDisplay $im
1
dervish> regDel $im
|
Other region utilities can be found using:
dervish> info command reg*
|
You can load a FITS binary table into a chain using
fits2Schema, and write to a fits file using
schema2Fits.
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> set h [hdrNew]
h0
dervish> fits2Schema tsObj-001339-5-40-0075.fit OBJ $h -hdu 1
h3
dervish> chainElementGetByPos h3 1
h2
dervish> exprGet h2.ra
260.994934505629
dervish> exit
|
You can also look through the chain interactively using
Chains have operations corresponding to typical list tools. This transcript should provide some clues:
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> typedef struct {char name[80]; int x; int y} SAMP;
dervish> genericNew SAMP
h0
dervish> genericNew SAMP
h1
dervish> handleSet h0 { Fred 2 3}
dervish> handleSet h0 { Saturn 1001 2001}
dervish> chainNew GENERIC
h2
dervish> handleSet h1 { Saturn 1001 2001}
dervish> chainElementAddByPos h2 h1
dervish> chainElementAddByPos h2 h0
dervish> chainElementGetByPos h2 1
h0
dervish> chainElementGetByPos h2 0
h1
dervish> chainCursorNew h2
h3
dervish> chainWalk h2 h3 THIS
h1
dervish> chainWalk h2 h3 NEXT
h0
dervish> chainWalk h2 h3 THIS
h0
dervish> chainWalk h2 h3 PREVIOUS
h1
dervish> chainWalk h2 h3 PREVIOUS
dervish> chainWalk h2 h3 THIS
h0
dervish> chainCursorDel h2 h3
dervish> chainCopy h2
h3
dervish> chainDel h2
dervish> exprGet h0
{name Saturn} {x 1001} {y 2001}
dervish> chainDestroy h3
0
dervish> exprGet h0
Error: Unknown handle: h0
Error: h is not open
p_shTclHandleAddrGet: unknown handle name h0.
dervish> exit
sdsslnx33$
|
There are many other chain utilities. These can be listed using:
dervish> info command chain*
|
Dervish uses an interface to
pgPlot
for plotting, using fairly direct mappings to the C and Fortran
interfaces, as discribed in the
pgPlot subroutine list and the
pgPlot subroutine discription pages. Here is an example of its
use in Dervish:
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> set pg [pgstateNew]
h0
dervish> pgstateSet $pg -device "/XWINDOW"
dervish> pgstateOpen $pg
Open /XWINDOW
h0
dervish> set xmin 0; set xmax 10; set ymin 0; set ymax 100
100
dervish> set color 1; set symbol 2
2
dervish> pgSci $color
dervish> pgEnv $xmin $xmax $ymin $ymax 0 0
Type <RETURN> for next page:
dervish> pgLabel "X values" "Y values"
dervish> set color 2
2
dervish> pgSci $color
dervish> set x 5; set y 8
8
dervish> pgPoint $x $y $symbol
dervish> set x 8; set y 80
80
dervish> pgPoint $x $y $symbol
dervish> pgstateClose $pg
Close /XWINDOW
Type <RETURN> for next page:
dervish> exit
sdsslnx33$
|
You can got a list of pgPlot routines from within Dervish:
dervish> info command pg*
|
and use the standard Dervish help facility to get usage:
Note that pgPlot uses a separate process to show plots in X
Windows. If you are using ssh to log into a remote host, and are
making plots from that host, it is sometimes helpful to start it
locally rather than remotely, so that ssh does not wait for the
process to end when you try to log out. You start the server thus:
A Dervish "vector" is a one dimensional array of floats. One can
operate on all elements of a vector at once:
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> vectorExprEval {{42 7 4344 2}}
h0
dervish> vectorExprPrint h0
42 7 4344 2
dervish> vectorExprEval 2*h0
h1
dervish> vectorExprPrint h1
84 14 8688 4
dervish> vectorExprPrint h0
42 7 4344 2
dervish> vectorExprSet h0 3*h0
h0
dervish> vectorExprPrint h0
126 21 13032 6
dervish> vectorExprDel h0
dervish> vectorExprDel h1
dervish> exit
sdsslnx33$
|
Note that the distinguishing feature between
vectorExprEval and
vectorExprSet
is that the former will allocate the new vector for you, but the later
requires that it already exist. As usual, you can get a list of vector commands
using
One can create vectors from elements in a chain of structures using
vFromChain, and set elements in a chain of
structures from a vector using
vToChain, which
makes them particularly useful for operating on columns in FITS tables
or Yanny par files:
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> hdrNew
h0
dervish> fits2Schema tsObj-001339-5-40-0075.fit OBJ h0 -hdu 1
h3
dervish> vFromChain h3 rowv
h2
dervish> vMean h2
-505.0784
dervish> vectorExprEval h2*2
h1
dervish> vMean h1
-1010.157
dervish> hdrDel h0
dervish> vectorExprDel h1
dervish> vectorExprDel h2
dervish> chainDestroy h3
0
dervish> exit
sdsslnx33$
|
Dervish inherits two different help string handling systems from
its dependencies. One supplies a help string when a user invokes the
help command, the other when the user runs the
procedure and supplies "-help" as one of the arguments. Because not
all procedures in all products handle the "-help" argument properly,
and some interpret it as some other argument, users are safer trying
the former first. When writing a precedure, devopers should make sure
both are supplied. Here is an example:
lappend thisFiles_procs doSomething
proc doSomething { args } {
set opts [list [list [info level 0] "The proc is an example of how to do something, but does nothing."]]
lappend opts [list {<happyArg>} STRING "" happy "The desired value for happy"]
lappend opts [list {<sadArg>} STRING "" sad "The desired value for sad."]
if {[shTclParseArg $args $opts [info level 0]] == 0} {
return
}
puts $happy
puts $sad
}
# This goes at the end of the source file
set_ftclHelp samples thisFiles_procs ;# perform ftclHelpDefine's on all listed procs
|
Note that the
opts argument of
shTclParseArg is a list of lists. The first element
of this list is a two element list, the first element of which is the
name of the procedure (determined automagically above using
info), and the second is a text description of the
command. The remaining elements of this list are lists describing the
options. These lists contain the following elements:
The label for the argument given in the help
string. This string must be surrounded by angle brackets ("<happy>"),
in which case the argument is required; preceeded by a dash
("-happy"), in which case the argument may either be present or
absent; or surrounded by square brackets ("[happy]"), in which case
the default value (given by element 3 in this list) is used as a
default.
The type of the argument, which may be
INTEGER,
CONSTANT,
STRING, or
DOUBLE.
The default value for the variable. If there is no
default value, a placeholder must be used.
The name of the tcl variable to be set by this option.
A text description of the option to be displayed in
the help string.
Given that we can allocate and deallocate memory manually in
Dervish, the possibility of memory leaks is an issue. Dervish provides
some tools for debugging them, the most useful of which may be
memStatsPrint.
sdsslnx33$ dervish
Executing commands in /sdss/ups/prd/dervish/v8_18/Linux-2-4-2-3-2/etc/dervishStartup.tcl:
dervish> memStatsPrint
Number of memory allocation requests: 0
Number of memory de-allocation requests: 0
Total bytes currently in use: 0
Total bytes in Free Memory Pool: 0
Total bytes allocated by malloc(): 0
Percentage of memory allocation requests
satisfied from Free Memory Pool: 0.00 %
Percentage of memory allocation requests
satisfied from the Operating System: 0.00 %
dervish> typedef struct {char name[80]; int x; int y} SAMP;
dervish> genericNew SAMP
h0
dervish> memStatsPrint
Number of memory allocation requests: 1
Number of memory de-allocation requests: 0
Total bytes currently in use: 88
Total bytes in Free Memory Pool: 0
Total bytes allocated by malloc(): 168
Percentage of memory allocation requests
satisfied from Free Memory Pool: 0.00 %
Percentage of memory allocation requests
satisfied from the Operating System: 100.00%
dervish> genericDel h0
dervish> memStatsPrint
Number of memory allocation requests: 1
Number of memory de-allocation requests: 1
Total bytes currently in use: 0
Total bytes in Free Memory Pool: 128
Total bytes allocated by malloc(): 168
Percentage of memory allocation requests
satisfied from Free Memory Pool: 0.00 %
Percentage of memory allocation requests
satisfied from the Operating System: 100.00%
dervish> exit
sdsslnx33$
|
You can, of course, find the others using
dervish> info command mem*
|
A region is simply an array, usually an image. For example, this
program creates a 512 by 512 array, initializes the whole array to 42,
prints the value of pixel 128, 128, multiplies the whole array by 10,
and prints new value of the pixel.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "region.h"
#include "shCRegUtils.h"
int
main(int argc, char **argv)
{
char *image="im";
REGION *r;
r=shRegNew(image,512,512,TYPE_U16);
shRegSetWithDbl(42.0,r);
printf("The starting value is %g\n", shRegPixGetAsDbl(r,128,128));
shRegMultWithDbl(10.0,r,r);
printf("The ending value is %g\n", shRegPixGetAsDbl(r,128,128));
return 1;
}
|
It can be compiled and run like this:
sdsslnx33$ setup dervish
sdsslnx33$ gcc -o reg_arith reg_arith.c -DSDSS_LITTLE_ENDIAN -DSTAND_ALONE -Wall -I$DERVISH_DIR/include -I$LIBFITS_DIR/include -L$DERVISH_DIR/lib -L$LIBFITS_DIR/lib -ldervishnotcl -lfits
sdsslnx33$ ./reg_arith
The starting value is 42
The ending value is 420
sdsslnx33$
|
A chain is a doubly linked list. This sample creates a chain
of a given structure, prints each member, and dealloctes the
memory used.
#include <stdio.h>
#include <stdlib.h>
#include <shChain.h>
#include <shCGarbage.h>
/* This declaration should be in shChain.h, but isn't */
int shChainDestroy(CHAIN *pChain, void (*pDelFunc)(void *));
/* Definition and create and destroy procs for our structure */
typedef struct {
int bar;
int baz;
} FOO;
FOO *fooNew(int bar, int baz)
{
FOO *f;
f=(FOO *)shMalloc(sizeof(FOO));
f->bar=bar;
f->baz=baz;
return f;
}
void fooDel(FOO *f)
{
shFree(f);
}
/* Make a short chain of FOOs, print them, and free memory */
int
main(int argc, char **argv)
{
FOO *f;
CHAIN *c;
CURSOR_T cursor;
c=shChainNew("FOO");
f=fooNew(1,2);
shChainElementAddByPos(c, f, "FOO", TAIL, AFTER);
f=fooNew(11,12);
shChainElementAddByPos(c, f, "FOO", TAIL, AFTER);
f=fooNew(21,22);
shChainElementAddByPos(c, f, "FOO", TAIL, AFTER);
cursor=shChainCursorNew(c);
while((f= (FOO*) shChainWalk(c,cursor,NEXT))!=NULL) {
printf("bar=%d, baz=%d\n",f->bar,f->baz);
}
shChainCursorDel(c, cursor);
shChainDestroy(c,(void (*)(void *))fooDel);
return 1;
}
|
It can be compiled and run like this:
sdsslnx33$ setup dervish
sdsslnx33$ gcc -o chain_sample chain_sample.c -DSDSS_LITTLE_ENDIAN -DSTAND_ALONE -Wall -I$DERVISH_DIR/include -I$LIBFITS_DIR/include -L$DERVISH_DIR/lib -L$LIBFITS_DIR/lib -ldervishnotcl -lfits -lm
sdsslnx33$ ./chain_sample
bar=1, baz=2
bar=11, baz=12
bar=21, baz=22
sdsslnx33$
|