Datawolves for the SDSS
The SDSS is constructing a 2.5 Terabyte dataset. We must explore machines that make possible efficient analysis of this data.If Beowulfs are clusters optimized for efficient message passing parallel processing, Datawolves are clusters optimized for both high I/O rate scans through large data sets and and for bringing to bear high compute power onto large datasets.
GriPhyN and the SDSS
GriPhyN is a collaboration studying how to handle the science of large datasets: they have chosen to focus on the idea of virtual data.We have written a draft document for GriPhyN. It outlines the SDSS project, the SDSS pipelines, how we approach turning the pipelines into a factory, and a suggestion for a SDSS problem that the Griphyn collboration could consider when designing thier tools, that of data management for the Southern Survey.
TAM
The Experimental Astrophysics Group is building a Terabyte Analysis Machine for its science analyses and as a research prototype for database testing and advanced filesystems.
The Terabyte Analysis Machine:
A research cluster aimed at exploring large distributed
astronomical databases with a 7 dual node Linux cluster,
500 gig of local disk, 1 Terabyte of global disk and with
SX, a distributed, spatially parititoned database designed
for fast queries on a complicated Terabyte scale data.
One aim: repartition and re-index the whole database onto
local disk for specialized queries, e.g., kth nearest neighbors.
We are designing this system around four archetypal analysis tasks:
- looking for quasars: one by one searching of a catalog for attributes meeting some match or range critera. I/O limited.
- weak lensing measurements: one by one re-analyzing of the atlas images. CPU limited.
- cluster finding: searches in an N-dimensional space, requiring all objects in a spatial area. CPU limited.
- casual exploratory data analysis. Astronomer limited.
TAM As Built:
- Linux NetworX: cluster and integration
- DotHill Fibre Channel RAID, the SanNet 4200
- Qlogic the Fibre Channel HBA, the QLA2100 HBA
- GFS the Global File System.
- Enstore data storage system, where one thinks of a terabyte of disk as cache; system details and transfer rates.
SX Bricks
Dell Poweredge 4400 make fine SX database bricks:
- PCI bus: They have 64 bit, 66 Mhz PCI buses, a requirement for dual processor machines to not be memory bus bandwidth limited at about 100 Mbytes/sec.
- SCSI bus: they have a split internal SCSI backplane, somewhat rare. Without this, one cannot use two SCSI channels at maximum rate.
- 2 Single channel SCSI controllers: there are 2 channel controllers available. We have tested Adaptec versions, and found no speed up over a single one channel controller. Two independent SCSI chains seem to be required to get linear scale up. We have tested these Adaptec cards. After we convinced them to fix the bugs in the software drives, they perform well.
- Software raid: we tested several varieties of internal hardware RAID cards (external hardware RAID cannot begin to compete on price/performance): all are limited to about 50 Mbytes/sec/channel. We use the Linux kernel software RAID; our tests show it capable of 85-90 Mbytes/sec/channel.
- 4 disks/channel: our tests of the Seagate Cheetah 10,000 rpm drives show them capable of 33 Mbytes/sec. 4 of these in a RAID-0 set will max out the SCSI channel (again, which we believe is memory bus limited).
The first of ours is sdssdp5.
IDE Disk Farms
We need to hold the reduced frames and atlas image database.
EIDE disk are so cheap that putting Terabytes onto a single node is attractive, if read/write performance is not an issue. See: Working with Arrays of Inexpensive EIDE Disk Drives by Sanders, Riley, Cremaldi, Summers, and Petravick.
The first of ours is sdssdp6.
What is the image at bottom? Caustics in a pool of water. Remarkably similar to the large scale distribution of galaxies in the Universe that the SDSS is designed to study. Clusters of galaxies would be the bright knots; I wish to find them.
James Annis
June 1, 2000
Last Updated: Monday, 05-Jun-2006 07:50:02 CDT
