Tom Handley (Caltech IPAC IRSA), Jin Ma (Caltech IPAC), John Good (Caltech IPAC), Robert Brunner (Caltech)
NPACI Digital Sky - A Technology Demonstration
With current survey's such as 2MASS, DPOSS and Sloan, the data available to the science community is exploding. Discovering and extracting the information within a single data set is challenging, however discovering and extracting the information hidden among datasets even more challenging.
Digital Sky's, an NPACI (National Partnership for Advanced Computational Infrastructure) technology demonstration, main goals are 1- Develop tools to manipulate Terabyte data sets (catalogs and images) 2- Develop analysis tools which take advantage of advances in parallel supercomputing 3- Develop tools to provide common interface to heterogeneous databases 4- Contribute to standards for encapsulation of relationships between objects in different catalogs, and 5- Contribute to standards for astronomical database metadata,
Section one will describe the overall architecture and services within Digital Sky.
Section Two will describe initial results which include: 1- O(N) cross-identification algorithm among heterogeneous databases, and 2- Prototyped tools to provide common interface to heterogeneous and distributed databases.
Section three will describe the future directions being taken based on the lessons learned, the fundamental goals, and an infusion of emerging archive technology. These directions include: 1- "Information exchange", what archive information services are needed to provide syntactic, semantic and bulk transfer interfaces? 2- Source Cross-Identification 3- System Metadata 4- Characterization of astronomical Object interrelationships 5- Cooperative process management 6- Optimized bulk data transfer 7- Cooperative cost-based, query optimization and 8- Suggest Community Standardization, which most likely will include, spatial indexing, system metadata, inter-archive communication, and data stream formatting.