Hostname: page-component-84b7d79bbc-5lx2p Total loading time: 0 Render date: 2024-07-26T19:31:14.139Z Has data issue: false hasContentIssue false

ATLAS: Big Data in a Small Package?

Published online by Cambridge University Press:  01 March 2016

Larry Denneau Jr.*
Affiliation:
Institute for Astronomy, University of Hawaii, 2680 Woodlawn Dr., Honolulu, HIUSA email: denneau@hawaii.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

For even small astronomy projects, the petabyte scale is now upon us. The Asteroid Terrestrial-impact Last Alert System (Tonry 2011) will survey the entire visible sky from Hawaii multiple times per night to search for near-Earth asteroids on impact trajectories. While the ATLAS optical system is modest by modern astronomical standards — two 0.5 m F/2.0 telescopes — each night the ATLAS system will measure nearly 109 astronomical sources to a photometric accuracy of <5%, totaling 1012 individual observations over its initial 3-year mission. This ever-growing dataset must be searched in real-time for moving objects and transients then archived for further analysis, and alerts for newly discovered near-Earth asteroids (NEAs) disseminated within tens of minutes from detection. ATLAS's all-sky coverage ensures it will discover many ‘rifle shot’ near-misses moving rapidly on the sky as they shoot past the Earth, so the system will need software to automatically detect highly-trailed sources and discriminate them from the thousands of low-Earth orbit (LEO) and geosynchronous orbit (GEO) satellites ATLAS will see each night. Additional interrogation will identify interesting phenomena from millions of transient sources per night beyond the solar system. The data processing and storage requirements for ATLAS demand a ‘big data’ approach typical of commercial internet enterprises. We describe our experience in deploying a nimble, scalable and reliable data processing infrastructure, and suggest ATLAS as steppingstone to data processing capability needed as we enter the era of LSST.

Type
Contributed Papers
Copyright
Copyright © International Astronomical Union 2016 

References

Busso, A.et al. 2009, AAS Abstracts, 41Google Scholar
Brewer, E. 2000, PODC KeynoteGoogle Scholar
Dean, J. & Ghemawat, S. 2008, Communications of the ACM, 51, 1, 107CrossRefGoogle Scholar
Denneau, L.et al. 2013, PASP, 125, 357CrossRefGoogle Scholar
Gilbert, S. & Lynch, N. 2002, ACM SIGACT News, 33, 2, 51CrossRefGoogle Scholar
Kaiser, N.et al. 2002, SPIE Conference Series, 4836, 154Google Scholar
Larson, S., Brownle, J., Hergenrother, C., & Spahr, T. 1998, Bulletin of the American Astronomical Society, 30, 1037Google Scholar
Mainzer, A. K.et al. 2011, ApJ, 743, 156CrossRefGoogle Scholar
Moore, G. 1965, Electronics Magazine, 38, 8Google Scholar
Thain, D., Tannenbaum, T., & Livny, M. 2005, Concurrency - Practice and Experience, 17, 323CrossRefGoogle Scholar
Tonry, J. L. 2011, PASP, 123, 58CrossRefGoogle Scholar
White, T. 2009, Hadoop: The Definitive Guide (O'Reilly Media, Inc.)Google Scholar