SKA: The Ultimate Big Data Project
Authors: Dr. Happy Sithole (Center for High Performance Computing in South Africa)
Abstract: The Square Kilometre Array (SKA) will be the world’s largest radio-telescope, to be built in Australia and South Africa in an 11 countries' effort. SKA1 design will finish in 2018 with construction scheduled 2019 – 2025. SKA will produce Big Data on a scale that cannot be processed with today's technologies. This BoF brings together vendors and OEMs with participants from SKA's computing platform design to discuss topics such as:
* Data Management Challenges,
* Frameworks for SKA-scale projects,
* Compute platform design, OS, Middleware,
* Accelerators and HPC software,
* Power considerations toward exascale computing and others.
Long Description: The Square Kilometre Array (SKA) will be the world’s largest and most powerful radio-telescope, to be built in remote locations in Australia and South Africa as a combined effort of various countries (ten countries are currently full members of the SKA Organisation). Pre-construction design for the SKA1 (first phase of the project) commenced in 2013 and is due to finish in 2018. SKA1 construction is scheduled 2019 – 2025 and aims to build a system that will answer fundamental questions about our Universe. Over the next 50 years the SKA will produce Big Data on a scale that the ICT Industry is not prepared to process with today's technologies but probably will start to be facing in 5-10 years time. Therefore the SKA design work is giving an early exposure to new ICT technologies and potential spinoffs for next-generation Big Data applications (genomics, smart cities, environmental, etc) and could help generate new companies, steer technology evolution, IP generation and first-to-market ideas. This BoF will bring together several participants in the design of the computing platform of the SKA and interested parties and discuss topics such as: * Data Management Challenges, * High Performance Data Analytics, * Data availability when and where it is needed, * Frameworks required for SKA-scale projects, * Mega-science projects – management and implementation, * Compute platform design, prototyping and specification, * Role of hardware accelerators and HPC software, * Middleware, OpenStack, Software Development Environment, * Power considerations towards exascale computing and others. * BoF website https://openparallel.com/events-2/ska-the-ultimate-big-data-project-bof/
Experts and practitioners in Big Data, High Performance Data Analytics, Data Management in general, Computer Vision and Image Processing, Data Transport Software and Systems Engineering Modeling, prototyping and specifications, Mega-Projects (management, development, integration, maintenance), Diverse hardware (accelerators, FPGA, others), Vendors and organisations interested in exploring partnerships
We will have panel speakers from vendors and OEMs -will be looking for some opposing views.
We held the first, previous SKA Big Data BOF at ISC 2017 in Frankfurt, Germany. The room was packed, with more than 100 attendees.
Outcome: We will publish all of the slides, and have started a website for sharing information. We intend to write a very short white paper on the areas of shared research for SKA and HPC Big Data.
Conference Presentation: pdf
Birds of a Feather Index