Optimizing the Query Performance of Block Index Through
Data Analysis and I/O Modeling
Event Type
Paper
Data management
I/O
Performance
Storage
TimeTuesday, November 14th11:30am -
12pm
Location301-302-303
DescriptionIndexing technique has become an efficient tool to
enable scientists to directly access the most relevant
data records. But, the time and space requirements of
building and storing indexes are expensive in the
traditional approaches, such as R-tree and bitmaps.
Recently, we started to address this issue by proposing the idea of "block index", and our previous work has shown promising results from comparing it against other well-known solutions, including ADIOS, SciDB, and FastBit. In this work, we further improve the technique from both theoretical and implementation perspectives. Driven by an extensive effort in characterizing scientific datasets and modeling I/O systems, we presented a theoretical model to analyze its query performance with respect to a given block size configuration. We also introduce three optimization techniques to achieve a 2.3x query time reduction compared to the original implementation.
Recently, we started to address this issue by proposing the idea of "block index", and our previous work has shown promising results from comparing it against other well-known solutions, including ADIOS, SciDB, and FastBit. In this work, we further improve the technique from both theoretical and implementation perspectives. Driven by an extensive effort in characterizing scientific datasets and modeling I/O systems, we presented a theoretical model to analyze its query performance with respect to a given block size configuration. We also introduce three optimization techniques to achieve a 2.3x query time reduction compared to the original implementation.
Download PDF:
here
