Moving data through memory has become a major performance bottleneck, with the bandwidth per FLOP decreasing and the complexity of the memory hierarchy quickly outgrowing software programming environment abstractions. The Cray European Research Lab and our partners are addressing this concern by designing a fundamental data-centric abstraction that represents user data in various forms as data objects, regardless of its current location in the memory system. This abstraction allows middleware to reason about data movement and dataflow inside applications and workflows. Separately we are designing a memory abstraction framework including user code tiling that can lean on such data object reasoning to make decisions about memory tiling and promotion. For persistent storage the same abstraction allows us to move beyond configuration, application and library optimization to looking at the whole workflow and to concentrate on data movement driven by dependencies. We believe that being able to schedule application components and coupled applications while being able to reason about their data production, data locality, and data consumption, and accounting for the cost of moving data between tiers of the memory hierarchy will enable efficient execution of coupled applications in complex workflows.
Details
Publication Type: Minisymposium
Year of Publication: 2019
Authors: Christopher Haine, Utz-Uwe Haus, Adrian Tate
Publisher: PASC 19
Place of publication: Zurich, Switzerland
Event link: https://pasc19.pasc-conference.org/program/schedule/index.html%3Fpost_type=page&p=10&id=msa226&sess=sess170.html
Leave a Reply