Tiered Storage for HPC: Matching Data to the Right Storage Technology | Nor-Tech
HPC environments rarely have a single storage requirement. Active simulation data may demand extremely high throughput, temporary datasets may require low-latency access, and completed projects may need to remain available for years without consuming premium storage capacity.
Treating all of that data identically can result in an infrastructure that is either too slow or unnecessarily expensive.
Tiered storage addresses the problem by matching data to storage technology according to performance, accessibility and retention requirements. Instead of designing the entire environment around the highest-performance workload, organizations can reserve premium resources for the data that actually benefits from them. A typical HPC storage strategy might take into account:
- NVMe or all-flash storage for latency-sensitive workloads and high-performance scratch
- High-throughput parallel storage for active HPC datasets accessed concurrently by many nodes
- High-capacity disk storage for less performance-sensitive project data
- Object, cloud or tape storage for archival and long-term retention
The challenge is determining where those boundaries belong. Data age alone is not enough. Integrators should consider access frequency, file size, I/O patterns, concurrency, required throughput and how quickly archived data may need to return to active use.
Parallel file systems may play an important role in the performance tier. Nor-Tech’s guide to parallel storage for AI and HPC discusses workload requirements including throughput, IOPS, latency, scalability and concurrent access.
Nor-Tech Executive Vice President Jeff Olson said, “Tiered storage is really about putting performance where it creates value. You don’t want expensive high-performance capacity sitting under data that rarely moves, and you don’t want active compute jobs waiting for storage that can’t keep up.”
Successful tiering requires that the network, file system, compute environment and data-management strategy work together. Nor-Tech’s storage solutions encompass commercial, software-defined, distributed, parallel, high-availability and archival technologies.
By engineering these resources as part of a fully integrated HPC cluster, organizations can balance performance, capacity and cost—while creating a system capable of evolving as workloads and datasets grow.
Nor-Tech has demos available for a wide range of software and applications. To get started or schedule a no-cost, in-depth consultation, call 952-808-1000, email engineering@nor-tech.com/ or visit https://www.nor-tech.com
Why Nor-Tech is the Best Choice for Your Business
In 1998 we established ourselves as one of the leading providers of quality HPC solutions. Our servers are backed by an expert team that is available to provide support and assistance, ensuring that your business always has access to the resources you need. Contact us for more information or a quick quote: 952-808-1000; engineering@nor-tech.com/ or click on the Contact tab at https://nor-tech.com/contact.
Nor-Tech is on CRN’s list of the top 40 Data Center Infrastructure Providers along with IBM, Oracle, Dell, and Supermicro and is also a member of Hyperion Research’s prestigious HPC Technical Computing Advisory Panel. The company is a complete high performance computer solution provider for two Nobel Physics Award-contending/winning projects. Nor-Tech engineers average 20+ years of experience. This strong industry reputation and deep partner relationships also enable the company to be a leading supplier of cost-effective Lenovo desktops, laptops, tablets and Chromebooks to schools and enterprises. All of Nor-Tech’s high-performance technology is developed by Nor-Tech in Minnesota and supported by Nor-Tech around the world. The company is headquartered in Burnsville, Minn. just outside of Minneapolis. Nor-Tech holds the following contracts: Minnesota State IT, University of Wisconsin System, and NASA SEWP V. To contact Nor-Tech call 952-808-1000 or visit https://www.nor-tech.com.