For a 50-year old organization that operates in some of the strictest security environments, Aerospace Corporation might seem like a company that would shy away from the cloud. However, like so many that have crossed the chasm into cloud computing, Aerospace Corporation now has a flexible and secure environment that is an indispensable part of their overall high performance computing (HPC) strategy.

This assumption about the limitations of cloud environments could sound even more improbable when one considers the demanding high performance computing applications they run for users in the government security, space, and intelligence spheres. After all, the most frequent concern in HPC circles is application performance when the node-to-node communication capabilities are sliced.

However, Aerospace Corporation is a living example of how three of the most frequently discussed challenges for HPC clouds — performance, security, and cost — can be navigated and overcome. By working toward a seamless blend between the company's existing on-site infrastructure and the scalable resources in the secure AWS GovCloud, Aerospace Corporation is ensuring all three elements are pulled together so researchers and engineers can focus on the tasks at hand — not manage IT issues or skirt a range of potential queue times and core-count restrictions.

Prior to their ability to offer users access to scalable resources, Aerospace Corporation's HPC infrastructure and application teams were continuously navigating user demands around their hosts of clusters. While they were able to control the security of the environment, the scalability limitations meant frustration for users when large jobs would come in that occupied the entire cluster — sometimes for several days. Other users would submit hundreds of jobs sequentially, which meant that queue times were long. And still others were forced to change their applications to meet smaller core count limits to avoid taking over the entire machine.

"We wanted to respond to increasing demand for services and needed a way to do that in a short period of time," explained Bill Williams, who manages the high performance group at Aerospace Corporation. The problem is, adding more resources typically means a procurement process that lasts a year or more — too long for mission-critical requirements, and in some cases, too long to be flexible if project requirements change. In addition to the lengthy procurement cycles, the flexibility extended to the company’s overall ROI. By scaling up and down with demand, Aerospace Corporation could make their costs flex along with actual demand.

Williams and his team found that they could use cloud services in the AWS GovCloud Region, which leverages a framework that adheres to U.S. Federal Risk and Authorization Management Program (FedRAMP) standards. This bolstered the stable of resources for the scientific and engineering community by building an elastic cluster that now chews on a range of high performance computing applications. From Monte Carlo, trace analysis, and MPI jobs, the Aerospace Corporation's HPC group is now nimble enough to deploy based on in-the-moment requirements, spinning up additional nodes as the workload demands and powering them down once the job is complete, making room for large runs that need to occupy the team's physical resources for large-scale, time-sensitive jobs."Our users are finally able to submit their jobs based upon need and get what resources they need at the time instead of tailoring their jobs to particular clusters," said Williams.

While the elasticity adds flexibility to the number and scale of jobs Aerospace Corporation can handle, the real key to the company's success lies in providing a seamless, secure experience. An early challenge the team faced was providing a single source of authentication so users wouldn't need to be concerned with where their jobs were running — they simply signed on and solved problems. By using a single image that can be quickly scaled to meet user demand and that already has all the predefined security baked in, Aerospace Corporation is continuing to push the limits of HPC applications in the cloud.

You can watch Bill's presentation on this topic from AWS re:Invent 2013.

Real-world Cloud HPC at Scale, for Production Workloads (BDT212) | AWS re:Invent 2013

(video)

Running high-performance scientific and engineering applications is challenging no matter where you do it. Join IT executives from Hitachi Global Storage Tec...