NewsEvents Careers Tech Support Internship
Select Region
United StatesIndiaAustralia & New Zealand
US & North America

StarCCM+ CFD Simulation on Azure HPC Environment

Computational Fluid Dynamics (CFD) simulations need significant compute power and time along with specialized hardware and network interfaces. The CAD industry is not new to utilizing flexible cloud environments to run their HPC (High-Performance Computing) workloads and overcome the challenges of procuring and maintenance of specialized costly infrastructure.

Microsoft Azure is consistently investing time and money to deliver and improve the broadest range of HPC offerings in their public cloud. From network interfaces like InfiniBand enabled VM families, Hyperscale like Cray supercomputing for running compute-intensive HPC, AI and Machine Learning workloads on the cloud. In this blog, we will be looking at Azure CycleCloud which can be used to set up and maintain an HPC cluster on Azure to submit CFD Solver jobs.

IoT CFD workload on an HPC environment created by Azure CycleCloud

Infrastructure:

  • NFS Mount: Network File system (NFS) is a distributed File system used on Linux Machines. Azure NFS 3.0 protocol added preview support for Azure Blob Storage. We are currently using the Azure NFS 4.1 protocol. In the above architecture, we have uploaded all the Star CCM+ Software binaries, CFD Simulation case files into an Azure Storage Container. We are then using Azure NFS protocol to mount this container as a File system directory /share on the Cluster Head Node. Now we can simply copy, paste, and operate on these files as if they are system files.
  • CycleCloud Server: CycleCloud is an orchestration and configuration tool for configuring an HPC Cluster on the Azure Cloud environment. We can deploy the CycleCloud application on-premises or on Azure VM. We access the IP address of the machine to open the CycleCloud Web UI. It is a GUI tool used to set up HPC Cluster information like max Core count, Job Scheduler type, NFS Mounting paths, pre or post initialization scripts, Infiniband network driver extension and Head and Computes node VM Sizes.
CFD workload on HPC environment created by Azure CycleCloud

The above diagram shows a High-level architecture of the components involved in running a CFD workload on an HPC environment created by Azure CycleCloud.

  • Infiniband Network: In a distributed nodes cluster, the jobs are broken down into pieces and executed across multiple nodes parallelly to get faster results. This is achieved by placing those cluster nodes physically closer or using a high bandwidth network interface which allows for high levels of inter-node communication using MPI (Message Passing interface) or over TCP/IP protocol. Azure Infiniband is such a network interface driver extension for eligible VM families which offloads the network communication load from the CPU to the driver. This improves the node efficiency and helps in high speed & low-latency communication with other nodes which is highly recommended in Deep Learning jobs and HPC workloads.
  • Cluster Head Node: The Cluster Head node in our implementation is a general-purpose optimized D4_v3 VM with SSH access pre-configured via the CycleCloud Server. It is also configured with Job Scheduler openPBS and CentOS 7 as the Operating System. We can use the Head Node to use the Job scheduler to submit a distributed jobs into the queue (qsub CLI).
  • Compute Node Cluster: The Compute Node Cluster in our implementation comprises of F8s_v2 VMs. Once the Job in the queue is picked up by the scheduler, the scheduler will take care of spinning up VMs and auto-scaling the VM cluster size based on workload and core count requirements.

Working

  • We have used Azure Cycle Cloud server tool to create and initialize the HPC Cluster with Head and Compute Nodes with max core count, OS type, NFS Mount directory path, VNET and subnet configurations.
  • CycleCloud takes approximately 10 minutes to create and initialize the head and compute node cluster infrastructure with OS & dependency configurations.
  • From Cyclecloud, we SSH into the Head Node and install all relevant Applications like StarCCM+ in our case and load the Simulation file using the NFS Mounted directory path /share.
  • We use CLI qsub Command to submit the Job to the scheduler queue. Once the Job is picked up for execution, the scheduler auto-scales the Compute cluster to satisfy the workload core count. Once the job is completed, the Compute nodes are automatically terminated by the scheduler and we can transfer the job results to a convenient location for further operations.

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

keyboard_arrow_up