Few people are familiar with high-performance computing, also known as massively parallel computing. This may be set to change as the provision of HPC is recognised as an important strategic resource for both commercial and public sectors. The e-Infrastructure Special Interest Group, part of the UK’s innovation-focused Knowledge Transfer Network (KTN), recently commissioned FirstPartner to create a HPC Market Map. The aim of the project was to create a visual overview of the HPC’s ecosystem, the provision of access methods to HPC resources and the variety application verticals requiring HPC resources. A further goal of the project was to compare the UK’s HPC capabilities with those found in mainland Europe.
What is HPC?
High performance computing facilities are distinct from other systems in their superior size, power and parallel processing capacity. Unlike cloud computing systems that provide a high degree of scalability through the on demand scaling of flexible virtual computing resources, HPC uses tightly coupled local hardware resources to minimise parallel processing latency. These can either be provided by clustering of high end servers from existing hardware vendors or for maximum execution performance, by hardware from specialist suppliers such as Cray, IBM, HP & Bull. Specialist HPC hardware typically tightly cluster upwards of 15 to 50 thousand CPU cores. In addition, specialist peripheral vendors offer HPC specific solutions to optimise memory, storage, energy and resilience around the high processing speed and large data transfer requirements.
Current optimised HPC installations achieve several petaFLOPS (1015 floating point operations per second) and research programmes are already underway to deliver exascale level performance (exaFLOPS or 1018 floating point calculations per second) within the next decade. The delivery of exascale performance will require new directions to achieve an order of magnitude step in processing power, whilst maintaining the current thermal power budgets and system resilience performance. Also developments in supporting massive parallelism software management are being pursued to optimise the throughput performance and efficiency of code run on HPCs through specialised compliers and middleware.
How is it used?
Applications of HPC are varied but typically involve tasks that are either computationally, numerically or data-intensive. These tasks require a high number of complex computations to be carried out on vast data sets, demanding a high level of processing power to be completed within a reasonable timeframe. These large datasets create a challenge in access to HPC resources, since few organisations are able to fund these capabilities for their own use.
In the past, HPC resources have been confined to specific strategic governmental and academic research programmes, but this scenario is rapidly changing. Provision of HPC resources are now viewed as a key driver to development of many commercial sectors and demand for access to their capabilities is no longer restricted to large budget R&D programmes. Access to HPC resources is becoming a critical factor in the development of such sectors as Life Sciences, Creative Media, Environmental Sciences, Financial Services and Automotive. In turn, these sectors are fuelled by the creativeness of the SME sector, who gain accelerated time to market and competitiveness through its use. The challenge is to provide these SME’s with the necessary resource and skills to quickly and efficiently benefit from the access to HPC resources.
In addition, connectivity is required that supports SME’s to efficiently transfer terabyte datasets securely to the HPC service centres. In many cases the cost of installing the necessary connectivity infrastructure is prohibitive, and the only solution is to courier the datasets stored on disk arrays.
What is Europe’s position in the world of HPC?
Suddenly the ability to ‘out compute’, means to ‘out compete’ and superior HPC technology has become a source of competitive advantage for nations. Despite traditionally being strong in HPC application development, European companies lag behind their counterparts in countries such as the US, which lead the field in terms of the design and manufacture of HPC hardware. As the map illustrates, whilst 83% of HPC application software used in Europe is indigenous, Europe’s share of HPC system vendors is less than 5%.
From a network perspective, some provision has been made to provide connectivity between European HPC centres. The academic communities have well equipped themselves with interconnect networks both at national (Janet in the UK) and pan-European levels through Géant interconnecting the national providers. Access is however limited to the academic community. The private sector, however, has lacked the ability to develop the necessary consortia to provide similar community solutions.
Europe has initiated a number of initiatives to provide competitive HPC technology development. In particular the ETP4HPC initiative as a joint collaboration between commercial and public sector HPC research to coordinate HPC development within a global agenda.
What is the future of HPC outside academia in the UK?
The HPC Market Map demonstrates the rapidly growing importance of HPC to industrial competitiveness of both the UK and Europe. In the past the largest HPC installations have been located in the USA, but this is changing rapidly. In November 2014, TOP500 reported the largest HPC installation in the World was located at the National Super Computer Center in Guangzhou, China with a performance of 33.86 petaflops/sec from over 3 million Intel cores configured in 16,000 nodes. Over 50 systems worldwide currently have a performance greater than 1 petaflop/sec.
The USA is the leading consumer of HPC systems with 231 recorded. Europe’s share was reported at 130 systems, with UK, France and Germany shares at 30, 30 and 26 respectively.
The challenge now is to surmount the technical barriers of achieving Exascale, along with the necessary improvements in software development to fully exploit this advance. Alongside the technology challenge, a further challenge needs to be addressed in fully opening up these resources so they can be fully exploited by the inherently fragmented SME communities. Achievement of this goal needs to be performed at an accelerated pace, as this is strategically important to fuel their innovation rates and fully utilise their agile approaches to drive global competitiveness. To achieve this will require the breakdown of the academic sector orientation of the HPC sector and placing these resources on a simple to access commercial model.
