Issuu

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

news Issue 82 AUTUMN 2017

In this issue

Driving the data revolution

HPC for business Building the road to Exascale ARCHER community for HPC researchers Our training and education programmes

City Deal seeks to create European capital of data High performance data engineering National Safe Haven for health data

HPC-UK: a new resource for HPC researchers

From our Director

EPCC is well known worldwide – I often think we are better known outside Edinburgh than we are in our home town. Perhaps the City Deal project which features in this edition will change this (see pages 4-5). EPCC is at the heart of this ambitious plan to transform the

Edinburgh City Region into the Data Capital of Europe by embedding data driven innovation into everything our schools, businesses and public-sector organisations do. Elsewhere in this edition you’ll also find articles on the wide range of activities around the UK and Europe that EPCC has always been involved in.

Mark Parsons EPCC Executive Director m.parsons@epcc.ed.ac.uk

HPC-Europa EC-funded research visits: apply now! The interdisciplinary HPCEuropa programme, which provides access to some of Europe’s top HPC facilities, is now running again. The programme pays travel and living costs for short collaborative research visits to a scientific host in another country. Applications can be submitted at any time, with four closing dates per year. One of the objectives of the HPCEuropa3 project is the extension of the geographic coverage of the Transnational Access programme. In this context, researchers from the Baltic and South East European countries (including Western Balkans and Eastern Partnership

New starts Meet our new colleagues EPCC at SC’17 Our booth and activities

City Deal EPCC’s role in building the Data Capital of Europe

Data engineering Our current big data projects

Safe Haven for health data EPCC hosts and supports this national resource

HPC-UK A new resource for HPC researchers

I hope you enjoy reading what we’ve achieved since the Spring.

HPC Europa

Infrastructure on High Performance Computing Pan-European Research

We always prepare the end of year edition of EPCC News in time for the Supercomputing conference in the USA. The 2017 conference is in Denver and EPCC will once again be well represented with 13 staff attending.

Contents

countries) are particularly encouraged to apply. Moreover, the new Regional Access sub-programme provides a new opportunity for those researchers from these countries who have less experience in HPC. Contact staff@hpc-europa.org to find out more. Catherine Inglis Project Manager, EPCC c.inglis@epcc.ed.ac.uk For further information and closing dates see: www.hpc-europa.eu Twitter: @HPCEuropa3 www.facebook.com/hpceuropa

Fortissimo Simulation & modelling for business

SHAPE programme HPC for smaller enterprises

The road to Exascale How we’re rising to the challenge

ARCHER community An open community for users of HPC

Staff development Cybersecurity degree

Our online courses Flexible learning in HPC and Data Science

Our Masters programmes An update

Supercomputing MOOC A review of the second run

Personalised medicine ‘Building a Virtual Human’

Cover image credits: European Space Agency (main image); Nguyen Anh Koa Doan (inset).

info@epcc.ed.ac.uk

+44 (0)131 650 5030

Twitter: @EPCCed

EPCC is a supercomputing centre based at The University of Edinburgh, which is a charitable body registered in Scotland with registration number SC005336.

New staff at EPCC Photograph of Roderick by Daniel Gleed.

Callum Fenna-Roberts Computing Support Officer I recently completed the MSc in High Performance Computing at EPCC before returning in September 2017 to join the Systems Development Team as a Computing Support Officer. Before that I worked in geophysics and later in intellectual property. Outside of work I enjoy going to the cinema, learning foreign languages, and playing the guitar and piano.

Kris Findlay Systems Developer Prior to EPCC I have been involved in render farms within the animation industry and embedded system optimisation. I regularly attend local Makerspaces and various Tech Meetups where I often give talks and encourage other members to learn and become involved. In my spare time I enjoy flying/ racing quadcopters, building and designing small electronics projects, and discovering new places.

Roderick Snedden Systems Administrator Intern I recently started a year-long placement working at EPCC’s Advanced Computing Facility. I am currently in my third year at Edinburgh Napier University where I am studying Computer Security and Forensics. I have enjoyed my first few weeks here at EPCC and I look forward to learning many new skills working with the exciting technologies available. Outside of work I am thoroughly interested in music and play guitar and piano.

EPCC at SC17 If you’re attending Supercomputing 2017, come and say hello! EPCC and Wee Archie, our Raspberry Pi supercomputer, will be at Booth number 201. EPCC has been part of Supercomputing since we first attended in Albuquerque in 1991. This year Wee Archie will take pride of place on our booth, running our CFD demo (see pic). Drop by to design, build and perhaps even fly your own virtual aircraft! EPCC staff will present several Birds-of-a-Feather sessions on topics including outreach to the HPC community, careers for software engineers, and HPC Carpentry.

In addition there will be posters and presentations about our current work, and colleagues will oversee conference events and give talks.

Adrian Jackson Research Architect, EPCC a.jackson@epcc.ed.ac.uk

Women in HPC will run a workshop and various other events at SC. Full details can be found online: www.womeninhpc.org/whpc-sc17/ Finally, look out for EPCCassociated projects including SHAPE, INTERTWinE and EXDCI (European Extreme Data and Computing Initiative) on the PRACE booth (number 2171).

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

For the full list of our activities at SC’17, see our website: www.epcc.ed.ac.uk 3

City Deal: University of Edinburgh to play pivotal role in creating Data Capital of Europe The University will be a key partner in a major initiative that seeks to make Edinburgh and its surrounding region the Data Capital of Europe. The move to transform the region into a digital powerhouse is a key strand of the recently announced £1.1 billion Edinburgh and SouthEast Scotland City Deal.

and public sectors to grow awareness of data science’s implications. As it takes effect, the programme will enable both established and start-up businesses to capture and adopt these opportunities.

Under the agreement, significant investment from the UK and Scottish Governments will be provided for major infrastructure projects.

“The University is working from a strong base. For 50 years, we have led advances that have shaped the computer age. Our research in new data technologies and the skills of our outstanding graduates have helped establish Edinburgh as a leading player in the global digital economy. We will seek to build on these strengths, to help turn the data science opportunity into new skills, jobs and companies for the Edinburgh and South East Scotland City Region.”

With the University at its centre, the City Deal’s Data Driven Innovation (DDI) programme will give businesses and people in Edinburgh and its surrounding areas maximum advantage in the data revolution. Over 10 years, the University and its partners will train over 100,000 people in the application of data across the region’s major industry sectors. It will work with the private

Professor Jonathan Seckl, VicePrincipal Planning, Resources and Research Policy, The University of Edinburgh

The Bayes Centre will be a hothouse of multidisciplinary activity acting as a hub supporting data technologies, data intensive research and data analytics in education. Work on the site started in earnest alongside the Informatics Forum at the end of February 2016 with planned completion by summer 2018. Image: Bennetts Associates Ltd (Architects)

©iStock.com/georgeclerk

The City Deal and what it means to EPCC EPCC is a key part of the Data Driven Innovation programme of the City Deal through three distinct activities: • Firstly, we will receive significant financial investment to build a new high resiliency computer room at our Advanced Computing Facility (ACF) to house the socalled World Class Data Infrastructure (WCDI) which underpins the whole City Deal. • Secondly, we will move into the Bayes Centre for Data Science and Technology. • Thirdly, we will be the delivery engine for a large number of projects across the sector-based industry and public sector programmes. At the time of writing, considerable progress is being made. The initial work on the new computer room at

the ACF is now underway and the plan is to develop the designs over the next 6 months so that we can quickly enter the build phase once the money is confirmed. We are extremely busy making preparations for the move into the Bayes Centre next summer and we are about to undertake a significant recruitment drive so that we have enough staff to deliver DDI projects across the region. “All of this activity is very positive. EPCC has a very high profile in the City Deal and, through the WCDI, we will underpin the whole programme. “At the same time, we’re seeing more and more interaction with other University of Edinburgh departments – Informatics and the Management School in particular – and such interaction is key to the success of our long-term strategy.”

Maureen Simpson, EPCC m.simpson@epcc.ed.ac.uk

City Deals A City Deal is an agreement between government and a city. It gives the city and its surrounding area certain powers and freedom to: • take charge and responsibility of decisions that affect their area • do what they think is best to help businesses grow • create economic growth • decide how public money should be spent.

Professor Mark Parsons, Director, EPCC

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Investigating high-performance data engineering Big data has always been a part of high-performance computing and the science it supports, but new open-source technologies are now being applied to a wider range of scientific and business problems. We’ve spent time this year testing some of the big data toolkits. Two of the largest drivers of big data applications have been mobile applications and the Internet-ofThings (IoT).

working with Edinburgh mobile app firm XDesign to gather and store sensor data from smartphones in moving vehicles.

Sensor data

The challenge with both cases is not in creating an app to record data but in creating an operational back-end that can scale to many data streams from many devices.

Smartphones contain a remarkable array of sensors and are very interesting as mobile devices for sensing and recording a user’s environment. The increasing prevalence of low-power sensors monitoring air and water quality, traffic density and so on gives an opportunity to take better socioeconomic decisions with better data. And, whether from mobile phone or static sensor, all those data have to go somewhere. EPCC has been engineering back-end systems for mobile data streams in two important areas: disaster risk reduction and smarter cities. Working with the Schools of Informatics and Geosciences, and with international NGO Concern Worldwide, we have created both the back-end and front-end for a system to record smartphone sensor data with the aim of predicting and improving earthquake resilience in buildings (see EPCC News 81). In related work we have been 6

In exploring these challenges we have been building expertise in Amazon Web Service’s building blocks for streaming data applications – AWS Lambda, API Gateway, DynamoDB – and exploring the use of Spark and Elastic MapReduce as platforms for onward analysis. We have learned that the AWS toolkit is functionally impressive, but that performance for both data gathering and data analysis still needs careful design and some technical finessing.

Machine learning frameworks We’ve also been looking at the effectiveness of machine learning frameworks on both GPUs and CPUs. Our main test case here was inspired by a commercial partner and involves identifying features

Sentinel-2A image of the Putorana Plateau in northern Central Siberia.

Left: mapping an earthquake. In October 2016, a 6.5 magnitude earthquake struck central Italy. Scientists from Italy’s Institute for Electromagnetic Sensing of the Environment used radar images to analyse ground displacements caused by the quake. The results show ground deformations extending across about 130 sq km with a maximum displacement of about 70 cm (in the direction of the satellite). Opposite and EPCC News cover: Sentinel-5P daily global coverage. With a swath width of 2600 km, Sentinel-5P maps the atmosphere around the entire planet every day. Information from this new mission will be used through the Copernicus Atmosphere Monitoring Service for air quality forecasts and for decisionmaking. All images: European Space Agency.

from Earth observation image data (our target is the Sentinel satellite data from the ESA Copernicus programme). Image analysis is a classic machine learning example: throw a large number of images at an algorithm (in our case a 27-layer convolution neural network); train the algorithm to identify the patterns we’re looking for; and test that algorithm on another large number of previously unseen images. The machine learning framework we’ve chosen for the neural network is PyTorch, a Python-based scientific computing package which provides a deep learning platform using the power of GPUs. GPUs (general-purpose graphical processing units) are particularly well-suited to this sort of algorithm, and are attracting a lot of interest in machine learning circles. We trained the network on DeepLearn, a small in-house GPU server set up for running deep learning frameworks. This server contains a single NVIDIA Titan X (Maxwell) GPU, and single quad-core Intel Q6600 CPU. Training the network to classify images over 120 “learning epochs” took 8 hours. In terms of results, the network has a recall score of 83.78% (ie 83.78%

of the pixels that belong to our objects of interest are correctly predicted as such), and a precision score of 84.54% (ie 84.54% of the predicted pixels of interest are genuine pixels of interest), which is encouraging. We wanted to know how much better than CPUs are GPUs for this kind of problem. This led us to explore optimisations of PyTorch running on Cirrus, our Tier-2 HPC system. PyTorch supports use of the multithreaded OpenBLAS linear algebra library, and each Cirrus compute node contains two 18core Intel Xeon E5 (Broadwell) processors, with each core supporting two hardware hyperthreads. A Cirrus node therefore has a possible 72 parallel hardware threads with which to compete with the Titan GPU. And compete it does: the 120 learning epochs that take 8 hours on DeepLearn take 10 on one Cirrus node. Slower, yes, but Cirrus has 280 nodes like this. Our next step will be to experiment with the brand new MPI-like parallel version of PyTorch to try to scale our image classifier across the whole machine.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Rob Baxter, Group Manager, EPCC r.baxter@epcc.ed.ac.uk Marc Sabaté Applications Consultant, EPCC m.sabate@epcc.ed.ac.uk

This is just a selection of our recent work. There are more interesting things going on at EPCC in the space of large-scale data engineering than we’ve room for here – benchmarking large graph databases, for instance. We’ll save those for another issue!

Building earthquake resilience in at-risk communities, EPCC News 81 (p12): http://edin.ac/2i14Xvk 7

The National Safe Haven for research using unconsented NHS data NHS Scotland allows research using routinely collected, unconsented patient data. Additionally, these data can be linked to social data such as education. The research this enables can have an enormous public benefit but the use of these data must be managed very carefully to safeguard privacy and maintain public trust and support.

process to ensure privacy is never breached.

To use these data, researchers must apply to the Public Benefit and Privacy Panel specifying the data that is required and the purpose of the research. If granted permission, the data will be selected, anonymised and linked (a process called pseudo-anonymisation) before being placed within the National Services Scotland (NSS) National Safe Haven. Researchers must process their data from within this National Safe Haven infrastructure. To access the National Safe Haven, researchers must use remote desktop software and log in using a high-security protocol. Researchers are then able to use the remote desktop session to access, process and analyse the data they have requested. The remote computer is installed with several statistics packages that the researcher can use. Crucially, the remote computer offers no access to the Internet neither to receive nor send data. Researchers can request files to be transferred into, or out of, the Safe Haven but such requests are subject to a manual verification

We are currently working with the Health Informatics Centre at the University of Dundee to provide researchers with access to the Scottish NHS imaging data (X-ray, CT, MRI, ultrasound etc) dating from 2010 onwards. Such data offers tremendous opportunities for a wide variety of research including examining early/preclinical diagnosis, disease progression, personalised medicine, genotypephenotype associations, and the development of novel computer vision and machine learning algorithms. Additionally, the ability to link this image data with patient outcome data is unique to Scotland and offers even greater potential for world leading research that will have a major contribution to the future health of the nation.

This approach to providing the National Safe Haven has proven very successful and has supported more than 200 research projects over the two years it has been running. EPCC is responsible for building, supporting and hosting the infrastructure and continues to develop the infrastructure and software to further enhance the service.

Providing imaging data to researchers to support their research goals is technically challenging, and we are excited to be involved in the delivery of a solution that offers the potential of huge health benefits in the future.

Ally Hume Software Architect, EPCC a.hume@epcc.ed.ac.uk

EPCC is responsible for building, supporting and hosting the infrastructure of the National Safe Haven, and we continue to develop the infrastructure and software to further enhance the service.

NHS Data Safe Havens Read more on the NHS website: http://edin.ac/2goWm5b Data projects at EPCC www.epcc.ed.ac.uk/research/data

HPC-UK: a new resource for UK researchers Image: Nguyen Anh Koa Doan, Research Student, University of Cambridge

HPC-UK provides a central location for researchers to find out about available HPC facilities and to provide support to help them get the most out of using HPC in their research. The core resource is an open source, community website that provides links to facilities, information on HPC training and how to access HPC facilities, and support to enable researchers to get the most out of HPC applications.

Facilities It is difficult for researchers to know what HPC facilities are available, and the differences between them. HPC-UK provides an accurate list of the HPC facilities available to UK researchers, both from the UK and internationally. To make the list as relevant as possible, it concentrates on those that have publicised access routes for UK researchers.

Access It can also be difficult to know how to access HPC facilities. HPC-UK brings together all access routes, making it easy for researchers to assess their options and plan applications. Access routes can vary according to the scale of facilities, and HPC-UK offers a simple way to find out which routes are available on which facilities.

Training Providing researchers with the skills to exploit HPC facilities is key to getting the most out of the UK investment in HPC. HPC-UK offers a single portal to the HPC training

available across the UK and online. The training section contains links to different HPC training options, but in future we will provide a live list of upcoming courses to make it as easy as possible to find out what is available.

Applications HPC software generally has a much longer lifetime than the hardware in a particular facility, and is at least as important to researchers for the effective use of HPC. HPC-UK provides information on the performance of applications on different HPC facilities to help researchers choose the correct facility for their research and optimise their use of resources. Descriptions of how to compile different applications on various HPC facilities are also provided to help researchers and those who support them to compile their applications on different HPC architectures.

Andrew Turner Project Manager, EPCC aturner@epcc.ed.ac.uk

HPC-UK is a community resource that brings together information to support researchers using HPC, and the people (such as research software engineers) who support those researchers. It provides a coherent view of what is available, how to access it, where to get training, and how to get the most out of HPC applications. Anyone can contribute to the website, note issues or request features through GitHub.

Future plans We intend to provide a list of local UK HPC Champions to allow researchers to speak in person about their use (or potential use) of HPC, and to increase the amount of application benchmarking information available. HPC-UK is a collaboration between EPCC, the UK Association of Research Software Engineers, and sites running UK national HPC facilities.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

To contribute to HPC-UK please feel free to use the GitHub repository or get in touch. www.hpc-uk.ac.uk GitHub repository https://github.com/hpc-uk/ hpcuk-website 9

Fortissimo: simulation and modelling software for business Since it began in 2013, the Fortissimo project has helped over 100 companies to apply HPC to advanced simulation and modelling problems, and has led to the creation of the Fortissimo Marketplace where users can access simulation software and services. A main objective for Fortissimo was to lower the barriers to HPC for small companies by reducing the cost and risk of using it. The project has carried out over 50 experiments to develop solutions to advanced simulation and modelling problems and enable companies, in particular SMEs, to benefit from HPC. Fortissimo uses a business model derived from cloud computing whereby the end-users access software running on a distributed HPC infrastructure rather than buying and operating their own systems. The pay-per-use model encourages small companies to make experimental use of HPC without a major outlay. If they find it successful, there is an easy route for an end-user to scale up their usage (as with cloud computing). The following examples illustrate the success of Fortissimo in three different fields: Light aircraft design Pipistrel is an SME based in Slovenia. It was established in 1989 and is a leading designer and manufacturer of light aircraft. To develop its product line, Pipistrel needs to understand how air flows over its aircraft. Performing simulations which were sufficiently detailed to model real physical effects accurately was a challenge for Pipistrel. Such simulations require expensive 10

computer resources which are normally beyond the means of an SME. The use of cloud-based HPC allowed Pipistrel to run simulations of a higher fidelity than was possible with its in-house systems, closely modelling real-world behaviour and giving accurate information on how the aircraft would behave in flight. Such a problem would either be too big for the in-house systems or would take too long to run (around 20 to 30 days) to be part of an effective design process. However on the HPC system accessed through Fortissimo, a typical large model would run in only two to three days. Fortissimo therefore enabled Pipistrel to obtain results of much more complex simulations in a reasonable time. It also offered a cost-effective solution to running such large simulations. Innovative construction Spanish SME Maderas Besteiro specialises in timber solutions for structural applications, such as house-building. These products are designed on demand by its technical staff. It has recently taken an interest in offering a new product line of box-framed timber beams. Designing and testing a new structural product is an extremely costly process, especially for composite wood and engineered

Mark Sawyer Business Development Manager & Project Manager, EPCC m.sawyer@epcc.ed.ac.uk Maderas Besteiroâ€™s box-framed timber beams, part of the virtual design process enabled by Fortissimo. The weight of the beams can be reduced by more than ten per cent using the new design process, leading to lower transportation costs and reduced environmental impact. Images: Maderas Besteiro

Image: GexCon AS

wood products. However, the company has reduced development costs and design times by introducing a virtual design process through the Fortissimo Marketplace, which enables access to an HPC platform through a pay-per-use model. The design process is now at least 19% faster. This has significantly reduced development costs and made the design optimisation process feasible. Improved safety in the oil and gas industry Gexcon AS is a market leader in providing software for modelling dispersion, fire, and explosion in hazardous industrial production areas. Gexcon’s software, FLACS, is a multi-physics modelling and simulation tool used worldwide for safety studies and to identify where to place gas detectors in large industrial complexes. Typical gas detection systems comprise several tens of detectors, hence the savings can run into millions of Euros. The new service developed under Fortissimo enables Gexcon’s customers to tap into HPC-cloud resources seamlessly from a user-friendly graphical interface in the same way that, for example, a local cluster would be accessed. It is expected that the new service will increase the utilisation of FLACS in the industry as a whole, as it

removes the financial overhead of investing in hardware that is used only at peak times during projects. Many of Gexcon’s customers are SME consultancy companies whose work is characterised by peak requirements during project execution, interspersed with periods of less activity. This makes the use of on-demand resources attractive, since an investment in computing hardware of several tens of thousands of Euros is replaced by project-based costs. At the same time, the project’s computing phase can typically be shortened by a factor of up to ten, based on computing power that the HPC facility can offer compared to a typical cluster owned by an SME company. High performance data analytics In its latest round of experiments, Fortissimo has expanded its scope to include using high-performance data analytics (HPDA) to solve industrial problems. The applications include analysing data collected from machinery to identify faults and avoid costly breakdowns of production lines. In 2018 Fortissimo will move into a new phase in which the Marketplace will be self-sustaining. We are on the lookout for more partners to provide innovative simulation and HPDA solutions, and of course, customers.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

A major cost of HPC is the capital cost of hardware. HPC systems are very expensive to buy, and only provide a return on investment if they are heavily utilised. In addition, ownership means that specialist staff are needed to run the system, adding to the cost and creating a reliance on skilled personnel who may be hard to recruit and retain. Together these factors make it very risky for a small company to experiment with HPC.

www.fortissimo-marketplace.com www.i4ms.eu 11

SMEs come in all SHAPEs and sizes... SHAPE (SME HPC Adoption Programme in Europe) is a pan-European programme that promotes high performance computing adoption by small and medium sized enterprises. It can be challenging for small and medium sized enterprises (SMEs) to adopt HPC. They may have no in-house expertise, no access to hardware, or be unable to commit resources to a potentially risky endeavour. This is where SHAPE comes in, by making it easier for SMEs to make use of highperformance computing in their business: to improve product quality, reduce time to delivery or provide innovative new services. Successful applicants to the SHAPE programme get effort from a PRACE HPC expert and access to machine time at a PRACE centre to try out their ideas for utilising HPC to enhance their business. So far SHAPE has helped 33 companies to benefit from using HPC in their business, and following the fifth call for applications, six more SMEs will be able to start work with PRACE. The companies cover a diverse range of areas, including prosthetics, electromagnetic simulation, gas burners and propeller design. The latest SHAPE call opened in October and there will be further calls during the PRACE project, so there are plenty of opportunities for SMEs to take part and try out HPC in their business.

SHAPE at EPCC Renuda is a London-based SME specialising in computational fluid dynamics. EPCC is currently working with Renuda to develop CodeRenuda, one of the key serial CFD codes it uses internally, into an HPC code. Renuda has recognised expertise in using and developing software for a large range of industrial thermal and fluid mechanics applications. Working mostly with blue chip clients in the UK and Europe, Renuda is involved in solving a wide range of problems including: how to simulate ventilation and contaminant dispersion in nuclear plants, the investigation and optimisation of mixing in chemical processing industries, and improving the design of gearbox oil cooling circuits. Renuda also has expertise in multi-language, multi-purpose software development of Graphical User Interfaces (GUIs). This SHAPE project will enable a step change in CodeRenuda’s performance. The projected business impact will be significant for Renuda, potentially expanding its client base and strengthening its role as a leading SME in the UK’s industrial CFD community.

Child wearing cranial orthosis (helmet) ©iStock.com/jenjen42

Main image: Instantaneous velocities â€“ velocity field in a hydraulic Francis turbine. Image: Renuda.

Forthcoming projects Disior Ltd, Finland: Numerical optimisation of bone fracture treatment This companyâ€™s software helps with pre-operative planning of bone fracture treatments. It plans to use HPC to develop a very fast method for implant optimisation, which would impact on surgery costs and patient quality of life.

AxesSim, France: HPC for connected objects AxesSim produces electromagnetic simulation software, and the project wants to use HPC to perform simulations in a reasonable timescale for connected objects, eg looking at how the E-M waves produced by small antennae interact with biological tissues.

Brabant Alucast, Italy: Multiphasic simulation in high pressure die casting process in HPC platforms using open-source CFD Brabant is a specialist in advanced engineering, supporting the design and development of high pressure die-casting components for the automotive industry. It wants to use HPC and open-source technologies to evaluate potential benefits to cost, time-to-results for designs, and number of designs that can be investigated in a day.

E&M Combustion S.L., Spain: High-fidelity simulation of an industrial swirling combustor E&M designs and manufactures combustion systems, such as burners and boilers, mainly for the energy, oil and gas, steel and metal industries. Its market demands higher efficiency in the burners and combustors in terms of fuel consumption and flexibility of operation, but also in reducing emissions, which has clear societal impact. It hopes to use HPC in optimising its combustor designs.

Invent Medical Group, Czech Republic: Numerical Modelling of 3D Printed Cranial Orthoses This medical start-up aims to use 3D scanning, computational modelling, and 3D printing in the field of custom-made orthotics and prosthetics. It designs and makes cranial orthoses, and wishes to replace physical testing with virtual prototyping using numerical modelling and simulation.

Svenska Flygtekniska Institutet AB, Sweden: AdaptiveRotor This SME (the Swedish Aeronautical Institute) focusses on research, education and consulting work in aeronautics. This project aims to increase the propulsive efficiency of aircraft propellers and so reduce pollution and noise. It wants to use HPC to evaluate different design candidates for propeller blades.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Paul Graham, Software Architect and SHAPE coordinator, EPCC p.graham@epcc.ed.ac.uk

SHAPE is supported as part of the PRACE (Partnership for Advanced Computing in Europe) initiative.

Contact the SHAPE team: shape@prace-ri.eu SHAPE website www.prace-ri.eu/hpc-access/ shape-programme 13

Building the road to Exascale With support from the European Commission’s Horizon 2020 programme and collaborating with peer supercomputing centres around Europe, we have assembled a portfolio of R&D projects to address the panoply of Exascale challenges. Three projects in particular focus on enabling next generation HPC and high-performance data analytics (HPDA) systems, aiming towards tens of millions of processing elements and requiring extremescale parallelism, data management, I/O, resiliency, and efficiency. NEXTGenIO EPCC leads the NEXTGenIO project, collaborating to design and build a prototype system that uses a new form of non-volatile technology, Intel and Micron’s 3D XPointTM memory. This includes the design and development of the software and systemware required to enable applications to use this new form of memory hardware. 3D XPointTM is non-volatile memory in DIMM form, meaning it can be connected directly to the same memory controller as current DRAM technology. Applications can access this memory directly, reading and writing bytes as required. 3D XpointTM has much higher storage capacity than standard memory and, being non-volatile, will keep data even when the host system is powered off. While this new memory technology promises great performance benefits for applications – enabling many times faster I/O and data access than current storage technology (even SSDs), and enabling new ways to store and access data (such as object stores) – there are many technical challenges to be addressed to enable such memory to be integrated into large HPC or HPDA systems. 14

Having non-volatile memory in compute nodes puts requirements on job schedulers to extend beyond scheduling jobs based on number of nodes and maximum runtime. Understanding data dependencies between applications, and storage requirements of user workflows, will enable efficient and effective use of the storage-class memory present in a system. We are focusing on integrating awareness of non-volatile memory into resource management: designing and developing dataaware schedulers; investigating check-pointing software; and looking at I/O and communication libraries, to identify necessary modifications and optimisation to support this new method of working. DEEP-EST The DEEP-EST (DEEP - ExtremeScale Technologies) project builds on two previous European projects, DEEP (Dynamical Exascale Entry Platform) and DEEP-ER (DEEPExtended Reach), to design and implement a Modular Supercomputer Architecture (MSA). The MSA integrates compute modules (multi-core nodes, manycore nodes, GPU nodes etc) with different performance characteristics into a single heterogeneous system. Each module is a parallel computer in its own rights, and modules are connected by specialised networks. The MSA is designed to enable heterogeneous applications or application workflows to run efficiently, by running different parts

George Beckett Project Manager, EPCC g.beckett@epcc.ed.ac.uk Adrian Jackson Research Architect, EPCC a.jackson@epcc.ed.ac.uk Michèle Weiland Project Manager, EPCC m.weiland@epcc.ed.ac.uk

The path to Exascale is on the minds of many HPC professionals at the moment, and EPCC is no different. We recognise that Exascale is a multifaceted problem, requiring radical advances in a number of areas – processor design, energy efficiency, memory and storage, hardware resilience and software. We also recognise that the path to Exascale needs to be forged by real requirements and by end-user science ambitions that can only be realised at the peak of scientific computing.

©iStock.com/Alexander_Bushenkov

of an application (or workflow) on hardware that best matches its performance requirements. This should improving both time-tosolution and energy use. The DEEP-EST prototype will include three modules: a general purpose Cluster Module (ie standard multi-core nodes); an Extreme Scale Booster (ie many-core or GPU nodes); and a Data Analytics Module specifically designed for HPDA workloads. As well as integrating heterogeneous hardware into a single system by adapting job scheduling and resource management system, DEEP-EST undertakes co-design with a range of HPC and HPDA applications to ensure the prototype delivers the performance benefits required for Exascale computing and to evaluate the DEEP-EST prototype. Within DEEP-EST, we collaborate on energy monitoring and modelling, data analytics programming environments, and the effective integration and efficient use of non-volatile memory in such a system. INTERTWinE INTERTWinE addresses software, tackling the problem of running the most important HPC applications efficiently and effectively on an Exascale-like platform. The INTERTWinE team anticipates that very few applications can be re-written to be “Exascale-ready” in time for the emergence of the first such systems, and thus the vast majority of application users will rely on software written with today’s prevalent programming models,

such as MPI and OpenMP. Because of this, INTERTWinE is investigating and addressing the barriers to efficiency and scalability when today’s programming models are combined (‘hybridisation’) to create hierarchies of parallelism. Led by EPCC, INTERTWinE is a collaboration of nine partners, with representatives from world-class programming-model and parallel interface teams around Europe. EPCC is exploiting its long track record in development of MPI and OpenMP standards; the GASPI team from Fraunhofer ITWM bring great experience of efficient message-passing and distributed memory management; KTH, Inria and BSC contribute significant expertise in task-based programming and parallelisation; Universitat Jaume I de Castellon, T-Systems, and the University of Manchester provide expertise in algorithmic and hybridised parallelisation strategies; and commercial concerns are addressed by Deutsches Zentrum fuer Luft und Raumfahrt (DLR). Standardisation and training are at the heart of INTERTWinE’s philosophy. The team has leading roles in the MPI Forum, GASPI Forum, and OpenMP Architecture Advisory Board, ensuring interoperability is a priority for their work. The project also has a significant training portfolio, covering advanced parallelisation and hybridisation techniques, recognising that today’s application developers are the people who will prepare software for Exascale.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

NextGenIO www.nextgenio.eu DEEP-EST www.deep-projects.eu INTERTWinE www.intertwine-project.eu

ARCHER community: get involved! The ARCHER service has been developing an open community where expertise and knowledge can be shared. It is designed to be inclusive and to engage with users of all HPC services in the UK: ARCHER, Tier-2 and University-level systems. There are a number of ways you can get involved...

ARCHER Champions This programme promotes a coherent access structure to HPC resources across the UK, with coordination between tiers. It supports and promotes activities designed to provide career development to research software engineers seeking a career in HPC, and provides a support network for staff members whose role involves advising users on access to local, regional and national HPC resources. We run a series of workshops at various locations across the UK.

ARCHER Outreach Ambassadors This activity provides ARCHER users with the resources to engage in outreach activities and act as Outreach Ambassadors for the ARCHER service. We are developing an outreach pack that will contain a series of printable resources, ideas for outreach demonstrations and 16

appropriate videos/presentation material. Ambassadors will be able to use their material in local schools and at local science events, offering outreach opportunities to young people across the UK.

Diversity in HPC Gender, ethnicity, age, family or any other personal characteristic should not be a barrier to participating in this exciting field. Anyone with an interest in learning about HPC or using it in their research can start here to find out about the people who work in HPC, their careers, and the routes they have taken to get there. We believe that we need to actively foster change within our workforce to ensure that those from underrepresented groups have the fullest opportunity to engage and contribute to the HPC community. ARCHER has developed a new resource celebrating Diversity in HPC, showcasing inspiring Faces of HPC, and also sharing best practices to bring about positive

Lorna Smith Group Manager, EPCC l.smith@epcc.ed.ac.uk Toni Collis Applications Consultant, EPCC a.collis@epcc.ed.ac.uk

ARCHER Champions and ARCHER Ambassadors If you are interested in becoming involved in either initiative, please get in touch: support@archer.ac.uk

Diversity in HPC www.hpc-diversity.ac.uk

change within organisations and our activities. Faces of HPC Faces of HPC highlights real people working in HPC today, and some inspiring role models. We want to provide role models for everyone, highlighting the important contribution that everyone makes to the future of our work, irrespective of their personal characteristics. Faces of HPC has highlighted the variety of people working in HPC, from social scientists to physicists, and the importance of interdisciplinary contributions to the community. Bringing about change in the community As part of the Diversity in HPC work we have gathered best practice for fostering change to improve diversity. We have guides on: • Improving Accessibility to HPC training • Improving diversity at HPC conferences and events.

Women in High Performance Computing Women in High Performance Computing (WHPC) is part funded by the ARCHER Outreach project, including running the series of international workshops. Our activities focus on bringing equality of opportunity for all to the community, promoting and building a diverse and inclusive HPC workforce by enabling and energising those in the community to increase the participation of women, and highlighting their contribution to the success of supercomputing. WHPC needs you! WHPC is for everyone - not just women. We are a community of like-minded people who wish to address the under-representation of women. Individual membership is free, and members receive our monthly newsletter.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Women in HPC We rely on volunteers to help run our activities, events and fundraising. If you are interested in getting involved, email us for more information: info@womeninhpc.org Or join now: www.womeninhpc.org/ membership

Faces of HPC We are always looking for new contributions. If you love HPC and want to share what encourages you to participate in the community, get in touch: info@hpc-diversity.ac.uk. You can read some of the interviews here: www.hpc-diversity.ac.uk/facesof-hpc

Bridging the skills gap The skills gap in the IT, digital and cybersecurity industries in Scotland is widening with many employers, including EPCC, finding that both new and existing staff lack technical, practical or job specific skills. To help bridge this gap, Edinburgh Napier University’s School of Computing has received funding from Skills Development Scotland (SDS) to develop and deliver new work-based Graduate Level Apprenticeship (GLA) courses. GLAs provide a development pathway for individuals to attain the necessary knowledge, skills and competencies required by employers. There is no upper age limit for entry and courses are open to new staff or existing employees looking to develop their career. We are delighted to be participating in the GLA scheme as it will contribute to the success of EPCC and enable us to retain top talent by supporting and investing in career development. Emilio Perez, an HPC Systems Administrator at EPCC’s Advanced Computing Facility (ACF), started a four-year BEng (Hons) in Cybersecurity this September. Here is what he has to say: “I believe the GLA is a great way to study for a degree as it is fully tailored to the limited availability of a full-time employee like myself. “The lectures that normal students 18

Emilio Perez, HPC Systems Administrator at EPCC’s Advanced Computing Facility, has started a four-year BEng (Hons) in Cybersecurity.

attend during the week are adapted for GLA students so they all take place on the same day. The fact that GLA students have a day off work every week for the first two years and once a month for years 3 and 4 to attend lectures is also key as it allows us to dedicate the time that a degree requires.

Emilio Perez HPC Systems Administrator, EPCC e.perez@epcc.ed.ac.uk

“One of the main reasons I decided to do the GLA is because my first degree is not an IT degree and I believe that getting a degree in IT will help me to consolidate my current knowledge, improve my background skills and my desire to specialise in Security.

The available Graduate Level Apprenticeship courses are:

“Due to my background in System Administration for the past 10 years and my Ethical Hacker certification I have been always attracted to the security side of System Administration and I think the Cybersecurity degree is an extraordinary opportunity for me to keep specialising in this area. “It will also be very beneficial for the team at the ACF as maintaining the security of our systems and services is a vital part of our work, so the skills I will obtain from this degree will be very useful.”

Maureen Simpson, EPCC m.simpson@epcc.ed.ac.uk

• BSc (Hons) Software Development • BSc (Hons) IT Management for Business • BEng (Hons) Cybersecurity

www.napier.ac.uk/study-withus/apprenticeships

Our online courses in data science and HPC

Our online postgraduate courses in High Performance Computing and HPC with Data Science will run for the third time in January 2018. These can be stand-alone courses or taken as part of an online Masters, Diploma or Certificate on the University of Edinburgh’s online Data Science, Technology and Innovation (DSTI) programme. I am the Course Organiser for Practical Introduction to Data Science, and I’ve been excited to be part of this effort since we started to put these courses together in 2015. I’m also now taking over as Programme Director for the wider DSTI programme. This unique programme brings together courses from all three of the University’s Colleges and offers varied perspectives on data science and related fields. From medical informatics to machine vision and robotics, there are courses for people with an array of backgrounds and interests. On these courses you’ll have access to many of the same machines and resources as our on-campus students, including (in our Intro to HPC course) ARCHER, the UK’s national supercomputer service. These are postgraduate courses, so they are not open to everyone in the way that a MOOC1 is, but they are designed to be accessible to

anyone with an undergraduate degree in a numerate discipline or equivalent work experience. We don’t require knowledge of any particular programming language, but courses like mine are practical and will require you to do some programming, so if you’ve never done any programming before you should be ready for a steep learning curve. You’ll be studying these subjects to the same level as you would in an on-campus Masters and you will receive an equivalent level of personalised support and feedback, but because it’s online, you can take the courses over a period of up to six years from wherever in the world you are. What’s more, if you’re thinking of signing up for the Masters programme, you can start with (and therefore pay for) just a single course. If you then want to take things further, you can move on to the full programme and count the credits you have already earned2. But if you want “free” and “open” we also run a Supercomputing MOOC. See page 22.

Providing you finish the programme within the maximum prescribed time.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Electric field generated by the heart in the human body at the beginning of a heartbeat. The use of ARCHER is allowing the investigation of multiple diseases of the human heart. Image: Dr Alfonso Bueno Orovio, Department of Computer Science, University of Oxford.

Adam Carter Project Manager, EPCC a.carter@epcc.ed.ac.uk

Practical Introduction to Data Science gives a hands-on introduction to many of the important aspects of data science, from data management to data mining. Practical Introduction to HPC covers all the fundamental concepts that underpin modern HPC and will allow you to explore these topics by running parallel programs on real HPC systems.

EPCC’s online courses: www.epcc.ed.ac.uk/onlinecourses University of Edinburgh’s DSTI programme: www.ed.ac.uk/data-science/ postgraduate

MSc programmes update Autumn can be a busy time for those involved in the MSc programmes in High Performance Computing (HPC) and HPC with Data Science. The dissertations of the previous year’s MSc students are marked, and we welcome a new intake of students and update materials for the new year.

Farewell to the Class of 2017 Although they have finished their MSc studies, the students will definitely not be forgotten. Their graduations on 30 November will be celebrated; one student has just started working in EPCC’s Systems Development Team, and another has made the short journey to the University’s Central Area to begin a PhD in Informatics. Others have well and truly flown the nest with graduate destinations around the world. We look forward to catching up with their progress in a future edition of EPCC News. The MSc programmes are a constantly evolving ecosystem and we’re grateful to the Class of 2017 for their feedback so we can continue to improve the programmes, especially as we prepare for one of the biggest changes in the programmes’ history. 20

New home, new start In 2018, EPCC will relocate to the Bayes Centre in the University’s Central Area. This means that from the start of the 2018/19 academic year, teaching will leave the King’s Buildings campus for the first time. We are excited about the possibilities this opens up and the changes it may require. The ability to step straight from classes into the historic heart of the city of Edinburgh is certainly one which we hope our future years of students will enjoy.

Changes Academic year 2017/18 was not without its changes either. We are delighted to welcome MSc alumnus Manos Farsarakis as the new Course Organiser of the HPC Architectures Course, and Dr Rupert Nash as the Course Organiser of the ever-popular Parallel Programming Languages optional course.

Above and left: some of the Class of 2018. Below left: MSc students and staff take part in a team-building exercise by Dr Frances Parry from the University’s Institute for Academic Development. All photographs taken by EPCC’s Mario Antonioletti during Induction Week.

Dr Stephen Booth’s great passion and expertise in the development and teaching of the Performance Programming course cannot be overstated and although it is most certainly not the end of his teaching on the course, he steps down as Course Organiser to be replaced by MSc alumnus Adrian Jackson. We are delighted to welcome students from all over the world to their studies here at EPCC. The Class of 2018 is one of our most diverse and international groups ever, and it has already developed a strong cohort dynamic and social outlook – demonstrating that the University experience can be just as much about learning from your classmates and peers as from the lecturers themselves. The student induction process this year was expanded to three days and we were thankful to Dr Frances Parry from the Institute for Academic Development for her input to the first day. This included

an overview of the differences between undergraduate and postgraduate study, and an effective demonstration of how teamwork and dialogue with peers can be crucial to success. The ‘additional’ day in the induction calendar gave the new students the opportunity to visit the secure Advanced Computing Facility (ACF), allowing them a glimpse of the hardware so central to their studies, and to hear directly from the team responsible for maintaining such nationally important infrastructure. The MSc team is hugely grateful to our ACF colleagues for hosting us and providing the students with such an opportunity so early in their studies. We will launch a new blog series later in 2017, which will give you the opportunity to keep up to date with the MSc programmes, their staff, and students in our final year on the King’s Buildings campus.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Ben Morse MSc Programmes Officer, EPCC b.morse@epcc.ed.ac.uk

See our website to find out about our MSc programmes in High Performance Computing (HPC) and HPC with Data Science: http://edin.ac/epccmsc Institute for Academic Development www.ed.ac.uk/instituteacademic-development

Supercomputing MOOC: it’s massive! We are nearing the end of the second run of our Massive Open Online Course (MOOC) on “Supercomputing”. So how has it all gone? Our Supercomputing MOOC is hosted on FutureLearn, the social learning platform. For the first run in March this year, 3,263 people registered and 1,721 people accessed content, which is a typical conversion rate for FutureLearn. The course went very well, with the most surprising (and pleasing!) aspect being how much interaction there was between learners in the various discussion sessions. For the second run we chose to make only a few updates: • Re-record some of the “talking head” videos to improve audio quality. • Add more informal quizzes so learners can check how well they are following the material. • Write a couple of short articles addressing artificial intelligence, which was a hot topic of conversation during the first run. Even with this additional material, rolling out the second run was much less effort (and far less stressful!) than the first. 22

By the end of the five-week course we had over 2,800 registrations with almost 1,700 accessing content, which is a higher conversion rate than for the first run. As “Educators” we try to check up on how things are going and contribute comments on a daily basis, but just as before there is an enormous amount of mutual support among the learners. Our expectation was that it would be a lot of work to develop the material in the first place (compared to a traditional “chalk-and-talk” lecture course), but less effort to run. This has been borne out in practice in spades: as MOOC novices we found creating interactive content and videos quite challenging, but I am constantly surprised at how much the MOOC almost runs itself on a week-byweek basis (although helped to a great extent by the many pearls of wisdom contributed by the Educators!). The MOOC was developed in collaboration with SURFsara (Netherlands) on behalf of the Partnership for Advanced Computing in Europe (PRACE).

David Henty, Group Manager, EPCC d.henty@epcc.ed.ac.uk

This course is for anyone interested in leading-edge computing technology, supercomputers or the role that computer simulation takes in science and engineering. There is no requirement to write computer programmes.

We plan to run the MOOC twice a year with the next run in January 2018. See: www.futurelearn.com/courses/ supercomputing For other MOOCs offered by PRACE see: www.futurelearn.com/partners/ prace

Building a Virtual Human: the movie In September the CompBioMed Centre of Excellence unveiled the film “Building a Virtual Human” at the Science Museum’s IMAX theatre in London in front of 100 invited guests, and 300 members of the public. To create truly personalised medicine, doctors dream of creating a digital Doppelgänger of you, one that can be a guinea pig, crash test dummy, and a drug trial volunteer all rolled into one. That way they can perfect medical treatments on your virtual clone before they try them out on you. Virtual organs are already taking shape as a result of efforts by many groups worldwide. These include the CompBioMed Centre of Excellence which carries out medical simulations using supercomputers such as ARCHER, the UK national supercomputer; Marenostrum, at the Barcelona Supercomputing Centre, Spain; and SuperMUC at the Leibniz Supercomputing Centre, Germany. The CompBioMed IMAX event was part of the Science Museum Lates series of themed evenings, which each attract 4,000-7,000 people. The hour-long event described recreating a human being in silico, and included the showing of CompBioMed’s IMAX video, which shows stunning simulations of aspects of computational biomedicine. It proved to be very popular, with the free tickets all going before the event.

Production of the film involved numerous Core and Associate Partners within the CompBioMed consortium. Led by University College London and directed at Barcelona Supercomputing Center, the diverse computational models were incorporated in the film to take the audience through the simulation of various aspects of the human body, and the future possibility of creating a virtual human. The leaders of the research groups whose work was featured in the film each gave a 5-minute presentation, followed by questions to a panel:

Emily Lumley Project Manager, CompBioMed e.lumley@ucl.ac.uk EPCC is one of 15 core partners in CompBioMed. This user-driven Centre of Excellence in Computational Biomedicine will nurture and promote the uptake and exploitation of high performance computing within the biomedical modelling community. Its users come from academia, industry and clinical practice.

• Prof Peter Coveney (UCL), CompBioMed consortium lead, discussed simulating how drugs work in the body • Prof Blanca Rodriguez (Oxford University) spoke about virtual hearts • Prof Marco Viceconti (Sheffield University), a key player in the Virtual Physiological Human Initiative • Prof Alfons Hoekstra (University of Amsterdam), spoke about virtual blood vessels and more. The film will be shown at a number of film festivals and events.

The newsletter of EPCC, the supercomputing centre at the University of Edinburgh

Main image: Dynamics of mutated cancer therapy target (EGFR). Above: Atomistic simulation of the thermal motion of a supercoiled DNA plasmid.

CompBioMed website: www.compbiomed.eu 23

Study HPC in the heart of Edinburgh

Master’s degrees in High Performance Computing (HPC) and in HPC with Data Science EPCC is the UK’s leading supercomputing centre. We are a major provider of HPC training in Europe, and have an international reputation for excellence in HPC education and research. Our two MSc programmes have a strong practical focus and provide access to leadingedge systems such as ARCHER, the UK National HPC Service, and Cirrus, an EPSRC Tier-2 National HPC facility. All students can undertake an industry-based dissertation project.

“High standard of lecturers and access to world-class computing facilities.” 2016/17 MSc in HPC student

Both programmes will be located in the new Bayes Centre in central Edinburgh from September 2018. The University of Edinburgh is consistently ranked among the top 50 universities in the world.* * Times Higher World University Ranking

“The use of exceptional hardware under the guidance of knowledgeable professors.” 2016/17 MSc in HPC with Data Science student