You are browsing the Blog for blog-technical.

SciNet’s publication about Niagara deployment

August 3, 2019 in blog, blog-general, blog-technical, for_press, for_researchers, for_users, frontpage, news, Road_to_Niagara

Have you ever wondered how a supercomputer is designed and brought to life?
Read SciNet’s latest paper on the deployment of Canada’s fastest supercomputer: Niagara.

Niagara is currently the fastest supercomputer accessible to academics in Canada.
In this paper we describe the transition process from our previous systems, the TCS and GPC, the procurement and deployment processes, as well as the unique features that make Niagara a one-of-a-kind machine in Canada.

Please cite this paper when using Niagara to run your computations, simulations or analysis:
“Deploying a Top-100 Supercomputer for Large Parallel Workloads: the Niagara Supercomputer”, Ponce et al, “Proceedings of PEARC’19: Practice and Experience in Advanced Research Computing on Rise of the Machines (Learning)”, 34 (2019).

Learn more about SciNet’s research and publications by visiting the following link.

Comments Off on SciNet’s publication about Niagara deployment

Road to Niagara 3: Hardware setup

March 5, 2018 in blog-technical, for_press, for_researchers, for_users, news, Road_to_Niagara, Uncategorized

This is the fourth of a series of posts on the transition to SciNet’s new supercomputer called “Niagara”, which will replace the General Purpose Cluster (GPC) and Tightly Coupled Cluster (TCS). The transition to Niagara will take place in the fall of 2017, and the system is planned to be available to users in early 2018.

The University of Toronto has awarded the contract for Niagara to Lenovo, and some of the details of the hardware specifications of the Niagara system have been released:

The system will have the following hardware components:

  • 1,500 nodes.
  • Each node will have 40 Intel Skylake cores (making a total of 60,000 cores) at 2.4 GHz.
  • Each node will have 200 GB (188 GiB)of DDR4 memory.
  • The interconnect between the nodes will be Mellanox EDR Infiniband in a Dragonfly+ topology.
  • A ~9PB usable shared parallel filesystem (GPFS) will be mounted on all nodes.
  • A 256TB Excelero burst buffer (NVMe fabric, up to 160 GB/s) will be available for fast I/O.
  • Peak theoretical speed: 4.61 PetaFLOPS

Niagara is estimated to be installed and operational towards in March 2018, and ready for users not too long after.

Even before official ready-date, there will a period in which select users can try out and port their codes to Niagara.

After the friendly-user period, all current users of the GPC (and former users of the TCS) will get access to Niagara.

The large core count, ample memory per core, and fast interconnect support Niagara’s intended purpose to enable large parallel compute jobs of 512 cores or more.

The software setup will also be tailored to large parallel computations. Nonetheless, there will still be a fair amount of backfill opportunity for smaller jobs.

The setup of Niagara is intended to be similar in spirit to the GPC, but different in form: scheduling per node, a home, scratch and possibly project directory defined in environment variables, a module system, and access to our team of analyst to help you get your codes running, and running well.

Comments Off on Road to Niagara 3: Hardware setup

Gravitation waves detected, again!

June 15, 2016 in blog, blog-general, blog-technical, for_press, for_researchers, for_users, in_the_news, news, success_story, Testimonials

We congratulate the LIGO and Virgo collaborations to the second-ever observation of gravitational waves from colliding black holes.

SciNet is proud to have contributed to the computation of the waveform templates that were used in this latest discovery of LIGO. LIGO measured about 55 gravitational wave cycles for this new binary black hole system. This large number of cycles made detailed computations of the expected wave-shapes more important than for the first detected black hole merger that was announced in February.

Canada is a leader of numerical calculations of colliding black holes, research led by Professor Harald Pfeiffer, Canada Research Chair for Numerical Relativity and Gravitational Wave Astrophysics at the University of Toronto. Pfeiffer states: I am very grateful for the sustained support of the SciNet team during the last 7 years; their support and the access to computing time on SciNet’s supercomputers have been crucial for my research program and its profound contributions to the LIGO discovery.

frame06508

Above: The in-spiral and collision of two black holes similar to GW151226. The top portion of the frame shows the horizons of the two holes, in this case, at the moment close to the merger of the black holes. The middle portion of the frame shows the gravitational waveform projected onto the LIGO Livingston detector. The bottom part shows the frequency of the gravitational waves, gradually increasing from about 35Hz to above 700Hz. For this system, LIGO could observe many more gravitational wave cycles than for the first discoved system (named GW150914).

Visualization done by University of Toronto Undergraduate student Aliya Babul & Prof. Harald Pfeiffer, within the SXS Collaboration/www.black-holes.org.

Comments Off on Gravitation waves detected, again!