HMS IT is upgrading to a new storage system designed to efficiently manage large amounts of data at high speed. The Research Data Migration project aims to streamline the storage architecture to a centralized, stable, and secure infrastructure. As of March 15, 2024, O2 Home folders and scratch have already been migrated to the new storage successfully, with several benefits and efficiencies already achieved:  

  • Automated deduplication has increased the total available capacity in these directories by 35%.  
  • Automated data compression functionality has reduced data traffic to the storage system by 60%. 
  • The new storage system reduced energy usage by 75% compared to energy use on Isilon. 
  • The storage footprint has been reduced by 66%, leading to significant savings in future data center operational costs. 

Migration plan and impact 

The project will run through August 2024, and data migration to the new storage system will be completed in phases to minimize disrupting researcher workflows.  

Outages of specific filesystems in the existing storage system will be needed to migrate data between the old and the new environment. Outages will occur every six weeks, lasting less than 48 hours. Unless otherwise noted on the project update website, it is expected that O2 will remain fully operational during outage windows, while the filesystem being migrated will be unavailable during the specified outage window. Outages will be announced 60 days in advance via the O2 announce email list.  

Review the detailed timeline that follows for additional details about scheduled upgrades and maintenance windows. 

All data will be migrated by September 2024, when the HMS community can look forward to a stable, modern, and more performant storage environment.

Outages and impacts

StartEndFilesystemImpact description 
February 13 at 5:00 PMFebruary 15 at 10:00 PMO2 HomeInaccessible 
February 13 at 5:00 PMFebruary 15 at 10:00 PMSlurm softwareJob scheduling unavailable 
February 13 at 5:00 PMFebruary 15 at 10:00 PMO2 AppsInaccessible 
April 8 at 9:00 AMApril 9 at 10:00 AMo2groups/htemRead and write unavailable, viewing not affected 
April 13 at 9:00 AMApril 16 at 5:00 PM/n/data1Inaccessible 
April 13 at 9:00 AMApril 16 at 5:00 PMO2 clusterCertain O2 tools or commands cannot be run 
May 3 at 5:00 PMMay 4 at 5:00 PMCryoEMData Inaccessible 
May 3 at 5:00 PMMay 4 at 5:00 PMCellbio_gygi_VM Inaccessible 
May 24May 26/n/data3Read and write unavailable, viewing not affected 
May 27 at 5:00 PMMay 28 at 5:00 PML4Inaccessible 
May 29 at 9:00 AMMay 31 at 9:00 AM/n/data2Inaccessible 
May 29 at 9:00 AMMay 31 at 9:00 AM/n/no_backup2Inaccessible 
May 29 at 9:00 AMMay 31 at 9:00 AMOMEROInaccessible 
June 25 at 5:00 PMJune 27 at 5:00 PM/n/groupsInaccessible 
September 2 at 5:00 PMSeptember 5 at 10:00 AMStandby StandardInaccessible 

Consultation and support 

As usual, throughout the process, support for researchers will be available through rchelp@hms.harvard.edu