Overview:  

Standby is a new storage offering, built as an alternative to Tier 2. Standby should be leveraged for infrequently accessed data that is still directly available for reference, retrieval or analysis.  Standby can act as an intermediary location; a space to organize and prepare research data for long-term retention.  

Standby can also be used to store large data sets, transferring portions of the data back and forth to the Active space for analysis.  Data stored in Standby does not need to be recalled or downloaded for access, the data is available immediately.  

Standby is accessible via desktop or transfer nodes on O2, providing a similar degree of data protection, including snapshots and off-site backup.  

 

How to Access: 

Multi-protocol Access 

Standby Storage can be provisioned in both NFS and SMB. Windows and MacOS clients typically access shared storage over SMB, whereas Linux hosts access Standby Storage via NFS.

    Access via:  

    • Windows: \\standby.files.med.harvard.edu  
    • Mac OSX: smb://standby.files.med.harvard.edu/  

    To access and transfer data to/from Standby and the O2 cluster: 

    • Login to the transfer cluster at transfer.rc.hms.harvard.edu; /n/standby is not available from O2 login or compute nodes.
    • Please ensure you are connected to the HMS VPN network. 
    • To migrate data from an existing O2 folder to a Standby folder, we recommend you utilize this approved method: 
      • rsync - Copies files either to or from a remote host, or locally on the current host  
        • (e.g. rsync -a /n/groups/lab/tier2 /n/standby/hms/dept/lab/) 

    To transfer data to/from Standby and research.files (collaborations):

    • Connect to remote desktop to mount research.files.med.harvard.edu and standby.files.med.harvard.edu  
    • Open two windows, one to your research.files folder and one to your standby.files folder
    • Begin to 'drag and drop' folders into the Standby ‘collaborations’ subfolder

    IMPORTANT: At this time, please only use the drag/drop method to transfer data to Standby. Users should refrain from copying/pasting data or using the Windows “Move to Folder” option as this has led to confusion with duplicate datasets. Please do not migrate data from n/files to the Standby 'collaborate' subfolder using the transfer server on O2, this will interfere with the existing permissions structure. 

    • Note: Transferring files while using VPN will be quite slow and can interfere with other users accessing HMS resources over the VPN.  
    • If a remote desktop is not available, please contact rdmhelp@hms.harvard.edu to discuss further options.  

     

    Eligibility  

    HMS, HSDM, and HSPH quad-based faculty and staff are currently eligible to use Standby storage. Other HMS affiliates may also qualify for access. Please email rdmhelp@hms.harvard.edu to discuss available storage options.  

     

    Directory Structure  

    For new labs, a Standby storage folder will be created as part of the Active storage request, to encourage integration of Standby in your lab’s data workflow. If you belong to an established lab, you can request creation of a Standby folder by filling out the Standby Storage Form. Modification of the directory structure is available to all users of the directory. Researchers are advised to structure the folders to correspond to how the records were generated, to complement proposed or existing workflows. 

      

    Permissions 

    When a Standby folder is created, HMS Storage will add two subfolders within the lab folder called 'collaborations’ and ‘compute,’ which are designed to mirror permissions set on Active storage. The ‘compute’ subfolder will store data generated from computations on O2 and the ‘collaborations’ subfolder will store data originating from collaboration folders (research.files). Please make sure data are moved into the correct subfolder, depending on the source location; this is necessary to maintain correct permissions in Standby.

     

    Filesystem Quotas 

    Filesystem quotas will be applied to all lab level directories. They enable HMS to appropriately scale and lifecycle stored research data. Defining and limiting filesystem sizes allows for adequate provisioning of standby capacity to the HMS community. 

     

    Quota Notifications  

    Storage limits are designed to allow researchers to continue to conduct research while optimizing the use and cost of your storage, and can be modified to accommodate changing research needs. When a folder grows above the pre-established limit, automatic email notifications are sent to the responsible party (often the PI) and the Lab Data Manager (if one has been designated by the PI) so they can work with lab members to manage the lab’s storage usage. If you are the responsible party for more than one folder, you will receive individual emails with details about each folder.  

     

    Request Storage:  

    Allocation amounts dependent on lab needs and available resources.  

    To request storage: Standby Storage Request Form

     

    Contact:  

    Research Data Management: rdmhelp@hms.harvard.edu