Globus and the Cloud

Here is a list of what is (and is not) supported. Click on a cloud service’s name for more information.

Amazon S3

Amazon S3 buckets—in all regions—may be accessed through the SRCC SCG AWS S3 collection.

Go here for detailed information on how to use Globus with Amazon S3.

Using Globus with Amazon S3 requires an IAM User with appropriate permissions to list buckets, list objects within buckets, download objects, (optionally) upload objects (which includes managing multi-part uploads), and (optionally) delete objects. Globus’ documentation has an example IAM Policy, or see the step-by-step instructions.

Globus for Amazon S3 supports working with buckets in other accounts (cross-account access), as long as access is granted directly to your IAM User. Access via IAM Roles is not supported. Both you and the account owner must give your IAM User access to the bucket.

Globus does not support Requester-Pays buckets, but this feature has been requested. An announcement will be posted when this feature is available.

Globus also does not support custom metadata/tags, ACLs, and additional checksum algorithms. Only the latest version of objects will be accessed. Uploaded objects will be placed into the S3 Standard storage class.

S3-compatible storage

The Globus S3 connector supports more than just Amazon S3. If you run a storage platform that supports the S3 API, the Globus-for-S3 add-on may support it.

As of this page’s last update, besides Amazon S3, we are aware of the Globus S3 connector being used with Dell EMC ECS on-prem, and with Wasabi (as mentioned later on this page).

If you would like to use the Globus S3 connector with your own storage platform, you may do so! You will need to set up a Globus Connect Server v5 endpoint, along with an S3 storage gateway and at least one mapped collection.

For compability across multiple products, the Globus S3 connector uses a limited subset of the S3 REST API. Your storage platform will need to support the following aspects and operations:

• For authentication, v4 signatures are used with Access Keys and Secret Keys.

• Globus uses path-style for bucket access, instead of virtual-host style.

• Globus makes a GetBucketLocation call before anything else, to identify which Amazon S3 region’s endpoint should be used. As long as your storage platform returns something (instead of an error), Globus will ignore responses it cannot parse. If your platform has multiple instances or endpoints, you may need a separate storage gateway for each.

• The ListObjects and GetObject calls are used for directory listings and downloads, respectively. Globus follows the convention of using the forward-slash character as a ‘directory’ separator.

• Uploading objects use the following operations: PutObject, ListMultipartUploads, ListParts, CreateMultipartUpload, CompleteMultipartUpload, UploadPart, and AbortMultipartUpload.

• Deleting objects uses the DeleteObject operation.

Unfortunately, SRCC are unable to host or administer a Globus endpoint for your storage connector, but you may still contact us for general advice.

Box

The Stanford Box service was retired on February 28, 2023. At that time, Globus for Box stopped working. It is no longer possible to access Box through Stanford’s Globus subscription.

This collection previously gave you access to your Box files, as well as to folders which have been shared with you.

Globus does not copy any custom metadata or permissions from files it downloads. Uploaded items will inherit the permissions of the parent folder.

When uploading, Globus will respect the permissions set on the destination; if you do not have write permission, uploads will fail with a “Permission Denied” error.

Ceph

Although Stanford’s subscription does support the Ceph (object storage) connector, University IT does not provide Ceph storage as a service.

If you run your own Ceph service, you may be able to run your own Globus endpoint with the Ceph connector. Contact us if you are interested in this.

Dropbox

Although Stanford’s subscription does support the Dropbox connector, University IT does not provide Dropbox storage as a service.

Some groups within the University do use Dropbox. If you use Dropbox, you may be able to run your own Globus endpoint with the Dropbox connector. Contact us if you are interested in this.

Google Cloud Storage

Google Cloud Storage may be accessed through the Stanford Google Cloud collection.

When using Google Cloud Storage, Globus will act on your behalf. You will be able to access any buckets outside of the VPC Service Perimeter. In other words, you will not be able to access buckets associated with Nero projects, and other High Risk projects.

Google Drive

Google Drive may be accessed through the Stanford Google Drive collection.

Go here for detailed information on how to use Globus with Google Drive.

When using Globus with Google Drive, Globus will act on your behalf. You will be able to access files on your own Drive, on Shared Drives (formerly known as Team Drives), and to folders & items Shared With You.

Globus does not support setting or copying custom permissions. Uploaded items will inherit the permissions of the parent folder.

Globus is also not able to copy Drive Shortcuts. Drive Shortcuts are similar to shortcuts on Windows or aliases on macOS. At this time, Globus is not able to copy or follow Shortcuts; attempting to copy a Shortcut will create an empty file at the destination, and generate a checksum (integrity-checking) error. Support for Shortcuts is currently on Globus’ backlog for implementation.

Finally, Globus is not able to copy ‘files’ which are created by Google products, such as Docs, Sheets, Slides, etc.. Google does not actually store data for Google products in Drive; instead, Drive holds a ‘pointer’ to the specific Google product. Attempting to copy these ‘files’ will generate an error.

iRODS

Although Stanford’s subscription does support the iRODS connector, University IT does not provide iRODS storage as a service.

If you run your own iRODS installation, you may be able to run your own Globus endpoint with the iRODS connector. Contact us if you are interested in this.

Microsoft Azure Blob Storage

Stanford’s subscription supports the Microsoft Azure Blob Storage connector.

Microsoft Azure Blob Storage has unique properties that make it different to Amazon S3 and Google Cloud Storage. For that reason, before we provide an Azure Blob Storage Globus service, we would like to hear from existing Azure Blob Storage users.

If you use Microsoft Azure Blob Storage within the University, and you are interested in accessing it through Globus, please reach out!

Microsoft OneDrive

Microsoft OneDrive may be accessed through the Stanford OneDrive collection.

Go here for detailed information on how to use Globus with Microsoft OneDrive.

When using Globus with Microsoft OneDrive, Globus will act on your behalf. You will be able to access files on your own OneDrive, files shared with you by others in Stanford University, and files in Stanford University SharePoint sites.

Globus does not support setting or copying custom permissions. Uploaded items will inherit the permissions of the parent folder.

Globus is not able to access non-University content. That includes things shared by folks outside of Stanford University, and it also includes things shared by members of Stanford Health Care and Stanford Children’s Health.

Wasabi

Access to Wasabi is possible throguh region-specific collections:

• us-west-1: SRCC Oak Wasabi S3

You must use the collection specific to your Wasabi region. Wasabi accounts created through Stanford University IT are placed into the us-west-1 region by default. If you need access to a different region, let us know.