Skip to main content

Globus and Elm

Elm may be accessed through the Stanford Elm Storage (project: campus) collection.

Before continuing, you may need to do some one-time setup work. You will need to…

  • Review the service limitations, to see if they will affect your usage. If you are still OK using Globus, your next step will be to…

  • Create an Access Key, which will give you an access key (officially an “Access Key ID”) and a secret key (officially a “Secret Access Key”). Once your access key is created, you will need to…

  • Load the credentials into Globus. This gives Globus access to Elm, with the credentials you provide.

Once the one-time setup work is complete, you should proceed to access your files on Elm.

Each step is described in detail below.

Service Limitations

Globus has a number of limitations when working with Elm. These limits might affect your use case—especially when dealing with research data or shared content—so you should review them before starting to use Globus with OneDrive.

Elm Limitations

Even though Elm is accessed through the S3 protocol, it has a number of limitations compared to Amazon S3. In particular…

  • Big Data is Best: Elm works best with large files (hundreds of megabytes and larger). If you are thinking of uploading smaller files, you may wish to combine them into a larger .tar or .zip file before you copy them to Elm.

  • Expect Slow Restores: Data on Elm are eventually sent to tape. Directory listings will be fast, but downloads from Elm will be slower than expected, and you may see timeouts.

  • Workgroups Rule: Even if you give Globus full read, write, and delete access, your workgroup membership will limit what Globus can do. Specifically…

    • You must be a member of your bucket’s main workgroup to have any access.

    • You must be a member of your bucket’s “Writers” workgroup to have read/write access.

    • You must be a member of your bucket’s “Editors” workgroup to have full access.

  • No Cross-Account Access: Elm does not support S3 cross-account bucket access. Instead, you should add users to the appropriate work, so they can create their own access keys.

Globus Limitations

The Globus File Manager only supports one access key per Mapped Collection, but it is possible to have a separate access key for each bucket. If you are someone who has multiple buckets on Elm, contact us for assistance.

Globus for Elm does not support the following S3 features:

  • Custom Metadata / Tags: Custom Metadata and tags on existing objects are ignored when those objects are downloaded, and new objects do not have custom metadata or tags set.

  • Versions: When downloading a file from an S3 bucket, Globus will always access the latest version.

  • ACLs: ACLs on existing objects (and the bucket) will influence what you can download through Globus, but those ACLs are not copied out of S3. When uploading new objects, ACLs are not explicitly set, and so inherit any bucket-level ACL that is set.

  • Additional Checksum Algorithms: At this time, if an object in Elm needs to be verified, Globus will re-download it in order to compute the checksum.

These limitations are present because Globus supports only a common set of features between storage platforms, to make file transfers as portable as possible.

If you are OK with the limitations above, you should move on to creating an Access Key, which Globus will need to interact with Elm.

Creating an Access Key

Once your Elm bucket is created, and you have been given access to the appropriate workgroup, you should log in to Elm. After going through Stanford Login, you will see your buckets.

The Elm main page, showing your buckets.

Select the “Access Keys” option:

The Elm main page, with the 'Access Keys' link hilighted.

You will see the list of access keys. Click “Create access key”:

The Elm 'Access Keys' page, with 'Create Access Key' hilighted.

You’ll now be presented with the screen to configure the new access key.

The Create Access Key page.

To start, in the Name field, enter your name and the word “Globus”, so that others will know what you are using the Access Key for:

The Create Access Key page, with a name filled in.

Next, turn on the “Restrict beyond user policy” setting:

The Create Access Key page, with a name filled in.

You will now be presented with the current User Policy, allowing you to change it.

The Create Access Key page, with a name filled in.

Setting a User Policy

The default User Policy allows more access than Globus needs. So, you should change it to a policy that restricts Globus to just the buckets, and actions, that you want to allow.

Below, fill in the names of the buckets you want to access through Globus, and choose if you want to allow Globus to upload and/or delete files.

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Action": [
    "s3:ListAllMyBuckets",
    "s3:GetBucketLocation"
   ],
   "Resource": [
    "arn:aws:s3:::*"
   ]
  },
  {
   "Effect": "Allow",
   "Action": [
    "s3:ListBucket",
    "s3:ListBucketMultipartUploads"
   ],
   "Resource": [
    "arn:aws:s3:::allcats" <------ ONE LINE PER BUCKET
   ]
  },
  {
   "Effect": "Allow",
   "Action": [
    "s3:DeleteObject",               <------- If allowing deletions
    "s3:PutObject",                  <------\
    "s3:ListMultipartUploadParts",   <------|- If allowing uploads
    "s3:AbortMultipartUpload",       <------/
    "s3:GetObject"
   ],
   "Resource": [
    "arn:aws:s3:::allcats/*" <------ ONE LINE PER BUCKET
   ]
  }
 ]
}

Replace the default Current User Policy with the custom policy above, and click “Create”:

The Create Access Key page, with a name and policy filled in.

Once created, you will be given your new Access Key and Secret Key. Make a note of these, to load into Globus.

The Create Access Key page, showing the newly-created Access Key and Secret Key.

Loading Credentials into Globus

With an Access Key configured, you can now upload your credentials to Globus.

Using the link at the top of the page, access the Elm collection. You might be asked to log in; if so, log in through Stanford University.

The Elm collection, with the Credentials tab hilighted.

You will be taken to the main page for the collection. Click on the ‘Credentials’ tab.

A prompt to give Globus consent to manage your Elm credentials.

If this is the first time you accessed this collection, you will be asked to give consent for Globus to store your Elm credentials. Click ‘Continue’.

A prompt to select an identity for giving consent.

Some institutions allow you to have multiple accounts. Stanford only allows one SUNetID per person, so click on your SUNetID.

A prompt to give consent to the Globus Web app to manage your IAM User credentials in the S3 collection.

Finally, click on ‘Allow’ to give Globus permission to store your Elm credentials.

A form to enter your IAM User's Access Key ID and Secret Access Key.

Once consent is granted, you will be asked to enter your Elm credentials. Enter the Access Key and Secret Key from when you created your access key.

The credentials page after an Access Key has been entered, showing the Access Key ID and associated Globus identity.

If you go to the Credentials tab after entering an Access Key, you will see the Access Key ID, along with an option to replace the credential (the gear icon) or delete the credential (the trashcan icon).

You should now proceed to accessing the collection!

Accessing Files on Elm

With Elm Access Key loaded and permissions granted, you may now proceed to access your data on Elm through Globus!

Using the link at the top of the page, access the Elm collection. You might be asked to log in; if so, log in through Stanford University.

The Elm collection, with the Overview tab and File Manager buttons hilighted.

Click on the “Open in File Manager” button. That will take you to the File Manager and connect to Elm.

First-Time Access

The first time you access the Elm collection, you will be asked for consent.

The Elm collection, saying that consent is required.

When you first loaded your credentials, you gave Globus consent to store those credentials for you. Now, you are giving Globus consent to actually use those credentials to talk to Elm. Click the “Continue” button.

A prompt to select an identity for giving consent.

Some institutions allow you to have multiple accounts. Stanford only allows one SUNetID per person, so click on your SUNetID.

A prompt to give consent to the Globus Web app to access Elm on your behalf.

Finally, click on ‘Allow’ to give Globus permission to use your AWS IAM User credentials to access Elm.

Subsequent Accesses

When you access the S3 collection—assuming you have previously provided consent—you should be greeted with a list of the buckets from Elm.

The File Manager showing the root of the Elm collection, listing buckets.

To access one of the buckets—assuming your Access Key has permissions—double-click on the bucket’s name.

The File Manager showing the root of an S3 bucket.

If your Access Key does not have permissions, attempting to list the contents of the bucket will give an error.

The File Manager, showing an erorr when trying to access a bucket in the Elm collection.

Once you have access to a bucket, you can transfer files in and out like any other Globus collection.