Locked History Actions

FTPUpload

File Upload via FTP

Please note the updated server to use for FTP: usegalaxy.org. This replaces the prior server "main.g2.bx.psu" in all materials below.

With the continued advancement of sequencing technology, we've seen the size of files uploaded to Galaxy grow quite large. Although our current file upload methods have worked fine for years, they are not well suited for the extremely large files in common use today. Uploading directly from the browser can be unreliable and browsers don't provide feedback on upload progress and state like they do for downloads.

Because of this, we have implemented file uploads to Galaxy via FTP on both Galaxy Test and Galaxy Main. FTP will allow you to monitor as mentioned above, as well as resume interrupted transfers. Compression types .gz/.gzip, .bz/.bzip, .bz2/.bzip2, and single-file .zip are supported.

Video demonstrating how it works: Get Data: Upload File
also in archive screencast FTP Upload

If viewing the "Upload File" tool, you'll notice a new field:

FTP on the Upload File tool form

To get started using FTP, you'll need to have registered a regular Galaxy account.

Once registered, you can initiate an FTP connection in your preferred FTP client (in this example I'm using Cyberduck for Mac OS X) to the same hostname provided on the "Upload File" tool form, (http://usegalaxy.org, for Galaxy Main, likewise for Test) using your registered email address and password for the login details:

FTP client connection details

As usual with FTP, you can view upload progress and completion time estimates:

FTP upload progress

Files uploaded to the FTP server won't automatically be imported to Galaxy - rather, you will be presented with a list of the contents of your FTP directory on the standard "Upload File" tool interface:

FTP files on the Upload File tool form

Files not imported within 3 days will be cleaned up from the FTP site.

Please note that it may not always be practical to use the public Galaxy servers. If you're routinely working with very large data and having to wait for it to upload, a local Galaxy server could be a more practical solution. Instructions on installing your own server can be found at Admin/Config/Performance/ProductionServer.

In addition, since it's possible to upload to Amazon's Simple Storage Service (S3) in parallel, using Galaxy CloudMan may be a faster alternative. We are investigating incorporating easy access to S3 buckets for Galaxy instances on the Amazon Elastic Compute Cloud (EC2). But you don't need to wait for the pretty interface, you can already access contents of S3 buckets by pasting links to their contents in the "URL/Text:" field of the "Upload File" tool. For an example of how to do this, see the screencast on this page entitled "Watch how the complete analysis can be performed on the Amazon Cloud."

FTP upload can be enabled in local installations of Galaxy, instructions to do so can be found at Admin/Config/Upload via FTP.