BaseMount: A Linux command line interface for BaseSpace
BaseSpace was designed with a graphical, web-based interface to enable push-button analytics, and sharing of NGS data. We also realize that there are some use cases, especially while working with bulk data, where a Linux-based command line interface (CLI) is more convenient. We are therefore happy to announce the early access release of BaseMount, a tool that can be used to mount your BaseSpace data as a Linux file system. With BaseMount you can navigate through projects, samples, runs, and app results, and interact directly with the associated files exactly as you would with any other local file system. BaseMount is a FUSE driver, which operates in user-space and uses the BaseSpace API to generate the file system structure for both the metadata and actual file-based data.
To install, please go to the BaseMount page: basemount.basespace.illumina.com and run the installation script. We have several tutorial videos as well as a Help Page to get you started. The Help page also has information on hardware and operation system requirements.
Once you install BaseMount, and authenticate your BaseSpace credentials, you are ready to get started.
Once you navigate to the root folder of your BaseSpace mount, you will see the structure of the data within your account. Within this folder, you will see all of the Runs and Projects that you have access to in BaseSpace. Each of those directories will contain metadata information from the API about its contents as well as the folders and files associated with that Run or Project.
Finally, you will navigate to places where the actual files are listed. This is where the magic happens! Nothing is actually downloaded until you start to access each file. Portions of the file are pulled on demand and cached, so you can mount your whole account containing potentially terabytes of data, but only have a couple gigabytes of local storage. There are potentially many things you can now do with access to all of your data.
For example, you can simply use BaseMount to download data to a local filesystem. Just use cp, rsync or any command line tool you prefer to copy the files from your BaseMount space to your chosen destination. Although BaseMount does facilitate file download, we would recommend that since BaseMount allows convenient, fast, cached access to your BaseSpace metadata and files, you may find that many operations can be carried out without the need to download locally. During our testing, we have used BaseMount to grep through fastq files, extract blocks of reads from bam files and even use IGV on the bam files directly all without downloading files locally. This can be more convenient than including a download step and saves on the overheads of local storage.
Please note that this is an early access version of BaseMount, and does have some limitations. We encourage you to test the feature and provide any feedback you may have on our developer google group.