New BaseSpaceCLI Tools

We are pleased to announce the third release (version 0.8) of BaseSpaceCLI, which includes a major new tool and a number of minor features. To install the new release, run this script:

$ sudo bash -c "$(curl -L https://bintray.com/artifact/download/basespace/helper/install.sh)"

BaseSpace Copy

With this release of BaseSpaceCLI we are introducing a new high-performance copy tool called BaseSpace Copy or bscp, which provides the following features:

  • Rich and flexible URI mechanism to specify source and target for copies
  • Multi-threaded for performance and to cope with high-latency connections
  • Excellent data integrity, all part data is hashed and verified. Ability to export md5sum compatible sum file for future data verification.
  • Resumable downloading

Examples

Example Download (using default configuration):

$ bs authenticate # only needed first time
$ bs cp //./Projects/MyProject/AppResults/RunData ./RunData

Example Upload (using custom configuration):

$ bs authenticate --config custom
$ bs cp ./RunData //~custom/Projects/MyProject/AppResults/NewAppResult

For more information see the help documentation.

Should I use BaseMount or bscp?

BaseMount can also be used to copy files to and from BaseSpace. This table helps you choose which tool is most suitable for your application. A more detailed table is available in the help page.

Application
Recommended Tool
Discussion
Browsing and interacting directly with files without having to download them locally BaseMount
Interactive selective copying BaseMount The interactive nature of BaseMount makes it easier to select files to copy.
Programmatic selective copying bscp Bscp streams file metadata during copy and does not require the entire file tree to be loaded before copy starts.

Bscp requires no additional memory for complex filters.

Scripted or programmatic copying bscp Using bscp in scripts avoids having to use a mount step.

Bscp has better error reporting as it has end-to-end control of data transfer.

Bscp will automatically close the AppSession when finished.

Downloading runfolders with thousands of small files bscp BaseMount relies on Unix tools like cp and rsync, which only copy one file at a time. By copying more than one file in parallel, bscp performs better on these examples.

 

Additional Improvements For This Release

Listing Runs

To complement the other listing functions in BaseSpaceCLI we have introduced the list runs command, which works just like the other entity listing tools.

WhoAmI

For users working with multiple BaseSpace configurations, it can be useful to see details of the access tokens associated with each. The whoami command provides this token reflection.

$ bs whoami -c hoth
+-----------------+-----------------------------------+
| key             | value
+-----------------+-----------------------------------+
| Name            | BaseSpaceCLI Illumina
| Id              | 975975
| Email           | BaseSpace.CLI@gmail.com
| ApplicationName | BaseSpaceCLI
| DateCreated     | 2015-11-12 10:57:52
| Scopes          | BROWSE GLOBAL
+-----------------+-----------------------------------+

Again, the yaml, csv and json output options are also supported.

History

For Enterprise tier users, BaseSpace now provides support for examining a user’s event history and BaseSpaceCLI allows access to this stream of events. This can be accessed using the bs history command. For more details, see the documentation.

Feedback

As with previous releases, report bugs or other feedback on the Google Groups board.

About Eric Smith

Eric Smith is a product owner for BaseSpace.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: