Archive by Author | Greg Roberts

MiSeq Analysis Reports Upgrade

We are happy to announce newly revamped reports for all MiSeq based analyses.  BaseSpace automatically runs an analysis package called MiSeq Reporter, similar to what is available on instrument, upon completion of MiSeq run uploads. With the advent of the rich reporting tools available in the Native App Engine, we felt these analysis reports needed an upgrade.  While the underlying analysis result data has not been altered, the visualizations and reports on our site have been upgraded to this new style for all MiSeq analyses in the past 2 years.

Here are some example reports and charts:

msr-report-1

msr-report-2

image

Even with these upgraded reports, we also encourage our MiSeq users to try out all of the latest apps in the app store!

Introducing the New BaseSpace HelpCenter

The team is proud to announce the release of our new help site at help.basespace.illumina.com.  The new HelpCenter makes it easier to navigate and find the relevant information about BaseSpace. It also enables greater user participation (see below for details).

image

image

Site Contents

  • Overview section for new users
  • Articles explaining the usage of different features
  • Release notes
  • Videos
  • Latest news
  • Search
  • Links to other BaseSpace related content like our Developer’s Site

Feedback

This is just the beginning of our content here and it will continue to be expanded and refined.  If you would like to suggest a topic to be covered or want us to clarify an existing article, all of the content is open source and hosted on github. You can create issues on current or new content and submit pull requests as well.

image

We still have direct access to support via our “Contact Us” widget within BaseSpace along with email and phone support through the normal Illumina support channels.

Thanks!

Native App Report Engine Improvements

We recently released improvements to our reporting engine for native apps that make it much easier to do some common tasks.  For those of you that are new to the “Native App Engine”, it’s comprised of three main developer components that allow a developer to package up any analytic workflow and publish it in the BaseSpace App store.  

Those three components are:

  1. Form Builder – Custom input form designer
  2. Docker – Linux container technology that your app runs in
  3. Report Builder – HTML 5 template engine

It’s the same toolset that we use internally to publish apps and we are constantly looking to make improvements based on internal and external feedback. 

Report Generation

report-editor-shot2

The reporting engine, which is essentially a html template engine,  is the last step in building a native app.  When your application finishes processing, we build a meta model around all the data that was produced and allow you to bind that to an html template.  We leverage an open source technology called Liquid which is a markup template engine used in lots of other companies with similar needs.  Along with some of the basic filters that are defined in liquid, we have extended the syntax to include BaseSpace specific needs.

New Features

Find Filter

XPath operator on XML files

  • Takes in an xpath expression and returns the resultant xml text

{{ statsFile | find: "/Statistics/Overall/Stats[SampleID='10002 - R1']/NumberOfClustersPF" }}

 

Stringify Improvements

Stringify is a custom filter that allows you to serialize contents of a drop, csv, or xml segment to JSON.

var globals = {}; 
globals.sample = {{ sample | stringify }};
globals.sample.chromosomes = {{ sample.chromosomes | stringify }};
globals.sample.statsByChromosome = {{ statsFile.parse.StatisticsResequencing.Samples.SampleStatistics | stringify }}; 

 

Custom Dictionary Filters

  • where.starts_with – returns the values where the dictionary key starts with provided string
  • where.ends_with – returns the values where the dictionary key starts with provided string
  • where.contains – returns the values where the dictionary key starts with provided string
  • first – returns the first value, errors if nothing there
  • first_or_default – returns the first value or default (usually null)
{% assign_object datafile1 = result.files.where.starts_with["datafile1"].first %}
{% assign_object anyXml = result.files.where.ends_with[".xml"].first_or_default %}
{% assign_object datafile3 = result.files.where.contains["file3"].first %}

 

Break and Continue Tags

Allows breaking and continuing in Liquid loops.

{% for file in files %}
	{% if file.href == null %}
		{% continue %}
	{% endif %}
	{% if file.href == 'http://special.com' %}
		{% assign specialHref = file.href %}
		{% break %}
	{% endif %}
{% endfor %}

 

Select Columns

Allows the selection of columns in a csv by column name or index. 

{% assign grid2 = result.files[key].select['0,1,2'].take[1].parse %}
{% assign grid3 = result.files[key].select['LastName','City','Phone'].take[5].parse %}

 

Take

Ability to take a subset of rows from a csv file.

{% assign grid2 = result.files[key].select['0,1,2'].take[1].parse %}
{% assign grid3 = result.files[key].select['LastName','City','Phone'].take[5].parse %}

 

ToArray

Ability to output csv data rows to a 2-dimensional data array.

{{ result.files[key].parse.to_array | stringify }}

 

Assign Improvements

Assign now allows assignment of any liquid object, not just primitives.

{% assign myFiles = result.files %}
{% assign myCsv = myFiles.where.ends_with[".csv"].first.select["0"].take[2].parse.to_array | stringify %}
{{ myCsv }}

 

Summary

We hope developers will leverage these new features to build great interactive reports.  If you want to learn more about native apps, then read our intro post, check out our developer portal, or follow our native app tutorial.

Taking Out the Trash

If you’ve been using BaseSpace for a while you may have noticed that there wasn’t a way to permanently remove data from your account.  I say that in the past tense because it is no longer true.    The wait is over! “Move to Trash” is now available on Runs and Analyses.

MoveToTrash02

Trash Overview

This has been one of the most important features for us to get right because it has to do with removing your data and we take that very seriously.  That is why we are introducing a two-step delete process that will help prevent accidental deletes and give you the confidence you need to safely manage your data.

First, you will notice a new action available on run and analysis list and detail pages, called “Move to Trash”. On the list pages, you must first highlight the row that you want before it’s available.

MoveToTrash

This action is very similar to moving files on your desktop to the trash or recycle bin. Just like your desktop, the data can be recovered, but it can no longer be viewed or acted upon.

Trashed Items Side-Effects:

  • If the items were shared, all share recipients will lose access to that data
  • All API access is immediately removed and will return the HTTP status code of 410 (“Gone”)
  • Any attempt to view this data on the website will take the user to an 410 error page stating the content is “Gone”
  • Data, while in the trash, can only be  “Restored” or “Emptied” by the owner.
  • Purging data will cause it to be permanently removed and cannot be undone.

Moving Runs to the Trash

  • Runs can be put in the trash from the list or the detail pages.
  • Runs cannot be removed if they are in a non-terminal state.  The most common non-terminal states would be: running, uploading, analyzing.
  • The dialog may also present you with the option to remove all associated analyses that used the run as input.
    • All sequencing runs will have at least 1 associated analysis unless they were failed or used just for remote monitoring.
  • If you are not the owner of the run, moving this item to the trash will simply remove your access and cannot be undone.
    •   To restore access, just contact the owner or click on the previously sent share link if it’s still active.

movetotrash-animation

Moving Analyses to the Trash

  • Analyses can be put in the trash from the list and detail pages.
  • Analyses cannot be removed if in a non-terminal state. The most common non-terminal states would be: pending execution and running.
  • If a project is being transferred, some of the analyses may not be removed until after the transfer has been completed.
  • Apps that are leveraging data as input may fail if items are moved to the trash.
  • If you have items in the trash, we prevent project transfers until all items in that project are restored or emptied.

Emptying and Restoring Items in the Trash

The trash page can be accessed from most of the project and run list pages.  The icon is always in the right side of the grid and labeled, “View Trash”.

TrashIcon

There are only two actions currently on the Trash page: Empty and Restore.

Empty will permanently delete all items, and Restore allows you to return the items back to being active.

Restored items will keep all of their original attributes except for the share recipients.

restore

User Agreement Updates

Because of all of these changes, we have also updated our User agreements to reflect the behavior of these new features. In particular, item 7 states that even though data can be removed it may have been previously shared with other users or apps and subsequently downloaded or copied.   You will be prompted to accept these new terms upon your next login.  If you have any questions, don’t hesitate to ask!

Thank you,

-Greg

MiSeq Reporter version 2.4 Now Available in BaseSpace

With the latest update of BaseSpace, we have updated the MiSeq Reporter (MSR) Workflows to v2.4! This means all users streaming MiSeq data to BaseSpace with automatic analysis launch will now have the latest and greatest features incorporated since the release of MSR v2.2 in BaseSpace. MSR v2.4 in BaseSpace will also provide consistent results with MSR local v2.4, released in March, 2014.

For full details on the MSR v2.4 release in BaseSpace, please see the Customer Release Notes.

New platform level features

  • gVCF file output for TruSeq Amplicon, Enrichment, and PCR Amplicon workflows. The gVCF files provide data at all genomic coordinates assayed, regardless of the variant status, and show the reference allele if no variant is found. For more information on gVCF files, please see the following link: https://sites.google.com/site/gvcftools/home/about-gvcf
  • BAM and VCF/gVCF files are now stamped with a header containing algorithm versions and parameter settings to easily track your analysis details. msr vcf stamp
  • Updated the Starling variant caller to v2.0.3 for use with TruSeq Amplicon, Resequencing, and PCR Amplicon workflows. The default variant caller for these workflows remains GATK v1.6.
  • Enabled StitchedReads in the TruSeq Amplicon and GenerateFASTQ workflows. Read stitching allows construction of a single, longer read from two overlapping, shorter paired reads. For more information on Stitched Reads, please see the Customer Release Notes.

Highlights for specific workflows

  • The 16S Metagenomics workflow now allows species-level identification (previous versions stopped at genus-level). To classify down the species level, add the sample sheet setting described in the Customer Release Notes, or use Illumina Experiment Manager v1.7 to build your sample sheet. We also added a new, interactive HTML output report to dive deep into your 16S data. Lastly, the classification algorithm has been updated to provide faster results.
    16s Metagenomics
  • The Enrichment workflow now allows users to specify padding values in the sample sheet. Please see the Customer Release Notes for additional details.

If you have any older runs that you want to push through these new workflows, please contact support through the “Contact Us” button on the site.

-Greg

Introducing BaseSpace RNA-Seq Core Apps!

The BaseSpace team is excited to announce the release of our most anticipated applications since the announcement of the Native App Engine last October.

app-icons

  • The TopHat Alignment App can be used to align RNA reads as well as detect gene fusions using the industry-standard method. Illumina’s Isaac method further enables the calling of SNVs and small indels.
  • The Cufflinks Assembly & Differential Expression App enables gene expression profiling and detection of novel transcript isoforms.

 

RNA-Seq-03

These applications are part of the BaseSpace Core Apps and were both written with our Native App Engine, taking advantage of the parallelization features which allows TopHat to process 96 samples in less than a couple hours with fusion calling enabled.  This parallelization is seamless and automatic, bringing the scalability of the cloud to your analysis – so that if you have 1 sample or 100, they run in the same amount of time.

In the coming days, we will be posting a sample project with fully analyzed data so that you can quickly view the output from the RNA-Seq Core Apps.  There is also example data comparing Universal Human Reference and Human Brain mRNA that you can import into your BaseSpace account from BaseSpace’s Public Data repository:

dataCentral

For more information about these apps, check out the detailed announcement here.  If you’d like to try out the new RNA-Seq Apps in BaseSpace, head over to BaseSpace Apps.

Security Update Concerning OpenSSL Heartbleed

We take the security of your data very seriously at Illumina.  As one of many safeguarding measures, BaseSpace uses encryption software to ensure the privacy and security of both your user login information and your sequencing data as it travels from Illumina’s sequencing instruments into our cloud environment.

Recently, a security threat with a common encryption library that is widely used by many websites (including BaseSpace) was found to have a major vulnerability in it.  The security threat was published on April 7th, 2014 for the “OpenSSL” library (CVE-2014-0160 ) and quickly came to our attention.  The vulnerability, nicknamed “Heartbleed”, had the potential to allow an attacker to steal private keys or other sensitive information that is normally encrypted via SSL.  The nickname came about because of the way an attacker could gain access to a server’s memory through the “OpenSSL” specific heartbeat protocol.

Once we assessed the situation we quickly worked with our platform vendor and internal teams to ensure the following was completed:

  • All affected servers were patched with the updated/non-affected version of OpenSSL.  This was completed less than 24 hours after the announcement.
  • All of our public facing SSL certificates were replaced.  All of the old SSL certificates are now being revoked.
  • All login sessions older than 24 hours were automatically invalidated.

We have no information that knowledge of this vulnerability was used against BaseSpace or its users.  However, as with all websites that have updated their OpenSSL libraries, we encourage everyone to change their BaseSpace password.  You should also check your notification history for any suspicious activity.

At BaseSpace, protecting your data is our top priority, and we continue to assess the risks in response to this issue.  We have and will continue to work together with our infrastructure provider, Amazon Web Services, to ensure that all of our services respond quickly to security threats.

For further information about BaseSpace data security, please see our Data Security Technical Note.  If you have any questions or concerns, please feel free to contact support.

Technical Support
Monday – Friday
7:00am – 5:00pm (PT)
858.202.4766 (Fax)
TechSupport@illumina.com
customerservice@illumina.com