Frequently Asked Questions

Pipeline Submission Tips

  • Paired End: Paired-end sequencing involves sequencing DNA from both ends of a fragment. Choose this option if your sequence files are paired and have been demultiplexed. For example, Illumina MiSeq Paired End FASTQ files consist of two files for each sample with names ending on "R1_001.fastq" and "R2_001.fastq"
  • Single End: Single-end sequencing involves sequencing DNA from only one end. Choose this option if your samples were only sequenced from one end or if you don’t want to use the reverse FASTQ file.
  • If you need further information to help you decide what type of file you have, see the SRA file type information page.
Each sequence file should only contain data from a single sample. For paired-end data, you should have 2 files per sample, and for single-end, just one file per sample. Nephele only accepts demultiplexed data at this time.
The WGS pipeline uses whole genome shotgun data for taxonomic and functional characterization. It is not designed to work with amplicon data from a single region, but rather makes use of marker sequences from along the entire genome. Furthermore, the QC step of the bioBakery pipeline removes many 16S (and other amplicon) sequences, as these are often over-abundant in samples due to contamination. For more information, see the bioBakery kneadData QC and MetaPhlAn2 tutorial wiki pages.
If you have reads that are already joined, submit the dataset as single-end samples.

The optional parameters were carefully chosen based on (1) the most common scenarios of NGS data analysis, (2) the suggestions from the developers, and (3) published results. The different pipelines available on Nephele target different kinds of NGS studies, such as whole genome shotgun sequencing, 16S microbiome survey, and functional annotation of microbial community.

Most users submit their jobs with default values of the optional parameters. In our experience, more experienced bioinformaticians change the parameters to optimize their input data. We also have received feedback from novice microbiome researchers or students that they study the optional parameters (reading help text and testing different values-even if it fails) as a part of learning microbiome analysis.

Rather than uploading large sequence files from your computer (and potentially over long distances or slow/wireless networks), you can likely save time by first uploading your files to a publicly-accessible FTP server, such as NIH Helix FTP server, and then supplying the URL as your input. Nephele will verify that it can access the files, and then it will retrieve them after you click submit. Please see 'how to upload via ftp' in Step 3.
Yes, you can, but you will need to use your mobile device to do so. Below are the steps you can try. Here we use Google Drive on an iOS device as an example, but the steps are pretty much the same for Dropbox and/or Android.
  1. Make sure your files (fastq or fastq.gz) are already stored in your Google Drive account.
  2. Download the Google Drive app to your device.
  3. Once the download is completed, log into Google Drive with your account.
  4. Go to https://nephele.niaid.nih.gov in your browser.
  5. Tap on analysis type, for example, 16S, enter your email address and tap on Upload from Local.
  6. Tap the Add files button.
  7. In the menu that pops up, tap on Browse.
  8. You might see "iCloud Drive" as an option. But you should tap on Locations on the top left, and enable Google Drive/Drive under Locations (for Android, it may say Documents instead).
  9. Once you enable "Drive," you can navigate to your Google Drive account and select a file.
  10. Once you select a file, tap Start Upload. **Please note that you will need to upload one file at a time. Selecting all files from Drive and uploading them in a batch will not work.
  11. Once you upload all the files from Drive, you can submit your job to the pipeline!
Please let us know if this works for your device or other cloud-based storage or any other tips!
On average, it takes about 1.4 hours to process 1G of compressed data. Nephele uses multiple processor cores to speed up the analysis pipelines.

Studies show that quality filtering can greatly improve microbiome analysis results. Best practices on working with sequencing data include doing a series of QC steps to verify and even improve the quality of the data. Our Pre-processing QC pipeline was designed to run a quality check by default, so the user can run it without choosing any options and receive FastQC tables and graphs providing information on the quality of individual samples. After evaluating these results, the user can submit their files to an analysis pipeline or return to the QC pipeline to trim reads and merge read pairs as needed.

Even though our 16S and WGS pipelines include quality filtering, trimming and merging steps, it may be best to run those processing steps separately ahead of time. We have incorporated the tools cutadapt and Trimmomatic in our Pre-processing QC pipeline steps to give users more control for modifying parameters, which can be helpful for some datasets, especially if the amplicon region is variable length. For the read merging step, we have integrated the FLASH merger, which some results show might provide better precision and recall than the native tools used by QIIME 1 and mothur. For longer amplicon regions with a short overlap between paired reads, FLASH may perform better than the DADA2 merger. So, we designed the QC pipeline to provide these programs for our users as well to help them get better results. For, more information about the tools we use see the details page.

Some usage examples:

  • Run paired-end data through this pipeline, choosing to merge the reads, and then submit the resulting FASTQ files to the DADA2 or QIIME1 Single End pipelines.
  • Examine the average per-base quality scores from the FastQC results of the pipeline, and use that information to set the Truncation length parameter in DADA2 or the Minimum Phred quality score parameter in QIIME 1.

Troubleshooting Nephele Errors

We recommend that you start by examining the logfile.txt file, which can be found directly on the results download page as well as in the PipelineResults.JOBID.tar.gz directories. Specifically, you can do a text search for ERROR to see some common errors that can arise with data analyses on Nephele. Many of these errors are described further in additional FAQs here, which provide detailed suggestions or solutions. If you continue to have issues, please do not hesitate to send us a support request.
This is most likely due to corrupted sequence data files. File corruption is not uncommon on slow or unreliable network connections. If the file is gzipped and cannot be uncompressed successfully, Nephele will report an error of missing files. If this happens to your submission, there are possible remedies: (1) ensure that the file is not already corrupted on your computer, (2) try to find a faster, reliable connection, and (3) use software with file integrity check option such as FileZilla with ftp upload.
This is most likely the result of having a bad or corrupt .gz files. You'll need to recreate your .gz files before resubmitting your pipeline. The .gz files should only contain a single sequence data file, and NO FOLDERS. MacOS users should create your .gz file using the command line, (Terminal.app instructions).

Problems with Output Files

The QIIME core diversity analysis and the bioBakery wmgx_vis workflow each require a minimum of 3 samples. If you submitted a mapping file with more than three samples, check the contents of logfile.txt. It is possible that one or more of your samples did not have the minimum number of OTUs or reads and was excluded from further analysis. This will be indicated in the logfile.txt output.
Missing samples are typically the result of poor or low OTU or sequence variant counts for those samples. To identify which samples have been excluded from your final analysis look at the samples_being_ignored.txt file. You may also look at the logfile.txt to see why those samples have been excluded. Samples that have low OTU or sequence variant counts are sometimes removed because of the Sampling depth cutoff parameter. If you do not specify the parameter, Nephele will use the default value of 10,000; see FAQ:How is the sampling depth calculated? for more information. If you open the otu_summary_table.txt file, you can see OTU counts for all of your samples. Adjusting the Sampling depth parameter accordingly (i.e., entering a value that will include all of your samples) in a new run with the same data will resolve this issue. The parameter can be set under the Analysis tab of the job submission page, and you can use the job resubmission feature of Nephele to more easily resubmit your data with a different value.

The DADA2 pipeline is highly sensitive to sequence quality and primer trimming. It is very important to specify the correct primer lengths at job submission (or remove the primers from the data before submitting), as these sequences may interfere with the denoising of the reads as well as with chimera removal (if you are in doubt of the primer lengths, we advise you not to choose the chimera removal option). See this DADA2 FAQ for more information.

The DADA2 pipeline produces quality profile plots that you can look at to gauge the quality of your data (qualityProfile_R1/2.pdf). If the data is poor quality, the reads may be filtered out during the filterAndTrim step. You can also see a table in the log file of how many reads pass this step. Additionally, if the data is poor quality, reads that pass the filter may be trimmed too much in the filterAndTrim step, and may not merge properly in the mergePairs step. You can search the log file for paired-reads for how many reads successfully merged for each sample. Sometimes, it is helpful to use a trimming program such as cutadapt, Trimmomatic, or BBDuk to trim for quality (and/or primers) prior to running DADA2. You can use Nephele's QC pipeline to do this pre-processing of your data; see here for more information.

The interactive heatmaps are produced using Morpheus by the Broad Institute. They work best in Google Chrome or Firefox. You may have problems with Microsoft Edge, Internet Explorer, or Safari.

Miscellaneous

Computers running Windows (7, 8, or 10) requires a third-party program like WinZip (commercial) or 7-Zip (open source). A file with a .tgz or .tar.gz extension is a Gnu Zip compressed Tar Archive file. They are made up of files that have been placed in a TAR archive and then compressed using Gnu Zip.

In articles and other publications, please cite Nephele as follows:

Office of Cyber Infrastructure and Computational Biology (OCICB), National Institute of Allergy and Infectious Diseases (NIAID). Nephele v2. http://nephele.niaid.nih.gov (2018)

Alternatively, use the following acknowledgement:

This study used the Nephele platform from the National Institute of Allergy and Infectious Diseases (NIAID) Office of Cyber Infrastructure and Computational Biology (OCICB) in Bethesda, MD.

Please also report the publication of articles which made use of Nephele by providing the reference in an email to the Nephele Project Team.

Please refer to the Release Notes to see when Nephele updates were made. Also, in the initial email you receive for each job, you will find the version of Nephele that corresponds to the Release Notes, as well as a copy of all the parameters that were selected for that job. Software package versions for the pipelines are also listed in the log files.

Choosing a sampling depth is generally arbitrary. Generally, it's recommended to choose a value high enough that you are able to capture the diversity present in samples with high read counts, but low enough to include the majority of your samples. For a simple community with only a handful of abundant members, for example, a sampling depth of 5,000 or less may suffice for an accurate estimate of diversity. For a more complex community with many low abundant members, however, a much higher value for sampling depth, 10,000 or higher, is generally necessary.

Nephele specifies the sampling depth of 10,000 reads as the minimum requirement for all downstream analysis. The pipelines use the following logics to determine the sampling depth:

  1. apply the user-specified sampling depth, if available
  2. set the sampling depth based on the sample with the least number of reads if it is greater than or equal to 10,000
  3. otherwise, no downstream analysis is performed.

Note: Users are encouraged to specify the sampling depth that is most appropriate for their studies. There is really no formula that can precisely determine the most appropriate value simply based on the distribution of read counts and the number of samples. If the pipeline does not generate any downstream analysis for you samples, it is most likely that the sample with the least number of reads is below 10,000. You will need to lower the sampling depth in order to run the downstream analysis.

Nephele is currently provided to the research community free of charge (this may change in the future).
Since the public launch in 2016, Georgetown University, North Carolina State University, and University of Florida have used Nephele in their microbiology classes. Nephele has mainly served to process FASTQ files, and then students have studied the result files by visualizing the processed data and making sense of what it means biologically. Besides the classroom, Ph.D. candidates or graduate students who do not have access to a high computing environment use Nephele for their research. If you are interested in using Nephele in your class, please let us know to see how we can help.
We also like learning about new and different pipelines that could better serve your research and educational needs! If you have a suggestion of a tool or analysis for Nephele, please fill out this form. We are interested in hearing more about the research needs of our users!