How to submit a job

Thank you for choosing Nephele. Here you will find instructions on how to submit a successful job. If you would like to try submitting with a small dataset, below are the sample fastq files and a mapping file.

Note: These data are from a 2017 BioProject in NCBI entitled "Experimental Microbial Dysbiosis Does Not Promote Disease Progression in SIV-Infected Macaques" (https://www.ncbi.nlm.nih.gov/bioproject/PRJNA417022)

File (Type) Size Description
Sequences 72MB Contains paired-end data (forward and reverse) from 10 samples sequenced on an Illumina MiSeq
Mapping File (Excel) 9KB Metadata file used in Nephele job submissions that describes samples, groups, etc. for analysis

You can follow along with this user guide using the sample data.

Step 1: Select analysis type.

Please select the analysis you are interested in. Nephele provides amplicon analysis including 16S, ITS and WGS metagenomics analysis.
Select 16S if you are using the sample data.

Analysis type select panel with Amplicon and WGS type selection buttons

Step 2: Select data type (demultiplexed file only)

Nephele supports the following data types for analysis types.

  • 16S
    • demultiplexed paired end fastq (select this option if you are using the sample data)
    • demultiplexed single end fastq
  • ITS
    • As ITS supports only demultiplexed paired end fastq, the system will assume the data type and proceed to next page
  • WGS
    • demultiplexed paired end fastq
    • demultiplexed single end fastq
Select the type appropriate to your data (e.g. Paired End FASTQ) and click “Next”.

Step 3: Select upload method.

Nephele provides two methods to upload input files: local and FTP. Local file upload allows you to upload files from your local computer. This method is suitable for less than 300MB per fastq or fastq.gz file or 10GB size in total. We recommend that you upload compressed .gz files for performance. Select this option if you are using the sample data. If you have a large data set, we recommend using the FTP method.

upload options panel with from local and FTP buttons
  • Local uploads
    1. Click "+ Add files" and select fastq or fastq.gz files or Drag and drop the files.
    2. Click Start upload to start upload your input files.
      *Dependent on your network, the upload speed and total time can vary. We recommend stable and high-speed internet for this method.
    3. After completing the uploads, click Next.
    4. You can cancel the upload anytime by clicking the "Cancel upload" button.
    5. You can select all files by clicking "Select all."
    6. If you wish to delete files, you can delete them by clicking the "Delete selected" button.
    local upload panel with file uploads in progress
  • FTP (NIH Helix) (recommended for large data)
    • If you don’t have an FTP server that is readily accessible to you, please use NIH Felix FTP (ftp://helix.nih.gov/pub).
      1. Upload your files to FTP, for example NIH helix. If you are not familiar with using FTP, please see this guide.
      2. Copy the ftp URL—for example, ftp://helix.nih.gov/pub/test. The URL should point to a location of a folder that contains fastq or fastq.gz files.
        Note: please make the folder with only files without nested folders.
      3. Select Upload via FTP from the upload options page
      4. Paste the URL in the text box.
        Note: please enter the URL as exactly it is. If there is a space in the folder name, you can put a space in the path.
        local upload panel with file uploads in progress
      5. Click Upload.
    • In general, Nephele FTP method can retrieve data from publicly available URL that does not require a username or password. (Nephele FTP method does not support Google Drive or Box currently. Read more in the FAQ).

Step 4: Upload mapping files

After uploading your input files, Nephele pipelines require a mapping file. To upload your mapping file, simply click Browse, select your mapping file, and click Upload. Upload the sample mapping file (xx.xlsx).

Nephele validates your mapping file instantly so that you can correct any error interactively. In order to prevent any mapping file errors, we recommend that you start with mapping file templates provided on the map file upload page for the pipeline you're running.

Step 4.1: How to use Nephele interactive mapping file validation

If you have an error in your mapping file, for example, a typo in your fastq file name column, here is an example of how to correct it using the interactive error page.

  • On the validation error page, move your mouse over to the area that is highlighted in red.
  • This example shows that the file names in both ForwardFastqFile and the ReverseFastqFile column names are the same: "3_S3_L001_R1_001s.fastq". There is a typo on the ReverseFastqFile column.
    map file error page with two columns in first row highlighted and error displayed
  • Click on the ReverseFastqFile column and correct "3_S3_L001_R1_001s.fastq" to "3_S3_L001_R2_001s.fastq."
    map file error page with two columns in first row highlighted, editing second column
  • Click "Save & Retry."
  • The mapping file passes the validation and moves to the next page, Select Pipelines, automatically.

Notes on map file validation:

  1. "BarcodeSequence" and "LinkerPrimerSequence" columns are no longer required by Nephele pipelines. Please remove them if you are using older mapping files.
  2. The file names listed in the mapping file should exactly match the names of the files you have uploaded for analysis. If you have uploaded *.fastq.gz files, make sure to add the .gz extension to the file names in the mapping file. Do not include full file paths.
  3. Nephele 2 accepts tab-delimited text (.txt) files and Excel (.xlsx or .xls) files. Comma-separated (.csv) files are not supported.

Step 5: Select a pipeline

After uploading mapping file and pass the validation, you can select a pipeline. Simply click "Select" on the pipeline you wish to use. Select QIIME if you use the sample data.
You can learn more about pipelines in Nephele in the Nephele Pipelines section.

image of the pipeline selection page

Step 6: Update parameters (optional)

Once you select a pipeline, you will see a submission page. You can enter a description of your job so that you can identify different jobs easily.

image of Job Details tab of pipeline options page

On this page, there are different tabs such as "Pre-processing" and "Analysis" (dependent on the pipeline) which allow you to change the parameters the pipeline is run with. Don’t forget to have a look if you are interested in different options. See Nephele's Pipeline Section for more details regarding parameters.

image of Job Details tab of pipeline options page

Finally, click Validate and Submit!

Congratulations! You have submitted a job! You will receive a Pipeline Started email shortly.

How to resubmit your job

You can resubmit a job without uploading all of your input files again.

Step 1: Enter the Nephele JobID that you would like to resubmit.

image of Home page with job resubmission box highlighted

Step 2: As long as your job hasn't expired, your input data will be retrieved. The rest of steps are the same as submitting a job. You can re-upload a mapping file and change parameters for the resubmission.