ProvisionFiles (SeqWare 1.1.2 API)

java.lang.Object
- net.sourceforge.seqware.pipeline.module.Module
- - net.sourceforge.seqware.pipeline.modules.utilities.ProvisionFiles

All Implemented Interfaces:

ModuleInterface
```
public class ProvisionFiles
extends Module
```
Purpose: This module takes one or more inputs (S3 URL, HTTP/HTTPS URL, or local file path) and copies the file to the specified output (S3 bucket URL, HTTP/HTTPS URL, or local directory path). For S3 this bundle supports large, multipart file upload which is needed for files >2G. FIXME: needs to return errors on failures/exceptions

Version:

$Id: $Id

Author:

boconnor

Field Summary

Fields
Modifier and Type	Field and Description
`protected String`	`algorithmName`
`protected joptsimple.OptionSet`	`options`
`protected int`	`s3ConnectionTimeout`
`protected int`	`s3MaxConnections`
`protected int`	`s3MaxErrorRetry`
`protected int`	`s3SocketTimeout`

Fields inherited from class net.sourceforge.seqware.pipeline.module.Module
processingAccession, VERSION

Constructor Summary

Constructors
Constructor and Description

ProvisionFiles()

Constructors
Constructor and Description
`ProvisionFiles()`

Method Summary

Methods
Modifier and Type	Method and Description
`ReturnValue`	`do_run()` do_run.
`ReturnValue`	`do_test()` do_test.
`ReturnValue`	`do_verify_input()` do_verify_input.
`ReturnValue`	`do_verify_output()` do_verify_output.
`ReturnValue`	`do_verify_parameters()` do_verify_parameters.
`String`	`get_syntax()` get_syntax.
`protected Cipher`	`getDecryptCipher()` getDecryptCipher.
`protected Cipher`	`getEncryptCipher()` getEncryptCipher.
`protected joptsimple.OptionParser`	`getOptionParser()` getOptionParser.
`protected boolean`	`provisionFile(String input, String output, boolean skipIfMissing, ArrayList<FileMetadata> fileArray, boolean fullOutputPath, FileMetadata metadata)` provisionFile.
`protected boolean`	`putDestination(BufferedInputStream reader, String output, int bufLen, String input, ArrayList<FileMetadata> fileArray, boolean fullOutputPath, FileMetadata metadata)` HTTP writeback currently not supported S3 uses multi-part upload which should deal with failed uploads (maybe) I'm not really dealing with failed upload recovery here...

Methods inherited from class net.sourceforge.seqware.pipeline.module.Module
clean_up, get_galaxy_xml, getAlgorithm, getMetadata, getParameters, getProcessingAccession, getStderrFile, getStdoutFile, init, setAlgorithm, setMetadata, setParameters, setProcessingAccession, setStderrFile, setStdoutFile

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - options
```
protected joptsimple.OptionSet options
```
  - algorithmName
```
protected String algorithmName
```
  - s3ConnectionTimeout
```
protected int s3ConnectionTimeout
```
  - s3MaxConnections
```
protected int s3MaxConnections
```
  - s3MaxErrorRetry
```
protected int s3MaxErrorRetry
```
  - s3SocketTimeout
```
protected int s3SocketTimeout
```
- Constructor Detail
  - ProvisionFiles
```
public ProvisionFiles()
```
- Method Detail
  - getOptionParser
```
protected joptsimple.OptionParser getOptionParser()
```
    getOptionParser.
    
    Overrides:
    
    getOptionParser in class Module
    
    Returns:
    a OptionParser object.
  - get_syntax
```
public String get_syntax()
```
    get_syntax.
    
    Specified by:
    
    get_syntax in interface ModuleInterface
    
    Overrides:
    
    get_syntax in class Module
    
    Returns:
    a String object.
  - do_test
```
public ReturnValue do_test()
```
    do_test.
    Things to check: * FIXME
    
    Specified by:
    
    do_test in interface ModuleInterface
    
    Specified by:
    
    do_test in class Module
    
    Returns:
  - do_verify_parameters
```
public ReturnValue do_verify_parameters()
```
    do_verify_parameters.
    
    Specified by:
    
    do_verify_parameters in interface ModuleInterface
    
    Specified by:
    
    do_verify_parameters in class Module
    
    Returns:
  - do_verify_input
```
public ReturnValue do_verify_input()
```
    do_verify_input.
    
    Specified by:
    
    do_verify_input in interface ModuleInterface
    
    Specified by:
    
    do_verify_input in class Module
    
    Returns:
  - do_run
```
public ReturnValue do_run()
```
    do_run.
    
    Specified by:
    
    do_run in interface ModuleInterface
    
    Specified by:
    
    do_run in class Module
    
    Returns:
  - provisionFile
```
protected boolean provisionFile(String input,
                    String output,
                    boolean skipIfMissing,
                    ArrayList<FileMetadata> fileArray,
                    boolean fullOutputPath,
                    FileMetadata metadata)
```
    provisionFile.
    
    Parameters:
    input - original path to the file, a file path if local, otherwise a http path or s3 path
    output - path to the destination, a file path if local, otherwise a http path or s3 path
    skipIfMissing - skip missing files with only a warning
    fileArray - metadata about the file (files) to be copied
    fullOutputPath - the full output path overrides any concatenation of variables
    metadata - the specific filemetadata for this file, if there is clearly one
    
    Returns:
    a boolean.
  - putDestination
```
protected boolean putDestination(BufferedInputStream reader,
                     String output,
                     int bufLen,
                     String input,
                     ArrayList<FileMetadata> fileArray,
                     boolean fullOutputPath,
                     FileMetadata metadata)
```
    HTTP writeback currently not supported S3 uses multi-part upload which should deal with failed uploads (maybe) I'm not really dealing with failed upload recovery here... Keep in mind only the writeout to local file will attempt to recover from failed reader
    
    Parameters:
    output - where the file should be copied to
    reader - a reader open to the file that needs to be copied
    bufLen - passed right through to filesUtil
    input - where the file is copied from
    fileArray - a ArrayList object.
    fullOutputPath -
    metadata - metadata entry for the specific file
    
    Returns:
    a boolean.
  - getDecryptCipher
```
protected Cipher getDecryptCipher()
```
    getDecryptCipher.
    
    Returns:
    a Cipher object.
  - getEncryptCipher
```
protected Cipher getEncryptCipher()
```
    getEncryptCipher.
    
    Returns:
    a Cipher object.
  - do_verify_output
```
public ReturnValue do_verify_output()
```
    do_verify_output.
    
    Specified by:
    
    do_verify_output in interface ModuleInterface
    
    Specified by:
    
    do_verify_output in class Module
    
    Returns:

Class ProvisionFiles

Field Summary

Fields inherited from class net.sourceforge.seqware.pipeline.module.Module

Constructor Summary

Method Summary

Methods inherited from class net.sourceforge.seqware.pipeline.module.Module

Methods inherited from class java.lang.Object

Field Detail

options

algorithmName

s3ConnectionTimeout

s3MaxConnections

s3MaxErrorRetry

s3SocketTimeout

Constructor Detail

ProvisionFiles

Method Detail

getOptionParser

get_syntax

do_test

do_verify_parameters

do_verify_input

do_run

provisionFile

putDestination

getDecryptCipher

getEncryptCipher

do_verify_output