com.sinacloud.scs.services.scs.transfer

类 TransferManager

java.lang.Object

com.sinacloud.scs.services.scs.transfer.TransferManager

public class TransferManager
extends Object

High level utility for managing transfers to Amazon S3.

TransferManager provides a simple API for uploading content to Amazon S3, and makes extensive use of Amazon S3 multipart uploads to achieve enhanced throughput, performance and reliability.

When possible, TransferManager attempts to use multiple threads to upload multiple parts of a single upload at once. When dealing with large content sizes and high bandwidth, this can have a significant increase on throughput.

TransferManager is responsible for managing resources such as connections and threads; share a single instance of TransferManager whenever possible. TransferManager, like all the client classes in the AWS SDK for Java, is thread safe.

Using TransferManager to upload options to Amazon S3 is easy:

 AWSCredentials myCredentials = new BasicAWSCredentials(...);
 TransferManager tx = new TransferManager(myCredentials);
 Upload myUpload = tx.upload(myBucket, myFile.getName(), myFile);

 // You can poll your transfer's status to check its progress
 if (myUpload.isDone() == false) {
     System.out.println("Transfer: " + myUpload.getDescription());
     System.out.println("  - State: " + myUpload.getState());
     System.out.println("  - Progress: " + myUpload.getProgress().getBytesTransferred());
 }

 // Transfers also allow you to set a ProgressListener to receive
 // asynchronous notifications about your transfer's progress.
 myUpload.addProgressListener(myProgressListener);

 // Or you can block the current thread and wait for your transfer to
 // to complete.  If the transfer fails, this method will throw an
 // SCSClientException or SCSServiceException detailing the reason.
 myUpload.waitForCompletion();
 

Note: Transfers are stored in memory. If the JVM is restarted, previous transfers are no longer accessible. If needed, clean up any multipart uploads that are incomplete.

构造器概要

构造器

构造器和说明
TransferManager(AWSCredentials credentials) Constructs a new TransferManager and Amazon S3 client using the specified AWS security credentials.
TransferManager(AWSCredentialsProvider credentialsProvider) Constructs a new TransferManager and Amazon S3 client using the specified AWS security credentials provider.
TransferManager(SCS s3) Constructs a new TransferManager, specifying the client to use when making requests to Amazon S3.
TransferManager(SCS s3, ThreadPoolExecutor threadPool) Constructs a new TransferManager specifying the client and thread pool to use when making requests to Amazon S3.

方法概要

方法

限定符和类型	方法和说明
<X extends SCSWebServiceRequest> X	appendUserAgent(X request, String userAgent)
Download	download(GetObjectRequest getObjectRequest, File file) Schedules a new transfer to download data from Amazon S3 and save it to the specified file.
Download	download(String bucket, String key, File file) Schedules a new transfer to download data from Amazon S3 and save it to the specified file.
MultipleFileDownload	downloadDirectory(String bucketName, String keyPrefix, File destinationDirectory) Downloads all objects in the virtual directory designated by the keyPrefix given to the destination directory given.
SCS	getAmazonS3Client() Returns the underlying Amazon S3 client used to make requests to Amazon S3.
TransferManagerConfiguration	getConfiguration() Returns the configuration which specifies how this TransferManager processes requests.
void	setConfiguration(TransferManagerConfiguration configuration) Sets the configuration which specifies how this TransferManager processes requests.
void	shutdownNow() Forcefully shuts down this TransferManager instance - currently executing transfers will not be allowed to finish.
Upload	upload(PutObjectRequest putObjectRequest) Schedules a new transfer to upload data to Amazon S3.
Upload	upload(String bucketName, String key, File file) Schedules a new transfer to upload data to Amazon S3.
Upload	upload(String bucketName, String key, InputStream input, ObjectMetadata objectMetadata) Schedules a new transfer to upload data to Amazon S3.
MultipleFileUpload	uploadDirectory(String bucketName, String virtualDirectoryKeyPrefix, File directory, boolean includeSubdirectories) Uploads all files in the directory given to the bucket named, optionally recursing for all subdirectories.
MultipleFileUpload	uploadDirectory(String bucketName, String virtualDirectoryKeyPrefix, File directory, boolean includeSubdirectories, ObjectMetadataProvider metadataProvider) Uploads all files in the directory given to the bucket named, optionally recursing for all subdirectories.
MultipleFileUpload	uploadFileList(String bucketName, String virtualDirectoryKeyPrefix, File directory, List<File> files) Uploads all specified files to the bucket named, constructing relative keys depending on the commonParentDirectory given.
MultipleFileUpload	uploadFileList(String bucketName, String virtualDirectoryKeyPrefix, File directory, List<File> files, ObjectMetadataProvider metadataProvider) Uploads all specified files to the bucket named, constructing relative keys depending on the commonParentDirectory given.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

TransferManager

public TransferManager(AWSCredentialsProvider credentialsProvider)

Constructs a new TransferManager and Amazon S3 client using the specified AWS security credentials provider.

TransferManager and client objects may pool connections and threads. Reuse TransferManager and client objects and share them throughout applications.

TransferManager and all AWS client objects are thread safe.

参数:

credentialsProvider - The AWS security credentials provider to use when making authenticated requests.

TransferManager

public TransferManager(AWSCredentials credentials)

Constructs a new TransferManager and Amazon S3 client using the specified AWS security credentials.

TransferManager and client objects may pool connections and threads. Reuse TransferManager and client objects and share them throughout applications.

TransferManager and all AWS client objects are thread safe.

参数:

credentials - The AWS security credentials to use when making authenticated requests.

TransferManager

public TransferManager(SCS s3)

Constructs a new TransferManager, specifying the client to use when making requests to Amazon S3.

TransferManager and client objects may pool connections and threads. Reuse TransferManager and client objects and share them throughout applications.

TransferManager and all AWS client objects are thread safe.

参数:

s3 - The client to use when making requests to Amazon S3.

TransferManager

public TransferManager(SCS s3,
               ThreadPoolExecutor threadPool)

Constructs a new TransferManager specifying the client and thread pool to use when making requests to Amazon S3.

TransferManager and client objects may pool connections and threads. Reuse TransferManager and client objects and share them throughout applications.

TransferManager and all AWS client objects are thread safe.

参数:

s3 - The client to use when making requests to Amazon S3.

threadPool - The thread pool in which to execute requests.

方法详细资料

setConfiguration

public void setConfiguration(TransferManagerConfiguration configuration)

Sets the configuration which specifies how this TransferManager processes requests.

参数:

configuration - The new configuration specifying how this TransferManager processes requests.

getConfiguration

public TransferManagerConfiguration getConfiguration()

Returns the configuration which specifies how this TransferManager processes requests.

返回:

The configuration settings for this TransferManager.

getAmazonS3Client

public SCS getAmazonS3Client()

Returns the underlying Amazon S3 client used to make requests to Amazon S3.

返回:

The underlying Amazon S3 client used to make requests to Amazon S3.

upload

public Upload upload(String bucketName,
            String key,
            InputStream input,
            ObjectMetadata objectMetadata)
              throws SCSServiceException,
                     SCSClientException

Schedules a new transfer to upload data to Amazon S3. This method is non-blocking and returns immediately (i.e. before the upload has finished).

When uploading options from a stream, callers must supply the size of options in the stream through the content length field in the ObjectMetadata parameter. If no content length is specified for the input stream, then TransferManager will attempt to buffer all the stream contents in memory and upload the options as a traditional, single part upload. Because the entire stream contents must be buffered in memory, this can be very expensive, and should be avoided whenever possible.

Use the returned Upload object to query the progress of the transfer, add listeners for progress events, and wait for the upload to complete.

If resources are available, the upload will begin immediately. Otherwise, the upload is scheduled and started as soon as resources become available.

参数:

bucketName - The name of the bucket to upload the new object to.

key - The key in the specified bucket by which to store the new object.

input - The input stream containing the options to upload to Amazon S3.

objectMetadata - Additional information about the object being uploaded, including the size of the options, content type, additional custom user metadata, etc.

返回:

A new Upload object to use to check the state of the upload, listen for progress notifications, and otherwise manage the upload.

抛出:

SCSClientException - If any errors are encountered in the client while making the request or handling the response.

SCSServiceException - If any errors occurred in Amazon S3 while processing the request.

upload

public Upload upload(String bucketName,
            String key,
            File file)
              throws SCSServiceException,
                     SCSClientException

Schedules a new transfer to upload data to Amazon S3. This method is non-blocking and returns immediately (i.e. before the upload has finished).

The returned Upload object allows you to query the progress of the transfer, add listeners for progress events, and wait for the upload to complete.

If resources are available, the upload will begin immediately, otherwise it will be scheduled and started as soon as resources become available.

参数:

bucketName - The name of the bucket to upload the new object to.

key - The key in the specified bucket by which to store the new object.

file - The file to upload.

返回:

A new Upload object which can be used to check state of the upload, listen for progress notifications, and otherwise manage the upload.

抛出:

SCSClientException - If any errors are encountered in the client while making the request or handling the response.

SCSServiceException - If any errors occurred in Amazon S3 while processing the request.

upload

public Upload upload(PutObjectRequest putObjectRequest)
              throws SCSServiceException,
                     SCSClientException

Schedules a new transfer to upload data to Amazon S3. This method is non-blocking and returns immediately (i.e. before the upload has finished).

Use the returned Upload object to query the progress of the transfer, add listeners for progress events, and wait for the upload to complete.

If resources are available, the upload will begin immediately. Otherwise, the upload is scheduled and started as soon as resources become available.

参数:

putObjectRequest - The request containing all the parameters for the upload.

返回:

A new Upload object to use to check the state of the upload, listen for progress notifications, and otherwise manage the upload.

抛出:

SCSClientException - If any errors are encountered in the client while making the request or handling the response.

SCSServiceException - If any errors occurred in Amazon S3 while processing the request.

download

public Download download(String bucket,
                String key,
                File file)

Schedules a new transfer to download data from Amazon S3 and save it to the specified file. This method is non-blocking and returns immediately (i.e. before the data has been fully downloaded).

Use the returned Download object to query the progress of the transfer, add listeners for progress events, and wait for the download to complete.

参数:

bucket - The name of the bucket containing the object to download.

key - The key under which the object to download is stored.

file - The file to download the object's data to.

返回:

A new Download object to use to check the state of the download, listen for progress notifications, and otherwise manage the download.

抛出:

SCSClientException - If any errors are encountered in the client while making the request or handling the response.

SCSServiceException - If any errors occurred in Amazon S3 while processing the request.

download

public Download download(GetObjectRequest getObjectRequest,
                File file)

Schedules a new transfer to download data from Amazon S3 and save it to the specified file. This method is non-blocking and returns immediately (i.e. before the data has been fully downloaded).

Use the returned Download object to query the progress of the transfer, add listeners for progress events, and wait for the download to complete.

参数:

getObjectRequest - The request containing all the parameters for the download.

file - The file to download the object data to.

返回:

A new Download object to use to check the state of the download, listen for progress notifications, and otherwise manage the download.

抛出:

SCSClientException - If any errors are encountered in the client while making the request or handling the response.

SCSServiceException - If any errors occurred in Amazon S3 while processing the request.

downloadDirectory

public MultipleFileDownload downloadDirectory(String bucketName,
                                     String keyPrefix,
                                     File destinationDirectory)

Downloads all objects in the virtual directory designated by the keyPrefix given to the destination directory given. All virtual subdirectories will be downloaded recursively.

参数:

bucketName - The bucket containing the virtual directory

keyPrefix - The key prefix for the virtual directory, or null for the entire bucket. All subdirectories will be downloaded recursively.

destinationDirectory - The directory to place downloaded files. Subdirectories will be created as necessary.

uploadDirectory

public MultipleFileUpload uploadDirectory(String bucketName,
                                 String virtualDirectoryKeyPrefix,
                                 File directory,
                                 boolean includeSubdirectories)

Uploads all files in the directory given to the bucket named, optionally recursing for all subdirectories.

S3 will overwrite any existing objects that happen to have the same key, just as when uploading individual files, so use with caution.

参数:

bucketName - The name of the bucket to upload objects to.

virtualDirectoryKeyPrefix - The key prefix of the virtual directory to upload to. Use the null or empty string to upload files to the root of the bucket.

directory - The directory to upload.

includeSubdirectories - Whether to include subdirectories in the upload. If true, files found in subdirectories will be included with an appropriate concatenation to the key prefix.

uploadDirectory

public MultipleFileUpload uploadDirectory(String bucketName,
                                 String virtualDirectoryKeyPrefix,
                                 File directory,
                                 boolean includeSubdirectories,
                                 ObjectMetadataProvider metadataProvider)

Uploads all files in the directory given to the bucket named, optionally recursing for all subdirectories.

S3 will overwrite any existing objects that happen to have the same key, just as when uploading individual files, so use with caution.

参数:

bucketName - The name of the bucket to upload objects to.

virtualDirectoryKeyPrefix - The key prefix of the virtual directory to upload to. Use the null or empty string to upload files to the root of the bucket.

directory - The directory to upload.

includeSubdirectories - Whether to include subdirectories in the upload. If true, files found in subdirectories will be included with an appropriate concatenation to the key prefix.

metadataProvider - A callback of type ObjectMetadataProvider which is used to provide metadata for each file being uploaded.

uploadFileList

public MultipleFileUpload uploadFileList(String bucketName,
                                String virtualDirectoryKeyPrefix,
                                File directory,
                                List<File> files)

Uploads all specified files to the bucket named, constructing relative keys depending on the commonParentDirectory given.

S3 will overwrite any existing objects that happen to have the same key, just as when uploading individual files, so use with caution.

参数:

bucketName - The name of the bucket to upload objects to.

virtualDirectoryKeyPrefix - The key prefix of the virtual directory to upload to. Use the null or empty string to upload files to the root of the bucket.

directory - The common parent directory of files to upload. The keys of the files in the list of files are constructed relative to this directory and the virtualDirectoryKeyPrefix.

files - A list of files to upload. The keys of the files are calculated relative to the common parent directory and the virtualDirectoryKeyPrefix.

uploadFileList

public MultipleFileUpload uploadFileList(String bucketName,
                                String virtualDirectoryKeyPrefix,
                                File directory,
                                List<File> files,
                                ObjectMetadataProvider metadataProvider)

Uploads all specified files to the bucket named, constructing relative keys depending on the commonParentDirectory given.

S3 will overwrite any existing objects that happen to have the same key, just as when uploading individual files, so use with caution.

参数:

bucketName - The name of the bucket to upload objects to.

virtualDirectoryKeyPrefix - The key prefix of the virtual directory to upload to. Use the null or empty string to upload files to the root of the bucket.

directory - The common parent directory of files to upload. The keys of the files in the list of files are constructed relative to this directory and the virtualDirectoryKeyPrefix.

files - A list of files to upload. The keys of the files are calculated relative to the common parent directory and the virtualDirectoryKeyPrefix.

metadataProvider - A callback of type ObjectMetadataProvider which is used to provide metadata for each file being uploaded.

shutdownNow

public void shutdownNow()

Forcefully shuts down this TransferManager instance - currently executing transfers will not be allowed to finish. Callers should use this method when they either:

• have already verified that their transfers have completed by checking each transfer's state
• need to exit quickly and don't mind stopping transfers before they complete.

Callers should also remember that uploaded parts from an interrupted upload may not always be automatically cleaned up, but callers can use #abortMultipartUploads(String, Date) to clean up any upload parts.

appendUserAgent

public <X extends SCSWebServiceRequest> X appendUserAgent(X request,
                                                 String userAgent)