Generated using TypeDoc
161 |Generated using TypeDoc
166 |Generated using TypeDoc
179 |Generated using TypeDoc
237 |Parameter type for the getObjectStream function.
74 |Generated using TypeDoc
274 |The JavaScript SDK Managed Download can be used for custom S3 downloads. The client can configure options for the Managed Download such as chunk size and number of concurrent transfers.
67 |This package only works in Node.js versions 6+ currently. Use the npm package manager for Node.js to install the Managed Download package. Type the following command in a terminal window.
70 | npm install @aws/s3-managed-download
71 | You can find the full documentation for the Managed Download package here.
73 |You can customize the part size and number of concurrent downloads for the Managed Download by setting the maxPartSize and maxConcurrency options. The maxPartSize option controls the size in bytes of each part of the transfer. The maxConcurrency option specifies the number of parts that will be downloaded in parallel and the maximum number of parts that will be buffered in memory at any time.
75 |You will need an S3 client in order to create a managed download, so import the aws-sdk along with the managed download package.
77 | const S3 = require('aws-sdk/clients/s3');
79 | const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
80 | import * as S3 from 'aws-sdk/clients/s3';
82 | import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
83 | Create an AWS S3 client and pass the client into the Managed Download constructor to create a Managed Download with a part size of 5MB and concurrency of 1.
85 | const s3 = new S3();
87 | const managedDownloader = new ManagedDownloader(s3);
88 | const s3:S3 = new S3();
90 | const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
91 | Create an AWS S3 client and Managed Download options. Pass the client and the options into the Managed Download constructor to create a Managed Download with a custom part size of 10MB and concurrency of 5.
93 | const s3 = new S3();
95 | const options = {
96 | maxPartSize: 10 * 1024 * 1024,
97 | maxConcurrency: 5
98 | };
99 | const managedDownloader = new ManagedDownloader(s3, options);
100 | const s3:S3 = new S3();
102 | const options:ManagedDownloaderOptons = {
103 | maxPartSize: 10 * 1024 * 1024,
104 | maxConcurrency: 5
105 | };
106 | const managedDownloader:ManagedDownloader = new ManagedDownloader(s3, options);
107 | Currently, the Managed Download only contains a streaming download operation which can work for an entire file or a specific range or part of a file. Here are some examples on 109 | how to use the Managed Download's getObjectStream download operation.
110 |Get the file 'example-key' from the bucket 'example-bucket' and use the getObjectStream method to create a local file at 'example-file-path'.
112 | const S3 = require('aws-sdk/clients/s3');
114 | const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
115 | const fs = require('fs');
116 |
117 | const s3 = new S3();
118 | const managedDownloader = new ManagedDownloader(s3);
119 |
120 | const params = {
121 | Bucket: 'example-bucket',
122 | Key: 'example-key'
123 | };
124 | // create a write stream for a file
125 | const writeStream = fs.createWriteStream('example-file-path');
126 |
127 | managedDownloader.getObjectStream(params)
128 | .then((stream) => {
129 | stream.pipe(writeStream);
130 | }, (err) => {
131 | writeStream.end();
132 | console.error(err);
133 | });
134 | import * as S3 from 'aws-sdk/clients/s3';
136 | import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
137 | import * as fs from 'fs';
138 |
139 | const s3:S3 = new S3();
140 | const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
141 |
142 | const params:GetObjectStreamInput = {
143 | Bucket: 'example-bucket',
144 | Key: 'example-key'
145 | };
146 | // create a write stream for a file
147 | const writeStream:fs.WriteStream = fs.createWriteStream('example-file-path');
148 |
149 | managedDownloader.getObjectStream(params)
150 | .then((stream) => {
151 | stream.pipe(writeStream);
152 | }, (err) => {
153 | writeStream.end();
154 | console.error(err);
155 | });
156 | Get the file 'example-key' from the bucket 'example-bucket' and use the getObjectStream method to write to bytes 100-150 of a local file at 'example-file-path'.
158 | const S3 = require('aws-sdk/clients/s3');
160 | const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
161 | const fs = require('fs');
162 |
163 | const s3 = new S3();
164 | const managedDownloader = new ManagedDownloader(s3);
165 |
166 | const params = {
167 | Bucket: 'example-bucket',
168 | Key: 'example-key',
169 | Range: '100-150'
170 | };
171 | // create a write stream for a file starting at byte 100
172 | const writeStream = fs.createWriteStream('example-file-path', {start:100});
173 |
174 | managedDownloader.getObjectStream(params)
175 | .then((stream) => {
176 | stream.pipe(writeStream);
177 | }, (err) => {
178 | writeStream.end();
179 | console.error(err);
180 | });
181 | import * as S3 from 'aws-sdk/clients/s3';
183 | import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
184 | import * as fs from 'fs';
185 |
186 | const s3:S3 = new S3();
187 | const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
188 |
189 | const params:GetObjectStreamInput = {
190 | Bucket: 'example-bucket',
191 | Key: 'example-key',
192 | Range: '100-150'
193 | };
194 | // create a write stream for a file starting at byte 100
195 | const writeStream:fs.WriteStream = fs.createWriteStream('example-file-path', {start:100});
196 |
197 | managedDownloader.getObjectStream(params)
198 | .then((stream) => {
199 | stream.pipe(writeStream);
200 | }, (err) => {
201 | writeStream.end();
202 | console.error(err);
203 | });
204 | If you find any bugs or want to request a new feature, please first check the existing issues. If there is not already an issue, then open a new issue. If the issue is regarding a bug, please include the stack trace and a method to recreate it.
206 |This library is licensed under the Apache 2.0 License.
208 |Generated using TypeDoc
292 |The ManagedDownloader class handles custom S3 downloads. 74 | A client can set options for the download such as 75 | the part size and the concurrency.
76 |Create a ManagedDownloader object with an 134 | s3 client and custom options if provided.
135 |an S3 client to make requests.
155 |configuration for the managed download.
163 |an S3 client to make requests.
186 |Download the individual parts of the file 238 | and write them to the stream sequentially. 239 | Download n = this.maxConcurrency parts 240 | in parallel and store parts which 241 | aren't the next part in memory until the 242 | stream is ready to read it.
243 |params object contains the bucket 251 | and key.
252 |the ManagedDownloaderStream stream 258 | transferring the parts of a file.
259 |the length of the file.
265 |if available, the amount to add to start 271 | and end byte when calculating the range.
272 |the part number to start downloading from.
278 |Get a download stream for a file stored in s3. 301 | Download the first part of size [maxPartSize] to get 302 | the content length and set headers for the stream. 303 | If the file length is less than [maxPartSize], or the 304 | client provides a part number, then return the stream. 305 | If not, call downloadByParts to split the rest of the file into 306 | parts of size [maxPartSize] and download [maxConcurrency] parts 307 | in parallel.
308 |an object in the format 321 | { 322 | Bucket:string, 323 | Key:string, 324 | Range(optional):string, 325 | PartNumber(optional):number 326 | }.
327 |ManagedDownloaderStream the PassThrough stream to read 332 | file data from.
333 |wait for a stream to emit a 352 | drain event when it is ready to 353 | receive data again.
354 |the stream 362 | that is being watched for the drain 363 | event.
364 |Generated using TypeDoc
491 |
Custom options for the Managed Download.
74 |