8 | This package handles chunked files upload requests in order to safely save chunked files and, once the upload has been completed, merge all the chunks into a single file.
9 |
10 |
11 | Report Bug
12 | ·
13 | Request Feature
14 |
15 |
16 |
17 | [](https://github.com/jobtech-dev/laravel-chunky/blob/master/LICENSE.txt)
18 | [](https://github.com/jobtech-dev/laravel-chunky/actions)
19 | [](https://github.styleci.io/repos/291024576?branch=master)
20 | [](https://github.com/jobtech-dev/laravel-chunky/stargazers)
21 | [](https://github.com/jobtech-dev/laravel-chunky/issues)
22 | [](https://linkedin.com/in/jobtech-srl)
23 |
24 | ## Table of Contents
25 |
26 | * [Getting Started](#getting-started)
27 | * [Main features](#main-features)
28 | * [Installation](#installation)
29 | * [Usage](#usage)
30 | * [Chunks](#chunks)
31 | * [Merge handler](#merge-handler)
32 | * [Testing](#testing)
33 | * [Roadmap](#roadmap)
34 | * [Changelog](#changelog)
35 | * [Contributing](#contributing)
36 | * [License](#license)
37 | * [Contact](#contact)
38 | * [Credits](#credits)
39 |
40 | ## Laravel compatibility
41 |
42 | Laravel | laravel-chunky
43 | :-------------|:----------
44 | 10.x | 3.0.0
45 | 9.x | 2.0.0
46 | 8.x | 1.4.1
47 | 7.x | 1.4.1
48 | 6.x | 1.4.1
49 |
50 | ## Getting Started
51 |
52 | Laravel chunky is a package that can handle chunk upload for large files in Laravel 6.x, 7.x. and 8.x. Its main goal is automatically handle the upload request (see the [usage](#usage) section below) and save all the chunks into the desired disk.
53 |
54 | Once the upload completes, the package will dispatch a job in order to merge all the files into a single one and save in the same chunks disks or in another one.
55 |
56 | ### Main features
57 | * Handle chunks upload with custom save disks and folders.
58 | * Handle file merge with custom save disks and folders.
59 | * Once the merge is done, the chunks folder is automatically cleared.
60 |
61 | ### Installation
62 |
63 | In order to install Laravel Chunky into your project you have to require it via composer.
64 |
65 | ```sh
66 | $ composer require jobtech/laravel-chunky
67 | ```
68 |
69 | Laravel uses Package Auto-Discovery and the `ChunkyServiceProvider` will be automatically registered. If you are using a Laravel version lower than 5.5 or you're not using autodiscovery, manually register the service provider:
70 |
71 | ```php
72 | // config/app.php
73 | [
74 | // [...]
75 | 'providers' => [
76 | // [...]
77 | Jobtech\LaravelChunky\ChunkyServiceProvider::class,
78 | ]
79 | ];
80 | ```
81 |
82 | You can also register an alias for the `Chunky` facade:
83 |
84 | ```php
85 | // config/app.php
86 | [
87 | // [...]
88 | 'aliases' => [
89 | // [...]
90 | 'Chunky' => Jobtech\LaravelChunky\Facades\Chunky::class,
91 | ]
92 | ];
93 | ```
94 |
95 | #### Configuration
96 |
97 | To publish the configuration file, run the following command:
98 |
99 | ```sh
100 | $ php artisan vendor:publish --provider="Jobtech\LaravelChunky\ChunkyServiceProvider" --tag="config"
101 | ```
102 |
103 | #### Lumen
104 |
105 | This package can also work with Lumen, just register the service provider in `bootstrap/app.php`:
106 |
107 | ```php
108 | $app->register(Jobtech\LaravelChunky\ChunkyServiceProvider::class);
109 | ```
110 |
111 | In order to configure the package, since lumen doesn't include the `vendor:publish` command, copy the configuration file to your config folder and enable it:
112 |
113 | ```php
114 | $app->configure('chunky');
115 | ```
116 |
117 | ## Usage
118 |
119 | This package has been designed to leave you the full control of the chunks upload and simple use the helper methods to handle the files merge as well as an _all-in-one_ solution for a fast scaffolding of the controllers delegated to handle large files upload.
120 |
121 | At the moment, this package doesn't include any wrapper for the frontend forms for the file uploads but, in the `config/chunky.php` configuration file, you can find two ways of integrate the package with [Dropzone](https://www.dropzonejs.com/) and [ResumableJs](http://resumablejs.com/).
122 |
123 | ### Chunks
124 |
125 | Laravel Chunky handles the chunks as an _ordered list_ of files. This is a **must** and if a wrong file index has been uploaded, an exception will be thrown in order to guarantee the integrity of the final merged file. Once all the chunks have been uploaded, and the merge process is executing, another integrity check will be made to all the chunks. If the sum of each file size is lower than the original file size, another exception will be thrown. For this reason a chunk request must include both the chunk and these attributes:
126 |
127 | * An `index`: indicates the current chunk that is uploading. The first index can be set in the configuration file.
128 | * A `file size`: the original file size. Will be used for the integrity check.
129 | * A `chunk size`: the chunk file size. Will be used for the integrity check.
130 |
131 | #### Configuration
132 |
133 | ```php
134 | // config/chunky.php
135 | [
136 |
137 | /*
138 | |--------------------------------------------------------------------------
139 | | Default disks
140 | |--------------------------------------------------------------------------
141 | |
142 | | This option defines the disks on which to store the chunks from an upload
143 | | request as well as the final merged file. If you don't need to save the
144 | | files into a sub folder just set null as value.
145 | |
146 | */
147 |
148 | 'disks' => [
149 | 'chunks' => [
150 | 'disk' => env('CHUNKY_CHUNK_DISK'),
151 | 'folder' => 'chunks',
152 | ],
153 | ],
154 |
155 | /*
156 | |--------------------------------------------------------------------------
157 | | Default index
158 | |--------------------------------------------------------------------------
159 | |
160 | | This option defines if chunky should start counting the chunks indexes
161 | | from 0 (ChunkySettings::INDEX_ZERO) or 1 (ChunkySettings::INDEX_ONE). You
162 | | can override this feature with any number, but the indexes must always
163 | | be index + n or the integrity check for the chunks folder will throw an
164 | | exception.
165 | |
166 | */
167 |
168 | 'index' => \Jobtech\LaravelChunky\ChunkySettings::INDEX_ZERO,
169 |
170 | /*
171 | |--------------------------------------------------------------------------
172 | | Additional options
173 | |--------------------------------------------------------------------------
174 | |
175 | | This option defines the additional settings that chunky should pass to
176 | | the `storeAs` method while saving chunks or the merged file. This can be
177 | | useful, for example, when storing public files in S3 storage.
178 | |
179 | */
180 |
181 | 'options' => [
182 | 'chunks' => [
183 | // 'visibility' => 'public'
184 | ],
185 | ],
186 | ];
187 | ```
188 |
189 | #### Chunks methods
190 |
191 | If you want to manually save a chunk from a request you can use the `addChunk` method. It gets in input the uploaded file, the chunk index and, optionally, a folder name. If no folder is passed, the chunks anyway will be stored into a chunk's root subfolder. This folder will be named as the slug of the uploaded file basename.
192 |
193 | This method will return a `Jobtech\LaravelChunky\Chunk` object, that implements the `Illuminate\Contracts\Support\Responsable` contract so you can easily return a JSON response. If the requests has the `Accept application/json` header, the object will be automatically transformed into a `Jobtech\LaravelChunky\Http\Resources\ChunkResource` object. Furthermore, every time a chunk is added a `Jobtech\LaravelChunky\Events\ChunkAdded` event is fired.
194 |
195 | ```php
196 | // ...
197 | $chunk = Chunky::addChunk(
198 | $request->file('your-file-key'),
199 | $request->input('your-index-key'),
200 | 'folder-is-optional'
201 | );
202 |
203 | return $chunk->hideFileInfo()->toResponse();
204 | // This will return
205 | // {
206 | // "data": {
207 | // "name": "my-very-big-file.ext",
208 | // "extension": "ext",
209 | // "index": 0,
210 | // "last": false
211 | // }
212 | // }
213 |
214 | return $chunk->showFileInfo()->toResponse();
215 | // This will return
216 | // {
217 | // "data": {
218 | // "name": "my-very-big-file.ext",
219 | // "extension": "ext",
220 | // "index": 0,
221 | // "last": false,
222 | // "file": "/path/to/my-very-big-file.ext",
223 | // "path": "/path/to"
224 | // }
225 | // }
226 | ```
227 |
228 | Everytime a chunk is added, a `Jobtech\LaravelChunky\Events\ChunkDeleted` event is fired.
229 |
230 | > If you're trying to add a chunk that violates the integrity of the chunks folder an exception will be thrown.
231 | > for example:
232 | >
233 | >```php
234 | > |- chunks
235 | > |- folder
236 | > |- 0_chunk.ext
237 | > |- 1_chunk.ext
238 | > |- 2_chunk.ext
239 | >
240 | > Chunk::addChunk($chunk, 4);
241 | >```
242 | > This will throw a `Jobtech\LaravelChunky\Exceptions\ChunksIntegrityException`
243 |
244 | If you're using, for example, Dropzone you can block the upload action, in that case you will delete the currently uploaded chunks:
245 |
246 |
247 | ```php
248 | Chunky::deleteChunks('chunks-folder');
249 | ```
250 | Everytime a chunk is deleted, a `Jobtech\LaravelChunky\Events\ChunkDeleted` event is fired.
251 |
252 | ---
253 |
254 | The package include a method that, given the chunks folder, will return a sorted collection. Each item contains the relative chunk's path and index.
255 |
256 | ```php
257 | $chunks = Chunky::listChunks('chunks-folder-name');
258 |
259 | foreach($chunks as $chunk) {
260 | /** @var \Jobtech\LaravelChunky\Chunk $chunk */
261 | print_r($chunk->toArray());
262 | }
263 |
264 | // [
265 | // 'index' => 0,
266 | // 'path' => '/path/to/chunks-folder-name/0_chunk.ext',
267 | // [...]
268 | // ],
269 | // [
270 | // 'index' => 1,
271 | // 'path' => '/path/to/chunks-folder-name/1_chunk.ext',
272 | // [...]
273 | // ],
274 | // [
275 | // 'index' => 2,
276 | // 'path' => '/path/to/chunks-folder-name/2_chunk.ext',
277 | // [...]
278 | // ],
279 | // ...
280 | ```
281 |
282 | #### Chunks request
283 |
284 | If you want to automate the chunks upload and merge (requires the value `true` for the `auto_merge` config key), you can use the `Jobtech\LaravelChunky\Http\Requests\AddChunkRequest` class. For more informations about form request please have a look at the [official documentation](https://laravel.com/docs/6.x/validation#form-request-validation).
285 |
286 | Include the form request in your method and simply call the `handle` method of the Chunky facade. The package will automatically handle the upload and return a `Jobtech\LaravelChunky\Chunk` object.
287 |
288 | ```php
289 | // Example with `auto_merge = false`
290 |
291 | use Jobtech\LaravelChunky\Http\Requests\AddChunkRequest;
292 |
293 | class UploadController extends Controller {
294 | // [...]
295 |
296 | public function chunkUpload(AddChunkRequest $request) {
297 | $chunk = Chunky::handle($request, 'folder-is-optional');
298 |
299 | if($chunk->isLast()) {
300 | // See section below for merge or
301 | // implement your own logic
302 | }
303 |
304 | return $chunk->toResponse();
305 | }
306 | }
307 | ```
308 |
309 | ### Merge handler
310 |
311 | If you need to merge chunks into a single file, you can call the `merge` function that will use the configured merge handler to concatenate all the uploaded chunks into a single file.
312 |
313 | ```
314 | public function chunkUpload(AddChunkRequest $request) {
315 | $chunk = Chunky::handle($request, 'folder-is-optional');
316 |
317 | if($chunk->isLast()) {
318 | Chunky::merge('upload-folder', 'your/merge/file.ext');
319 | }
320 |
321 | return $chunk->toResponse();
322 | }
323 | ```
324 |
325 | Once the last chunk has been uploaded and the `auto_merge` config key has `true` value, the package will automatically merge the chunks. A `Jobtech\LaravelChunky\Jobs\MergeChunks` job will be dispatched on the given connection and queue if these options have been set.
326 |
327 | ```php
328 | // config/chunky.php
329 | [
330 | // [...]
331 |
332 | /*
333 | |--------------------------------------------------------------------------
334 | | Merge settings
335 | |--------------------------------------------------------------------------
336 | |
337 | | This option defines the merge handler that should be used to perform the
338 | | chunks merge once the upload is completed (automagically depending on
339 | | `auto_merge` config value.
340 | |
341 | | `connection` and `queue` keys define which queue and which connection
342 | | should be used for the merge job. If connection is null, a synchronous
343 | | job will be dispatched
344 | */
345 |
346 | 'merge' => [
347 | 'handler' => \Jobtech\LaravelChunky\Handlers\MergeHandler::class,
348 |
349 | 'connection' => env('CHUNKY_MERGE_CONNECTION', 'sync'),
350 |
351 | 'queue' => env('CHUNKY_MERGE_QUEUE'),
352 | ],
353 | ];
354 | ```
355 |
356 | You can manually dispatch the job (or if you're not using the `Jobtech\LaravelChunky\Http\Requests\AddChunkRequest` form request create your own):
357 |
358 | ```php
359 | use Jobtech\LaravelChunky\Http\Requests\AddChunkRequest;
360 | use Jobtech\LaravelChunky\Jobs\MergeChunks;
361 |
362 | class UploadController extends Controller {
363 | // [...]
364 |
365 | public function chunkUpload(AddChunkRequest $request) {
366 | $chunk = Chunky::handle($request, 'folder-is-optional');
367 |
368 | if($chunk->isLast()) {
369 | $job = new MergeChunks($request, 'chunks-folder', 'destination/path/to/merge.ext');
370 |
371 | dispatch(
372 | $job->onConnection('your-connection')
373 | ->onQueue('your-queue')
374 | );
375 | }
376 |
377 | return $chunk->toResponse();
378 | }
379 | }
380 | ```
381 |
382 | Once the job is completed, a `Jobtech\LaravelChunky\Events\ChunksMerged` event is fired as well as once the merge file is moved to destination a `Jobtech\LaravelChunky\Events\MergeAdded` event is fired.
383 |
384 | #### Custom handler
385 |
386 | If you want to integrate your own handler, remember to implement the `Jobtech\LaravelChunky\Contracts\MergeHandler` contract (or at least implement the same methods) in your class, and update the related `handler` configuration option:
387 |
388 | ```php
389 | use Jobtech\LaravelChunky\Contracts\MergeHandler;
390 |
391 | class MyHandler implements MergeHandler {
392 | private ChunkyManager $manager;
393 |
394 | /**
395 | * @param \Jobtech\LaravelChunky\Contracts\ChunkyManager $manager
396 | * @return \Jobtech\LaravelChunky\Handlers\MergeHandler
397 | */
398 | public function setManager(ChunkyManager $manager): MergeHandler
399 | {
400 | $this->manager = $manager;
401 |
402 | return $this;
403 | }
404 |
405 | /**
406 | * @return \Jobtech\LaravelChunky\Contracts\ChunkyManager
407 | * @throws \Illuminate\Contracts\Container\BindingResolutionException
408 | */
409 | public function manager(): ChunkyManager
410 | {
411 | return $this->manager;
412 | }
413 |
414 | /**
415 | * @param \Jobtech\LaravelChunky\Http\Requests\AddChunkRequest $request
416 | * @param string $folder
417 | *
418 | * @return \Illuminate\Foundation\Bus\PendingDispatch|string
419 | */
420 | public function dispatchMerge(AddChunkRequest $request, string $folder)
421 | {
422 | // Your logic here
423 | }
424 |
425 | /**
426 | * @param string $chunks_folder
427 | * @param string $merge_destination
428 | *
429 | * @return string
430 | */
431 | public function merge(string $chunks_folder, string $merge_destination): string
432 | {
433 | // Your logic here
434 | }
435 |
436 | /**
437 | * @return \Jobtech\LaravelChunky\Contracts\MergeHandler
438 | */
439 | public static function instance(): MergeHandler
440 | {
441 | return new static();
442 | }
443 | }
444 | ```
445 |
446 | ## Testing
447 |
448 | You can run the tests with PHP unit:
449 |
450 | ```sh
451 | $ vendor/bin/phpunit
452 | ```
453 |
454 | If you want to set custom environment variable, you can add a `.env` file for custom disks, queue or whatever you need. Tests anyway set a temporary local disk by default.
455 |
456 | ```
457 | CHUNKY_CHUNK_DISK=s3
458 | CHUNKY_MERGE_DISK=public
459 | CHUNKY_AUTO_MERGE=false
460 | CHUNKY_MERGE_CONNECTION=redis
461 | CHUNKY_MERGE_QUEUE=my-custom-queue
462 | ```
463 |
464 | ## Roadmap
465 |
466 | See the [open issues](https://github.com/jobtech-dev/laravel-chunky/issues) for a list of proposed features (and known issues).
467 |
468 | We're working on:
469 |
470 | * Integrate frontend chunk upload (Not sure if necessary... there are so many packages that does it).
471 | * Custom concatenation, at the moment we're using a third party package.
472 | * Better tests.
473 | * Laravel 5.5+ compatibility.
474 |
475 | ## Changelog
476 |
477 | Please see [CHANGELOG.md](https://github.com/jobtech-dev/laravel-chunky/blob/master/CHANGELOG.md) for more information what has changed recently.
478 |
479 | ## Contributing
480 | This package comes with a docker container based on php 8.1 and composer 2.2. To start it simply run `make start`. To enter the container shell you can use `make shell`.
481 |
482 | Please see [CONTRIBUTING.md](https://github.com/jobtech-dev/laravel-chunky/blob/master/CONTRIBUTING.md) for more details.
483 |
484 | ## License
485 |
486 | Distributed under the MIT License. See [LICENSE](https://github.com/jobtech-dev/laravel-chunky/blob/master/LICENSE) for more information.
487 |
488 | ## Contact
489 |
490 | Jobtech dev team - [dev@jobtech.it](mailto:dev@jobtech.it)
491 |
492 | ## Credits
493 |
494 | Laravel Chunky is a Laravel package made with :heart: by the JT nerds.
495 |
496 | Thanks to:
497 |
498 | * Filippo Galante ([ilGala](https://github.com/ilgala))
499 | * [All contributors](https://github.com/jobtech-dev/laravel-chunky/graphs/contributors)
500 |
501 | We've used these packages for the chunks concatenation:
502 |
503 | * Keven Godet - [Flysystem concatenate](https://github.com/kevengodet/flysystem-concatenate)
504 |
505 | And this repository for the readme boilerplate:
506 |
507 | * Othneil Drew - [Best-README-Template](https://github.com/othneildrew/Best-README-Template)
508 |
--------------------------------------------------------------------------------
/composer.json:
--------------------------------------------------------------------------------
1 | {
2 | "name": "jobtech/laravel-chunky",
3 | "description": "A laravel manager to handle chunked files upload",
4 | "keywords": ["upload", "chunk", "laravel", "jobtech", "chunky"],
5 | "license": "MIT",
6 | "authors": [
7 | {
8 | "name": "ilGala",
9 | "email": "filippo.galante@jobtech.it",
10 | "homepage": "https://jobtech.it"
11 | }
12 | ],
13 | "homepage": "https://github.com/jobtech-dev/laravel-chunky",
14 | "require": {
15 | "php": "^8.1",
16 | "ext-fileinfo": "*",
17 | "ext-json": "*",
18 | "illuminate/contracts": "^10.0",
19 | "illuminate/filesystem": "^10.0",
20 | "illuminate/support": "^10.0",
21 | "keven/append-stream": "^1.0.5"
22 | },
23 | "require-dev": {
24 | "phpunit/phpunit": "^10.0",
25 | "mockery/mockery": "^1.4",
26 | "aws/aws-sdk-php": "^3.155",
27 | "league/flysystem-aws-s3-v3": "^3.0",
28 | "orchestra/testbench": "^8.21",
29 | "friendsofphp/php-cs-fixer": "^3.49"
30 | },
31 | "suggest": {
32 | "league/flysystem-aws-s3-v3": "Required to use AWS S3 file storage"
33 | },
34 | "autoload": {
35 | "psr-4": {
36 | "Jobtech\\LaravelChunky\\": "src"
37 | }
38 | },
39 | "autoload-dev": {
40 | "psr-4": {
41 | "Jobtech\\LaravelChunky\\Tests\\": "tests"
42 | }
43 | },
44 | "extra": {
45 | "laravel": {
46 | "providers": [
47 | "Jobtech\\LaravelChunky\\ChunkyServiceProvider"
48 | ]
49 | }
50 | },
51 | "scripts": {
52 | "test": "vendor/bin/phpunit",
53 | "lint": "vendor/bin/php-cs-fixer fix"
54 | },
55 | "minimum-stability": "dev",
56 | "prefer-stable": true
57 | }
58 |
--------------------------------------------------------------------------------
/config/chunky.php:
--------------------------------------------------------------------------------
1 | [
20 | 'chunks' => [
21 | 'disk' => env('CHUNKY_CHUNK_DISK', 'local'),
22 | 'folder' => 'chunks',
23 | ],
24 | 'merge' => [
25 | 'disk' => env('CHUNKY_MERGE_DISK', 'local'),
26 | 'folder' => null,
27 | ],
28 | 'tmp' => [
29 | 'disk' => env('CHUNKY_TMP_DISK', 'local'),
30 | 'folder' => null,
31 | ],
32 | ],
33 |
34 | /*
35 | |--------------------------------------------------------------------------
36 | | Default index
37 | |--------------------------------------------------------------------------
38 | |
39 | | This option defines if chunky should start counting the chunks indexes
40 | | from 0 (ChunkySettings::INDEX_ZERO) or 1 (ChunkySettings::INDEX_ONE). You
41 | | can override this feature with any number, but the indexes must always
42 | | be index + n or the integrity check for the chunks folder will throw an
43 | | exception.
44 | |
45 | */
46 |
47 | 'index' => ChunkySettings::INDEX_ZERO,
48 |
49 | /*
50 | |--------------------------------------------------------------------------
51 | | Additional options
52 | |--------------------------------------------------------------------------
53 | |
54 | | This option defines the additional settings that chunky should pass to
55 | | the `storeAs` method while saving chunks or the merged file. This can be
56 | | useful, for example, when storing public files in S3 storage.
57 | |
58 | */
59 |
60 | 'options' => [
61 | 'chunks' => [],
62 |
63 | 'merge' => [
64 | // 'visibility' => 'public'
65 | ],
66 | ],
67 |
68 | /*
69 | |--------------------------------------------------------------------------
70 | | Validation rules
71 | |--------------------------------------------------------------------------
72 | |
73 | | When using a chunk request these rules will be used to validate the input.
74 | | `index`, `file`, `chunkSize` and `totalSize` attributes are mandatory in
75 | | order to let chunky work properly, so if missing from the configuration
76 | | file an exception will be thrown.
77 | */
78 |
79 | 'validation' => [
80 | // Mandatory
81 | 'index' => [
82 | 'key' => 'index',
83 | 'rules' => ['required', 'integer', 'min:0'],
84 | ],
85 | 'file' => [
86 | 'key' => 'file',
87 | 'rules' => ['required', 'file'],
88 | ],
89 | 'chunkSize' => [
90 | 'key' => 'chunkSize',
91 | 'rules' => ['required', 'integer', 'min:1'],
92 | ],
93 | 'totalSize' => [
94 | 'key' => 'totalSize',
95 | 'rules' => ['required', 'integer', 'min:1'],
96 | ],
97 | // Optional
98 | 'folder' => [
99 | 'key' => 'folder',
100 | 'rules' => ['filled', 'string'],
101 | ],
102 | // --------------------------------------------------------------------------
103 | // Dropzone chunk uploads example
104 | // --------------------------------------------------------------------------
105 | // 'index' => [
106 | // 'key' => 'dzchunkindex',
107 | // 'rules' => ['required', 'integer', 'min:0']
108 | // ],
109 | // 'file' => [
110 | // 'key' => 'file',
111 | // 'rules' => ['required', 'file']
112 | // ],
113 | // 'chunkSize' => [
114 | // 'key' => 'dzchunksize',
115 | // 'rules' => ['required', 'integer', 'min:1']
116 | // ],
117 | // 'totalSize' => [
118 | // 'key' => 'dztotalfilesize',
119 | // 'rules' => ['required', 'integer', 'min:1']
120 | // ],
121 | // // Optional
122 | // 'folder' => [
123 | // 'key' => 'dzuuid',
124 | // 'rules' => ['required', 'string']
125 | // ]
126 | // --------------------------------------------------------------------------
127 | // Resumable js chunk uploads example
128 | // --------------------------------------------------------------------------
129 | // 'index' => [
130 | // 'key' => 'resumableChunkNumber',
131 | // 'rules' => ['required', 'integer', 'min:0']
132 | // ],
133 | // 'file' => [
134 | // 'key' => 'file',
135 | // 'rules' => ['required', 'file']
136 | // ],
137 | // 'chunkSize' => [
138 | // 'key' => 'resumableChunkSize',
139 | // 'rules' => ['required', 'integer', 'min:1']
140 | // ],
141 | // 'totalSize' => [
142 | // 'key' => 'resumableTotalSize',
143 | // 'rules' => ['required', 'integer', 'min:1']
144 | // ],
145 | // // Optional
146 | // 'folder' => [
147 | // 'key' => 'resumableIdentifier',
148 | // 'rules' => ['required', 'string']
149 | // ]
150 | ],
151 |
152 | /*
153 | |--------------------------------------------------------------------------
154 | | Chunk json resource
155 | |--------------------------------------------------------------------------
156 | |
157 | | This option defines the class to use when a chunk is transformed to a
158 | | json resource and returned as response.
159 | |
160 | */
161 |
162 | 'resource' => ChunkResource::class,
163 |
164 | /*
165 | |--------------------------------------------------------------------------
166 | | Automatically merge chunks
167 | |--------------------------------------------------------------------------
168 | |
169 | | This option defines if chunky should automatically dispatch a merge job
170 | | once the last chunk has been upload.
171 | |
172 | */
173 |
174 | 'auto_merge' => env('CHUNKY_AUTO_MERGE', true),
175 |
176 | /*
177 | |--------------------------------------------------------------------------
178 | | Merge settings
179 | |--------------------------------------------------------------------------
180 | |
181 | | This option defines the merge handler that should be used to perform the
182 | | chunks merge once the upload is completed (automagically depending on
183 | | `auto_merge` config value.
184 | |
185 | | `connection` and `queue` keys define which queue and which connection
186 | | should be used for the merge job. If connection is null, a synchronous
187 | | job will be dispatched
188 | */
189 |
190 | 'merge' => [
191 | 'handler' => MergeHandler::class,
192 |
193 | 'connection' => env('CHUNKY_MERGE_CONNECTION', 'sync'),
194 |
195 | 'queue' => env('CHUNKY_MERGE_QUEUE'),
196 | ],
197 | ];
198 |
--------------------------------------------------------------------------------
/docker-compose.yml:
--------------------------------------------------------------------------------
1 | version: "3.9"
2 |
3 | services:
4 | laravel-chunky:
5 | build:
6 | context: .
7 | dockerfile: Dockerfile
8 | container_name: laravel-chunky
9 | working_dir: /var/www/html
10 | volumes:
11 | - ./:/var/www/html
12 |
--------------------------------------------------------------------------------
/docs/_config.yml:
--------------------------------------------------------------------------------
1 | theme: jekyll-theme-minimal
2 | title: Laravel Chunky
3 | description: A laravel manager to handle chunked files upload
4 | logo: https://user-images.githubusercontent.com/1577699/100456224-4ab28680-30c0-11eb-8452-e6a674f3dcdb.png
5 | show_downloads: true
--------------------------------------------------------------------------------
/docs/_layouts/default.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 | {% seo %}
9 |
10 |
13 |
14 |
15 |
8 | This package handles chunked files upload requests in order to safely save chunked files and, once the upload has been completed, merge all the chunks into a single file.
9 |
10 |
11 | Report Bug
12 | ·
13 | Request Feature
14 |
15 |
16 |
17 | [](https://github.com/jobtech-dev/laravel-chunky/blob/master/LICENSE.txt)
18 | [](https://github.com/jobtech-dev/laravel-chunky/actions)
19 | [](https://github.styleci.io/repos/291024576?branch=master)
20 | [](https://github.com/jobtech-dev/laravel-chunky/stargazers)
21 | [](https://github.com/jobtech-dev/laravel-chunky/issues)
22 | [](https://linkedin.com/in/jobtech-srl)
23 |
24 | ## Table of Contents
25 |
26 | * [Getting Started](#getting-started)
27 | * [Main features](#main-features)
28 | * [Installation](#installation)
29 | * [Usage](#usage)
30 | * [Chunks](#chunks)
31 | * [Merge handler](#merge-handler)
32 | * [Testing](#testing)
33 | * [Roadmap](#roadmap)
34 | * [Changelog](#changelog)
35 | * [Contributing](#contributing)
36 | * [License](#license)
37 | * [Contact](#contact)
38 | * [Credits](#credits)
39 |
40 | ## Getting Started
41 |
42 | Laravel chunky is a package that can handle chunk upload for large files in Laravel 6.x, 7.x. and 8.x. Its main goal is automatically handle the upload request (see the [usage](#usage) section below) and save all the chunks into the desired disk.
43 |
44 | Once the upload completes, the package will dispatch a job in order to merge all the files into a single one and save in the same chunks disks or in another one.
45 |
46 | ### Main features
47 | * Handle chunks upload with custom save disks and folders.
48 | * Handle file merge with custom save disks and folders.
49 | * Once the merge is done, the chunks folder is automatically cleared.
50 |
51 | ### Installation
52 |
53 | In order to install Laravel Chunky into your project you have to require it via composer.
54 |
55 | ```sh
56 | $ composer require jobtech/laravel-chunky
57 | ```
58 |
59 | Laravel uses Package Auto-Discovery and the `ChunkyServiceProvider` will be automatically registered. If you are using a Laravel version lower than 5.5 or you're not using autodiscovery, manually register the service provider:
60 |
61 | ```php
62 | // config/app.php
63 | [
64 | // [...]
65 | 'providers' => [
66 | // [...]
67 | Jobtech\LaravelChunky\ChunkyServiceProvider::class,
68 | ]
69 | ];
70 | ```
71 |
72 | You can also register an alias for the `Chunky` facade:
73 |
74 | ```php
75 | // config/app.php
76 | [
77 | // [...]
78 | 'aliases' => [
79 | // [...]
80 | 'Chunky' => Jobtech\LaravelChunky\Facades\Chunky::class,
81 | ]
82 | ];
83 | ```
84 |
85 | #### Configuration
86 |
87 | To publish the configuration file, run the following command:
88 |
89 | ```sh
90 | $ php artisan vendor:publish --provider="Jobtech\LaravelChunky\ChunkyServiceProvider" --tag="config"
91 | ```
92 |
93 | #### Lumen
94 |
95 | This package can also work with Lumen, just register the service provider in `bootstrap/app.php`:
96 |
97 | ```php
98 | $app->register(Jobtech\LaravelChunky\ChunkyServiceProvider::class);
99 | ```
100 |
101 | In order to configure the package, since lumen doesn't include the `vendor:publish` command, copy the configuration file to your config folder and enable it:
102 |
103 | ```php
104 | $app->configure('chunky');
105 | ```
106 |
107 | ## Usage
108 |
109 | This package has been designed to leave you the full control of the chunks upload and simple use the helper methods to handle the files merge as well as an _all-in-one_ solution for a fast scaffolding of the controllers delegated to handle large files upload.
110 |
111 | At the moment, this package doesn't include any wrapper for the frontend forms for the file uploads but, in the `config/chunky.php` configuration file, you can find two ways of integrate the package with [Dropzone](https://www.dropzonejs.com/) and [ResumableJs](http://resumablejs.com/).
112 |
113 | ### Chunks
114 |
115 | Laravel Chunky handles the chunks as an _ordered list_ of files. This is a **must** and if a wrong file index has been uploaded, an exception will be thrown in order to guarantee the integrity of the final merged file. Once all the chunks have been uploaded, and the merge process is executing, another integrity check will be made to all the chunks. If the sum of each file size is lower than the original file size, another exception will be thrown. For this reason a chunk request must include both the chunk and these attributes:
116 |
117 | * An `index`: indicates the current chunk that is uploading. The first index can be set in the configuration file.
118 | * A `file size`: the original file size. Will be used for the integrity check.
119 | * A `chunk size`: the chunk file size. Will be used for the integrity check.
120 |
121 | #### Configuration
122 |
123 | ```php
124 | // config/chunky.php
125 | [
126 |
127 | /*
128 | |--------------------------------------------------------------------------
129 | | Default disks
130 | |--------------------------------------------------------------------------
131 | |
132 | | This option defines the disks on which to store the chunks from an upload
133 | | request as well as the final merged file. If you don't need to save the
134 | | files into a sub folder just set null as value.
135 | |
136 | */
137 |
138 | 'disks' => [
139 | 'chunks' => [
140 | 'disk' => env('CHUNKY_CHUNK_DISK'),
141 | 'folder' => 'chunks',
142 | ],
143 | ],
144 |
145 | /*
146 | |--------------------------------------------------------------------------
147 | | Default index
148 | |--------------------------------------------------------------------------
149 | |
150 | | This option defines if chunky should start counting the chunks indexes
151 | | from 0 (ChunkySettings::INDEX_ZERO) or 1 (ChunkySettings::INDEX_ONE). You
152 | | can override this feature with any number, but the indexes must always
153 | | be index + n or the integrity check for the chunks folder will throw an
154 | | exception.
155 | |
156 | */
157 |
158 | 'index' => \Jobtech\LaravelChunky\ChunkySettings::INDEX_ZERO,
159 |
160 | /*
161 | |--------------------------------------------------------------------------
162 | | Additional options
163 | |--------------------------------------------------------------------------
164 | |
165 | | This option defines the additional settings that chunky should pass to
166 | | the `storeAs` method while saving chunks or the merged file. This can be
167 | | useful, for example, when storing public files in S3 storage.
168 | |
169 | */
170 |
171 | 'options' => [
172 | 'chunks' => [
173 | // 'visibility' => 'public'
174 | ],
175 | ],
176 | ];
177 | ```
178 |
179 | #### Chunks methods
180 |
181 | If you want to manually save a chunk from a request you can use the `addChunk` method. It gets in input the uploaded file, the chunk index and, optionally, a folder name. If no folder is passed, the chunks anyway will be stored into a chunk's root subfolder. This folder will be named as the slug of the uploaded file basename.
182 |
183 | This method will return a `Jobtech\LaravelChunky\Chunk` object, that implements the `Illuminate\Contracts\Support\Responsable` contract so you can easily return a JSON response. If the requests has the `Accept application/json` header, the object will be automatically transformed into a `Jobtech\LaravelChunky\Http\Resources\ChunkResource` object. Furthermore, every time a chunk is added a `Jobtech\LaravelChunky\Events\ChunkAdded` event is fired.
184 |
185 | ```php
186 | // ...
187 | $chunk = Chunky::addChunk(
188 | $request->file('your-file-key'),
189 | $request->input('your-index-key'),
190 | 'folder-is-optional'
191 | );
192 |
193 | return $chunk->hideFileInfo()->toResponse();
194 | // This will return
195 | // {
196 | // "data": {
197 | // "name": "my-very-big-file.ext",
198 | // "extension": "ext",
199 | // "index": 0,
200 | // "last": false
201 | // }
202 | // }
203 |
204 | return $chunk->showFileInfo()->toResponse();
205 | // This will return
206 | // {
207 | // "data": {
208 | // "name": "my-very-big-file.ext",
209 | // "extension": "ext",
210 | // "index": 0,
211 | // "last": false,
212 | // "file": "/path/to/my-very-big-file.ext",
213 | // "path": "/path/to"
214 | // }
215 | // }
216 | ```
217 |
218 | Everytime a chunk is added, a `Jobtech\LaravelChunky\Events\ChunkDeleted` event is fired.
219 |
220 | > If you're trying to add a chunk that violates the integrity of the chunks folder an exception will be thrown.
221 | > for example:
222 | >
223 | >```php
224 | > |- chunks
225 | > |- folder
226 | > |- 0_chunk.ext
227 | > |- 1_chunk.ext
228 | > |- 2_chunk.ext
229 | >
230 | > Chunk::addChunk($chunk, 4);
231 | >```
232 | > This will throw a `Jobtech\LaravelChunky\Exceptions\ChunksIntegrityException`
233 |
234 | If you're using, for example, Dropzone you can block the upload action, in that case you will delete the currently uploaded chunks:
235 |
236 |
237 | ```php
238 | Chunky::deleteChunks('chunks-folder');
239 | ```
240 | Everytime a chunk is deleted, a `Jobtech\LaravelChunky\Events\ChunkDeleted` event is fired.
241 |
242 | ---
243 |
244 | The package include a method that, given the chunks folder, will return a sorted collection. Each item contains the relative chunk's path and index.
245 |
246 | ```php
247 | $chunks = Chunky::listChunks('chunks-folder-name');
248 |
249 | foreach($chunks as $chunk) {
250 | /** @var \Jobtech\LaravelChunky\Chunk $chunk */
251 | print_r($chunk->toArray());
252 | }
253 |
254 | // [
255 | // 'index' => 0,
256 | // 'path' => '/path/to/chunks-folder-name/0_chunk.ext',
257 | // [...]
258 | // ],
259 | // [
260 | // 'index' => 1,
261 | // 'path' => '/path/to/chunks-folder-name/1_chunk.ext',
262 | // [...]
263 | // ],
264 | // [
265 | // 'index' => 2,
266 | // 'path' => '/path/to/chunks-folder-name/2_chunk.ext',
267 | // [...]
268 | // ],
269 | // ...
270 | ```
271 |
272 | #### Chunks request
273 |
274 | If you want to automate the chunks upload and merge (requires the value `true` for the `auto_merge` config key), you can use the `Jobtech\LaravelChunky\Http\Requests\AddChunkRequest` class. For more informations about form request please have a look at the [official documentation](https://laravel.com/docs/6.x/validation#form-request-validation).
275 |
276 | Include the form request in your method and simply call the `handle` method of the Chunky facade. The package will automatically handle the upload and return a `Jobtech\LaravelChunky\Chunk` object.
277 |
278 | ```php
279 | // Example with `auto_merge = false`
280 |
281 | use Jobtech\LaravelChunky\Http\Requests\AddChunkRequest;
282 |
283 | class UploadController extends Controller {
284 | // [...]
285 |
286 | public function chunkUpload(AddChunkRequest $request) {
287 | $chunk = Chunky::handle($request, 'folder-is-optional');
288 |
289 | if($chunk->isLast()) {
290 | // See section below for merge or
291 | // implement your own logic
292 | }
293 |
294 | return $chunk->toResponse();
295 | }
296 | }
297 | ```
298 |
299 | ### Merge handler
300 |
301 | If you need to merge chunks into a single file, you can call the `merge` function that will use the configured merge handler to concatenate all the uploaded chunks into a single file.
302 |
303 | ```
304 | public function chunkUpload(AddChunkRequest $request) {
305 | $chunk = Chunky::handle($request, 'folder-is-optional');
306 |
307 | if($chunk->isLast()) {
308 | Chunky::merge('upload-folder', 'your/merge/file.ext');
309 | }
310 |
311 | return $chunk->toResponse();
312 | }
313 | ```
314 |
315 | Once the last chunk has been uploaded and the `auto_merge` config key has `true` value, the package will automatically merge the chunks. A `Jobtech\LaravelChunky\Jobs\MergeChunks` job will be dispatched on the given connection and queue if these options have been set.
316 |
317 | ```php
318 | // config/chunky.php
319 | [
320 | // [...]
321 |
322 | /*
323 | |--------------------------------------------------------------------------
324 | | Merge settings
325 | |--------------------------------------------------------------------------
326 | |
327 | | This option defines the merge handler that should be used to perform the
328 | | chunks merge once the upload is completed (automagically depending on
329 | | `auto_merge` config value.
330 | |
331 | | `connection` and `queue` keys define which queue and which connection
332 | | should be used for the merge job. If connection is null, a synchronous
333 | | job will be dispatched
334 | */
335 |
336 | 'merge' => [
337 | 'handler' => \Jobtech\LaravelChunky\Handlers\MergeHandler::class,
338 |
339 | 'connection' => env('CHUNKY_MERGE_CONNECTION', 'sync'),
340 |
341 | 'queue' => env('CHUNKY_MERGE_QUEUE'),
342 | ],
343 | ];
344 | ```
345 |
346 | You can manually dispatch the job (or if you're not using the `Jobtech\LaravelChunky\Http\Requests\AddChunkRequest` form request create your own):
347 |
348 | ```php
349 | use Jobtech\LaravelChunky\Http\Requests\AddChunkRequest;
350 | use Jobtech\LaravelChunky\Jobs\MergeChunks;
351 |
352 | class UploadController extends Controller {
353 | // [...]
354 |
355 | public function chunkUpload(AddChunkRequest $request) {
356 | $chunk = Chunky::handle($request, 'folder-is-optional');
357 |
358 | if($chunk->isLast()) {
359 | $job = new MergeChunks($request, 'chunks-folder', 'destination/path/to/merge.ext');
360 |
361 | dispatch(
362 | $job->onConnection('your-connection')
363 | ->onQueue('your-queue')
364 | );
365 | }
366 |
367 | return $chunk->toResponse();
368 | }
369 | }
370 | ```
371 |
372 | Once the job is completed, a `Jobtech\LaravelChunky\Events\ChunksMerged` event is fired as well as once the merge file is moved to destination a `Jobtech\LaravelChunky\Events\MergeAdded` event is fired.
373 |
374 | #### Custom handler
375 |
376 | If you want to integrate your own handler, remember to implement the `Jobtech\LaravelChunky\Contracts\MergeHandler` contract (or at least implement the same methods) in your class, and update the related `handler` configuration option:
377 |
378 | ```php
379 | use Jobtech\LaravelChunky\Contracts\MergeHandler;
380 |
381 | class MyHandler implements MergeHandler {
382 | private ChunkyManager $manager;
383 |
384 | /**
385 | * @param \Jobtech\LaravelChunky\Contracts\ChunkyManager $manager
386 | * @return \Jobtech\LaravelChunky\Handlers\MergeHandler
387 | */
388 | public function setManager(ChunkyManager $manager): MergeHandler
389 | {
390 | $this->manager = $manager;
391 |
392 | return $this;
393 | }
394 |
395 | /**
396 | * @return \Jobtech\LaravelChunky\Contracts\ChunkyManager
397 | * @throws \Illuminate\Contracts\Container\BindingResolutionException
398 | */
399 | public function manager(): ChunkyManager
400 | {
401 | return $this->manager;
402 | }
403 |
404 | /**
405 | * @param \Jobtech\LaravelChunky\Http\Requests\AddChunkRequest $request
406 | * @param string $folder
407 | *
408 | * @return \Illuminate\Foundation\Bus\PendingDispatch|string
409 | */
410 | public function dispatchMerge(AddChunkRequest $request, string $folder)
411 | {
412 | // Your logic here
413 | }
414 |
415 | /**
416 | * @param string $chunks_folder
417 | * @param string $merge_destination
418 | *
419 | * @return string
420 | */
421 | public function merge(string $chunks_folder, string $merge_destination): string
422 | {
423 | // Your logic here
424 | }
425 |
426 | /**
427 | * @return \Jobtech\LaravelChunky\Contracts\MergeHandler
428 | */
429 | public static function instance(): MergeHandler
430 | {
431 | return new static();
432 | }
433 | }
434 | ```
435 |
436 | ## Testing
437 |
438 | You can run the tests with PHP unit:
439 |
440 | ```sh
441 | $ vendor/bin/phpunit
442 | ```
443 |
444 | If you want to set custom environment variable, you can add a `.env` file for custom disks, queue or whatever you need. Tests anyway set a temporary local disk by default.
445 |
446 | ```
447 | CHUNKY_CHUNK_DISK=s3
448 | CHUNKY_MERGE_DISK=public
449 | CHUNKY_AUTO_MERGE=false
450 | CHUNKY_MERGE_CONNECTION=redis
451 | CHUNKY_MERGE_QUEUE=my-custom-queue
452 | ```
453 |
454 | ## Roadmap
455 |
456 | See the [open issues](https://github.com/jobtech-dev/laravel-chunky/issues) for a list of proposed features (and known issues).
457 |
458 | We're working on:
459 |
460 | * Integrate frontend chunk upload (Not sure if necessary... there are so many packages that does it).
461 | * Custom concatenation, at the moment we're using a third party package.
462 | * Better tests.
463 | * Laravel 5.5+ compatibility.
464 |
465 | ## Changelog
466 |
467 | Please see [CHANGELOG.md](https://github.com/jobtech-dev/laravel-chunky/blob/master/CHANGELOG.md) for more information what has changed recently.
468 |
469 | ## Contributing
470 |
471 | Please see [CONTRIBUTING.md](https://github.com/jobtech-dev/laravel-chunky/blob/master/CONTRIBUTING.md) for more details.
472 |
473 | ## License
474 |
475 | Distributed under the MIT License. See [LICENSE](https://github.com/jobtech-dev/laravel-chunky/blob/master/LICENSE) for more information.
476 |
477 | ## Contact
478 |
479 | Jobtech dev team - [dev@jobtech.it](mailto:dev@jobtech.it)
480 |
481 | ## Credits
482 |
483 | Laravel Chunky is a Laravel package made with :heart: by the JT nerds.
484 |
485 | Thanks to:
486 |
487 | * Filippo Galante ([ilGala](https://github.com/ilgala))
488 | * [All contributors](https://github.com/jobtech-dev/laravel-chunky/graphs/contributors)
489 |
490 | We've used these packages for the chunks concatenation:
491 |
492 | * Keven Godet - [Flysystem concatenate](https://github.com/kevengodet/flysystem-concatenate)
493 |
494 | And this repository for the readme boilerplate:
495 |
496 | * Othneil Drew - [Best-README-Template](https://github.com/othneildrew/Best-README-Template)
497 |
--------------------------------------------------------------------------------
/src/Chunk.php:
--------------------------------------------------------------------------------
1 | index = $index;
38 | $this->path = $path;
39 | $this->disk = $disk;
40 | $this->last = $last;
41 | }
42 |
43 | /**
44 | * @return int
45 | */
46 | public function getIndex(): int
47 | {
48 | return $this->index;
49 | }
50 |
51 | /**
52 | * @return string
53 | */
54 | public function getPath(): string
55 | {
56 | if ($this->path instanceof File) {
57 | return $this->path->getRealPath();
58 | }
59 |
60 | return $this->path;
61 | }
62 |
63 | /**
64 | * @return mixed
65 | */
66 | public function getOriginalPath()
67 | {
68 | return $this->path;
69 | }
70 |
71 | /**
72 | * @param $path
73 | */
74 | public function setPath($path)
75 | {
76 | $this->path = $path;
77 | }
78 |
79 | /**
80 | * @param string|null $suffix
81 | *
82 | * @return string
83 | */
84 | public function getFilename($suffix = null): string
85 | {
86 | if ($this->path instanceof UploadedFile) {
87 | return basename($this->path->getClientOriginalName(), $suffix);
88 | }
89 |
90 | if ($this->path instanceof File) {
91 | return $this->path->getBasename($suffix);
92 | }
93 |
94 | return basename($this->path, $suffix);
95 | }
96 |
97 | /**
98 | * @return string
99 | */
100 | public function getName(): string
101 | {
102 | return pathinfo(
103 | $this->getFilename($this->getExtension()),
104 | PATHINFO_FILENAME
105 | );
106 | }
107 |
108 | /**
109 | * @return string|string[]
110 | */
111 | public function getExtension()
112 | {
113 | return pathinfo($this->getFilename(), PATHINFO_EXTENSION);
114 | }
115 |
116 | /**
117 | * @return string
118 | */
119 | public function getSlug(): string
120 | {
121 | return $this->index.'_'.Str::slug($this->getName()).'.'.$this->getExtension();
122 | }
123 |
124 | /**
125 | * Retrieve the chunk file disk.
126 | *
127 | * @return string|null
128 | */
129 | public function getDisk(): ?string
130 | {
131 | return $this->disk;
132 | }
133 |
134 | /**
135 | * Set the chunk file disk.
136 | *
137 | * @param string|null $disk
138 | */
139 | public function setDisk($disk = null)
140 | {
141 | $this->disk = $disk;
142 | }
143 |
144 | /**
145 | * @return bool
146 | */
147 | public function isLast(): bool
148 | {
149 | return $this->last;
150 | }
151 |
152 | /**
153 | * @param bool $last
154 | */
155 | public function setLast(bool $last): void
156 | {
157 | $this->last = $last;
158 | }
159 |
160 | /**
161 | * If this method is called, when a chunk is turned to array, the file path and real path
162 | * will be omitted.
163 | *
164 | * @return $this
165 | */
166 | public function hideFileInfo()
167 | {
168 | $this->show_file_info = false;
169 |
170 | return $this;
171 | }
172 |
173 | /**
174 | * If this method is called, when a chunk is turned to array, the file path and real path
175 | * will be included.
176 | *
177 | * @return $this
178 | */
179 | public function showFileInfo()
180 | {
181 | $this->show_file_info = true;
182 |
183 | return $this;
184 | }
185 |
186 | /**
187 | * {@inheritdoc}
188 | */
189 | public function toArray(): array
190 | {
191 | $extension = $this->getExtension();
192 |
193 | $data = [
194 | 'name' => $this->getName(),
195 | 'extension' => $extension,
196 | 'index' => $this->getIndex(),
197 | 'last' => $this->isLast(),
198 | ];
199 |
200 | if ($this->show_file_info) {
201 | $data['file'] = $this->getFilename();
202 | $data['path'] = $this->getPath();
203 | }
204 |
205 | return $data;
206 | }
207 |
208 | /**
209 | * {@inheritdoc}
210 | */
211 | public function toJson($options = 0): string
212 | {
213 | return json_encode($this->toArray(), $options);
214 | }
215 |
216 | /**
217 | * {@inheritdoc}
218 | */
219 | public function toResponse($request)
220 | {
221 | return $this->toResource()
222 | ->toResponse($request);
223 | }
224 |
225 | /**
226 | * Transforms the current model into a json resource.
227 | */
228 | public function toResource(): JsonResource
229 | {
230 | /** @var JsonResource $resource */
231 | $resource = config('chunky.resource', ChunkResource::class);
232 |
233 | return new $resource($this);
234 | }
235 |
236 | /**
237 | * @param string|\Symfony\Component\HttpFoundation\File\File $file
238 | * @param int $index
239 | * @param array $options
240 | *
241 | * @return Chunk
242 | */
243 | public static function create($file, int $index, $options = [])
244 | {
245 | return new static($index, $file, Arr::pull($options, 'disk'));
246 | }
247 | }
248 |
--------------------------------------------------------------------------------
/src/ChunkyManager.php:
--------------------------------------------------------------------------------
1 | settings = $settings;
38 |
39 | $this->setChunksFilesystem($settings->chunksDisk(), $settings->chunksFolder());
40 | $this->setMergeFilesystem($settings->mergeDisk(), $settings->mergeFolder());
41 | }
42 |
43 | /**
44 | * {@inheritdoc}
45 | */
46 | public function settings(): ChunkySettings
47 | {
48 | return $this->settings;
49 | }
50 |
51 | /**
52 | * {@inheritdoc}
53 | */
54 | public function setChunksFilesystem(string $disk, string $folder): ChunkyManager
55 | {
56 | $this->chunksFilesystem = ChunksFilesystem::instance(
57 | compact('disk', 'folder')
58 | );
59 |
60 | return $this;
61 | }
62 |
63 | /**
64 | * {@inheritdoc}
65 | */
66 | public function chunksFilesystem(): ChunksFilesystem
67 | {
68 | return $this->chunksFilesystem;
69 | }
70 |
71 | /**
72 | * {@inheritdoc}
73 | */
74 | public function setMergeFilesystem(string $disk, string $folder): ChunkyManager
75 | {
76 | $this->mergeFilesystem = MergeFilesystem::instance(
77 | compact('disk', 'folder')
78 | );
79 |
80 | return $this;
81 | }
82 |
83 | /**
84 | * {@inheritdoc}
85 | */
86 | public function mergeFilesystem(): MergeFilesystem
87 | {
88 | return $this->mergeFilesystem;
89 | }
90 |
91 | /**
92 | * {@inheritdoc}
93 | *
94 | * @codeCoverageIgnore
95 | */
96 | public function mergeHandler(): MergeHandler
97 | {
98 | return $this->settings->mergeHandler()->setManager($this);
99 | }
100 |
101 | /**
102 | * {@inheritdoc}
103 | */
104 | public function chunksDisk(): ?string
105 | {
106 | return $this->settings
107 | ->chunksDisk();
108 | }
109 |
110 | /**
111 | * {@inheritdoc}
112 | */
113 | public function mergeDisk(): ?string
114 | {
115 | return $this->settings
116 | ->mergeDisk();
117 | }
118 |
119 | /**
120 | * {@inheritdoc}
121 | */
122 | public function chunksFolder(): string
123 | {
124 | return $this->settings
125 | ->chunksFolder();
126 | }
127 |
128 | /**
129 | * {@inheritdoc}
130 | */
131 | public function mergeFolder(): string
132 | {
133 | return $this->settings
134 | ->mergeFolder();
135 | }
136 |
137 | /**
138 | * {@inheritdoc}
139 | */
140 | public function chunksOptions(): array
141 | {
142 | return array_merge([
143 | 'disk' => $this->chunksDisk(),
144 | ], $this->settings->additionalChunksOptions());
145 | }
146 |
147 | /**
148 | * {@inheritdoc}
149 | */
150 | public function mergeOptions(): array
151 | {
152 | return $this->settings->additionalMergeOptions();
153 | }
154 |
155 | /**
156 | * {@inheritdoc}
157 | */
158 | public function listChunks(?string $folder = null): Collection
159 | {
160 | return $this->chunksFilesystem()->listChunks($folder);
161 | }
162 |
163 | /**
164 | * {@inheritdoc}
165 | */
166 | public function readChunk($chunk)
167 | {
168 | if ($chunk instanceof Chunk) {
169 | $chunk = $chunk->getPath();
170 | }
171 |
172 | return $this->chunksFilesystem()
173 | ->readChunk($chunk);
174 | }
175 |
176 | /**
177 | * {@inheritdoc}
178 | */
179 | public function addChunk(UploadedFile $file, int $index, string $folder): Chunk
180 | {
181 | // Check integrity
182 | if (!$this->checkChunks($folder, $index)) {
183 | throw new ChunksIntegrityException("Uploaded chunk with index {$index} violates the integrity");
184 | }
185 |
186 | $chunk = Chunk::create($file, $index, $this->chunksOptions());
187 |
188 | // Store chunk
189 | return $this->chunksFilesystem()
190 | ->store($chunk, $folder, $this->chunksOptions());
191 | }
192 |
193 | /**
194 | * {@inheritdoc}
195 | */
196 | public function deleteChunks(string $folder): bool
197 | {
198 | return $this->chunksFilesystem()->delete($folder);
199 | }
200 |
201 | /**
202 | * @param string $folder
203 | *
204 | * @return bool
205 | */
206 | public function isValidChunksFolder(string $folder): bool
207 | {
208 | return $this->chunksFilesystem()->exists($folder);
209 | }
210 |
211 | /**
212 | * {@inheritdoc}
213 | */
214 | public function checkChunks(string $folder, int $index): bool
215 | {
216 | $default = $this->settings->defaultIndex();
217 |
218 | if (!$this->chunksFilesystem()->exists($folder) && $index != $default) {
219 | return false;
220 | }
221 |
222 | if ($this->chunksFilesystem()->exists($folder)) {
223 | if ($default != ChunkySettings::INDEX_ZERO) {
224 | $index -= $default;
225 | }
226 |
227 | return $this->chunksFilesystem()->chunksCount($folder) == $index;
228 | }
229 |
230 | if ($index == $default) {
231 | if (!$this->chunksFilesystem()->makeDirectory($folder)) {
232 | throw new ChunksIntegrityException("Cannot create chunks folder {$folder}");
233 | }
234 | }
235 |
236 | return true;
237 | }
238 |
239 | /**
240 | * {@inheritdoc}
241 | */
242 | public function checkChunksIntegrity(string $folder, int $chunk_size, int $total_size): bool
243 | {
244 | $total = 0;
245 | $chunks = $this->listChunks($folder);
246 |
247 | foreach ($chunks as $chunk) {
248 | $size = $this->chunksFilesystem()->chunkSize($chunk->getPath());
249 |
250 | if ($size < $chunk_size && !$chunk->isLast()) {
251 | return false;
252 | }
253 |
254 | $total += $size;
255 | }
256 |
257 | return $total >= $total_size;
258 | }
259 |
260 | /**
261 | * {@inheritdoc}
262 | */
263 | public function handle(AddChunkRequest $request, ?string $folder = null): Chunk
264 | {
265 | // Store chunk
266 | $folder = $this->checkFolder($request, $folder);
267 | $chunk = $this->addChunk(
268 | $request->fileInput(),
269 | $request->indexInput(),
270 | $folder
271 | );
272 |
273 | $chunk->setLast(
274 | $this->isLastIndex($request)
275 | );
276 |
277 | // Check merge
278 | if ($chunk->isLast() && $this->settings->autoMerge()) {
279 | $this->mergeHandler()->dispatchMerge($request, $folder);
280 | }
281 |
282 | return $chunk;
283 | }
284 |
285 | /**
286 | * {@inheritdoc}
287 | */
288 | public function merge(string $chunks_folder, string $merge_path): string
289 | {
290 | return $this->mergeHandler()->merge($chunks_folder, $merge_path);
291 | }
292 | }
293 |
--------------------------------------------------------------------------------
/src/ChunkyServiceProvider.php:
--------------------------------------------------------------------------------
1 | setupConfig();
24 | }
25 |
26 | /**
27 | * Register any application services.
28 | */
29 | public function register()
30 | {
31 | $this->registerBindings();
32 | $this->registerCommands();
33 |
34 | $this->app->alias(ChunkyManagerContract::class, 'chunky');
35 | }
36 |
37 | protected function setupConfig()
38 | {
39 | $source = realpath(__DIR__.'/../config/chunky.php');
40 |
41 | if ($this->app instanceof LaravelApplication) {
42 | $this->publishes([
43 | $source => config_path('chunky.php'),
44 | ], 'config');
45 | } elseif ($this->app instanceof LumenApplication) {
46 | $this->app->configure('chunky');
47 | }
48 |
49 | $this->mergeConfigFrom($source, 'chunky');
50 | }
51 |
52 | private function registerCommands()
53 | {
54 | $this->app->bind('command.chunky:clear', ClearChunks::class);
55 |
56 | $this->commands([
57 | 'command.chunky:clear',
58 | ]);
59 | }
60 |
61 | private function registerBindings()
62 | {
63 | $this->app->singleton(TempFilesystem::class, function () {
64 | $config = $this->app->make('config');
65 | $filesystem = new TempFilesystem(app()->make(Factory::class));
66 |
67 | $filesystem->disk($config->get('chunky.disks.tmp.disk', $config->get('filesystems.default')));
68 | $filesystem->folder($config->get('chunky.disks.tmp.folder'));
69 |
70 | return $filesystem;
71 | });
72 |
73 | $this->app->bind(ChunksFilesystem::class, ChunksFilesystem::class);
74 | $this->app->bind(MergeFilesystem::class, MergeFilesystem::class);
75 |
76 | $this->app->singleton(ChunkySettings::class, function (Container $app) {
77 | return new ChunkySettings($app->make('config'));
78 | });
79 |
80 | $this->app->singleton(ChunkyManagerContract::class, function (Container $app) {
81 | return new ChunkyManager($app->make(ChunkySettings::class));
82 | });
83 | }
84 | }
85 |
--------------------------------------------------------------------------------
/src/ChunkySettings.php:
--------------------------------------------------------------------------------
1 | config = $config->get('chunky');
27 | }
28 |
29 | /**
30 | * Retrieve the chunky configurations.
31 | *
32 | * @return array
33 | */
34 | public function config(): array
35 | {
36 | return $this->config;
37 | }
38 |
39 | /**
40 | * Retrieve the default chunks disk.
41 | *
42 | * @return string|null
43 | */
44 | public function chunksDisk(): ?string
45 | {
46 | return Arr::get($this->config, 'disks.chunks.disk');
47 | }
48 |
49 | /**
50 | * Retrieve the chunks destination folder.
51 | *
52 | * @return string
53 | */
54 | public function chunksFolder(): string
55 | {
56 | $folder = Arr::get($this->config, 'disks.chunks.folder');
57 |
58 | if ($folder === null) {
59 | return '';
60 | }
61 |
62 | if (!Str::endsWith($folder, '/')) {
63 | $folder .= DIRECTORY_SEPARATOR;
64 | }
65 |
66 | return $folder;
67 | }
68 |
69 | /**
70 | * Retrieve the default merge file disk.
71 | *
72 | * @return string|null
73 | */
74 | public function mergeDisk(): ?string
75 | {
76 | return Arr::get($this->config, 'disks.merge.disk');
77 | }
78 |
79 | /**
80 | * Retrieve the merge file destination folder.
81 | *
82 | * @return string
83 | */
84 | public function mergeFolder(): string
85 | {
86 | $folder = Arr::get($this->config, 'disks.merge.folder');
87 |
88 | if ($folder === null) {
89 | return '';
90 | }
91 |
92 | if (!Str::endsWith($folder, '/')) {
93 | $folder .= DIRECTORY_SEPARATOR;
94 | }
95 |
96 | return $folder;
97 | }
98 |
99 | /**
100 | * Retrieve the default index value for chunks.
101 | *
102 | * @return int
103 | */
104 | public function defaultIndex(): int
105 | {
106 | return Arr::get($this->config, 'index', self::INDEX_ZERO)
107 | ?: self::INDEX_ZERO;
108 | }
109 |
110 | /**
111 | * Retrieve the additional options for chunk store.
112 | *
113 | * @return array
114 | */
115 | public function additionalChunksOptions(): array
116 | {
117 | return Arr::get($this->config, 'options.chunks', []);
118 | }
119 |
120 | /**
121 | * Retrieve the additional options for merge store.
122 | *
123 | * @return array
124 | */
125 | public function additionalMergeOptions(): array
126 | {
127 | return Arr::get($this->config, 'options.merge', []);
128 | }
129 |
130 | /**
131 | * Retrieve the auto merge option.
132 | *
133 | * @return bool
134 | */
135 | public function autoMerge(): bool
136 | {
137 | return Arr::get($this->config, 'auto_merge', false);
138 | }
139 |
140 | /**
141 | * Retrieve the default merge handler.
142 | *
143 | * @return MergeHandler
144 | */
145 | public function mergeHandler(): MergeHandler
146 | {
147 | if ($this->handler === null) {
148 | $handler = Arr::get($this->config, 'merge.handler');
149 |
150 | if (!class_exists($handler)) {
151 | throw new ChunkyException("Undefined handler {$handler}");
152 | }
153 |
154 | $this->handler = $handler::instance();
155 | }
156 |
157 | return $this->handler;
158 | }
159 |
160 | /**
161 | * Retrieve the queue connection for the merge job.
162 | *
163 | * @return string
164 | */
165 | public function connection()
166 | {
167 | return Arr::get($this->config, 'merge.connection');
168 | }
169 |
170 | /**
171 | * Retrieve the queue for the merge job.
172 | *
173 | * @return array|\ArrayAccess|mixed
174 | */
175 | public function queue()
176 | {
177 | return Arr::get($this->config, 'merge.queue');
178 | }
179 | }
180 |
--------------------------------------------------------------------------------
/src/Commands/ClearChunks.php:
--------------------------------------------------------------------------------
1 | manager = $manager;
51 |
52 | if (!$this->confirmToProceed()) {
53 | return;
54 | }
55 |
56 | $root = $this->manager->chunksFolder();
57 | $folder = $this->argument('folder');
58 |
59 | if (!empty($folder)) {
60 | $root .= $folder;
61 | $this->deleteFolder($root);
62 |
63 | return;
64 | }
65 |
66 | $folders = $this->manager->chunksFilesystem()->chunkFolders();
67 | $bar = $this->output->createProgressBar(count($folders));
68 |
69 | $bar->start();
70 |
71 | foreach ($folders as $folder) {
72 | $this->deleteFolder($folder);
73 | }
74 |
75 | $bar->finish();
76 | $this->info('Chunks folders have been deleted!');
77 | }
78 |
79 | /**
80 | * @param string $folder
81 | */
82 | private function deleteFolder(string $folder)
83 | {
84 | if (!$this->manager->deleteChunks($folder)) {
85 | $this->error("An error occurred while deleting folder {$folder}");
86 |
87 | return;
88 | }
89 |
90 | $this->info("folder {$folder} has been deleted!");
91 | }
92 | }
93 |
--------------------------------------------------------------------------------
/src/Concerns/ChunkyRequestHelpers.php:
--------------------------------------------------------------------------------
1 | totalSizeInput();
21 | $chunk_size = $request->chunkSizeInput();
22 |
23 | if ($total_size < $chunk_size) {
24 | // In this case usually it means that there's only a chunk
25 | return 1;
26 | }
27 |
28 | return ceil($total_size / $chunk_size);
29 | }
30 |
31 | /**
32 | * Check if current index refers to the last chunk.
33 | *
34 | * @param AddChunkRequest $request
35 | *
36 | * @return bool
37 | */
38 | public function isLastIndex(AddChunkRequest $request): bool
39 | {
40 | $last_index = $this->lastIndexFrom($request)
41 | + ($this->settings->defaultIndex() - 1);
42 |
43 | return $request->indexInput() == $last_index;
44 | }
45 |
46 | /**
47 | * Check if folder is a valid string, otherwise guess the folder from the
48 | * request file input.
49 | *
50 | * @param AddChunkRequest $request
51 | * @param string|null $folder
52 | *
53 | * @return string
54 | */
55 | protected function checkFolder(AddChunkRequest $request, ?string $folder)
56 | {
57 | $file = $request->fileInput();
58 |
59 | if ($folder !== null) {
60 | return Str::slug($folder);
61 | }
62 |
63 | return $this->chunkFolderNameFor(
64 | str_replace(
65 | $file->guessExtension(),
66 | '',
67 | $file->getClientOriginalName()
68 | )
69 | );
70 | }
71 | }
72 |
--------------------------------------------------------------------------------
/src/Contracts/ChunkyManager.php:
--------------------------------------------------------------------------------
1 | chunk = $chunk;
17 | }
18 | }
19 |
--------------------------------------------------------------------------------
/src/Events/ChunkDeleted.php:
--------------------------------------------------------------------------------
1 | chunk = $chunk;
17 | }
18 | }
19 |
--------------------------------------------------------------------------------
/src/Events/ChunksMerged.php:
--------------------------------------------------------------------------------
1 | path = $path;
16 | }
17 | }
18 |
--------------------------------------------------------------------------------
/src/Events/MergeAdded.php:
--------------------------------------------------------------------------------
1 | path = $path;
16 | }
17 | }
18 |
--------------------------------------------------------------------------------
/src/Exceptions/ChunksIntegrityException.php:
--------------------------------------------------------------------------------
1 | manager = $manager;
43 | $this->temp_filesystem = Container::getInstance()->make(TempFilesystem::class);
44 | }
45 |
46 | public function __call($method, $parameters)
47 | {
48 | if (!method_exists($this, $method)) {
49 | return $this->forwardCallTo($this->manager(), $method, $parameters);
50 | }
51 |
52 | return $this->{$method}(...$parameters);
53 | }
54 |
55 | /**
56 | * {@inheritdoc}
57 | *
58 | * @codeCoverageIgnore
59 | */
60 | public function setManager(ChunkyManager $manager): MergeHandler
61 | {
62 | $this->manager = $manager;
63 |
64 | return $this;
65 | }
66 |
67 | /**
68 | * {@inheritdoc}
69 | *
70 | * @codeCoverageIgnore
71 | */
72 | public function manager(): ChunkyManager
73 | {
74 | if ($this->manager === null) {
75 | $this->manager = Container::getInstance()->make('chunky');
76 | }
77 |
78 | return $this->manager;
79 | }
80 |
81 | /**
82 | * {@inheritdoc}
83 | */
84 | public function dispatchMerge(AddChunkRequest $request, string $folder)
85 | {
86 | $merge_path = $request->fileInput()->getClientOriginalName();
87 | $chunk_size = $request->chunkSizeInput();
88 | $total_size = $request->totalSizeInput();
89 |
90 | if (empty($connection = $this->settings()->connection())) {
91 | if (!$this->checkChunksIntegrity($folder, $chunk_size, $total_size)) {
92 | throw new ChunksIntegrityException('Chunks total file size doesnt match with original file size');
93 | }
94 |
95 | return $this->merge($folder, $merge_path);
96 | }
97 |
98 | return MergeChunks::dispatch(
99 | $folder,
100 | $merge_path,
101 | $chunk_size,
102 | $total_size
103 | )->onConnection($connection)
104 | ->onQueue($this->settings()->queue());
105 | }
106 |
107 | /**
108 | * {@inheritdoc}
109 | */
110 | public function merge(string $chunks_folder, string $merge_path): string
111 | {
112 | // Check chunks folder
113 | if (!$this->isValidChunksFolder($chunks_folder)) {
114 | throw new ChunkyException("Invalid chunks folder {$chunks_folder}");
115 | }
116 |
117 | /** @var resource $origin */
118 | $path = $this->concatenate($chunks_folder, $merge_path);
119 |
120 | // Final check and cleanup
121 | if (!$path) {
122 | throw new ChunkyException('An error occurred while moving merge to destination');
123 | }
124 |
125 | $this->deleteChunks($chunks_folder);
126 |
127 | event(new ChunksMerged($path));
128 |
129 | return $path;
130 | }
131 |
132 | /**
133 | * {@inheritdoc}
134 | */
135 | public static function instance(?ChunkyManager $manager = null): MergeHandlerContract
136 | {
137 | return new static($manager);
138 | }
139 |
140 | /**
141 | * @param string $folder
142 | * @param string $target
143 | *
144 | * @return string
145 | *
146 | * @throws \Illuminate\Contracts\Filesystem\FileExistsException|\Illuminate\Contracts\Filesystem\FileNotFoundException
147 | */
148 | private function concatenate(string $folder, string $target): string
149 | {
150 | if (!$this->chunksFilesystem()->isLocal()) {
151 | return $this->temporaryConcatenate($target, $this->listChunks($folder));
152 | }
153 |
154 | $chunks = $this->listChunks($folder)->map(function (Chunk $item) {
155 | return $item->getPath();
156 | });
157 | $merge = $chunks->first();
158 |
159 | if (!$this->chunksFilesystem()->concatenate($merge, $chunks->toArray())) {
160 | throw new ChunkyException('Unable to concatenate chunks');
161 | }
162 |
163 | return $this->mergeFilesystem()
164 | ->store(
165 | $target,
166 | $this->readChunk($merge),
167 | $this->mergeOptions()
168 | );
169 | }
170 |
171 | /**
172 | * @param string $target
173 | * @param Collection $chunks
174 | *
175 | * @return string
176 | *
177 | * @throws FileExistsException
178 | * @throws FileNotFoundException
179 | */
180 | private function temporaryConcatenate(string $target, Collection $chunks)
181 | {
182 | $stream = new AppendStream();
183 | $chunks->each(function (Chunk $chunk) use ($stream) {
184 | $path = $this->temp_filesystem->store('tmp-'.$chunk->getFilename(), $this->chunksFilesystem()->readChunk($chunk->getPath()));
185 | $stream->append($this->temp_filesystem->readFile($path));
186 | });
187 |
188 | $tmp_merge = $this->temp_filesystem->store($target, $stream->getResource());
189 | $path = $this->mergeFilesystem()->store($target, $this->temp_filesystem->readFile($tmp_merge), $this->mergeOptions());
190 |
191 | $this->temp_filesystem->clean();
192 |
193 | return $path;
194 | }
195 | }
196 |
--------------------------------------------------------------------------------
/src/Http/Requests/AddChunkRequest.php:
--------------------------------------------------------------------------------
1 | $rules,
32 | ];
33 | }
34 |
35 | /**
36 | * Retrieve file rules.
37 | *
38 | * @return array
39 | */
40 | public function fileRules(): array
41 | {
42 | $key = config('chunky.validation.file.key');
43 | $rules = array_unique(
44 | array_merge([
45 | 'required', 'file',
46 | ], config('chunky.validation.file.rules', []))
47 | );
48 |
49 | if (empty($key)) {
50 | throw new ChunkyException('File key cannot be null');
51 | }
52 |
53 | return [
54 | $key => $rules,
55 | ];
56 | }
57 |
58 | /**
59 | * Retrieve chunk file size rules.
60 | *
61 | * @return array
62 | */
63 | public function chunkSizeRules(): array
64 | {
65 | $key = config('chunky.validation.chunkSize.key');
66 | $rules = array_unique(
67 | array_merge([
68 | 'required', 'integer', 'min:1',
69 | ], config('chunky.validation.chunkSize.rules', []))
70 | );
71 |
72 | if (empty($key)) {
73 | throw new ChunkyException('Chunk file size key cannot be null');
74 | }
75 |
76 | return [
77 | $key => $rules,
78 | ];
79 | }
80 |
81 | /**
82 | * Retrieve total file size rules.
83 | *
84 | * @return array
85 | */
86 | public function totalSizeRules(): array
87 | {
88 | $key = config('chunky.validation.totalSize.key');
89 | $rules = array_unique(
90 | array_merge([
91 | 'required', 'integer', 'min:1',
92 | ], config('chunky.validation.totalSize.rules', []))
93 | );
94 |
95 | if (empty($key)) {
96 | throw new ChunkyException('Total file size key cannot be null');
97 | }
98 |
99 | return [
100 | $key => $rules,
101 | ];
102 | }
103 |
104 | public function additionalRules(): array
105 | {
106 | $rules = [];
107 |
108 | foreach (config('chunky.validation') as $input => $config) {
109 | if (
110 | !in_array($input, ['index', 'file', 'chunkSize', 'totalSize'])
111 | && Arr::has($config, 'key')
112 | && Arr::has($config, 'rules')
113 | ) {
114 | $rules[$config['key']] = Arr::get($config, 'rules', []);
115 | }
116 | }
117 |
118 | return $rules;
119 | }
120 |
121 | /**
122 | * Retrieve index input.
123 | *
124 | * @return int
125 | */
126 | public function indexInput(): int
127 | {
128 | $key = config('chunky.validation.index.key');
129 |
130 | if (empty($key)) {
131 | throw new ChunkyException('Index key cannot be null');
132 | }
133 |
134 | return $this->input($key);
135 | }
136 |
137 | /**
138 | * Retrieve file input.
139 | *
140 | * @return UploadedFile
141 | */
142 | public function fileInput(): UploadedFile
143 | {
144 | $key = config('chunky.validation.file.key');
145 |
146 | if (empty($key)) {
147 | throw new ChunkyException('File key cannot be null');
148 | }
149 |
150 | return $this->file($key);
151 | }
152 |
153 | /**
154 | * Retrieve chunk file size input.
155 | *
156 | * @return int
157 | */
158 | public function chunkSizeInput(): int
159 | {
160 | $key = config('chunky.validation.chunkSize.key');
161 |
162 | if (empty($key)) {
163 | throw new ChunkyException('Chunk file size key cannot be null');
164 | }
165 |
166 | return $this->input($key);
167 | }
168 |
169 | /**
170 | * Retrieve total file size input.
171 | *
172 | * @return int
173 | */
174 | public function totalSizeInput(): int
175 | {
176 | $key = config('chunky.validation.totalSize.key');
177 |
178 | if (empty($key)) {
179 | throw new ChunkyException('Total file size key cannot be null');
180 | }
181 |
182 | return $this->input($key);
183 | }
184 |
185 | /**
186 | * Get the validation rules that apply to the request.
187 | *
188 | * @return array
189 | */
190 | public function rules()
191 | {
192 | return array_merge(
193 | $this->additionalRules(),
194 | $this->indexRules(),
195 | $this->fileRules(),
196 | $this->chunkSizeRules(),
197 | $this->totalSizeRules()
198 | );
199 | }
200 | }
201 |
--------------------------------------------------------------------------------
/src/Http/Resources/ChunkResource.php:
--------------------------------------------------------------------------------
1 | resource->toArray();
20 | }
21 | }
22 |
--------------------------------------------------------------------------------
/src/Jobs/MergeChunks.php:
--------------------------------------------------------------------------------
1 | chunks_folder = $chunks_folder;
39 | $this->merge_path = $merge_path;
40 | $this->chunk_size = $chunk_size;
41 | $this->total_size = $total_size;
42 | }
43 |
44 | /**
45 | * Execute the job.
46 | */
47 | public function handle()
48 | {
49 | if (!Chunky::checkChunksIntegrity($this->chunks_folder, $this->chunk_size, $this->total_size)) {
50 | throw new ChunksIntegrityException('Chunks total file size doesnt match with original file size');
51 | }
52 |
53 | Chunky::merge($this->chunks_folder, $this->merge_path);
54 | }
55 | }
56 |
--------------------------------------------------------------------------------
/src/Support/ChunksFilesystem.php:
--------------------------------------------------------------------------------
1 | path($folder);
29 | $files = $this->list($folder);
30 |
31 | return collect($files)
32 | ->map(function ($path) use ($folder, $files) {
33 | $filename = str_replace($folder.DIRECTORY_SEPARATOR, '', $path);
34 | $exploded_name = explode('_', $filename);
35 |
36 | $index = array_shift($exploded_name);
37 | $last = count($files) - 1 == $index;
38 |
39 | return new Chunk((int) $index, $path, $this->disk(), $last);
40 | })->sortBy(function (Chunk $chunk) {
41 | return $chunk->getIndex();
42 | })->values();
43 | }
44 |
45 | /**
46 | * @return array
47 | *
48 | * @throws FilesystemException
49 | */
50 | public function chunkFolders(): array
51 | {
52 | return $this->filesystem()->disk($this->disk())->listContents($this->folder(), false)
53 | ->filter(fn (StorageAttributes $attributes) => $attributes->isDir())
54 | ->sortByPath()
55 | ->map(fn (StorageAttributes $attributes) => $attributes->path())
56 | ->toArray();
57 | }
58 |
59 | /**
60 | * @param string $folder
61 | *
62 | * @return int
63 | */
64 | public function chunksCount(string $folder): int
65 | {
66 | return count($this->list($this->path($folder)));
67 | }
68 |
69 | /**
70 | * @param string $path
71 | *
72 | * @return int
73 | */
74 | public function chunkSize(string $path): int
75 | {
76 | return $this->filesystem()->disk($this->disk())->size($this->path($path));
77 | }
78 |
79 | /**
80 | * @param string $path
81 | *
82 | * @return resource|null
83 | */
84 | public function readChunk(string $path)
85 | {
86 | return $this->filesystem()->disk($this->disk())->readStream($this->path($path));
87 | }
88 |
89 | /**
90 | * @param Chunk $chunk
91 | * @param string $folder
92 | * @param array $options
93 | *
94 | * @return Chunk
95 | */
96 | public function store(Chunk $chunk, string $folder, $options = []): Chunk
97 | {
98 | if (!$chunk->getOriginalPath() instanceof File) {
99 | throw new ChunkyException('Path must be a file');
100 | }
101 |
102 | // Build destination
103 | $suffix = Str::endsWith($folder, DIRECTORY_SEPARATOR) ? '' : DIRECTORY_SEPARATOR;
104 | $destination = $this->path($folder.$suffix.$chunk->getSlug());
105 |
106 | // Copy file
107 | $file = fopen($chunk->getPath(), 'r');
108 |
109 | Arr::pull($options, 'disk');
110 | $this->filesystem()->disk($this->disk())->put($destination, $file, $options);
111 | fclose($file);
112 |
113 | // Return chunk
114 | $chunk->setPath($destination);
115 | event(new ChunkAdded($chunk));
116 |
117 | return $chunk;
118 | }
119 |
120 | /**
121 | * Delete all chunks and, once empty, delete the folder.
122 | *
123 | * @param Chunk $chunk
124 | *
125 | * @return bool
126 | */
127 | public function deleteChunk(Chunk $chunk): bool
128 | {
129 | if (!$this->filesystem()->disk($this->disk())->exists($chunk->getPath())) {
130 | return true;
131 | }
132 |
133 | $deleted = $this->filesystem()->disk($this->disk())->delete($chunk->getPath());
134 |
135 | if ($deleted) {
136 | event(new ChunkDeleted($chunk));
137 | }
138 |
139 | return $deleted;
140 | }
141 |
142 | /**
143 | * Delete all chunks and, once empty, delete the folder.
144 | *
145 | * @param string $folder
146 | *
147 | * @return bool
148 | */
149 | public function delete(string $folder): bool
150 | {
151 | $folder = $this->path($folder);
152 |
153 | if (!$this->filesystem()->disk($this->disk())->exists($folder)) {
154 | return true;
155 | }
156 |
157 | foreach ($this->listChunks($folder) as $chunk) {
158 | $this->deleteChunk($chunk);
159 | }
160 |
161 | return $this->filesystem()->disk($this->disk())
162 | ->deleteDirectory($folder);
163 | }
164 |
165 | /**
166 | * Concatenate all chunks into final merge.
167 | *
168 | * @param string $chunk
169 | * @param array $chunks
170 | *
171 | * @return bool
172 | *
173 | * @throws FileNotFoundException|FilesystemException
174 | */
175 | public function concatenate(string $chunk, array $chunks): bool
176 | {
177 | foreach ($chunks as $path) {
178 | if (!$this->filesystem()->disk($this->disk())->has($path)) {
179 | throw new FileNotFoundException($path);
180 | }
181 | }
182 |
183 | $overwrite = in_array($chunk, $chunks, true);
184 | if ($overwrite) {
185 | $this->filesystem()->disk($this->disk())->move($chunk, $chunkBackupPath = $chunk.'.backup');
186 | $key = array_search($chunk, $chunks, true);
187 | $chunks[$key] = $chunkBackupPath;
188 | }
189 |
190 | $stream = new AppendStream();
191 | foreach ($chunks as $fragment) {
192 | $stream->append($this->filesystem()->disk($this->disk())->readStream($fragment));
193 | }
194 | $this->filesystem()->disk($this->disk())->writeStream($chunk, $resource = $stream->getResource());
195 |
196 | if ($overwrite) {
197 | $this->filesystem()->disk($this->disk())->delete($chunkBackupPath);
198 | }
199 |
200 | return true;
201 | }
202 | }
203 |
--------------------------------------------------------------------------------
/src/Support/Filesystem.php:
--------------------------------------------------------------------------------
1 | filesystem = $filesystem;
30 | }
31 |
32 | public function __call($method, $parameters)
33 | {
34 | if (!method_exists($this, $method)) {
35 | return $this->forwardCallTo($this->filesystem(), $method, $parameters);
36 | }
37 |
38 | return $this->{$method}($parameters);
39 | }
40 |
41 | /**
42 | * @return Factory
43 | *
44 | * @codeCoverageIgnore
45 | */
46 | public function filesystem(): Factory
47 | {
48 | return $this->filesystem;
49 | }
50 |
51 | /**
52 | * Disk getter and setter.
53 | *
54 | * @param string|null $disk
55 | *
56 | * @return string|null
57 | */
58 | public function disk($disk = null): ?string
59 | {
60 | if (!empty($disk) && is_string($disk)) {
61 | $this->disk = $disk;
62 | }
63 |
64 | return $this->disk;
65 | }
66 |
67 | /**
68 | * Folder getter and setter.
69 | *
70 | * @param string|null $folder
71 | *
72 | * @return string|null
73 | */
74 | public function folder($folder = null): ?string
75 | {
76 | if (!empty($folder) && is_string($folder)) {
77 | $suffix = Str::endsWith($folder, DIRECTORY_SEPARATOR) ? '' : DIRECTORY_SEPARATOR;
78 |
79 | $this->folder = $folder.$suffix;
80 | }
81 |
82 | return $this->folder;
83 | }
84 |
85 | /**
86 | * @return bool
87 | *
88 | * @codeCoverageIgnore
89 | */
90 | public function isLocal(): bool
91 | {
92 | $adapter = $this->filesystem()
93 | ->disk($this->disk)
94 | ->getAdapter();
95 |
96 | return $adapter instanceof LocalFilesystemAdapter;
97 | }
98 |
99 | /**
100 | * @param string $path
101 | *
102 | * @return bool
103 | */
104 | public function exists(string $path): bool
105 | {
106 | return $this->filesystem()
107 | ->disk($this->disk())
108 | ->exists($this->path($path));
109 | }
110 |
111 | /**
112 | * Retrieve every chunks' folder.
113 | *
114 | * @return array
115 | */
116 | public function folders(): array
117 | {
118 | return $this->filesystem()->disk($this->disk())->listContents($this->folder(), false)
119 | ->filter(fn (StorageAttributes $attributes) => $attributes->isDir())
120 | ->sortByPath()
121 | ->map(fn (StorageAttributes $attributes) => $attributes->path())
122 | ->toArray();
123 | }
124 |
125 | /**
126 | * @param string|null $folder
127 | *
128 | * @return array
129 | */
130 | public function list($folder = ''): array
131 | {
132 | return $this->filesystem()->disk(
133 | $this->disk()
134 | )->files(
135 | $this->path($folder)
136 | );
137 | }
138 |
139 | /**
140 | * @param string $folder
141 | *
142 | * @return bool
143 | */
144 | public function makeDirectory(string $folder): bool
145 | {
146 | return $this->filesystem->disk($this->disk)
147 | ->makeDirectory($this->path($folder));
148 | }
149 |
150 | /**
151 | * @param array $config
152 | *
153 | * @return mixed
154 | *
155 | * @throws BindingResolutionException
156 | */
157 | public static function instance(array $config)
158 | {
159 | $filesystem = Container::getInstance()->make(get_called_class());
160 |
161 | $filesystem->disk(Arr::get($config, 'disk'));
162 | $filesystem->folder(Arr::get($config, 'folder'));
163 |
164 | return $filesystem;
165 | }
166 |
167 | /**
168 | * @param string $path
169 | *
170 | * @return string
171 | */
172 | protected function path(string $path): string
173 | {
174 | if (!Str::startsWith($path, $this->folder)) {
175 | return $this->folder().$path;
176 | }
177 |
178 | return $path;
179 | }
180 | }
181 |
--------------------------------------------------------------------------------
/src/Support/MergeFilesystem.php:
--------------------------------------------------------------------------------
1 | path($destination);
24 |
25 | if ($this->filesystem()->disk($this->disk)->writeStream($destination, $origin, $options)) {
26 | event(new MergeAdded($destination));
27 |
28 | return $destination;
29 | }
30 |
31 | return false;
32 | }
33 | }
34 |
--------------------------------------------------------------------------------
/src/Support/TempFilesystem.php:
--------------------------------------------------------------------------------
1 | filesystem()->disk($this->disk())->readStream($this->path($path));
22 | }
23 |
24 | /**
25 | * @param string $path
26 | * @param resource|string $resource
27 | * @param array $options
28 | *
29 | * @return string
30 | *
31 | * @throws FileExistsException
32 | */
33 | public function store(string $path, $resource = '', $options = []): string
34 | {
35 | $path = $this->path($path);
36 | $this->filesystem()->disk($this->disk())->writeStream($path, $resource, $options);
37 | $this->temp_files[] = $path;
38 |
39 | return $path;
40 | }
41 |
42 | public function clean()
43 | {
44 | foreach ($this->temp_files as $file) {
45 | $this->filesystem()->disk($this->disk())->delete($file);
46 | }
47 | }
48 | }
49 |
--------------------------------------------------------------------------------
5 |
6 |
7 |