├── .github └── workflows │ └── elixir.yml ├── .gitignore ├── CHANGELOG.md ├── Dockerfile ├── ISSUE_TEMPLATE.md ├── README.md ├── config └── config.exs ├── docker-compose.yml ├── documentation ├── development.md ├── examples │ ├── local.md │ └── s3.md └── livebooks │ └── custom_transformation.livemd ├── example.env ├── lib ├── mix │ └── tasks │ │ └── g.ex ├── waffle.ex └── waffle │ ├── actions │ ├── delete.ex │ ├── store.ex │ └── url.ex │ ├── behaviors │ └── storage_behaviour.ex │ ├── definition.ex │ ├── definition │ ├── storage.ex │ └── versioning.ex │ ├── exceptions.ex │ ├── file.ex │ ├── processor.ex │ ├── storage │ ├── local.ex │ └── s3.ex │ └── transformations │ └── convert.ex ├── mix.exs ├── mix.lock └── test ├── actions ├── store_test.exs └── url_test.exs ├── processor_test.exs ├── storage ├── local_test.exs └── s3_test.exs ├── support ├── image two.png ├── image+three.png ├── image.png └── invalid_image.png └── test_helper.exs /.github/workflows/elixir.yml: -------------------------------------------------------------------------------- 1 | name: Elixir CI 2 | 3 | on: 4 | push: 5 | branches: [ master ] 6 | pull_request: 7 | branches: [ master ] 8 | 9 | jobs: 10 | build_and_test: 11 | 12 | name: Build and test 13 | runs-on: ubuntu-latest 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - name: Build the Docker image 18 | run: docker build . --file Dockerfile --tag waffle 19 | 20 | - name: Run linters 21 | run: docker run waffle mix credo --strict 22 | 23 | - name: Run tests 24 | run: docker run waffle mix test 25 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /_build 2 | /deps 3 | erl_crash.dump 4 | *.ez 5 | .DS_Store 6 | /waffletest 7 | /doc 8 | .env* 9 | /priv 10 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## v1.1.9 (2024-06-14) 4 | 5 | - Fix module.function notation warnings in Elixir 1.17 #123 6 | - Added max_body_length to hackney options #122 7 | - Fix images and source links in hexdocs #125 8 | - Add links to view source in docs #124 9 | - Update link for waffle_gcs #118 10 | 11 | ## v1.1.8 (2024-02-07) 12 | 13 | - Fix documentation typos (#111) 14 | - Refactor local storage example: whitelist » acceptlist (#112) 15 | - Fix warnings emitted by elixir 1.16 (#115) 16 | - Add support for extensions to custom transformations (#117) 17 | 18 | ## v1.1.7 (2023-02-11) 19 | 20 | - Make Waffle.Storage.S3.s3_key/3 public, so that other modules can 21 | have access to the object path (#100) 22 | - Handle recv_timeout in Waffle.File (#102) 23 | - Fix typos (#107) 24 | - Support custom functions for transformations (#110) 25 | 26 | 27 | ## v1.1.6 (2022-02-20) 28 | * Improve Waffle.Storage.S3 moduledoc (#90) 29 | * Fix links to other storage providers (#92) 30 | * Allow setting a custom bucket from scope (#31) 31 | * Update dependencies 32 | 33 | ## v1.1.5 (2021-08-12) 34 | * Update dependencies 35 | * Add support for custom validation error message (#84) 36 | 37 | ## v1.1.4 (2021-02-25) 38 | * Upgrade deps for fixing compile warning in Elixir 1.11 (#74) 39 | * Use elixir v1.11 as base image 40 | * Make readme contain less marketing links 41 | * Migrate to GitHub Actions from CodeShip CI 42 | 43 | 44 | ## v1.1.3 (2020-09-14) 45 | * fix missing comma in configuration (#65) 46 | * request headers for remote file (#61) 47 | 48 | By default, when downloading files from remote path request headers are empty, 49 | but if you wish to provide your own, you can override the `remote_file_headers/1` 50 | function in your definition module. 51 | 52 | ## v1.1.2 (2020-09-03) 53 | * Improve docs of getting started and multiple minor changes (#55) 54 | * refactor Waffle.File.do_generate_temporary_path/1 (#56) 55 | 56 | ## v1.1.1 (2020-08-02) 57 | * update dependencies 58 | * add correct S3 setup to documentation 59 | * Use correct extension for temp files (#53) 60 | * Fix typo in comment (#52) 61 | * Update path in getting started guide to the correct generated path (#48) 62 | * add Aliyun OSS storage provider link (#45) 63 | 64 | ## v1.1.0 (2020-05-12) 65 | * update dependencies 66 | * respect `content-disposition` header (#41) 67 | * `ex_aws` needs to be at least 2.1.2 (#43) 68 | * add attribution to the original work (#39) 69 | 70 | ### Notes 71 | Now, the Waffle respects `content-disposition` header. It means that 72 | for remote uploads by url, we'll check for this header and will 73 | respect the filename from it. In other words, we'll save a file with a 74 | name from `content-disposition` header. 75 | 76 | ## v1.0.1 (2020-03-23) 77 | * Handle special S3 escaping (#32) 78 | * add branding for project (#34) 79 | 80 | ## v1.0.0 (2020-02-04) 81 | * remove poison and update dependencies (#30) 82 | * define remote url as an attribute #29 83 | * respect spaces in remote filenames (#28) 84 | 85 | ### Upgrade instructions 86 | `ExAws` dependency was upgraded to the current version. Since most of 87 | community packages are migrating to `Jason` as a default library to 88 | work with JSON, we've decided to migrate `Waffle` as well. You can 89 | still use `Poison` as your JSON adapter, in such case just add it as a 90 | dependency. 91 | 92 | #### Before 93 | ``` 94 | config :ex_aws, 95 | ... 96 | ``` 97 | #### After 98 | ``` 99 | config :ex_aws, 100 | json_codec: Jason 101 | ``` 102 | 103 | ## v0.0.4 (2019-12-16) 104 | * Fixes link to waffle_ecto (#16) 105 | * add credo to project (#17) 106 | * correct misspellings in documentation (#19) 107 | * Change reference to arc_gcs to refer to waffle_gcs instead (#22) 108 | * add documentation for modules (#25) 109 | * upgrade hackney to fix remote file downloading (#26) 110 | 111 | ## v0.0.3 (2019-09-09) 112 | * add new file from remote path with new filename (#13) 113 | * support specifying asset host for local storage (#12) 114 | * allow storage path to be different from url (#11) 115 | 116 | ## v0.0.2 (2019-09-02) 117 | * Clean up all temp files created during processing and storage (#9) 118 | * Bypass delete on skipped versions (#8) 119 | * add documentation for phoenix integration (#7) 120 | * add documentation for local development setup (#5) 121 | * update dependencies (#4) 122 | * add codeship integration (#2) 123 | 124 | ## v0.0.1 (2019-08-25) 125 | * Move project to a new repository and change name to `Waffle` 126 | 127 | ## v0.11.0 (2018-10-04) 128 | * (Dependency) `:httpoison` removed in favor of `:hackney` 129 | * (Enhancement) Proper generator file location for Phoenix 1.3+ 130 | * (Enhancement) Support setting asset_host to `false` in the app config to revert to the default 131 | * (Enhancement) Allow overriding asset_host in an individual definition module 132 | * (Enhancement) Definitions can conditionally skip a version or transformation 133 | 134 | ## v0.10.0 (2018-06-19) 135 | * (Dependency) `:ex_aws` increased to `~> 2.0` 136 | * (Dependency) `:ex_aws_s3` added at `~> 2.0` 137 | 138 | ## v0.9.0 (2018-06-19) 139 | * (Enhancement) Allow overriding the destination bucket in an upload definition. See (https://github.com/stavro/arc/pull/206) 140 | * (Enhancement) Allow overriding the `storage_dir` via configuration 141 | * (Enhancement) Skip uploading all files if any of the versions fail (PR: https://github.com/stavro/arc/pull/218) 142 | 143 | ## v0.8.0 (2017-04-20) 144 | * (Enhancement) Fix elixir warnings. 145 | * (Enhancement) Allow delete/1 to be overridden. 146 | * (Enhancement) Deletions follow same async behavior as uploads. 147 | * (Minor Breaking Change) URL encode returned urls. If you were explicitly encoding them yourself, you don't need to do this anymore. 148 | 149 | ## v0.7.0 (2017-02-07) 150 | * Require Elixir v1.4 151 | * Relax package dependencies 152 | * Fix Elixir v1.4 warnings 153 | * (Enhancement) Disable asynchronous processing via module attribute `@async false`. 154 | * (Enhancement) Add retry functionality to remote path uploader 155 | 156 | > v0.7.0 Requires Elixir 1.4 or above, due to enhancements made with ExAws and Task Streaming 157 | 158 | ## v0.6.0 (2016-12-19) 159 | * (Enhancement) Allow asset host to be set via an environment variable 160 | * (Enhancement) Allow downloading and saving remote files 161 | * (Enhancement) Move Arc storage module to config 162 | * (Bugfix) Split conversion arguments correctly when a file name has a space in it 163 | * (Bugfix) S3 object headers must be transferred to ExAws as a keyword list, not a map 164 | * (Bugfix) Don't prepend a forward-slash to local storage urls if the url already starts with a forward-slash. 165 | 166 | ## v0.6.0-rc3 (2016-10-20) 167 | * (Dependencies) - Upgrade `ex_aws` to rc3 168 | 169 | ## v0.6.0-rc2 (2016-10-20) 170 | * (Dependencies) - Upgrade `ex_aws` to rc2 171 | 172 | ## v0.6.0-rc1 (2016-10-04) 173 | * (Dependencies) - Removed `httpoison` as an optional dependency, added `sweet_xml` and `hackney` as optional dependencies (required if using S3). 174 | * (Enhancement) File streaming to S3 - Allows the uploading of large files to S3 without reading to memory first. 175 | * (Enhancement) Allow Arc to transform and store directly on binary input. 176 | * (Bugfix - backwards incompatible) Return error tuple rather than raising `Arc.ConvertError` if the transformation fails. 177 | * (Bugfix) Update `:crypto` usage to `:crypto.strong_rand_bytes` 178 | * (Enhancement) Optionally set S3 bucket from runtime env var (`config :arc, bucket: {:system, "S3_BUCKET"}`) 179 | * (Enhancement) Temporary files created during transformations now include the file extension. 180 | * (Bugfix) Add a leading slash to **urls** generated from the Local storage adapter. 181 | 182 | ## v0.5.3 (2016-06-21) 183 | * (Enhancement) Relax ex_aws dependency to allow `~> 0.5.0` 184 | 185 | ## v0.5.2 (2016-04-27) 186 | * (Enhancement) Allow returning a list of arguments for transformations to preserve desired groupings. 187 | 188 | ## v0.5.1 (2016-03-02) 189 | * (Enhancement) Raise a more helpful error message when attempting a transformation with an executable which cannot be found. 190 | 191 | ## v0.5.0 (2016-03-02) 192 | * (Enhancement) Allow transforms via arbitrary system executables. 193 | * (Enhancement) Allow transforms to supply a function to define the transformation args. 194 | * (Deprecation) Deprecate usage of {:noaction} in favor of :noaction for transformation responses. 195 | 196 | Upgrade instructions from 0.4.x to 0.5.x: 197 | 198 | Arc now favors explicitness in file extension changes rather than scanning with a Regex. If you have a `convert` transformation which changes the file extension (through the parameter `-format png` argument), you must explicitly add a third tuple argument in the conversion noting the final extension. 199 | 200 | Example: 201 | 202 | ```elixir 203 | # Change this: 204 | def transform(:thumb, _) do 205 | {:convert, "-strip -thumbnail 250x250^ -gravity center -extent 250x250 -format png"} 206 | end 207 | 208 | # To this: 209 | def transform(:thumb, _) do 210 | {:convert, "-strip -thumbnail 250x250^ -gravity center -extent 250x250 -format png", :png} #<--- Note the third tuple argument with the output file extension 211 | end 212 | ``` 213 | 214 | 215 | ## v0.4.1 (2016-02-28) 216 | * (Bugfix) Fix regression using the local filesystem introduced via v0.4.0. 217 | 218 | ## v0.4.0 (2016-02-25) 219 | * (Bugfix) Surface errors from ExAws put operations. Parse ExAws errors and return tuple of form `{:error, List.t}` when an error is encountered. 220 | 221 | To upgrade and properly support parsing aws errors, add `:poison` to your list of dependencies. 222 | 223 | > Optional dependency added, prompting a minor version bump. While not a strict backwards incompatibility, Arc users should take note of the change as more than an internal change. 224 | 225 | ## v0.3.0 (2016-01-22) 226 | * (Enhancement) Introduce `Definition.delete/2` 227 | 228 | > While there is no strict backwards incompatibility with the public API, a number of users have been using `Arc.Storage.S3.delete/3` as a public API due to a lack of a fully supported delete method. This internal method has now changed slightly, thus prompting more than a patch release. 229 | 230 | ## v0.2.3 (2016-01-22) 231 | * (Enhancement) Allow specifying custom s3 object headers through the definition module via `s3_object_headers/2`. 232 | 233 | ## v0.2.2 (12-14-2015) 234 | * (Enhancement) Allow the version transformation and storage timeout to be specified in configuration `config :arc, version_timeout: 15_000`. 235 | 236 | ## v0.2.1 (12-11-2015) 237 | * (Bugfix) Raise `Arc.ConvertError` if ImageMagick's `convert` tool exits unsuccessfully. 238 | 239 | ## v0.2.0 (12-11-2015) 240 | * (Breaking Change) Erlcloud has been removed in favor of ExAws. 241 | * (Enhancement) Added a configuration parameter to generate urls in the `virtual_host` style. 242 | 243 | ### Upgrade Instructions 244 | Since `erlcloud` has been removed from `arc`, you must also remove it from your dependency graph as well as your application list. In its place, add `ex_aws` and `httpoison` to your dependencies as well as application list. Next, remove the aws credential configuration from arc: 245 | 246 | ```elixir 247 | # BEFORE 248 | config :arc, 249 | access_key_id: "###", 250 | secret_access_key: "###", 251 | bucket: "uploads" 252 | 253 | #AFTER 254 | config :arc, 255 | bucket: "uploads" 256 | 257 | # (this is the default ex_aws config... if your keys are not in environment variables you can override it here) 258 | config :ex_aws, 259 | access_key_id: [{:system, "AWS_ACCESS_KEY_ID"}, :instance_role], 260 | secret_access_key: [{:system, "AWS_SECRET_ACCESS_KEY"}, :instance_role] 261 | ``` 262 | 263 | Read more about how ExAws manages configuration [here](https://github.com/CargoSense/ex_aws). 264 | 265 | ## v0.1.4 (11-10-2015) 266 | * (Enhancement: Local Storage) Filenames which contain path separators will flatten out as expected prior to moving copying the file to its destination. 267 | 268 | ## v0.1.3 (09-15-2015) 269 | 270 | * (Enhancement: Url Generation) `default_url/2` introduced to definition module which passes the given scope as the second parameter. Backwards compatibility is maintained for `default_url/1`. 271 | 272 | ## v0.1.2 (09-08-2015) 273 | 274 | * (Bugfix: Storage) Bugfix for referencing atoms in the file name. 275 | 276 | ## v0.1.1 277 | 278 | * (Enhancement: Storage) Add the local filesystem as a storage option. 279 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM elixir:1.13-alpine 2 | 3 | # add `convert` utility from imagemagick for image transformations 4 | RUN apk add --no-cache imagemagick 5 | 6 | RUN mix local.hex --force && \ 7 | mix local.rebar --force 8 | 9 | WORKDIR /srv/app 10 | 11 | COPY . . 12 | 13 | ENV MIX_ENV=test 14 | 15 | RUN mix deps.get && mix compile 16 | 17 | CMD mix test 18 | -------------------------------------------------------------------------------- /ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 8 | 9 | ### Environment 10 | 11 | * Elixir version (elixir -v): 12 | * Waffle version (mix deps): 13 | * Waffle dependencies when applicable (mix deps): 14 | * Operating system: 15 | 16 | ### Expected behavior 17 | 18 | 19 | ### Actual behavior 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [hex-img]: http://img.shields.io/hexpm/v/waffle.svg 2 | 3 | [hexdocs-img]: http://img.shields.io/badge/hexdocs-documentation-brightgreen.svg 4 | 5 | [evrone-img]: https://img.shields.io/badge/Sponsored_by-Evrone-brightgreen.svg 6 | 7 | # Waffle [![Sponsored by Evrone][evrone-img]](https://evrone.com?utm_source=waffle) 8 | 9 | [![Hex.pm Version][hex-img]](https://hex.pm/packages/waffle) 10 | [![waffle documentation][hexdocs-img]](https://hexdocs.pm/waffle) 11 | 12 | ![Waffle is a flexible file upload library for Elixir](https://elixir-waffle.github.io/waffle/assets/logo.svg) 13 | 14 | Waffle is a flexible file upload library for Elixir with straightforward integrations for Amazon S3 and ImageMagick. 15 | 16 | [Documentation](https://hexdocs.pm/waffle) 17 | 18 | ## Installation 19 | 20 | Add the latest stable release to your `mix.exs` file, along with the 21 | required dependencies for `ExAws` if appropriate: 22 | 23 | ```elixir 24 | defp deps do 25 | [ 26 | {:waffle, "~> 1.1"}, 27 | 28 | # If using S3: 29 | {:ex_aws, "~> 2.1.2"}, 30 | {:ex_aws_s3, "~> 2.0"}, 31 | {:hackney, "~> 1.9"}, 32 | {:sweet_xml, "~> 0.6"} 33 | ] 34 | end 35 | ``` 36 | 37 | Then run `mix deps.get` in your shell to fetch the dependencies. 38 | 39 | ## Usage 40 | 41 | After installing Waffle, another two things should be done: 42 | 43 | 1. setup a storage provider 44 | 2. define a definition module 45 | 46 | ### Setup a storage provider 47 | 48 | Waffle has two built-in storage providers: 49 | 50 | * `Waffle.Storage.Local` 51 | * `Waffle.Storage.S3` 52 | 53 | [Other available storage providers](#other-storage-providers) 54 | are supported by the community. 55 | 56 | An example for setting up `Waffle.Storage.Local`: 57 | 58 | ```elixir 59 | config :waffle, 60 | storage: Waffle.Storage.Local, 61 | asset_host: "http://static.example.com" # or {:system, "ASSET_HOST"} 62 | ``` 63 | 64 | An example for setting up `Waffle.Storage.S3`: 65 | 66 | ```elixir 67 | config :waffle, 68 | storage: Waffle.Storage.S3, 69 | bucket: "custom_bucket", # or {:system, "AWS_S3_BUCKET"} 70 | asset_host: "http://static.example.com" # or {:system, "ASSET_HOST"} 71 | 72 | config :ex_aws, 73 | json_codec: Jason 74 | # any configurations provided by https://github.com/ex-aws/ex_aws 75 | ``` 76 | 77 | ### Define a definition module 78 | 79 | Waffle requires a **definition module** which contains the relevant 80 | functions to store and retrieve files: 81 | 82 | * Optional transformations of the uploaded file 83 | * Where to put your files (the storage directory) 84 | * How to name your files 85 | * How to secure your files (private? Or publicly accessible?) 86 | * Default placeholders 87 | 88 | This module can be created manually or generated by `mix waffle.g` 89 | automatically. 90 | 91 | As an example, we will generate a definition module for handling 92 | avatars: 93 | 94 | mix waffle.g avatar 95 | 96 | This should generate a file at `lib/[APP_NAME]_web/uploaders/avatar.ex`. 97 | Check this file for descriptions of configurable options. 98 | 99 | ## Examples 100 | 101 | * [An example for Local storage driver](documentation/examples/local.md) 102 | * [An example for S3 storage driver](documentation/examples/s3.md) 103 | 104 | ## Usage with Ecto 105 | 106 | Waffle comes with a companion package for use with Ecto. If you 107 | intend to use Waffle with Ecto, it is highly recommended you also 108 | add the 109 | [`waffle_ecto`](https://github.com/elixir-waffle/waffle_ecto) 110 | dependency. Benefits include: 111 | 112 | * Changeset integration 113 | * Versioned urls for cache busting (`.../thumb.png?v=63601457477`) 114 | 115 | ## Other Storage Providers 116 | 117 | * **Rackspace** - [arc_rackspace](https://github.com/lokalebasen/arc_rackspace) 118 | 119 | * **Manta** - [arc_manta](https://github.com/onyxrev/arc_manta) 120 | 121 | * **OVH** - [arc_ovh](https://github.com/stephenmoloney/arc_ovh) 122 | 123 | * **Google Cloud Storage** - [waffle_gcs](https://github.com/elixir-waffle/waffle_gcs) 124 | 125 | * **Microsoft Azure Storage** - [arc_azure](https://github.com/phil-a/arc_azure) 126 | 127 | * **Aliyun OSS Storage** - [waffle_aliyun_oss](https://github.com/ug0/waffle_aliyun_oss) 128 | 129 | ## Testing 130 | 131 | The basic test suite can be run with without supplying any S3 information: 132 | 133 | ``` 134 | mix test 135 | ``` 136 | 137 | In order to test S3 capability, you must have access to an S3/equivalent bucket. For 138 | S3 buckets, the bucket must be configured to allow ACLs and it must allow public 139 | access. 140 | 141 | The following environment variables will be used by the test suite: 142 | 143 | * WAFFLE_TEST_BUCKET 144 | * WAFFLE_TEST_BUCKET2 145 | * WAFFLE_TEST_S3_KEY 146 | * WAFFLE_TEST_S3_SECRET 147 | * WAFFLE_TEST_REGION 148 | 149 | After setting these variables, you can run the full test suite with `mix test --include s3:true`. 150 | 151 | ## Attribution 152 | 153 | Great thanks to Sean Stavropoulos (@stavro) for the original awesome work on the library. 154 | 155 | This project is forked from [Arc](https://github.com/stavro/arc) from the version `v0.11.0`. 156 | 157 | ## License 158 | 159 | Copyright 2019 Boris Kuznetsov 160 | 161 | Copyright 2015 Sean Stavropoulos 162 | 163 | Licensed under the Apache License, Version 2.0 (the "License"); 164 | you may not use this file except in compliance with the License. 165 | You may obtain a copy of the License at 166 | 167 | http://www.apache.org/licenses/LICENSE-2.0 168 | 169 | Unless required by applicable law or agreed to in writing, software 170 | distributed under the License is distributed on an "AS IS" BASIS, 171 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 172 | See the License for the specific language governing permissions and 173 | limitations under the License. 174 | -------------------------------------------------------------------------------- /config/config.exs: -------------------------------------------------------------------------------- 1 | import Config 2 | 3 | config :waffle, 4 | storage: Waffle.Storage.S3 5 | 6 | config :ex_aws, 7 | json_codec: Jason 8 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "2.4" 2 | services: 3 | waffle: 4 | build: 5 | context: . 6 | dockerfile: Dockerfile 7 | command: tail -f /dev/null 8 | env_file: 9 | - .env 10 | volumes: 11 | - .:/srv/app:cached 12 | -------------------------------------------------------------------------------- /documentation/development.md: -------------------------------------------------------------------------------- 1 | # Development 2 | 3 | Development documentation with instructions how to setup the project for local development. 4 | 5 | ## Preliminary 6 | 7 | * Docker 8 | 9 | ```sh 10 | # screen 1 11 | $ cp example.env .env 12 | $ docker-compose up 13 | 14 | # screen 2 15 | $ docker-compose exec waffle sh 16 | $ > mix deps.get 17 | ``` 18 | 19 | ## Tests with S3 integration 20 | 21 | AWS S3 setup 22 | - create a new user with FullS3Access 23 | - copy `key_id` and `secret` 24 | - create a new bucket with *public access* 25 | - comment the `s3` exclusion inside `test/test_helper.exs` 26 | - update `.env` file 27 | 28 | ```sh 29 | $ mix test 30 | ``` 31 | 32 | ## Common tasks 33 | 34 | ```sh 35 | # to run linter 36 | $ mix credo --strict 37 | 38 | # to generate documentation 39 | $ MIX_ENV=dev mix docs 40 | 41 | # to publish package 42 | $ MIX_ENV=dev mix hex.publish 43 | 44 | # to publish only documentation 45 | $ MIX_ENV=dev mix hex.publish docs 46 | ``` 47 | -------------------------------------------------------------------------------- /documentation/examples/local.md: -------------------------------------------------------------------------------- 1 | # An Example for Local 2 | 3 | Setup the storage provider: 4 | ```elixir 5 | config :waffle, 6 | storage: Waffle.Storage.Local, 7 | asset_host: "http://static.example.com" # or {:system, "ASSET_HOST"} 8 | ``` 9 | 10 | Define a definition module: 11 | ```elixir 12 | defmodule Avatar do 13 | use Waffle.Definition 14 | 15 | @versions [:original, :thumb] 16 | @extensions ~w(.jpg .jpeg .gif .png) 17 | 18 | def validate({file, _}) do 19 | file_extension = file.file_name |> Path.extname |> String.downcase 20 | 21 | case Enum.member?(@extensions, file_extension) do 22 | true -> :ok 23 | false -> {:error, "file type is invalid"} 24 | end 25 | end 26 | 27 | def transform(:thumb, _) do 28 | {:convert, "-thumbnail 100x100^ -gravity center -extent 100x100 -format png", :png} 29 | end 30 | 31 | def filename(version, _) do 32 | version 33 | end 34 | 35 | def storage_dir(_, {file, user}) do 36 | "uploads/avatars/#{user.id}" 37 | end 38 | end 39 | ``` 40 | 41 | Store or Get files: 42 | ```elixir 43 | # Given some current_user record 44 | current_user = %{id: 1} 45 | 46 | # Store any accessible file 47 | Avatar.store({"/path/to/my/selfie.png", current_user}) 48 | #=> {:ok, "selfie.png"} 49 | 50 | # ..or store directly from the `params` of a file upload within your controller 51 | Avatar.store({%Plug.Upload{}, current_user}) 52 | #=> {:ok, "selfie.png"} 53 | 54 | # and retrieve the url later 55 | Avatar.url({"selfie.png", current_user}, :thumb) 56 | #=> "uploads/avatars/1/thumb.png" 57 | ``` 58 | -------------------------------------------------------------------------------- /documentation/examples/s3.md: -------------------------------------------------------------------------------- 1 | # An Example for S3 2 | 3 | Setup the storage provider: 4 | ```elixir 5 | config :waffle, 6 | storage: Waffle.Storage.S3, 7 | bucket: "custom_bucket", # or {:system, "AWS_S3_BUCKET"} 8 | asset_host: "http://static.example.com" # or {:system, "ASSET_HOST"} 9 | 10 | config :ex_aws, 11 | json_codec: Jason 12 | # any configurations provided by https://github.com/ex-aws/ex_aws 13 | ``` 14 | 15 | Define a definition module: 16 | ```elixir 17 | defmodule Avatar do 18 | use Waffle.Definition 19 | 20 | @versions [:original, :thumb] 21 | @extension_whitelist ~w(.jpg .jpeg .gif .png) 22 | 23 | def acl(:thumb, _), do: :public_read 24 | 25 | def validate({file, _}) do 26 | file_extension = file.file_name |> Path.extname |> String.downcase 27 | Enum.member?(@extension_whitelist, file_extension) 28 | end 29 | 30 | def transform(:thumb, _) do 31 | {:convert, "-thumbnail 100x100^ -gravity center -extent 100x100 -format png", :png} 32 | end 33 | 34 | def filename(version, _) do 35 | version 36 | end 37 | 38 | def storage_dir(_, {file, user}) do 39 | "uploads/avatars/#{user.id}" 40 | end 41 | 42 | def default_url(:thumb) do 43 | "https://placehold.it/100x100" 44 | end 45 | end 46 | ``` 47 | 48 | Store or Get files: 49 | ```elixir 50 | # Given some current_user record 51 | current_user = %{id: 1} 52 | 53 | # Store any accessible file 54 | Avatar.store({"/path/to/my/selfie.png", current_user}) 55 | #=> {:ok, "selfie.png"} 56 | 57 | # ..or store directly from the `params` of a file upload within your controller 58 | Avatar.store({%Plug.Upload{}, current_user}) 59 | #=> {:ok, "selfie.png"} 60 | 61 | # and retrieve the url later 62 | Avatar.url({"selfie.png", current_user}, :thumb) 63 | #=> "https://s3.amazonaws.com/custom_bucket/uploads/avatars/1/thumb.png" 64 | ``` 65 | -------------------------------------------------------------------------------- /documentation/livebooks/custom_transformation.livemd: -------------------------------------------------------------------------------- 1 | # Processing with custom functions 2 | 3 | ```elixir 4 | lib = 5 | Regex.named_captures(~r/(?.+)documentation\/livebooks/, __DIR__) 6 | |> Map.get("lib") 7 | 8 | Mix.install([ 9 | # for local development 10 | # {:waffle, path: lib} 11 | :waffle 12 | ]) 13 | ``` 14 | 15 | ## Definition 16 | 17 | All starts from creating the definition and custom processing function 18 | 19 | ```elixir 20 | defmodule Avatar do 21 | use Waffle.Definition 22 | @versions [:original, :thumb] 23 | 24 | def __storage, do: Waffle.Storage.Local 25 | 26 | def filename(version, _), do: version 27 | 28 | def transform(:thumb, _) do 29 | &process/2 30 | end 31 | 32 | @spec process( 33 | atom(), 34 | Waffle.File.t() 35 | ) :: {:ok, Waffle.File.t()} | {:error, String.t()} 36 | def process(_version, original_file) do 37 | {:ok, file} 38 | end 39 | end 40 | ``` 41 | 42 | Then, you can store the file by calling `Avatar.store/1` 43 | 44 | ```elixir 45 | image = lib <> "test/support/image.png" 46 | 47 | Avatar.store(image) 48 | ``` 49 | 50 | ## Understanding custom transformations 51 | 52 | 53 | 54 | ```elixir 55 | def process(_version, original_file) do 56 | {:ok, file} 57 | end 58 | ``` 59 | 60 | To generate a temporary path for the new file version 61 | 62 | 63 | 64 | ```elixir 65 | tmp_path = Waffle.File.generate_temporary_path(file) 66 | ``` 67 | 68 | or if the new version is going to have a new extension 69 | 70 | 71 | 72 | ```elixir 73 | tmp_path = Waffle.File.generate_temporary_path("png") 74 | ``` 75 | 76 | then, you can process your file and put the result into tmp_path. 77 | 78 | To return the processed tmp file into the pipeline and clean it afterwards, create a new file struct 79 | 80 | 81 | 82 | ```elixir 83 | {:ok, %Waffle.File{file | path: tmp_path, is_tempfile?: true}} 84 | ``` 85 | 86 | You can combine it all together to use `ExOptimizer` library 87 | 88 | 89 | 90 | ```elixir 91 | def process(_version, original_file) do 92 | tmp_path = Waffle.File.generate_temporary_path(original_file) 93 | 94 | File.cp!(original_file.path, tmp_path) 95 | 96 | with {:ok, _} <- ExOptimizer.optimize(tmp_path) do 97 | { 98 | :ok, 99 | %Waffle.File{original_file | path: tmp_path, is_tempfile?: true} 100 | } 101 | end 102 | end 103 | ``` 104 | 105 | ## All together 106 | 107 | We can create a bit more complex example, where we combine transformation done by external binary with transformation done by existing elixir library. 108 | 109 | ```elixir 110 | defmodule Avatar do 111 | use Waffle.Definition 112 | @versions [:original, :thumb] 113 | 114 | def __storage, do: Waffle.Storage.Local 115 | 116 | def filename(version, _), do: version 117 | 118 | def transform(:thumb, _) do 119 | &process/2 120 | end 121 | 122 | @spec process( 123 | atom(), 124 | Waffle.File.t() 125 | ) :: {:ok, Waffle.File.t()} | {:error, String.t()} 126 | def process(_version, original_file) do 127 | # convert .jpg to .png 128 | args = "-strip -thumbnail 100x100^ -gravity center -extent 100x100 -format png" 129 | 130 | with {:ok, file} <- 131 | Waffle.Transformations.Convert.apply( 132 | :convert, 133 | original_file, 134 | args, 135 | :png 136 | ), 137 | {:ok, _} <- ExOptimizer.optimize(file.path) do 138 | {:ok, file} 139 | end 140 | end 141 | end 142 | ``` 143 | -------------------------------------------------------------------------------- /example.env: -------------------------------------------------------------------------------- 1 | WAFFLE_TEST_BUCKET= 2 | WAFFLE_TEST_BUCKET2= 3 | WAFFLE_TEST_S3_KEY= 4 | WAFFLE_TEST_S3_SECRET= 5 | -------------------------------------------------------------------------------- /lib/mix/tasks/g.ex: -------------------------------------------------------------------------------- 1 | defmodule Mix.Tasks.Waffle do 2 | defmodule G do 3 | use Mix.Task 4 | import Mix.Generator 5 | import Macro, only: [camelize: 1, underscore: 1] 6 | 7 | @shortdoc "For Waffle definition generation code" 8 | 9 | @moduledoc """ 10 | A task for generating waffle uploader modules. 11 | 12 | If generating an uploader in a Phoenix project, your a uploader will be generated in 13 | lib/[APP_NAME]_web/uploaders/ 14 | 15 | ## Example 16 | 17 | mix waffle.g avatar # creates lib/[APP_NAME]_web/uploaders/avatar.ex 18 | 19 | 20 | If not generating an uploader in a Phoenix project, then you must pass the path to where the 21 | uploader should be generated. 22 | 23 | ## Example 24 | 25 | mix waffle.g avatar uploaders/avatar.ex # creates uploaders/avatar.ex 26 | """ 27 | 28 | def run([model_name]) do 29 | app_name = Mix.Project.config()[:app] 30 | if File.exists?("lib/#{app_name}_web/") do 31 | project_module_name = camelize(to_string(app_name)) 32 | generate_phx_uploader_file(model_name, project_module_name) 33 | else 34 | raise "path must be passed when generating an uploader." 35 | end 36 | end 37 | 38 | def run([model_name, path]) do 39 | app_name = Mix.Project.config()[:app] 40 | project_module_name = camelize(to_string(app_name)) 41 | generate_uploader_file(model_name, project_module_name, path) 42 | end 43 | 44 | def run(_) do 45 | IO.puts "Incorrect syntax. Please try mix waffle.g " 46 | end 47 | 48 | defp generate_uploader_file(model_name, project_module_name, path) do 49 | model_destination = Path.join(File.cwd!(), "#{path}/#{underscore(model_name)}.ex") 50 | create_uploader(model_name, project_module_name, model_destination) 51 | end 52 | 53 | defp generate_phx_uploader_file(model_name, project_module_name) do 54 | app_name = Mix.Project.config()[:app] 55 | model_destination = Path.join(File.cwd!(), "/lib/#{app_name}_web/uploaders/#{underscore(model_name)}.ex") 56 | create_uploader(model_name, project_module_name, model_destination) 57 | end 58 | 59 | defp create_uploader(model_name, project_module_name, model_destination) do 60 | create_file model_destination, uploader_template( 61 | model_name: model_name, 62 | uploader_model_name: Module.concat(project_module_name, camelize(model_name)) 63 | ) 64 | end 65 | 66 | embed_template :uploader, """ 67 | defmodule <%= inspect @uploader_model_name %> do 68 | use Waffle.Definition 69 | 70 | # Include ecto support (requires package waffle_ecto installed): 71 | # use Waffle.Ecto.Definition 72 | 73 | @versions [:original] 74 | 75 | # To add a thumbnail version: 76 | # @versions [:original, :thumb] 77 | 78 | # Override the bucket on a per definition basis: 79 | # def bucket do 80 | # :custom_bucket_name 81 | # end 82 | 83 | # def bucket({_file, scope}) do 84 | # scope.bucket || bucket() 85 | # end 86 | 87 | # Whitelist file extensions: 88 | # def validate({file, _}) do 89 | # file_extension = file.file_name |> Path.extname() |> String.downcase() 90 | # 91 | # case Enum.member?(~w(.jpg .jpeg .gif .png), file_extension) do 92 | # true -> :ok 93 | # false -> {:error, "invalid file type"} 94 | # end 95 | # end 96 | 97 | # Define a thumbnail transformation: 98 | # def transform(:thumb, _) do 99 | # {:convert, "-strip -thumbnail 250x250^ -gravity center -extent 250x250 -format png", :png} 100 | # end 101 | 102 | # Override the persisted filenames: 103 | # def filename(version, _) do 104 | # version 105 | # end 106 | 107 | # Override the storage directory: 108 | # def storage_dir(version, {file, scope}) do 109 | # "uploads/user/avatars/\#{scope.id}" 110 | # end 111 | 112 | # Provide a default URL if there hasn't been a file uploaded 113 | # def default_url(version, scope) do 114 | # "/images/avatars/default_\#{version}.png" 115 | # end 116 | 117 | # Specify custom headers for s3 objects 118 | # Available options are [:cache_control, :content_disposition, 119 | # :content_encoding, :content_length, :content_type, 120 | # :expect, :expires, :storage_class, :website_redirect_location] 121 | # 122 | # def s3_object_headers(version, {file, scope}) do 123 | # [content_type: MIME.from_path(file.file_name)] 124 | # end 125 | end 126 | """ 127 | 128 | end 129 | end 130 | -------------------------------------------------------------------------------- /lib/waffle.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle do 2 | @moduledoc false 3 | end 4 | -------------------------------------------------------------------------------- /lib/waffle/actions/delete.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Actions.Delete do 2 | @moduledoc ~S""" 3 | Delete files from a defined adapter. 4 | 5 | After an object is stored through Waffle, you may optionally remove 6 | it. To remove a stored object, pass the same path identifier and 7 | scope from which you stored the object. 8 | 9 | # Without a scope: 10 | {:ok, original_filename} = Avatar.store("/Images/me.png") 11 | :ok = Avatar.delete(original_filename) 12 | 13 | # With a scope: 14 | user = Repo.get!(User, 1) 15 | {:ok, original_filename} = Avatar.store({"/Images/me.png", user}) 16 | # example 1 17 | :ok = Avatar.delete({original_filename, user}) 18 | # example 2 19 | user = Repo.get!(User, 1) 20 | :ok = Avatar.delete({user.avatar, user}) 21 | 22 | """ 23 | 24 | alias Waffle.Actions.Delete 25 | 26 | defmacro __using__(_) do 27 | quote do 28 | def delete(args), do: Delete.delete(__MODULE__, args) 29 | 30 | defoverridable [{:delete, 1}] 31 | end 32 | end 33 | 34 | def delete(definition, {filepath, scope}) when is_binary(filepath) do 35 | do_delete(definition, {%{file_name: filepath}, scope}) 36 | end 37 | 38 | def delete(definition, filepath) when is_binary(filepath) do 39 | do_delete(definition, {%{file_name: filepath}, nil}) 40 | end 41 | 42 | # 43 | # Private 44 | # 45 | 46 | defp version_timeout do 47 | Application.get_env(:waffle, :version_timeout) || 15_000 48 | end 49 | 50 | defp do_delete(definition, {file, scope}) do 51 | if definition.async() do 52 | definition.__versions() 53 | |> Enum.map(fn(r) -> async_delete_version(definition, r, {file, scope}) end) 54 | |> Enum.each(fn(task) -> Task.await(task, version_timeout()) end) 55 | else 56 | definition.__versions() 57 | |> Enum.each(fn(version) -> delete_version(definition, version, {file, scope}) end) 58 | end 59 | :ok 60 | end 61 | 62 | defp async_delete_version(definition, version, {file, scope}) do 63 | Task.async(fn -> delete_version(definition, version, {file, scope}) end) 64 | end 65 | 66 | defp delete_version(definition, version, {file, scope}) do 67 | conversion = definition.transform(version, {file, scope}) 68 | if conversion == :skip do 69 | :ok 70 | else 71 | definition.__storage().delete(definition, version, {file, scope}) 72 | end 73 | end 74 | end 75 | -------------------------------------------------------------------------------- /lib/waffle/actions/store.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Actions.Store do 2 | @moduledoc ~S""" 3 | Store files to a defined adapter. 4 | 5 | The definition module responds to `Avatar.store/1` which 6 | accepts either: 7 | 8 | * A path to a local file 9 | 10 | * A path to a remote `http` or `https` file 11 | 12 | * A map with a filename and path keys (eg, a `%Plug.Upload{}`) 13 | 14 | * A map with a filename and binary keys (eg, `%{filename: "image.png", binary: <<255,255,255,...>>}`) 15 | * A map with a filename and stream keys (eg, `%{filename: "image.png", stream: %Stream{...}}`) 16 | 17 | * A two-element tuple consisting of one of the above file formats as well as a scope map 18 | 19 | Example usage as general file store: 20 | 21 | # Store any locally accessible file 22 | Avatar.store("/path/to/my/file.png") #=> {:ok, "file.png"} 23 | 24 | # Store any remotely accessible file 25 | Avatar.store("http://example.com/file.png") #=> {:ok, "file.png"} 26 | 27 | # Store a file directly from a `%Plug.Upload{}` 28 | Avatar.store(%Plug.Upload{filename: "file.png", path: "/a/b/c"}) #=> {:ok, "file.png"} 29 | 30 | # Store a file from a connection body 31 | {:ok, data, _conn} = Plug.Conn.read_body(conn) 32 | Avatar.store(%{filename: "file.png", binary: data}) 33 | 34 | Example usage as a file attached to a `scope`: 35 | 36 | scope = Repo.get(User, 1) 37 | Avatar.store({%Plug.Upload{}, scope}) #=> {:ok, "file.png"} 38 | 39 | This scope will be available throughout the definition module to be 40 | used as an input to the storage parameters (eg, store files in 41 | `/uploads/#{scope.id}`). 42 | 43 | """ 44 | 45 | alias Waffle.Actions.Store 46 | alias Waffle.Definition.Versioning 47 | 48 | defmacro __using__(_) do 49 | quote do 50 | def store(args), do: Store.store(__MODULE__, args) 51 | end 52 | end 53 | 54 | def store(definition, {file, scope}) 55 | when is_binary(file) or is_map(file) do 56 | put(definition, {Waffle.File.new(file, definition), scope}) 57 | end 58 | 59 | def store(definition, filepath) 60 | when is_binary(filepath) or is_map(filepath) do 61 | store(definition, {filepath, nil}) 62 | end 63 | 64 | # 65 | # Private 66 | # 67 | 68 | defp put(_definition, {error = {:error, _msg}, _scope}), do: error 69 | 70 | defp put(definition, {%Waffle.File{} = file, scope}) do 71 | case definition.validate({file, scope}) do 72 | result when result == true or result == :ok -> 73 | put_versions(definition, {file, scope}) 74 | |> cleanup!(file) 75 | 76 | {:error, message} -> 77 | {:error, message} 78 | 79 | _ -> 80 | {:error, :invalid_file} 81 | end 82 | end 83 | 84 | defp put_versions(definition, {file, scope}) do 85 | if definition.async() do 86 | definition.__versions() 87 | |> Enum.map(fn(r) -> async_process_version(definition, r, {file, scope}) end) 88 | |> Enum.map(fn(task) -> Task.await(task, version_timeout()) end) 89 | |> ensure_all_success 90 | |> Enum.map(fn({v, r}) -> async_put_version(definition, v, {r, scope}) end) 91 | |> Enum.map(fn(task) -> Task.await(task, version_timeout()) end) 92 | |> handle_responses(file.file_name) 93 | else 94 | definition.__versions() 95 | |> Enum.map(fn(version) -> process_version(definition, version, {file, scope}) end) 96 | |> ensure_all_success 97 | |> Enum.map(fn({version, result}) -> put_version(definition, version, {result, scope}) end) 98 | |> handle_responses(file.file_name) 99 | end 100 | end 101 | 102 | defp ensure_all_success(responses) do 103 | errors = Enum.filter(responses, fn({_version, resp}) -> elem(resp, 0) == :error end) 104 | if Enum.empty?(errors), do: responses, else: errors 105 | end 106 | 107 | defp handle_responses(responses, filename) do 108 | errors = Enum.filter(responses, fn(resp) -> elem(resp, 0) == :error end) |> Enum.map(fn(err) -> elem(err, 1) end) 109 | if Enum.empty?(errors), do: {:ok, filename}, else: {:error, errors} 110 | end 111 | 112 | defp version_timeout do 113 | Application.get_env(:waffle, :version_timeout) || 15_000 114 | end 115 | 116 | defp async_process_version(definition, version, {file, scope}) do 117 | Task.async(fn -> 118 | process_version(definition, version, {file, scope}) 119 | end) 120 | end 121 | 122 | defp async_put_version(definition, version, {result, scope}) do 123 | Task.async(fn -> 124 | put_version(definition, version, {result, scope}) 125 | end) 126 | end 127 | 128 | defp process_version(definition, version, {file, scope}) do 129 | {version, Waffle.Processor.process(definition, version, {file, scope})} 130 | end 131 | 132 | defp put_version(definition, version, {result, scope}) do 133 | case result do 134 | {:error, error} -> {:error, error} 135 | {:ok, nil} -> {:ok, nil} 136 | {:ok, file} -> 137 | file_name = Versioning.resolve_file_name(definition, version, {file, scope}) 138 | file = %Waffle.File{file | file_name: file_name} 139 | result = definition.__storage().put(definition, version, {file, scope}) 140 | 141 | case definition.transform(version, {file, scope}) do 142 | :noaction -> 143 | # We don't have to cleanup after `:noaction` transformations 144 | # because final `cleanup!` will remove the original temporary file. 145 | result 146 | _ -> 147 | cleanup!(result, file) 148 | end 149 | end 150 | end 151 | 152 | defp cleanup!(result, file) do 153 | # If we were working with binary data or a remote file, a tempfile was 154 | # created that we need to clean up. 155 | if file.is_tempfile? do 156 | File.rm!(file.path) 157 | end 158 | 159 | result 160 | end 161 | 162 | end 163 | -------------------------------------------------------------------------------- /lib/waffle/actions/url.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Actions.Url do 2 | @moduledoc ~S""" 3 | Url generation. 4 | 5 | Saving your files is only the first half of any decent storage 6 | solution. Straightforward access to your uploaded files is equally 7 | as important as storing them in the first place. 8 | 9 | Often times you will want to regain access to the stored files. As 10 | such, `Waffle` facilitates the generation of urls. 11 | 12 | # Given some user record 13 | user = %{id: 1} 14 | 15 | Avatar.store({%Plug.Upload{}, user}) #=> {:ok, "selfie.png"} 16 | 17 | # To generate a regular, unsigned url (defaults to the first version): 18 | Avatar.url({"selfie.png", user}) 19 | #=> "https://bucket.s3.amazonaws.com/uploads/1/original.png" 20 | 21 | # To specify the version of the upload: 22 | Avatar.url({"selfie.png", user}, :thumb) 23 | #=> "https://bucket.s3.amazonaws.com/uploads/1/thumb.png" 24 | 25 | # To generate a signed url: 26 | Avatar.url({"selfie.png", user}, :thumb, signed: true) 27 | #=> "https://bucket.s3.amazonaws.com/uploads/1/thumb.png?AWSAccessKeyId=AKAAIPDF14AAX7XQ&Signature=5PzIbSgD1V2vPLj%2B4WLRSFQ5M%3D&Expires=1434395458" 28 | 29 | # To generate urls for all versions: 30 | Avatar.urls({"selfie.png", user}) 31 | #=> %{original: "https://.../original.png", thumb: "https://.../thumb.png"} 32 | 33 | **Default url** 34 | 35 | In cases where a placeholder image is desired when an uploaded file 36 | is not present, Waffle allows the definition of a default image to 37 | be returned gracefully when requested with a `nil` file. 38 | 39 | def default_url(version) do 40 | MyApp.Endpoint.url <> "/images/placeholders/profile_image.png" 41 | end 42 | 43 | Avatar.url(nil) #=> "http://example.com/images/placeholders/profile_image.png" 44 | Avatar.url({nil, scope}) #=> "http://example.com/images/placeholders/profile_image.png" 45 | 46 | **Virtual Host** 47 | 48 | To support AWS regions other than US Standard, it may be required to 49 | generate urls in the 50 | [`virtual_host`](http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html) 51 | style. This will generate urls in the style: 52 | `https://#{bucket}.s3.amazonaws.com` instead of 53 | `https://s3.amazonaws.com/#{bucket}`. 54 | 55 | To use this style of url generation, your bucket name must be DNS 56 | compliant. 57 | 58 | This can be enabled with: 59 | 60 | config :waffle, 61 | virtual_host: true 62 | 63 | > When using virtual hosted–style buckets with SSL, the SSL wild card certificate only matches buckets that do not contain periods. To work around this, use HTTP or write your own certificate verification logic. 64 | 65 | **Asset Host** 66 | 67 | You may optionally specify an asset host rather than using the 68 | default `bucket.s3.amazonaws.com` format. 69 | 70 | In your application configuration, you'll need to provide an `asset_host` value: 71 | 72 | config :waffle, 73 | asset_host: "https://d3gav2egqolk5.cloudfront.net", # For a value known during compilation 74 | asset_host: {:system, "ASSET_HOST"} # For a value not known until runtime 75 | 76 | """ 77 | 78 | alias Waffle.Actions.Url 79 | alias Waffle.Definition.Versioning 80 | 81 | defmacro __using__(_) do 82 | quote do 83 | def urls(file, options \\ []) do 84 | Enum.into __MODULE__.__versions(), %{}, fn(r) -> 85 | {r, __MODULE__.url(file, r, options)} 86 | end 87 | end 88 | 89 | def url(file), do: url(file, nil) 90 | def url(file, options) when is_list(options), do: url(file, nil, options) 91 | def url(file, version), do: url(file, version, []) 92 | def url(file, version, options), do: Url.url(__MODULE__, file, version, options) 93 | 94 | defoverridable [{:url, 3}] 95 | end 96 | end 97 | 98 | # Apply default version if not specified 99 | def url(definition, file, nil, options), 100 | do: url(definition, file, Enum.at(definition.__versions(), 0), options) 101 | 102 | # Transform standalone file into a tuple of {file, scope} 103 | def url(definition, file, version, options) when is_binary(file) or is_map(file) or is_nil(file), 104 | do: url(definition, {file, nil}, version, options) 105 | 106 | # Transform file-path into a map with a file_name key 107 | def url(definition, {file, scope}, version, options) when is_binary(file) do 108 | url(definition, {%{file_name: file}, scope}, version, options) 109 | end 110 | 111 | def url(definition, {file, scope}, version, options) do 112 | build(definition, version, {file, scope}, options) 113 | end 114 | 115 | # 116 | # Private 117 | # 118 | 119 | defp build(definition, version, {nil, scope}, _options) do 120 | definition.default_url(version, scope) 121 | end 122 | 123 | defp build(definition, version, file_and_scope, options) do 124 | case Versioning.resolve_file_name(definition, version, file_and_scope) do 125 | nil -> nil 126 | _ -> 127 | definition.__storage().url(definition, version, file_and_scope, options) 128 | end 129 | end 130 | end 131 | -------------------------------------------------------------------------------- /lib/waffle/behaviors/storage_behaviour.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.StorageBehavior do 2 | @moduledoc """ 3 | Defines the behavior for file storage. 4 | 5 | ## Callbacks 6 | 7 | - `put/3`: Saves a file and returns the file name or an error. 8 | - `url/3`: Generates a URL for accessing a file. 9 | - `delete/3`: Deletes a file and returns the result of the operation. 10 | """ 11 | 12 | @callback put(definition :: atom, version :: atom, file_and_scope :: {Waffle.File.t(), any}) :: 13 | {:ok, file_name :: String.t()} | {:error, reason :: any} 14 | 15 | @callback url(definition :: atom, version :: atom, file_and_scope :: {Waffle.File.t(), any}) :: 16 | String.t() 17 | 18 | @callback delete(definition :: atom, version :: atom, file_and_scope :: {Waffle.File.t(), any}) :: 19 | atom 20 | end 21 | -------------------------------------------------------------------------------- /lib/waffle/definition.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Definition do 2 | @moduledoc ~S""" 3 | Defines uploader to manage files. 4 | 5 | defmodule Avatar do 6 | use Waffle.Definition 7 | end 8 | 9 | Consists of several components to manage different parts of file 10 | managing process. 11 | 12 | * `Waffle.Definition.Versioning` 13 | 14 | * `Waffle.Definition.Storage` 15 | 16 | * `Waffle.Actions.Store` 17 | 18 | * `Waffle.Actions.Delete` 19 | 20 | * `Waffle.Actions.Url` 21 | 22 | """ 23 | 24 | defmacro __using__(_options) do 25 | quote do 26 | use Waffle.Definition.Versioning 27 | use Waffle.Definition.Storage 28 | 29 | use Waffle.Actions.Store 30 | use Waffle.Actions.Delete 31 | use Waffle.Actions.Url 32 | end 33 | end 34 | end 35 | -------------------------------------------------------------------------------- /lib/waffle/definition/storage.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Definition.Storage do 2 | @moduledoc ~S""" 3 | Uploader configuration. 4 | 5 | Add `use Waffle.Definition` inside your module to use it as uploader. 6 | 7 | ## Storage directory 8 | 9 | By default, the storage directory is `uploads`. But, it can be customized 10 | in two ways. 11 | 12 | ### By setting up configuration 13 | 14 | Customize storage directory via configuration option `:storage_dir`. 15 | 16 | config :waffle, 17 | storage_dir: "my/dir" 18 | 19 | ### By overriding the relevant functions in definition modules 20 | 21 | Every definition module has a default `storage_dir/2` which is overridable. 22 | 23 | For example, a common pattern for user avatars is to store each user's 24 | uploaded images in a separate subdirectory based on primary key: 25 | 26 | def storage_dir(version, {file, scope}) do 27 | "uploads/users/avatars/#{scope.id}" 28 | end 29 | 30 | > **Note**: If you are "attaching" a file to a record on creation (eg, while inserting the record at the same time), then you cannot use the model's `id` as a path component. You must either (1) use a different storage path format, such as UUIDs, or (2) attach and update the model after an id has been given. [Read more about how to integrate it with Ecto](https://hexdocs.pm/waffle_ecto/filepath-with-id.html#content) 31 | 32 | > **Note**: The storage directory is used for both local filestorage (as the relative or absolute directory), and S3 storage, as the path name (not including the bucket). 33 | 34 | ## Asynchronous File Uploading 35 | 36 | If you specify multiple versions in your definition module, each 37 | version is processed and stored concurrently as independent Tasks. 38 | To prevent an overconsumption of system resources, each Task is 39 | given a specified timeout to wait, after which the process will 40 | fail. By default, the timeout is `15_000` milliseconds. 41 | 42 | If you wish to change the time allocated to version transformation 43 | and storage, you can add a configuration option: 44 | 45 | config :waffle, 46 | :version_timeout, 15_000 # milliseconds 47 | 48 | To disable asynchronous processing, add `@async false` to your 49 | definition module. 50 | 51 | ## Storage of files 52 | 53 | Waffle currently supports: 54 | 55 | * `Waffle.Storage.Local` 56 | * `Waffle.Storage.S3` 57 | 58 | Override the `__storage` function in your definition module if you 59 | want to use a different type of storage for a particular uploader. 60 | 61 | ## File Validation 62 | 63 | While storing files on S3 eliminates some malicious attack vectors, 64 | it is strongly encouraged to validate the extensions of uploaded 65 | files as well. 66 | 67 | Waffle delegates validation to a `validate/1` function with a tuple 68 | of the file and scope. As an example, in order to validate that an 69 | uploaded file conforms to popular image formats, you can use: 70 | 71 | defmodule Avatar do 72 | use Waffle.Definition 73 | @extension_whitelist ~w(.jpg .jpeg .gif .png) 74 | 75 | def validate({file, _}) do 76 | file_extension = file.file_name |> Path.extname() |> String.downcase() 77 | 78 | case Enum.member?(@extension_whitelist, file_extension) do 79 | true -> :ok 80 | false -> {:error, "invalid file type"} 81 | end 82 | end 83 | end 84 | 85 | Validation will be considered successful if the function returns `true` or `:ok`. 86 | A customized error message can be returned in the form of `{:error, message}`. 87 | Any other return value will return `{:error, :invalid_file}` when passed through 88 | to `Avatar.store`. 89 | 90 | ## Passing custom headers when downloading from remote path 91 | 92 | By default, when downloading files from remote path request headers are empty, 93 | but if you wish to provide your own, you can override the `remote_file_headers/1` 94 | function in your definition module. For example: 95 | 96 | defmodule Avatar do 97 | use Waffle.Definition 98 | 99 | def remote_file_headers(%URI{host: "elixir-lang.org"}) do 100 | credentials = Application.get_env(:my_app, :avatar_credentials) 101 | token = Base.encode64(credentials[:username] <> ":" <> credentials[:password]) 102 | 103 | [{"Authorization", "Basic #{token}")}] 104 | end 105 | end 106 | 107 | This code would authenticate request only for specific domain. Otherwise, it would send 108 | empty request headers. 109 | 110 | """ 111 | defmacro __using__(_) do 112 | quote do 113 | @acl :private 114 | @async true 115 | 116 | def bucket, do: Application.fetch_env!(:waffle, :bucket) 117 | def bucket({_file, _scope}), do: bucket() 118 | def asset_host, do: Application.get_env(:waffle, :asset_host) 119 | def filename(_, {file, _}), do: Path.basename(file.file_name, Path.extname(file.file_name)) 120 | def storage_dir_prefix, do: Application.get_env(:waffle, :storage_dir_prefix, "") 121 | def storage_dir(_, _), do: Application.get_env(:waffle, :storage_dir, "uploads") 122 | def validate(_), do: true 123 | def default_url(version, _), do: default_url(version) 124 | def default_url(_), do: nil 125 | def __storage, do: Application.get_env(:waffle, :storage, Waffle.Storage.S3) 126 | 127 | defoverridable storage_dir_prefix: 0, 128 | storage_dir: 2, 129 | filename: 2, 130 | validate: 1, 131 | default_url: 1, 132 | default_url: 2, 133 | __storage: 0, 134 | bucket: 0, 135 | bucket: 1, 136 | asset_host: 0 137 | 138 | @before_compile Waffle.Definition.Storage 139 | end 140 | end 141 | 142 | defmacro __before_compile__(_env) do 143 | quote do 144 | def acl(_, _), do: @acl 145 | def s3_object_headers(_, _), do: [] 146 | def async, do: @async 147 | def remote_file_headers(_), do: [] 148 | end 149 | end 150 | end 151 | -------------------------------------------------------------------------------- /lib/waffle/definition/versioning.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Definition.Versioning do 2 | @moduledoc ~S""" 3 | Define proper name for a version. 4 | 5 | It may be undesirable to retain original filenames (eg, it may 6 | contain personally identifiable information, vulgarity, 7 | vulnerabilities with Unicode characters, etc). 8 | 9 | You may specify the destination filename for uploaded versions 10 | through your definition module. 11 | 12 | A common pattern is to combine directories scoped to a particular 13 | model's primary key, along with static filenames. (eg: 14 | `user_avatars/1/thumb.png`). 15 | 16 | # To retain the original filename, but prefix the version and user id: 17 | def filename(version, {file, scope}) do 18 | file_name = Path.basename(file.file_name, Path.extname(file.file_name)) 19 | "#{scope.id}_#{version}_#{file_name}" 20 | end 21 | 22 | # To make the destination file the same as the version: 23 | def filename(version, _), do: version 24 | 25 | """ 26 | 27 | defmacro __using__(_) do 28 | quote do 29 | @versions [:original] 30 | @before_compile Waffle.Definition.Versioning 31 | end 32 | end 33 | 34 | def resolve_file_name(definition, version, {file, scope}) do 35 | name = definition.filename(version, {file, scope}) 36 | conversion = definition.transform(version, {file, scope}) 37 | 38 | case conversion do 39 | :skip -> 40 | nil 41 | 42 | {_, _, ext} -> 43 | [name, ext] |> Enum.join(".") 44 | 45 | {fn_transform, fn_extension} 46 | when is_function(fn_transform) and is_function(fn_extension) -> 47 | [name, fn_extension.(version, file)] |> Enum.join(".") 48 | 49 | _ -> 50 | [name, Path.extname(file.file_name)] |> Enum.join() 51 | end 52 | end 53 | 54 | defmacro __before_compile__(_env) do 55 | quote do 56 | def transform(_, _), do: :noaction 57 | def __versions, do: @versions 58 | end 59 | end 60 | end 61 | -------------------------------------------------------------------------------- /lib/waffle/exceptions.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.MissingExecutableError do 2 | defexception [:message] 3 | 4 | def exception(opts) do 5 | message = Keyword.fetch!(opts, :message) 6 | 7 | msg = """ 8 | Cannot locate executable: #{message} 9 | """ 10 | 11 | %__MODULE__{message: msg} 12 | end 13 | end 14 | -------------------------------------------------------------------------------- /lib/waffle/file.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.File do 2 | @moduledoc false 3 | 4 | defstruct [:path, :file_name, :binary, :is_tempfile?, :stream] 5 | 6 | def generate_temporary_path(item \\ nil) do 7 | do_generate_temporary_path(item) 8 | end 9 | 10 | # 11 | # Handle a remote file 12 | # 13 | 14 | # Given a remote file 15 | # (respects content-disposition header) 16 | def new(remote_path = "http" <> _, definition) do 17 | uri = URI.parse(remote_path) 18 | filename = uri.path |> Path.basename() |> URI.decode() 19 | 20 | case save_file(uri, filename, definition) do 21 | {:ok, local_path, filename_from_content_disposition} -> 22 | %Waffle.File{ 23 | path: local_path, 24 | file_name: filename_from_content_disposition, 25 | is_tempfile?: true 26 | } 27 | 28 | {:ok, local_path} -> 29 | %Waffle.File{path: local_path, file_name: filename, is_tempfile?: true} 30 | 31 | {:error, _reason} = err -> 32 | err 33 | 34 | :error -> 35 | {:error, :invalid_file_path} 36 | end 37 | end 38 | 39 | # Given a remote file with a filename 40 | def new( 41 | %{filename: filename, remote_path: remote_path} = %{filename: _, remote_path: "http" <> _}, 42 | definition 43 | ) do 44 | uri = URI.parse(remote_path) 45 | 46 | case save_file(uri, filename, definition) do 47 | {:ok, local_path} -> 48 | %Waffle.File{path: local_path, file_name: filename, is_tempfile?: true} 49 | 50 | {:error, _reason} = err -> 51 | err 52 | 53 | :error -> 54 | {:error, :invalid_file_path} 55 | end 56 | end 57 | 58 | # Rejects invalid remote file path 59 | def new( 60 | %{filename: _filename, remote_path: _remote_path} = %{filename: _, remote_path: _}, 61 | _definition 62 | ) do 63 | {:error, :invalid_file_path} 64 | end 65 | 66 | # 67 | # Handle a binary blob 68 | # 69 | 70 | def new(%{filename: filename, binary: binary}, _definition) do 71 | %Waffle.File{binary: binary, file_name: Path.basename(filename)} 72 | |> write_binary() 73 | end 74 | 75 | # 76 | # Handle a local file 77 | # 78 | 79 | # Accepts a path 80 | def new(path, _definition) when is_binary(path) do 81 | case File.exists?(path) do 82 | true -> %Waffle.File{path: path, file_name: Path.basename(path)} 83 | false -> {:error, :invalid_file_path} 84 | end 85 | end 86 | 87 | # Accepts a map conforming to %Plug.Upload{} syntax 88 | def new(%{filename: filename, path: path}, _definition) do 89 | case File.exists?(path) do 90 | true -> %Waffle.File{path: path, file_name: filename} 91 | false -> {:error, :invalid_file_path} 92 | end 93 | end 94 | 95 | # 96 | # Handle a stream 97 | # 98 | def new(%{filename: filename, stream: stream}, _definition) when is_struct(stream) do 99 | %Waffle.File{stream: stream, file_name: Path.basename(filename)} 100 | end 101 | 102 | # 103 | # Support functions 104 | # 105 | 106 | # 107 | # 108 | # Temp file with exact extension. 109 | # Used for converting formats when passing extension in transformations 110 | # 111 | 112 | defp do_generate_temporary_path(%Waffle.File{path: path}) do 113 | Path.extname(path || "") 114 | |> do_generate_temporary_path() 115 | end 116 | 117 | defp do_generate_temporary_path(extension) do 118 | ext = extension |> to_string() 119 | 120 | string_extension = 121 | cond do 122 | String.starts_with?(ext, ".") -> 123 | ext 124 | 125 | ext == "" -> 126 | "" 127 | 128 | true -> 129 | ".#{ext}" 130 | end 131 | 132 | file_name = 133 | :crypto.strong_rand_bytes(20) 134 | |> Base.encode32() 135 | |> Kernel.<>(string_extension) 136 | 137 | Path.join(System.tmp_dir(), file_name) 138 | end 139 | 140 | defp write_binary(file) do 141 | path = generate_temporary_path(file) 142 | File.write!(path, file.binary) 143 | 144 | %__MODULE__{ 145 | file_name: file.file_name, 146 | path: path, 147 | is_tempfile?: true 148 | } 149 | end 150 | 151 | defp save_file(uri, filename, definition) do 152 | local_path = 153 | generate_temporary_path() 154 | |> Kernel.<>(Path.extname(filename)) 155 | 156 | case save_temp_file(local_path, uri, definition) do 157 | {:ok, filename} -> {:ok, local_path, filename} 158 | :ok -> {:ok, local_path} 159 | err -> err 160 | end 161 | end 162 | 163 | defp save_temp_file(local_path, remote_path, definition) do 164 | remote_file = get_remote_path(remote_path, definition) 165 | 166 | case remote_file do 167 | {:ok, body, filename} -> 168 | case File.write(local_path, body) do 169 | :ok -> {:ok, filename} 170 | _ -> :error 171 | end 172 | 173 | {:ok, body} -> 174 | File.write(local_path, body) 175 | 176 | {:error, _reason} = err -> 177 | err 178 | end 179 | end 180 | 181 | # hackney :connect_timeout - timeout used when establishing a connection, in milliseconds 182 | # hackney :recv_timeout - timeout used when receiving from a connection, in milliseconds 183 | # hackney :max_body_length - maximum size of the file to download, in bytes. Defaults to :infinity 184 | # :backoff_max - maximum backoff time, in milliseconds 185 | # :backoff_factor - a backoff factor to apply between attempts, in milliseconds 186 | defp get_remote_path(remote_path, definition) do 187 | headers = definition.remote_file_headers(remote_path) 188 | 189 | options = [ 190 | follow_redirect: true, 191 | recv_timeout: Application.get_env(:waffle, :recv_timeout, 5_000), 192 | connect_timeout: Application.get_env(:waffle, :connect_timeout, 10_000), 193 | max_retries: Application.get_env(:waffle, :max_retries, 3), 194 | backoff_factor: Application.get_env(:waffle, :backoff_factor, 1000), 195 | backoff_max: Application.get_env(:waffle, :backoff_max, 30_000) 196 | ] 197 | 198 | request(remote_path, headers, options) 199 | end 200 | 201 | defp request(remote_path, headers, options, tries \\ 0) do 202 | with {:ok, 200, response_headers, client_ref} <- 203 | :hackney.get(URI.to_string(remote_path), headers, "", options), 204 | res when elem(res, 0) == :ok <- body(client_ref, response_headers) do 205 | res 206 | else 207 | {:error, %{reason: :timeout}} -> 208 | case retry(tries, options) do 209 | {:ok, :retry} -> request(remote_path, headers, options, tries + 1) 210 | {:error, :out_of_tries} -> {:error, :timeout} 211 | end 212 | 213 | {:error, :timeout} -> 214 | case retry(tries, options) do 215 | {:ok, :retry} -> request(remote_path, headers, options, tries + 1) 216 | {:error, :out_of_tries} -> {:error, :recv_timeout} 217 | end 218 | 219 | {:ok, 503, _headers, client_ref} = response -> 220 | case retry(tries, options) do 221 | {:ok, :retry} -> 222 | request(remote_path, headers, options, tries + 1) 223 | 224 | {:error, :out_of_tries} -> 225 | :hackney.close(client_ref) 226 | {:error, {:waffle_hackney_error, response}} 227 | end 228 | 229 | {:ok, _, _, client_ref} = response -> 230 | :hackney.close(client_ref) 231 | {:error, {:waffle_hackney_error, response}} 232 | 233 | _err -> 234 | {:error, :waffle_hackney_error} 235 | end 236 | end 237 | 238 | defp body(client_ref, response_headers) do 239 | max_body_length = Application.get_env(:waffle, :max_body_length, :infinity) 240 | 241 | case :hackney.body(client_ref, max_body_length) do 242 | {:ok, body} -> 243 | response_headers = :hackney_headers.new(response_headers) 244 | filename = content_disposition(response_headers) 245 | 246 | if is_nil(filename) do 247 | {:ok, body} 248 | else 249 | {:ok, body, filename} 250 | end 251 | 252 | err -> 253 | err 254 | end 255 | end 256 | 257 | defp content_disposition(headers) do 258 | case :hackney_headers.get_value("content-disposition", headers) do 259 | :undefined -> 260 | nil 261 | 262 | value -> 263 | case :hackney_headers.content_disposition(value) do 264 | {_, [{"filename", filename} | _]} -> 265 | filename 266 | 267 | _ -> 268 | nil 269 | end 270 | end 271 | end 272 | 273 | defp retry(tries, options) do 274 | if tries < options[:max_retries] do 275 | backoff = round(options[:backoff_factor] * :math.pow(2, tries - 1)) 276 | backoff = :erlang.min(backoff, options[:backoff_max]) 277 | :timer.sleep(backoff) 278 | {:ok, :retry} 279 | else 280 | {:error, :out_of_tries} 281 | end 282 | end 283 | end 284 | -------------------------------------------------------------------------------- /lib/waffle/processor.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Processor do 2 | @moduledoc ~S""" 3 | Apply transformation to files. 4 | 5 | Waffle can be used to facilitate transformations of uploaded files 6 | via any system executable. Some common operations you may want to 7 | take on uploaded files include resizing an uploaded avatar with 8 | ImageMagick or extracting a still image from a video with FFmpeg. 9 | 10 | To transform an image, the definition module must define a 11 | `transform/2` function which accepts a version atom and a tuple 12 | consisting of the uploaded file and corresponding scope. 13 | 14 | This transform handler accepts the version atom, as well as the 15 | file/scope argument, and is responsible for returning one of the 16 | following: 17 | 18 | * `:noaction` - The original file will be stored as-is. 19 | 20 | * `:skip` - Nothing will be stored for the provided version. 21 | 22 | * `{executable, args}` - The `executable` will be called with 23 | `System.cmd` with the format 24 | `#{original_file_path} #{args} #{transformed_file_path}`. 25 | 26 | * `{executable, fn(input, output) -> args end}` If your executable 27 | expects arguments in a format other than the above, you may 28 | supply a function to the conversion tuple which will be invoked 29 | to generate the arguments. The arguments can be returned as a 30 | string (e.g. – `" #{input} -strip -thumbnail 10x10 #{output}"`) 31 | or a list (e.g. – `[input, "-strip", "-thumbnail", "10x10", 32 | output]`) for even more control. 33 | 34 | * `{executable, args, output_extension}` - If your transformation 35 | changes the file extension (eg, converting to `png`), then the 36 | new file extension must be explicit. 37 | 38 | * `fn version, file -> {:ok, file} end` - Implement custom 39 | transformation as elixir function, 40 | [read more about custom transformations](custom_transformation.livemd) 41 | 42 | * `{&transform/2, fn version, file -> :png end}` - A custom 43 | transformation converting a file into a different extension 44 | 45 | ## ImageMagick transformations 46 | 47 | As images are one of the most commonly uploaded filetypes, Waffle 48 | has a recommended integration with ImageMagick's `convert` tool for 49 | manipulation of images. Each definition module may specify as many 50 | versions as desired, along with the corresponding transformation for 51 | each version. 52 | 53 | The expected return value of a `transform` function call must either 54 | be `:noaction`, in which case the original file will be stored 55 | as-is, `:skip`, in which case nothing will be stored, or `{:convert, 56 | transformation}` in which the original file will be processed via 57 | ImageMagick's `convert` tool with the corresponding transformation 58 | parameters. 59 | 60 | The following example stores the original file, as well as a squared 61 | 100x100 thumbnail version which is stripped of comments (eg, GPS 62 | coordinates): 63 | 64 | defmodule Avatar do 65 | use Waffle.Definition 66 | 67 | @versions [:original, :thumb] 68 | 69 | def transform(:thumb, _) do 70 | {:convert, "-strip -thumbnail 100x100^ -gravity center -extent 100x100"} 71 | end 72 | end 73 | 74 | Other examples: 75 | 76 | # Change the file extension through ImageMagick's `format` parameter: 77 | {:convert, "-strip -thumbnail 100x100^ -gravity center -extent 100x100 -format png", :png} 78 | 79 | # Take the first frame of a gif and process it into a square jpg: 80 | {:convert, fn(input, output) -> "#{input}[0] -strip -thumbnail 100x100^ -gravity center -extent 100x100 -format jpg #{output}", :jpg} 81 | 82 | For more information on defining your transformation, please consult 83 | [ImageMagick's convert 84 | documentation](http://www.imagemagick.org/script/convert.php). 85 | 86 | > **Note**: Keep this transformation function simple and deterministic based on the version, file name, and scope object. The `transform` function is subsequently called during URL generation, and the transformation is scanned for the output file format. As such, if you conditionally format the image as a `png` or `jpg` depending on the time of day, you will be displeased with the result of Waffle's URL generation. 87 | 88 | > **System Resources**: If you are accepting arbitrary uploads on a public site, it may be prudent to add system resource limits to prevent overloading your system resources from malicious or nefarious files. Since all processing is done directly in ImageMagick, you may pass in system resource restrictions through the [-limit](http://www.imagemagick.org/script/command-line-options.php#limit) flag. One such example might be: `-limit area 10MB -limit disk 100MB`. 89 | 90 | ## FFmpeg transformations 91 | 92 | Common transformations of uploaded videos can be also defined 93 | through your definition module: 94 | 95 | # To take a thumbnail from a video: 96 | {:ffmpeg, fn(input, output) -> "-i #{input} -f jpg #{output}" end, :jpg} 97 | 98 | # To convert a video to an animated gif 99 | {:ffmpeg, fn(input, output) -> "-i #{input} -f gif #{output}" end, :gif} 100 | 101 | ## Complex Transformations 102 | 103 | `Waffle` requires the output of your transformation to be located at 104 | a predetermined path. However, the transformation may be done 105 | completely outside of `Waffle`. For fine-grained transformations, 106 | you should create an executable wrapper in your $PATH (eg. bash 107 | script) which takes these proper arguments, runs your 108 | transformation, and then moves the file into the correct location. 109 | 110 | For example, to use `soffice` to convert a doc to an html file, you 111 | should place the following bash script in your $PATH: 112 | 113 | #!/usr/bin/env sh 114 | 115 | # `soffice` doesn't allow for output file path option, and waffle can't find the 116 | # temporary file to process and copy. This script has a similar argument list as 117 | # what waffle expects. See https://github.com/stavro/arc/issues/77. 118 | 119 | set -e 120 | set -o pipefail 121 | 122 | function convert { 123 | soffice \ 124 | --headless \ 125 | --convert-to html \ 126 | --outdir $TMPDIR \ 127 | "$1" 128 | } 129 | 130 | function filter_new_file_name { 131 | awk -F$TMPDIR '{print $2}' \ 132 | | awk -F" " '{print $1}' \ 133 | | awk -F/ '{print $2}' 134 | } 135 | 136 | converted_file_name=$(convert "$1" | filter_new_file_name) 137 | 138 | cp $TMPDIR/$converted_file_name "$2" 139 | rm $TMPDIR/$converted_file_name 140 | 141 | And perform the transformation as such: 142 | 143 | def transform(:html, _) do 144 | {:soffice_wrapper, fn(input, output) -> [input, output] end, :html} 145 | end 146 | 147 | """ 148 | alias Waffle.Transformations.Convert 149 | 150 | def process(definition, version, {file, scope}) do 151 | transform = definition.transform(version, {file, scope}) 152 | apply_transformation(file, transform, version) 153 | end 154 | 155 | @spec apply_transformation( 156 | Waffle.File.t(), 157 | (Waffle.File.t() -> {:ok, Waffle.File.t()} | {:error, String.t()}), 158 | atom() 159 | ) :: {:ok, Waffle.File.t()} | {:error, String.t()} 160 | 161 | defp apply_transformation(_, :skip, _), do: {:ok, nil} 162 | defp apply_transformation(file, :noaction, _), do: {:ok, file} 163 | # Deprecated 164 | defp apply_transformation(file, {:noaction}, _), do: {:ok, file} 165 | 166 | defp apply_transformation(file, func, version) when is_function(func), do: func.(version, file) 167 | 168 | defp apply_transformation(file, {func, _}, version) when is_function(func), 169 | do: func.(version, file) 170 | 171 | defp apply_transformation(file, {cmd, conversion}, _) do 172 | Convert.apply(cmd, file, conversion) 173 | end 174 | 175 | defp apply_transformation(file, {cmd, conversion, extension}, _) do 176 | Convert.apply(cmd, file, conversion, extension) 177 | end 178 | end 179 | -------------------------------------------------------------------------------- /lib/waffle/storage/local.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Storage.Local do 2 | @moduledoc ~S""" 3 | Local storage provides facility to store files locally. 4 | 5 | ## Local configuration 6 | 7 | config :waffle, 8 | storage: Waffle.Storage.Local, 9 | # in order to have a different storage directory from url 10 | storage_dir_prefix: "priv/waffle/private", 11 | # add custom host to url 12 | asset_host: "https://example.com" 13 | 14 | If you want to handle your attachments by phoenix application, 15 | configure the endpoint to serve it. 16 | 17 | defmodule AppWeb.Endpoint do 18 | plug Plug.Static, 19 | at: "/uploads", 20 | from: Path.expand("./priv/waffle/public/uploads"), 21 | gzip: false 22 | end 23 | """ 24 | 25 | @behaviour Waffle.StorageBehavior 26 | 27 | alias Waffle.Definition.Versioning 28 | 29 | @impl true 30 | def put(definition, version, {file, scope}) do 31 | destination_path = Path.join([ 32 | definition.storage_dir_prefix(), 33 | definition.storage_dir(version, {file, scope}), 34 | file.file_name 35 | ]) 36 | destination_path |> Path.dirname() |> File.mkdir_p!() 37 | 38 | if binary = file.binary do 39 | File.write!(destination_path, binary) 40 | else 41 | File.copy!(file.path, destination_path) 42 | end 43 | 44 | {:ok, file.file_name} 45 | end 46 | 47 | @impl true 48 | def url(definition, version, file_and_scope, _options \\ []) do 49 | local_path = Path.join([ 50 | definition.storage_dir(version, file_and_scope), 51 | Versioning.resolve_file_name(definition, version, file_and_scope) 52 | ]) 53 | host = host(definition) 54 | 55 | if host == nil do 56 | Path.join("/", local_path) 57 | else 58 | Path.join([host, local_path]) 59 | end 60 | |> URI.encode() 61 | end 62 | 63 | @impl true 64 | def delete(definition, version, file_and_scope) do 65 | Path.join([ 66 | definition.storage_dir_prefix(), 67 | definition.storage_dir(version, file_and_scope), 68 | Versioning.resolve_file_name(definition, version, file_and_scope) 69 | ]) 70 | |> File.rm() 71 | end 72 | 73 | defp host(definition) do 74 | case definition.asset_host() do 75 | {:system, env_var} when is_binary(env_var) -> System.get_env(env_var) 76 | url -> url 77 | end 78 | end 79 | end 80 | -------------------------------------------------------------------------------- /lib/waffle/storage/s3.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Storage.S3 do 2 | @moduledoc ~S""" 3 | The module to facilitate integration with S3 through ExAws.S3 4 | 5 | config :waffle, 6 | storage: Waffle.Storage.S3, 7 | bucket: {:system, "AWS_S3_BUCKET"} 8 | 9 | Along with any configuration necessary for ExAws. 10 | 11 | [ExAws](https://github.com/CargoSense/ex_aws) is used to support Amazon S3. 12 | 13 | To store your attachments in Amazon S3, you'll need to provide a 14 | bucket destination in your application config: 15 | 16 | config :waffle, 17 | bucket: "uploads" 18 | 19 | You may also set the bucket from an environment variable: 20 | 21 | config :waffle, 22 | bucket: {:system, "S3_BUCKET"} 23 | 24 | In addition, ExAws must be configured with the appropriate Amazon S3 25 | credentials. 26 | 27 | ExAws has by default the following configuration (which you may 28 | override if you wish): 29 | 30 | config :ex_aws, 31 | json_codec: Jason, 32 | access_key_id: [{:system, "AWS_ACCESS_KEY_ID"}, :instance_role], 33 | secret_access_key: [{:system, "AWS_SECRET_ACCESS_KEY"}, :instance_role] 34 | 35 | This means it will first look for the AWS standard 36 | `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment 37 | variables, and fall back using instance meta-data if those don't 38 | exist. You should set those environment variables to your 39 | credentials, or configure an instance that this library runs on to 40 | have an iam role. 41 | 42 | ## Specify multiple buckets 43 | 44 | Waffle lets you specify a bucket on a per definition basis. In case 45 | you want to use multiple buckets, you can specify a bucket in the 46 | definition module like this: 47 | 48 | def bucket, do: :some_custom_bucket_name 49 | 50 | You can also use the current scope to define a target bucket 51 | 52 | def bucket({_file, scope}), do: scope.bucket || bucket() 53 | 54 | ## Access Control Permissions 55 | 56 | Waffle defaults all uploads to `private`. In cases where it is 57 | desired to have your uploads public, you may set the ACL at the 58 | module level (which applies to all versions): 59 | 60 | @acl :public_read 61 | 62 | Or you may have more granular control over each version. As an 63 | example, you may wish to explicitly only make public a thumbnail 64 | version of the file: 65 | 66 | def acl(:thumb, _), do: :public_read 67 | 68 | Supported access control lists for Amazon S3 are: 69 | 70 | | ACL | Permissions Added to ACL | 71 | |------------------------------|---------------------------------------------------------------------------------| 72 | | `:private` | Owner gets `FULL_CONTROL`. No one else has access rights (default). | 73 | | `:public_read` | Owner gets `FULL_CONTROL`. The `AllUsers` group gets READ access. | 74 | | `:public_read_write` | Owner gets `FULL_CONTROL`. The `AllUsers` group gets `READ` and `WRITE` access. | 75 | | | Granting this on a bucket is generally not recommended. | 76 | | `:authenticated_read` | Owner gets `FULL_CONTROL`. The `AuthenticatedUsers` group gets `READ` access. | 77 | | `:bucket_owner_read` | Object owner gets `FULL_CONTROL`. Bucket owner gets `READ` access. | 78 | | `:bucket_owner_full_control` | Both the object owner and the bucket owner get `FULL_CONTROL` over the object. | 79 | 80 | For more information on the behavior of each of these, please 81 | consult Amazon's documentation for [Access Control List (ACL) 82 | Overview](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). 83 | 84 | ## S3 Object Headers 85 | 86 | The definition module may specify custom headers to pass through to 87 | S3 during object creation. The available custom headers include: 88 | 89 | * `:cache_control` 90 | * `:content_disposition` 91 | * `:content_encoding` 92 | * `:content_length` 93 | * `:content_type` 94 | * `:expect` 95 | * `:expires` 96 | * `:storage_class` 97 | * `:website_redirect_location` 98 | * `:encryption` (set to "AES256" for encryption at rest) 99 | 100 | As an example, to explicitly specify the content-type of an object, 101 | you may define a `s3_object_headers/2` function in your definition, 102 | which returns a Keyword list, or Map of desired headers. 103 | 104 | def s3_object_headers(version, {file, scope}) do 105 | [content_type: MIME.from_path(file.file_name)] # for "image.png", would produce: "image/png" 106 | end 107 | 108 | ## Alternate S3 configuration example 109 | 110 | If you are using a region other than US-Standard, it is necessary to 111 | specify the correct configuration for `ex_aws`. A full example 112 | configuration for both waffle and ex_aws is as follows: 113 | 114 | config :waffle, 115 | bucket: "my-frankfurt-bucket" 116 | 117 | config :ex_aws, 118 | json_codec: Jason, 119 | access_key_id: "my_access_key_id", 120 | secret_access_key: "my_secret_access_key", 121 | region: "eu-central-1", 122 | s3: [ 123 | scheme: "https://", 124 | host: "s3.eu-central-1.amazonaws.com", 125 | region: "eu-central-1" 126 | ] 127 | 128 | > For your host configuration, please examine the approved [AWS Hostnames](http://docs.aws.amazon.com/general/latest/gr/rande.html). There are often multiple hostname formats for AWS regions, and it will not work unless you specify the correct one. 129 | 130 | """ 131 | require Logger 132 | 133 | @behaviour Waffle.StorageBehavior 134 | 135 | alias ExAws.Config 136 | alias ExAws.Request.Url 137 | alias ExAws.S3 138 | alias ExAws.S3.Upload 139 | alias Waffle.Definition.Versioning 140 | 141 | @default_expiry_time 60 * 5 142 | 143 | @impl true 144 | def put(definition, version, {file, scope}) do 145 | destination_dir = definition.storage_dir(version, {file, scope}) 146 | s3_bucket = s3_bucket(definition, {file, scope}) 147 | s3_key = Path.join(destination_dir, file.file_name) 148 | acl = definition.acl(version, {file, scope}) 149 | 150 | s3_options = 151 | definition.s3_object_headers(version, {file, scope}) 152 | |> ensure_keyword_list() 153 | |> Keyword.put(:acl, acl) 154 | 155 | do_put(file, {s3_bucket, s3_key, s3_options}) 156 | end 157 | 158 | @impl true 159 | def url(definition, version, file_and_scope, options \\ []) do 160 | case Keyword.get(options, :signed, false) do 161 | false -> build_url(definition, version, file_and_scope, options) 162 | true -> build_signed_url(definition, version, file_and_scope, options) 163 | end 164 | end 165 | 166 | @impl true 167 | def delete(definition, version, {file, scope}) do 168 | s3_bucket(definition, {file, scope}) 169 | |> S3.delete_object(s3_key(definition, version, {file, scope})) 170 | |> ExAws.request() 171 | 172 | :ok 173 | end 174 | 175 | def s3_key(definition, version, file_and_scope) do 176 | Path.join([ 177 | definition.storage_dir(version, file_and_scope), 178 | Versioning.resolve_file_name(definition, version, file_and_scope) 179 | ]) 180 | end 181 | 182 | # 183 | # Private 184 | # 185 | 186 | defp ensure_keyword_list(list) when is_list(list), do: list 187 | defp ensure_keyword_list(map) when is_map(map), do: Map.to_list(map) 188 | 189 | # If the file is stored as a binary in-memory, send to AWS in a single request 190 | defp do_put(file = %Waffle.File{binary: file_binary}, {s3_bucket, s3_key, s3_options}) 191 | when is_binary(file_binary) do 192 | S3.put_object(s3_bucket, s3_key, file_binary, s3_options) 193 | |> ExAws.request() 194 | |> case do 195 | {:ok, _res} -> {:ok, file.file_name} 196 | {:error, error} -> {:error, error} 197 | end 198 | end 199 | 200 | # If the file is a stream, send it to AWS as a multi-part upload 201 | defp do_put(file = %Waffle.File{stream: file_stream}, {s3_bucket, s3_key, s3_options}) 202 | when is_struct(file_stream) do 203 | file_stream 204 | |> chunk_stream() 205 | |> do_put_stream(file, {s3_bucket, s3_key, s3_options}) 206 | end 207 | 208 | # Stream the file and upload to AWS as a multi-part upload 209 | defp do_put(file, {s3_bucket, s3_key, s3_options}) do 210 | file.path 211 | |> Upload.stream_file() 212 | |> do_put_stream(file, {s3_bucket, s3_key, s3_options}) 213 | end 214 | 215 | defp do_put_stream(stream, file, {s3_bucket, s3_key, s3_options}) do 216 | stream 217 | |> S3.upload(s3_bucket, s3_key, s3_options) 218 | |> ExAws.request() 219 | |> case do 220 | {:ok, %{status_code: 200}} -> {:ok, file.file_name} 221 | {:ok, :done} -> {:ok, file.file_name} 222 | {:error, error} -> {:error, error} 223 | end 224 | rescue 225 | e in ExAws.Error -> 226 | Logger.error(inspect(e)) 227 | Logger.error(e.message) 228 | {:error, :invalid_bucket} 229 | end 230 | 231 | defp build_url(definition, version, file_and_scope, _options) do 232 | asset_path = 233 | s3_key(definition, version, file_and_scope) 234 | |> Url.sanitize(:s3) 235 | 236 | Path.join(host(definition, file_and_scope), asset_path) 237 | end 238 | 239 | defp build_signed_url(definition, version, file_and_scope, options) do 240 | # Previous waffle argument was expire_in instead of expires_in 241 | # check for expires_in, if not present, use expire_at. 242 | options = put_in(options[:expires_in], Keyword.get(options, :expires_in, options[:expire_in])) 243 | # fallback to default, if neither is present. 244 | options = put_in(options[:expires_in], options[:expires_in] || @default_expiry_time) 245 | options = put_in(options[:virtual_host], virtual_host()) 246 | config = Config.new(:s3, Application.get_all_env(:ex_aws)) 247 | s3_key = s3_key(definition, version, file_and_scope) 248 | s3_bucket = s3_bucket(definition, file_and_scope) 249 | {:ok, url} = S3.presigned_url(config, :get, s3_bucket, s3_key, options) 250 | url 251 | end 252 | 253 | defp host(definition, file_and_scope) do 254 | case asset_host(definition, file_and_scope) do 255 | {:system, env_var} when is_binary(env_var) -> System.get_env(env_var) 256 | url -> url 257 | end 258 | end 259 | 260 | defp asset_host(definition, file_and_scope) do 261 | case definition.asset_host() do 262 | false -> default_host(definition, file_and_scope) 263 | nil -> default_host(definition, file_and_scope) 264 | host -> host 265 | end 266 | end 267 | 268 | defp default_host(definition, file_and_scope) do 269 | case virtual_host() do 270 | true -> "https://#{s3_bucket(definition, file_and_scope)}.s3.amazonaws.com" 271 | _ -> "https://s3.amazonaws.com/#{s3_bucket(definition, file_and_scope)}" 272 | end 273 | end 274 | 275 | defp virtual_host do 276 | Application.get_env(:waffle, :virtual_host) || false 277 | end 278 | 279 | defp s3_bucket(definition, file_and_scope) do 280 | definition.bucket(file_and_scope) |> parse_bucket() 281 | end 282 | 283 | defp parse_bucket({:system, env_var}) when is_binary(env_var), do: System.get_env(env_var) 284 | defp parse_bucket(name), do: name 285 | 286 | defp chunk_stream(stream, chunk_size \\ 5 * 1024 * 1024) do 287 | Stream.chunk_while( 288 | stream, 289 | "", 290 | fn element, acc -> 291 | if String.length(acc) >= chunk_size do 292 | {:cont, acc, element} 293 | else 294 | {:cont, acc <> element} 295 | end 296 | end, 297 | fn 298 | [] -> {:cont, []} 299 | acc -> {:cont, acc, []} 300 | end 301 | ) 302 | end 303 | end 304 | -------------------------------------------------------------------------------- /lib/waffle/transformations/convert.ex: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Transformations.Convert do 2 | @moduledoc false 3 | 4 | def apply(cmd, file, args, extension \\ nil) do 5 | new_path = 6 | if extension, 7 | do: Waffle.File.generate_temporary_path(extension), 8 | else: Waffle.File.generate_temporary_path(file) 9 | 10 | args = 11 | if is_function(args), 12 | do: args.(file.path, new_path), 13 | else: [file.path | String.split(args, " ") ++ [new_path]] 14 | 15 | program = to_string(cmd) 16 | 17 | ensure_executable_exists!(program) 18 | 19 | result = System.cmd(program, args_list(args), stderr_to_stdout: true) 20 | 21 | case result do 22 | {_, 0} -> 23 | {:ok, %Waffle.File{file | path: new_path, is_tempfile?: true}} 24 | 25 | {error_message, _exit_code} -> 26 | {:error, error_message} 27 | end 28 | end 29 | 30 | defp args_list(args) when is_list(args), do: args 31 | defp args_list(args), do: ~w(#{args}) 32 | 33 | defp ensure_executable_exists!(program) do 34 | unless System.find_executable(program) do 35 | raise Waffle.MissingExecutableError, message: program 36 | end 37 | end 38 | end 39 | -------------------------------------------------------------------------------- /mix.exs: -------------------------------------------------------------------------------- 1 | defmodule Waffle.Mixfile do 2 | use Mix.Project 3 | 4 | @version "1.1.9" 5 | 6 | def project do 7 | [ 8 | app: :waffle, 9 | version: @version, 10 | elixir: "~> 1.4", 11 | source_url: "https://github.com/elixir-waffle/waffle", 12 | deps: deps(), 13 | docs: docs(), 14 | 15 | # Hex 16 | description: description(), 17 | package: package() 18 | ] 19 | end 20 | 21 | defp description do 22 | """ 23 | Flexible file upload and attachment library for Elixir. 24 | """ 25 | end 26 | 27 | defp package do 28 | [ 29 | maintainers: ["Boris Kuznetsov"], 30 | licenses: ["Apache 2.0"], 31 | links: %{"GitHub" => "https://github.com/elixir-waffle/waffle"}, 32 | files: ~w(mix.exs README.md CHANGELOG.md lib) 33 | ] 34 | end 35 | 36 | defp docs do 37 | [ 38 | main: "readme", 39 | source_ref: "v#{@version}", 40 | extras: [ 41 | "README.md", 42 | "documentation/examples/local.md", 43 | "documentation/examples/s3.md", 44 | "documentation/livebooks/custom_transformation.livemd" 45 | ] 46 | ] 47 | end 48 | 49 | def application do 50 | [ 51 | extra_applications: [ 52 | :logger, 53 | # Used by Mix.generator.embed_template/2 54 | :eex 55 | ] 56 | ] 57 | end 58 | 59 | defp deps do 60 | [ 61 | {:hackney, "~> 1.9"}, 62 | 63 | # If using Amazon S3 64 | {:ex_aws, "~> 2.1", optional: true}, 65 | {:ex_aws_s3, "~> 2.1", optional: true}, 66 | {:sweet_xml, "~> 0.6", optional: true}, 67 | 68 | # Test 69 | {:mock, "~> 0.3", only: :test}, 70 | 71 | # Dev 72 | {:ex_doc, "~> 0.21", only: :dev}, 73 | 74 | # Dev, Test 75 | {:credo, "~> 1.4", only: [:dev, :test], runtime: false} 76 | ] 77 | end 78 | end 79 | -------------------------------------------------------------------------------- /mix.lock: -------------------------------------------------------------------------------- 1 | %{ 2 | "bunt": {:hex, :bunt, "0.2.1", "e2d4792f7bc0ced7583ab54922808919518d0e57ee162901a16a1b6664ef3b14", [:mix], [], "hexpm", "a330bfb4245239787b15005e66ae6845c9cd524a288f0d141c148b02603777a5"}, 3 | "certifi": {:hex, :certifi, "2.9.0", "6f2a475689dd47f19fb74334859d460a2dc4e3252a3324bd2111b8f0429e7e21", [:rebar3], [], "hexpm", "266da46bdb06d6c6d35fde799bcb28d36d985d424ad7c08b5bb48f5b5cdd4641"}, 4 | "credo": {:hex, :credo, "1.6.7", "323f5734350fd23a456f2688b9430e7d517afb313fbd38671b8a4449798a7854", [:mix], [{:bunt, "~> 0.2.1", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2.8", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "41e110bfb007f7eda7f897c10bf019ceab9a0b269ce79f015d54b0dcf4fc7dd3"}, 5 | "earmark": {:hex, :earmark, "1.4.4", "4821b8d05cda507189d51f2caeef370cf1e18ca5d7dfb7d31e9cafe6688106a4", [:mix], [], "hexpm"}, 6 | "earmark_parser": {:hex, :earmark_parser, "1.4.39", "424642f8335b05bb9eb611aa1564c148a8ee35c9c8a8bba6e129d51a3e3c6769", [:mix], [], "hexpm", "06553a88d1f1846da9ef066b87b57c6f605552cfbe40d20bd8d59cc6bde41944"}, 7 | "ex_aws": {:hex, :ex_aws, "2.4.1", "d1dc8965d1dc1c939dd4570e37f9f1d21e047e4ecd4f9373dc89cd4e45dce5ef", [:mix], [{:configparser_ex, "~> 4.0", [hex: :configparser_ex, repo: "hexpm", optional: true]}, {:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:jsx, "~> 2.8 or ~> 3.0", [hex: :jsx, repo: "hexpm", optional: true]}, {:mime, "~> 1.2 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:sweet_xml, "~> 0.7", [hex: :sweet_xml, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.3 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "803387db51b4e91be4bf0110ba999003ec6103de7028b808ee9b01f28dbb9eee"}, 8 | "ex_aws_s3": {:hex, :ex_aws_s3, "2.4.0", "ce8decb6b523381812798396bc0e3aaa62282e1b40520125d1f4eff4abdff0f4", [:mix], [{:ex_aws, "~> 2.0", [hex: :ex_aws, repo: "hexpm", optional: false]}, {:sweet_xml, ">= 0.0.0", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm", "85dda6e27754d94582869d39cba3241d9ea60b6aa4167f9c88e309dc687e56bb"}, 9 | "ex_doc": {:hex, :ex_doc, "0.34.0", "ab95e0775db3df71d30cf8d78728dd9261c355c81382bcd4cefdc74610bef13e", [:mix], [{:earmark_parser, "~> 1.4.39", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_c, ">= 0.1.0", [hex: :makeup_c, repo: "hexpm", optional: true]}, {:makeup_elixir, "~> 0.14 or ~> 1.0", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1 or ~> 1.0", [hex: :makeup_erlang, repo: "hexpm", optional: false]}, {:makeup_html, ">= 0.1.0", [hex: :makeup_html, repo: "hexpm", optional: true]}], "hexpm", "60734fb4c1353f270c3286df4a0d51e65a2c1d9fba66af3940847cc65a8066d7"}, 10 | "file_system": {:hex, :file_system, "0.2.10", "fb082005a9cd1711c05b5248710f8826b02d7d1784e7c3451f9c1231d4fc162d", [:mix], [], "hexpm", "41195edbfb562a593726eda3b3e8b103a309b733ad25f3d642ba49696bf715dc"}, 11 | "hackney": {:hex, :hackney, "1.18.1", "f48bf88f521f2a229fc7bae88cf4f85adc9cd9bcf23b5dc8eb6a1788c662c4f6", [:rebar3], [{:certifi, "~> 2.9.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~> 6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~> 1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~> 1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.3.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~> 1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "a4ecdaff44297e9b5894ae499e9a070ea1888c84afdd1fd9b7b2bc384950128e"}, 12 | "idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"}, 13 | "jason": {:hex, :jason, "1.4.0", "e855647bc964a44e2f67df589ccf49105ae039d4179db7f6271dfd3843dc27e6", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "79a3791085b2a0f743ca04cec0f7be26443738779d09302e01318f97bdb82121"}, 14 | "makeup": {:hex, :makeup, "1.1.2", "9ba8837913bdf757787e71c1581c21f9d2455f4dd04cfca785c70bbfff1a76a3", [:mix], [{:nimble_parsec, "~> 1.2.2 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "cce1566b81fbcbd21eca8ffe808f33b221f9eee2cbc7a1706fc3da9ff18e6cac"}, 15 | "makeup_elixir": {:hex, :makeup_elixir, "0.16.2", "627e84b8e8bf22e60a2579dad15067c755531fea049ae26ef1020cad58fe9578", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.2.3 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "41193978704763f6bbe6cc2758b84909e62984c7752b3784bd3c218bb341706b"}, 16 | "makeup_erlang": {:hex, :makeup_erlang, "1.0.0", "6f0eff9c9c489f26b69b61440bf1b238d95badae49adac77973cbacae87e3c2e", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm", "ea7a9307de9d1548d2a72d299058d1fd2339e3d398560a0e46c27dab4891e4d2"}, 17 | "meck": {:hex, :meck, "0.9.2", "85ccbab053f1db86c7ca240e9fc718170ee5bda03810a6292b5306bf31bae5f5", [:rebar3], [], "hexpm", "81344f561357dc40a8344afa53767c32669153355b626ea9fcbc8da6b3045826"}, 18 | "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm", "69b09adddc4f74a40716ae54d140f93beb0fb8978d8636eaded0c31b6f099f16"}, 19 | "mime": {:hex, :mime, "2.0.3", "3676436d3d1f7b81b5a2d2bd8405f412c677558c81b1c92be58c00562bb59095", [:mix], [], "hexpm", "27a30bf0db44d25eecba73755acf4068cbfe26a4372f9eb3e4ea3a45956bff6b"}, 20 | "mimerl": {:hex, :mimerl, "1.2.0", "67e2d3f571088d5cfd3e550c383094b47159f3eee8ffa08e64106cdf5e981be3", [:rebar3], [], "hexpm", "f278585650aa581986264638ebf698f8bb19df297f66ad91b18910dfc6e19323"}, 21 | "mock": {:hex, :mock, "0.3.7", "75b3bbf1466d7e486ea2052a73c6e062c6256fb429d6797999ab02fa32f29e03", [:mix], [{:meck, "~> 0.9.2", [hex: :meck, repo: "hexpm", optional: false]}], "hexpm", "4da49a4609e41fd99b7836945c26f373623ea968cfb6282742bcb94440cf7e5c"}, 22 | "nimble_parsec": {:hex, :nimble_parsec, "1.4.0", "51f9b613ea62cfa97b25ccc2c1b4216e81df970acd8e16e8d1bdc58fef21370d", [:mix], [], "hexpm", "9c565862810fb383e9838c1dd2d7d2c437b3d13b267414ba6af33e50d2d1cf28"}, 23 | "parse_trans": {:hex, :parse_trans, "3.3.1", "16328ab840cc09919bd10dab29e431da3af9e9e7e7e6f0089dd5a2d2820011d8", [:rebar3], [], "hexpm", "07cd9577885f56362d414e8c4c4e6bdf10d43a8767abb92d24cbe8b24c54888b"}, 24 | "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.7", "354c321cf377240c7b8716899e182ce4890c5938111a1296add3ec74cf1715df", [:make, :mix, :rebar3], [], "hexpm", "fe4c190e8f37401d30167c8c405eda19469f34577987c76dde613e838bbc67f8"}, 25 | "sweet_xml": {:hex, :sweet_xml, "0.7.3", "debb256781c75ff6a8c5cbf7981146312b66f044a2898f453709a53e5031b45b", [:mix], [], "hexpm", "e110c867a1b3fe74bfc7dd9893aa851f0eed5518d0d7cad76d7baafd30e4f5ba"}, 26 | "telemetry": {:hex, :telemetry, "1.2.1", "68fdfe8d8f05a8428483a97d7aab2f268aaff24b49e0f599faa091f1d4e7f61c", [:rebar3], [], "hexpm", "dad9ce9d8effc621708f99eac538ef1cbe05d6a874dd741de2e689c47feafed5"}, 27 | "unicode_util_compat": {:hex, :unicode_util_compat, "0.7.0", "bc84380c9ab48177092f43ac89e4dfa2c6d62b40b8bd132b1059ecc7232f9a78", [:rebar3], [], "hexpm", "25eee6d67df61960cf6a794239566599b09e17e668d3700247bc498638152521"}, 28 | } 29 | -------------------------------------------------------------------------------- /test/actions/store_test.exs: -------------------------------------------------------------------------------- 1 | defmodule WaffleTest.Actions.Store do 2 | use ExUnit.Case, async: false 3 | 4 | @img "test/support/image.png" 5 | @remote_img_with_space_image_two "https://github.com/elixir-waffle/waffle/blob/master/test/support/image%20two.png" 6 | 7 | import Mock 8 | 9 | defmodule DummyDefinition do 10 | use Waffle.Actions.Store 11 | use Waffle.Definition.Storage 12 | 13 | def validate({file, _}), 14 | do: String.ends_with?(file.file_name, ".png") || String.ends_with?(file.file_name, ".ico") 15 | 16 | def transform(:skipped, _), do: :skip 17 | def transform(_, _), do: :noaction 18 | def __versions, do: [:original, :thumb, :skipped] 19 | end 20 | 21 | defmodule DummyDefinitionWithExtension do 22 | use Waffle.Actions.Store 23 | use Waffle.Definition.Storage 24 | 25 | def validate({file, _}), do: String.ends_with?(file.file_name, ".png") 26 | 27 | def transform(:convert_to_jpg, _), 28 | do: {:convert, "-format jpg", :jpg} 29 | 30 | def transform(:custom_to_jpg, {file, _}) do 31 | { 32 | fn _, _ -> {:ok, file} end, 33 | fn _, _ -> :jpg end 34 | } 35 | end 36 | 37 | def __versions, do: [:convert_to_jpg, :custom_to_jpg] 38 | end 39 | 40 | defmodule DummyDefinitionWithHeaders do 41 | use Waffle.Actions.Store 42 | use Waffle.Definition.Storage 43 | 44 | def transform(_, _), do: :noaction 45 | def __versions, do: [:original, :thumb, :skipped] 46 | def remote_file_headers(%URI{host: "www.google.com"}), do: [{"User-Agent", "MyApp"}] 47 | end 48 | 49 | defmodule DummyDefinitionWithValidationError do 50 | use Waffle.Actions.Store 51 | use Waffle.Definition.Storage 52 | 53 | def validate(_), do: {:error, "invalid file type"} 54 | def transform(_, _), do: :noaction 55 | def __versions, do: [:original, :thumb, :skipped] 56 | end 57 | 58 | test "custom transformations change a file extension" do 59 | with_mock Waffle.Storage.S3, 60 | put: fn DummyDefinitionWithExtension, _, {%{file_name: "image.jpg", path: _}, nil} -> 61 | {:ok, "resp"} 62 | end do 63 | assert DummyDefinitionWithExtension.store(@img) == {:ok, "image.png"} 64 | end 65 | end 66 | 67 | test "checks file existence" do 68 | assert DummyDefinition.store("non-existent-file.png") == {:error, :invalid_file_path} 69 | end 70 | 71 | test "delegates to definition validation" do 72 | assert DummyDefinition.store(__ENV__.file) == {:error, :invalid_file} 73 | end 74 | 75 | test "supports custom validation error message" do 76 | assert DummyDefinitionWithValidationError.store(__ENV__.file) == {:error, "invalid file type"} 77 | end 78 | 79 | test "single binary argument is interpreted as file path" do 80 | with_mock Waffle.Storage.S3, 81 | put: fn DummyDefinition, _, {%{file_name: "image.png", path: @img}, nil} -> 82 | {:ok, "resp"} 83 | end do 84 | assert DummyDefinition.store(@img) == {:ok, "image.png"} 85 | end 86 | end 87 | 88 | test "two-tuple argument interpreted as path and scope" do 89 | with_mock Waffle.Storage.S3, 90 | put: fn DummyDefinition, _, {%{file_name: "image.png", path: @img}, :scope} -> 91 | {:ok, "resp"} 92 | end do 93 | assert DummyDefinition.store({@img, :scope}) == {:ok, "image.png"} 94 | end 95 | end 96 | 97 | test "map with a filename and path" do 98 | with_mock Waffle.Storage.S3, 99 | put: fn DummyDefinition, _, {%{file_name: "image.png", path: @img}, nil} -> 100 | {:ok, "resp"} 101 | end do 102 | assert DummyDefinition.store(%{filename: "image.png", path: @img}) == {:ok, "image.png"} 103 | end 104 | end 105 | 106 | test "two-tuple with Plug.Upload and a scope" do 107 | with_mock Waffle.Storage.S3, 108 | put: fn DummyDefinition, _, {%{file_name: "image.png", path: @img}, :scope} -> 109 | {:ok, "resp"} 110 | end do 111 | assert DummyDefinition.store({%{filename: "image.png", path: @img}, :scope}) == 112 | {:ok, "image.png"} 113 | end 114 | end 115 | 116 | test "error from ExAws on upload to S3" do 117 | with_mock Waffle.Storage.S3, 118 | put: fn DummyDefinition, _, {%{file_name: "image.png", path: @img}, :scope} -> 119 | {:error, {:http_error, 404, "XML"}} 120 | end do 121 | assert DummyDefinition.store({%{filename: "image.png", path: @img}, :scope}) == 122 | {:error, [{:http_error, 404, "XML"}, {:http_error, 404, "XML"}]} 123 | end 124 | end 125 | 126 | test "timeout" do 127 | Application.put_env(:waffle, :version_timeout, 1) 128 | 129 | catch_exit do 130 | with_mock Waffle.Storage.S3, 131 | put: fn DummyDefinition, _, {%{file_name: "image.png", path: @img}, :scope} -> 132 | :timer.sleep(100) && {:ok, "favicon.ico"} 133 | end do 134 | assert DummyDefinition.store({%{filename: "image.png", path: @img}, :scope}) == 135 | {:ok, "image.png"} 136 | end 137 | end 138 | 139 | Application.put_env(:waffle, :version_timeout, 15_000) 140 | end 141 | 142 | test "recv_timeout" do 143 | Application.put_env(:waffle, :recv_timeout, 1) 144 | 145 | with_mock Waffle.Storage.S3, 146 | put: fn DummyDefinition, _, {%{file_name: "favicon.ico", path: _}, nil} -> 147 | {:ok, "favicon.ico"} 148 | end do 149 | assert DummyDefinition.store("https://www.google.com/favicon.ico") == 150 | {:error, :recv_timeout} 151 | end 152 | 153 | Application.put_env(:waffle, :recv_timeout, 5_000) 154 | end 155 | 156 | test "recv_timeout with a filename" do 157 | Application.put_env(:waffle, :recv_timeout, 1) 158 | 159 | with_mock Waffle.Storage.S3, 160 | put: fn DummyDefinition, _, {%{file_name: "newfavicon.ico", path: _}, nil} -> 161 | {:ok, "newfavicon.ico"} 162 | end do 163 | assert DummyDefinition.store(%{ 164 | remote_path: "https://www.google.com/favicon.ico", 165 | filename: "newfavicon.ico" 166 | }) == 167 | {:error, :recv_timeout} 168 | end 169 | 170 | Application.put_env(:waffle, :recv_timeout, 5_000) 171 | end 172 | 173 | test "accepts remote files" do 174 | with_mock Waffle.Storage.S3, 175 | put: fn DummyDefinition, _, {%{file_name: "favicon.ico", path: _}, nil} -> 176 | {:ok, "favicon.ico"} 177 | end do 178 | assert DummyDefinition.store("https://www.google.com/favicon.ico") == {:ok, "favicon.ico"} 179 | end 180 | end 181 | 182 | test "sets remote filename from content-disposition header when available" do 183 | with_mocks([ 184 | { 185 | :hackney_headers, 186 | [:passthrough], 187 | get_value: fn "content-disposition", _headers -> 188 | "attachment; filename=\"image three.png\"" 189 | end 190 | }, 191 | { 192 | Waffle.Storage.S3, 193 | [], 194 | put: fn DummyDefinition, _, {%{file_name: "image three.png", path: _}, nil} -> 195 | {:ok, "image three.png"} 196 | end 197 | } 198 | ]) do 199 | assert DummyDefinition.store(@remote_img_with_space_image_two) == 200 | {:ok, "image three.png"} 201 | end 202 | end 203 | 204 | test "sets HTTP headers for request to remote file" do 205 | with_mocks([ 206 | { 207 | :hackney, 208 | [:passthrough], 209 | [] 210 | }, 211 | { 212 | Waffle.Storage.S3, 213 | [], 214 | put: fn DummyDefinitionWithHeaders, _, {%{file_name: "favicon.ico", path: _}, nil} -> 215 | {:ok, "favicon.ico"} 216 | end 217 | } 218 | ]) do 219 | DummyDefinitionWithHeaders.store("https://www.google.com/favicon.ico") 220 | 221 | assert_called( 222 | :hackney.get("https://www.google.com/favicon.ico", [{"User-Agent", "MyApp"}], "", :_) 223 | ) 224 | end 225 | end 226 | 227 | test "accepts remote files with spaces" do 228 | with_mock Waffle.Storage.S3, 229 | put: fn DummyDefinition, _, {%{file_name: "image two.png", path: _}, nil} -> 230 | {:ok, "image two.png"} 231 | end do 232 | assert DummyDefinition.store(@remote_img_with_space_image_two) == {:ok, "image two.png"} 233 | end 234 | end 235 | 236 | test "accepts remote files with filenames" do 237 | with_mock Waffle.Storage.S3, 238 | put: fn DummyDefinition, _, {%{file_name: "newfavicon.ico", path: _}, nil} -> 239 | {:ok, "newfavicon.ico"} 240 | end do 241 | assert DummyDefinition.store(%{ 242 | remote_path: "https://www.google.com/favicon.ico", 243 | filename: "newfavicon.ico" 244 | }) == {:ok, "newfavicon.ico"} 245 | end 246 | end 247 | 248 | test "rejects remote files with filenames and invalid remote path" do 249 | with_mock Waffle.Storage.S3, 250 | put: fn DummyDefinition, _, {%{file_name: "newfavicon.ico", path: _}, nil} -> 251 | {:ok, "newfavicon.ico"} 252 | end do 253 | assert DummyDefinition.store(%{remote_path: "path/favicon.ico", filename: "newfavicon.ico"}) == 254 | {:error, :invalid_file_path} 255 | end 256 | end 257 | end 258 | -------------------------------------------------------------------------------- /test/actions/url_test.exs: -------------------------------------------------------------------------------- 1 | defmodule WaffleTest.Actions.Url do 2 | use ExUnit.Case, async: false 3 | import Mock 4 | 5 | defmodule DummyDefinition do 6 | use Waffle.Actions.Url 7 | use Waffle.Definition.Storage 8 | 9 | def __versions, do: [:original, :thumb, :skipped] 10 | def transform(:skipped, _), do: :skip 11 | def transform(_, _), do: :noaction 12 | def default_url(version, scope) when is_nil(scope), do: "dummy-#{version}" 13 | def default_url(version, scope), do: "dummy-#{version}-#{scope}" 14 | def __storage, do: Waffle.Storage.S3 15 | end 16 | 17 | test "delegates default_url generation to the definition when given a nil file" do 18 | assert DummyDefinition.url(nil) == "dummy-original" 19 | assert DummyDefinition.url(nil, :thumb) == "dummy-thumb" 20 | assert DummyDefinition.url({nil, :scope}, :thumb) == "dummy-thumb-scope" 21 | end 22 | 23 | test "handles skipped versions" do 24 | assert DummyDefinition.url("file.png", :skipped) == nil 25 | end 26 | 27 | test_with_mock "delegates url generation to the storage engine", Waffle.Storage.S3, 28 | [url: fn(DummyDefinition, :original, {%{file_name: "file.png"}, nil}, []) -> :ok end] do 29 | assert DummyDefinition.url("file.png") == :ok 30 | end 31 | 32 | test_with_mock "optional atom as a second argument specifies the version", Waffle.Storage.S3, 33 | [url: fn(DummyDefinition, :thumb, {%{file_name: "file.png"}, nil}, []) -> :ok end] do 34 | assert DummyDefinition.url("file.png", :thumb) == :ok 35 | end 36 | 37 | test_with_mock "optional list as a second argument specifies the options", Waffle.Storage.S3, 38 | [url: fn(DummyDefinition, :original, {%{file_name: "file.png"}, nil}, [signed: true, expires_in: 10]) -> :ok end] do 39 | assert DummyDefinition.url("file.png", signed: true, expires_in: 10) == :ok 40 | end 41 | 42 | test_with_mock "optional tuple for file including scope", Waffle.Storage.S3, 43 | [url: fn(DummyDefinition, :original, {%{file_name: "file.png"}, :scope}, []) -> :ok end] do 44 | assert DummyDefinition.url({"file.png", :scope}) == :ok 45 | end 46 | 47 | test_with_mock "optional tuple for file including scope 2", Waffle.Storage.S3, 48 | [url: fn 49 | (DummyDefinition, :original, {%{file_name: "file.png"}, :scope}, [signed: true]) -> :ok 50 | (DummyDefinition, :thumb, {%{file_name: "file.png"}, :scope}, [signed: true]) -> :ok 51 | end] do 52 | assert DummyDefinition.urls({"file.png", :scope}, signed: true) == %{original: :ok, thumb: :ok, skipped: nil} 53 | end 54 | end 55 | -------------------------------------------------------------------------------- /test/processor_test.exs: -------------------------------------------------------------------------------- 1 | defmodule WaffleTest.Processor do 2 | use ExUnit.Case, async: false 3 | @img "test/support/image.png" 4 | @img2 "test/support/image two.png" 5 | 6 | defmodule DummyDefinition do 7 | use Waffle.Actions.Store 8 | use Waffle.Definition.Storage 9 | 10 | alias Waffle.Transformations.Convert 11 | 12 | def validate({file, _}), do: String.ends_with?(file.file_name, ".png") 13 | def transform(:original, _), do: :noaction 14 | def transform(:thumb, _), do: {:convert, "-strip -thumbnail 10x10"} 15 | 16 | def transform(:med, _), 17 | do: {:convert, fn input, output -> " #{input} -strip -thumbnail 10x10 #{output}" end, :jpg} 18 | 19 | def transform(:small, _), 20 | do: 21 | {:convert, fn input, output -> [input, "-strip", "-thumbnail", "10x10", output] end, :jpg} 22 | 23 | def transform(:custom, _) do 24 | fn _version, file -> 25 | Convert.apply( 26 | :convert, 27 | file, 28 | fn input, output -> [input, "-strip", "-thumbnail", "1x1", output] end, 29 | :jpg 30 | ) 31 | end 32 | end 33 | 34 | def transform(:custom_with_ext, _) do 35 | {&transform_custom/2, &transform_custom_ext/2} 36 | end 37 | 38 | def transform(:skipped, _), do: :skip 39 | 40 | defp transform_custom(version, file) do 41 | Convert.apply( 42 | :convert, 43 | file, 44 | fn input, output -> [input, "-strip", "-thumbnail", "1x1", output] end, 45 | transform_custom_ext(version, file) 46 | ) 47 | end 48 | 49 | defp transform_custom_ext(_, _), do: :jpg 50 | 51 | def __versions, do: [:original, :thumb] 52 | end 53 | 54 | defmodule BrokenDefinition do 55 | use Waffle.Actions.Store 56 | use Waffle.Definition.Storage 57 | 58 | def validate({file, _}), do: String.ends_with?(file.file_name, ".png") 59 | def transform(:original, _), do: :noaction 60 | def transform(:thumb, _), do: {:convert, "-strip -invalidTransformation 10x10"} 61 | def __versions, do: [:original, :thumb] 62 | end 63 | 64 | defmodule MissingExecutableDefinition do 65 | use Waffle.Definition 66 | 67 | def transform(:original, _), do: {:blah, ""} 68 | end 69 | 70 | test "returns the original path for :noaction transformations" do 71 | {:ok, file} = 72 | Waffle.Processor.process( 73 | DummyDefinition, 74 | :original, 75 | {Waffle.File.new(@img, DummyDefinition), nil} 76 | ) 77 | 78 | assert file.path == @img 79 | end 80 | 81 | test "returns nil for :skip transformations" do 82 | assert {:ok, nil} = 83 | Waffle.Processor.process( 84 | DummyDefinition, 85 | :skipped, 86 | {Waffle.File.new(@img, DummyDefinition), nil} 87 | ) 88 | end 89 | 90 | test "transforms a copied version of file according to the specified transformation" do 91 | {:ok, new_file} = 92 | Waffle.Processor.process( 93 | DummyDefinition, 94 | :thumb, 95 | {Waffle.File.new(@img, DummyDefinition), nil} 96 | ) 97 | 98 | assert new_file.path != @img 99 | # original file untouched 100 | assert "128x128" == geometry(@img) 101 | assert "10x10" == geometry(new_file.path) 102 | cleanup(new_file.path) 103 | end 104 | 105 | test "transforms a copied version of file according to a function transformation that returns a string" do 106 | {:ok, new_file} = 107 | Waffle.Processor.process( 108 | DummyDefinition, 109 | :med, 110 | {Waffle.File.new(@img, DummyDefinition), nil} 111 | ) 112 | 113 | assert new_file.path != @img 114 | # original file untouched 115 | assert "128x128" == geometry(@img) 116 | assert "10x10" == geometry(new_file.path) 117 | # new tmp file has correct extension 118 | assert Path.extname(new_file.path) == ".jpg" 119 | cleanup(new_file.path) 120 | end 121 | 122 | test "transforms a copied version of file according to a function transformation that returns a list" do 123 | {:ok, new_file} = 124 | Waffle.Processor.process( 125 | DummyDefinition, 126 | :small, 127 | {Waffle.File.new(@img, DummyDefinition), nil} 128 | ) 129 | 130 | assert new_file.path != @img 131 | # original file untouched 132 | assert "128x128" == geometry(@img) 133 | assert "10x10" == geometry(new_file.path) 134 | cleanup(new_file.path) 135 | end 136 | 137 | test "transforms with a custom function" do 138 | {:ok, new_file} = 139 | Waffle.Processor.process( 140 | DummyDefinition, 141 | :custom, 142 | {Waffle.File.new(@img, DummyDefinition), nil} 143 | ) 144 | 145 | assert new_file.path != @img 146 | # original file untouched 147 | assert "128x128" == geometry(@img) 148 | assert "1x1" == geometry(new_file.path) 149 | cleanup(new_file.path) 150 | end 151 | 152 | test "transforms with custom functions" do 153 | {:ok, new_file} = 154 | Waffle.Processor.process( 155 | DummyDefinition, 156 | :custom_with_ext, 157 | {Waffle.File.new(@img, DummyDefinition), nil} 158 | ) 159 | 160 | assert new_file.path != @img 161 | # original file untouched 162 | assert "128x128" == geometry(@img) 163 | assert "1x1" == geometry(new_file.path) 164 | assert Path.extname(new_file.path) == ".jpg" 165 | cleanup(new_file.path) 166 | end 167 | 168 | test "transforms a file given as a binary" do 169 | img_binary = File.read!(@img) 170 | 171 | {:ok, new_file} = 172 | Waffle.Processor.process( 173 | DummyDefinition, 174 | :small, 175 | {Waffle.File.new(%{binary: img_binary, filename: "image.png"}, DummyDefinition), nil} 176 | ) 177 | 178 | assert new_file.path != @img 179 | # original file untouched 180 | assert "128x128" == geometry(@img) 181 | assert "10x10" == geometry(new_file.path) 182 | # new tmp file has correct extension 183 | assert Path.extname(new_file.path) == ".jpg" 184 | cleanup(new_file.path) 185 | end 186 | 187 | test "file names with spaces" do 188 | {:ok, new_file} = 189 | Waffle.Processor.process( 190 | DummyDefinition, 191 | :thumb, 192 | {Waffle.File.new(@img2, DummyDefinition), nil} 193 | ) 194 | 195 | assert new_file.path != @img2 196 | # original file untouched 197 | assert "128x128" == geometry(@img2) 198 | assert "10x10" == geometry(new_file.path) 199 | cleanup(new_file.path) 200 | end 201 | 202 | test "returns tuple in an invalid transformation" do 203 | assert {:error, _} = 204 | Waffle.Processor.process( 205 | BrokenDefinition, 206 | :thumb, 207 | {Waffle.File.new(@img, BrokenDefinition), nil} 208 | ) 209 | end 210 | 211 | test "raises an error if the given transformation executable cannot be found" do 212 | assert_raise Waffle.MissingExecutableError, ~r"blah", fn -> 213 | Waffle.Processor.process( 214 | MissingExecutableDefinition, 215 | :original, 216 | {Waffle.File.new(@img, MissingExecutableDefinition), nil} 217 | ) 218 | end 219 | end 220 | 221 | defp geometry(path) do 222 | {identify, 0} = System.cmd("identify", ["-verbose", path], stderr_to_stdout: true) 223 | Enum.at(Regex.run(~r/Geometry: ([^+]*)/, identify), 1) 224 | end 225 | 226 | defp cleanup(path) do 227 | File.rm(path) 228 | end 229 | end 230 | -------------------------------------------------------------------------------- /test/storage/local_test.exs: -------------------------------------------------------------------------------- 1 | defmodule WaffleTest.Storage.Local do 2 | use ExUnit.Case 3 | 4 | alias Waffle.Storage.Local 5 | 6 | @img "test/support/image.png" 7 | @badimg "test/support/invalid_image.png" 8 | @custom_asset_host "http://static.example.com" 9 | 10 | setup do 11 | File.mkdir_p("waffletest/uploads") 12 | File.mkdir_p("waffletest/tmp") 13 | System.put_env("TMPDIR", "waffletest/tmp") 14 | 15 | on_exit fn -> 16 | File.rm_rf("waffletest/uploads") 17 | File.rm_rf("waffletest/tmp") 18 | end 19 | end 20 | 21 | def with_env(app, key, value, fun) do 22 | previous = Application.get_env(app, key, :nothing) 23 | 24 | Application.put_env(app, key, value) 25 | fun.() 26 | 27 | case previous do 28 | :nothing -> Application.delete_env(app, key) 29 | _ -> Application.put_env(app, key, previous) 30 | end 31 | end 32 | 33 | defmodule DummyDefinition do 34 | use Waffle.Definition 35 | 36 | @versions [:original, :thumb, :skipped] 37 | 38 | def transform(:thumb, _), do: {:convert, "-strip -thumbnail 10x10"} 39 | def transform(:original, _), do: :noaction 40 | def transform(:skipped, _), do: :skip 41 | 42 | def storage_dir(_, _), do: "waffletest/uploads" 43 | def __storage, do: Local 44 | 45 | def filename(:original, {file, _}), do: "original-#{Path.basename(file.file_name, Path.extname(file.file_name))}" 46 | def filename(:thumb, {file, _}), do: "1/thumb-#{Path.basename(file.file_name, Path.extname(file.file_name))}" 47 | def filename(:skipped, {file, _}), do: "1/skipped-#{Path.basename(file.file_name, Path.extname(file.file_name))}" 48 | end 49 | 50 | defmodule DummyDefinitionWithPrefix do 51 | use Waffle.Definition 52 | 53 | @versions [:original, :thumb] 54 | 55 | def transform(:thumb, _), do: {:convert, "-strip -thumbnail 10x10"} 56 | 57 | def storage_dir_prefix, do: "priv/waffle/private" 58 | def storage_dir(_, _), do: "waffletest/uploads" 59 | def __storage, do: Local 60 | 61 | def filename(:original, {file, _}), do: "original-#{Path.basename(file.file_name, Path.extname(file.file_name))}" 62 | def filename(:thumb, {file, _}), do: "1/thumb-#{Path.basename(file.file_name, Path.extname(file.file_name))}" 63 | end 64 | 65 | test "put, delete, get" do 66 | assert {:ok, "original-image.png"} == 67 | Local.put( 68 | DummyDefinition, 69 | :original, 70 | {Waffle.File.new(%{filename: "original-image.png", path: @img}, DummyDefinition), 71 | nil} 72 | ) 73 | 74 | assert {:ok, "1/thumb-image.png"} == 75 | Local.put( 76 | DummyDefinition, 77 | :thumb, 78 | {Waffle.File.new(%{filename: "1/thumb-image.png", path: @img}, DummyDefinition), 79 | nil} 80 | ) 81 | 82 | assert File.exists?("waffletest/uploads/original-image.png") 83 | assert File.exists?("waffletest/uploads/1/thumb-image.png") 84 | assert "/waffletest/uploads/original-image.png" == DummyDefinition.url("image.png", :original) 85 | assert "/waffletest/uploads/1/thumb-image.png" == DummyDefinition.url("1/image.png", :thumb) 86 | 87 | :ok = Local.delete(DummyDefinition, :original, {%{file_name: "image.png"}, nil}) 88 | :ok = Local.delete(DummyDefinition, :thumb, {%{file_name: "image.png"}, nil}) 89 | refute File.exists?("waffletest/uploads/original-image.png") 90 | refute File.exists?("waffletest/uploads/1/thumb-image.png") 91 | end 92 | 93 | test "put, delete, get with storage prefix" do 94 | assert {:ok, "original-image.png"} == 95 | Local.put( 96 | DummyDefinitionWithPrefix, 97 | :original, 98 | {Waffle.File.new( 99 | %{filename: "original-image.png", path: @img}, 100 | DummyDefinitionWithPrefix 101 | ), nil} 102 | ) 103 | 104 | assert {:ok, "1/thumb-image.png"} == 105 | Local.put( 106 | DummyDefinitionWithPrefix, 107 | :thumb, 108 | {Waffle.File.new( 109 | %{filename: "1/thumb-image.png", path: @img}, 110 | DummyDefinitionWithPrefix 111 | ), nil} 112 | ) 113 | 114 | assert File.exists?("priv/waffle/private/waffletest/uploads/original-image.png") 115 | assert File.exists?("priv/waffle/private/waffletest/uploads/1/thumb-image.png") 116 | 117 | :ok = Local.delete(DummyDefinitionWithPrefix, :original, {%{file_name: "image.png"}, nil}) 118 | :ok = Local.delete(DummyDefinitionWithPrefix, :thumb, {%{file_name: "image.png"}, nil}) 119 | refute File.exists?("priv/waffle/private/waffletest/uploads/original-image.png") 120 | refute File.exists?("priv/waffle/private/waffletest/uploads/1/thumb-image.png") 121 | end 122 | 123 | test "deleting when there's a skipped version" do 124 | DummyDefinition.store(@img) 125 | assert :ok = DummyDefinition.delete(@img) 126 | end 127 | 128 | test "get, delete with :asset_host set" do 129 | with_env :waffle, :asset_host, @custom_asset_host, fn -> 130 | assert {:ok, "original-image.png"} == 131 | Local.put( 132 | DummyDefinition, 133 | :original, 134 | {Waffle.File.new(%{filename: "original-image.png", path: @img}, DummyDefinition), 135 | nil} 136 | ) 137 | 138 | assert {:ok, "1/thumb-image.png"} == 139 | Local.put( 140 | DummyDefinition, 141 | :thumb, 142 | {Waffle.File.new(%{filename: "1/thumb-image.png", path: @img}, DummyDefinition), 143 | nil} 144 | ) 145 | 146 | assert File.exists?("waffletest/uploads/original-image.png") 147 | assert File.exists?("waffletest/uploads/1/thumb-image.png") 148 | assert @custom_asset_host <> "/waffletest/uploads/original-image.png" == DummyDefinition.url("image.png", :original) 149 | assert @custom_asset_host <> "/waffletest/uploads/1/thumb-image.png" == DummyDefinition.url("1/image.png", :thumb) 150 | 151 | :ok = Local.delete(DummyDefinition, :original, {%{file_name: "image.png"}, nil}) 152 | :ok = Local.delete(DummyDefinition, :thumb, {%{file_name: "image.png"}, nil}) 153 | refute File.exists?("waffletest/uploads/original-image.png") 154 | refute File.exists?("waffletest/uploads/1/thumb-image.png") 155 | end 156 | end 157 | 158 | test "save binary" do 159 | Local.put( 160 | DummyDefinition, 161 | :original, 162 | {Waffle.File.new(%{binary: "binary", filename: "binary.png"}, DummyDefinition), nil} 163 | ) 164 | 165 | assert true == File.exists?("waffletest/uploads/binary.png") 166 | end 167 | 168 | test "encoded url" do 169 | url = 170 | DummyDefinition.url( 171 | Waffle.File.new(%{binary: "binary", filename: "binary file.png"}, DummyDefinition), 172 | :original 173 | ) 174 | 175 | assert "/waffletest/uploads/original-binary%20file.png" == url 176 | end 177 | 178 | test "url for skipped version" do 179 | url = 180 | DummyDefinition.url( 181 | Waffle.File.new(%{binary: "binary", filename: "binary file.png"}, DummyDefinition), 182 | :skipped 183 | ) 184 | 185 | assert url == nil 186 | end 187 | 188 | test "if one transform fails, they all fail" do 189 | filepath = @badimg 190 | [filename] = String.split(@img, "/") |> Enum.reverse |> Enum.take(1) 191 | assert File.exists?(filepath) 192 | DummyDefinition.store(filepath) 193 | 194 | assert !File.exists?("waffletest/uploads/original-#{filename}") 195 | assert !File.exists?("waffletest/uploads/1/thumb-#{filename}") 196 | end 197 | 198 | test "temp files from processing are cleaned up" do 199 | filepath = @img 200 | DummyDefinition.store(filepath) 201 | assert Enum.empty?(File.ls!("waffletest/tmp")) 202 | end 203 | 204 | test "temp files from handling binary data are cleaned up" do 205 | filepath = @img 206 | filename = "image.png" 207 | DummyDefinition.store(%{binary: File.read!(filepath), filename: filename}) 208 | assert File.exists?("waffletest/uploads/original-#{filename}") 209 | assert Enum.empty?(File.ls!("waffletest/tmp")) 210 | end 211 | 212 | test "temp files from handling remote URLs are cleaned up" do 213 | DummyDefinition.store("https://www.google.com/favicon.ico") 214 | assert Enum.empty?(File.ls!("waffletest/tmp")) 215 | end 216 | end 217 | -------------------------------------------------------------------------------- /test/storage/s3_test.exs: -------------------------------------------------------------------------------- 1 | defmodule WaffleTest.Storage.S3 do 2 | use ExUnit.Case, async: false 3 | 4 | @img "test/support/image.png" 5 | @img_with_space "test/support/image two.png" 6 | @img_with_plus "test/support/image+three.png" 7 | 8 | defmodule DummyDefinition do 9 | use Waffle.Definition 10 | 11 | @acl :public_read 12 | def storage_dir(_, _), do: "waffletest/uploads" 13 | def acl(_, {_, :private}), do: :private 14 | 15 | def s3_object_headers(:original, {_, :with_content_type}), do: [content_type: "image/gif"] 16 | 17 | def s3_object_headers(:original, {_, :with_content_disposition}), 18 | do: %{content_disposition: "attachment; filename=abc.png"} 19 | end 20 | 21 | defmodule DefinitionWithThumbnail do 22 | use Waffle.Definition 23 | @versions [:thumb] 24 | @acl :public_read 25 | 26 | def transform(:thumb, _) do 27 | {"convert", "-strip -thumbnail 100x100^ -gravity center -extent 100x100 -format jpg", :jpg} 28 | end 29 | end 30 | 31 | defmodule DefinitionWithSkipped do 32 | use Waffle.Definition 33 | @versions [:skipped] 34 | @acl :public_read 35 | 36 | def transform(:skipped, _), do: :skip 37 | end 38 | 39 | defmodule DefinitionWithScope do 40 | use Waffle.Definition 41 | @acl :public_read 42 | def storage_dir(_, {_, scope}), do: "uploads/with_scopes/#{scope.id}" 43 | end 44 | 45 | defmodule DefinitionWithBucket do 46 | use Waffle.Definition 47 | def bucket, do: System.get_env("WAFFLE_TEST_BUCKET") 48 | end 49 | 50 | defmodule DefinitionWithBucketInScope do 51 | use Waffle.Definition 52 | @acl :public_read 53 | def bucket({_, scope}), do: scope[:bucket] || bucket() 54 | def bucket, do: System.get_env("WAFFLE_TEST_BUCKET") 55 | end 56 | 57 | defmodule DefinitionWithAssetHost do 58 | use Waffle.Definition 59 | def asset_host, do: "https://example.com" 60 | end 61 | 62 | def env_bucket do 63 | System.get_env("WAFFLE_TEST_BUCKET") 64 | end 65 | 66 | defmacro delete_and_assert_not_found(definition, args) do 67 | quote bind_quoted: [definition: definition, args: args] do 68 | :ok = definition.delete(args) 69 | signed_url = DummyDefinition.url(args, signed: true) 70 | {:ok, {{_, code, msg}, _, _}} = :httpc.request(to_charlist(signed_url)) 71 | 72 | # If buckets aren't configured to be public at bucket-level, 73 | # deleted objects may return 403 Forbidden instead of 404 Not Found, 74 | # even with a signed url 75 | assert code in [403, 404] 76 | end 77 | end 78 | 79 | defmacro assert_header(definition, args, header, value) do 80 | quote bind_quoted: [definition: definition, args: args, header: header, value: value] do 81 | url = definition.url(args) 82 | {:ok, {{_, 200, 'OK'}, headers, _}} = :httpc.request(to_charlist(url)) 83 | 84 | char_header = to_charlist(header) 85 | 86 | assert to_charlist(value) == 87 | Enum.find_value(headers, fn 88 | {^char_header, value} -> value 89 | _ -> nil 90 | end) 91 | end 92 | end 93 | 94 | defmacro assert_private(definition, args) do 95 | quote bind_quoted: [definition: definition, args: args] do 96 | unsigned_url = definition.url(args) 97 | {:ok, {{_, code, msg}, _, _}} = :httpc.request(to_charlist(unsigned_url)) 98 | assert code == 403 99 | assert msg == 'Forbidden' 100 | 101 | signed_url = definition.url(args, signed: true) 102 | {:ok, {{_, code, msg}, headers, _}} = :httpc.request(to_charlist(signed_url)) 103 | assert code == 200 104 | assert msg == 'OK' 105 | end 106 | end 107 | 108 | defmacro assert_public(definition, args) do 109 | quote bind_quoted: [definition: definition, args: args] do 110 | url = definition.url(args) 111 | {:ok, {{_, code, msg}, headers, _}} = :httpc.request(to_charlist(url)) 112 | assert code == 200 113 | assert msg == 'OK' 114 | end 115 | end 116 | 117 | defmacro assert_public_with_extension(definition, args, version, extension) do 118 | quote bind_quoted: [ 119 | definition: definition, 120 | version: version, 121 | args: args, 122 | extension: extension 123 | ] do 124 | url = definition.url(args, version) 125 | {:ok, {{_, code, msg}, headers, _}} = :httpc.request(to_charlist(url)) 126 | assert code == 200 127 | assert msg == 'OK' 128 | assert Path.extname(url) == extension 129 | end 130 | end 131 | 132 | setup_all do 133 | Application.ensure_all_started(:hackney) 134 | Application.ensure_all_started(:ex_aws) 135 | Application.put_env(:waffle, :virtual_host, true) 136 | Application.put_env(:waffle, :bucket, {:system, "WAFFLE_TEST_BUCKET"}) 137 | 138 | # Application.put_env :ex_aws, :s3, [scheme: "https://", host: "s3.amazonaws.com", region: "us-west-2"] 139 | Application.put_env(:ex_aws, :access_key_id, System.get_env("WAFFLE_TEST_S3_KEY")) 140 | Application.put_env(:ex_aws, :secret_access_key, System.get_env("WAFFLE_TEST_S3_SECRET")) 141 | Application.put_env(:ex_aws, :region, System.get_env("WAFFLE_TEST_REGION", "eu-north-1")) 142 | # Application.put_env :ex_aws, :scheme, "https://" 143 | end 144 | 145 | def with_env(app, key, value, fun) do 146 | previous = Application.get_env(app, key, :nothing) 147 | 148 | Application.put_env(app, key, value) 149 | fun.() 150 | 151 | case previous do 152 | :nothing -> Application.delete_env(app, key) 153 | _ -> Application.put_env(app, key, previous) 154 | end 155 | end 156 | 157 | @tag :s3 158 | @tag timeout: 15_000 159 | test "virtual_host" do 160 | with_env(:waffle, :virtual_host, true, fn -> 161 | assert "https://#{env_bucket()}.s3.amazonaws.com/waffletest/uploads/image.png" == 162 | DummyDefinition.url(@img) 163 | end) 164 | 165 | with_env(:waffle, :virtual_host, false, fn -> 166 | assert "https://s3.amazonaws.com/#{env_bucket()}/waffletest/uploads/image.png" == 167 | DummyDefinition.url(@img) 168 | end) 169 | end 170 | 171 | @tag :s3 172 | @tag timeout: 15_000 173 | test "custom asset_host" do 174 | custom_asset_host = "https://some.cloudfront.com" 175 | 176 | with_env(:waffle, :asset_host, custom_asset_host, fn -> 177 | assert "#{custom_asset_host}/waffletest/uploads/image.png" == DummyDefinition.url(@img) 178 | end) 179 | 180 | with_env(:waffle, :asset_host, {:system, "WAFFLE_ASSET_HOST"}, fn -> 181 | System.put_env("WAFFLE_ASSET_HOST", custom_asset_host) 182 | assert "#{custom_asset_host}/waffletest/uploads/image.png" == DummyDefinition.url(@img) 183 | end) 184 | 185 | with_env(:waffle, :asset_host, false, fn -> 186 | assert "https://#{env_bucket()}.s3.amazonaws.com/waffletest/uploads/image.png" == 187 | DummyDefinition.url(@img) 188 | end) 189 | end 190 | 191 | @tag :s3 192 | @tag timeout: 150_000 193 | test "custom asset_host in definition" do 194 | custom_asset_host = "https://example.com" 195 | 196 | assert "#{custom_asset_host}/uploads/image.png" == DefinitionWithAssetHost.url(@img) 197 | end 198 | 199 | @tag :s3 200 | @tag timeout: 15_000 201 | test "encoded url" do 202 | url = DummyDefinition.url(@img_with_space) 203 | assert "https://#{env_bucket()}.s3.amazonaws.com/waffletest/uploads/image%20two.png" == url 204 | end 205 | 206 | @tag :s3 207 | @tag timeout: 15_000 208 | test "encoded url with S3-specific escaping" do 209 | url = DummyDefinition.url(@img_with_plus) 210 | assert "https://#{env_bucket()}.s3.amazonaws.com/waffletest/uploads/image%2Bthree.png" == url 211 | end 212 | 213 | @tag :s3 214 | @tag timeout: 15_000 215 | test "public put and get" do 216 | assert {:ok, "image.png"} == DummyDefinition.store(@img) 217 | assert_public(DummyDefinition, "image.png") 218 | delete_and_assert_not_found(DummyDefinition, "image.png") 219 | end 220 | 221 | @tag :s3 222 | @tag timeout: 15_000 223 | test "public put stream" do 224 | img_map = %{filename: "image.png", stream: File.stream!(@img)} 225 | assert {:ok, "image.png"} == DummyDefinition.store(img_map) 226 | assert_public(DummyDefinition, "image.png") 227 | delete_and_assert_not_found(DummyDefinition, "image.png") 228 | end 229 | 230 | @tag :s3 231 | @tag timeout: 15_000 232 | test "private put and signed get" do 233 | # put the image as private 234 | assert {:ok, "image.png"} == DummyDefinition.store({@img, :private}) 235 | assert_private(DummyDefinition, "image.png") 236 | delete_and_assert_not_found(DummyDefinition, "image.png") 237 | end 238 | 239 | @tag :s3 240 | @tag timeout: 15_000 241 | test "content_type" do 242 | {:ok, "image.png"} = DummyDefinition.store({@img, :with_content_type}) 243 | assert_header(DummyDefinition, "image.png", "content-type", "image/gif") 244 | delete_and_assert_not_found(DummyDefinition, "image.png") 245 | end 246 | 247 | @tag :s3 248 | @tag timeout: 15_000 249 | test "content_disposition" do 250 | {:ok, "image.png"} = DummyDefinition.store({@img, :with_content_disposition}) 251 | 252 | assert_header( 253 | DummyDefinition, 254 | "image.png", 255 | "content-disposition", 256 | "attachment; filename=abc.png" 257 | ) 258 | 259 | delete_and_assert_not_found(DummyDefinition, "image.png") 260 | end 261 | 262 | @tag :s3 263 | @tag timeout: 150_000 264 | test "delete with scope" do 265 | scope = %{id: 1} 266 | {:ok, path} = DefinitionWithScope.store({"test/support/image.png", scope}) 267 | 268 | assert "https://#{env_bucket()}.s3.amazonaws.com/uploads/with_scopes/1/image.png" == 269 | DefinitionWithScope.url({path, scope}) 270 | 271 | assert_public(DefinitionWithScope, {path, scope}) 272 | delete_and_assert_not_found(DefinitionWithScope, {path, scope}) 273 | end 274 | 275 | @tag :s3 276 | @tag timeout: 150_000 277 | test "delete with bucket in scope" do 278 | bucket = System.get_env("WAFFLE_TEST_BUCKET2") 279 | scope = %{id: 1, bucket: bucket} 280 | {:ok, path} = DefinitionWithBucketInScope.store({"test/support/image.png", scope}) 281 | 282 | assert "https://#{bucket}.s3.amazonaws.com/uploads/image.png" == 283 | DefinitionWithBucketInScope.url({path, scope}) 284 | 285 | assert_public(DefinitionWithBucketInScope, {path, scope}) 286 | delete_and_assert_not_found(DefinitionWithBucketInScope, {path, scope}) 287 | end 288 | 289 | @tag :s3 290 | @tag timeout: 150_000 291 | test "with bucket" do 292 | url = "https://#{env_bucket()}.s3.amazonaws.com/uploads/image.png" 293 | assert url == DefinitionWithBucket.url("test/support/image.png") 294 | assert {:ok, "image.png"} == DefinitionWithBucket.store("test/support/image.png") 295 | delete_and_assert_not_found(DefinitionWithBucket, "test/support/image.png") 296 | end 297 | 298 | @tag :s3 299 | @tag timeout: 150_000 300 | test "put with error" do 301 | Application.put_env(:waffle, :bucket, "unknown-bucket") 302 | {:error, res} = DummyDefinition.store("test/support/image.png") 303 | Application.put_env(:waffle, :bucket, env_bucket()) 304 | assert res 305 | end 306 | 307 | @tag :s3 308 | @tag timeout: 150_000 309 | test "put with converted version" do 310 | assert {:ok, "image.png"} == DefinitionWithThumbnail.store(@img) 311 | assert_public_with_extension(DefinitionWithThumbnail, "image.png", :thumb, ".jpg") 312 | delete_and_assert_not_found(DefinitionWithThumbnail, "image.png") 313 | end 314 | 315 | @tag :s3 316 | @tag timeout: 150_000 317 | test "url for a skipped version" do 318 | assert nil == DefinitionWithSkipped.url("image.png") 319 | end 320 | end 321 | -------------------------------------------------------------------------------- /test/support/image two.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/elixir-waffle/waffle/d6614e8671ca31cd07d50de2d9cbf80b341f7b69/test/support/image two.png -------------------------------------------------------------------------------- /test/support/image+three.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/elixir-waffle/waffle/d6614e8671ca31cd07d50de2d9cbf80b341f7b69/test/support/image+three.png -------------------------------------------------------------------------------- /test/support/image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/elixir-waffle/waffle/d6614e8671ca31cd07d50de2d9cbf80b341f7b69/test/support/image.png -------------------------------------------------------------------------------- /test/support/invalid_image.png: -------------------------------------------------------------------------------- 1 | This is not a PNG file. 2 | -------------------------------------------------------------------------------- /test/test_helper.exs: -------------------------------------------------------------------------------- 1 | ExUnit.configure exclude: [:s3] 2 | ExUnit.start() 3 | --------------------------------------------------------------------------------