├── .gitignore ├── CHANGELOG.md ├── Dockerfile ├── LICENSE ├── Makefile ├── README.md ├── VERSION ├── default.nix ├── examples ├── filesystem_config.json ├── http_config.json └── s3_config.json ├── halfshell ├── config.go ├── halfshell.go ├── image.go ├── image_processor.go ├── logger.go ├── route.go ├── server.go ├── source.go ├── source_filesystem.go ├── source_http.go ├── source_s3.go ├── statter.go └── templates.go ├── main.go └── release.nix /.gitignore: -------------------------------------------------------------------------------- 1 | .env 2 | bin 3 | *.py[co] 4 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## 0.1.1-dev 4 | 5 | ### Features: 6 | 7 | - Allowed disabling of StatsD reporting 8 | - Allowed customizing StatsD host and port 9 | - Added ETag headers 10 | 11 | ### Maintenance: 12 | 13 | - Go vet/lint cleanup 14 | 15 | ## 0.1.1 (2014-03-13) 16 | 17 | ### Features: 18 | 19 | - Added source for reading from local filesystem. 20 | 21 | ## 0.1.0 (2014-03-12) 22 | 23 | - Initial release 24 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:14.04 2 | MAINTAINER Rafik Salama 3 | 4 | WORKDIR /opt/go/src/github.com/oysterbooks/halfshell 5 | ENV GOPATH /opt/go 6 | 7 | RUN apt-get update && apt-get install -qy \ 8 | build-essential \ 9 | git \ 10 | wget \ 11 | libmagickcore-dev \ 12 | libmagickwand-dev \ 13 | imagemagick \ 14 | golang 15 | 16 | ADD . /opt/go/src/github.com/oysterbooks/halfshell 17 | RUN cd /opt/go/src/github.com/oysterbooks/halfshell && make deps && make build 18 | 19 | ENTRYPOINT ["/opt/go/src/github.com/oysterbooks/halfshell/bin/halfshell"] 20 | 21 | EXPOSE 8080 22 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014 Oyster 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | OK_COLOR=\033[32;01m 2 | NO_COLOR=\033[0m 3 | 4 | build: 5 | @echo "$(OK_COLOR)==> Compiling binary$(NO_COLOR)" 6 | go build -o bin/halfshell 7 | 8 | clean: 9 | @rm -rf bin/ 10 | @rm -rf result/ 11 | 12 | deps: 13 | @echo "$(OK_COLOR)==> Installing dependencies$(NO_COLOR)" 14 | @go get -d -v ./... 15 | @go list -f '{{range .TestImports}}{{.}} {{end}}' ./... | xargs -n1 go get -d 16 | 17 | format: 18 | go fmt ./... 19 | 20 | .PHONY: clean format deps build 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Halfshell 2 | 3 | Halfshell is a proxy server for processing images on the fly. It allows you to dynamically resize (and apply effects to) images hosted on S3, a local filesystem or an http source via query parameters. It supports creating “families” of images which can read from distinct image sources and enable different configuration values for image processing and retrieval. See the [introduction blog post](http://engineering.oysterbooks.com/post/79458380259/resizing-images-on-the-fly-with-go). 4 | 5 | Current version: `0.1.1` 6 | 7 | ## Architecture 8 | 9 | Halfshell was architected to be extensible from the beginning. The system is composed of a few components with their own configuration and simple interfaces. 10 | 11 | ### Sources 12 | 13 | Sources are repositories from which an “original” image can be loaded. They return an image given a path. Currently, sources for downloading images from S3, a local filesystem and http are included. 14 | 15 | ### Processors 16 | 17 | Processors perform all image manipulation. They accept an image and a set of options and return a modified image. Out of the box, the default processor supports resizing and blurring images. Each processor can be configured with maximum and default image dimensions and enable/disable certain features. 18 | 19 | ### Routes 20 | 21 | Routes bind URL rules (regular expressions) with a source and a processor. Halfshell supports setting up an arbitrary number of routes, and sources and processors do not need to correspond 1-1 with routes. 22 | 23 | When Halfshell receives a request, it determines the matching route, retrieves the image from its source, and processes the image using its processor. 24 | 25 | This simple architecture has allowed us to serve images from multiple S3 buckets and maintain isolated configuration settings for each family of images. 26 | 27 | ## Usage and Configuration 28 | 29 | Halfshell uses a JSON file for configuration. An example is shown below: 30 | 31 | ```json 32 | { 33 | "server": { 34 | "port": 8080, 35 | "read_timeout": 5, 36 | "write_timeout": 30 37 | }, 38 | "sources": { 39 | "default": { 40 | "type": "s3", 41 | "s3_access_key": "", 42 | "s3_secret_key": "" 43 | }, 44 | "blog-post-images": { 45 | "s3_bucket": "my-company-blog-post-images" 46 | }, 47 | "profile-photos": { 48 | "s3_bucket": "my-company-profile-photos" 49 | } 50 | }, 51 | "processors": { 52 | "default": { 53 | "image_compression_quality": 85, 54 | "default_scale_mode": "aspect_fit", 55 | "max_blur_radius_percentage": 0, 56 | "max_image_height": 0, 57 | "max_image_width": 1000 58 | 59 | }, 60 | "profile-photos": { 61 | "default_image_width": 120 62 | } 63 | }, 64 | "routes": { 65 | "^/blog(?P/.*)$": { 66 | "name": "blog-post-images", 67 | "source": "blog-post-images", 68 | "processor": "default", 69 | "cache_control": "no-transform,public,max-age=2592000,s-maxage=31104000" 70 | }, 71 | "^/users(?P/.*)$": { 72 | "name": "profile-photos", 73 | "source": "profile-photos", 74 | "processor": "profile-photos" 75 | } 76 | } 77 | } 78 | ``` 79 | 80 | To start the server, pass configuration file path as an argument. 81 | 82 | ```bash 83 | $ ./bin/halfshell config.json 84 | ``` 85 | 86 | This will start the server on port 8080, and service requests whose path begins with /users/ or /blog/, e.g.: 87 | 88 | http://localhost:8080/users/joe/default.jpg?w=100&h=100 89 | http://localhost:8080/blog/posts/announcement.jpg?w=600&h=200 90 | 91 | The image_host named group in the route pattern match (e.g., `^/users(?P/.*)$`) gets extracted as the request path for the source. In this instance, the file “joe/default.jpg” is requested from the “my-company-profile-photos” S3 bucket. The processor resizes the image to a width and height of 100. 92 | 93 | ### Server 94 | 95 | The `server` configuration block accepts the following settings: 96 | 97 | ##### port 98 | 99 | The port to run the server on. 100 | 101 | ##### read_timeout 102 | 103 | The timeout in seconds for reading the initial data from the connection. 104 | 105 | ##### write_timeout 106 | 107 | The timeout in seconds for writing the image data backto the connection. 108 | 109 | ### Sources 110 | 111 | The `sources` block is a mapping of source names to source configuration values. 112 | Values from a source named `default` will be inherited by all other sources. 113 | 114 | ##### type 115 | 116 | The type of image source. Currently `s3` or `filesystem`. 117 | 118 | ##### s3_access_key 119 | 120 | For the S3 source type, the access key to read from S3. 121 | 122 | ##### s3_secret_key 123 | 124 | For the S3 source type, the secret key to read from S3. 125 | 126 | ##### s3_bucket 127 | 128 | For the S3 source type, the bucket to request images from. 129 | 130 | ##### directory 131 | 132 | For the Filesystem source type, the local directory to request images from. Required. 133 | For the S3 source type, `directory` corresponds to an optional base directory in the S3 bucket. 134 | 135 | ### Processors 136 | 137 | The `processors` block is a mapping of processor names to processor configuration values. 138 | Values from a processor named `default` will be inherited by all other processors. 139 | 140 | ##### image_compression_quality 141 | 142 | The compression quality to use for JPEG images. 143 | 144 | ##### maintain_aspect_ratio 145 | 146 | DEPRECATED: Use the `aspect_fit` `scale_mode` instead. 147 | 148 | If this is set to true, the resized images will always maintain the original 149 | aspect ratio. When set to false, the image will be stretched to fit the width 150 | and height requested. 151 | 152 | ##### default_scale_mode 153 | 154 | When changing the dimensions of an image, you may want to crop edges or 155 | constrain proportions. Use the `default_scale_mode` setting to define these 156 | rules (`scale_mode` as a URL query parameter). 157 | 158 | A value of `aspect_fit` will change the image size to fit in the given 159 | dimensions while retaining original proportions. No part of the image will be 160 | cut away. 161 | 162 | A value of `aspect_fill` will change the image size to at least fit the given 163 | dimensions while retaining original proportions. No part of the image will be 164 | cut away. 165 | 166 | A value of `aspect_crop` will change the image size to fit in the given 167 | dimensions while retaining original proportions. Edges that do not fit in the 168 | given dimensions will be cut off. 169 | 170 | The default behavior is to `fill`, which changes the image size to fit the given 171 | dimensions and will NOT retain the original proportions. 172 | 173 | Cheat Sheet: 174 | 175 | Image dimensions: 500x800 176 | Requested dimensions: 400x400 177 | 178 | Scale mode: fill 179 | New dimensions: 400x400 180 | Maintain aspect ratio: NO 181 | Cropping: NO 182 | 183 | Scale mode: aspect_fit 184 | New dimensions: 250x400 185 | Maintain aspect ratio: YES 186 | Cropping: NO 187 | 188 | Scale mode: aspect_fill 189 | New dimensions: 400x640 190 | Maintain aspect ratio: YES 191 | Cropping: NO 192 | 193 | Scale mode: aspect_crop 194 | New dimensions: 400x400 195 | Maintain aspect ratio: YES 196 | Cropping: YES 197 | 198 | ##### default_image_width 199 | 200 | In the absence of a width parameter in the request, use this as image width. A 201 | value of `0` sets no default. 202 | ##### default_image_height 203 | 204 | In the absence of a height parameter in the request, use this as image height. 205 | A value of `0` sets no default. 206 | 207 | ##### max_image_width 208 | 209 | Set a maximum image width. A value of `0` specifies no maximum. 210 | 211 | ##### max_image_height 212 | 213 | Set a maximum image height. A value of `0` specifies no maximum. 214 | 215 | ##### max_blur_radius_percentage 216 | 217 | Set a maximum blur radius percentage. A value of `0` disables blurring images. 218 | For Gaussian blur, the radius used is this value * the image width. This allows 219 | you to use a blur parameter (from 0-1) which will apply the same proportion of 220 | blurring to each image size. 221 | 222 | ##### auto_orient 223 | 224 | If set to true, the image processor will respect EXIF rotation data. A common 225 | case are photos taken with a camera (eg: iPhone, digital camera) in landscape 226 | mode. The built-in gyroscope will embed rotation data in the image via EXIF. 227 | 228 | Disabled by default. 229 | 230 | ##### formats 231 | 232 | ``` 233 | formats: { 234 | "large": { "width": 1280, "height": 768, "blur": 0 }, 235 | "medium": { "width": 640, "height": 480, "blur": 0 } 236 | } 237 | ``` 238 | 239 | If specified, the `w`, `h` and `blur` parameters will be ignored from the 240 | request. Instead will only be read the `format` parameter. 241 | 242 | ### Routes 243 | 244 | The `routes` block is a mapping of route patterns to route configuration values. 245 | 246 | The route pattern is a regular expression with a captured group for `image_path`. 247 | The subexpression match is the path that is requested from the image source. 248 | 249 | ##### name 250 | 251 | The name to use for the route. This is currently used in logging and StatsD key 252 | names. 253 | 254 | ##### source 255 | 256 | The name of the source to use for the route. 257 | 258 | ##### processor 259 | 260 | The name of the processor to use for the route. 261 | 262 | ##### cache_control 263 | 264 | The Cache-Control response header to set. If left empty or unspecified, `no-transform,public,max-age=86400,s-maxage=2592000` will be set. 265 | 266 | ### Health Checks 267 | 268 | You can check the server health at `/healthcheck` and `/health`. If the server 269 | is up and running, the HTTP client will receive a response with status code 270 | `200`. 271 | 272 | ## Adopters 273 | 274 | - [Oyster](https://www.oysterbooks.com) 275 | - [Storehouse](https://www.storehouse.co) 276 | 277 | If your organization is using Halfshell, consider adding a link and sending us a pull request! 278 | 279 | ## Contributing 280 | 281 | Contributions are welcome. 282 | 283 | ### Building 284 | 285 | There's a Vagrant file set up to ease development. After you have the 286 | Vagrant box set up, cd to the /vagrant directory and run `make`. 287 | 288 | ### Notes 289 | 290 | Run `make format` before sending any pull requests. 291 | 292 | ### Questions? 293 | 294 | File an issue or send an email to rafik@oysterbooks.com. 295 | -------------------------------------------------------------------------------- /VERSION: -------------------------------------------------------------------------------- 1 | 0.1.1-dev -------------------------------------------------------------------------------- /default.nix: -------------------------------------------------------------------------------- 1 | { nixpkgs ? 2 | , name ? "halfshell" 3 | , src ? ./. }: 4 | 5 | with import nixpkgs {}; 6 | 7 | with goPackages; let 8 | 9 | buildSrc = src; 10 | 11 | go-s3 = buildGoPackage rec { 12 | name = "go-s3"; 13 | goPackagePath = "github.com/oysterbooks/s3"; 14 | src = fetchFromGitHub { 15 | rev = "master"; 16 | owner = "oysterbooks"; 17 | repo = "s3"; 18 | sha256 = "0ql1i7b8qjrvh6bbh43vka9va7q15s98s1x2h7b1c5q3nsgn77sy"; 19 | }; 20 | }; 21 | 22 | go-imagick = buildGoPackage rec { 23 | name = "go-imagick"; 24 | goPackagePath = "github.com/rafikk/imagick"; 25 | propagatedBuildInputs = [ pkgconfig imagemagick ]; 26 | src = fetchFromGitHub { 27 | rev = "master"; 28 | owner = "rafikk"; 29 | repo = "imagick"; 30 | sha256 = "1paarlszxn63cwawgb5m0b1p8k35n6r34raps3383w5wnrqf6w2a"; 31 | }; 32 | }; 33 | 34 | go-halfshell = buildGoPackage rec { 35 | name = "go-halfshell"; 36 | goPackagePath = "github.com/oysterbooks/halfshell/halfshell"; 37 | propagatedBuildInputs = [ go-s3 go-imagick ]; 38 | src = builtins.toPath "${buildSrc}/halfshell"; 39 | }; 40 | 41 | in buildGoPackage { 42 | goPackagePath = "github.com/oysterbooks/halfshell"; 43 | name = name; 44 | src = buildSrc; 45 | propagatedBuildInputs = [ go-halfshell ]; 46 | } // { 47 | name = name; 48 | meta = { 49 | homepage = "https://github.com/oysterbooks/halfshell"; 50 | maintainers = [ 51 | "Rafik Salama " 52 | ]; 53 | }; 54 | } 55 | -------------------------------------------------------------------------------- /examples/filesystem_config.json: -------------------------------------------------------------------------------- 1 | { 2 | "server": { 3 | "port": 8080, 4 | "read_timeout": 5, 5 | "write_timeout": 30 6 | }, 7 | "statsd": { 8 | "host": "0", 9 | "port": 12345 10 | }, 11 | "sources": { 12 | "default": { 13 | "type": "filesystem", 14 | "directory": "/tmp" 15 | }, 16 | "blog-post-images": { 17 | "directory": "/tmp/my-company-blog-post-images" 18 | }, 19 | "profile-photos": { 20 | "directory": "/tmp/my-company-profile-photos" 21 | } 22 | }, 23 | "processors": { 24 | "default": { 25 | "image_compression_quality": 85, 26 | "default_scale_mode": "aspect_fit", 27 | "max_blur_radius_percentage": 0, 28 | "max_image_height": 0, 29 | "max_image_width": 1000 30 | }, 31 | "profile-photos": { 32 | "default_image_width": 120 33 | } 34 | }, 35 | "routes": { 36 | "^/blog(?P/.*)$": { 37 | "name": "blog-post-images", 38 | "source": "blog-post-images", 39 | "processor": "default" 40 | }, 41 | "^/users(?P/.*)$": { 42 | "name": "profile-photos", 43 | "source": "profile-photos", 44 | "processor": "profile-photos" 45 | } 46 | } 47 | } 48 | -------------------------------------------------------------------------------- /examples/http_config.json: -------------------------------------------------------------------------------- 1 | { 2 | "server": { 3 | "port": 8081, 4 | "read_timeout": 5, 5 | "write_timeout": 30 6 | }, 7 | "statsd": { 8 | "enabled": false 9 | }, 10 | "sources": { 11 | "default": { 12 | "type": "http", 13 | "host": "placekitten.com" 14 | } 15 | }, 16 | "processors": { 17 | "default": { 18 | "image_compression_quality": 85, 19 | "default_scale_mode": "aspect_fill", 20 | "max_blur_radius_percentage": 0, 21 | "max_image_height": 0, 22 | "max_image_width": 1000, 23 | "formats": { 24 | "large": { "width": 1700, "height": 1275, "blur": 0 }, 25 | "medium": { "width": 1136, "height": 852, "blur": 0 }, 26 | "small": { "width": 750, "height": 562, "blur": 0 }, 27 | "thumb": { "width": 120, "height": 90, "blur": 0 } 28 | } 29 | } 30 | }, 31 | "routes": { 32 | "(?P/.*)": { 33 | "name": "images", 34 | "source" :"default", 35 | "processor": "default" 36 | } 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /examples/s3_config.json: -------------------------------------------------------------------------------- 1 | { 2 | "server": { 3 | "port": 8080, 4 | "read_timeout": 5, 5 | "write_timeout": 30 6 | }, 7 | "statsd": { 8 | "enabled": false 9 | }, 10 | "sources": { 11 | "default": { 12 | "type": "s3", 13 | "s3_access_key": "", 14 | "s3_secret_key": "" 15 | }, 16 | "blog-post-images": { 17 | "s3_bucket": "my-company-blog-post-images" 18 | }, 19 | "profile-photos": { 20 | "s3_bucket": "my-company-profile-photos" 21 | } 22 | }, 23 | "processors": { 24 | "default": { 25 | "image_compression_quality": 85, 26 | "default_scale_mode": "aspect_fill", 27 | "max_blur_radius_percentage": 0, 28 | "max_image_height": 0, 29 | "max_image_width": 1000 30 | 31 | }, 32 | "profile-photos": { 33 | "default_image_width": 120 34 | } 35 | }, 36 | "routes": { 37 | "^/blog(?P/.*)$": { 38 | "name": "blog-post-images", 39 | "source": "blog-post-images", 40 | "processor": "default" 41 | }, 42 | "^/users(?P/.*)$": { 43 | "name": "profile-photos", 44 | "source": "profile-photos", 45 | "processor": "profile-photos" 46 | } 47 | } 48 | } 49 | -------------------------------------------------------------------------------- /halfshell/config.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "encoding/json" 25 | "fmt" 26 | "os" 27 | "reflect" 28 | "regexp" 29 | "strings" 30 | ) 31 | 32 | // Config is the primary configuration of Halfshell. It contains the server 33 | // configuration as well as a list of route configurations. 34 | type Config struct { 35 | ServerConfig *ServerConfig 36 | StatterConfig *StatterConfig 37 | RouteConfigs []*RouteConfig 38 | } 39 | 40 | // ServerConfig holds the configuration settings relevant for the HTTP server. 41 | type ServerConfig struct { 42 | Port uint64 43 | ReadTimeout uint64 44 | WriteTimeout uint64 45 | } 46 | 47 | // RouteConfig holds the configuration settings for a particular route. 48 | type RouteConfig struct { 49 | Name string 50 | CacheControl string 51 | Pattern *regexp.Regexp 52 | ImagePathIndex int 53 | SourceConfig *SourceConfig 54 | ProcessorConfig *ProcessorConfig 55 | } 56 | 57 | // SourceConfig holds the type information and configuration settings for a 58 | // particular image source. 59 | type SourceConfig struct { 60 | Name string 61 | Type ImageSourceType 62 | S3AccessKey string 63 | S3Bucket string 64 | S3SecretKey string 65 | Directory string 66 | Host string 67 | } 68 | 69 | // ProcessorConfig holds the configuration settings for the image processor. 70 | type ProcessorConfig struct { 71 | Name string 72 | ImageCompressionQuality uint64 73 | DefaultScaleMode uint 74 | DefaultImageHeight uint64 75 | DefaultImageWidth uint64 76 | MaxImageDimensions ImageDimensions 77 | MaxBlurRadiusPercentage float64 78 | AutoOrient bool 79 | Formats map[string]FormatConfig 80 | 81 | // DEPRECATED 82 | MaintainAspectRatio bool 83 | } 84 | 85 | type FormatConfig struct { 86 | Width uint64 87 | Height uint64 88 | Blur float64 89 | } 90 | 91 | // StatterConfig holds configuration data for StatsD 92 | type StatterConfig struct { 93 | Host string 94 | Port uint64 95 | Enabled bool 96 | } 97 | 98 | // NewConfigFromFile parses a JSON configuration file and returns a pointer to 99 | // a new Config object. 100 | func NewConfigFromFile(filepath string) *Config { 101 | parser := newConfigParser(filepath) 102 | config := parser.parse() 103 | return config 104 | } 105 | 106 | type configParser struct { 107 | filepath string 108 | data map[string]interface{} 109 | } 110 | 111 | func newConfigParser(filepath string) *configParser { 112 | file, err := os.Open(filepath) 113 | if err != nil { 114 | fmt.Fprintf(os.Stderr, "Unable to open file %s\n", filepath) 115 | os.Exit(1) 116 | } 117 | decoder := json.NewDecoder(file) 118 | parser := configParser{filepath: filepath} 119 | decoder.Decode(&parser.data) 120 | return &parser 121 | } 122 | 123 | func (c *configParser) parse() *Config { 124 | config := Config{ 125 | ServerConfig: c.parseServerConfig(), 126 | StatterConfig: c.parseStatterConfig(), 127 | } 128 | 129 | sourceConfigsByName := make(map[string]*SourceConfig) 130 | processorConfigsByName := make(map[string]*ProcessorConfig) 131 | 132 | for sourceName := range c.data["sources"].(map[string]interface{}) { 133 | sourceConfigsByName[sourceName] = c.parseSourceConfig(sourceName) 134 | } 135 | 136 | for processorName := range c.data["processors"].(map[string]interface{}) { 137 | processorConfigsByName[processorName] = c.parseProcessorConfig(processorName) 138 | } 139 | 140 | routesData := c.data["routes"].(map[string]interface{}) 141 | for routePatternString := range routesData { 142 | routeConfig := &RouteConfig{ImagePathIndex: -1} 143 | routeData := routesData[routePatternString].(map[string]interface{}) 144 | pattern, err := regexp.Compile(routePatternString) 145 | if err != nil { 146 | fmt.Fprintf(os.Stderr, "Invalid route pattern %s: %v\n", routePatternString, err) 147 | os.Exit(1) 148 | } 149 | 150 | for i, expName := range pattern.SubexpNames() { 151 | if expName == "image_path" { 152 | routeConfig.ImagePathIndex = i 153 | } 154 | } 155 | 156 | if routeConfig.ImagePathIndex == -1 { 157 | fmt.Fprintf(os.Stderr, "No 'image_path' named group in regex: %s\n", routePatternString) 158 | os.Exit(1) 159 | } 160 | 161 | processorKey := routeData["processor"].(string) 162 | sourceKey := routeData["source"].(string) 163 | 164 | routeConfig.Name = routeData["name"].(string) 165 | routeConfig.Pattern = pattern 166 | routeConfig.ProcessorConfig = processorConfigsByName[processorKey] 167 | routeConfig.SourceConfig = sourceConfigsByName[sourceKey] 168 | if _, ok := routeData["cache_control"]; ok { 169 | routeConfig.CacheControl = routeData["cache_control"].(string) 170 | } 171 | 172 | config.RouteConfigs = append(config.RouteConfigs, routeConfig) 173 | } 174 | 175 | return &config 176 | } 177 | 178 | func (c *configParser) parseServerConfig() *ServerConfig { 179 | return &ServerConfig{ 180 | Port: c.uintForKeypath("server.port"), 181 | ReadTimeout: c.uintForKeypath("server.read_timeout"), 182 | WriteTimeout: c.uintForKeypath("server.write_timeout"), 183 | } 184 | } 185 | 186 | func (c *configParser) parseStatterConfig() *StatterConfig { 187 | statsd, _ := c.data["statsd"].(map[string]interface{}) 188 | 189 | host, _ := statsd["host"].(string) 190 | if host == "" { 191 | host = "0" 192 | } 193 | 194 | port, _ := statsd["port"].(float64) 195 | if port == 0 { 196 | port = 8125 197 | } 198 | 199 | enabled, ok := statsd["enabled"] 200 | if ok { 201 | enabled, _ = enabled.(bool) 202 | } else { 203 | enabled = true 204 | } 205 | 206 | return &StatterConfig{ 207 | Host: host, 208 | Port: uint64(port), 209 | Enabled: enabled.(bool), 210 | } 211 | } 212 | 213 | func (c *configParser) parseSourceConfig(sourceName string) *SourceConfig { 214 | return &SourceConfig{ 215 | Name: sourceName, 216 | Type: ImageSourceType(c.stringForKeypath("sources.%s.type", sourceName)), 217 | S3AccessKey: c.stringForKeypath("sources.%s.s3_access_key", sourceName), 218 | S3SecretKey: c.stringForKeypath("sources.%s.s3_secret_key", sourceName), 219 | S3Bucket: c.stringForKeypath("sources.%s.s3_bucket", sourceName), 220 | Directory: c.stringForKeypath("sources.%s.directory", sourceName), 221 | Host: c.stringForKeypath("sources.%s.host", sourceName), 222 | } 223 | } 224 | 225 | func (c *configParser) parseProcessorConfig(processorName string) *ProcessorConfig { 226 | scaleModeName := c.stringForKeypath("processors.%s.default_scale_mode", processorName) 227 | scaleMode, _ := ScaleModes[scaleModeName] 228 | if scaleMode == 0 { 229 | scaleMode = ScaleFill 230 | } 231 | 232 | maxDimensions := ImageDimensions{ 233 | Width: uint(c.uintForKeypath("processors.%s.max_image_width", processorName)), 234 | Height: uint(c.uintForKeypath("processors.%s.max_image_height", processorName)), 235 | } 236 | 237 | formats := make(map[string]FormatConfig) 238 | processor := c.data["processors"].(map[string]interface{})[processorName].(map[string]interface{}) 239 | if _, ok := processor["formats"]; ok { 240 | for formatName := range processor["formats"].(map[string]interface{}) { 241 | format := FormatConfig{ 242 | Width: c.uintForKeypath("processors.%s.formats.%s.width", processorName, formatName), 243 | Height: c.uintForKeypath("processors.%s.formats.%s.height", processorName, formatName), 244 | Blur: c.floatForKeypath("processors.%s.formats.%s.blur", processorName, formatName), 245 | } 246 | formats[formatName] = format 247 | } 248 | } 249 | 250 | config := &ProcessorConfig{ 251 | Name: processorName, 252 | ImageCompressionQuality: c.uintForKeypath("processors.%s.image_compression_quality", processorName), 253 | DefaultScaleMode: scaleMode, 254 | DefaultImageHeight: c.uintForKeypath("processors.%s.default_image_height", processorName), 255 | DefaultImageWidth: c.uintForKeypath("processors.%s.default_image_width", processorName), 256 | MaxImageDimensions: maxDimensions, 257 | MaxBlurRadiusPercentage: c.floatForKeypath("processors.%s.max_blur_radius_percentage", processorName), 258 | AutoOrient: c.boolForKeypath("processors.%s.auto_orient", processorName), 259 | Formats: formats, 260 | 261 | // DEPRECATED 262 | MaintainAspectRatio: c.boolForKeypath("processors.%s.maintain_aspect_ratio", processorName), 263 | } 264 | 265 | if config.MaintainAspectRatio { 266 | config.DefaultScaleMode = ScaleAspectFit 267 | } 268 | 269 | return config 270 | } 271 | 272 | func (c *configParser) valueForKeypath(valueType reflect.Kind, keypathFormat string, v ...interface{}) interface{} { 273 | keypath := fmt.Sprintf(keypathFormat, v...) 274 | components := strings.Split(keypath, ".") 275 | var currentData = c.data 276 | for _, component := range components[:len(components)-1] { 277 | currentData = currentData[component].(map[string]interface{}) 278 | } 279 | value := currentData[components[len(components)-1]] 280 | if value == nil && len(v) > 0 { 281 | return c.valueForKeypath(valueType, fmt.Sprintf(keypathFormat, "default")) 282 | } 283 | 284 | switch value.(type) { 285 | case string, bool, float64: 286 | return value 287 | case nil: 288 | switch valueType { 289 | case reflect.Float64: 290 | return float64(0) 291 | case reflect.String: 292 | return "" 293 | case reflect.Bool: 294 | return false 295 | default: 296 | panic("Unreachable") 297 | } 298 | default: 299 | panic("Unreachable") 300 | } 301 | } 302 | 303 | func (c *configParser) stringForKeypath(keypathFormat string, v ...interface{}) string { 304 | return c.valueForKeypath(reflect.String, keypathFormat, v...).(string) 305 | } 306 | 307 | func (c *configParser) floatForKeypath(keypathFormat string, v ...interface{}) float64 { 308 | return c.valueForKeypath(reflect.Float64, keypathFormat, v...).(float64) 309 | } 310 | 311 | func (c *configParser) uintForKeypath(keypathFormat string, v ...interface{}) uint64 { 312 | return uint64(c.floatForKeypath(keypathFormat, v...)) 313 | } 314 | 315 | func (c *configParser) boolForKeypath(keypathFormat string, v ...interface{}) bool { 316 | return c.valueForKeypath(reflect.Bool, keypathFormat, v...).(bool) 317 | } 318 | -------------------------------------------------------------------------------- /halfshell/halfshell.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "os" 25 | "text/template" 26 | 27 | "github.com/rafikk/imagick/imagick" 28 | ) 29 | 30 | // Halfshell is the primary struct of the program. It holds onto the 31 | // configuration, the HTTP server, and all the routes. 32 | type Halfshell struct { 33 | Pid int 34 | Config *Config 35 | Routes []*Route 36 | Server *Server 37 | Logger *Logger 38 | } 39 | 40 | // NewWithConfig creates a new Halfshell instance from an instance of Config. 41 | func NewWithConfig(config *Config) *Halfshell { 42 | routes := make([]*Route, 0, len(config.RouteConfigs)) 43 | for _, routeConfig := range config.RouteConfigs { 44 | routes = append(routes, NewRouteWithConfig(routeConfig, config.StatterConfig)) 45 | } 46 | 47 | return &Halfshell{ 48 | Pid: os.Getpid(), 49 | Config: config, 50 | Routes: routes, 51 | Server: NewServerWithConfigAndRoutes(config.ServerConfig, routes), 52 | Logger: NewLogger("main"), 53 | } 54 | } 55 | 56 | // Run starts the Halfshell program. Performs global (de)initialization, and 57 | // starts the HTTP server. 58 | func (h *Halfshell) Run() { 59 | var tmpl, _ = template.New("start").Parse(StartupTemplateString) 60 | _ = tmpl.Execute(os.Stdout, h) 61 | 62 | imagick.Initialize() 63 | defer imagick.Terminate() 64 | 65 | h.Server.ListenAndServe() 66 | } 67 | -------------------------------------------------------------------------------- /halfshell/image.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "io" 26 | "io/ioutil" 27 | "os" 28 | "strconv" 29 | "strings" 30 | 31 | "github.com/rafikk/imagick/imagick" 32 | ) 33 | 34 | var EmptyImageDimensions = ImageDimensions{} 35 | var EmptyResizeDimensions = ResizeDimensions{} 36 | var DefaultFocalPoint = Focalpoint{0.5, 0.5} 37 | 38 | type Image struct { 39 | Wand *imagick.MagickWand 40 | Signature string 41 | destroyed bool 42 | } 43 | 44 | func NewImageFromBuffer(buffer io.Reader) (image *Image, err error) { 45 | bytes, err := ioutil.ReadAll(buffer) 46 | if err != nil { 47 | return nil, err 48 | } 49 | 50 | image = &Image{Wand: imagick.NewMagickWand()} 51 | err = image.Wand.ReadImageBlob(bytes) 52 | if err != nil { 53 | return nil, err 54 | } 55 | 56 | return image, nil 57 | } 58 | 59 | func NewImageFromFile(file *os.File) (image *Image, err error) { 60 | image, err = NewImageFromBuffer(file) 61 | return image, err 62 | } 63 | 64 | func (i *Image) GetMIMEType() string { 65 | return fmt.Sprintf("image/%s", strings.ToLower(i.Wand.GetImageFormat())) 66 | } 67 | 68 | func (i *Image) GetBytes() (bytes []byte, size int) { 69 | bytes = i.Wand.GetImageBlob() 70 | size = len(bytes) 71 | return bytes, size 72 | } 73 | 74 | func (i *Image) GetWidth() uint { 75 | return i.Wand.GetImageWidth() 76 | } 77 | 78 | func (i *Image) GetHeight() uint { 79 | return i.Wand.GetImageHeight() 80 | } 81 | 82 | func (i *Image) GetDimensions() ImageDimensions { 83 | return ImageDimensions{i.GetWidth(), i.GetHeight()} 84 | } 85 | 86 | func (i *Image) GetSignature() string { 87 | return i.Wand.GetImageSignature() 88 | } 89 | 90 | func (i *Image) Destroy() { 91 | if !i.destroyed { 92 | i.Wand.Destroy() 93 | i.destroyed = true 94 | } 95 | } 96 | 97 | type ImageDimensions struct { 98 | Width uint 99 | Height uint 100 | } 101 | 102 | func (d ImageDimensions) AspectRatio() float64 { 103 | return float64(d.Width) / float64(d.Height) 104 | } 105 | 106 | func (d ImageDimensions) String() string { 107 | return fmt.Sprintf("%dx%d", d.Width, d.Height) 108 | } 109 | 110 | type ResizeDimensions struct { 111 | Scale ImageDimensions 112 | Crop ImageDimensions 113 | } 114 | 115 | // Focalpoint is a float pair representing the location of the image subject. 116 | // (0.5, 0.5) is the middle. (1, 1) is the bottom right. (0, 0) is the top left. 117 | type Focalpoint struct { 118 | X float64 119 | Y float64 120 | } 121 | 122 | // NewFocalpointFromString splits the given string into a Focalpoint struct. The 123 | // string format should be: "X,Y". For example: "0.1,0.1". 124 | func NewFocalpointFromString(s string) (fp Focalpoint) { 125 | pair := strings.Split(s, ",") 126 | if len(pair) != 2 { 127 | return DefaultFocalPoint 128 | } 129 | 130 | x, err := strconv.ParseFloat(pair[0], 64) 131 | if err != nil { 132 | return DefaultFocalPoint 133 | } 134 | 135 | y, err := strconv.ParseFloat(pair[1], 64) 136 | if err != nil { 137 | return DefaultFocalPoint 138 | } 139 | 140 | return Focalpoint{x, y} 141 | } 142 | -------------------------------------------------------------------------------- /halfshell/image_processor.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "math" 25 | 26 | "github.com/rafikk/imagick/imagick" 27 | ) 28 | 29 | const ( 30 | ScaleFill = 10 31 | ScaleAspectFit = 21 32 | ScaleAspectFill = 22 33 | ScaleAspectCrop = 23 34 | ) 35 | 36 | var ScaleModes = map[string]uint{ 37 | "fill": ScaleFill, 38 | "aspect_fit": ScaleAspectFit, 39 | "aspect_fill": ScaleAspectFill, 40 | "aspect_crop": ScaleAspectCrop, 41 | } 42 | 43 | type ImageProcessor interface { 44 | ProcessImage(*Image, *ImageProcessorOptions) error 45 | } 46 | 47 | type ImageProcessorOptions struct { 48 | Dimensions ImageDimensions 49 | BlurRadius float64 50 | ScaleMode uint 51 | Focalpoint Focalpoint 52 | } 53 | 54 | type imageProcessor struct { 55 | Config *ProcessorConfig 56 | Logger *Logger 57 | } 58 | 59 | func NewImageProcessorWithConfig(config *ProcessorConfig) ImageProcessor { 60 | return &imageProcessor{ 61 | Config: config, 62 | Logger: NewLogger("image_processor.%s", config.Name), 63 | } 64 | } 65 | 66 | func (ip *imageProcessor) ProcessImage(img *Image, req *ImageProcessorOptions) error { 67 | if req.Dimensions == EmptyImageDimensions { 68 | req.Dimensions.Width = uint(ip.Config.DefaultImageWidth) 69 | req.Dimensions.Height = uint(ip.Config.DefaultImageHeight) 70 | } 71 | 72 | var err error 73 | 74 | err = ip.orient(img, req) 75 | if err != nil { 76 | ip.Logger.Errorf("Error orienting image: %s", err) 77 | return err 78 | } 79 | 80 | err = ip.resize(img, req) 81 | if err != nil { 82 | ip.Logger.Errorf("Error resizing image: %s", err) 83 | return err 84 | } 85 | 86 | err = ip.blur(img, req) 87 | if err != nil { 88 | ip.Logger.Errorf("Error blurring image: %s", err) 89 | return err 90 | } 91 | 92 | return nil 93 | } 94 | 95 | func (ip *imageProcessor) orient(img *Image, req *ImageProcessorOptions) error { 96 | if !ip.Config.AutoOrient { 97 | return nil 98 | } 99 | 100 | orientation := img.Wand.GetImageOrientation() 101 | 102 | switch orientation { 103 | case imagick.ORIENTATION_UNDEFINED: 104 | case imagick.ORIENTATION_TOP_LEFT: 105 | return nil 106 | } 107 | 108 | transparent := imagick.NewPixelWand() 109 | defer transparent.Destroy() 110 | transparent.SetColor("none") 111 | 112 | var err error 113 | 114 | switch orientation { 115 | case imagick.ORIENTATION_TOP_RIGHT: 116 | err = img.Wand.FlopImage() 117 | case imagick.ORIENTATION_BOTTOM_RIGHT: 118 | err = img.Wand.RotateImage(transparent, 180) 119 | case imagick.ORIENTATION_BOTTOM_LEFT: 120 | err = img.Wand.FlipImage() 121 | case imagick.ORIENTATION_LEFT_TOP: 122 | err = img.Wand.TransposeImage() 123 | case imagick.ORIENTATION_RIGHT_TOP: 124 | err = img.Wand.RotateImage(transparent, 90) 125 | case imagick.ORIENTATION_RIGHT_BOTTOM: 126 | err = img.Wand.TransverseImage() 127 | case imagick.ORIENTATION_LEFT_BOTTOM: 128 | err = img.Wand.RotateImage(transparent, 270) 129 | } 130 | 131 | if err != nil { 132 | return err 133 | } 134 | 135 | return img.Wand.SetImageOrientation(imagick.ORIENTATION_TOP_LEFT) 136 | } 137 | 138 | func (ip *imageProcessor) resize(img *Image, req *ImageProcessorOptions) error { 139 | scaleMode := req.ScaleMode 140 | if scaleMode == 0 { 141 | scaleMode = ip.Config.DefaultScaleMode 142 | } 143 | 144 | resize, err := ip.resizePrepare(img.GetDimensions(), req.Dimensions, scaleMode) 145 | if err != nil { 146 | return err 147 | } 148 | 149 | if resize.Scale != EmptyImageDimensions { 150 | err = ip.resizeApply(img, resize.Scale) 151 | if err != nil { 152 | return err 153 | } 154 | } 155 | 156 | if resize.Crop != EmptyImageDimensions { 157 | err = ip.cropApply(img, resize.Crop, req.Focalpoint) 158 | if err != nil { 159 | return err 160 | } 161 | } 162 | 163 | return nil 164 | } 165 | 166 | func (ip *imageProcessor) resizePrepare(oldDimensions, reqDimensions ImageDimensions, scaleMode uint) (*ResizeDimensions, error) { 167 | resize := &ResizeDimensions{ 168 | Scale: ImageDimensions{}, 169 | Crop: ImageDimensions{}, 170 | } 171 | 172 | if reqDimensions == EmptyImageDimensions { 173 | return resize, nil 174 | } 175 | if oldDimensions == reqDimensions { 176 | return resize, nil 177 | } 178 | 179 | reqDimensions = clampDimensionsToMaxima(oldDimensions, reqDimensions, ip.Config.MaxImageDimensions) 180 | oldAspectRatio := oldDimensions.AspectRatio() 181 | 182 | // Unspecified dimensions are automatically computed relative to the specified 183 | // dimension using the old image's aspect ratio. 184 | if reqDimensions.Width > 0 && reqDimensions.Height == 0 { 185 | reqDimensions.Height = aspectHeight(oldAspectRatio, reqDimensions.Width) 186 | } else if reqDimensions.Height > 0 && reqDimensions.Width == 0 { 187 | reqDimensions.Width = aspectWidth(oldAspectRatio, reqDimensions.Height) 188 | } 189 | 190 | // Retain the aspect ratio while at least filling the bounds requested. No 191 | // cropping will occur but the image will be resized. 192 | if scaleMode == ScaleAspectFit { 193 | newAspectRatio := reqDimensions.AspectRatio() 194 | if newAspectRatio > oldAspectRatio { 195 | resize.Scale.Width = aspectWidth(oldAspectRatio, reqDimensions.Height) 196 | resize.Scale.Height = reqDimensions.Height 197 | } else if newAspectRatio < oldAspectRatio { 198 | resize.Scale.Width = reqDimensions.Width 199 | resize.Scale.Height = aspectHeight(oldAspectRatio, reqDimensions.Width) 200 | } else { 201 | resize.Scale.Width = reqDimensions.Width 202 | resize.Scale.Height = reqDimensions.Height 203 | } 204 | return resize, nil 205 | } 206 | 207 | // Retain the aspect ratio while filling the bounds requested completely. New 208 | // dimensions are at least as large as the requested dimensions. No cropping 209 | // will occur but the image will be resized. 210 | if scaleMode == ScaleAspectFill { 211 | newAspectRatio := reqDimensions.AspectRatio() 212 | if newAspectRatio < oldAspectRatio { 213 | resize.Scale.Width = aspectWidth(oldAspectRatio, reqDimensions.Height) 214 | resize.Scale.Height = reqDimensions.Height 215 | } else if newAspectRatio > oldAspectRatio { 216 | resize.Scale.Width = reqDimensions.Width 217 | resize.Scale.Height = aspectHeight(oldAspectRatio, reqDimensions.Width) 218 | } else { 219 | resize.Scale.Width = reqDimensions.Width 220 | resize.Scale.Height = reqDimensions.Height 221 | } 222 | return resize, nil 223 | } 224 | 225 | // Use exact width/height and clip off the parts that bleed. The image is 226 | // first resized to ensure clipping occurs on the smallest edges possible. 227 | if scaleMode == ScaleAspectCrop { 228 | newAspectRatio := reqDimensions.AspectRatio() 229 | if newAspectRatio > oldAspectRatio { 230 | resize.Scale.Width = reqDimensions.Width 231 | resize.Scale.Height = aspectHeight(oldAspectRatio, reqDimensions.Width) 232 | } else if newAspectRatio < oldAspectRatio { 233 | resize.Scale.Width = aspectWidth(oldAspectRatio, reqDimensions.Height) 234 | resize.Scale.Height = reqDimensions.Height 235 | } else { 236 | resize.Scale.Width = reqDimensions.Width 237 | resize.Scale.Height = reqDimensions.Height 238 | } 239 | resize.Crop.Width = reqDimensions.Width 240 | resize.Crop.Height = reqDimensions.Height 241 | return resize, nil 242 | } 243 | 244 | // Use the new dimensions exactly as is. Don't correct for aspect ratio and 245 | // don't do any cropping. This is equivalent to ScaleFill. 246 | resize.Scale = reqDimensions 247 | return resize, nil 248 | } 249 | 250 | func (ip *imageProcessor) resizeApply(img *Image, dimensions ImageDimensions) error { 251 | if dimensions == EmptyImageDimensions { 252 | return nil 253 | } 254 | 255 | err := img.Wand.ResizeImage(dimensions.Width, dimensions.Height, imagick.FILTER_LANCZOS, 1) 256 | if err != nil { 257 | ip.Logger.Errorf("Failed resizing image: %s", err) 258 | return err 259 | } 260 | 261 | err = img.Wand.SetImageInterpolateMethod(imagick.INTERPOLATE_PIXEL_BICUBIC) 262 | if err != nil { 263 | ip.Logger.Errorf("Failed getting interpolation method: %s", err) 264 | return err 265 | } 266 | 267 | err = img.Wand.StripImage() 268 | if err != nil { 269 | ip.Logger.Errorf("Failed stripping image metadata: %s", err) 270 | return err 271 | } 272 | 273 | if img.Wand.GetImageFormat() == "JPEG" { 274 | err = img.Wand.SetInterlaceScheme(imagick.INTERLACE_PLANE) 275 | if err != nil { 276 | ip.Logger.Errorf("Failed setting image interlace scheme: %s", err) 277 | return err 278 | } 279 | 280 | err = img.Wand.SetImageCompression(imagick.COMPRESSION_JPEG) 281 | if err != nil { 282 | ip.Logger.Errorf("Failed setting image compression type: %s", err) 283 | return err 284 | } 285 | 286 | err = img.Wand.SetImageCompressionQuality(uint(ip.Config.ImageCompressionQuality)) 287 | if err != nil { 288 | ip.Logger.Errorf("Failed setting compression quality: %s", err) 289 | return err 290 | } 291 | } 292 | 293 | return nil 294 | } 295 | 296 | func (ip *imageProcessor) cropApply(img *Image, reqDimensions ImageDimensions, focalpoint Focalpoint) error { 297 | oldDimensions := img.GetDimensions() 298 | x := int(focalpoint.X * (float64(oldDimensions.Width) - float64(reqDimensions.Width))) 299 | y := int(focalpoint.Y * (float64(oldDimensions.Height) - float64(reqDimensions.Height))) 300 | w := reqDimensions.Width 301 | h := reqDimensions.Height 302 | return img.Wand.CropImage(w, h, x, y) 303 | } 304 | 305 | func (ip *imageProcessor) blur(image *Image, request *ImageProcessorOptions) error { 306 | if request.BlurRadius == 0 { 307 | return nil 308 | } 309 | blurRadius := float64(image.GetWidth()) * request.BlurRadius * ip.Config.MaxBlurRadiusPercentage 310 | return image.Wand.GaussianBlurImage(blurRadius, blurRadius) 311 | } 312 | 313 | func aspectHeight(aspectRatio float64, width uint) uint { 314 | return uint(math.Floor(float64(width)/aspectRatio + 0.5)) 315 | } 316 | 317 | func aspectWidth(aspectRatio float64, height uint) uint { 318 | return uint(math.Floor(float64(height)*aspectRatio + 0.5)) 319 | } 320 | 321 | func clampDimensionsToMaxima(imgDimensions, reqDimensions, maxDimensions ImageDimensions) ImageDimensions { 322 | if maxDimensions.Width > 0 && reqDimensions.Width > maxDimensions.Width { 323 | reqDimensions.Width = maxDimensions.Width 324 | reqDimensions.Height = aspectHeight(imgDimensions.AspectRatio(), maxDimensions.Width) 325 | return clampDimensionsToMaxima(imgDimensions, reqDimensions, maxDimensions) 326 | } 327 | 328 | if maxDimensions.Height > 0 && reqDimensions.Height > maxDimensions.Height { 329 | reqDimensions.Width = aspectWidth(imgDimensions.AspectRatio(), maxDimensions.Height) 330 | reqDimensions.Height = maxDimensions.Height 331 | return clampDimensionsToMaxima(imgDimensions, reqDimensions, maxDimensions) 332 | } 333 | 334 | return reqDimensions 335 | } 336 | -------------------------------------------------------------------------------- /halfshell/logger.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "log" 26 | "os" 27 | ) 28 | 29 | type Logger struct { 30 | *log.Logger 31 | Name string 32 | } 33 | 34 | func NewLogger(nameFormat string, v ...interface{}) *Logger { 35 | return &Logger{ 36 | log.New(os.Stdout, "", log.Ldate|log.Lmicroseconds), 37 | fmt.Sprintf(nameFormat, v...), 38 | } 39 | } 40 | 41 | func (l *Logger) Logf(level, format string, v ...interface{}) { 42 | l.Printf("[%s] [%s] %s", level, l.Name, fmt.Sprintf(format, v...)) 43 | } 44 | 45 | func (l *Logger) Debugf(format string, v ...interface{}) { 46 | l.Logf("DEBUG", format, v...) 47 | } 48 | 49 | func (l *Logger) Infof(format string, v ...interface{}) { 50 | l.Logf("INFO", format, v...) 51 | } 52 | 53 | func (l *Logger) Warnf(format string, v ...interface{}) { 54 | l.Logf("WARNING", format, v...) 55 | } 56 | 57 | func (l *Logger) Errorf(format string, v ...interface{}) { 58 | l.Logf("ERROR", format, v...) 59 | } 60 | -------------------------------------------------------------------------------- /halfshell/route.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "net/http" 25 | "regexp" 26 | "strconv" 27 | ) 28 | 29 | // A Route handles the business logic of a Halfshell request. It contains a 30 | // Processor and a Source. When a request is serviced, the appropriate route 31 | // is chosen after which the image is retrieved from the source and 32 | // processed by the processor. 33 | type Route struct { 34 | Name string 35 | Pattern *regexp.Regexp 36 | ImagePathIndex int 37 | Processor ImageProcessor 38 | Formats map[string]FormatConfig 39 | Source ImageSource 40 | CacheControl string 41 | Statter Statter 42 | } 43 | 44 | // NewRouteWithConfig returns a pointer to a new Route instance created using 45 | // the provided configuration settings. 46 | func NewRouteWithConfig(config *RouteConfig, statterConfig *StatterConfig) *Route { 47 | return &Route{ 48 | Name: config.Name, 49 | Pattern: config.Pattern, 50 | ImagePathIndex: config.ImagePathIndex, 51 | CacheControl: config.CacheControl, 52 | Processor: NewImageProcessorWithConfig(config.ProcessorConfig), 53 | Formats: config.ProcessorConfig.Formats, 54 | Source: NewImageSourceWithConfig(config.SourceConfig), 55 | Statter: NewStatterWithConfig(config, statterConfig), 56 | } 57 | } 58 | 59 | // ShouldHandleRequest accepts an HTTP request and returns a bool indicating 60 | // whether the route should handle the request. 61 | func (p *Route) ShouldHandleRequest(r *http.Request) bool { 62 | return p.Pattern.MatchString(r.URL.Path) 63 | } 64 | 65 | // SourceAndProcessorOptionsForRequest parses the source and processor options 66 | // from the request. 67 | func (p *Route) SourceAndProcessorOptionsForRequest(r *http.Request) ( 68 | *ImageSourceOptions, *ImageProcessorOptions) { 69 | 70 | matches := p.Pattern.FindAllStringSubmatch(r.URL.Path, -1)[0] 71 | path := matches[p.ImagePathIndex] 72 | 73 | var width, height uint64 74 | var blurRadius float64 75 | if formatName := r.FormValue("format"); formatName == "" { 76 | width, _ = strconv.ParseUint(r.FormValue("w"), 10, 32) 77 | height, _ = strconv.ParseUint(r.FormValue("h"), 10, 32) 78 | blurRadius, _ = strconv.ParseFloat(r.FormValue("blur"), 64) 79 | } else { 80 | width = p.Formats[formatName].Width 81 | height = p.Formats[formatName].Height 82 | blurRadius = p.Formats[formatName].Blur 83 | } 84 | 85 | focalpoint := r.FormValue("focalpoint") 86 | scaleModeName := r.FormValue("scale_mode") 87 | scaleMode, _ := ScaleModes[scaleModeName] 88 | 89 | return &ImageSourceOptions{Path: path}, &ImageProcessorOptions{ 90 | Dimensions: ImageDimensions{uint(width), uint(height)}, 91 | BlurRadius: blurRadius, 92 | ScaleMode: uint(scaleMode), 93 | Focalpoint: NewFocalpointFromString(focalpoint), 94 | } 95 | } 96 | -------------------------------------------------------------------------------- /halfshell/server.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "net" 26 | "net/http" 27 | "time" 28 | ) 29 | 30 | type Server struct { 31 | *http.Server 32 | Routes []*Route 33 | Logger *Logger 34 | } 35 | 36 | func NewServerWithConfigAndRoutes(config *ServerConfig, routes []*Route) *Server { 37 | httpServer := &http.Server{ 38 | Addr: fmt.Sprintf(":%d", config.Port), 39 | ReadTimeout: time.Duration(config.ReadTimeout) * time.Second, 40 | WriteTimeout: time.Duration(config.WriteTimeout) * time.Second, 41 | MaxHeaderBytes: 1 << 20, 42 | } 43 | server := &Server{httpServer, routes, NewLogger("server")} 44 | httpServer.Handler = server 45 | return server 46 | } 47 | 48 | func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { 49 | hw := s.NewResponseWriter(w) 50 | hr := s.NewRequest(r) 51 | defer s.LogRequest(hw, hr) 52 | switch { 53 | case "/healthcheck" == hr.URL.Path || "/health" == hr.URL.Path: 54 | hw.Write([]byte("OK")) 55 | default: 56 | s.ImageRequestHandler(hw, hr) 57 | } 58 | } 59 | 60 | func (s *Server) ImageRequestHandler(w *ResponseWriter, r *Request) { 61 | if r.Route == nil { 62 | w.WriteError(fmt.Sprintf("No route available to handle request: %v", 63 | r.URL.Path), http.StatusNotFound) 64 | return 65 | } 66 | 67 | defer func() { go r.Route.Statter.RegisterRequest(w, r) }() 68 | 69 | s.Logger.Infof("Handling request for image %s with dimensions %v", 70 | r.SourceOptions.Path, r.ProcessorOptions.Dimensions) 71 | 72 | image, err := r.Route.Source.GetImage(r.SourceOptions) 73 | if err != nil { 74 | w.WriteError("Not Found", http.StatusNotFound) 75 | return 76 | } 77 | defer image.Destroy() 78 | 79 | err = r.Route.Processor.ProcessImage(image, r.ProcessorOptions) 80 | if err != nil { 81 | s.Logger.Warnf("Error processing image data %s to dimensions: %v", r.ProcessorOptions.Dimensions) 82 | w.WriteError("Internal Server Error", http.StatusNotFound) 83 | return 84 | } 85 | 86 | s.Logger.Infof("Returning resized image %s to dimensions %v", 87 | r.SourceOptions.Path, r.ProcessorOptions.Dimensions) 88 | 89 | cacheControl := r.Route.CacheControl 90 | if r.Route.CacheControl == "" { 91 | cacheControl = "no-transform,public,max-age=86400,s-maxage=2592000" 92 | } 93 | w.SetHeader("Cache-Control", cacheControl) 94 | w.WriteImage(image) 95 | } 96 | 97 | func (s *Server) LogRequest(w *ResponseWriter, r *Request) { 98 | logFormat := "%s - - [%s] \"%s %s %s\" %d %d\n" 99 | host, _, err := net.SplitHostPort(r.RemoteAddr) 100 | if err != nil { 101 | host = r.RemoteAddr 102 | } 103 | fmt.Printf(logFormat, host, r.Timestamp.Format("02/Jan/2006:15:04:05 -0700"), 104 | r.Method, r.URL.RequestURI(), r.Proto, w.Status, w.Size) 105 | } 106 | 107 | type Request struct { 108 | *http.Request 109 | Timestamp time.Time 110 | Route *Route 111 | SourceOptions *ImageSourceOptions 112 | ProcessorOptions *ImageProcessorOptions 113 | } 114 | 115 | func (s *Server) NewRequest(r *http.Request) *Request { 116 | request := &Request{r, time.Now(), nil, nil, nil} 117 | for _, route := range s.Routes { 118 | if route.ShouldHandleRequest(r) { 119 | request.Route = route 120 | } 121 | } 122 | 123 | if request.Route != nil { 124 | request.SourceOptions, request.ProcessorOptions = 125 | request.Route.SourceAndProcessorOptionsForRequest(r) 126 | } 127 | 128 | return request 129 | } 130 | 131 | // ResponseWriter is a wrapper around http.ResponseWriter that provides 132 | // access to the response status and size after they have been set. 133 | type ResponseWriter struct { 134 | w http.ResponseWriter 135 | Status int 136 | Size int 137 | } 138 | 139 | // NewResponseWriter creates a new ResponseWriter by wrapping http.ResponseWriter. 140 | func (s *Server) NewResponseWriter(w http.ResponseWriter) *ResponseWriter { 141 | return &ResponseWriter{w: w} 142 | } 143 | 144 | // WriteHeader forwards to http.ResponseWriter's WriteHeader method. 145 | func (hw *ResponseWriter) WriteHeader(status int) { 146 | hw.Status = status 147 | hw.w.WriteHeader(status) 148 | } 149 | 150 | // SetHeader sets the value for a response header. 151 | func (hw *ResponseWriter) SetHeader(name, value string) { 152 | hw.w.Header().Set(name, value) 153 | } 154 | 155 | // Writes data the output stream. 156 | func (hw *ResponseWriter) Write(data []byte) (int, error) { 157 | hw.Size += len(data) 158 | return hw.w.Write(data) 159 | } 160 | 161 | // WriteError writes an error response. 162 | func (hw *ResponseWriter) WriteError(message string, status int) { 163 | hw.SetHeader("Content-Type", "text/plain; charset=utf-8") 164 | hw.WriteHeader(status) 165 | hw.Write([]byte(message)) 166 | } 167 | 168 | // WriteImage writes an image to the output stream and sets the appropriate headers. 169 | func (hw *ResponseWriter) WriteImage(image *Image) { 170 | bytes, size := image.GetBytes() 171 | hw.SetHeader("Content-Type", image.GetMIMEType()) 172 | hw.SetHeader("Content-Length", fmt.Sprintf("%d", size)) 173 | hw.SetHeader("ETag", image.GetSignature()) 174 | hw.WriteHeader(http.StatusOK) 175 | hw.Write(bytes) 176 | } 177 | -------------------------------------------------------------------------------- /halfshell/source.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "os" 26 | ) 27 | 28 | type ImageSourceType string 29 | type ImageSourceFactoryFunction func(*SourceConfig) ImageSource 30 | 31 | var ( 32 | imageSourceTypeToFactoryFunctionMap = make(map[ImageSourceType]ImageSourceFactoryFunction) 33 | ) 34 | 35 | type ImageSource interface { 36 | GetImage(*ImageSourceOptions) (*Image, error) 37 | } 38 | 39 | type ImageSourceOptions struct { 40 | Path string 41 | } 42 | 43 | func RegisterSource(sourceType ImageSourceType, factory ImageSourceFactoryFunction) { 44 | imageSourceTypeToFactoryFunctionMap[sourceType] = factory 45 | } 46 | 47 | func NewImageSourceWithConfig(config *SourceConfig) ImageSource { 48 | factory := imageSourceTypeToFactoryFunctionMap[config.Type] 49 | if factory == nil { 50 | fmt.Fprintf(os.Stderr, "Unknown image source type: %s\n", config.Type) 51 | os.Exit(1) 52 | } 53 | return factory(config) 54 | } 55 | -------------------------------------------------------------------------------- /halfshell/source_filesystem.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "os" 25 | "path/filepath" 26 | "strings" 27 | ) 28 | 29 | const ( 30 | ImageSourceTypeFilesystem ImageSourceType = "filesystem" 31 | ) 32 | 33 | type FileSystemImageSource struct { 34 | Config *SourceConfig 35 | Logger *Logger 36 | } 37 | 38 | func NewFileSystemImageSourceWithConfig(config *SourceConfig) ImageSource { 39 | source := &FileSystemImageSource{ 40 | Config: config, 41 | Logger: NewLogger("source.fs.%s", config.Name), 42 | } 43 | 44 | baseDirectory, err := os.Open(source.Config.Directory) 45 | if os.IsNotExist(err) { 46 | source.Logger.Infof(source.Config.Directory, " does not exit. Creating.") 47 | _ = os.MkdirAll(source.Config.Directory, 0700) 48 | baseDirectory, err = os.Open(source.Config.Directory) 49 | } 50 | 51 | if err != nil { 52 | source.Logger.Fatal(err) 53 | } 54 | 55 | fileInfo, err := baseDirectory.Stat() 56 | if err != nil || !fileInfo.IsDir() { 57 | source.Logger.Fatal("Directory ", source.Config.Directory, " not a directory", err) 58 | } 59 | 60 | return source 61 | } 62 | 63 | func (s *FileSystemImageSource) GetImage(request *ImageSourceOptions) (*Image, error) { 64 | fileName := s.fileNameForRequest(request) 65 | 66 | file, err := os.Open(fileName) 67 | if err != nil { 68 | s.Logger.Warnf("Failed to open file: %v", err) 69 | return nil, err 70 | } 71 | 72 | image, err := NewImageFromFile(file) 73 | if err != nil { 74 | s.Logger.Warnf("Failed to read image: %v", err) 75 | return nil, err 76 | } 77 | 78 | return image, nil 79 | } 80 | 81 | func (s *FileSystemImageSource) fileNameForRequest(request *ImageSourceOptions) string { 82 | // Remove the leading / from the file name and replace the 83 | // directory separator (/) with something safe for file names (_) 84 | return filepath.Join(s.Config.Directory, strings.Replace(strings.TrimLeft(request.Path, string(filepath.Separator)), string(filepath.Separator), "_", -1)) 85 | } 86 | 87 | func init() { 88 | RegisterSource(ImageSourceTypeFilesystem, NewFileSystemImageSourceWithConfig) 89 | } 90 | -------------------------------------------------------------------------------- /halfshell/source_http.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "io/ioutil" 26 | "net/http" 27 | "net/url" 28 | "strings" 29 | ) 30 | 31 | const ( 32 | ImageSourceTypeHttp ImageSourceType = "http" 33 | ) 34 | 35 | type HttpImageSource struct { 36 | Config *SourceConfig 37 | Logger *Logger 38 | } 39 | 40 | func NewHttpImageSourceWithConfig(config *SourceConfig) ImageSource { 41 | return &HttpImageSource{ 42 | Config: config, 43 | Logger: NewLogger("source.http.%s", config.Name), 44 | } 45 | } 46 | 47 | func (s *HttpImageSource) GetImage(request *ImageSourceOptions) (*Image, error) { 48 | httpRequest := s.getHttpRequest(request) 49 | httpResponse, err := http.DefaultClient.Do(httpRequest) 50 | defer httpResponse.Body.Close() 51 | if err != nil { 52 | s.Logger.Warnf("Error downlading image: %v", err) 53 | return nil, err 54 | } 55 | if httpResponse.StatusCode != 200 { 56 | return nil, fmt.Errorf("Error downlading image (url=%v)", httpRequest.URL) 57 | } 58 | image, err := NewImageFromBuffer(httpResponse.Body) 59 | if err != nil { 60 | responseBody, _ := ioutil.ReadAll(httpResponse.Body) 61 | s.Logger.Warnf("Unable to create image from response body: %v (url=%v)", string(responseBody), httpRequest.URL) 62 | return nil, err 63 | } 64 | s.Logger.Infof("Successfully retrieved image from http: %v", httpRequest.URL) 65 | return image, nil 66 | } 67 | 68 | func (s *HttpImageSource) getHttpRequest(request *ImageSourceOptions) *http.Request { 69 | path := s.Config.Directory + request.Path 70 | imageURLPathComponents := strings.Split(path, "/") 71 | 72 | for index, component := range imageURLPathComponents { 73 | component = url.QueryEscape(component) 74 | imageURLPathComponents[index] = component 75 | } 76 | requestURL := &url.URL{ 77 | Opaque: strings.Join(imageURLPathComponents, "/"), 78 | Scheme: "http", 79 | Host: s.Config.Host, 80 | } 81 | 82 | httpRequest, _ := http.NewRequest("GET", requestURL.RequestURI(), nil) 83 | httpRequest.URL = requestURL 84 | 85 | return httpRequest 86 | } 87 | 88 | func init() { 89 | RegisterSource(ImageSourceTypeHttp, NewHttpImageSourceWithConfig) 90 | } 91 | -------------------------------------------------------------------------------- /halfshell/source_s3.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "io/ioutil" 26 | "net/http" 27 | "net/url" 28 | "strings" 29 | "time" 30 | 31 | "github.com/oysterbooks/s3" 32 | ) 33 | 34 | const ( 35 | ImageSourceTypeS3 ImageSourceType = "s3" 36 | ) 37 | 38 | type S3ImageSource struct { 39 | Config *SourceConfig 40 | Logger *Logger 41 | } 42 | 43 | func NewS3ImageSourceWithConfig(config *SourceConfig) ImageSource { 44 | return &S3ImageSource{ 45 | Config: config, 46 | Logger: NewLogger("source.s3.%s", config.Name), 47 | } 48 | } 49 | 50 | func (s *S3ImageSource) GetImage(request *ImageSourceOptions) (*Image, error) { 51 | httpRequest := s.signedHTTPRequestForRequest(request) 52 | httpResponse, err := http.DefaultClient.Do(httpRequest) 53 | defer httpResponse.Body.Close() 54 | if err != nil { 55 | s.Logger.Warnf("Error downlading image: %v", err) 56 | return nil, err 57 | } 58 | if httpResponse.StatusCode != 200 { 59 | return nil, fmt.Errorf("Error downlading image (url=%v)", httpRequest.URL) 60 | } 61 | image, err := NewImageFromBuffer(httpResponse.Body) 62 | if err != nil { 63 | responseBody, _ := ioutil.ReadAll(httpResponse.Body) 64 | s.Logger.Warnf("Unable to create image from response body: %v (url=%v)", string(responseBody), httpRequest.URL) 65 | return nil, err 66 | } 67 | s.Logger.Infof("Successfully retrieved image from S3: %v", httpRequest.URL) 68 | return image, nil 69 | } 70 | 71 | func (s *S3ImageSource) signedHTTPRequestForRequest(request *ImageSourceOptions) *http.Request { 72 | path := s.Config.Directory + request.Path 73 | imageURLPathComponents := strings.Split(path, "/") 74 | 75 | for index, component := range imageURLPathComponents { 76 | component = url.QueryEscape(component) 77 | imageURLPathComponents[index] = component 78 | } 79 | requestURL := &url.URL{ 80 | Opaque: strings.Join(imageURLPathComponents, "/"), 81 | Scheme: "http", 82 | Host: fmt.Sprintf("%s.s3.amazonaws.com", s.Config.S3Bucket), 83 | } 84 | 85 | httpRequest, _ := http.NewRequest("GET", requestURL.RequestURI(), nil) 86 | httpRequest.URL = requestURL 87 | httpRequest.Header.Set("Date", time.Now().UTC().Format(http.TimeFormat)) 88 | s3.Sign(httpRequest, s3.Keys{ 89 | AccessKey: s.Config.S3AccessKey, 90 | SecretKey: s.Config.S3SecretKey, 91 | }) 92 | 93 | return httpRequest 94 | } 95 | 96 | func init() { 97 | RegisterSource(ImageSourceTypeS3, NewS3ImageSourceWithConfig) 98 | } 99 | -------------------------------------------------------------------------------- /halfshell/statter.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | import ( 24 | "fmt" 25 | "net" 26 | "net/http" 27 | "os" 28 | "time" 29 | ) 30 | 31 | type Statter interface { 32 | RegisterRequest(*ResponseWriter, *Request) 33 | } 34 | 35 | type statsdStatter struct { 36 | conn *net.UDPConn 37 | addr *net.UDPAddr 38 | Name string 39 | Hostname string 40 | Logger *Logger 41 | Enabled bool 42 | } 43 | 44 | func NewStatterWithConfig(routeConfig *RouteConfig, statterConfig *StatterConfig) Statter { 45 | logger := NewLogger("stats.%s", routeConfig.Name) 46 | hostname, _ := os.Hostname() 47 | 48 | addr, err := net.ResolveUDPAddr( 49 | "udp", fmt.Sprintf("%s:%d", statterConfig.Host, statterConfig.Port)) 50 | if err != nil { 51 | logger.Errorf("Unable to resolve UDP address: %v", err) 52 | return nil 53 | } 54 | 55 | conn, err := net.DialUDP("udp", nil, addr) 56 | if err != nil { 57 | logger.Errorf("Unable to create UDP connection: %v", err) 58 | return nil 59 | } 60 | 61 | return &statsdStatter{ 62 | conn: conn, 63 | addr: addr, 64 | Name: routeConfig.Name, 65 | Hostname: hostname, 66 | Logger: logger, 67 | Enabled: statterConfig.Enabled, 68 | } 69 | } 70 | 71 | func (s *statsdStatter) RegisterRequest(w *ResponseWriter, r *Request) { 72 | if !s.Enabled { 73 | return 74 | } 75 | 76 | now := time.Now() 77 | 78 | status := "success" 79 | if w.Status != http.StatusOK { 80 | status = "failure" 81 | } 82 | 83 | s.count(fmt.Sprintf("http.status.%d", w.Status)) 84 | s.count(fmt.Sprintf("image_resized.%s", status)) 85 | s.count(fmt.Sprintf("image_resized_%s.%s", r.ProcessorOptions.Dimensions, status)) 86 | 87 | if status == "success" { 88 | durationInMs := (now.UnixNano() - r.Timestamp.UnixNano()) / 1000000 89 | s.time("image_resized", durationInMs) 90 | s.time(fmt.Sprintf("image_resized_%s", r.ProcessorOptions.Dimensions), durationInMs) 91 | } 92 | } 93 | 94 | func (s *statsdStatter) count(stat string) { 95 | stat = fmt.Sprintf("%s.halfshell.%s.%s", s.Hostname, s.Name, stat) 96 | s.Logger.Infof("Incrementing counter: %s", stat) 97 | s.send(stat, "1|c") 98 | } 99 | 100 | func (s *statsdStatter) time(stat string, time int64) { 101 | stat = fmt.Sprintf("%s.halfshell.%s.%s", s.Hostname, s.Name, stat) 102 | s.Logger.Infof("Registering time: %s (%d)", stat, time) 103 | s.send(stat, fmt.Sprintf("%d|ms", time)) 104 | } 105 | 106 | func (s *statsdStatter) send(stat string, value string) { 107 | data := fmt.Sprintf("%s:%s", stat, value) 108 | n, err := s.conn.Write([]byte(data)) 109 | if err != nil { 110 | s.Logger.Errorf("Error sending data to statsd: %v", err) 111 | } else if n == 0 { 112 | s.Logger.Errorf("No bytes were written") 113 | } 114 | } 115 | -------------------------------------------------------------------------------- /halfshell/templates.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package halfshell 22 | 23 | const StartupTemplateString = ` 24 | ┬ ┬┌─┐┬ ┌─┐┌─┐┬ ┬┌─┐┬ ┬ 25 | ├─┤├─┤│ ├┤ └─┐├─┤├┤ │ │ 26 | ┴ ┴┴ ┴┴─┘└ └─┘┴ ┴└─┘┴─┘┴─┘ 27 | 28 | Running on process {{.Pid}} 29 | 30 | Server settings: 31 | Port: {{.Config.ServerConfig.Port}} 32 | Read Timeout: {{.Config.ServerConfig.ReadTimeout}} 33 | Write Timeout: {{.Config.ServerConfig.WriteTimeout}} 34 | 35 | StatsD settings: 36 | Host: {{.Config.StatterConfig.Host}} 37 | Port: {{.Config.StatterConfig.Port}} 38 | Enabled: {{.Config.StatterConfig.Enabled}} 39 | 40 | Routes: 41 | {{ range $index, $route := .Routes }} {{ $route.Name }}: 42 | Pattern: {{ $route.Pattern }} 43 | {{ end }} 44 | ` 45 | -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2014 Oyster 2 | // 3 | // Permission is hereby granted, free of charge, to any person obtaining a copy 4 | // of this software and associated documentation files (the "Software"), to deal 5 | // in the Software without restriction, including without limitation the rights 6 | // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | // copies of the Software, and to permit persons to whom the Software is 8 | // furnished to do so, subject to the following conditions: 9 | // 10 | // The above copyright notice and this permission notice shall be included in 11 | // all copies or substantial portions of the Software. 12 | // 13 | // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | // IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | // AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | // THE SOFTWARE. 20 | 21 | package main 22 | 23 | import ( 24 | "fmt" 25 | "github.com/oysterbooks/halfshell/halfshell" 26 | "os" 27 | ) 28 | 29 | func main() { 30 | if len(os.Args) < 2 || os.Args[1] == "" { 31 | fmt.Fprintf(os.Stderr, "usage: %s [config]\n", os.Args[0]) 32 | os.Exit(1) 33 | } 34 | 35 | config := halfshell.NewConfigFromFile(os.Args[1]) 36 | halfshell := halfshell.NewWithConfig(config) 37 | halfshell.Run() 38 | } 39 | -------------------------------------------------------------------------------- /release.nix: -------------------------------------------------------------------------------- 1 | { nixpkgs ? 2 | , src ? { outPath = ./.; gitTag = "dirty"; } 3 | , system ? builtins.currentSystem }: 4 | 5 | let 6 | pkgs = import nixpkgs { inherit system; }; 7 | 8 | in with pkgs; rec { 9 | tarball = releaseTools.sourceTarball { 10 | src = src; 11 | name = "halfshell"; 12 | version = src.gitTag; 13 | versionSuffix = ""; 14 | doBuild = true; 15 | distPhase = '' 16 | mkdir -p $out/tarballs/ 17 | tar czf $out/tarballs/halfshell-${src.gitTag}.tar.gz . 18 | ''; 19 | }; 20 | 21 | build = import ./default.nix { 22 | inherit nixpkgs; 23 | name = "halfshell-${src.gitTag}"; 24 | src = src; 25 | }; 26 | } 27 | --------------------------------------------------------------------------------