├── .gitignore ├── Gemfile ├── Gemfile.lock ├── README.md ├── _config.yml ├── build_site.sh ├── jekyllmail.rb ├── lib ├── blog.rb ├── j_m_logger.rb └── j_mail.rb └── run_jekyllmail.sh /.gitignore: -------------------------------------------------------------------------------- 1 | _config.yml.real 2 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source "http://rubygems.org" 2 | 3 | gem 'mail' 4 | gem 'nokogiri' 5 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: http://rubygems.org/ 3 | specs: 4 | mail (2.7.1) 5 | mini_mime (>= 0.1.1) 6 | mini_mime (1.0.1) 7 | mini_portile2 (2.4.0) 8 | nokogiri (1.10.1) 9 | mini_portile2 (~> 2.4.0) 10 | rufo (0.6.0) 11 | 12 | PLATFORMS 13 | ruby 14 | 15 | DEPENDENCIES 16 | mail 17 | nokogiri 18 | rufo 19 | 20 | BUNDLED WITH 21 | 1.17.2 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # JekyllMail # 2 | 3 | JekyllMail enables you to post to your [Jekyll](https://github.com/mojombo/jekyll) 4 | or [Octopress](http://octopress.org/) powered blog by email. 5 | 6 | ## How it Works ## 7 | Once configured (see below) JekyllMail will log into a POP3 account, check for messages with a pre-defined secret in the subject line, convert them into appropriately named files, and save them in your `_posts` directory. Images will be extracted and saved in a date specific directory under your Jekyll images directory. 8 | 9 | Please note. JekyllMail assumes that the address it is checking is *exclusively* for its use and will only be used to post emails to a single blog. JekyllMail does support multiple blogs but you will need a seperate e-mail account for each one. 10 | 11 | 12 | ## Usage ## 13 | The magic is all in the subject line. In order to differentiate your email from the spam that's almost guaranteed to find your account eventually suck in the appropriate metadata A subject line for JekyllMail has two parts the title (of your post) and the metadata which will go into the YAML frontmatter Jekyll needs. The metadata is a series of key value pairs separated by slashes. One of those key value pairs *must* be "secret" and the secret listed in your configuration. Note that the keys must contain no spaces and be immediately followed by a colon. 14 | 15 | || key: value / key: value / key: value, value, value 16 | An example: 17 | 18 | My Awesome Post || secret: more-1337 / tags: awesome, excellent, spectacular 19 | 20 | Your secret should be short, easy to remember, easy to type, and very unlikely to show up in an e-mail from another human or spammer. 21 | 22 | Your e-mail can be formatted in Markdown, Textile, or HTML. 23 | 24 | ### Subject Metadata ### 25 | Metadata is separated from your subject by a double pipe. There are a handful of keys that JekyllMail is specifically looking for in the subject. 26 | **All of these are optional except "secret"**: 27 | 28 | * published: defaults to true. Set this to "false" to prevent the post from being published. 29 | * markup: can be: `html`, `markdown`, or `textile` 30 | * tags: expects a comma separated list of tags for your post 31 | * slug: the "slug" for the file-name. E.g. yyyy-mm-dd-*slug*.extension 32 | 33 | If you don't provide a slug JekyllMail will just convert your title to a slug 34 | 35 | ### Images ### 36 | Image attachments will be extracted by JekyllMail and placed in dated directory 37 | that corresponds with the date of the posting. 38 | 39 | For example If you attached flag.jpg to a post sent on July 4th 2012 it would be 40 | stored in `/2012/07/04/flag.jpg` 41 | 42 | 43 | JekyllMail will look for the image tags in your document that reference the image 44 | filename and update them to point to the correct published file path. For example 45 | it will convert `![alt text](flag.jpg)` in a Markdown document to 46 | `![alt text](http://example.com/path/to/images/dir/2012/07/04/flag.jpg)`. 47 | Textile and HTML posts are also supported. 48 | 49 | In practice this simply means that if you insert a `![alt text](flag.jpg)` 50 | tag and attach an image named `flag.jpg` to the same email everything will 51 | show up as expected in your post even though JekyllMail has moved that image 52 | off to a dated subdirectory (just like the post's url). 53 | 54 | ## Installation ## 55 | Clone this git repo on your server, cd into the resulting directory, and 56 | run `bundle install` to make sure all the required gems are present. 57 | 58 | Required Gems: 59 | 60 | * bundler 61 | * jekyll 62 | * nokogiri 63 | * mail 64 | 65 | After editing the `_config.yml` file (in JekyllMail) you'll need to wire things up to rebuild after each commit. Instructions are below. 66 | 67 | ## Configuration ## 68 | JekyllMail is configured via a \_config.yml file in its root directory. 69 | Within this are a couple global settings and a series of "blog" stanzas one for each blog you'll have it checking mail for. 70 | 71 | A config file for a single blog will look something like this: 72 | 73 | ```yaml 74 | --- 75 | debug: false 76 | blogs: 77 | - name: my blog name 78 | active: true 79 | markup: markdown 80 | jekyll_blog_dir: /Users/masukomi/workspace/jekyllmail_test_site 81 | images_dir_under_jekyll: assets/img 82 | posts_dir_under_jekyll: _posts 83 | images_dir_under_site_url: /assets/img 84 | 85 | origin_repo_branch: master 86 | 87 | local_repo: /Users/masukomi/workspace/jekyllmail_test_site 88 | origin_repo: /Users/masukomi/workspace/jekyllmail_test_site 89 | pop_server: mail.example.com 90 | pop_user: jekyllmail@example.com 91 | pop_password: a_really_good_password_goes_here 92 | secret: jekyllmail 93 | markup: markdown 94 | site_url: http://blog.example.com 95 | commit_after_save: true 96 | delete_after_run: true 97 | 98 | ``` 99 | 100 | ### Configuration Notes ### 101 | 102 | `jekyll_blog_dir` is the absolute path to your Jekyll install on the same box as the `jekyllmail.rb` script. 103 | 104 | `images_dir_under_jekyll` defaults to `assets/img` in current Jekyll deploys. It represents the directory under your Jekyll root where images are stored. 105 | 106 | `post_dir_under_jekyll` defaults to `_posts` in current Jekyll deploys. 107 | 108 | `images_dir_under_site_url` is `/assets/img` with a default Jekyll configuration. It means that you would serve an image from `https:///assets/img/foo.jpg` 109 | 110 | `origin_repo_branch` JekyllMail can push changes from your local 111 | Jekyll install to a remote repo. It assumes that remote repo will be named `origin` but you can configure what branch it will push to. 112 | 113 | `local_repo` JekyllMail assumes that your local Jekyll install is a git repository. if `origin_repo` is different it will try and push to it. 114 | 115 | The `secret` is a short piece of text that must appear in the subject of 116 | each email. This is used to filter out the spam and will never be posted. 117 | 118 | `local_repo` is where JekyllMail will store your files during its run. This will be automatically configured to push to `origin_repo` if `commit_after_save` and `origin_repo` are specified. Any new posts and images to the `local_repo` will be pushed to `origin_repo`, and then deleted after the run is complete. 119 | 120 | `delete_after_run` tells JekyllMail to delete the emails after successfully processing them. You probably want this set to true unless you're debugging and want to reprocess the same email(s) repeatedly. 121 | 122 | 123 | Please note that paths must *not* end with a slash. Your `pop_user` doesn't have to be an e-mail address. It might just be "jekyllmail", or whatever username you've chosen for the e-mail account. It all depends on how your server is configured. It's probably best to use something other than "jekyllmail" though. 124 | 125 | ## Wiring Things up ## 126 | You need to schedule a cronjob to run regularly to kick of JekyllMail and check for new e-mails. 127 | 128 | To kick of JekyllMail you'll want a script that looks something like this. 129 | You can use the `run_jekyllmail.sh` file that comes with JekyllMail as 130 | a template. 131 | 132 | ```bash 133 | #!/bin/sh 134 | cd /full/path/to/jekyllmail 135 | bundle exec ruby jekyllmail.rb 136 | ``` 137 | 138 | 139 | Save the file anywhere that isn't served up to the public, make it executable, and add a new line to your [crontab](http://crontab.org/) to run it every five minutes or so. This is an example crontab line to do this 140 | 141 | ``` 142 | 4,9,14,19,24,29,34,39,44,49,54,59 * * * * /home/my_username/jekyllmail_repo/run_jekyllmail.sh 143 | ``` 144 | 145 | When JekyllMail finds something it will save the files in the appropriate locations and commit them to the appropriate git repo. When it does we can leverage git's `hooks/post-commit` to regenerate the HTML without needing to unnecessarily rebuild it on a regular interval. 146 | 147 | The `hooks/post-commit` file in your repo should look something like this. Don't forget to make it executable. You can use the `build_site.sh` file that comes with JekyllMail as a templote. 148 | 149 | ```bash 150 | #!/bin/sh 151 | cd /full/path/to/blog 152 | jekyll build 153 | cp -r _site /full/path/to/the/directory/your/site/is/hosted/from 154 | ``` 155 | 156 | Depending on your server's ruby / gem configuration you may have to add some additional info to the top of those scripts ( just below the `#!/bin/sh` ). On a system with a locally installed RVM and gems directory the top of your script might look something like this: 157 | 158 | ```bash 159 | #!/bin/sh 160 | [[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function* 161 | GEM_PATH=$GEM_PATH:/home/my_username/.gems 162 | PATH=$PATH:/home/my_username/.gems/bin 163 | ``` 164 | 165 | ## Warning ## 166 | At the end of every run JekyllMail *deletes every e-mail* in the account. 167 | This is for two reasons: 168 | 169 | 1. We don't want to have to maintain a list of what e-mails we've already ingested and posted 170 | 2. Once an e-mail's been ingested we don't need it 171 | 3. There are probably 400 spam e-mails in the account that should be deleted anyway. 172 | 4. less e-mail in the box means faster runs 173 | 174 | Ok, four reasons. 175 | 176 | If you want to disable this set the `delete_after_run` configuration setting to false. 177 | 178 | ## Developers ## 179 | If you set the `debug` option at the top if the configuration file to `true` it will cause a bunch of debug statements to be printed during the run, and logged to the log file. It will also prevent it from deleting the e-mails at the end. It's much easier to work on JekyllMail when you don't have to keep sending it new e-mails. 180 | 181 | Have fun, and remember to send in pull-requests. :) 182 | 183 | ### Known Issues ### 184 | Check out the [Issues page](https://github.com/masukomi/JekyllMail/issues) on 185 | Github for the current list of known issues (if any). 186 | 187 | ## Credit where credit is due ## 188 | JekyllMail was based on a [post & gist](http://tedkulp.com/2011/05/18/send-email-to-jekyll/) by [Ted Kulp](http://tedkulp.com/), but has come a long way since then. 189 | 190 | ## License ## 191 | JekyllMail is distributed under the [MIT License](http://www.opensource.org/licenses/mit-license.php). 192 | 193 | 194 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # log_file is optional. Simply leave it undefined to not log to a file 3 | log_file: /Users/masukomi/workspace/temp/jekyllmail.log 4 | ## Setting debug to true places the script 5 | ## in a debug state, which results in some verbose 6 | ## output and overrides delete_after_run 7 | debug: true 8 | blogs: 9 | - name: my blog name 10 | active: true 11 | # set active: false to temporarily pause the ingestion 12 | # of emails for this blog without having to disable 13 | # the cron job 14 | markup: markdown 15 | jekyll_blog_dir: /Users/masukomi/workspace/jekyllmail_test_site 16 | images_dir_under_jekyll: assets/img 17 | posts_dir_under_jekyll: _posts 18 | # SEE https://jekyllrb.com/docs/static-files/ 19 | # for details on how to configure where your static files live 20 | # by default images are expected to be stored in 21 | # /assets/img in jekyll 22 | # so the default value for images_dir_under_site_url 23 | # would be /assets/img because the web page would 24 | # link to an image at https:///assets/img/foo.jpg 25 | images_dir_under_site_url: /assets/img 26 | 27 | origin_repo_branch: master 28 | 29 | local_repo: /Users/masukomi/workspace/jekyllmail_test_site 30 | # typically this is a local repo that JekyllMail updates 31 | # and pushes to an origin repo, but it can just be a local 32 | # directory you want JekyllMail to store your files under. 33 | origin_repo: /Users/masukomi/workspace/jekyllmail_test_site 34 | # origin_repo could also be me@example.com:path/to/blog.git 35 | # orign_repo_branch defaults to master 36 | pop_server: mail.example.com 37 | pop_user: jekyllmail@example.com 38 | pop_password: a_really_good_password_here 39 | secret: jekyllmail 40 | markup: markdown 41 | site_url: http://blog.example.com 42 | commit_after_save: true 43 | delete_after_run: true 44 | -------------------------------------------------------------------------------- /build_site.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # CHANGE ME TO MEET THE NEEDS OF YOUR SERVER CONFIGURATION 4 | 5 | [[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function* 6 | GEM_PATH=$GEM_PATH:/home/my_username/.gems 7 | PATH=$PATH:/home/my_username/.gems/bin 8 | cd /home/my_username/jekyll/ 9 | bundle exec rake generate 10 | -------------------------------------------------------------------------------- /jekyllmail.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # -*- coding: utf-8 -*- #specify UTF-8 (unicode) characters 3 | 4 | #Email to Jekyll script 5 | #(c)2011 Ted Kulp 6 | # Portions copyright 2011 masukomi 7 | # POP3 support and Git config integration added by masukomi 8 | #MIT license -- Have fun 9 | #Most definitely a work in progress 10 | 11 | # TODO 12 | # error handling: 13 | # - complain if any of the required blog are not defined 14 | 15 | $LOAD_PATH.push File.expand_path(File.join(File.dirname(__FILE__), "lib")) 16 | 17 | require 'rubygems' 18 | require 'yaml' 19 | require 'net/pop' 20 | require 'mail' 21 | require 'nokogiri' 22 | require 'fileutils' 23 | #require 'grit' 24 | #include Grit 25 | 26 | 27 | require 'j_m_logger' # lib/j_m_logger.rb 28 | require 'blog' # lib/blog.rb 29 | require 'j_mail' # lib/j_mail.rb 30 | 31 | # The following constants can all be overridden in the config file 32 | @@globals={:debug=>false, :delete_after_run=>true} 33 | 34 | #JEKYLLMAIL_USER= Actor.from_string("JekyllMail Script ") 35 | 36 | 37 | # this is the _config.yml file in your JekyllMail install 38 | # NOT the _config.yml in your Jekyll install 39 | yaml = YAML::load(File.open('_config.yml')) 40 | @@globals[:delete_after_run] = yaml['delete_after_run'] ? true : false 41 | @@globals[:workspace] = yaml['workspace'] 42 | 43 | 44 | @@logger = JMLogger.new(yaml['log_file'], yaml['debug']) 45 | 46 | 47 | blogs = yaml['blogs'] 48 | blogs.each do | blog_data | 49 | blog = Blog.new(blog_data, @@logger) 50 | 51 | #TODO break this out into its own class 52 | Mail.defaults do 53 | retriever_method :pop3, :address => blog.pop_server, 54 | :port => 995, 55 | :user_name => blog.pop_user, 56 | :password => blog.pop_password, 57 | :enable_ssl => true 58 | end 59 | 60 | emails = Mail.all 61 | 62 | if (emails.length == 0 ) 63 | @@logger.log( "No Emails found") 64 | next #move on to the next blog's config 65 | else 66 | @@logger.log( "#{emails.length} email(s) found" ) 67 | end 68 | 69 | emails.each do | mail | 70 | jmail = JMail.new(blog, @@logger) 71 | valid_mail = jmail.process_mail(mail) 72 | if @@globals[:debug] && ! vail_mail 73 | @@logger.log("mail was invalid:\n#{mail.inspect}") 74 | end 75 | end 76 | Mail.delete_all() unless @@globals[:debug] == true or @@globals[:delete_after_run] == false 77 | # when debugging it's much easier to just leave the emails there and re-use them 78 | 79 | end 80 | -------------------------------------------------------------------------------- /lib/blog.rb: -------------------------------------------------------------------------------- 1 | class Blog 2 | DIRECTORY_KEYS=['jekyll_blog_dir', 3 | 'images_dir_under_jekyll', 4 | 'posts_dir_under_jekyll' 5 | ] 6 | 7 | attr_reader :data 8 | # TODO: CORRECT THE FOLLOWING COMMENTS 9 | # the blog hash contains 10 | # local_repo => absolute path to the root of a repo/dir 11 | # exclusively for JekyllMail's use 12 | # origin_repo => where JekyllMail should push new files to 13 | # origin_repo_branch => (optional) the name of the branch to push to 14 | # pop_server => domain name 15 | # pop_user => username 16 | # pop_password => plaintext password 17 | # secret => the secret that must appear in the email subject 18 | # markup => markup or textile 19 | # site_url => the http://.... url to the root of the public web site 20 | # commit_after_save => boolean 21 | # git_branch => the name of the git branch to commit to 22 | ## git_branch is Unused until we get Grit working correctly 23 | def initialize(yaml_data, logger) 24 | @data = yaml_data 25 | @logger = logger 26 | @data['images_dir_under_jekyll'] ||= 'assets/img' 27 | @data['posts_dir_under_jekyll'] ||= '_posts' 28 | @data['jekyll_blog_dir'] ||= 'source' 29 | @data['name'] ||= 'default blog' 30 | @data['origin_repo_branch'] ||= 'master' 31 | @data['markup'] ||= 'markdown' 32 | DIRECTORY_KEYS.each do | key | 33 | @data[key].sub!(/\/$/, '') # remove any trailing slashes from directory paths 34 | end 35 | @data.each do |key, value| 36 | @logger.log( "#{@data['name']} #{key}: #{value}" ) if @logger 37 | end 38 | confirm_local_repo(@data) 39 | end 40 | 41 | def confirm_local_repo(data) 42 | unless Dir.exists? data['jekyll_blog_dir'] 43 | raise "Can't find jekyll_blog_dir for #{data['name']} at #{data['jekyll_blog_dir']}" 44 | end 45 | end 46 | 47 | def jekyll_dir 48 | return @data['jekyll_blog_dir'] 49 | end 50 | def posts_dir 51 | return jekyll_dir() + "/#{@data['posts_dir_under_jekyll']}" 52 | end 53 | def images_dir 54 | return jekyll_dir() + "/#{@data['images_dir_under_jekyll']}" 55 | end 56 | def images_dir_under_jekyll 57 | return @data['images_dir_under_jekyll'] 58 | end 59 | def images_dir_under_site_url 60 | return @data['images_dir_under_site_url'] 61 | end 62 | 63 | def pop_server 64 | return @data['pop_server'] 65 | end 66 | 67 | def pop_user 68 | return @data['pop_user'] 69 | end 70 | 71 | def pop_password 72 | return @data['pop_password'] 73 | end 74 | 75 | def markup 76 | return @data['markup'] 77 | end 78 | 79 | def secret 80 | return @data['secret'] 81 | end 82 | 83 | def site_url 84 | return @data['site_url'] 85 | end 86 | 87 | def add_to_git? 88 | true 89 | end 90 | 91 | def commit(files_to_commit, slug) 92 | Dir.chdir(@data['jekyll_blog_dir']) 93 | files_to_commit.each do |file| 94 | # relative_file_name = file.sub(/.*?source\//, 'source/') 95 | @logger.log("adding #{file}") 96 | #index.add(relative_file_name, open(file, "rb") {|io| io.read }) 97 | #repo_specific_file_name, binary_data 98 | # git doesn't care if you use absolute file paths 99 | # from within a repo 100 | `git add #{file}` 101 | end 102 | @logger.log("committing") 103 | #sha = index.commit("Adding post #{slug} via JekyllMail", parents, JEKYLLMAIL_USER, nil, blog['git_branch']) 104 | #puts "sha = #{sha}" if @@globals[:debug] 105 | `git commit -m "Adding post #{slug} via JekyllMail"` 106 | # `git pull --rebase origin #{@data['origin_repo_branch']}` 107 | if (@data['jekyll_blog_dir'] != @data['origin_repo']) 108 | result = `git push origin #{@data['origin_repo_branch']} 2>&1` 109 | if (result.match(/\[rejected\]/)) 110 | @logger.log("error pushing #{@data['name']}'s new post to git:", true) 111 | result.split(/\n/).each do | line | 112 | @logger.log(line, true) 113 | end 114 | end 115 | end 116 | 117 | end 118 | 119 | end 120 | -------------------------------------------------------------------------------- /lib/j_m_logger.rb: -------------------------------------------------------------------------------- 1 | class JMLogger 2 | 3 | def initialize(log_file, debug) 4 | @debug = debug ? true : false 5 | @log_file = log_file ? File.open(log_file, "a") : nil 6 | if (@debug) 7 | log("initialized logger") 8 | end 9 | end 10 | 11 | def log(message, write_to_file=false) 12 | puts message if @debug 13 | if @debug or (@log_file and write_to_file) 14 | t = Time.now 15 | @log_file.puts "[#{t.strftime("%m/%d/%y %H:%M:%S")}] #{message}" 16 | end 17 | end 18 | 19 | 20 | 21 | end 22 | -------------------------------------------------------------------------------- /lib/j_mail.rb: -------------------------------------------------------------------------------- 1 | class JMail 2 | MARKUP_EXTENSIONS = { :html => "html", :markdown => "md", :md => "md", :textile => "textile", :txt => "textile" } 3 | @files_to_commit = [] 4 | attr_reader :files_to_commit 5 | 6 | def initialize(blog, logger) 7 | @blog = blog 8 | @logger = logger 9 | end 10 | 11 | 12 | #TODO: OMG THIS IS WAY TOO LONG 13 | def process_mail(mail) 14 | @files_to_commit = [] 15 | keyvals = { 16 | :tags => "", # a YAML array 17 | :markup => @blog.markup, 18 | :slug => nil, 19 | :published => true, 20 | :layout => "post", 21 | } 22 | 23 | subject = mail.subject 24 | 25 | return false unless is_subject_valid?(subject, keyvals) 26 | 27 | # vvv -- updates the contents of keyvals 28 | # and adds kevals[:title] 29 | keyvals = extract_data_from_subject(subject, keyvals) 30 | return false unless is_secret_valid?(@blog, keyvals[:secret], subject) 31 | @logger.log("XXX secret was valid") 32 | keyvals = update_and_infer_keyvals(@blog, keyvals) 33 | body = "" 34 | images_needing_replacement = {} 35 | if mail.multipart? 36 | #vvv updates images_needing_replacement 37 | body = process_multipart_mail(mail, body, images_needing_replacement, keyvals) 38 | else 39 | #Just grab the body no matter what it is 40 | body = mail.body.decoded 41 | end 42 | #If we have no body after all that, bail 43 | @logger.log("XXX will return if body empty") 44 | return false if body.strip.empty? 45 | 46 | @logger.log("XXX body wasn't empty") 47 | 48 | body = cleanup_html_and_replace_images( 49 | body, 50 | keyvals[:markup], 51 | images_needing_replacement 52 | ) 53 | 54 | @logger.log("XXX calling write_to_disk") 55 | write_to_disk(body, keyvals) 56 | @logger.log("XXX wrote to disk") 57 | 58 | log_files_to_commit(@files_to_commit) if @files_to_commit.size > 0 59 | 60 | 61 | if @blog.add_to_git? and @files_to_commit.size() > 0 62 | @blog.commit(@files_to_commit, keyvals[:slug]) 63 | end 64 | 65 | return @files_to_commit.size > 0 66 | end 67 | 68 | def log_files_to_commit(files_to_commit) 69 | message = "JekyllMail ingested these files:" 70 | files_to_commit.each do |f| 71 | message += "\n\t#{f}" 72 | end 73 | @logger.log(message, true) 74 | message 75 | end 76 | 77 | def cleanup_html_and_replace_images(body, markup, images_needing_replacement) 78 | body = cleanup_html(body, markup) 79 | body = replace_all_images( 80 | images_needing_replacement, 81 | body, 82 | markup 83 | ) 84 | body 85 | end 86 | 87 | def cleanup_html(body, markup) 88 | return body if markup != 'html' 89 | Nokogiri::HTML::DocumentFragment.parse(body.strip).to_html 90 | end 91 | 92 | def replace_all_images(images_needing_replacement, body, markup) 93 | images_needing_replacement.each do |filename, path| 94 | body = replace_images(body, markup, filename, path) 95 | end 96 | body 97 | end 98 | 99 | def replace_images(body, markup, filename, path) 100 | case markup 101 | when 'markdown' 102 | return replace_markdown_images(body, markup, filename, path) 103 | when 'textile' 104 | return replace_textile_images(body, markup, filename, path) 105 | when 'html' 106 | return replace_html_images(body, markup, filename, path) 107 | else 108 | raise "Unrecognized markup for image replacement: \"#{markup}\"" 109 | end 110 | end 111 | 112 | def set_name_and_time(keyvals) 113 | time = Time.now 114 | keyvals[:name] = "%02d-%02d-%02d-%s.%s" % [time.year, time.month, time.day, keyvals[:slug], MARKUP_EXTENSIONS[keyvals[:markup].to_sym]] 115 | keyvals[:time] = time 116 | keyvals 117 | end 118 | 119 | def set_slug(keyvals) 120 | keyvals[:slug] ||= keyvals[:title].gsub(/[^[:alnum:]]+/, "-").downcase.strip.gsub(/\A\-+|\-+\z/, "") 121 | @logger.log("new slug: #{keyvals[:slug]} from title: #{keyvals[:title]}") 122 | keyvals 123 | end 124 | 125 | def update_and_infer_keyvals(blog, keyvals) 126 | keyvals.delete(:secret) 127 | keyvals = set_slug(keyvals) 128 | keyvals = set_name_and_time(keyvals) # needs a slug 129 | keyvals 130 | end 131 | 132 | def is_secret_valid?(blog, test_secret, subject) 133 | # if it doesn't contain the secret we can assume it to be spam 134 | valid = (blog.secret == test_secret) 135 | @logger.log( 136 | "skipping email with invalid / non-existent secret.\n\tSubject:#{subject}\n\tSecret was: \"#{test_secret}\"", 137 | true) unless valid 138 | return valid 139 | end 140 | 141 | def is_subject_valid?(subject, keyvals) 142 | # || key: value / key: value / key: value, value, value 143 | @logger.log("processing email with subject: #{subject}") 144 | return false if subject.strip.empty? 145 | return false unless /\s*\S+.*\|\|/.match(subject) 146 | return false unless /[\/ ]secret: \S+/.match(subject) 147 | return true 148 | end 149 | def extract_data_from_subject(subject, keyvals) 150 | (title, raw_data) = subject.split(/\|\|/) # two pipes separate subject from data 151 | title.gsub!(/^\s+|\s+$/, "") 152 | unless raw_data.nil? 153 | datums = raw_data.split("/") 154 | datums.each do |datum| 155 | next if datum.nil? 156 | 157 | (key, val) = datum.split(/:\s?/) 158 | key.gsub!(/\s+/, "") 159 | val.gsub!(/\s+$/, "") 160 | keyvals[key.to_sym] = val 161 | end 162 | end 163 | keyvals[:title] = title 164 | keyvals 165 | end 166 | 167 | # TODO: refactor me into multiple methods 168 | def process_multipart_mail(mail, body, images_needing_replacement, keyvals) 169 | html_part = -1 170 | txt_part = -1 171 | 172 | #Figure out which part is html and which 173 | #is text 174 | mail.parts.each_with_index do |p, idx| 175 | if p.content_type.start_with?("text/html") 176 | html_part = idx 177 | elsif p.content_type.start_with?("text/plain") 178 | txt_part = idx 179 | end 180 | end 181 | 182 | mail.attachments.each do |attachment| 183 | #TODO: break this out into a separate method. 184 | if (attachment.content_type.start_with?("image/")) 185 | attachment_filename = attachment.filename 186 | images_dir = @blog.images_dir_under_jekyll + ("/%02d/%02d/%02d" % [keyvals[:time].year, keyvals[:time].month, keyvals[:time].day]) 187 | local_images_dir = "#{@blog.jekyll_dir}/#{images_dir}" 188 | FileUtils.mkdir_p(local_images_dir) 189 | puts "local_images_dir: #{local_images_dir}" 190 | images_needing_replacement[attachment_filename] = "/#{images_dir}/#{attachment_filename}" 191 | puts "image url: #{images_needing_replacement[attachment_filename]}" 192 | begin 193 | local_filename = "#{@blog.jekyll_dir}/#{images_dir}/#{attachment_filename}" 194 | @logger.log("saving image to #{local_filename}") 195 | unless File.writable?(local_images_dir) 196 | $stderr.puts("ERROR: #{local_images_dir} is unwritable. Exiting.") 197 | end 198 | File.open(local_filename, "w+b", 0644) { |f| f.write attachment.body.decoded } 199 | # @files_to_commit << "#{images_dir}/#{attachment_filename}" 200 | @files_to_commit << local_filename 201 | rescue Exception => e 202 | $stderr.puts "Unable to save data for #{attachment_filename} because #{e.message}" 203 | end 204 | end 205 | end 206 | 207 | #If the markup isn't html, try and use the 208 | #text if it exists. Anything else, use the html 209 | #version 210 | if txt_part > -1 and keyvals[:markup] != "html" 211 | body = mail.parts[txt_part].body.decoded 212 | elsif html_part > -1 213 | body = mail.parts[html_part].body.decoded 214 | end 215 | body 216 | end 217 | 218 | def write_to_disk(body, keyvals) 219 | post_filename = "#{@blog.posts_dir}/#{keyvals[:name]}" 220 | 221 | if File.writable?("#{@blog.posts_dir}") 222 | @logger.log("saving post to #{post_filename}") 223 | open(post_filename, 'w'){|f| 224 | f.puts body 225 | } 226 | else 227 | $stderr.puts "ERROR: #{@blog.posts_dir} is not writable" 228 | exit 0 229 | end 230 | open(post_filename, "w") do |str| 231 | str << "---\n" 232 | str << "title: '#{keyvals[:title]}'\n" 233 | str << "date: %02d-%02d-%02d %02d:%02d:%02d\n" % [keyvals[:time].year, keyvals[:time].month, keyvals[:time].day, keyvals[:time].hour, keyvals[:time].min, keyvals[:time].sec] 234 | keyvals.keys.sort.each do |key| 235 | if key != :tags and key != :slug 236 | str << "#{key}: #{keyvals[key]}\n" 237 | elsif key == :tags 238 | unless keyvals[:tags].empty? 239 | str << "tags: \n" 240 | keyvals[:tags].split(",").each do |string| 241 | str << "- " + string.strip + "\n" 242 | end 243 | end 244 | end 245 | end 246 | str << "---\n" 247 | str << body 248 | end 249 | @files_to_commit << post_filename 250 | end 251 | 252 | private 253 | 254 | def replace_markdown_images(body, markup, filename, path) 255 | return body if markup != 'markdown' 256 | body.gsub(/(\(|\]:\s|<)#{Regexp.escape(filename)}/, "\\1#{path}") 257 | end 258 | 259 | def replace_textile_images(body, markup, filename, path) 260 | return body if markup != 'textile' 261 | body.gsub(/!#{Regexp.escape(filename)}(!|\()/, "!#{path}\\1") 262 | end 263 | 264 | def replace_html_images(body, markup, filename, path) 265 | return body if markup != 'html' 266 | body.gsub(/(src=(?:'|")|href=(?:'|"))#{Regexp.escape(filename)}/i, "\\1#{path}") 267 | # WARNING: won't address urls in css 268 | # Is case insensitive so it won't differentiatee FOO.jpg from foo.jpg or FoO.jpg 269 | # people shouldn't be using the same name for different files anyway. :P 270 | end 271 | end 272 | -------------------------------------------------------------------------------- /run_jekyllmail.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # CHANGE ME TO MEET THE NEEDS OF YOUR SERVER CONFIGURATION 4 | 5 | [[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function* 6 | GEM_PATH=$GEM_PATH:/home/my_username/.gems 7 | PATH=$PATH:/home/my_username/.gems/bin 8 | cd /home/my_username/jekyllmail_repo/ 9 | bundle exec ruby jekyllmail.rb 10 | --------------------------------------------------------------------------------