├── .env.sample ├── Gemfile ├── Gemfile.lock ├── README.md ├── LICENSE.txt ├── .gitignore ├── script.rb └── kibela.rb /.env.sample: -------------------------------------------------------------------------------- 1 | ESA_ACCESS_TOKEN=tokendummytokendummytoken 2 | ESA_TEAM_NAME=esa 3 | KIBELA_TEAM=kibela 4 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | source "https://rubygems.org" 4 | 5 | git_source(:github) {|repo_name| "https://github.com/#{repo_name}" } 6 | 7 | gem 'pry' 8 | gem 'front_matter_parser' 9 | gem 'esa' 10 | gem 'nokogiri' 11 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | coderay (1.1.2) 5 | esa (1.14.0) 6 | faraday (~> 0.9) 7 | faraday_middleware 8 | mime-types (>= 2.6) 9 | multi_xml (>= 0.5.5) 10 | faraday (0.15.4) 11 | multipart-post (>= 1.2, < 3) 12 | faraday_middleware (0.12.2) 13 | faraday (>= 0.7.4, < 1.0) 14 | front_matter_parser (0.2.0) 15 | method_source (0.9.2) 16 | mime-types (3.2.2) 17 | mime-types-data (~> 3.2015) 18 | mime-types-data (3.2018.0812) 19 | mini_portile2 (2.8.5) 20 | multi_xml (0.6.0) 21 | multipart-post (2.0.0) 22 | nokogiri (1.16.2) 23 | mini_portile2 (~> 2.8.2) 24 | racc (~> 1.4) 25 | pry (0.12.2) 26 | coderay (~> 1.1.0) 27 | method_source (~> 0.9.0) 28 | racc (1.7.3) 29 | 30 | PLATFORMS 31 | ruby 32 | 33 | DEPENDENCIES 34 | esa 35 | front_matter_parser 36 | nokogiri 37 | pry 38 | 39 | BUNDLED WITH 40 | 1.17.2 41 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # kibela2esa 2 | 3 | ## 概要 4 | kibelaからexportされたファイル、記事をesaへインポートするスクリプトになります。 5 | 6 | ## 使いかた 7 | ### 1. API tokenの取得 8 | esaに対してwrite権限があるAPI tokenを発行し、環境変数 `ESA_ACCESS_TOKEN` へ格納してください。 9 | 10 | ![create token with write scope](https://user-images.githubusercontent.com/4487291/52120119-ae9fcb80-265e-11e9-8557-fe63afa6a08a.png) 11 | 12 | ### 2. 環境変数の定義 13 | `.env.sample` を参考に、 `ESA_TEAM_NAME` と `KIBELA_TEAM` を設定してください。 14 | (dotenvを読むようにはなっていないことに注意してください。) 15 | 16 | ### 3. kibela -> esa におけるユーザー名のmappingを行なうHashの定義 17 | 18 | ```ruby 19 | KIBELA_ESA_USER_MAP = { 20 | 'kibela_user_name1' => 'esa_user_name1', 21 | 'kibela_user_name2' => 'esa_user_name2', 22 | # ... 23 | } 24 | ``` 25 | 26 | ### 4. dry runで実行し、移行が問題ないか確認する 27 | ```shell 28 | $ bundle exec ruby script.rb 29 | ``` 30 | 31 | 上記コマンドを実行すると、migraterがnewされた時点でpry consoleに入ります。 32 | 33 | この時点で、 `migrater.migrate` を実行すると、API callするparamsをlogに表示して終了します。このlogを見て、移行が正しく行なわれそうかを確認するようにしてください。 34 | 35 | ### 5. migrateを実行する 36 | `migrater.migrate(dry_run: false)` により、実際にAPI callを行ない移行を開始します。 37 | 38 | 移行対象のオブジェクトが多い場合にはAPIの利用制限がかかることがあります。`sleep` の値を調整したり、サポートに連絡を行うなどで対応してください。 39 | 40 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2019 Kuniaki Hori, Yusuke Nakamura 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | *.rbc 3 | /.config 4 | /coverage/ 5 | /InstalledFiles 6 | /pkg/ 7 | /spec/reports/ 8 | /spec/examples.txt 9 | /test/tmp/ 10 | /test/version_tmp/ 11 | /tmp/ 12 | 13 | # Used by dotenv library to load environment variables. 14 | # .env 15 | 16 | ## Specific to RubyMotion: 17 | .dat* 18 | .repl_history 19 | build/ 20 | *.bridgesupport 21 | build-iPhoneOS/ 22 | build-iPhoneSimulator/ 23 | 24 | ## Specific to RubyMotion (use of CocoaPods): 25 | # 26 | # We recommend against adding the Pods directory to your .gitignore. However 27 | # you should judge for yourself, the pros and cons are mentioned at: 28 | # https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control 29 | # 30 | # vendor/Pods/ 31 | 32 | ## Documentation cache and generated files: 33 | /.yardoc/ 34 | /_yardoc/ 35 | /doc/ 36 | /rdoc/ 37 | 38 | ## Environment normalization: 39 | /.bundle/ 40 | /vendor/bundle 41 | /lib/bundler/man/ 42 | 43 | # for a library or gem, you might want to ignore these files since the code is 44 | # intended to run in multiple environments; otherwise, check them in: 45 | # Gemfile.lock 46 | # .ruby-version 47 | # .ruby-gemset 48 | 49 | # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: 50 | .rvmrc 51 | -------------------------------------------------------------------------------- /script.rb: -------------------------------------------------------------------------------- 1 | require 'fileutils' 2 | require 'pry' 3 | require 'esa' 4 | require 'logger' 5 | require_relative './kibela' 6 | 7 | ESA_TEAM = ENV['ESA_TEAM_NAME'] 8 | 9 | class Migrater 10 | attr_reader :notes, :attachment_list 11 | 12 | def initialize 13 | @logger = Logger.new(STDOUT) 14 | @file_logger = Logger.new("log_#{Time.now.to_i}.log") 15 | @client = Esa::Client.new(access_token: ENV['ESA_ACCESS_TOKEN'], current_team: ESA_TEAM) 16 | end 17 | 18 | def prepare 19 | @notes = Dir.glob("./kibela-#{ENV['KIBELA_TEAM']}-*/**/*.md").map do |path| 20 | File.open(path) do |f| 21 | Kibela::Note.new(f) 22 | end 23 | end 24 | 25 | @attachments = Dir.glob("./kibela-#{ENV['KIBELA_TEAM']}-*/attachments/*").map do |path| 26 | File.open(path) do |f| 27 | Kibela::Attachment.new(f) 28 | end 29 | end 30 | 31 | @attachment_list = {} 32 | @attachments.each do |a| 33 | @attachment_list[a.name] = a 34 | end 35 | 36 | self 37 | end 38 | 39 | def migrate(dry_run: true) 40 | prepare 41 | upload_attachments unless dry_run 42 | 43 | @notes.each do |note| 44 | request = note.esafy(@attachment_list) 45 | @logger.info request 46 | @file_logger.info request 47 | 48 | unless dry_run 49 | response = @client.create_post(request) 50 | @file_logger.info response 51 | note.response = response 52 | sleep 0.5 53 | end 54 | 55 | note.comments.each do |comment| 56 | request = comment.esafy(@attachment_list) 57 | @logger.info request 58 | @file_logger.info request 59 | 60 | unless dry_run 61 | response = @client.create_comment(note.esa_number, request) 62 | @file_logger.info response 63 | sleep 0.5 64 | end 65 | end 66 | end 67 | end 68 | 69 | def upload_attachments 70 | @attachments.each do |attachment| 71 | @logger.info attachment.path 72 | @file_logger.info attachment.path 73 | response = @client.upload_attachment(attachment.path) 74 | @logger.info response 75 | @file_logger.info response 76 | next if response.body['error'] 77 | attachment.esa_path = response.body['attachment']['url'] 78 | sleep 0.5 79 | end 80 | end 81 | end 82 | 83 | migrater = Migrater.new 84 | 85 | binding.pry 86 | -------------------------------------------------------------------------------- /kibela.rb: -------------------------------------------------------------------------------- 1 | require 'front_matter_parser' 2 | require 'nokogiri' 3 | 4 | module Kibela 5 | TEAM = ENV['KIBELA_TEAM'] 6 | TITLE_PATTERN = %r[\A.*/kibela-#{TEAM}-\d+/(?wikis|blogs|notes)/(?[[:print:]]*/)*(?\d)+-(?[[:print:]]*)\.md\z] 7 | ATTACHMENT_PATTERN = %r[^(?~http)(\.\./)*attachments/(?\d+\.(png|JPG|jpg|jpeg|gif|PNG))] 8 | KIBELA_ESA_USER_MAP = { 9 | 'unasuke' => 'unasuke' 10 | } 11 | 12 | class Note 13 | attr_accessor :name, :category, :body, :frontmatter, :author, :comments, :response 14 | 15 | def initialize(file) 16 | raise ArgumentError unless file.is_a?(File) 17 | regexp = TITLE_PATTERN.match(file.path) 18 | markdown = FrontMatterParser::Parser.new(:md).call(file.read) 19 | 20 | @name = regexp[:name] 21 | @category = "from_kibela/#{regexp[:path]}".delete_suffix('/') 22 | @kind = regexp[:kind].to_sym 23 | @id = regexp[:id].to_i 24 | @body = markdown.content 25 | @frontmatter = markdown.front_matter 26 | @author = @frontmatter['author'].delete_prefix('@') 27 | @comments = @frontmatter['comments'].map { |c| Comment.new(c) } 28 | end 29 | 30 | def blog? 31 | @kind == :blogs 32 | end 33 | 34 | def wiki? 35 | @kind == :wikis 36 | end 37 | 38 | def wip? 39 | %r[wip]i.match?(@name) 40 | end 41 | 42 | def replace_attachment_names(attachment_list) 43 | parsed_body = Nokogiri::HTML(@body) 44 | parsed_body.css('img').map do |elem| 45 | next unless elem.attributes['src'] 46 | match = ATTACHMENT_PATTERN.match(elem.attributes['src'].value) 47 | # 文中に出現するkibelaの画像URLをesaの画像URLに置換する 48 | next unless match 49 | next unless attachment_list[match['attachment_name']] 50 | @body.gsub!(elem.attributes['src'], attachment_list[match['attachment_name']].esa_path) if match 51 | end 52 | end 53 | 54 | def esafy(attachment_list) 55 | replace_attachment_names(attachment_list) 56 | @category = 'blog' if blog? 57 | { 58 | post: { 59 | name: @name, 60 | body_md: @body, 61 | tags: [], 62 | category: @category, 63 | user: esafy_user_name(@author), 64 | wip: wip?, 65 | message: 'migrate from kibela', 66 | } 67 | } 68 | end 69 | 70 | def esafy_user_name(kibela_user) 71 | KIBELA_ESA_USER_MAP[kibela_user] || 'esa_bot' 72 | end 73 | 74 | def esa_number 75 | @response.body['number'] 76 | end 77 | end 78 | 79 | class Comment 80 | def initialize(comment) 81 | @raw = comment 82 | @content = @raw['content'] 83 | @user = @raw['user'] 84 | end 85 | 86 | def replace_attachment_names(attachment_list) 87 | parsed_comment = Nokogiri::HTML(@content) 88 | parsed_comment.css('img').map do |elem| 89 | match = ATTACHMENT_PATTERN.match(elem.attributes['src'].value) 90 | @content.gsub!(elem.attributes['src'], attachment_list[match['attachment_name']].esa_path) if match 91 | end 92 | end 93 | 94 | def esafy_user_name(kibela_user) 95 | KIBELA_ESA_USER_MAP[kibela_user] || 'esa_bot' 96 | end 97 | 98 | def esafy(attachment_list) 99 | replace_attachment_names(attachment_list) 100 | { 101 | body_md: @content, 102 | user: esafy_user_name(@user) 103 | } 104 | end 105 | end 106 | 107 | class Attachment 108 | attr_accessor :name, :path, :esa_path 109 | 110 | def initialize(file) 111 | raise ArgumentError unless file.is_a?(File) 112 | 113 | @name = File.basename(file.path) 114 | @path = file.path 115 | @esa_path = 'dummy' 116 | end 117 | end 118 | end 119 | --------------------------------------------------------------------------------