├── COPYING ├── README.ja.md ├── README.md ├── bin └── install-word-vector-web-api ├── conf ├── mime.types ├── sample-word-vector-web-api-master-solo.conf ├── sample-word-vector-web-api-master.conf ├── sample-word-vector-web-api-slave1.conf ├── sample-word-vector-web-api-slave2.conf └── sample-word-vector-web-api-slave3.conf ├── docker ├── sample_s1 │ └── Dockerfile └── sample_s3 │ └── Dockerfile ├── libexec ├── boot-word-vector-web-api-3-slaves-sample.sh ├── boot-word-vector-web-api-sample-1-slave.sh ├── boot-word-vector-web-api-sample-3-slaves.sh ├── boot-word-vector-web-api-sample.sh ├── create-docker-image-sample-word-vector-web-api-s1.sh ├── create-docker-image-sample-word-vector-web-api-s3.sh ├── create-symlink-from-nginxdir-to-wwwdir.sh ├── download-sample-jawiki-model.sh ├── install-mecab-and-mecab-ipadic.sh ├── make-install-nginx-with-msgpack-rpc-moulle.sh ├── make-install-word2vec-msgpack-rpc-server.sh ├── make-install-word2vec.sh ├── make_osx_env.sh ├── quit-word-vector-web-api-3-slaves-sample.sh ├── quit-word-vector-web-api-sample.sh ├── run-docker-container-of-sample-word-vector-web-api-s1.sh ├── setup-word-vector-web-api.sh └── update-mecab-ipadic-neologd.sh ├── patch └── word2vec.rev42.local.patch └── www ├── documentation.css ├── github-markdown.css └── reset.css /COPYING: -------------------------------------------------------------------------------- 1 | Copyright (C) 2015 Toshinori Sato (@overlast) 2 | 3 | https://github.com/overlast/word-vector-web-api 4 | 5 | Licensed under the Apache License, Version 2.0 (the "License"); 6 | you may not use this file except in compliance with the License. 7 | You may obtain a copy of the License at 8 | 9 | http://www.apache.org/licenses/LICENSE-2.0 10 | 11 | Unless required by applicable law or agreed to in writing, software 12 | distributed under the License is distributed on an "AS IS" BASIS, 13 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | See the License for the specific language governing permissions and 15 | limitations under the License. 16 | -------------------------------------------------------------------------------- /README.ja.md: -------------------------------------------------------------------------------- 1 | # word-vector-web-api : Implementation in order to operate a web API of word vector models which are generated by Word2Vec, GloVe or e.t.c. 2 | 3 | ## word-vector-web-api とは 4 | word2vec や GloVe などで構築済みな単語ベクトルのモデルを使った結果を HTTP 経由で JSON/JSONP 形式で取得することができる Web API です。 5 | 6 | word-vector-web-api を利用することで、様々なライブラリや資源を組み合わせて同様の Web API を構築するために必要な作業コストを軽減できます。 7 | 8 | ## 特徴 9 | ### 利点 10 | - Web API なので、どのようなプログラミング言語からでも結果を取得可能 11 | - リクエスト時に callback パラメタを指定すれば、フロントエンドから JSONP でレスポンスを取得・可視化も可能 12 | - HTTP サーバに nginx を使っており、処理結果がキャッシュされる 13 | - 割と高速、割と省メモリ 14 | - モデルの読み込みと検索は C++ な MessagePack RPC サーバでおこなっている 15 | - Dockerfile が用意されている 16 | - アプリケーションごとに ENDPOINT に使うスクリプトを改変して image を作り直せばよい 17 | 18 | ### 欠点 19 | - まだメソッドが充実しきっていない。 20 | - 自分のアプリケーションに利用するためには、結局のところサンプルを眺めて理解したり、 word-vector-web-api の各コードを読む必要がある 21 | 22 | ## word-vector-web-api のコードの入手方法 23 | 24 | 更新は GitHub 経由で行います。 25 | 26 | 初回は以下のコマンドでgit cloneしてください。 27 | 28 | $ git clone --depth 1 https://github.com/overlast/word-vector-web-api.git 29 | 30 | または 31 | 32 | $ git clone --depth 1 git@github.com:overlast/word-vector-web-api.git 33 | 34 | もしも、リポジトリの全変更履歴を入手したい方は「--depth 1」を消してcloneして下さい。 35 | 36 | ## 使用例1. Docker + サンプルモデルを使って結果を確認 37 | 以下では日本語 Wikipedia を使って作ったサンプルモデルを使ってデータを取得してみます。 38 | 39 | 環境構築の際の不具合をできるだけ減らすために Docker を使います。 40 | 41 | Dockerfile からイメージを作り、起動したコンテナ内でプロセスを立ち上げて、結果を取得するまでの手順を示します。 42 | 43 | ### 事前に用意する必要があるもの 44 | #### Linux 45 | 46 | - (登録していない場合) epel リポジトリ 47 | 48 | $ sudo yum -y clean all 49 | 50 | $ sudo yum -y install epel-release 51 | 52 | $ sudo yum -y update 53 | 54 | - docker 55 | 56 | $ sudo yum --enablerepo=epel -y install docker-io 57 | 58 | - Docker に関する知識 59 | - docker コマンドで CentOS の docker image を pull して run できる程度に 60 | 61 | - Google のアカウント 62 | - Google Drive からサンプルモデルをダウンロードするために必要です 63 | 64 | #### OSX 65 | - [boot2docker](http://boot2docker.io/) 66 | - パッケージをインストールして、 以下の3コマンドを実行しておいて下さい 67 | - $ boot2docker init 68 | - $ boot2docker start 69 | - $ eval "$(boot2docker shellinit)" 70 | - shellinit すると docker コマンドが使えるようになります 71 | 72 | - Docker と boot2docker に関する知識 73 | - boot2docker を使って CentOS の docker image を pull して run できる程度に 74 | 75 | - Google のアカウント 76 | - Google Drive からサンプルモデルをダウンロードするために必要です 77 | 78 | ### 手順 79 | #### Step1. サンプルモデルの入手 80 | はじめに以下のコマンドを実行してサンプルモデル(約 800 MByte)を Google Drive からダウンロードします。 81 | 82 | $ ./libexec/download-sample-jawiki-model.sh 83 | 84 | 上記で実行したスクリプトは、golang で実装された drive コマンドのコンパイルと、それを使った Google Drive に対するリクエストを行います。 85 | 86 | たとえば以下の様な出力が表示されます。 87 | 88 | $./libexec/download-sample-jawiki-model.sh 89 | [sample-model-downloader] : Start.. 90 | [sample-model-downloader] : Linux is supported 91 | [sample-model-downloader] : Install go and hg 92 | 読み込んだプラグイン:fastestmirror, priorities, security 93 | インストール処理の設定をしています 94 | Loading mirror speeds from cached hostfile 95 | * base: www.ftp.ne.jp 96 | * epel: ftp.kddilabs.jp 97 | * extras: www.ftp.ne.jp 98 | * rpmforge: ftp.kddilabs.jp 99 | * updates: www.ftp.ne.jp 100 | 169 packages excluded due to repository priority protections 101 | パッケージ golang-1.4.2-2.el6.x86_64 はインストール済みか最新バージョンです 102 | パッケージ mercurial-1.4-3.el6.x86_64 はインストール済みか最新バージョンです 103 | 何もしません 104 | [sample-model-downloader] : go get github.com/prasmussen/gdrive/cli 105 | [sample-model-downloader] : go get github.com/voxelbrain/goptions 106 | [sample-model-downloader] : go get code.google.com/p/goauth2/oauth 107 | [sample-model-downloader] : git clone https://github.com/prasmussen/gdrive.git 108 | [sample-model-downloader] : cd /home/overlast/git/word-vector-web-api/libexec/../tmp/gdrive 109 | Already up-to-date. 110 | [sample-model-downloader] : Compile drive.go to create Google Drive CLI application 111 | command-line-arguments 112 | [sample-model-downloader] : cd /home/overlast/git/word-vector-web-api/libexec/../model/ 113 | 114 | うまくいくと、初めてdrive コマンドでモデルをダウンロードする場合には、 Google Drive からのダウンロードに必要な確認作業のための URL が表示されます。過去に drive コマンドを認証した情報が保持されているなら同意が不要なのでURLは表示されません。 115 | 116 | [sample-model-downloader] : Download sample model data file from Google Drive using drive command 117 | Go to the following link in your browser: 118 | https://accounts.google.com/o/oauth2/auth?client_id=367116221053-7n0vf5akeru7on6o2fjinrecpdoe99eg.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state=state 119 | 120 | Enter verification code: 121 | 122 | この URL にブラウザでアクセスして、同意画面で同意して、確認用の token を得たら、以下のようにコピペします。 123 | 124 | Enter verification code: 4/m62RCGf0d81HV5sooL********************* 125 | 126 | 確認が成功すると、xz 圧縮されたサンプルモデルが model ディレクトリにダウンロードされて、さらに unxz コマンドで解凍されます。 127 | 128 | [sample-model-downloader] : Decompress sample model data file using unxz command 129 | Downloaded 'jawiki.20150602.neologd.bin.xz' at 32.7 MB/s, total 849.3 MB 130 | [sample-model-downloader] : Finish.. 131 | 132 | 解凍されたサンプルモデルは以下の位置にあります。 133 | 134 | $ ls -al model/jawiki.20150602.neologd.bin 135 | -rw-rw-r-- 1 overlast overlast 917846510 7月 2 20:57 2015 model/jawiki.20150602.neologd.bin 136 | 137 | このモデルについて簡単なコメントを書いておきます。 138 | 139 | - [jawiki.20150602.neologd.bin.xz (810MByte)](https://drive.google.com/file/d/0B5QYYyltotqfM2lRQ3l4Mkc5Mk0/view?usp=sharing) 140 | - 日本語 Wikipedia の 2015-06-02 の dump データ(jawiki-latest-pages-articles.xml.bz2)から作成 141 | - jawiki-latest-pages-articles.xml.bz2 から plain text を抽出する際に New Version の [WikiExtractor.py](https://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/WikiExtractor.py) を使用 142 | - http://wiki.apertium.org/wiki/Wikipedia_Extractor 143 | - 分かち書きに MeCab と mecab-ipadic-NEologd を使用 144 | - Word2Vec のパラメタは"-cbow 0 -size 300 -window 5 -negative 0 -hs 1 -sample 1e-5" 145 | 146 | #### Step2. Docker image の作成 147 | 次に docker/sample_s1/Dockerfile を使って、起動すると 1.5 GByte 程度のメモリを使う Docker image を build します。 148 | 149 | Docker の build には以下のコマンドを使います。 150 | 151 | $ ./libexec/create-docker-image-sample-word-vector-web-api-s1.sh 152 | 153 | もしも最後まで処理がうまくいった場合は、以下の様なメッセージが表示されます。 154 | 155 | ... 156 | Step 57 : ENTRYPOINT /root/git/word-vector-web-api/libexec/boot-word-vector-web-api-sample-1-slave.sh 157 | ---> Running in bbec3c1833d9 158 | ---> 76493a06b018 159 | Removing intermediate container bbec3c1833d9 160 | Successfully built 76493a06b018 161 | [create-docker-image-sample-word-vector-web-api-s1] : Finish.. 162 | 163 | 表示されていない場合、何かの原因で build が失敗しています。 164 | 165 | ネットワーク環境に若干左右されるみたいなので、何度か時間を変えて実行してみてください。または以下のようにして一回作成中の image を消してリトライして下さい。 166 | 167 | $ docker rmi -f sample-word-vector-web-api-s1:0.0.1 168 | $ ./libexec/create-docker-image-sample-word-vector-web-api-s1.sh 169 | 170 | できあがった image は以下のコマンドで確認できます。 171 | 172 | $ docker images 173 | $docker images 174 | REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE 175 | sample-word-vector-web-api-s1 0.0.1 a6aaf2fcdc1a 44 seconds ago 698.6 MB 176 | centos centos6 a005304e4e74 13 days ago 203.1 MB 177 | 178 | サーバプロセスが 3 つ立ち上がる Docker image を build したいときは以下のコマンドを使います。 179 | 180 | $ libexec/create-docker-image-sample-word-vector-web-api-s3.sh 181 | 182 | 起動すると 4 GByte 程度のメモリを使うけど、単位時間あたりに同時に処理できるリクエスト数が増えます。 183 | 184 | #### Step3. Docker image を使ったコンテナの起動 185 | Step2 で作った Docker image を使ってコンテナを起動します。 186 | 187 | 起動には以下のコマンドを使います。 188 | 189 | $ ./libexec/run-docker-container-of-sample-word-vector-web-api-s1.sh 190 | 191 | このコマンドでコンテナを起動すると word-vector-web-api に 22670 番ポートからアクセスできるようになります。 192 | 193 | もしも localhost の 22670 番ポートをすでに使っている場合は、-p オプションで空いているポートを指定して下さい。 194 | 195 | $ ./libexec/run-docker-container-of-sample-word-vector-web-api-s1.sh 196 | [run-docker-container-of-sample-word-vector-web-api-s1] : Start.. 197 | [run-docker-container-of-sample-word-vector-web-api-s1] : OSX is supported 198 | [run-docker-container-of-sample-word-vector-web-api-s1] : Run docker container of sample 199 | 543a30bc4737c9c287953a8f8b1b9d5af52b55fa14a1a3737278908896a182cf 200 | Error response from daemon: Cannot start container 543a30bc4737c9c287953a8f8b1b9d5af52b55fa14a1a3737278908896a182cf: Bind for 0.0.0.0:22670 failed: port is already allocated 201 | $ 202 | $ ./libexec/run-docker-container-of-sample-word-vector-web-api-s1.sh -p 23670 203 | 204 | おそらく以下の様な表示が出ます。 205 | 206 | $./libexec/run-docker-container-of-sample-word-vector-web-api-s1.sh --port 23670 207 | [run-docker-container-of-sample-word-vector-web-api-s1] : OSX is supported 208 | [run-docker-container-of-sample-word-vector-web-api-s1] : Run docker container of sample 209 | 0077aef0e71a9858342e3f7f780bd5dbac0184253cbef917e278a17d27b7a5ff 210 | [run-docker-container-of-sample-word-vector-web-api-s1] : Finish.. 211 | 212 | コンテナの起動に成功した場合、以下のコマンドで起動したコンテナの状態を確認できます。 213 | 214 | $ docker ps 215 | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 216 | 0077aef0e71a sample-word-vector-web-api-s1:0.0.1 "/bin/sh -c /root/gi 58 seconds ago Up 42 seconds 22671-22680/tcp, 0.0.0.0:23670->22670/tcp stoic_turing 217 | 218 | Step2 でサーバプロセスが 3 つ立ち上がる Docker image を作った場合、そのイメージを使ったコンテナは以下のコマンドで起動できます。 219 | 220 | $ ./libexec/run-docker-container-of-sample-word-vector-web-api-s3.sh -p 22670 221 | 222 | #### Step4. API から結果を取得する 223 | Step3 でコンテナの起動に成功している場合、curl や ブラウザで結果を確認できます。 224 | 225 | ##### distance 226 | まず distance で 'タモリ'さんと Cosine 距離が近い単語を見てみましょう。ポート番号は自分で調整して下さい。 227 | 228 | - Linux の場合 229 | 230 | $ curl "http://0.0.0.0:22670/distance?a1=タモリ" 231 | 232 | - OSX の場合 233 | 234 | $ curl "http://$(boot2docker ip):22670/distance?a1=タモリ" 235 | 236 | その結果、以下の様な結果が得られました。 237 | 238 | {"format": "json", "items": [ 239 | {"term": "明石家さんま", "score": 0.81528389453887939}, {"term": "さんま", "score": 0.80035871267318726}, 240 | {"term": "ビートたけし", "score": 0.79251360893249512}, {"term": "所ジョージ", "score": 0.76938104629516602}, 241 | {"term": "とんねるず", "score": 0.7473946213722229}, {"term": "爆笑問題", "score": 0.73406845331192017}, 242 | {"term": "司会", "score": 0.72929728031158447}, {"term": "森田一義", "score": 0.72818166017532349}, 243 | {"term": "島田紳助", "score": 0.72627973556518555}, {"term": "中居正広", "score": 0.72301918268203735}, 244 | {"term": "山田邦子", "score": 0.71929085254669189}, {"term": "笑福亭鶴瓶", "score": 0.71266698837280273}, 245 | {"term": "ボキャブラ天国", "score": 0.70373982191085815}, {"term": "笑っていいとも!", "score": 0.70154005289077759}, 246 | {"term": "関根勤", "score": 0.70032232999801636}, {"term": "明石家マンション物語", "score": 0.70010024309158325}, 247 | {"term": "冠番組", "score": 0.69675129652023315}, {"term": "いいとも", "score": 0.69370967149734497}, 248 | {"term": "石橋貴明", "score": 0.68870562314987183}, {"term": "オレたちひょうきん族", "score": 0.68832921981811523}, 249 | {"term": "ものまね", "score": 0.68563121557235718}, {"term": "松村邦洋", "score": 0.68541419506072998}, 250 | {"term": "ボキャ天", "score": 0.68538647890090942}, {"term": "ウッチャンナンチャン", "score": 0.68278044462203979}, 251 | {"term": "テレフォンショッキング", "score": 0.68264764547348022}, {"term": "コーナー司会", "score": 0.68219667673110962}, 252 | {"term": "浅草キッド", "score": 0.68129265308380127}, {"term": "今夜は最高!", "score": 0.67805778980255127}, 253 | {"term": "今田耕司", "score": 0.67686200141906738}, {"term": "萩本欽一", "score": 0.67244911193847656}, 254 | {"term": "みのもんた", "score": 0.67096501588821411}, {"term": "北野ファンクラブ", "score": 0.66866558790206909}, 255 | {"term": "中居", "score": 0.66742992401123047}, {"term": "鶴瓶", "score": 0.66699719429016113}, 256 | {"term": "志村けん", "score": 0.66617244482040405}, {"term": "大橋巨泉", "score": 0.66485863924026489}, 257 | {"term": "上岡龍太郎", "score": 0.66450983285903931}, {"term": "ナインティナイン", "score": 0.66341793537139893}, 258 | {"term": "松本人志", "score": 0.66267943382263184}, {"term": "和田アキ子", "score": 0.66127783060073853} 259 | ], "query": "タモリ", "method": "distance", "sort": "cosine similarity", "status": "OK", "total_count": 40} 260 | 261 | 正解データがあるわけではないので、なるほど、って感じですね。 262 | 263 | 次に analogy で 「東京と東京タワー」との関係に近い「大阪」に対する東京タワー的なものが何かを見てみましょう。 264 | 265 | - Linux の場合 266 | 267 | $ curl "http://0.0.0.0:22670/analogy?a1=東京&a2=東京タワー&a3=大阪" 268 | 269 | - OSX の場合 270 | 271 | $ curl "http://$(boot2docker ip):22670/analogy?a1=東京&a2=東京タワー&a3=大阪" 272 | 273 | その結果、以下の様な結果が得られました。 274 | 275 | {"format": "json", "items": [ 276 | {"term": "電波塔", "score": 0.60608410835266113}, {"term": "東京スカイツリー", "score": 0.58792692422866821}, 277 | {"term": "あべのハルカス", "score": 0.52693915367126465}, {"term": "生駒山", "score": 0.52557224035263062}, 278 | {"term": "対馬オメガ局", "score": 0.52209371328353882}, {"term": "通天閣", "score": 0.51043027639389038}, 279 | {"term": "東山タワー", "score": 0.49114975333213806}, {"term": "名古屋テレビ塔", "score": 0.48477703332901001}, 280 | {"term": "鉄塔", "score": 0.47564679384231567}, {"term": "テレビ塔", "score": 0.45989611744880676}, 281 | {"term": "フットタウン", "score": 0.45694351196289062}, {"term": "瀬戸デジタルタワー", "score": 0.44819486141204834}, 282 | {"term": "神戸ポートタワー", "score": 0.4481755793094635}, {"term": "送信所", "score": 0.44717010855674744}, 283 | {"term": "新梅田シティ", "score": 0.44428074359893799}, {"term": "大阪マルビル", "score": 0.44200178980827332}, 284 | {"term": "大平山送信所", "score": 0.43856528401374817}, {"term": "イーグレひめじ", "score": 0.43271398544311523}, 285 | {"term": "大阪ビジネスパーク", "score": 0.43267670273780823}, {"term": "生駒山テレビ・FM送信所", "score": 0.43197193741798401}, 286 | {"term": "福岡タワー", "score": 0.42966717481613159}, {"term": "京都タワー", "score": 0.42821276187896729}, 287 | {"term": "阪急グランドビル", "score": 0.42394104599952698}, {"term": "天保山ハーバービレッジ", "score": 0.42309394478797913}, 288 | {"term": "御堂筋", "score": 0.4201604425907135}, {"term": "OBP", "score": 0.41969111561775208}, 289 | {"term": "扇町公園", "score": 0.41857218742370605}, {"term": "アンテナ", "score": 0.41681712865829468}, 290 | {"term": "ランドマークタワー", "score": 0.41671887040138245}, {"term": "大阪城公園", "score": 0.41529473662376404}, 291 | {"term": "日本電波塔", "score": 0.41485264897346497}, {"term": "在阪テレビジョン放送局", "score": 0.41319084167480469}, 292 | {"term": "大阪タワー", "score": 0.41302743554115295}, {"term": "エフエムゆめウェーブ", "score": 0.41291263699531555}, 293 | {"term": "阪急三番街", "score": 0.41132032871246338}, {"term": "大阪市内", "score": 0.40921291708946228}, 294 | {"term": "梅田スカイビル", "score": 0.40839734673500061}, {"term": "夢洲", "score": 0.4076114296913147}, 295 | {"term": "コスモタワー", "score": 0.40697681903839111}, {"term": "スカイツリー", "score": 0.4069637656211853} 296 | ], "query": "東京 東京タワー 大阪", "method": "analogy", "sort": "cosine similarity", "status": "OK", "total_count": 40} 297 | 298 | これも正解データがあるわけではないので、なるほど、って感じですね。人間だとどれが正解でどれが不正解か分かりますね。 299 | 300 | 実用するときは、モデルデータを改良して distance の結果を人手でフィルタリングして使うのがオススメです。 301 | 302 | API 自体の詳しい説明は以降の「word-vector-web-api の使い方」の節に書きます。 303 | 304 | #### Step5. Docker コンテナの停止 305 | 試し終わった後の Docker コンテナの停止は docker stop を使います。 306 | 307 | 最初にコンテナのIDを調べます。 308 | 309 | $ docker ps 310 | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 311 | 0077aef0e71a sample-word-vector-web-api-s1:0.0.1 "/bin/sh -c /root/gi 58 seconds ago Up 42 seconds 22671-22680/tcp, 0.0.0.0:23670->22670/tcp stoic_turing 312 | 313 | ID が分かったら docker stop [CONTAINER ID] と指定します。 314 | 315 | $ docker stop 0077aef0e71a 316 | 317 | これでコンテナは停止し破棄されます。 318 | 319 | ## 使用例2. 計算機に直にインストールする 320 | 個人的にはこの使い方がオススメです。 321 | 322 | Dockerfile を参考に環境構築をしてみて下さい。 323 | 324 | 今後、Linux と OSX に対応した環境構築用のスクリプトと手順も公開します。 325 | 326 | ## word-vector-web-api が依存しているライブラリ 327 | インストール時に以下のライブラリが必要に応じて順にインストールされます。 328 | 329 | - [nginx-1.8.0](http://nginx.org/) (独自のディレクトリにインストールされます) 330 | - [nginx-msgpack-rpc-module](https://github.com/overlast/nginx-msgpack-rpc-module) 331 | - [word2vec-msgpack-rpc-server](https://github.com/overlast/word2vec-msgpack-rpc-server) 332 | - [Word2Vec](http://code.google.com/p/word2vec/) (word-vector-web-api ディレクトリ内にインストールされます) 333 | 334 | 動作に必要なものは、上記ライブラリのインストール時に必要としている資源やライブラリです。 335 | 336 | ## word-vector-web-api の使い方 337 | word-vector-web-api を使いたいときは、自分の作成したいアプリケーションごとに conf ファイルをコピーして設定を調整して使うのが良いです。 338 | 339 | 以下に API の仕様や、カスタマイズ時に必要な作業についても書きます。 340 | 341 | ### API ドキュメント 342 | 343 | #### distance(string a) 344 | a という 1 単語のクエリを投げると、ベクトル空間上でベクトル a に最も近いベクトル b を探して答えます。 345 | 346 | 具体的には「タモリ = ?」の ? に何が入りそうかを考える様なものです。 347 | 348 | 答えがどうなって欲しいかも考えてみましょう。 349 | 350 | サンプルモデルは Wikipedia から作りました。ベクトルを構築する手法の性質から、タモリさんと同年代に同じ職種だった人の名前は同じような文脈で出現することが多そうで、距離が近くなりそうです。 351 | 352 | お笑いビッグ 3 みたいな概念がありますし、明石家さんまさんやビートたけしさんは近いかもしれませんね。。(それがうまくいくかはモデル作成に使ったコーパス次第です) 353 | 354 | ##### リクエストURL 355 | 356 | http://[host_name or ip_address]:[port_number]/distance?a1=[query_string] 357 | 358 | ##### リクエストパラメータ 359 | API は callback パラメタが指定されれ場合はレスポンスを JSONP 形式で返し、それ以外の場合は JSON 形式で返す。 360 | 361 | | parameter name | type | description | 362 | | --- | --- | --- | 363 | | a1(required)| string | Word2Vec の distance のクエリ文字列 | 364 | | callback | string | コールバック関数名 | 365 | 366 | ##### サンプルリクエストURL 367 | "http://0.0.0.0:22670/distance?a1=タモリ" 368 | "http://0.0.0.0:22670/distance?a1=タモリ&callback=sample" 369 | 370 | ##### レスポンスフィールド 371 | | field name | description | 372 | | --- | --- | 373 | | format | json のみ | 374 | | method | まさに distance メソッド | 375 | | sort | items の score の値の算出方法。今は cosine 距離のみ | 376 | | status | OK=結果が得られている, NG=何か不具合がある | 377 | | query | a1 パラメタで与えられた文字列 | 378 | | items | 検索結果の内容。term と score の組の配列。 | 379 | | term | score フィールドのスコアを持つ文字列 | 380 | | score | sort フィールドの手法で算出されたスコアの実数値 | 381 | | total_count | items フィールドに含まれる term と score の組数 | 382 | 383 | ##### サンプルレスポンス 384 | 以下は a1 パラメタに'タモリ'を指定した場合のレスポンスです 385 | 386 | {"format": "json", "items": [ 387 | {"term": "明石家さんま", "score": 0.81528389453887939}, {"term": "さんま", "score": 0.80035871267318726}, 388 | {"term": "ビートたけし", "score": 0.79251360893249512}, {"term": "所ジョージ", "score": 0.76938104629516602}, 389 | {"term": "とんねるず", "score": 0.7473946213722229}, {"term": "爆笑問題", "score": 0.73406845331192017}, 390 | (略) 391 | {"term": "上岡龍太郎", "score": 0.66450983285903931}, {"term": "ナインティナイン", "score": 0.66341793537139893}, 392 | {"term": "松本人志", "score": 0.66267943382263184}, {"term": "和田アキ子", "score": 0.66127783060073853} 393 | ], "query": "タモリ", "method": "distance", "sort": "cosine similarity", "status": "OK", "total_count": 40} 394 | 395 | 所ジョージさんですか。なるほど。 396 | 397 | #### analogy(string c, string b, string c) 398 | a、b と c の 3 単語のクエリを投げると、a - b = c - d という状態に一番近い d を見つけるために、ベクトル空間上でベクトル b - a + c に最も近いベクトル d を探して答える。 399 | 400 | 具体的には「東京 - 東京タワー = 大阪 - ?」の ? に何が入りそうかを考える様なものです。 401 | 402 | 答えがどうなって欲しいかも考えてみましょう。 403 | 404 | 東京から東京的な要素とタワー的な要素をひいたものを大阪から作るには、大阪から大阪的要素とタワー的要素をひくために、通天閣とかをひくとどうでしょうか。。(それがうまくいくかはモデル作成に使ったコーパス次第です) 405 | 406 | ##### リクエストURL 407 | 408 | http://[host_name or ip_address]:[port_number]/analogy?a1=[query_string]&a2=[query_string]&a3=[query_string] 409 | 410 | ##### リクエストパラメータ 411 | API は callback パラメタが指定されれ場合はレスポンスを JSONP 形式で返し、それ以外の場合は JSON 形式で返す。 412 | 413 | | parameter name | type | description | 414 | | --- | --- | --- | 415 | | a1(required)| string | Word2Vec の analogy のクエリ文字列a | 416 | | a2(required)| string | Word2Vec の analogy のクエリ文字列b | 417 | | a3(required)| string | Word2Vec の analogy のクエリ文字列c | 418 | | callback | string | コールバック関数名 | 419 | 420 | ##### サンプルリクエストURL 421 | "http://0.0.0.0:22670/analogy?a1=東京&a2=東京タワー&a3=大阪" 422 | "http://0.0.0.0:22670/analogy?a1=東京&a2=東京タワー&a3=大阪&callback=sample" 423 | 424 | ##### レスポンスフィールド 425 | | field name | description | 426 | | --- | --- | 427 | | format | json のみ | 428 | | method | まさに analogy メソッド | 429 | | sort | items の score の値の算出方法。今は cosine 距離のみ | 430 | | status | OK=結果が得られている, NG=何か不具合がある | 431 | | query | a1、a2、a3 パラメタで与えられた文字列を半角スペースで結合した文字列 | 432 | | items | 検索結果の内容。term と score の組の配列。 | 433 | | term | score フィールドのスコアを持つ文字列 | 434 | | score | sort フィールドの手法で算出されたスコアの実数値 | 435 | | total_count | items フィールドに含まれる term と score の組数 | 436 | 437 | ##### サンプルレスポンス 438 | 以下は a1 パラメタに'東京'、a2 パラメタに'東京タワー'、そして a3 パラメタに'大阪'を指定した場合のレスポンスです。 439 | 440 | {"format": "json", "items": [ 441 | {"term": "電波塔", "score": 0.60608410835266113}, {"term": "東京スカイツリー", "score": 0.58792692422866821}, 442 | {"term": "あべのハルカス", "score": 0.52693915367126465}, {"term": "生駒山", "score": 0.52557224035263062}, 443 | {"term": "対馬オメガ局", "score": 0.52209371328353882}, {"term": "通天閣", "score": 0.51043027639389038}, 444 | (略) 445 | {"term": "梅田スカイビル", "score": 0.40839734673500061}, {"term": "夢洲", "score": 0.4076114296913147}, 446 | {"term": "コスモタワー", "score": 0.40697681903839111}, {"term": "スカイツリー", "score": 0.4069637656211853} 447 | ], "query": "東京 東京タワー 大阪", "method": "analogy", "sort": "cosine similarity", "status": "OK", "total_count": 40} 448 | 449 | 電波塔ですか。そうですか。 450 | 451 | ちなみに overlast が普段使っているモデルでは上位 3 つは以下の通りでした。さっぽろテレビ塔・・・。 452 | 453 | {term: "通天閣",score: 0.6666724681854248}, 454 | {term: "さっぽろテレビ塔", score: 0.6369253993034363}, 455 | {term: "ライトアップ",score: 0.6251795291900635}, 456 | 457 | ### API の構成 458 | word2vec-message-pack-server のプロセス数に関わらず、nginx(master) が全てのリクエストをさばいて、結果を 24 時間キャッシュします。 459 | 460 | #### word2vec-message-pack-server プロセス数が 1 の場合 461 | 462 | | process name | port number | request to | return to | cache of response | protocol | 463 | | --- | --- | --- | --- | --- | --- | 464 | | nginx(master) | 22670 | nginx(slave) | user | on | HTTP | 465 | | nginx(slave) | 22671 | word2vec-message-pack-server | nginx(master) | - | HTTP & MessagePack-RPC | 466 | | word2vec-message-pack-server | 22676 | - | nginx(slave) | - | MessagePack-RPC | 467 | 468 | #### word2vec-message-pack-server プロセス数が 3 の場合 469 | 470 | | process name | port number | request to | return to | cache of response | protocol | 471 | | --- | --- | --- | --- | --- | --- | 472 | | nginx(master) | 22670 | nginx(slave 1、2と3 に均等に) | user | on | HTTP | 473 | | nginx(slave 1) | 22671 | word2vec-message-pack-server 1 | nginx(master) | - | HTTP & MessagePack-RPC | 474 | | nginx(slave 2) | 22672 | word2vec-message-pack-server 2 | nginx(master) | - | HTTP & MessagePack-RPC | 475 | | nginx(slave 3) | 22673 | word2vec-message-pack-server 3 | nginx(master) | - | HTTP & MessagePack-RPC | 476 | | word2vec-message-pack-server 1 | 22676 | - | nginx(slave 1) | - | MessagePack-RPC | 477 | | word2vec-message-pack-server 2 | 22677 | - | nginx(slave 2) | - | MessagePack-RPC | 478 | | word2vec-message-pack-server 3 | 22678 | - | nginx(slave 3) | - | MessagePack-RPC | 479 | 480 | ### カスタマイズ時の設定ファイルなどの編集方法 481 | いま書いてます。 482 | 483 | ## word-vector-web-api を通じて主張したいこと 484 | 研究的な実装を C/C++ で実装して [word2vec-msgpack-rpc-server](https://github.com/overlast/word2vec-msgpack-rpc-server) と同様に msgpack-rpc-server 化すれば、LL 言語で下手な実装をするより省メモリで高速な実装を各言語の MessagePack-RPC モジュール経由で利用できる。 485 | 486 | MessagePack-RPC は依存ライブラリが少なく、他のプロトコルに無い柔軟性があるので、最初は雑に作りたい研究的なものづくりに向いている。 487 | 488 | また、選択したプログラミング言語によるメモリの無駄が少ないことは、発明された新しい技術を実応用に近づけたり、結果を精緻化するうえでとても重要である。 489 | 490 | さらに、その実装を Web API 化する場合、[nginx-msgpack-rpc-module](https://github.com/overlast/nginx-msgpack-rpc-module) を利用すれば nginx の設定ファイルを書き換えるだけで msgpack-rpc-client と HTTP Server の用意が終わる。 491 | 492 | 新しい RPC server を書いた際に必要になる RPC client の実装工数がとても少なくなることは個人的にとても重要である。 493 | 494 | そして研究的な実装を一般のエンジニアに試してもらうには、Web API にリクエストして結果の JSON を見てもらうことが、個人的な経験上では一番早いのでオススメである。 495 | 496 | 何か有益なものを実装するときは、Python ではなく C/C++ で実装し、更に Web API 化することで、最終的により広い範囲の研究者やエンジニアがユーザになるだろう。 497 | 498 | word-vector-web-api を実用する際には、単語ベクトルのモデルを作る際の日本語文の単語分割の結果が形態素解析辞書によって変わり、その変化が性能に大きく影響するので、mecab-ipadic と [mecab-ipadic-NEologd](http://github.com/neologd/mecab-ipadic-neologd)を併用するのが望ましい。 499 | 500 | 以上です。 501 | 502 | ## 今後の発展 503 | 継続して開発しますので、気になるところはどんどん改善されます。 504 | 505 | ユーザの8割が気になる部分を優先して改善します。 506 | 507 | まずは gensim の models.word2vec で使える method のうち、よく使いそうなものを順に実装していく予定です。 508 | 509 | ## Bibtex 510 | 511 | もしも word-vector-web-api を論文から参照して下さる場合は、以下の bibtex をご利用ください。 512 | 513 | @misc{sato2015wordvectorwebapi, 514 | title = {Word Vector Web API}, 515 | author = {Toshinori, Sato}, 516 | url = {https://github.com/overlast/word-vector-web-api}, 517 | year = {2015} 518 | } 519 | 520 | ## Copyrights 521 | Copyright (c) 2015 Toshinori Sato (@overlast) All rights reserved. 522 | 523 | ライセンスは Apache License, Version 2.0 です。下記をご参照下さい。 524 | 525 | - https://github.com/overlast/word-vector-web-api/blob/master/COPYING 526 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # word-vector-web-api : Implementation in order to operate a web API of word vector models which are generated by Word2Vec, GloVe or e.t.c. 2 | 3 | ## For Japanese 4 | README.ja.md is written in Japanese. 5 | 6 | - https://github.com/overlast/word-vector-web-api/blob/master/README.ja.md 7 | -------------------------------------------------------------------------------- /bin/install-word-vector-web-api: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | BASEDIR=$(cd $(dirname $0);pwd) 20 | PROGRAM_NAME=install-word-vector-web-api 21 | ECHO_PREFIX="[${PROGRAM_NAME}] :" 22 | 23 | VERSION="install-word-vector-web-api version 0.0.1" 24 | 25 | usage() { 26 | echo "Usage: $PROGRAM_NAME [OPTIONS]" 27 | echo " This script is the installer of mecab-ipadic-neologd" 28 | echo 29 | echo "Options:" 30 | echo " -h, --help" 31 | echo 32 | echo " -v, --version" 33 | echo 34 | echo " -p, --prefix /PATH/TO/INSTALL/DIRECTORY" 35 | echo " Set any directory path where you want to install" 36 | echo 37 | echo " -y, --forceyes" 38 | echo " If you want to install regardless of the result of test" 39 | echo 40 | echo " -u, --asuser" 41 | echo " If you want to install to the user directory as an user" 42 | echo 43 | echo " -n, --newest" 44 | echo " If you want to update mecab-ipadic-neologd" 45 | echo 46 | exit 1 47 | } 48 | 49 | INSTALL_PATH_PREFIX="/usr/local" 50 | IS_FORCE_YES=0 51 | IS_AS_USER=0 52 | WANNA_UPDATE=0 53 | 54 | for OPT in "$@" 55 | do 56 | case "$OPT" in 57 | '-h'|'--help' ) 58 | usage 59 | exit 1 60 | ;; 61 | '-v'|'--version' ) 62 | echo $VERSION 63 | exit 1 64 | ;; 65 | '-p'|'--prefix' ) 66 | if [[ -z "$2" ]] || [[ "$2" =~ ^-+ ]]; then 67 | echo "${PROGRAM_NAME}: option requires an argument -- $1" 1>&2 68 | usage 69 | exit 1 70 | fi 71 | INSTALL_PATH_PREFIX="$2" 72 | shift 2 73 | ;; 74 | '-y'|'--forceyes' ) 75 | IS_FORCE_YES=1 76 | shift 1 77 | ;; 78 | '-u'|'--asuser' ) 79 | IS_AS_USER=1 80 | shift 1 81 | ;; 82 | '-n'|'--newest' ) 83 | WANNA_UPDATE=1 84 | shift 1 85 | ;; 86 | -*) 87 | echo "${PROGRAM_NAME}: illegal option -- '$(echo $1 | sed 's/^-*//')'" 1>&2 88 | usage 89 | exit 1 90 | ;; 91 | *) 92 | if [[ ! -z "$1" ]] && [[ ! "$1" =~ ^-+ ]]; then 93 | #param=( ${param[@]} "$1" ) 94 | param+=( "$1" ) 95 | shift 1 96 | fi 97 | ;; 98 | esac 99 | done 100 | 101 | echo "$ECHO_PREFIX Start.." 102 | 103 | echo "$ECHO_PREFIX Check the exists of libraries" 104 | COMMANDS=(find sort head cut egrep make curl sed cat diff tar unxz xargs grep iconv) 105 | for COMMAND in ${COMMANDS[@]};do 106 | if [ ! `which ${COMMAND}` ]; then 107 | echo "$ECHO_PREFIX ${COMMAND} is not found." 108 | exit 1 109 | else 110 | echo "$ECHO_PREFIX ${COMMAND} => ok" 111 | fi 112 | done 113 | 114 | if [ ${WANNA_UPDATE} = 1 ]; then 115 | echo 116 | echo "$ECHO_PREFIX Get the newest updated information using git" 117 | cd ${BASEDIR}/../ 118 | git fetch origin 119 | git reset --hard origin/master 120 | git pull 121 | fi 122 | 123 | if [ -z ${INSTALL_PATH_PREFIX} ]; then 124 | echo 125 | fi 126 | 127 | echo "" 128 | echo "$ECHO_PREFIX word-vector-web-api will be install to ${INSTALL_PATH_PREFIX}" 129 | echo "" 130 | sleep 3 131 | 132 | echo "$ECHO_PREFIX Make word-vector-web-api" 133 | ${BASEDIR}/../libexec/setup-word-vector-web-api.sh -p ${INSTALL_PATH_PREFIX} -u ${WANNA_CREATE_USER_DIC} 134 | 135 | if [ $? != 0 ]; then 136 | echo "" 137 | echo "$ECHO_PREFIX Failed to make word-vector-web-api" 138 | exit 1 139 | fi 140 | 141 | echo "$ECHO_PREFIX Finish.." 142 | -------------------------------------------------------------------------------- /conf/mime.types: -------------------------------------------------------------------------------- 1 | 2 | types { 3 | text/html html htm shtml; 4 | text/css css; 5 | text/xml xml; 6 | image/gif gif; 7 | image/jpeg jpeg jpg; 8 | application/javascript js; 9 | application/atom+xml atom; 10 | application/rss+xml rss; 11 | 12 | text/mathml mml; 13 | text/plain txt; 14 | text/vnd.sun.j2me.app-descriptor jad; 15 | text/vnd.wap.wml wml; 16 | text/x-component htc; 17 | 18 | image/png png; 19 | image/tiff tif tiff; 20 | image/vnd.wap.wbmp wbmp; 21 | image/x-icon ico; 22 | image/x-jng jng; 23 | image/x-ms-bmp bmp; 24 | image/svg+xml svg svgz; 25 | image/webp webp; 26 | 27 | application/font-woff woff; 28 | application/java-archive jar war ear; 29 | application/json json; 30 | application/mac-binhex40 hqx; 31 | application/msword doc; 32 | application/pdf pdf; 33 | application/postscript ps eps ai; 34 | application/rtf rtf; 35 | application/vnd.apple.mpegurl m3u8; 36 | application/vnd.ms-excel xls; 37 | application/vnd.ms-fontobject eot; 38 | application/vnd.ms-powerpoint ppt; 39 | application/vnd.wap.wmlc wmlc; 40 | application/vnd.google-earth.kml+xml kml; 41 | application/vnd.google-earth.kmz kmz; 42 | application/x-7z-compressed 7z; 43 | application/x-cocoa cco; 44 | application/x-java-archive-diff jardiff; 45 | application/x-java-jnlp-file jnlp; 46 | application/x-makeself run; 47 | application/x-perl pl pm; 48 | application/x-pilot prc pdb; 49 | application/x-rar-compressed rar; 50 | application/x-redhat-package-manager rpm; 51 | application/x-sea sea; 52 | application/x-shockwave-flash swf; 53 | application/x-stuffit sit; 54 | application/x-tcl tcl tk; 55 | application/x-x509-ca-cert der pem crt; 56 | application/x-xpinstall xpi; 57 | application/xhtml+xml xhtml; 58 | application/xspf+xml xspf; 59 | application/zip zip; 60 | 61 | application/octet-stream bin exe dll; 62 | application/octet-stream deb; 63 | application/octet-stream dmg; 64 | application/octet-stream iso img; 65 | application/octet-stream msi msp msm; 66 | 67 | application/vnd.openxmlformats-officedocument.wordprocessingml.document docx; 68 | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx; 69 | application/vnd.openxmlformats-officedocument.presentationml.presentation pptx; 70 | 71 | audio/midi mid midi kar; 72 | audio/mpeg mp3; 73 | audio/ogg ogg; 74 | audio/x-m4a m4a; 75 | audio/x-realaudio ra; 76 | 77 | video/3gpp 3gpp 3gp; 78 | video/mp2t ts; 79 | video/mp4 mp4; 80 | video/mpeg mpeg mpg; 81 | video/quicktime mov; 82 | video/webm webm; 83 | video/x-flv flv; 84 | video/x-m4v m4v; 85 | video/x-mng mng; 86 | video/x-ms-asf asx asf; 87 | video/x-ms-wmv wmv; 88 | video/x-msvideo avi; 89 | } 90 | -------------------------------------------------------------------------------- /conf/sample-word-vector-web-api-master-solo.conf: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | #user nobody; 18 | worker_processes 1; 19 | pid /var/run/nginx-master.pid; 20 | 21 | #error_log logs/error.log; 22 | #error_log logs/error.log notice; 23 | #error_log logs/error.log info; 24 | 25 | #pid logs/nginx.pid; 26 | 27 | 28 | events { 29 | worker_connections 1024; 30 | } 31 | 32 | 33 | http { 34 | charset UTF-8; 35 | include mime.types; 36 | default_type application/octet-stream; 37 | 38 | #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 39 | # '$status $body_bytes_sent "$http_referer" ' 40 | # '"$http_user_agent" "$http_x_forwarded_for"'; 41 | 42 | #access_log logs/access.log main; 43 | 44 | sendfile on; 45 | #tcp_nopush on; 46 | 47 | #keepalive_timeout 0; 48 | keepalive_timeout 65; 49 | 50 | #gzip on; 51 | 52 | # inactive=24hour(1440m or 1d) 53 | proxy_cache_path /var/cache/nginx/cache levels=1:2 keys_zone=word-vector-web-api:13m max_size=15m inactive=1440m; 54 | proxy_temp_path /var/cache/nginx/tmp; 55 | 56 | # header-information 57 | proxy_redirect off; 58 | proxy_set_header Host $host; 59 | proxy_set_header X-Real-IP $remote_addr; 60 | proxy_set_header X-Forwarded-Host $host; 61 | proxy_set_header X-Forwarded-Server $host; 62 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 63 | 64 | upstream word-vector-web-api { 65 | least_conn; 66 | server 127.0.0.1:22671; 67 | } 68 | 69 | server { 70 | listen 22670; 71 | server_name localhost; 72 | 73 | #charset koi8-r; 74 | 75 | #access_log logs/host.access.log main; 76 | 77 | location / { 78 | root www; 79 | index index.html index.htm; 80 | } 81 | 82 | location /distance { 83 | if ( $arg_callback ) { 84 | echo_before_body '$arg_callback('; 85 | echo_after_body ');'; 86 | } 87 | proxy_pass http://word-vector-web-api/distance; 88 | proxy_ignore_headers Cache-Control; 89 | proxy_cache word-vector-web-api; 90 | proxy_cache_valid 200 60m; 91 | proxy_cache_valid any 1m; 92 | } 93 | 94 | location /analogy { 95 | if ( $arg_callback ) { 96 | echo_before_body '$arg_callback('; 97 | echo_after_body ');'; 98 | } 99 | proxy_pass http://word-vector-web-api/analogy; 100 | proxy_ignore_headers Cache-Control; 101 | proxy_cache word-vector-web-api; 102 | proxy_cache_valid 200 60m; 103 | proxy_cache_valid any 1m; 104 | } 105 | 106 | #error_page 404 /404.html; 107 | 108 | # redirect server error pages to the static page /50x.html 109 | # 110 | error_page 500 502 503 504 /50x.html; 111 | location = /50x.html { 112 | root html; 113 | } 114 | 115 | # proxy the PHP scripts to Apache listening on 127.0.0.1:80 116 | # 117 | #location ~ \.php$ { 118 | # proxy_pass http://127.0.0.1; 119 | #} 120 | 121 | # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 122 | # 123 | #location ~ \.php$ { 124 | # root html; 125 | # fastcgi_pass 127.0.0.1:9000; 126 | # fastcgi_index index.php; 127 | # fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name; 128 | # include fastcgi_params; 129 | #} 130 | 131 | # deny access to .htaccess files, if Apache's document root 132 | # concurs with nginx's one 133 | # 134 | #location ~ /\.ht { 135 | # deny all; 136 | #} 137 | } 138 | 139 | 140 | # another virtual host using mix of IP-, name-, and port-based configuration 141 | # 142 | #server { 143 | # listen 8000; 144 | # listen somename:8080; 145 | # server_name somename alias another.alias; 146 | 147 | # location / { 148 | # root html; 149 | # index index.html index.htm; 150 | # } 151 | #} 152 | 153 | 154 | # HTTPS server 155 | # 156 | #server { 157 | # listen 443 ssl; 158 | # server_name localhost; 159 | 160 | # ssl_certificate cert.pem; 161 | # ssl_certificate_key cert.key; 162 | 163 | # ssl_session_cache shared:SSL:1m; 164 | # ssl_session_timeout 5m; 165 | 166 | # ssl_ciphers HIGH:!aNULL:!MD5; 167 | # ssl_prefer_server_ciphers on; 168 | 169 | # location / { 170 | # root html; 171 | # index index.html index.htm; 172 | # } 173 | #} 174 | 175 | } 176 | -------------------------------------------------------------------------------- /conf/sample-word-vector-web-api-master.conf: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | #user nobody; 18 | worker_processes 1; 19 | pid /var/run/nginx-master.pid; 20 | 21 | #error_log logs/error.log; 22 | #error_log logs/error.log notice; 23 | #error_log logs/error.log info; 24 | 25 | #pid logs/nginx.pid; 26 | 27 | 28 | events { 29 | worker_connections 1024; 30 | } 31 | 32 | 33 | http { 34 | charset UTF-8; 35 | include mime.types; 36 | default_type application/octet-stream; 37 | 38 | #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 39 | # '$status $body_bytes_sent "$http_referer" ' 40 | # '"$http_user_agent" "$http_x_forwarded_for"'; 41 | 42 | #access_log logs/access.log main; 43 | 44 | sendfile on; 45 | #tcp_nopush on; 46 | 47 | #keepalive_timeout 0; 48 | keepalive_timeout 65; 49 | 50 | #gzip on; 51 | 52 | # inactive=24hour(1440m or 1d) 53 | proxy_cache_path /var/cache/nginx/cache levels=1:2 keys_zone=word-vector-web-api:13m max_size=15m inactive=1440m; 54 | proxy_temp_path /var/cache/nginx/tmp; 55 | 56 | # header-information 57 | proxy_redirect off; 58 | proxy_set_header Host $host; 59 | proxy_set_header X-Real-IP $remote_addr; 60 | proxy_set_header X-Forwarded-Host $host; 61 | proxy_set_header X-Forwarded-Server $host; 62 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 63 | 64 | upstream word-vector-web-api { 65 | least_conn; 66 | server 127.0.0.1:22671; 67 | server 127.0.0.1:22672; 68 | server 127.0.0.1:22673; 69 | } 70 | 71 | server { 72 | listen 22670; 73 | server_name localhost; 74 | 75 | #charset koi8-r; 76 | 77 | #access_log logs/host.access.log main; 78 | 79 | location / { 80 | root www; 81 | index index.html index.htm; 82 | } 83 | 84 | location /distance { 85 | if ( $arg_callback ) { 86 | echo_before_body '$arg_callback('; 87 | echo_after_body ');'; 88 | } 89 | proxy_pass http://word-vector-web-api/distance; 90 | proxy_ignore_headers Cache-Control; 91 | proxy_cache word-vector-web-api; 92 | proxy_cache_valid 200 60m; 93 | proxy_cache_valid any 1m; 94 | } 95 | 96 | location /analogy { 97 | if ( $arg_callback ) { 98 | echo_before_body '$arg_callback('; 99 | echo_after_body ');'; 100 | } 101 | proxy_pass http://word-vector-web-api/analogy; 102 | proxy_ignore_headers Cache-Control; 103 | proxy_cache word-vector-web-api; 104 | proxy_cache_valid 200 60m; 105 | proxy_cache_valid any 1m; 106 | } 107 | 108 | #error_page 404 /404.html; 109 | 110 | # redirect server error pages to the static page /50x.html 111 | # 112 | error_page 500 502 503 504 /50x.html; 113 | location = /50x.html { 114 | root html; 115 | } 116 | 117 | # proxy the PHP scripts to Apache listening on 127.0.0.1:80 118 | # 119 | #location ~ \.php$ { 120 | # proxy_pass http://127.0.0.1; 121 | #} 122 | 123 | # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 124 | # 125 | #location ~ \.php$ { 126 | # root html; 127 | # fastcgi_pass 127.0.0.1:9000; 128 | # fastcgi_index index.php; 129 | # fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name; 130 | # include fastcgi_params; 131 | #} 132 | 133 | # deny access to .htaccess files, if Apache's document root 134 | # concurs with nginx's one 135 | # 136 | #location ~ /\.ht { 137 | # deny all; 138 | #} 139 | } 140 | 141 | 142 | # another virtual host using mix of IP-, name-, and port-based configuration 143 | # 144 | #server { 145 | # listen 8000; 146 | # listen somename:8080; 147 | # server_name somename alias another.alias; 148 | 149 | # location / { 150 | # root html; 151 | # index index.html index.htm; 152 | # } 153 | #} 154 | 155 | 156 | # HTTPS server 157 | # 158 | #server { 159 | # listen 443 ssl; 160 | # server_name localhost; 161 | 162 | # ssl_certificate cert.pem; 163 | # ssl_certificate_key cert.key; 164 | 165 | # ssl_session_cache shared:SSL:1m; 166 | # ssl_session_timeout 5m; 167 | 168 | # ssl_ciphers HIGH:!aNULL:!MD5; 169 | # ssl_prefer_server_ciphers on; 170 | 171 | # location / { 172 | # root html; 173 | # index index.html index.htm; 174 | # } 175 | #} 176 | 177 | } 178 | -------------------------------------------------------------------------------- /conf/sample-word-vector-web-api-slave1.conf: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | #user nobody; 18 | worker_processes 1; 19 | pid /var/run/nginx-slave1.pid; 20 | 21 | #error_log logs/error.log; 22 | #error_log logs/error.log notice; 23 | #error_log logs/error.log info; 24 | 25 | #pid logs/nginx.pid; 26 | 27 | events { 28 | worker_connections 1024; 29 | } 30 | 31 | http { 32 | charset UTF-8; 33 | include mime.types; 34 | default_type application/octet-stream; 35 | 36 | #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 37 | # '$status $body_bytes_sent "$http_referer" ' 38 | # '"$http_user_agent" "$http_x_forwarded_for"'; 39 | 40 | #access_log logs/access.log main; 41 | 42 | sendfile on; 43 | #tcp_nopush on; 44 | 45 | #keepalive_timeout 0; 46 | keepalive_timeout 65; 47 | 48 | #gzip on; 49 | 50 | server { 51 | listen 22671; 52 | server_name localhost; 53 | 54 | #charset koi8-r; 55 | 56 | #access_log logs/host.access.log main; 57 | 58 | location / { 59 | root html; 60 | index index.html index.htm; 61 | } 62 | 63 | location /distance { 64 | msgpack_rpc_client_call 127.0.0.1 22676 distance; 65 | } 66 | location /analogy { 67 | msgpack_rpc_client_call 127.0.0.1 22676 analogy; 68 | } 69 | 70 | #error_page 404 /404.html; 71 | 72 | # redirect server error pages to the static page /50x.html 73 | # 74 | error_page 500 502 503 504 /50x.html; 75 | location = /50x.html { 76 | root html; 77 | } 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /conf/sample-word-vector-web-api-slave2.conf: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | #user nobody; 18 | worker_processes 1; 19 | pid /var/run/nginx-slave1.pid; 20 | 21 | #error_log logs/error.log; 22 | #error_log logs/error.log notice; 23 | #error_log logs/error.log info; 24 | 25 | #pid logs/nginx.pid; 26 | 27 | events { 28 | worker_connections 1024; 29 | } 30 | 31 | http { 32 | charset UTF-8; 33 | include mime.types; 34 | default_type application/octet-stream; 35 | 36 | #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 37 | # '$status $body_bytes_sent "$http_referer" ' 38 | # '"$http_user_agent" "$http_x_forwarded_for"'; 39 | 40 | #access_log logs/access.log main; 41 | 42 | sendfile on; 43 | #tcp_nopush on; 44 | 45 | #keepalive_timeout 0; 46 | keepalive_timeout 65; 47 | 48 | #gzip on; 49 | 50 | server { 51 | listen 22672; 52 | server_name localhost; 53 | 54 | #charset koi8-r; 55 | 56 | #access_log logs/host.access.log main; 57 | 58 | location / { 59 | root html; 60 | index index.html index.htm; 61 | } 62 | 63 | location /distance { 64 | msgpack_rpc_client_call 127.0.0.1 22677 distance; 65 | } 66 | location /analogy { 67 | msgpack_rpc_client_call 127.0.0.1 22677 analogy; 68 | } 69 | 70 | #error_page 404 /404.html; 71 | 72 | # redirect server error pages to the static page /50x.html 73 | # 74 | error_page 500 502 503 504 /50x.html; 75 | location = /50x.html { 76 | root html; 77 | } 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /conf/sample-word-vector-web-api-slave3.conf: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | #user nobody; 18 | worker_processes 1; 19 | pid /var/run/nginx-slave1.pid; 20 | 21 | #error_log logs/error.log; 22 | #error_log logs/error.log notice; 23 | #error_log logs/error.log info; 24 | 25 | #pid logs/nginx.pid; 26 | 27 | events { 28 | worker_connections 1024; 29 | } 30 | 31 | http { 32 | charset UTF-8; 33 | include mime.types; 34 | default_type application/octet-stream; 35 | 36 | #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 37 | # '$status $body_bytes_sent "$http_referer" ' 38 | # '"$http_user_agent" "$http_x_forwarded_for"'; 39 | 40 | #access_log logs/access.log main; 41 | 42 | sendfile on; 43 | #tcp_nopush on; 44 | 45 | #keepalive_timeout 0; 46 | keepalive_timeout 65; 47 | 48 | #gzip on; 49 | 50 | server { 51 | listen 22673; 52 | server_name localhost; 53 | 54 | #charset koi8-r; 55 | 56 | #access_log logs/host.access.log main; 57 | 58 | location / { 59 | root html; 60 | index index.html index.htm; 61 | } 62 | 63 | location /distance { 64 | msgpack_rpc_client_call 127.0.0.1 22678 distance; 65 | } 66 | location /analogy { 67 | msgpack_rpc_client_call 127.0.0.1 22678 analogy; 68 | } 69 | 70 | #error_page 404 /404.html; 71 | 72 | # redirect server error pages to the static page /50x.html 73 | # 74 | error_page 500 502 503 504 /50x.html; 75 | location = /50x.html { 76 | root html; 77 | } 78 | } 79 | } 80 | -------------------------------------------------------------------------------- /docker/sample_s1/Dockerfile: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | FROM centos:centos6 18 | MAINTAINER Toshinori Sato <@overlast> 19 | 20 | RUN yum -y update 21 | RUN yum -y clean all 22 | RUN yum -y install gcc gcc-c++ automake libtool git tar openssl openssl-devel 23 | 24 | # local ldconfig 25 | RUN echo "/usr/local/lib" >> /etc/ld.so.conf.d/local.conf 26 | RUN echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf 27 | RUN /sbin/ldconfig 28 | 29 | # git files 30 | RUN mkdir /root/git 31 | 32 | # msgpack-c 33 | RUN git clone https://github.com/msgpack/msgpack-c.git /root/git/msgpack-c 34 | RUN cd /root/git/msgpack-c; git checkout refs/tags/cpp-0.5.9; ./bootstrap; ./configure; make; make install; 35 | RUN /sbin/ldconfig 36 | 37 | # mpio 38 | RUN yum -y install ruby 39 | RUN git clone https://github.com/frsyuki/mpio.git /root/git/mpio 40 | RUN cd /root/git/mpio; sed -i -e "s/ -rmpl / -r.\/mpl /g" ./preprocess; ./bootstrap; ./configure; make; make install; 41 | RUN /sbin/ldconfig 42 | 43 | # msgpack-rpc-cpp 44 | RUN git clone https://github.com/msgpack-rpc/msgpack-rpc-cpp.git /root/git/msgpack-rpc-cpp 45 | RUN cd /root/git/msgpack-rpc-cpp; ./bootstrap; ./configure; make; make install 46 | RUN /sbin/ldconfig 47 | 48 | # temporary directory 49 | RUN mkdir /root/tmp 50 | 51 | # waf 52 | RUN curl https://waf.io/waf-1.8.11 -o /root/tmp/waf; cp /root/tmp/waf /usr/local/bin/waf 53 | RUN chmod 755 /usr/local/bin/waf 54 | 55 | # msgpack-rpc-c 56 | RUN git clone https://github.com/overlast/msgpack-rpc-c.git /root/git/msgpack-rpc-c 57 | RUN cd /root/git/msgpack-rpc-c; waf configure; waf build; waf install 58 | RUN /sbin/ldconfig 59 | 60 | 61 | # nginx-with-msgpack-rpc-module 62 | RUN yum -y install pcre-devel 63 | RUN git clone https://github.com/openresty/echo-nginx-module.git /root/git/echo-nginx-module 64 | RUN git clone https://github.com/overlast/nginx-msgpack-rpc-module.git /root/git/nginx-msgpack-rpc-module 65 | RUN curl http://nginx.org/download/nginx-1.8.0.tar.gz -o /root/tmp/nginx-1.8.0.tar.gz 66 | RUN cd /root/tmp; tar xfvz ./nginx-1.8.0.tar.gz 67 | RUN cd /root/tmp/nginx-1.8.0; ./configure --add-module=/root/git/echo-nginx-module --add-module=/root/git/nginx-msgpack-rpc-module --prefix=/usr/local/nginx-with-msgpack-rpc-module; /root/git/nginx-msgpack-rpc-module/bin/fix_makefile.pl /root/tmp/nginx-1.8.0/objs/Makefile; make; make install 68 | RUN /sbin/ldconfig 69 | RUN mkdir -p /var/cache/nginx/cache 70 | 71 | # word2vec-msgpack-rpc-server 72 | RUN curl http://ftp-srv2.kddilabs.jp/Linux/distributions/fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm -o /root/tmp/epel-release-6-8.noarch.rpm 73 | RUN yum -y install epel-release 74 | RUN yum --enablerepo=epel -y install jansson jansson-devel 75 | RUN git clone https://github.com/overlast/word2vec-msgpack-rpc-server /root/git/word2vec-msgpack-rpc-server 76 | RUN cd /root/git/word2vec-msgpack-rpc-server; waf configure; waf build; waf install 77 | 78 | # mecab 79 | #RUN curl -L "https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7cENtOXlicTFaRUE" -o /root/tmp/mecab-0.996.tar.gz 80 | #RUN cd /root/tmp/; tar xfvz mecab-0.996.tar.gz 81 | #RUN cd /root/tmp/mecab-0.996; ./configure; make; make check; make install 82 | #RUN /sbin/ldconfig 83 | 84 | # mecab-ipadic 85 | #RUN curl -L "https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7MWVlSDBCSXZMTXM" -o /root/tmp/mecab-ipadic-2.7.0-20070801.tar.gz 86 | #RUN cd /root/tmp/;tar xfvz mecab-ipadic-2.7.0-20070801.tar.gz 87 | #RUN cd /root/tmp/mecab-ipadic-2.7.0-20070801; ./configure --with-charset=utf8; make; make install 88 | 89 | # commands 90 | RUN yum -y install patch which xz file 91 | 92 | # word-vector-web-api 93 | RUN git clone https://github.com/overlast/word-vector-web-api /root/git/word-vector-web-api 94 | RUN ln -s /root/git/word-vector-web-api/www /usr/local/nginx-with-msgpack-rpc-module/www 95 | 96 | # word2vec on word-vector-web-api 97 | RUN yum -y install svn 98 | RUN svn checkout http://word2vec.googlecode.com/svn/trunk/ /root/git/word-vector-web-api/word2vec 99 | RUN patch /root/git/word-vector-web-api/word2vec/word2vec.c < /root/git/word-vector-web-api/patch/word2vec.rev42.local.patch; 100 | RUN sed -i -e "s/-OFast/-O3/g" /root/git/word-vector-web-api/word2vec/makefile; 101 | RUN cd /root/git/word-vector-web-api/word2vec/; make 102 | 103 | EXPOSE 22670 104 | EXPOSE 22671 105 | EXPOSE 22672 106 | EXPOSE 22673 107 | EXPOSE 22674 108 | EXPOSE 22675 109 | EXPOSE 22676 110 | EXPOSE 22677 111 | EXPOSE 22678 112 | EXPOSE 22679 113 | EXPOSE 22680 114 | 115 | ENTRYPOINT /root/git/word-vector-web-api/libexec/boot-word-vector-web-api-sample-1-slave.sh 116 | -------------------------------------------------------------------------------- /docker/sample_s3/Dockerfile: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2015 Toshinori Sato (@overlast) 2 | # 3 | # https://github.com/overlast/word-vector-web-api 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | 17 | FROM centos:centos6 18 | MAINTAINER Toshinori Sato <@overlast> 19 | 20 | RUN yum -y update 21 | RUN yum -y clean all 22 | RUN yum -y install gcc gcc-c++ automake libtool git tar openssl openssl-devel 23 | 24 | # local ldconfig 25 | RUN echo "/usr/local/lib" >> /etc/ld.so.conf.d/local.conf 26 | RUN echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf 27 | RUN /sbin/ldconfig 28 | 29 | # git files 30 | RUN mkdir /root/git 31 | 32 | # msgpack-c 33 | RUN git clone https://github.com/msgpack/msgpack-c.git /root/git/msgpack-c 34 | RUN cd /root/git/msgpack-c; git checkout refs/tags/cpp-0.5.9; ./bootstrap; ./configure; make; make install; 35 | RUN /sbin/ldconfig 36 | 37 | # mpio 38 | RUN yum -y install ruby 39 | RUN git clone https://github.com/frsyuki/mpio.git /root/git/mpio 40 | RUN cd /root/git/mpio; sed -i -e "s/ -rmpl / -r.\/mpl /g" ./preprocess; ./bootstrap; ./configure; make; make install; 41 | RUN /sbin/ldconfig 42 | 43 | # msgpack-rpc-cpp 44 | RUN git clone https://github.com/msgpack-rpc/msgpack-rpc-cpp.git /root/git/msgpack-rpc-cpp 45 | RUN cd /root/git/msgpack-rpc-cpp; ./bootstrap; ./configure; make; make install 46 | RUN /sbin/ldconfig 47 | 48 | # temporary directory 49 | RUN mkdir /root/tmp 50 | 51 | # waf 52 | RUN curl https://waf.io/waf-1.8.11 -o /root/tmp/waf; cp /root/tmp/waf /usr/local/bin/waf 53 | RUN chmod 755 /usr/local/bin/waf 54 | 55 | # msgpack-rpc-c 56 | RUN git clone https://github.com/overlast/msgpack-rpc-c.git /root/git/msgpack-rpc-c 57 | RUN cd /root/git/msgpack-rpc-c; waf configure; waf build; waf install 58 | RUN /sbin/ldconfig 59 | 60 | # nginx-with-msgpack-rpc-module 61 | RUN yum -y install pcre-devel 62 | RUN git clone https://github.com/openresty/echo-nginx-module.git /root/git/echo-nginx-module 63 | RUN git clone https://github.com/overlast/nginx-msgpack-rpc-module.git /root/git/nginx-msgpack-rpc-module 64 | RUN curl http://nginx.org/download/nginx-1.8.0.tar.gz -o /root/tmp/nginx-1.8.0.tar.gz 65 | RUN cd /root/tmp; tar xfvz ./nginx-1.8.0.tar.gz 66 | RUN cd /root/tmp/nginx-1.8.0; ./configure --add-module=/root/git/echo-nginx-module --add-module=/root/git/nginx-msgpack-rpc-module --prefix=/usr/local/nginx-with-msgpack-rpc-module; /root/git/nginx-msgpack-rpc-module/bin/fix_makefile.pl /root/tmp/nginx-1.8.0/objs/Makefile; make; make install 67 | RUN /sbin/ldconfig 68 | RUN mkdir -p /var/cache/nginx/cache 69 | 70 | # word2vec-msgpack-rpc-server 71 | RUN curl http://ftp-srv2.kddilabs.jp/Linux/distributions/fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm -o /root/tmp/epel-release-6-8.noarch.rpm 72 | RUN yum -y install epel-release 73 | RUN yum --enablerepo=epel -y install jansson jansson-devel 74 | RUN git clone https://github.com/overlast/word2vec-msgpack-rpc-server /root/git/word2vec-msgpack-rpc-server 75 | RUN cd /root/git/word2vec-msgpack-rpc-server; waf configure; waf build; waf install 76 | 77 | # commands 78 | RUN yum -y install patch which xz file 79 | 80 | # word-vector-web-api 81 | RUN git clone https://github.com/overlast/word-vector-web-api /root/git/word-vector-web-api 82 | RUN ln -s /root/git/word-vector-web-api/www /usr/local/nginx-with-msgpack-rpc-module/www 83 | 84 | # word2vec on word-vector-web-api 85 | RUN yum -y install svn 86 | RUN svn checkout http://word2vec.googlecode.com/svn/trunk/ /root/git/word-vector-web-api/word2vec 87 | RUN patch /root/git/word-vector-web-api/word2vec/word2vec.c < /root/git/word-vector-web-api/patch/word2vec.rev42.local.patch; 88 | RUN sed -i -e "s/-OFast/-O3/g" /root/git/word-vector-web-api/word2vec/makefile; 89 | RUN cd /root/git/word-vector-web-api/word2vec/; make 90 | 91 | EXPOSE 22670 92 | EXPOSE 22671 93 | EXPOSE 22672 94 | EXPOSE 22673 95 | EXPOSE 22674 96 | EXPOSE 22675 97 | EXPOSE 22676 98 | EXPOSE 22677 99 | EXPOSE 22678 100 | EXPOSE 22679 101 | EXPOSE 22680 102 | 103 | ENTRYPOINT /root/git/word-vector-web-api/libexec/boot-word-vector-web-api-sample-3-slaves.sh 104 | -------------------------------------------------------------------------------- /libexec/boot-word-vector-web-api-3-slaves-sample.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | BASEDIR=`cd $(dirname $0); pwd` 4 | ECHO_PREFIX="[boot-word-vector-web-api-3-slaves-sample]" 5 | 6 | if [ ! -e /var/cache/nginx/cache ]; then 7 | echo "${ECHO_PREFIX} Create /var/cache/nginx/cache as cache directory.." 8 | sudo mkdir -p /var/cache/nginx/cache 9 | fi 10 | 11 | echo "${ECHO_PREFIX} Boot word2vec-server 1.." 12 | /usr/local/bin/word2vec-msgpack-rpc-server -m $BASEDIR/../model/jawiki.20150602.neologd.bin -p 22676 > /dev/null 2>&1 & 13 | 14 | echo "${ECHO_PREFIX} Boot word2vec-server 2.." 15 | /usr/local/bin/word2vec-msgpack-rpc-server -m $BASEDIR/../model/jawiki.20150602.neologd.bin -p 22677 > /dev/null 2>&1 & 16 | 17 | echo "${ECHO_PREFIX} Boot word2vec-server 3.." 18 | /usr/local/bin/word2vec-msgpack-rpc-server -m $BASEDIR/../model/jawiki.20150602.neologd.bin -p 22678 > /dev/null 2>&1 & 19 | 20 | echo "${ECHO_PREFIX} Boot nginx-slave 1.." 21 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave1.conf 22 | 23 | echo "${ECHO_PREFIX} Boot nginx-slave 2.." 24 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave2.conf 25 | 26 | echo "${ECHO_PREFIX} Boot nginx-slave 3.." 27 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave3.conf 28 | 29 | echo "${ECHO_PREFIX} Boot nginx-master.." 30 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-master.conf 31 | 32 | echo "${ECHO_PREFIX} Finish !!" 33 | -------------------------------------------------------------------------------- /libexec/boot-word-vector-web-api-sample-1-slave.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | /usr/local/bin/word2vec-msgpack-rpc-server -m /var/tmp/sample-word-vector-web-api/jawiki.20150602.neologd.bin -p 22676 > /dev/null 2>&1 & 4 | /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c /root/git/word-vector-web-api/conf/sample-word-vector-web-api-slave1.conf 5 | /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c /root/git/word-vector-web-api/conf/sample-word-vector-web-api-master-solo.conf 6 | 7 | while : 8 | do 9 | sleep 1 10 | done 11 | -------------------------------------------------------------------------------- /libexec/boot-word-vector-web-api-sample-3-slaves.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | /usr/local/bin/word2vec-msgpack-rpc-server -m /var/tmp/sample-word-vector-web-api/jawiki.20150602.neologd.bin -p 22676 > /dev/null 2>&1 & 4 | /usr/local/bin/word2vec-msgpack-rpc-server -m /var/tmp/sample-word-vector-web-api/jawiki.20150602.neologd.bin -p 22677 > /dev/null 2>&1 & 5 | /usr/local/bin/word2vec-msgpack-rpc-server -m /var/tmp/sample-word-vector-web-api/jawiki.20150602.neologd.bin -p 22678 > /dev/null 2>&1 & 6 | 7 | /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c /root/git/word-vector-web-api/conf/sample-word-vector-web-api-slave1.conf 8 | /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c /root/git/word-vector-web-api/conf/sample-word-vector-web-api-slave2.conf 9 | /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c /root/git/word-vector-web-api/conf/sample-word-vector-web-api-slave3.conf 10 | 11 | /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c /root/git/word-vector-web-api/conf/sample-word-vector-web-api-master.conf 12 | 13 | while true 14 | do 15 | sleep 86400 16 | done 17 | -------------------------------------------------------------------------------- /libexec/boot-word-vector-web-api-sample.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | BASEDIR=`cd $(dirname $0); pwd` 4 | ECHO_PREFIX="[boot-word-vector-web-api-sample]" 5 | 6 | if [ ! -e /var/cache/nginx/cache ]; then 7 | echo "${ECHO_PREFIX} Create /var/cache/nginx/cache as cache directory.." 8 | sudo mkdir -p /var/cache/nginx/cache 9 | fi 10 | 11 | echo "${ECHO_PREFIX} Boot word2vec-server.." 12 | /usr/local/bin/word2vec-msgpack-rpc-server -m $BASEDIR/../model/jawiki.20150602.neologd.bin -p 22676 > /dev/null 2>&1 & 13 | 14 | echo "${ECHO_PREFIX} Boot nginx-slave.." 15 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave1.conf 16 | 17 | echo "${ECHO_PREFIX} Boot nginx-master.." 18 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-master-solo.conf 19 | 20 | echo "${ECHO_PREFIX} Finish !!" 21 | -------------------------------------------------------------------------------- /libexec/create-docker-image-sample-word-vector-web-api-s1.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | set -eu 20 | #set -x 21 | 22 | BASEDIR=$(cd $(dirname $0);pwd) 23 | SCRIPT_NAME="[create-docker-image-sample-word-vector-web-api-s1] :" 24 | 25 | echo "${SCRIPT_NAME} Start.." 26 | 27 | if [ "$(uname)" == 'Darwin' ] || [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 28 | 29 | if [ "$(uname)" == 'Darwin' ]; then 30 | echo "${SCRIPT_NAME} OSX is supported" 31 | elif [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 32 | echo "${SCRIPT_NAME} Linux is supported" 33 | else 34 | exit 1 35 | fi 36 | 37 | echo "${SCRIPT_NAME} cd ${BASEDIR}/../docker/sample_s1/" 38 | cd ${BASEDIR}/../docker/sample_s1/ 39 | 40 | echo "${SCRIPT_NAME} docker build -t sample-word-vector-web-api-s1:v0.0.1" 41 | docker build -t sample-word-vector-web-api-s1:0.0.1 . 42 | 43 | else 44 | echo "${SCRIPT_NAME} Your platform ($(uname -a)) isn't supported" 45 | exit 1 46 | fi 47 | 48 | echo "${SCRIPT_NAME} Finish.." 49 | -------------------------------------------------------------------------------- /libexec/create-docker-image-sample-word-vector-web-api-s3.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | set -eu 20 | #set -x 21 | 22 | BASEDIR=$(cd $(dirname $0);pwd) 23 | SCRIPT_NAME="[create-docker-image-sample-word-vector-web-api-s3] :" 24 | 25 | echo "${SCRIPT_NAME} Start.." 26 | 27 | if [ "$(uname)" == 'Darwin' ] || [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 28 | 29 | if [ "$(uname)" == 'Darwin' ]; then 30 | echo "${SCRIPT_NAME} OSX is supported" 31 | elif [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 32 | echo "${SCRIPT_NAME} Linux is supported" 33 | else 34 | exit 1 35 | fi 36 | 37 | echo "${SCRIPT_NAME} cd ${BASEDIR}/../docker/sample_s3/" 38 | cd ${BASEDIR}/../docker/sample_s1/ 39 | 40 | echo "${SCRIPT_NAME} docker build -t sample-word-vector-web-api-s3:v0.0.1" 41 | docker build -t sample-word-vector-web-api-s1:0.0.1 . 42 | 43 | else 44 | echo "${SCRIPT_NAME} Your platform ($(uname -a)) isn't supported" 45 | exit 1 46 | fi 47 | 48 | echo "${SCRIPT_NAME} Finish.." 49 | -------------------------------------------------------------------------------- /libexec/create-symlink-from-nginxdir-to-wwwdir.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | BASEDIR=`cd $(dirname $0); pwd` 20 | ECHO_PREFIX="[link_from_nginx_html_dir_to_www_dir]" 21 | WWW_DIR=$BASEDIR/../www 22 | 23 | NGINX_DIR=/usr/local/nginx-with-msgpack-rpc-module 24 | 25 | echo "${ECHO_PREFIX} Create symlink to www dir" 26 | 27 | if [ -d ${NGINX_DIR} ] ;then 28 | if [ -d ${WWW_DIR} ]; then 29 | sudo ln -s ${WWW_DIR} ${NGINX_DIR} 30 | echo 31 | echo "${ECHO_PREFIX} ${WWW_DIR} is linked from ${NGINX_DIR}/www" 32 | echo 33 | ls -al ${NGINX_DIR}/www 34 | echo 35 | else 36 | echo 37 | echo "${ECHO_PREFIX} ${WWW_DIR} isn't exist.." 38 | echo 39 | fi 40 | else 41 | echo 42 | echo "${ECHO_PREFIX} ${NGINX_DIR} isn't exist.." 43 | echo 44 | fi 45 | 46 | echo "${ECHO_PREFIX} Finish !!" 47 | -------------------------------------------------------------------------------- /libexec/download-sample-jawiki-model.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | set -eu 20 | #set -x 21 | 22 | BASEDIR=$(cd $(dirname $0);pwd) 23 | SCRIPT_NAME="[sample-model-downloader] :" 24 | PREFIX=/usr/local 25 | 26 | echo "${SCRIPT_NAME} Start.." 27 | 28 | if [ "$(uname)" == 'Darwin' ] || [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 29 | if [ "$(uname)" == 'Darwin' ]; then 30 | echo "${SCRIPT_NAME} OSX is supported" 31 | if [ ! -d ${BASEDIR}/../tmp ]; then 32 | mkdir ${BASEDIR}/../tmp 33 | fi 34 | brew install -y go hg 35 | elif [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 36 | echo "${SCRIPT_NAME} Linux is supported" 37 | if [ ! -d ${BASEDIR}/../tmp ]; then 38 | echo "${SCRIPT_NAME} mkdir ${BASEDIR}/../tmp" 39 | mkdir ${BASEDIR}/../tmp 40 | fi 41 | echo "${SCRIPT_NAME} Install go and hg" 42 | sudo yum install -y go hg 43 | else 44 | exit 1 45 | fi 46 | 47 | echo "${SCRIPT_NAME} go get github.com/prasmussen/gdrive/cli" 48 | GOPATH=${BASEDIR}/../tmp/go go get -v github.com/prasmussen/gdrive/cli 49 | echo "${SCRIPT_NAME} go get github.com/voxelbrain/goptions" 50 | GOPATH=${BASEDIR}/../tmp/go go get -v github.com/voxelbrain/goptions 51 | echo "${SCRIPT_NAME} go get code.google.com/p/goauth2/oauth" 52 | GOPATH=${BASEDIR}/../tmp/go go get -v code.google.com/p/goauth2/oauth 53 | echo "${SCRIPT_NAME} git clone https://github.com/prasmussen/gdrive.git" 54 | 55 | if [ ! -d ${BASEDIR}/../tmp ]; then 56 | echo "${SCRIPT_NAME} mkdir ${BASEDIR}/../tmp" 57 | mkdir ${BASEDIR}/../tmp 58 | fi 59 | 60 | if [ ! -d ${BASEDIR}/../tmp/gdrive/ ] || [ ! -e ${BASEDIR}/../tmp/gdrive/.git ]; then 61 | echo "${SCRIPT_NAME} git clone github.com/prasmussen/gdrive" 62 | git clone https://github.com/prasmussen/gdrive.git ${BASEDIR}/../tmp/gdrive 63 | fi 64 | 65 | echo "${SCRIPT_NAME} cd ${BASEDIR}/../tmp/gdrive" 66 | cd ${BASEDIR}/../tmp/gdrive 67 | git pull 68 | 69 | echo "${SCRIPT_NAME} Compile drive.go to create Google Drive CLI application" 70 | GOPATH=${BASEDIR}/../tmp/go go build -v ./drive.go 71 | 72 | if [ ! -d ${BASEDIR}/../model ]; then 73 | echo "${SCRIPT_NAME} mkdir ${BASEDIR}/../model" 74 | mkdir ${BASEDIR}/../model 75 | fi 76 | 77 | echo "${SCRIPT_NAME} cd ${BASEDIR}/../model/" 78 | cd ${BASEDIR}/../model/ 79 | echo "${SCRIPT_NAME} Download sample model data file from Google Drive using drive command" 80 | ${BASEDIR}/../tmp/gdrive/drive download --id 0B5QYYyltotqfM2lRQ3l4Mkc5Mk0 81 | 82 | if [ -e ${BASEDIR}/../model/jawiki.20150602.neologd.bin ]; then 83 | echo "${SCRIPT_NAME} Delete old sample model file" 84 | rm ${BASEDIR}/../model/jawiki.20150602.neologd.bin 85 | fi 86 | 87 | echo "${SCRIPT_NAME} Decompress sample model data file using unxz command" 88 | unxz ./jawiki.20150602.neologd.bin.xz 89 | 90 | else 91 | echo "${SCRIPT_NAME} Your platform ($(uname -a)) isn't supported" 92 | exit 1 93 | fi 94 | 95 | echo "${SCRIPT_NAME} Finish.." 96 | -------------------------------------------------------------------------------- /libexec/install-mecab-and-mecab-ipadic.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | BASEDIR=$(cd $(dirname $0);pwd) 20 | ECHO_PREFIX="[install-mecab-and-mecab-ipadic] :" 21 | 22 | echo "${ECHO_PREFIX} Get sudo password" 23 | sudo pwd 24 | 25 | sudo rpm -ivh http://packages.groonga.org/centos/groonga-release-1.1.0-1.noarch.rpm 26 | sudo yum install -y mecab mecab-devel mecab-ipadic 27 | 28 | echo "${ECHO_PREFIX} Finish.." 29 | -------------------------------------------------------------------------------- /libexec/make-install-nginx-with-msgpack-rpc-moulle.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | #set -x # show executed commands 20 | set -e # die when an error will occur 21 | 22 | BASEDIR=`cd $(dirname $0); pwd` 23 | USER_ID=`/usr/bin/id -u` 24 | ECHO_PREFIX="[make-install-nginx-with-msgpack-rpc-moulle] : " 25 | 26 | TMPDIR=/var/tmp/nginx-with-msgpack-rpc-module 27 | 28 | NGX_VERSION=1.8.0 29 | NGX_DIR_NAME=nginx-${NGX_VERSION} 30 | 31 | NGX_DIR=nginx-with-msgpack-rpc-module 32 | INSTALL_DIR=/usr/local/${NGX_DIR} 33 | 34 | NGX_MODULE_DIR=${TMPDIR}/nginx-msgpack-rpc-module 35 | ECHO_MODULE_DIR=${TMPDIR}/echo-nginx-module 36 | 37 | echo "${ECHO_PREFIX} Get sudo password" 38 | sudo pwd 39 | 40 | echo "${ECHO_PREFIX} nginx will install to ${INSTALL_DIR}." 41 | 42 | while true;do 43 | echo "${ECHO_PREFIX} nginx will install to ${INSTALL_DIR}." 44 | echo "${ECHO_PREFIX} Type 'yes|y' or 'dir path which you want to install'." 45 | read answer 46 | case $answer in 47 | yes) 48 | echo -e "${ECHO_PREFIX} [yes]\n" 49 | echo -e "${ECHO_PREFIX} nginx will install to ${INSTALL_DIR}.\n" 50 | break 51 | ;; 52 | y) 53 | echo -e "${ECHO_PREFIX} [y]\n" 54 | echo -e "${ECHO_PREFIX} nginx will install to ${INSTALL_DIR}.\n" 55 | break 56 | ;; 57 | *) 58 | echo -e "${ECHO_PREFIX} [$answer]\n" 59 | INSTALL_DIR=$answer 60 | echo -e "${ECHO_PREFIX} instll dir is changed.\n" 61 | ;; 62 | esac 63 | done 64 | 65 | if [ ! -e /usr/local/lib/libmsgpack_rpc_client.so.0.0.1 ]; then 66 | echo "${ECHO_PREFIX} msgpack-rpc-c must be installed.." 67 | 68 | if [ "$(uname)" == 'Darwin' ]; then 69 | echo "$SCRIPT_NAME OSX is supported" 70 | $BASEDIR/../libexec/make_osx_env.sh 71 | elif [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 72 | echo "$SCRIPT_NAME Linux is supported" 73 | $BASEDIR/../libexec/make_centos_env.sh 74 | else 75 | echo "$SCRIPT_NAME Your platform ($(uname -a)) isn't supported" 76 | exit 1 77 | fi 78 | fi 79 | 80 | echo "${ECHO_PREFIX} cd to tmp dir" 81 | if [ -e ${TMPDIR} ]; then 82 | rm -rf ${TMPDIR} 83 | fi 84 | mkdir -p ${TMPDIR} 85 | cd ${TMPDIR} 86 | 87 | git clone https://github.com/overlast/nginx-msgpack-rpc-module.git 88 | git clone https://github.com/openresty/echo-nginx-module.git 89 | 90 | wget http://nginx.org/download/${NGX_DIR_NAME}.tar.gz 91 | tar xfvz ./${NGX_DIR_NAME}.tar.gz 92 | cd ${NGX_DIR_NAME} 93 | 94 | ./configure --add-module=${ECHO_MODULE_DIR} --add-module=${NGX_MODULE_DIR} --prefix=${INSTALL_DIR} 95 | 96 | ${NGX_MODULE_DIR}/bin/fix_makefile.pl ./objs/Makefile 97 | make 98 | sudo make install 99 | 100 | echo "${ECHO_PREFIX} Finish.." 101 | 102 | echo "${ECHO_PREFIX} nginx.conf is here => ${INSTALL_DIR}/conf/nginx.conf" 103 | echo "" 104 | echo "${ECHO_PREFIX} nginx with msgpack_rpc_module can start to exec ${INSTALL_DIR}/sbin/nginx" 105 | echo "Usage :" 106 | echo " Start : ${INSTALL_DIR}/sbin/nginx" 107 | echo " Stop : ${INSTALL_DIR}/sbin/nginx -s stop" 108 | echo " Quit after fetch request : ${INSTALL_DIR}/sbin/nginx -s quit" 109 | echo " Reopen the logfiles : ${INSTALL_DIR}/sbin/nginx -s reopen" 110 | echo " Reload nginx.conf : ${INSTALL_DIR}/sbin/nginx -s reloqd" 111 | echo 112 | -------------------------------------------------------------------------------- /libexec/make-install-word2vec-msgpack-rpc-server.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | #set -x # show executed commands 20 | set -e # die when an error will occur 21 | 22 | BASEDIR=`cd $(dirname $0); pwd` 23 | USER_ID=`/usr/bin/id -u` 24 | ECHO_PREFIX="[make-install-word2vec-msgpack-rpc-server] :" 25 | 26 | TMPDIR=/var/tmp/make-install-word2vec-msgpack-rpc-server 27 | 28 | echo "${ECHO_PREFIX} Get sudo password" 29 | sudo pwd 30 | 31 | echo "${ECHO_PREFIX} cd to tmp dir" 32 | if [ -e ${TMPDIR} ]; then 33 | rm -rf ${TMPDIR} 34 | fi 35 | mkdir -p ${TMPDIR} 36 | cd ${TMPDIR} 37 | 38 | if [ -e ${TMPDIR}/word2vec-msgpack-rpc-server ]; then 39 | echo "${ECHO_PREFIX} cd to tmp dir" 40 | rm -rf ${TMPDIR}/word2vec-msgpack-rpc-server 41 | fi 42 | 43 | echo "${ECHO_PREFIX} cd to tmp dir" 44 | git clone https://github.com/overlast/word2vec-msgpack-rpc-server.git 45 | cd word2vec-msgpack-rpc-server 46 | 47 | if [ ! -e /usr/local/lib/libmsgpack-rpc.so.1.0.0 ] || [ ! -e /usr/lib64/libjansson.so ] ; then 48 | echo "${ECHO_PREFIX} msgpack-rpc-c++ and jansson-devel must be installed.." 49 | ./sh/make_centos_env.sh 50 | fi 51 | 52 | echo "${ECHO_PREFIX} cd to tmp dir" 53 | ./sh/compile.sh 54 | 55 | echo "${ECHO_PREFIX} deleting ${TMPDIR}" 56 | cd ${BASEDIR}/../ 57 | rm -rf ${TMPDIR} 58 | 59 | INSTALL_DIR=/usr/local 60 | SCRIPT=word2vec-msgpack-rpc-server 61 | 62 | echo "${ECHO_PREFIX} Finish.." 63 | echo 64 | 65 | echo "${ECHO_PREFIX} ${SCRIPT} is here => ${INSTALL_DIR}/bin/word2vec-msgpack-rpc-server" 66 | ls -al ${INSTALL_DIR}/bin/word2vec-msgpack-rpc-server 67 | echo 68 | echo "${ECHO_PREFIX} nginx with msgpack_rpc_module can start to exec ${INSTALL_DIR}/bin/word2vec-msgpack-rpc-server" 69 | echo "Usage :" 70 | echo " Start : ${INSTALL_DIR}/bin/word2vec-msgpack-rpc-server -m [/path/to/word/vector/model] -p [port_number]" 71 | echo 72 | -------------------------------------------------------------------------------- /libexec/make-install-word2vec.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | BASEDIR=$(cd $(dirname $0);pwd) 20 | ECHO_PREFIX="[make-install-word2vec] :" 21 | 22 | echo "${ECHO_PREFIX} Get sudo password" 23 | sudo pwd 24 | 25 | WORD2VEC_DIR=$BASEDIR/../word2vec 26 | 27 | if [ -e $WORD2VEC_DIR ]; then 28 | echo "${ECHO_PREFIX} word2vec is already installed.." 29 | else 30 | echo "${ECHO_PREFIX} trying to install word2vec.." 31 | cd $BASEDIR/../ 32 | echo "${ECHO_PREFIX} Check out from googlecode.." 33 | svn checkout http://word2vec.googlecode.com/svn/trunk/ word2vec 34 | echo "${ECHO_PREFIX} Fix word2vec.c using patch.." 35 | patch $BASEDIR/../word2vec/word2vec.c < $BASEDIR/../patch/word2vec.rev42.local.patch 36 | echo "${ECHO_PREFIX} Fix C++ optimal option.." 37 | sed -i -e "s/-OFast/-O3/g" $BASEDIR/../word2vec/makefile 38 | cd $BASEDIR/../word2vec/ 39 | echo "${ECHO_PREFIX} Make.." 40 | make 41 | fi 42 | 43 | echo "${ECHO_PREFIX} Finish.." 44 | -------------------------------------------------------------------------------- /libexec/make_osx_env.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | BASEDIR=$(cd $(dirname $0);pwd) 4 | SCRIPT_NAME="[make_osx_env] :" 5 | TMP_DIR=/var/tmp/msgpack_rpc_c 6 | 7 | echo "$SCRIPT_NAME Get sudo password" 8 | sudo pwd 9 | 10 | mkdir $TMP_DIR 11 | cd $TMP_DIR 12 | 13 | if [ -e /usr/local/lib/libmsgpack.so.3.0.0 ] || [ -e /usr/local/lib/libmsgpack.so.4.0.0 ]; then 14 | echo "$SCRIPT_NAME msgpack-c is already installed.." 15 | else 16 | echo "$SCRIPT_NAME trying to install msgpack-c.." 17 | git clone https://github.com/msgpack/msgpack-c.git 18 | cd msgpack-c 19 | git checkout refs/tags/cpp-0.5.9 20 | ./bootstrap 21 | ./configure 22 | make 23 | sudo make install 24 | 25 | echo "$SCRIPT_NAME making clean msgpack-c directory.." 26 | cd $TMP_DIR 27 | rm -rf msgpack-c 28 | 29 | echo "$SCRIPT_NAME Refreshing the cache of shared library.." 30 | sudo ldconfig 31 | fi 32 | 33 | if [ -e /usr/local/lib/libmpio.so.0.0.0 ]; then 34 | echo "$SCRIPT_NAME mpio is already installed.." 35 | else 36 | echo "$SCRIPT_NAME trying to install mpio.." 37 | git clone https://github.com/frsyuki/mpio.git 38 | cd mpio 39 | sed -i -e "s/ -rmpl / -r.\/mpl /g" ./preprocess 40 | ./bootstrap 41 | ./configure 42 | make 43 | sudo make install 44 | 45 | echo "$SCRIPT_NAME making clean mpio directory.." 46 | cd $TMP_DIR 47 | rm -rf mpio 48 | 49 | echo "$SCRIPT_NAME Refreshing the cache of shared library.." 50 | sudo ldconfig 51 | fi 52 | 53 | if [ -e /usr/local/lib/libmsgpack-rpc.so.1.0.0 ]; then 54 | echo "$SCRIPT_NAME msgpack-rpc-cpp is already installed.." 55 | else 56 | echo "$SCRIPT_NAME trying to install msgpack-rpc-cpp.." 57 | git clone https://github.com/msgpack-rpc/msgpack-rpc-cpp.git 58 | cd msgpack-rpc-cpp 59 | ./bootstrap # if needed 60 | ./configure 61 | make 62 | sudo make install 63 | 64 | echo "$SCRIPT_NAME making clean msgpack-rpc-cpp directory.." 65 | cd $TMP_DIR 66 | rm -rf msgpack-rpc-cpp 67 | 68 | echo "$SCRIPT_NAME Refreshing the cache of shared library.." 69 | sudo ldconfig 70 | fi 71 | 72 | echo "$SCRIPT_NAME Making clean a build directory.." 73 | cd $BASEDIR 74 | rm -rf $TMP_DIR 75 | 76 | 77 | if [ -e /usr/lib64/libjansson.so ]; then 78 | echo "$SCRIPT_NAME msgpack-rpc-cpp is already installed.." 79 | else 80 | echo "$SCRIPT_NAME Trying to install jansson.." 81 | sudo yum install -y jansson jansson-devel 82 | fi 83 | 84 | echo "$SCRIPT_NAME Trying to install waf.." 85 | sudo yum install -y waf 86 | 87 | 88 | 89 | echo "$SCRIPT_NAME Finish.." 90 | -------------------------------------------------------------------------------- /libexec/quit-word-vector-web-api-3-slaves-sample.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | BASEDIR=`cd $(dirname $0); pwd` 4 | ECHO_PREFIX="[quit-word-vector-web-api-3-slaves-sample]" 5 | 6 | echo "${ECHO_PREFIX} Stop word2vec-server 1.." 7 | pkill -f "model/jawiki.20150602.neologd.bin -p 22676" 8 | 9 | echo "${ECHO_PREFIX} Stop word2vec-server 2.." 10 | pkill -f "model/jawiki.20150602.neologd.bin -p 22677" 11 | 12 | echo "${ECHO_PREFIX} Stop word2vec-server 3.." 13 | pkill -f "model/jawiki.20150602.neologd.bin -p 22678" 14 | 15 | echo "${ECHO_PREFIX} Quit nginx-master.." 16 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-master.conf -s quit 17 | 18 | echo "${ECHO_PREFIX} Quit nginx-slave 1.." 19 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave1.conf -s quit 20 | 21 | echo "${ECHO_PREFIX} Quit nginx-slave 2.." 22 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave2.conf -s quit 23 | 24 | echo "${ECHO_PREFIX} Quit nginx-slave 3.." 25 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave3.conf -s quit 26 | 27 | echo "${ECHO_PREFIX} Finish !!" 28 | -------------------------------------------------------------------------------- /libexec/quit-word-vector-web-api-sample.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | BASEDIR=`cd $(dirname $0); pwd` 4 | ECHO_PREFIX="[quit-word-vector-web-api-sample]" 5 | 6 | echo "${ECHO_PREFIX} Stop word2vec-server.." 7 | pkill -f "model/jawiki.20150602.neologd.bin -p 22676" 8 | 9 | echo "${ECHO_PREFIX} Quit nginx-master.." 10 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-master-solo.conf -s quit 11 | 12 | echo "${ECHO_PREFIX} Quit nginx-slave.." 13 | sudo /usr/local/nginx-with-msgpack-rpc-module/sbin/nginx -c $BASEDIR/../conf/sample-word-vector-web-api-slave1.conf -s quit 14 | 15 | echo "${ECHO_PREFIX} Finish !!" 16 | -------------------------------------------------------------------------------- /libexec/run-docker-container-of-sample-word-vector-web-api-s1.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | set -eu 20 | #set -x 21 | 22 | BASEDIR=$(cd $(dirname $0);pwd) 23 | SCRIPT_NAME="[run-docker-container-of-sample-word-vector-web-api-s1] :" 24 | HOST_PORT=22670 25 | 26 | 27 | for OPT in "$@" 28 | do 29 | case "$OPT" in 30 | '-p'|'--port' ) 31 | if [[ -z "$2" ]] || [[ "$2" =~ ^-+ ]]; then 32 | echo "${SCRIPT_NAME}: option requires an argument -- $1" 1>&2 33 | usage 34 | exit 1 35 | fi 36 | HOST_PORT="$2" 37 | shift 2 38 | break 39 | ;; 40 | -*) 41 | echo "${SCRIPT_NAME}: illegal option -- '$(echo $1 | sed 's/^-*//')'" 1>&2 42 | exit 1 43 | ;; 44 | esac 45 | done 46 | 47 | if [ "$(uname)" == 'Darwin' ] || [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 48 | 49 | if [ "$(uname)" == 'Darwin' ]; then 50 | echo "${SCRIPT_NAME} OSX is supported" 51 | elif [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 52 | echo "${SCRIPT_NAME} Linux is supported" 53 | else 54 | exit 1 55 | fi 56 | 57 | echo "${SCRIPT_NAME} Run docker container of sample " 58 | docker run -d -p 0.0.0.0:${HOST_PORT}:22670 -v ${BASEDIR}/../model:/var/tmp/sample-word-vector-web-api sample-word-vector-web-api-s1:0.0.1 /bin/bash 59 | 60 | else 61 | echo "${SCRIPT_NAME} Your platform ($(uname -a)) isn't supported" 62 | exit 1 63 | fi 64 | 65 | echo "${SCRIPT_NAME} Finish.." 66 | -------------------------------------------------------------------------------- /libexec/setup-word-vector-web-api.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | BASEDIR=$(cd $(dirname $0);pwd) 20 | SCRIPT_NAME="[setup-wordvector-api-server] :" 21 | PREFIX=/usr/local 22 | 23 | echo "$SCRIPT_NAME Start.." 24 | 25 | if [ "$(uname)" == 'Darwin' ]; then 26 | echo "$SCRIPT_NAME OSX is supported" 27 | elif [ "$(expr substr $(uname -s) 1 5)" == 'Linux' ]; then 28 | echo "$SCRIPT_NAME Linux is supported" 29 | else 30 | echo "$SCRIPT_NAME Your platform ($(uname -a)) isn't supported" 31 | exit 1 32 | fi 33 | 34 | echo "$SCRIPT_NAME Get sudo password" 35 | sudo pwd 36 | 37 | NGXWMPRPCM_DIR=${PREFIX}/nginx-with-msgpack-rpc-module 38 | if [ -d ${NGXWMPRPCM_DIR} ]; then 39 | echo "$SCRIPT_NAME nginx-with-msgpack-rpc-moulle is already installed" 40 | else 41 | echo "$SCRIPT_NAME Install nginx-with-msgpack-rpc-moulle" 42 | $BASEDIR/../libexec/make-install-nginx-with-msgpack-rpc-moulle.sh 43 | fi 44 | 45 | WWW_DIR_PATH=${NGXWMPRPCM_DIR}/www 46 | if [ -e ${WWW_DIR_PATH} ]; then 47 | echo "$SCRIPT_NAME symlink to www dir from nginx-with-msgpack-rpc-moulle dir is already created" 48 | else 49 | echo "$SCRIPT_NAME Create symlink to www dir from nginx-with-msgpack-rpc-moulle dir" 50 | $BASEDIR/../libexec/create_symlink_from_nginxdir_to_wwwdir.sh 51 | fi 52 | 53 | W2VMPRPCS_PATH=`which word2vec-msgpack-rpc-server` 54 | if [ -e ${W2VMPRPCS_PATH} ]; then 55 | echo "$SCRIPT_NAME word2vec-msgpack-rpc-server is already installed" 56 | else 57 | echo "$SCRIPT_NAME Install word2vec-msgpack-rpc-server" 58 | $BASEDIR/../libexec/make-install-word2vec-msgpack-rpc-server.sh 59 | fi 60 | 61 | MECAB_PATH=`which mecab` 62 | MECAB_DIC_DIR=`${MECAB_PATH}-config --dicdir` 63 | MECAB_IPADIC_DIR=${MECAB_DIC_DIR}/ipadic 64 | if [ -e ${MECAB_PATH}-config ] && [ -d ${MECAB_IPADIC_DIR} ]; then 65 | echo "$SCRIPT_NAME MeCab and mecab-ipadic is already installed" 66 | else 67 | echo "$SCRIPT_NAME Install MeCab and mecab-ipadic" 68 | $BASEDIR/../libexec/install-mecab-and-mecab-ipadic.sh 69 | fi 70 | 71 | MECAB_PATH=`which mecab` 72 | MECAB_DIC_DIR=`${MECAB_PATH}-config --dicdir` 73 | MECAB_IPADIC_NEOLOGD_DIR=${MECAB_DIC_DIR}/mecab-ipadic-neologd 74 | if [ -d ${MECAB_IPADIC_NEOLOGD_DIR} ]; then 75 | echo "$SCRIPT_NAME mecab-ipadic-NEologd is already installed" 76 | else 77 | echo "$SCRIPT_NAME Install mecab-ipadic-NEologd" 78 | $BASEDIR/../libexec/update-mecab-ipadic-neologd.sh 79 | fi 80 | 81 | WORD2VEC_DIR=$BASEDIR/../word2vec 82 | if [ -d ${WORD2VEC_DIR} ]; then 83 | echo "$SCRIPT_NAME Word2Vec is already installed" 84 | else 85 | echo "$SCRIPT_NAME Install Word2Vec" 86 | $BASEDIR/../libexec/make-install-word2vec.sh 87 | fi 88 | 89 | echo "$SCRIPT_NAME Finish.." 90 | -------------------------------------------------------------------------------- /libexec/update-mecab-ipadic-neologd.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Copyright (C) 2015 Toshinori Sato (@overlast) 4 | # 5 | # https://github.com/overlast/word-vector-web-api 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | BASEDIR=$(cd $(dirname $0);pwd) 20 | ECHO_PREFIX="[update-mecab-ipadic-neologd] :" 21 | TMP_DIR=/var/tmp/mecab-ipadic-overlast 22 | 23 | echo "${ECHO_PREFIX} Get sudo password" 24 | sudo pwd 25 | 26 | if [ -e $TMP_DIR ]; then 27 | rm -rf $TMP_DIR 28 | fi; 29 | mkdir $TMP_DIR 30 | cd $TMP_DIR 31 | 32 | git clone https://github.com/neologd/mecab-ipadic-neologd.git 33 | cd mecab-ipadic-neologd 34 | bin/install-mecab-ipadic-neologd -n -y 35 | 36 | cd $BASEDIR 37 | rm -rf $TMP_DIR 38 | 39 | echo "${ECHO_PREFIX} Finish.." 40 | -------------------------------------------------------------------------------- /patch/word2vec.rev42.local.patch: -------------------------------------------------------------------------------- 1 | --- ./word2vec.c.orig 2014-11-27 00:18:43.605973427 +0900 2 | +++ ./word2vec.c 2014-11-27 00:16:58.784458652 +0900 3 | @@ -43,6 +43,7 @@ long long vocab_max_size = 1000, vocab_s 4 | long long train_words = 0, word_count_actual = 0, iter = 5, file_size = 0, classes = 0; 5 | real alpha = 0.025, starting_alpha, sample = 1e-3; 6 | real *syn0, *syn1, *syn1neg, *expTable; 7 | +size_t local_cache_size = 512 * 1024, local_cache_update = 100; 8 | clock_t start; 9 | 10 | int hs = 0, negative = 5; 11 | @@ -248,10 +249,10 @@ void CreateBinaryTree() { 12 | if (b == vocab_size * 2 - 2) break; 13 | } 14 | vocab[a].codelen = i; 15 | - vocab[a].point[0] = vocab_size - 2; 16 | + vocab[a].point[0] = 0; 17 | for (b = 0; b < i; b++) { 18 | vocab[a].code[i - b - 1] = code[b]; 19 | - vocab[a].point[i - b] = point[b] - vocab_size; 20 | + vocab[a].point[i - b] = vocab_size * 2 - 2 - point[b]; 21 | } 22 | } 23 | free(count); 24 | @@ -368,8 +369,34 @@ void *TrainModelThread(void *id) { 25 | clock_t now; 26 | real *neu1 = (real *)calloc(layer1_size, sizeof(real)); 27 | real *neu1e = (real *)calloc(layer1_size, sizeof(real)); 28 | + size_t num_updates = 0; 29 | + size_t local_syn1_size = (size_t)(local_cache_size / (real)(layer1_size * sizeof(real))); 30 | + real *syn1_local = NULL, *syn1_diff = NULL, *syn1neg_local = NULL, *syn1neg_diff = NULL; 31 | FILE *fi = fopen(train_file, "rb"); 32 | fseek(fi, file_size / (long long)num_threads * (long long)id, SEEK_SET); 33 | + 34 | + if (0 < local_syn1_size) { 35 | + if (hs) { 36 | + a = posix_memalign((void **)&syn1_local, 128, (long long)local_syn1_size * layer1_size * sizeof(real)); 37 | + if (syn1_local == NULL) {printf("Memory allocation failed\n"); exit(1);} 38 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 39 | + syn1_local[a] = syn1[a]; 40 | + a = posix_memalign((void **)&syn1_diff, 128, (long long)local_syn1_size * layer1_size * sizeof(real)); 41 | + if (syn1_diff == NULL) {printf("Memory allocation failed\n"); exit(1);} 42 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 43 | + syn1_diff[a] = 0; 44 | + } 45 | + if (negative>0) { 46 | + a = posix_memalign((void **)&syn1neg_local, 128, (long long)local_syn1_size * layer1_size * sizeof(real)); 47 | + if (syn1neg_local == NULL) {printf("Memory allocation failed\n"); exit(1);} 48 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 49 | + syn1neg_local[a] = syn1neg[a]; 50 | + a = posix_memalign((void **)&syn1neg_diff, 128, (long long)local_syn1_size * layer1_size * sizeof(real)); 51 | + if (syn1neg_diff == NULL) {printf("Memory allocation failed\n"); exit(1);} 52 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 53 | + syn1neg_diff[a] = 0; 54 | + } 55 | + } 56 | while (1) { 57 | if (word_count - last_word_count > 10000) { 58 | word_count_actual += word_count - last_word_count; 59 | @@ -434,22 +461,24 @@ void *TrainModelThread(void *id) { 60 | if (cw) { 61 | for (c = 0; c < layer1_size; c++) neu1[c] /= cw; 62 | if (hs) for (d = 0; d < vocab[word].codelen; d++) { 63 | + real *_syn1 = (vocab[word].point[d] < local_syn1_size) ? syn1_local : syn1; 64 | f = 0; 65 | l2 = vocab[word].point[d] * layer1_size; 66 | // Propagate hidden -> output 67 | - for (c = 0; c < layer1_size; c++) f += neu1[c] * syn1[c + l2]; 68 | + for (c = 0; c < layer1_size; c++) f += neu1[c] * _syn1[c + l2]; 69 | if (f <= -MAX_EXP) continue; 70 | else if (f >= MAX_EXP) continue; 71 | else f = expTable[(int)((f + MAX_EXP) * (EXP_TABLE_SIZE / MAX_EXP / 2))]; 72 | // 'g' is the gradient multiplied by the learning rate 73 | g = (1 - vocab[word].code[d] - f) * alpha; 74 | // Propagate errors output -> hidden 75 | - for (c = 0; c < layer1_size; c++) neu1e[c] += g * syn1[c + l2]; 76 | + for (c = 0; c < layer1_size; c++) neu1e[c] += g * _syn1[c + l2]; 77 | // Learn weights hidden -> output 78 | - for (c = 0; c < layer1_size; c++) syn1[c + l2] += g * neu1[c]; 79 | + for (c = 0; c < layer1_size; c++) _syn1[c + l2] += g * neu1[c]; 80 | } 81 | // NEGATIVE SAMPLING 82 | if (negative > 0) for (d = 0; d < negative + 1; d++) { 83 | + real *_syn1neg = NULL; 84 | if (d == 0) { 85 | target = word; 86 | label = 1; 87 | @@ -460,14 +489,17 @@ void *TrainModelThread(void *id) { 88 | if (target == word) continue; 89 | label = 0; 90 | } 91 | + _syn1neg = (target < (long long)local_syn1_size) ? syn1neg_local : syn1neg; 92 | l2 = target * layer1_size; 93 | f = 0; 94 | - for (c = 0; c < layer1_size; c++) f += neu1[c] * syn1neg[c + l2]; 95 | + for (c = 0; c < layer1_size; c++) f += neu1[c] * _syn1neg[c + l2]; 96 | if (f > MAX_EXP) g = (label - 1) * alpha; 97 | else if (f < -MAX_EXP) g = (label - 0) * alpha; 98 | else g = (label - expTable[(int)((f + MAX_EXP) * (EXP_TABLE_SIZE / MAX_EXP / 2))]) * alpha; 99 | - for (c = 0; c < layer1_size; c++) neu1e[c] += g * syn1neg[c + l2]; 100 | - for (c = 0; c < layer1_size; c++) syn1neg[c + l2] += g * neu1[c]; 101 | + for (c = 0; c < layer1_size; c++) neu1e[c] += g * _syn1neg[c + l2]; 102 | + for (c = 0; c < layer1_size; c++) _syn1neg[c + l2] += g * neu1[c]; 103 | + if (target < (long long)local_syn1_size) 104 | + for (c = 0; c < layer1_size; c++) syn1neg_diff[c + l2] += g * neu1[c]; 105 | } 106 | // hidden -> in 107 | for (a = b; a < window * 2 + 1 - b; a++) if (a != window) { 108 | @@ -490,22 +522,26 @@ void *TrainModelThread(void *id) { 109 | for (c = 0; c < layer1_size; c++) neu1e[c] = 0; 110 | // HIERARCHICAL SOFTMAX 111 | if (hs) for (d = 0; d < vocab[word].codelen; d++) { 112 | + real *_syn1 = (vocab[word].point[d] < local_syn1_size) ? syn1_local : syn1; 113 | f = 0; 114 | l2 = vocab[word].point[d] * layer1_size; 115 | // Propagate hidden -> output 116 | - for (c = 0; c < layer1_size; c++) f += syn0[c + l1] * syn1[c + l2]; 117 | + for (c = 0; c < layer1_size; c++) f += syn0[c + l1] * _syn1[c + l2]; 118 | if (f <= -MAX_EXP) continue; 119 | else if (f >= MAX_EXP) continue; 120 | else f = expTable[(int)((f + MAX_EXP) * (EXP_TABLE_SIZE / MAX_EXP / 2))]; 121 | // 'g' is the gradient multiplied by the learning rate 122 | g = (1 - vocab[word].code[d] - f) * alpha; 123 | // Propagate errors output -> hidden 124 | - for (c = 0; c < layer1_size; c++) neu1e[c] += g * syn1[c + l2]; 125 | + for (c = 0; c < layer1_size; c++) neu1e[c] += g * _syn1[c + l2]; 126 | // Learn weights hidden -> output 127 | - for (c = 0; c < layer1_size; c++) syn1[c + l2] += g * syn0[c + l1]; 128 | + for (c = 0; c < layer1_size; c++) _syn1[c + l2] += g * syn0[c + l1]; 129 | + if (vocab[word].point[d] < local_syn1_size) 130 | + for (c = 0; c < layer1_size; c++) syn1_diff[c + l2] += g * syn0[c + l1]; 131 | } 132 | // NEGATIVE SAMPLING 133 | if (negative > 0) for (d = 0; d < negative + 1; d++) { 134 | + real *_syn1neg = NULL; 135 | if (d == 0) { 136 | target = word; 137 | label = 1; 138 | @@ -516,14 +552,17 @@ void *TrainModelThread(void *id) { 139 | if (target == word) continue; 140 | label = 0; 141 | } 142 | + _syn1neg = (target < local_syn1_size) ? syn1neg_local : syn1neg; 143 | l2 = target * layer1_size; 144 | f = 0; 145 | - for (c = 0; c < layer1_size; c++) f += syn0[c + l1] * syn1neg[c + l2]; 146 | + for (c = 0; c < layer1_size; c++) f += syn0[c + l1] * _syn1neg[c + l2]; 147 | if (f > MAX_EXP) g = (label - 1) * alpha; 148 | else if (f < -MAX_EXP) g = (label - 0) * alpha; 149 | else g = (label - expTable[(int)((f + MAX_EXP) * (EXP_TABLE_SIZE / MAX_EXP / 2))]) * alpha; 150 | - for (c = 0; c < layer1_size; c++) neu1e[c] += g * syn1neg[c + l2]; 151 | - for (c = 0; c < layer1_size; c++) syn1neg[c + l2] += g * syn0[c + l1]; 152 | + for (c = 0; c < layer1_size; c++) neu1e[c] += g * _syn1neg[c + l2]; 153 | + for (c = 0; c < layer1_size; c++) _syn1neg[c + l2] += g * syn0[c + l1]; 154 | + if (target < local_syn1_size) 155 | + for (c = 0; c < layer1_size; c++) syn1neg_diff[c + l2] += g * syn0[c + l1]; 156 | } 157 | // Learn weights input -> hidden 158 | for (c = 0; c < layer1_size; c++) syn0[c + l1] += neu1e[c]; 159 | @@ -534,8 +573,32 @@ void *TrainModelThread(void *id) { 160 | sentence_length = 0; 161 | continue; 162 | } 163 | + num_updates++; 164 | + if (local_cache_update <= num_updates) { 165 | + if (hs) { 166 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 167 | + syn1[a] += syn1_diff[a]; 168 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 169 | + syn1_local[a] = syn1[a]; 170 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 171 | + syn1_diff[a] = 0; 172 | + } 173 | + if (negative > 0) { 174 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 175 | + syn1neg[a] += syn1neg_diff[a]; 176 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 177 | + syn1neg_local[a] = syn1neg[a]; 178 | + for (a = 0; a < local_syn1_size * layer1_size; a++) 179 | + syn1neg_diff[a] = 0; 180 | + } 181 | + num_updates = 0; 182 | + } 183 | } 184 | fclose(fi); 185 | + free(syn1neg_diff); 186 | + free(syn1neg_local); 187 | + free(syn1_diff); 188 | + free(syn1_local); 189 | free(neu1); 190 | free(neu1e); 191 | pthread_exit(NULL); 192 | @@ -649,6 +712,8 @@ int main(int argc, char **argv) { 193 | printf("\t\tUse threads (default 12)\n"); 194 | printf("\t-iter \n"); 195 | printf("\t\tRun more training iterations (default 5)\n"); 196 | + printf("\t-cache-update \n"); 197 | + printf("\t\tSynchronize after updates (default 100)\n"); 198 | printf("\t-min-count \n"); 199 | printf("\t\tThis will discard words that appear less than times; default is 5\n"); 200 | printf("\t-alpha \n"); 201 | @@ -688,6 +753,7 @@ int main(int argc, char **argv) { 202 | if ((i = ArgPos((char *)"-negative", argc, argv)) > 0) negative = atoi(argv[i + 1]); 203 | if ((i = ArgPos((char *)"-threads", argc, argv)) > 0) num_threads = atoi(argv[i + 1]); 204 | if ((i = ArgPos((char *)"-iter", argc, argv)) > 0) iter = atoi(argv[i + 1]); 205 | + if ((i = ArgPos((char *)"-cache-update", argc, argv)) > 0) local_cache_update = atoi(argv[i + 1]); 206 | if ((i = ArgPos((char *)"-min-count", argc, argv)) > 0) min_count = atoi(argv[i + 1]); 207 | if ((i = ArgPos((char *)"-classes", argc, argv)) > 0) classes = atoi(argv[i + 1]); 208 | vocab = (struct vocab_word *)calloc(vocab_max_size, sizeof(struct vocab_word)); 209 | -------------------------------------------------------------------------------- /www/documentation.css: -------------------------------------------------------------------------------- 1 | @import 'octicons.css'; 2 | 3 | /*------------------------------------------------------------------------------ 4 | Global Documentation Styles 5 | ------------------------------------------------------------------------------*/ 6 | 7 | body { 8 | font: 13px/1.4em "Helvetica Neue", arial,freesans,clean,sans-serif; 9 | background-color: #fff; 10 | color: #393939; 11 | } 12 | 13 | p { 14 | margin: 1em 0; 15 | } 16 | 17 | h1 { 18 | font-size: 20px; 19 | padding: .5em 0; 20 | margin: 2em 0 1em; 21 | font-weight: normal; 22 | } 23 | 24 | .content h1 { 25 | font-weight: 300; 26 | -webkit-backface-visibility: hidden; 27 | font-size: 36px; 28 | margin: 0em 0 0.5em; 29 | position: relative; 30 | line-height: 30px; 31 | } 32 | 33 | .header-anchor { 34 | position: absolute; 35 | left: -32px; 36 | top: 0; 37 | opacity: 0; 38 | padding: 0 5px 0 10px; 39 | height: 100%; 40 | width: 20px; 41 | font: normal normal 16px/20px octicons; 42 | color: #000; 43 | -webkit-font-smoothing: antialiased; 44 | color: #333; 45 | 46 | -webkit-transition: opacity 0.3s ease-in-out 0s; 47 | -moz-transition: opacity 0.3s ease-in-out 0s; 48 | transition: opacity 0.3s ease-in-out 0s; 49 | } 50 | 51 | h1 .header-anchor { 52 | line-height: 55px; 53 | } 54 | 55 | .content h1:hover .header-anchor, .content h2:hover .header-anchor, .content h3:hover .header-anchor, .content h4:hover .header-anchor, .header-anchor:hover { 56 | opacity: 1; 57 | text-decoration: none; 58 | } 59 | 60 | .header-anchor:before { 61 | content:'\f05c'; 62 | } 63 | 64 | h2 { 65 | font-size: 22px; 66 | color: #333; 67 | margin: 2em auto 1em; 68 | position: relative; 69 | font-weight: 300; 70 | } 71 | 72 | .library-list h1 { 73 | margin: 20px 0 20px; 74 | padding: 20px 0 20px; 75 | } 76 | 77 | .library-list h2 { 78 | margin-bottom: -17px; 79 | margin-left: 0px; 80 | font-size: 16px; 81 | max-width: 190px; 82 | } 83 | 84 | .library-list ul { 85 | margin-bottom: 40px; 86 | list-style: none; 87 | } 88 | 89 | .library-list ul li { 90 | padding-left: 200px; 91 | padding-right: 200px; 92 | font-size: 14px; 93 | } 94 | 95 | .change h2, .sidebar-shell h2 { 96 | margin: 1.2em 0 1em; 97 | } 98 | 99 | .change > .title { 100 | line-height: 1.4em; 101 | margin: 1.2em 0 0px; 102 | padding-left: 0; 103 | font-size: 30px; 104 | } 105 | 106 | h2 span.step { 107 | color: #666; 108 | } 109 | 110 | h3 { 111 | font-size: 14px; 112 | color: #333; 113 | margin: 1.5em 0 .5em; 114 | position: relative; 115 | } 116 | 117 | h4 { 118 | margin: 1em 0; 119 | position: relative; 120 | } 121 | 122 | h5 { 123 | font-size: 13px; 124 | } 125 | 126 | h6 { 127 | font-size: 13px; 128 | color: #666; 129 | } 130 | 131 | a { 132 | color: #4183C4; 133 | text-decoration: none; 134 | } 135 | 136 | a:hover, 137 | a:active { 138 | text-decoration:underline; 139 | } 140 | 141 | blockquote { 142 | margin:0 -5px; 143 | padding: 0px 20px; 144 | } 145 | 146 | dt { 147 | font-weight: bold; 148 | } 149 | 150 | dd { 151 | padding-left: 1em; 152 | margin-bottom: 1em; 153 | } 154 | 155 | dd + dd { 156 | margin-bottom: 0; 157 | } 158 | 159 | a img { 160 | border: 0px; 161 | } 162 | 163 | .button, .button-secondary { 164 | background-color: #297fc7; 165 | color: #fff; 166 | font-size: 16px; 167 | padding: 15px; 168 | border-radius: 5px; 169 | } 170 | 171 | .button:hover { 172 | text-decoration: none; 173 | background-color: #3088d0; 174 | } 175 | 176 | .button-secondary { 177 | background-color: #738797; 178 | font-size: 15px; 179 | padding: 12px; 180 | -webkit-backface-visibility: hidden; 181 | } 182 | 183 | .button-secondary:hover { 184 | text-decoration: none; 185 | background-color: #7e909e; 186 | } 187 | 188 | .rss-subscribe { 189 | display: block; 190 | padding: 15px 14px 16px 47px; 191 | background: url(/images/feed-icon.png) 10px center no-repeat; 192 | color: #444; 193 | font-size: 13px; 194 | text-decoration: none; 195 | } 196 | 197 | .rss-subscribe:hover { 198 | background-color: #f4f4f7; 199 | text-decoration: none; 200 | } 201 | 202 | /*------------------------------------------------------------------------------ 203 | Header Styles 204 | ------------------------------------------------------------------------------*/ 205 | 206 | #header-wrapper { 207 | margin-bottom: 0; 208 | clear: both; 209 | height: 61px; 210 | background: url(../images/header.png) 0 0 repeat-x; 211 | } 212 | 213 | #header { 214 | margin: 0 auto; 215 | max-width: 980px; 216 | position: relative; 217 | } 218 | 219 | #header .logo { 220 | float: left; 221 | margin-top: 17px; 222 | display: inline-block; 223 | width: 186px; 224 | height: 27px; 225 | background: url(/images/logo_developer.png) 0 0 no-repeat; 226 | text-indent: -999em; 227 | } 228 | 229 | #header .nav { 230 | float: right; 231 | font-weight: bold; 232 | margin-top: 20px; 233 | text-shadow: white 1px 1px 0px; 234 | font-size: 14px; 235 | } 236 | 237 | #header .nav li { 238 | display: inline-block; 239 | margin: 0; 240 | list-style-type: none; 241 | line-height: 1.4em; 242 | } 243 | 244 | #header .nav a { 245 | color: #515151; 246 | outline: none; 247 | text-decoration: none; 248 | padding: 20px 11px 19px; 249 | -webkit-transition: color 0.2s ease-in-out 0s; 250 | -moz-transition: color 0.2s ease-in-out 0s; 251 | transition: color 0.2s ease-in-out 0s; 252 | } 253 | 254 | .overview #header .nav-overview, 255 | .api #header .nav-api, 256 | .blog #header .nav-blog, 257 | .developers #header .nav-developers { 258 | color: #333; 259 | border-bottom: 3px solid #c9c9c9; 260 | } 261 | 262 | #header .nav a:hover { 263 | color: #327fc7; 264 | } 265 | 266 | /** Search **/ 267 | 268 | #header .nav #search-container { 269 | position: relative; 270 | margin-left: 5px; 271 | text-shadow: none; 272 | } 273 | 274 | #searchfield { 275 | border: 1px solid #dddddd; 276 | line-height: 20px; 277 | height: 21px; 278 | padding: 2px 5px 2px 20px; 279 | display: inline-block; 280 | margin-top: -4px; 281 | width: 51px; 282 | border-radius: 3px; 283 | background: url(/images/search.png) 5px center no-repeat #fff; 284 | 285 | -webkit-transition: width 0.3s ease-in-out 0s; 286 | -moz-transition: width 0.3s ease-in-out 0s; 287 | transition: width 0.3s ease-in-out 0s; 288 | } 289 | 290 | /** Hide native X appearing in IE10 **/ 291 | 292 | #searchfield::-ms-clear { 293 | height: 0; 294 | width: 0; 295 | } 296 | 297 | #searchfield:focus, #search-container.active #searchfield { 298 | width: 120px; 299 | } 300 | 301 | #searchfield:focus + .search-placeholder, #search-container.active .search-placeholder { 302 | opacity: 0; 303 | } 304 | 305 | .search-placeholder { 306 | position: absolute; 307 | cursor: text; 308 | left: 22px; 309 | top: 0px; 310 | font-weight: normal; 311 | font-size: 12px; 312 | pointer-events: none; 313 | color: #aaa; 314 | line-height: 20px; 315 | 316 | -webkit-transition: opacity 0.3s ease-in-out 0s; 317 | -moz-appearance-transition: opacity 0.3s ease-in-out 0s; 318 | transition: opacity 0.3s ease-in-out 0s; 319 | } 320 | 321 | .cancel-search { 322 | position: absolute; 323 | right: 4px; 324 | top: 2px; 325 | background: url(/images/cancel.png) 0 0 no-repeat; 326 | width: 14px; 327 | height: 14px; 328 | display: none; 329 | } 330 | 331 | #search-container.active .cancel-search { 332 | display: block; 333 | } 334 | 335 | #search-results { 336 | position: absolute; 337 | top: 30px; 338 | right: 0px; 339 | width: 260px; 340 | background-color: #fff; 341 | border-radius: 4px; 342 | min-height: 40px; 343 | z-index: 100; 344 | overflow: hidden; 345 | box-shadow: 0px 1px 3px rgba(0,0,0,0.4); 346 | visibility: hidden; 347 | opacity: 0; 348 | } 349 | 350 | #search-container.active #searchfield:focus ~ #search-results, #search-results:active, #search-results:focus, #search-results:hover { 351 | visibility: visible; 352 | opacity: 1; 353 | -webkit-transition: opacity 0.3s ease-in-out 0s; 354 | -moz-transition: opacity 0.3s ease-in-out 0s; 355 | transition: opacity 0.3s ease-in-out 0s; 356 | } 357 | 358 | #search-results li { 359 | display: block; 360 | width: 100%; 361 | border-bottom: 1px solid #f2f2f2; 362 | } 363 | 364 | #header #search-results .result a { 365 | display: block; 366 | text-shadow: none; 367 | padding: 10px; 368 | } 369 | 370 | #search-results .placeholder { 371 | text-align: center; 372 | font-size: 12px; 373 | font-weight: normal; 374 | padding: 20px 0; 375 | } 376 | 377 | #search-results em { 378 | font-weight: bold; 379 | } 380 | 381 | #search-results .result em { 382 | display: block; 383 | font-weight: normal; 384 | font-style: normal; 385 | line-height: 1em; 386 | } 387 | 388 | .result small { 389 | font-size: 12px; 390 | color: #5c5c5c; 391 | line-height: 1em; 392 | } 393 | 394 | .result:hover, .result.selected { 395 | background-color: #297fc7 !important; 396 | } 397 | 398 | .result:hover em, .result.selected em, .result:hover small, .result.selected small { 399 | color: #fff !important; 400 | } 401 | 402 | #header #search-results li:last-child { 403 | border-bottom: none; 404 | } 405 | 406 | /*------------------------------------------------------------------------------ 407 | Subnav 408 | ------------------------------------------------------------------------------*/ 409 | 410 | .sub-nav { 411 | max-width: 980px; 412 | margin: 0 auto; 413 | position: relative; 414 | border-bottom: 1px solid #d8d8d8; 415 | } 416 | 417 | .sub-nav > h2, .sub-nav > h2 > a { 418 | color: #333; 419 | font-weight: normal; 420 | font-size: 22px; 421 | -webkit-backface-visibility: hidden; 422 | margin: 50px 0 10px 0; 423 | text-decoration: none; 424 | } 425 | 426 | .sub-nav > h2 > a { 427 | z-index: 10; 428 | position: relative; 429 | } 430 | 431 | .sub-nav ul { 432 | list-style: none; 433 | position: absolute; 434 | right: 0; 435 | bottom: 8px; 436 | width: 100%; 437 | text-align: right; 438 | } 439 | 440 | .sub-nav li { 441 | display: inline-block; 442 | } 443 | 444 | .sub-nav li a { 445 | color: #767676; 446 | font-size: 14px; 447 | margin-left: 16px; 448 | padding: 4px 2px 8px; 449 | } 450 | 451 | .sub-nav li a:hover { 452 | text-decoration: none; 453 | color: #327fc7; 454 | } 455 | 456 | .sub-nav li:last-child a { 457 | padding-right: 0; 458 | } 459 | 460 | .sub-nav li .active { 461 | color: #222; 462 | border-bottom: 2px solid #d8d8d8; 463 | } 464 | 465 | /*------------------------------------------------------------------------------ 466 | Features 467 | ------------------------------------------------------------------------------*/ 468 | 469 | .wrapper { 470 | max-width: 980px; 471 | margin: 0 auto; 472 | } 473 | 474 | .wrapper:before, .wrapper:after { 475 | content: " "; 476 | } 477 | 478 | .wrapper:after { 479 | clear: both; 480 | } 481 | 482 | .wrapper.blog { 483 | display: table; 484 | } 485 | 486 | .feature { 487 | position: relative; 488 | height: 420px; 489 | } 490 | 491 | .feature h1, .library-list .feature h1 { 492 | font-weight: 300; 493 | font-size: 42px; 494 | line-height: 1.2em; 495 | -webkit-backface-visibility: hidden; 496 | padding: 80px 0 0 0; 497 | margin: 0; 498 | text-align: left; 499 | } 500 | 501 | .library-list .feature { 502 | overflow: hidden; 503 | border-bottom: 1px solid #ddd; 504 | padding-bottom: 40px; 505 | } 506 | 507 | .feature .intro { 508 | color: #666; 509 | font-size: 18px; 510 | line-height: 1.5em; 511 | max-width: 400px; 512 | } 513 | 514 | .feature .button, .highlights .button-secondary { 515 | display: inline-block; 516 | } 517 | 518 | .feature .electrocat { 519 | position: absolute; 520 | right: 0; 521 | top: 45px; 522 | } 523 | 524 | .feature .rocketship { 525 | position: absolute; 526 | right: -50px; 527 | top: 30px; 528 | z-index: -1; 529 | } 530 | 531 | .feature .gundamcat { 532 | position: absolute; 533 | right: 0; 534 | top: 50px; 535 | } 536 | 537 | .feature .library-links { 538 | line-height: 40px; 539 | margin-top: 25px; 540 | } 541 | 542 | .feature .library-links li { 543 | padding: 0; 544 | font-size: 22px; 545 | } 546 | 547 | .feature .library-links span { 548 | width: 60px; 549 | text-align: right; 550 | padding-right: 10px; 551 | display: inline-block; 552 | } 553 | 554 | .feature + .full-width-divider { 555 | margin-top: -1px; 556 | } 557 | 558 | .full-width-divider { 559 | width: 100%; 560 | background: #f2f2f2; 561 | border-top: 1px solid #ddd; 562 | border-bottom: 1px solid #ddd; 563 | overflow: hidden; 564 | z-index: 1; 565 | position: relative; 566 | } 567 | 568 | .octicon-mail-read:before { 569 | content: "\f03c"; 570 | } 571 | 572 | .octicon-code:before { 573 | content: '\f05f'; 574 | } 575 | 576 | .octicon-file-text:before { 577 | content: '\f011' 578 | } 579 | 580 | .highlight-module { 581 | list-style-type: none; 582 | display: table-cell; 583 | padding: 40px 14px 30px; 584 | text-align: center; 585 | margin-top: 10px; 586 | font-size: 14px; 587 | line-height: 1.5em; 588 | } 589 | 590 | .highlight-module:first-child { 591 | padding-left: 0; 592 | padding-right: 14px; 593 | } 594 | 595 | .highlight-module:last-child { 596 | padding-left: 14px; 597 | padding-right: 0; 598 | } 599 | 600 | .highlight-module .mega-octicon { 601 | font-size: 70px; 602 | width: 70px; 603 | height: 70px; 604 | } 605 | 606 | .highlights h2 { 607 | margin: 25px 0 15px; 608 | font-size: 30px; 609 | line-height: 1.2; 610 | -webkit-backface-visibility: hidden; 611 | } 612 | 613 | a .mega-octicon { 614 | color: #333; 615 | } 616 | 617 | .full-width-divider + #footer { 618 | margin-top: -1px; 619 | } 620 | 621 | /*------------------------------------------------------------------------------ 622 | Sidebar 623 | ------------------------------------------------------------------------------*/ 624 | 625 | .sidebar-shell { 626 | position: relative; 627 | float: right; 628 | margin: 20px 0 0; 629 | width: 290px; 630 | } 631 | 632 | .sidebar-module { 633 | border-radius: 4px; 634 | overflow: hidden; 635 | margin-bottom: 20px; 636 | font-size: 12px; 637 | border: 1px solid #ddd; 638 | } 639 | 640 | .sidebar-module ul { 641 | margin: 0px; 642 | } 643 | 644 | .sidebar-module.notice { 645 | background: #fafafb; 646 | border: 1px solid #CACACA; 647 | border-radius: 4px; 648 | line-height: 1.5em; 649 | } 650 | 651 | .notice > p { 652 | padding: 12px; 653 | margin: 0; 654 | } 655 | 656 | .sidebar-module li { 657 | list-style-type: none; 658 | } 659 | 660 | .sidebar-module ul li:last-child h3, .headlines li:last-child { 661 | border-bottom: none; 662 | } 663 | 664 | .sidebar-module ul li:last-child ul li:first-child { 665 | border-top: 1px solid #eee; 666 | } 667 | 668 | .sidebar-module ul h3, .headlines > li { 669 | margin: 0px; 670 | color: #666; 671 | border-bottom: 1px solid #eee; 672 | font-size: 14px; 673 | } 674 | 675 | .headlines > li > a { 676 | color: #333; 677 | display: block; 678 | padding: 10px 10px; 679 | text-decoration: none; 680 | font-size: 15px; 681 | font-weight: 300; 682 | } 683 | 684 | .headlines > li > a:hover { 685 | color: #327fc7; 686 | } 687 | 688 | .headlines > li > a > .date { 689 | font-weight: normal; 690 | font-size: 12px; 691 | color: #888; 692 | } 693 | 694 | .headlines li:hover { 695 | background-color: #fdfdfd; 696 | } 697 | 698 | .sidebar-module h3 a { 699 | padding: 8px 10px 8px 10px; 700 | color: #555; 701 | text-decoration: none; 702 | display: block; 703 | } 704 | 705 | .sidebar-module h3 a:hover { 706 | text-decoration: none; 707 | color: #327fc7; 708 | background-color: #fdfdfd; 709 | } 710 | 711 | .sidebar-module ul ul li { 712 | border-bottom: 1px solid #eee; 713 | font-weight: bold; 714 | color: #666; 715 | background-color: #f9f9f9; 716 | } 717 | 718 | .sidebar-module ul li:last-child ul li:last-child { 719 | border-bottom: none; 720 | } 721 | 722 | .sidebar-module .disable > a { 723 | color: #333; 724 | border-left: 2px solid #d8d8d8; 725 | padding-left: 29px; 726 | } 727 | 728 | .sidebar-module ul ul li a { 729 | padding: 6px 0px 6px 31px; 730 | display: block; 731 | text-decoration: none; 732 | font-weight: normal; 733 | } 734 | 735 | .sidebar-module ul ul li a:hover { 736 | background-color: #f2f2f2; 737 | color: #327fc7; 738 | } 739 | 740 | 741 | /*****************************************************************************/ 742 | /* 743 | /* Footer 744 | /* 745 | /*****************************************************************************/ 746 | 747 | .api-status { 748 | text-align: center; 749 | padding: 20px 0; 750 | border-top: 1px solid #ddd; 751 | font-size: 14px; 752 | } 753 | 754 | .api-status a { 755 | color: #999; 756 | background-position: center bottom; 757 | background-repeat: no-repeat; 758 | padding-bottom: 40px; 759 | display: block; 760 | } 761 | 762 | .api-status .unknown { 763 | background-image: url(/images/status-icon-unknown.png); 764 | } 765 | 766 | .api-status .good { 767 | color: #227b41; 768 | background-image: url(/images/status-icon-good.png); 769 | } 770 | 771 | .api-status .minor { 772 | color: #ed8b3f; 773 | background-image: url(/images/status-icon-minor.png); 774 | } 775 | 776 | .api-status .major { 777 | color: #ad1a05; 778 | background-image: url(/images/status-icon-major.png); 779 | } 780 | 781 | .api-status a:hover { 782 | text-decoration: none; 783 | color: #327fc7; 784 | } 785 | 786 | #footer { 787 | position: relative; 788 | bottom:0; 789 | font-size:12px; 790 | color: #636363; 791 | margin: 0 auto; 792 | max-width: 980px; 793 | clear: both; 794 | } 795 | 796 | #footer a:hover { 797 | text-decoration: underline; 798 | } 799 | 800 | #footer .mega-octicon { 801 | color: #ccc; 802 | font-size: 24px; 803 | } 804 | 805 | #footer .mega-octicon:hover { 806 | color: #bbb; 807 | text-decoration: none; 808 | } 809 | 810 | .lower_footer { 811 | border-top: 1px solid #ddd; 812 | padding: 30px 0; 813 | } 814 | 815 | .footer-cell { 816 | display: table-cell; 817 | text-align: center; 818 | width: 10%; 819 | vertical-align: middle; 820 | } 821 | 822 | .footer-cell:first-child { 823 | text-align: left; 824 | padding-left: 0; 825 | } 826 | 827 | .footer-cell:last-child { 828 | text-align: right; 829 | padding-right: 0; 830 | } 831 | 832 | .footer-cell:last-child li { 833 | margin: 0 0 0 20px; 834 | padding-right: 0; 835 | } 836 | 837 | .footer-cell li { 838 | display: inline; 839 | margin-right: 20px; 840 | list-style: none; 841 | } 842 | 843 | #footer .wrapper { 844 | border-top: 1px solid #ddd; 845 | text-align: center; 846 | padding: 20px 0; 847 | } 848 | 849 | /* end */ 850 | 851 | /*------------------------------------------------------------------------------ 852 | Dev Program 853 | ------------------------------------------------------------------------------*/ 854 | .dev-program.feature { 855 | height: 420px; 856 | margin-top: -2px; 857 | position: relative; 858 | } 859 | 860 | .dev-program.feature, 861 | .dev-program-callout { 862 | background: #160625; 863 | } 864 | 865 | .dev-program.feature .intro { 866 | font-size: 20px; 867 | line-height: 1.5em; 868 | max-width: 500px; 869 | } 870 | 871 | .dev-program.feature h1 { 872 | width: 600px; 873 | } 874 | 875 | .dev-program.feature .earth { 876 | margin-top: -187px; 877 | position: absolute; 878 | right: 5%; 879 | top: 50%; 880 | z-index: 0; 881 | } 882 | 883 | .dev-program.feature .earth-short-loop { 884 | display: none; 885 | } 886 | 887 | .dev-program.feature .wrapper { 888 | position: relative; 889 | z-index: 1; 890 | } 891 | 892 | .dev-program-callout .wrapper { 893 | padding: 40px 0; 894 | background: #160625 url(/images/callout-earth-static.png) no-repeat top right / 30%; 895 | } 896 | 897 | .dev-program.feature h1, 898 | .dev-program-callout h2 { 899 | color: #eee; 900 | } 901 | 902 | .dev-program.feature .intro, 903 | .dev-program-callout p { 904 | color: #a397ad; 905 | } 906 | 907 | .dev-program-callout h2 { 908 | margin: 0; 909 | } 910 | 911 | .dev-program-callout p { 912 | font-size: 16px; 913 | margin: 10px 0 0; 914 | } 915 | 916 | /* end */ 917 | 918 | /*------------------------------------------------------------------------------ 919 | Not Footer 920 | ------------------------------------------------------------------------------*/ 921 | #wrapper { 922 | padding: 20px 0; 923 | height: auto; 924 | max-width: 980px; 925 | margin: 0 auto; 926 | } 927 | 928 | #wrapper:after, #wrapper:before { 929 | content: " "; 930 | display: table; 931 | } 932 | 933 | #wrapper:after { 934 | clear: both; 935 | } 936 | 937 | .content { 938 | width: 630px; 939 | position: relative; 940 | float: left; 941 | color: #393939; 942 | z-index: 2; 943 | font-size: 14px; 944 | line-height: 1.5em; 945 | } 946 | 947 | .blog .content { 948 | margin-bottom: 60px; 949 | } 950 | 951 | .content dt { 952 | color: #666; 953 | } 954 | 955 | .content ol { 956 | margin-left: 1.5em; 957 | } 958 | 959 | .content ul { 960 | margin: 1.5em; 961 | list-style-type: disc; 962 | } 963 | 964 | .content dd ul { 965 | margin-top: 0; 966 | } 967 | 968 | .content li { 969 | margin: 0.5em 0; 970 | } 971 | 972 | .content img { 973 | max-width: 100%; 974 | margin: 12px 0; 975 | } 976 | 977 | /* Taken from Help in order to show images in ordered lists inline */ 978 | ol { 979 | counter-reset: li; 980 | list-style: none; 981 | position: relative; 982 | padding-bottom: 10px; 983 | } 984 | ol > li { 985 | padding: 5px 0 5px 55px; 986 | position: relative; 987 | margin-bottom: 5px; 988 | } 989 | ol > li:before { 990 | content: counter(li); 991 | counter-increment: li; 992 | position: absolute; 993 | top: 0; 994 | left: 0; 995 | height: 100%; 996 | width: 30px; 997 | padding: 0 10px 0 0; 998 | color: #999; 999 | font-size: 22px; 1000 | font-weight: bold; 1001 | line-height: 35px; 1002 | text-align: right; 1003 | border-right: 1px solid #ddd; 1004 | } 1005 | ol > li > p:first-child { 1006 | margin-top: 0; 1007 | } 1008 | ol > li:after { 1009 | content: "."; 1010 | display: block; 1011 | clear: both; 1012 | visibility: hidden; 1013 | line-height: 0; 1014 | height: 0; 1015 | } 1016 | .content ol > li img { 1017 | max-width: 100px; 1018 | margin: 0 0 0 10px; 1019 | float: right; 1020 | border: 1px solid #ddd; 1021 | cursor: pointer; 1022 | } 1023 | .content ol > li img.expanded { 1024 | max-width: 400px; 1025 | } 1026 | 1027 | .content .full-image { 1028 | position: absolute; 1029 | top: 5px; 1030 | right: -20px; 1031 | z-index: 100; 1032 | } 1033 | .content .full-image img { 1034 | position: absolute; 1035 | top: 0; 1036 | right: 20px; 1037 | margin: 0; 1038 | max-width: 600px; 1039 | box-shadow: 0 0 3px rgba(0, 0, 0, 0.2); 1040 | } 1041 | .content .full-image:hover .octicon, .full-image:hover .mini-icon { 1042 | color: #666; 1043 | } 1044 | .content .full-image .octicon, .full-image .octicon-remove-close { 1045 | position: absolute; 1046 | top: 0px; 1047 | right: 0px; 1048 | color: #999; 1049 | cursor: pointer; 1050 | } 1051 | 1052 | .content .description { 1053 | margin-left: 20px; 1054 | } 1055 | 1056 | .content .verseblock-content { 1057 | padding: 3px; 1058 | } 1059 | 1060 | .content .verseblock-content, 1061 | .content .sectionbody .dlist dt, 1062 | .content p > tt, 1063 | .content dl code, 1064 | .content ul code, 1065 | p code { 1066 | border-radius: 3px; 1067 | border: 1px solid #ccc; 1068 | background-color: #f9f9f9; 1069 | padding: 0px 3px; 1070 | display: inline-block; 1071 | } 1072 | 1073 | .content .sectionbody .dlist dt { 1074 | margin-top: 10px; 1075 | } 1076 | 1077 | .content .verseblock-content { 1078 | padding: 3px; 1079 | } 1080 | 1081 | .content .intro { 1082 | color: #868686; 1083 | } 1084 | 1085 | .article-content ul { 1086 | margin: 1.5em; 1087 | } 1088 | 1089 | .change { 1090 | padding-bottom: 40px; 1091 | } 1092 | 1093 | 1094 | .change .meta { 1095 | font-size: 13px; 1096 | margin: 3px 0 20px; 1097 | } 1098 | 1099 | .change .meta ul { 1100 | margin: 0; 1101 | } 1102 | 1103 | .change .meta ul img { 1104 | margin: 0; 1105 | } 1106 | 1107 | .who_when a { 1108 | padding-left: 6px; 1109 | color: #999; 1110 | } 1111 | 1112 | .change .who_when .author { 1113 | color: #eee; 1114 | } 1115 | 1116 | .change .who_when .published { 1117 | color: #ccc; 1118 | } 1119 | 1120 | .meta li { 1121 | color: #999; 1122 | padding-right: 20px; 1123 | list-style: none; 1124 | display: inline; 1125 | } 1126 | 1127 | .published span { 1128 | color: #999; 1129 | vertical-align: middle; 1130 | padding-bottom: 5px; 1131 | } 1132 | 1133 | #markdown-toc { 1134 | margin: 1em 1.5em 2em; 1135 | } 1136 | 1137 | #markdown-toc li { 1138 | font-size: 14px; 1139 | list-style-type: lower-roman; 1140 | margin: 0.25em 0; 1141 | } 1142 | 1143 | #markdown-toc li a { 1144 | color: #333; 1145 | text-decoration: underline; 1146 | } 1147 | 1148 | #markdown-toc li a:hover { 1149 | color: #327fc7; 1150 | } 1151 | 1152 | .content table { 1153 | width: 100%; 1154 | overflow: auto; 1155 | display: block; 1156 | margin: 15px 0; 1157 | } 1158 | 1159 | .content thead { 1160 | margin: 0; 1161 | padding: 0; 1162 | border: 0; 1163 | font-size: 100%; 1164 | font: inherit; 1165 | vertical-align: baseline; 1166 | } 1167 | 1168 | .content table th { 1169 | border: 1px solid #ddd; 1170 | padding: 6px 13px; 1171 | font-weight: bold; 1172 | text-align: center; 1173 | } 1174 | 1175 | .content tbody { 1176 | vertical-align: middle; 1177 | border-color: inherit; 1178 | } 1179 | 1180 | .content table tr { 1181 | border-top: 1px solid #ccc; 1182 | background-color: #fff; 1183 | vertical-align: inherit; 1184 | } 1185 | 1186 | .content table td { 1187 | border: 1px solid #ddd; 1188 | padding: 6px 13px; 1189 | } 1190 | 1191 | /* @end */ 1192 | 1193 | /*------------------------------------------------------------------------------ 1194 | Pre/Code Styles 1195 | ------------------------------------------------------------------------------*/ 1196 | 1197 | code { 1198 | white-space: nowrap; 1199 | font: 12px Consolas, 'Liberation Mono', Courier, monospace; 1200 | } 1201 | 1202 | pre code {white-space: pre;} 1203 | 1204 | pre { 1205 | border: 1px solid #cacaca; 1206 | font: 12px/1.4em Consolas, 'Liberation Mono', Courier, monospace; 1207 | padding: 10px; 1208 | overflow:auto; 1209 | border-radius: 3px; 1210 | background-color: #fafafb; 1211 | color: #393939; 1212 | margin: 2em 0; 1213 | } 1214 | 1215 | ul + pre { 1216 | margin-top: 1em; 1217 | } 1218 | 1219 | pre span.comment {color: #aaa;} 1220 | 1221 | .headers { 1222 | margin-bottom: 0; 1223 | border-radius: 3px 3px; 1224 | border-bottom: 1px solid #CACACA; 1225 | background-color: #f4f4f4; 1226 | } 1227 | 1228 | .headers + pre { 1229 | border-radius: 0 0 3px 3px; 1230 | margin-top: 0; 1231 | border-top-color: #ddd; 1232 | } 1233 | 1234 | .headers + pre.body-response { 1235 | margin-top: -2px; 1236 | } 1237 | 1238 | .terminal { 1239 | background-color: #444; 1240 | color: #fff; 1241 | border-radius: 3px; 1242 | border: none; 1243 | position: relative; 1244 | } 1245 | 1246 | .terminal em { 1247 | color: #f9fe64; 1248 | } 1249 | 1250 | .terminal span.comment { 1251 | color: #ccc; 1252 | } 1253 | 1254 | .terminal span.output { 1255 | color: #63E463; 1256 | } 1257 | 1258 | /****************************/ 1259 | /* Expandable List Module */ 1260 | /****************************/ 1261 | 1262 | .sidebar-module a + a { 1263 | padding-left: 27px; 1264 | } 1265 | 1266 | .sidebar-module h3 { 1267 | position: relative; 1268 | } 1269 | 1270 | .sidebar-module .arrow-btn { 1271 | background-image: url(/images/expand-arrows.png); 1272 | background-repeat: no-repeat; 1273 | background-position: -3px -3px; 1274 | width: 13px; 1275 | height: 100%; 1276 | padding: 0 4px; 1277 | text-decoration: none; 1278 | position: absolute; 1279 | left: 0; 1280 | top: 0; 1281 | background-color: transparent; 1282 | } 1283 | 1284 | .arrow-btn.expanded { 1285 | background-position: -41px -3px; 1286 | } 1287 | 1288 | .arrow-btn.collapsed:hover { 1289 | background-position: -3px -43px; 1290 | background-color: transparent; 1291 | } 1292 | 1293 | .arrow-btn.expanded:hover { 1294 | background-position: -41px -43px; 1295 | background-color: transparent; 1296 | } 1297 | 1298 | .alert { 1299 | position:relative; 1300 | padding: 0 15px; 1301 | color: #264c72; 1302 | border: 1px solid #97c1da; 1303 | border-radius: 3px; 1304 | background-color: #d8ebf8; 1305 | } 1306 | 1307 | .warning { 1308 | position:relative; 1309 | padding: 0 15px; 1310 | color: #613A00; 1311 | border: 1px solid #dca874; 1312 | border-radius: 3px; 1313 | background-color: #ffe3c8; 1314 | } 1315 | 1316 | /*------------------------------------------------------------------------------ 1317 | Dev Program 1318 | ------------------------------------------------------------------------------*/ 1319 | 1320 | .program-info-column { 1321 | float: left; 1322 | position: relative; 1323 | margin: 20px 0 60px; 1324 | padding-left: 50px; 1325 | width: 415px; 1326 | } 1327 | 1328 | .program-info-column:first-child { 1329 | margin-right: 50px; 1330 | } 1331 | 1332 | .program-info-column h2 { 1333 | margin-bottom: 0; 1334 | font-size: 18px; 1335 | } 1336 | 1337 | .program-info-column p { 1338 | margin-top: 5px; 1339 | font-size: 16px; 1340 | line-height: 1.5; 1341 | color: #666; 1342 | } 1343 | 1344 | .program-info-column .mega-octicon { 1345 | position: absolute; 1346 | left: 0; 1347 | top: 32px; 1348 | color: #333; 1349 | } 1350 | 1351 | .program-info-column ul { 1352 | margin-left: 15px; 1353 | } 1354 | 1355 | /* @end */ 1356 | 1357 | /********************/ 1358 | /* Retina support */ 1359 | /********************/ 1360 | 1361 | @media only screen and (-webkit-min-device-pixel-ratio: 2), only screen and (min-resolution: 192dpi) { 1362 | 1363 | #header-wrapper { 1364 | background-image: url(../images/header@2x.png); 1365 | background-size: 8px 61px; 1366 | } 1367 | 1368 | #header .logo { 1369 | background-image: url(/images/logo_developer@2x.png); 1370 | background-size: 186px 27px; 1371 | } 1372 | 1373 | #searchfield { 1374 | background-image: url(/images/search@2x.png); 1375 | background-size: 13px 13px; 1376 | } 1377 | 1378 | .cancel-search { 1379 | background-image: url(/images/cancel@2x.png); 1380 | background-size: 14px 14px; 1381 | } 1382 | 1383 | .sidebar-module .arrow-btn { 1384 | background-image: url(/images/expand-arrows@2x.png); 1385 | background-size: 73px 80px; 1386 | } 1387 | 1388 | .api-status .unknown { 1389 | background-image: url(/images/status-icon-unknown@2x.png); 1390 | background-size: 32px 32px; 1391 | } 1392 | 1393 | .api-status .good { 1394 | background-image: url(/images/status-icon-good@2x.png); 1395 | background-size: 32px 32px; 1396 | } 1397 | 1398 | .api-status .minor { 1399 | background-image: url(/images/status-icon-minor@2x.png); 1400 | background-size: 32px 32px; 1401 | } 1402 | 1403 | .api-status .major { 1404 | background-image: url(/images/status-icon-major@2x.png); 1405 | background-size: 32px 32px; 1406 | } 1407 | 1408 | .rss-subscribe { 1409 | background-image: url(/images/feed-icon@2x.png); 1410 | background-size: 28px 28px; 1411 | } 1412 | 1413 | } 1414 | 1415 | /* @end */ 1416 | -------------------------------------------------------------------------------- /www/reset.css: -------------------------------------------------------------------------------- 1 | /* `XHTML, HTML4, HTML5 Reset 2 | ----------------------------------------------------------------------------------------------------*/ 3 | 4 | a, 5 | abbr, 6 | acronym, 7 | address, 8 | applet, 9 | article, 10 | aside, 11 | audio, 12 | b, 13 | big, 14 | blockquote, 15 | body, 16 | canvas, 17 | caption, 18 | center, 19 | cite, 20 | code, 21 | dd, 22 | del, 23 | details, 24 | dfn, 25 | dialog, 26 | div, 27 | dl, 28 | dt, 29 | em, 30 | embed, 31 | fieldset, 32 | figcaption, 33 | figure, 34 | font, 35 | footer, 36 | form, 37 | h1, 38 | h2, 39 | h3, 40 | h4, 41 | h5, 42 | h6, 43 | header, 44 | hgroup, 45 | hr, 46 | html, 47 | i, 48 | iframe, 49 | img, 50 | ins, 51 | kbd, 52 | label, 53 | legend, 54 | li, 55 | mark, 56 | menu, 57 | meter, 58 | nav, 59 | object, 60 | ol, 61 | output, 62 | p, 63 | pre, 64 | progress, 65 | q, 66 | rp, 67 | rt, 68 | ruby, 69 | s, 70 | samp, 71 | section, 72 | small, 73 | span, 74 | strike, 75 | strong, 76 | sub, 77 | summary, 78 | sup, 79 | table, 80 | tbody, 81 | td, 82 | tfoot, 83 | th, 84 | thead, 85 | time, 86 | tr, 87 | tt, 88 | u, 89 | ul, 90 | var, 91 | video, 92 | xmp { 93 | border: 0; 94 | margin: 0; 95 | padding: 0; 96 | font-size: 100%; 97 | } 98 | 99 | html, 100 | body { 101 | height: 100%; 102 | } 103 | 104 | article, 105 | aside, 106 | details, 107 | figcaption, 108 | figure, 109 | footer, 110 | header, 111 | hgroup, 112 | menu, 113 | nav, 114 | section { 115 | /* 116 | Override the default (display: inline) for 117 | browsers that do not recognize HTML5 tags. 118 | 119 | IE8 (and lower) requires a shiv: 120 | http://ejohn.org/blog/html5-shiv 121 | */ 122 | display: block; 123 | } 124 | 125 | b, 126 | strong { 127 | /* 128 | Makes browsers agree. 129 | IE + Opera = font-weight: bold. 130 | Gecko + WebKit = font-weight: bolder. 131 | */ 132 | font-weight: bold; 133 | } 134 | 135 | img { 136 | color: transparent; 137 | font-size: 0; 138 | vertical-align: middle; 139 | /* 140 | For IE. 141 | http://css-tricks.com/ie-fix-bicubic-scaling-for-images 142 | */ 143 | -ms-interpolation-mode: bicubic; 144 | } 145 | 146 | li { 147 | /* 148 | For IE6 + IE7. 149 | */ 150 | display: list-item; 151 | } 152 | 153 | table { 154 | border-collapse: collapse; 155 | border-spacing: 0; 156 | } 157 | 158 | th, 159 | td, 160 | caption { 161 | font-weight: normal; 162 | vertical-align: top; 163 | text-align: left; 164 | } 165 | 166 | svg { 167 | /* 168 | For IE9. 169 | */ 170 | overflow: hidden; 171 | } --------------------------------------------------------------------------------