├── nlpdocument ├── docs │ ├── baselines │ │ ├── aspect.md │ │ ├── dialogue.md │ │ ├── captioning.md │ │ ├── summarization.md │ │ ├── answer_selection.md │ │ └── question_answering.md │ ├── tutorial.md │ ├── helper │ │ ├── img │ │ │ ├── 01.png │ │ │ ├── 02.png │ │ │ ├── 03.png │ │ │ ├── 04.png │ │ │ ├── 05.png │ │ │ ├── 06.png │ │ │ ├── 07.png │ │ │ └── mechanism.png │ │ ├── index.md │ │ ├── element.md │ │ └── writing_paradigm.md │ ├── images │ │ ├── favicon.png │ │ └── logo.svg │ ├── dl_framework.md │ └── index.md ├── site │ ├── assets │ │ ├── javascripts │ │ │ ├── lunr │ │ │ │ ├── lunr.jp.js │ │ │ │ ├── lunr.multi.js │ │ │ │ ├── lunr.th.js │ │ │ │ ├── lunr.ja.js │ │ │ │ ├── lunr.stemmer.support.js │ │ │ │ ├── lunr.sv.js │ │ │ │ ├── lunr.da.js │ │ │ │ ├── lunr.no.js │ │ │ │ ├── lunr.nl.js │ │ │ │ ├── lunr.de.js │ │ │ │ ├── lunr.du.js │ │ │ │ ├── lunr.ru.js │ │ │ │ ├── lunr.fi.js │ │ │ │ ├── lunr.hu.js │ │ │ │ ├── lunr.pt.js │ │ │ │ ├── lunr.fr.js │ │ │ │ ├── lunr.ro.js │ │ │ │ ├── lunr.it.js │ │ │ │ ├── lunr.es.js │ │ │ │ ├── lunr.tr.js │ │ │ │ └── tinyseg.js │ │ │ └── modernizr.1f0bcf2b.js │ │ ├── images │ │ │ ├── favicon.png │ │ │ └── icons │ │ │ │ ├── github.f0b8504a.svg │ │ │ │ ├── bitbucket.1b09e088.svg │ │ │ │ └── gitlab.6dd19c00.svg │ │ └── fonts │ │ │ ├── specimen │ │ │ ├── FontAwesome.ttf │ │ │ ├── FontAwesome.woff │ │ │ ├── FontAwesome.woff2 │ │ │ ├── MaterialIcons-Regular.ttf │ │ │ ├── MaterialIcons-Regular.woff │ │ │ └── MaterialIcons-Regular.woff2 │ │ │ └── material-icons.css │ ├── sitemap.xml.gz │ ├── helper │ │ └── img │ │ │ ├── 01.png │ │ │ ├── 02.png │ │ │ ├── 03.png │ │ │ ├── 04.png │ │ │ ├── 05.png │ │ │ ├── 06.png │ │ │ ├── 07.png │ │ │ └── mechanism.png │ ├── images │ │ ├── favicon.png │ │ └── logo.svg │ ├── sitemap.xml │ └── 404.html └── mkdocs.yml ├── README.md └── LICENSE /nlpdocument/docs/baselines/aspect.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /nlpdocument/docs/baselines/dialogue.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /nlpdocument/docs/baselines/captioning.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /nlpdocument/docs/baselines/summarization.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /nlpdocument/docs/baselines/answer_selection.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.jp.js: -------------------------------------------------------------------------------- 1 | module.exports=require("./lunr.ja"); -------------------------------------------------------------------------------- /nlpdocument/docs/tutorial.md: -------------------------------------------------------------------------------- 1 | # Useful Tutorials 2 | under construction... 3 | 4 | 5 | 6 | -------------------------------------------------------------------------------- /nlpdocument/site/sitemap.xml.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/sitemap.xml.gz -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/01.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/02.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/03.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/04.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/05.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/05.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/06.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/06.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/07.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/07.png -------------------------------------------------------------------------------- /nlpdocument/docs/images/favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/images/favicon.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/01.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/02.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/03.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/04.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/05.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/05.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/06.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/06.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/07.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/07.png -------------------------------------------------------------------------------- /nlpdocument/site/images/favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/images/favicon.png -------------------------------------------------------------------------------- /nlpdocument/docs/helper/img/mechanism.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/docs/helper/img/mechanism.png -------------------------------------------------------------------------------- /nlpdocument/site/assets/images/favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/images/favicon.png -------------------------------------------------------------------------------- /nlpdocument/site/helper/img/mechanism.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/helper/img/mechanism.png -------------------------------------------------------------------------------- /nlpdocument/site/assets/fonts/specimen/FontAwesome.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/fonts/specimen/FontAwesome.ttf -------------------------------------------------------------------------------- /nlpdocument/site/assets/fonts/specimen/FontAwesome.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/fonts/specimen/FontAwesome.woff -------------------------------------------------------------------------------- /nlpdocument/site/assets/fonts/specimen/FontAwesome.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/fonts/specimen/FontAwesome.woff2 -------------------------------------------------------------------------------- /nlpdocument/site/assets/fonts/specimen/MaterialIcons-Regular.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/fonts/specimen/MaterialIcons-Regular.ttf -------------------------------------------------------------------------------- /nlpdocument/site/assets/fonts/specimen/MaterialIcons-Regular.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/fonts/specimen/MaterialIcons-Regular.woff -------------------------------------------------------------------------------- /nlpdocument/site/assets/fonts/specimen/MaterialIcons-Regular.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/siat-nlp/NLP-docs/HEAD/nlpdocument/site/assets/fonts/specimen/MaterialIcons-Regular.woff2 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # NLP-docs 2 | This repository contains all the documentation in natural language processing (NLP) / deep learning / machine learning, etc. Please visit https://siat-nlp.github.io/docs/ to read the docs. 3 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.multi.js: -------------------------------------------------------------------------------- 1 | !function(e,i){"function"==typeof define&&define.amd?define(i):"object"==typeof exports?module.exports=i():i()(e.lunr)}(this,function(){return function(o){o.multiLanguage=function(){for(var e=Array.prototype.slice.call(arguments),i=e.join("-"),t="",r=[],n=[],s=0;s 2 | 3 | <desc/> 4 | 5 | <g> 6 | <title>background 7 | 8 | 9 | 10 | Layer 1 11 | 12 | 13 | -------------------------------------------------------------------------------- /nlpdocument/site/images/logo.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | background 7 | 8 | 9 | 10 | Layer 1 11 | 12 | 13 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 SIAT-NLP 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/images/icons/github.f0b8504a.svg: -------------------------------------------------------------------------------- 1 | 3 | 18 | 19 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/images/icons/bitbucket.1b09e088.svg: -------------------------------------------------------------------------------- 1 | 3 | 20 | 21 | -------------------------------------------------------------------------------- /nlpdocument/site/sitemap.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | None 5 | 2019-05-06 6 | daily 7 | 8 | 9 | None 10 | 2019-05-06 11 | daily 12 | 13 | 14 | None 15 | 2019-05-06 16 | daily 17 | 18 | 19 | None 20 | 2019-05-06 21 | daily 22 | 23 | 24 | None 25 | 2019-05-06 26 | daily 27 | 28 | 29 | None 30 | 2019-05-06 31 | daily 32 | 33 | 34 | None 35 | 2019-05-06 36 | daily 37 | 38 | 39 | None 40 | 2019-05-06 41 | daily 42 | 43 | 44 | None 45 | 2019-05-06 46 | daily 47 | 48 | 49 | None 50 | 2019-05-06 51 | daily 52 | 53 | 54 | None 55 | 2019-05-06 56 | daily 57 | 58 | 59 | None 60 | 2019-05-06 61 | daily 62 | 63 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/images/icons/gitlab.6dd19c00.svg: -------------------------------------------------------------------------------- 1 | 3 | 4 | 7 | 8 | 9 | 12 | 13 | 14 | 17 | 18 | 19 | 22 | 23 | 24 | 27 | 28 | 29 | 32 | 33 | 34 | 37 | 38 | 39 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.ja.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(m){if(void 0===m)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===m.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var l="2"==m.version[0];m.ja=function(){this.pipeline.reset(),this.pipeline.add(m.ja.trimmer,m.ja.stopWordFilter,m.ja.stemmer),l?this.tokenizer=m.ja.tokenizer:(m.tokenizer&&(m.tokenizer=m.ja.tokenizer),this.tokenizerFn&&(this.tokenizerFn=m.ja.tokenizer))};var j=new m.TinySegmenter;m.ja.tokenizer=function(e){var r,t,i,n,o,s,p,a,u;if(!arguments.length||null==e||null==e)return[];if(Array.isArray(e))return e.map(function(e){return l?new m.Token(e.toLowerCase()):e.toLowerCase()});for(r=(t=e.toString().toLowerCase().replace(/^\s+/,"")).length-1;0<=r;r--)if(/\S/.test(t.charAt(r))){t=t.substring(0,r+1);break}for(o=[],i=t.length,p=a=0;a<=i;a++)if(s=a-p,t.charAt(a).match(/\s/)||a==i){if(0 items[j + 1]: 28 | items[j], items[j + 1] = items[j + 1], items[j] 29 | ``` 30 | ```` 31 | 效果： 32 | ``` python hl_lines="3 4" 33 | """ Bubble sort """ 34 | def bubble_sort(items): 35 | for i in range(len(items)): 36 | for j in range(len(items) - 1 - i): 37 | if items[j] > items[j + 1]: 38 | items[j], items[j + 1] = items[j + 1], items[j] 39 | ``` 40 | 更多有关代码的使用细节可以参考[CodeHilite](https://squidfunk.github.io/mkdocs-material/extensions/codehilite/) 41 | 42 | ## 表格 43 | ```markdown 44 | dog | bird | cat 45 | ----|------|---- 46 | foo | foo | foo 47 | bar | bar | bar 48 | baz | baz | baz 49 | ``` 50 | 效果： 51 | 52 | dog | bird | cat 53 | ----|------|---- 54 | foo | foo | foo 55 | bar | bar | bar 56 | baz | baz | baz 57 | 58 | ## 数学公式 59 | 文档支持使用Latex进行数学公式的编辑。 60 | 61 | ```markdown 62 | $$ 63 | \frac{n!}{k!(n-k)!} = \binom{n}{k} 64 | $$ 65 | ``` 66 | 效果： 67 | 68 | $$ 69 | \frac{n!}{k!(n-k)!} = \binom{n}{k} 70 | $$ 71 | 72 | ## Admonition 73 | 为了更加突出的展示显著信息，你可以使用Admonition，我们规定了两种比较常用的形式；分别是“Why”：用来描述/解释实验/代码的现象、“Ref”：用来介绍你所引用的参考文献。它们的实现方式如下所示。 74 | ```markdown 75 | !!! info "Ref" 76 | [Massive Exploration of Neural Machine Translation Architectures](https://arxiv.org/abs/1703.03906), Denny Britz, Anna Goldie et al. 77 | ``` 78 | 效果： 79 | 80 | !!! info "Ref" 81 | [Massive Exploration of Neural Machine Translation Architectures](https://arxiv.org/abs/1703.03906), Denny Britz, Anna Goldie et al. 82 | 83 | ```markdown 84 | !!! question "Why" 85 | From the authors: "*This way, [...] that makes it easy for SGD to “establish communication” between the input and the output. We found this simple data transformation to greatly improve the performance of the LSTM.*" 86 | ``` 87 | 效果： 88 | 89 | !!! question "Why" 90 | From the authors: "*This way, [...] that makes it easy for SGD to “establish communication” between the input and the output. We found this simple data transformation to greatly improve the performance of the LSTM.*" 91 | 92 | 更多有关Admonition的使用细节可以参考[Admonition](https://squidfunk.github.io/mkdocs-material/extensions/admonition/) 93 | 94 | -------------------------------------------------------------------------------- /nlpdocument/docs/helper/writing_paradigm.md: -------------------------------------------------------------------------------- 1 | # 文档编写模板 2 | 3 | ```markdown 4 | ## Network architecture 5 | ### Seq2Seq 6 | Some tricks to train RNN and seq2seq models: 7 | 8 | * Embedding size: 1024 or 512. Lower dimensionality like 256 can also lead to good performances. Higher does not necessarily lead to better performances. 9 | * For the decoder: LSTM > GRU > Vanilla-RNN 10 | * 2-4 layers seems generally enough. Deeper models with residual connections seems more difficult to converge (high variance). More tricks needs to be discovered. 11 | * ResD (dense residual connections) > Res (only connected to previous layer) > no residual connections 12 | * For encoder: Bidirectional > Unidirectional (reversed input) > Unidirectional 13 | * Attention (additive) > Attention (multiplicative) > No attention. Authors suggest that attention act more as a skip connection mechanism than as a memory for the decoder. 14 | 15 | !!! info "Ref" 16 | [Massive Exploration of Neural Machine Translation Architectures](https://arxiv.org/abs/1703.03906), Denny Britz, Anna Goldie et al. 17 | 18 | For seq2seq, reverse the order of the input sequence (\['I', 'am', 'hungry'\] becomes \['hungry', 'am', 'I'\]). Keep the target sequence intact. 19 | 20 | !!! question "Why" 21 | From the authors: "*This way, [...] that makes it easy for SGD to “establish communication” between the input and the output. We found this simple data transformation to greatly improve the performance of the LSTM.*" 22 | 23 | !!! info "Ref" 24 | [Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215), Ilya Sutskever et al. 25 | ``` 26 | 效果： 27 | 28 | ## Network architecture 29 | ### Seq2Seq 30 | Some tricks to train RNN and seq2seq models: 31 | 32 | * Embedding size: 1024 or 512. Lower dimensionality like 256 can also lead to good performances. Higher does not necessarily lead to better performances. 33 | * For the decoder: LSTM > GRU > Vanilla-RNN 34 | * 2-4 layers seems generally enough. Deeper models with residual connections seems more difficult to converge (high variance). More tricks needs to be discovered. 35 | * ResD (dense residual connections) > Res (only connected to previous layer) > no residual connections 36 | * For encoder: Bidirectional > Unidirectional (reversed input) > Unidirectional 37 | * Attention (additive) > Attention (multiplicative) > No attention. Authors suggest that attention act more as a skip connection mechanism than as a memory for the decoder. 38 | 39 | !!! info "Ref" 40 | [Massive Exploration of Neural Machine Translation Architectures](https://arxiv.org/abs/1703.03906), Denny Britz, Anna Goldie et al. 41 | 42 | For seq2seq, reverse the order of the input sequence (\['I', 'am', 'hungry'\] becomes \['hungry', 'am', 'I'\]). Keep the target sequence intact. 43 | 44 | !!! question "Why" 45 | From the authors: "*This way, [...] that makes it easy for SGD to “establish communication” between the input and the output. We found this simple data transformation to greatly improve the performance of the LSTM.*" 46 | 47 | !!! info "Ref" 48 | [Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215), Ilya Sutskever et al. 49 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.stemmer.support.js: -------------------------------------------------------------------------------- 1 | !function(r,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(r.lunr)}(this,function(){return function(r){r.stemmerSupport={Among:function(r,t,i,s){if(this.toCharArray=function(r){for(var t=r.length,i=new Array(t),s=0;s>3]&1<<(7&s))return this.cursor++,!0}return!1},in_grouping_b:function(r,t,i){if(this.cursor>this.limit_backward){var s=b.charCodeAt(this.cursor-1);if(s<=i&&t<=s&&r[(s-=t)>>3]&1<<(7&s))return this.cursor--,!0}return!1},out_grouping:function(r,t,i){if(this.cursor>3]&1<<(7&s)))return this.cursor++,!0}return!1},out_grouping_b:function(r,t,i){if(this.cursor>this.limit_backward){var s=b.charCodeAt(this.cursor-1);if(i>3]&1<<(7&s)))return this.cursor--,!0}return!1},eq_s:function(r,t){if(this.limit-this.cursor>1),a=0,f=u=(l=r[i]).s_size){if(this.cursor=e+l.s_size,!l.method)return l.result;var m=l.method();if(this.cursor=e+l.s_size,m)return l.result}if((i=l.substring_i)<0)return 0}},find_among_b:function(r,t){for(var i=0,s=t,e=this.cursor,n=this.limit_backward,u=0,o=0,h=!1;;){for(var c=i+(s-i>>1),a=0,f=u=(_=r[i]).s_size){if(this.cursor=e-_.s_size,!_.method)return _.result;var m=_.method();if(this.cursor=e-_.s_size,m)return _.result}if((i=_.substring_i)<0)return 0}},replace_s:function(r,t,i){var s=i.length-(t-r);return b=b.substring(0,r)+i+b.substring(t),this.limit+=s,this.cursor>=t?this.cursor+=s:this.cursor>r&&(this.cursor=r),s},slice_check:function(){if(this.bra<0||this.bra>this.ket||this.ket>this.limit||this.limit>b.length)throw"faulty slice operation"},slice_from:function(r){this.slice_check(),this.replace_s(this.bra,this.ket,r)},slice_del:function(){this.slice_from("")},insert:function(r,t,i){var s=this.replace_s(r,t,i);r<=this.bra&&(this.bra+=s),r<=this.ket&&(this.ket+=s)},slice_to:function(){return this.slice_check(),b.substring(this.bra,this.ket)},eq_v_b:function(r){return this.eq_s_b(r.length,r)}}}},r.trimmerSupport={generateTrimmer:function(r){var t=new RegExp("^[^"+r+"]+"),i=new RegExp("[^"+r+"]+$");return function(r){return"function"==typeof r.update?r.update(function(r){return r.replace(t,"").replace(i,"")}):r.replace(t,"").replace(i,"")}}}}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.sv.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,l,n;e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=(r=e.stemmerSupport.Among,l=e.stemmerSupport.SnowballProgram,n=new function(){var n,t,i=[new r("a",-1,1),new r("arna",0,1),new r("erna",0,1),new r("heterna",2,1),new r("orna",0,1),new r("ad",-1,1),new r("e",-1,1),new r("ade",6,1),new r("ande",6,1),new r("arne",6,1),new r("are",6,1),new r("aste",6,1),new r("en",-1,1),new r("anden",12,1),new r("aren",12,1),new r("heten",12,1),new r("ern",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",18,1),new r("or",-1,1),new r("s",-1,2),new r("as",21,1),new r("arnas",22,1),new r("ernas",22,1),new r("ornas",22,1),new r("es",21,1),new r("ades",26,1),new r("andes",26,1),new r("ens",21,1),new r("arens",29,1),new r("hetens",29,1),new r("erns",21,1),new r("at",-1,1),new r("andet",-1,1),new r("het",-1,1),new r("ast",-1,1)],s=[new r("dd",-1,-1),new r("gd",-1,-1),new r("nn",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1),new r("tt",-1,-1)],a=[new r("ig",-1,1),new r("lig",0,1),new r("els",-1,1),new r("fullt",-1,3),new r("löst",-1,2)],o=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],u=[119,127,149],m=new l;this.setCurrent=function(e){m.setCurrent(e)},this.getCurrent=function(){return m.getCurrent()},this.stem=function(){var e,r=m.cursor;return function(){var e,r=m.cursor+3;if(t=m.limit,0<=r||r<=m.limit){for(n=r;;){if(e=m.cursor,m.in_grouping(o,97,246)){m.cursor=e;break}if(m.cursor=e,m.cursor>=m.limit)return;m.cursor++}for(;!m.out_grouping(o,97,246);){if(m.cursor>=m.limit)return;m.cursor++}(t=m.cursor)=t&&(m.limit_backward=t,m.cursor=m.limit,m.ket=m.cursor,e=m.find_among_b(i,37),m.limit_backward=r,e))switch(m.bra=m.cursor,e){case 1:m.slice_del();break;case 2:m.in_grouping_b(u,98,121)&&m.slice_del()}}(),m.cursor=m.limit,e=m.limit_backward,m.cursor>=t&&(m.limit_backward=t,m.cursor=m.limit,m.find_among_b(s,7)&&(m.cursor=m.limit,m.ket=m.cursor,m.cursor>m.limit_backward&&(m.bra=--m.cursor,m.slice_del())),m.limit_backward=e),m.cursor=m.limit,function(){var e,r;if(m.cursor>=t){if(r=m.limit_backward,m.limit_backward=t,m.cursor=m.limit,m.ket=m.cursor,e=m.find_among_b(a,5))switch(m.bra=m.cursor,e){case 1:m.slice_del();break;case 2:m.slice_from("lös");break;case 3:m.slice_from("full")}m.limit_backward=r}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.da.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,m,i;e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=(r=e.stemmerSupport.Among,m=e.stemmerSupport.SnowballProgram,i=new function(){var i,t,n,s=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],o=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],a=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],u=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],c=new m;function l(){var e,r=c.limit-c.cursor;c.cursor>=t&&(e=c.limit_backward,c.limit_backward=t,c.ket=c.cursor,c.find_among_b(o,4)?(c.bra=c.cursor,c.limit_backward=e,c.cursor=c.limit-r,c.cursor>c.limit_backward&&(c.cursor--,c.bra=c.cursor,c.slice_del())):c.limit_backward=e)}this.setCurrent=function(e){c.setCurrent(e)},this.getCurrent=function(){return c.getCurrent()},this.stem=function(){var e,r=c.cursor;return function(){var e,r=c.cursor+3;if(t=c.limit,0<=r&&r<=c.limit){for(i=r;;){if(e=c.cursor,c.in_grouping(d,97,248)){c.cursor=e;break}if((c.cursor=e)>=c.limit)return;c.cursor++}for(;!c.out_grouping(d,97,248);){if(c.cursor>=c.limit)return;c.cursor++}(t=c.cursor)=t&&(r=c.limit_backward,c.limit_backward=t,c.ket=c.cursor,e=c.find_among_b(s,32),c.limit_backward=r,e))switch(c.bra=c.cursor,e){case 1:c.slice_del();break;case 2:c.in_grouping_b(u,97,229)&&c.slice_del()}}(),c.cursor=c.limit,l(),c.cursor=c.limit,function(){var e,r,i,n=c.limit-c.cursor;if(c.ket=c.cursor,c.eq_s_b(2,"st")&&(c.bra=c.cursor,c.eq_s_b(2,"ig")&&c.slice_del()),c.cursor=c.limit-n,c.cursor>=t&&(r=c.limit_backward,c.limit_backward=t,c.ket=c.cursor,e=c.find_among_b(a,5),c.limit_backward=r,e))switch(c.bra=c.cursor,e){case 1:c.slice_del(),i=c.limit-c.cursor,l(),c.cursor=c.limit-i;break;case 2:c.slice_from("løs")}}(),c.cursor=c.limit,c.cursor>=t&&(e=c.limit_backward,c.limit_backward=t,c.ket=c.cursor,c.out_grouping_b(d,97,248)?(c.bra=c.cursor,n=c.slice_to(n),c.limit_backward=e,c.eq_v_b(n)&&c.slice_del()):c.limit_backward=e),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.no.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,n,i;e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=(r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){var o,s,a=[new r("a",-1,1),new r("e",-1,1),new r("ede",1,1),new r("ande",1,1),new r("ende",1,1),new r("ane",1,1),new r("ene",1,1),new r("hetene",6,1),new r("erte",1,3),new r("en",-1,1),new r("heten",9,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",12,1),new r("s",-1,2),new r("as",14,1),new r("es",14,1),new r("edes",16,1),new r("endes",16,1),new r("enes",16,1),new r("hetenes",19,1),new r("ens",14,1),new r("hetens",21,1),new r("ers",14,1),new r("ets",14,1),new r("et",-1,1),new r("het",25,1),new r("ert",-1,3),new r("ast",-1,1)],m=[new r("dt",-1,-1),new r("vt",-1,-1)],l=[new r("leg",-1,1),new r("eleg",0,1),new r("ig",-1,1),new r("eig",2,1),new r("lig",2,1),new r("elig",4,1),new r("els",-1,1),new r("lov",-1,1),new r("elov",7,1),new r("slov",7,1),new r("hetslov",9,1)],u=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],d=[119,125,149,1],c=new n;this.setCurrent=function(e){c.setCurrent(e)},this.getCurrent=function(){return c.getCurrent()},this.stem=function(){var e,r,n,i,t=c.cursor;return function(){var e,r=c.cursor+3;if(s=c.limit,0<=r||r<=c.limit){for(o=r;;){if(e=c.cursor,c.in_grouping(u,97,248)){c.cursor=e;break}if(e>=c.limit)return;c.cursor=e+1}for(;!c.out_grouping(u,97,248);){if(c.cursor>=c.limit)return;c.cursor++}(s=c.cursor)=s&&(r=c.limit_backward,c.limit_backward=s,c.ket=c.cursor,e=c.find_among_b(a,29),c.limit_backward=r,e))switch(c.bra=c.cursor,e){case 1:c.slice_del();break;case 2:n=c.limit-c.cursor,c.in_grouping_b(d,98,122)?c.slice_del():(c.cursor=c.limit-n,c.eq_s_b(1,"k")&&c.out_grouping_b(u,97,248)&&c.slice_del());break;case 3:c.slice_from("er")}}(),c.cursor=c.limit,r=c.limit-c.cursor,c.cursor>=s&&(e=c.limit_backward,c.limit_backward=s,c.ket=c.cursor,c.find_among_b(m,2)?(c.bra=c.cursor,c.limit_backward=e,c.cursor=c.limit-r,c.cursor>c.limit_backward&&(c.cursor--,c.bra=c.cursor,c.slice_del())):c.limit_backward=e),c.cursor=c.limit,c.cursor>=s&&(i=c.limit_backward,c.limit_backward=s,c.ket=c.cursor,(n=c.find_among_b(l,11))?(c.bra=c.cursor,c.limit_backward=i,1==n&&c.slice_del()):c.limit_backward=i),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.nl.js: -------------------------------------------------------------------------------- 1 | !function(r,e){"function"==typeof define&&define.amd?define(e):"object"==typeof exports?module.exports=e():e()(r.lunr)}(this,function(){return function(r){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var v,q,e;r.nl=function(){this.pipeline.reset(),this.pipeline.add(r.nl.trimmer,r.nl.stopWordFilter,r.nl.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(r.nl.stemmer))},r.nl.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",r.nl.trimmer=r.trimmerSupport.generateTrimmer(r.nl.wordCharacters),r.Pipeline.registerFunction(r.nl.trimmer,"trimmer-nl"),r.nl.stemmer=(v=r.stemmerSupport.Among,q=r.stemmerSupport.SnowballProgram,e=new function(){var e,i,u,o=[new v("",-1,6),new v("á",0,1),new v("ä",0,1),new v("é",0,2),new v("ë",0,2),new v("í",0,3),new v("ï",0,3),new v("ó",0,4),new v("ö",0,4),new v("ú",0,5),new v("ü",0,5)],n=[new v("",-1,3),new v("I",0,2),new v("Y",0,1)],t=[new v("dd",-1,-1),new v("kk",-1,-1),new v("tt",-1,-1)],c=[new v("ene",-1,2),new v("se",-1,3),new v("en",-1,2),new v("heden",2,1),new v("s",-1,3)],a=[new v("end",-1,1),new v("ig",-1,2),new v("ing",-1,1),new v("lijk",-1,3),new v("baar",-1,4),new v("bar",-1,5)],l=[new v("aa",-1,-1),new v("ee",-1,-1),new v("oo",-1,-1),new v("uu",-1,-1)],m=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],d=[1,0,0,17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],f=[17,67,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],_=new q;function s(r){return(_.cursor=r)>=_.limit||(_.cursor++,!1)}function w(){for(;!_.in_grouping(m,97,232);){if(_.cursor>=_.limit)return!0;_.cursor++}for(;!_.out_grouping(m,97,232);){if(_.cursor>=_.limit)return!0;_.cursor++}return!1}function b(){return i<=_.cursor}function p(){return e<=_.cursor}function g(){var r=_.limit-_.cursor;_.find_among_b(t,3)&&(_.cursor=_.limit-r,_.ket=_.cursor,_.cursor>_.limit_backward&&(_.cursor--,_.bra=_.cursor,_.slice_del()))}function h(){var r;u=!1,_.ket=_.cursor,_.eq_s_b(1,"e")&&(_.bra=_.cursor,b()&&(r=_.limit-_.cursor,_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-r,_.slice_del(),u=!0,g())))}function k(){var r;b()&&(r=_.limit-_.cursor,_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-r,_.eq_s_b(3,"gem")||(_.cursor=_.limit-r,_.slice_del(),g())))}this.setCurrent=function(r){_.setCurrent(r)},this.getCurrent=function(){return _.getCurrent()},this.stem=function(){var r=_.cursor;return function(){for(var r,e,i,n=_.cursor;;){if(_.bra=_.cursor,r=_.find_among(o,11))switch(_.ket=_.cursor,r){case 1:_.slice_from("a");continue;case 2:_.slice_from("e");continue;case 3:_.slice_from("i");continue;case 4:_.slice_from("o");continue;case 5:_.slice_from("u");continue;case 6:if(_.cursor>=_.limit)break;_.cursor++;continue}break}for(_.cursor=n,_.bra=n,_.eq_s(1,"y")?(_.ket=_.cursor,_.slice_from("Y")):_.cursor=n;;)if(e=_.cursor,_.in_grouping(m,97,232)){if(i=_.cursor,_.bra=i,_.eq_s(1,"i"))_.ket=_.cursor,_.in_grouping(m,97,232)&&(_.slice_from("I"),_.cursor=e);else if(_.cursor=i,_.eq_s(1,"y"))_.ket=_.cursor,_.slice_from("Y"),_.cursor=e;else if(s(e))break}else if(s(e))break}(),_.cursor=r,i=_.limit,e=i,w()||((i=_.cursor)<3&&(i=3),w()||(e=_.cursor)),_.limit_backward=r,_.cursor=_.limit,function(){var r,e,i,n,o,t,s=_.limit-_.cursor;if(_.ket=_.cursor,r=_.find_among_b(c,5))switch(_.bra=_.cursor,r){case 1:b()&&_.slice_from("heid");break;case 2:k();break;case 3:b()&&_.out_grouping_b(f,97,232)&&_.slice_del()}if(_.cursor=_.limit-s,h(),_.cursor=_.limit-s,_.ket=_.cursor,_.eq_s_b(4,"heid")&&(_.bra=_.cursor,p()&&(e=_.limit-_.cursor,_.eq_s_b(1,"c")||(_.cursor=_.limit-e,_.slice_del(),_.ket=_.cursor,_.eq_s_b(2,"en")&&(_.bra=_.cursor,k())))),_.cursor=_.limit-s,_.ket=_.cursor,r=_.find_among_b(a,6))switch(_.bra=_.cursor,r){case 1:if(p()){if(_.slice_del(),i=_.limit-_.cursor,_.ket=_.cursor,_.eq_s_b(2,"ig")&&(_.bra=_.cursor,p()&&(n=_.limit-_.cursor,!_.eq_s_b(1,"e")))){_.cursor=_.limit-n,_.slice_del();break}_.cursor=_.limit-i,g()}break;case 2:p()&&(o=_.limit-_.cursor,_.eq_s_b(1,"e")||(_.cursor=_.limit-o,_.slice_del()));break;case 3:p()&&(_.slice_del(),h());break;case 4:p()&&_.slice_del();break;case 5:p()&&u&&_.slice_del()}_.cursor=_.limit-s,_.out_grouping_b(d,73,232)&&(t=_.limit-_.cursor,_.find_among_b(l,4)&&_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-t,_.ket=_.cursor,_.cursor>_.limit_backward&&(_.cursor--,_.bra=_.cursor,_.slice_del())))}(),_.cursor=_.limit_backward,function(){for(var r;;)if(_.bra=_.cursor,r=_.find_among(n,3))switch(_.ket=_.cursor,r){case 1:_.slice_from("y");break;case 2:_.slice_from("i");break;case 3:if(_.cursor>=_.limit)return;_.cursor++}}(),!0}},function(r){return"function"==typeof r.update?r.update(function(r){return e.setCurrent(r),e.stem(),e.getCurrent()}):(e.setCurrent(r),e.stem(),e.getCurrent())}),r.Pipeline.registerFunction(r.nl.stemmer,"stemmer-nl"),r.nl.stopWordFilter=r.generateStopWordFilter(" aan al alles als altijd andere ben bij daar dan dat de der deze die dit doch doen door dus een eens en er ge geen geweest haar had heb hebben heeft hem het hier hij hoe hun iemand iets ik in is ja je kan kon kunnen maar me meer men met mij mijn moet na naar niet niets nog nu of om omdat onder ons ook op over reeds te tegen toch toen tot u uit uw van veel voor want waren was wat werd wezen wie wil worden wordt zal ze zelf zich zij zijn zo zonder zou".split(" ")),r.Pipeline.registerFunction(r.nl.stopWordFilter,"stopWordFilter-nl")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.de.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var _,p,r;e.de=function(){this.pipeline.reset(),this.pipeline.add(e.de.trimmer,e.de.stopWordFilter,e.de.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.de.stemmer))},e.de.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.de.trimmer=e.trimmerSupport.generateTrimmer(e.de.wordCharacters),e.Pipeline.registerFunction(e.de.trimmer,"trimmer-de"),e.de.stemmer=(_=e.stemmerSupport.Among,p=e.stemmerSupport.SnowballProgram,r=new function(){var r,n,i,s=[new _("",-1,6),new _("U",0,2),new _("Y",0,1),new _("ä",0,3),new _("ö",0,4),new _("ü",0,5)],o=[new _("e",-1,2),new _("em",-1,1),new _("en",-1,2),new _("ern",-1,1),new _("er",-1,1),new _("s",-1,3),new _("es",5,2)],c=[new _("en",-1,1),new _("er",-1,1),new _("st",-1,2),new _("est",2,1)],u=[new _("ig",-1,1),new _("lich",-1,1)],a=[new _("end",-1,1),new _("ig",-1,2),new _("ung",-1,1),new _("lich",-1,3),new _("isch",-1,2),new _("ik",-1,2),new _("heit",-1,3),new _("keit",-1,4)],t=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,8,0,32,8],d=[117,30,5],l=[117,30,4],m=new p;function h(e,r,n){return!(!m.eq_s(1,e)||(m.ket=m.cursor,!m.in_grouping(t,97,252)))&&(m.slice_from(r),m.cursor=n,!0)}function w(){for(;!m.in_grouping(t,97,252);){if(m.cursor>=m.limit)return!0;m.cursor++}for(;!m.out_grouping(t,97,252);){if(m.cursor>=m.limit)return!0;m.cursor++}return!1}function f(){return i<=m.cursor}function b(){return n<=m.cursor}this.setCurrent=function(e){m.setCurrent(e)},this.getCurrent=function(){return m.getCurrent()},this.stem=function(){var e=m.cursor;return function(){for(var e,r,n,i,s=m.cursor;;)if(e=m.cursor,m.bra=e,m.eq_s(1,"ß"))m.ket=m.cursor,m.slice_from("ss");else{if(e>=m.limit)break;m.cursor=e+1}for(m.cursor=s;;)for(r=m.cursor;;){if(n=m.cursor,m.in_grouping(t,97,252)){if(i=m.cursor,m.bra=i,h("u","U",n))break;if(m.cursor=i,h("y","Y",n))break}if(n>=m.limit)return m.cursor=r;m.cursor=n+1}}(),m.cursor=e,function(){i=m.limit,n=i;var e=m.cursor+3;0<=e&&e<=m.limit&&(r=e,w()||((i=m.cursor)=m.limit)return;m.cursor++}}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return r.setCurrent(e),r.stem(),r.getCurrent()}):(r.setCurrent(e),r.stem(),r.getCurrent())}),e.Pipeline.registerFunction(e.de.stemmer,"stemmer-de"),e.de.stopWordFilter=e.generateStopWordFilter("aber alle allem allen aller alles als also am an ander andere anderem anderen anderer anderes anderm andern anderr anders auch auf aus bei bin bis bist da damit dann das dasselbe dazu daß dein deine deinem deinen deiner deines dem demselben den denn denselben der derer derselbe derselben des desselben dessen dich die dies diese dieselbe dieselben diesem diesen dieser dieses dir doch dort du durch ein eine einem einen einer eines einig einige einigem einigen einiger einiges einmal er es etwas euch euer eure eurem euren eurer eures für gegen gewesen hab habe haben hat hatte hatten hier hin hinter ich ihm ihn ihnen ihr ihre ihrem ihren ihrer ihres im in indem ins ist jede jedem jeden jeder jedes jene jenem jenen jener jenes jetzt kann kein keine keinem keinen keiner keines können könnte machen man manche manchem manchen mancher manches mein meine meinem meinen meiner meines mich mir mit muss musste nach nicht nichts noch nun nur ob oder ohne sehr sein seine seinem seinen seiner seines selbst sich sie sind so solche solchem solchen solcher solches soll sollte sondern sonst um und uns unse unsem unsen unser unses unter viel vom von vor war waren warst was weg weil weiter welche welchem welchen welcher welches wenn werde werden wie wieder will wir wird wirst wo wollen wollte während würde würden zu zum zur zwar zwischen über".split(" ")),e.Pipeline.registerFunction(e.de.stopWordFilter,"stopWordFilter-de")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.du.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var v,q,r;console.warn('[Lunr Languages] Please use the "nl" instead of the "du". The "nl" code is the standard code for Dutch language, and "du" will be removed in the next major versions.'),e.du=function(){this.pipeline.reset(),this.pipeline.add(e.du.trimmer,e.du.stopWordFilter,e.du.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.du.stemmer))},e.du.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.du.trimmer=e.trimmerSupport.generateTrimmer(e.du.wordCharacters),e.Pipeline.registerFunction(e.du.trimmer,"trimmer-du"),e.du.stemmer=(v=e.stemmerSupport.Among,q=e.stemmerSupport.SnowballProgram,r=new function(){var r,i,u,o=[new v("",-1,6),new v("á",0,1),new v("ä",0,1),new v("é",0,2),new v("ë",0,2),new v("í",0,3),new v("ï",0,3),new v("ó",0,4),new v("ö",0,4),new v("ú",0,5),new v("ü",0,5)],n=[new v("",-1,3),new v("I",0,2),new v("Y",0,1)],t=[new v("dd",-1,-1),new v("kk",-1,-1),new v("tt",-1,-1)],c=[new v("ene",-1,2),new v("se",-1,3),new v("en",-1,2),new v("heden",2,1),new v("s",-1,3)],a=[new v("end",-1,1),new v("ig",-1,2),new v("ing",-1,1),new v("lijk",-1,3),new v("baar",-1,4),new v("bar",-1,5)],l=[new v("aa",-1,-1),new v("ee",-1,-1),new v("oo",-1,-1),new v("uu",-1,-1)],m=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],d=[1,0,0,17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],f=[17,67,16,1,0,0,0,0,0,0,0,0,0,0,0,0,128],_=new q;function s(e){return(_.cursor=e)>=_.limit||(_.cursor++,!1)}function w(){for(;!_.in_grouping(m,97,232);){if(_.cursor>=_.limit)return!0;_.cursor++}for(;!_.out_grouping(m,97,232);){if(_.cursor>=_.limit)return!0;_.cursor++}return!1}function b(){return i<=_.cursor}function p(){return r<=_.cursor}function g(){var e=_.limit-_.cursor;_.find_among_b(t,3)&&(_.cursor=_.limit-e,_.ket=_.cursor,_.cursor>_.limit_backward&&(_.cursor--,_.bra=_.cursor,_.slice_del()))}function h(){var e;u=!1,_.ket=_.cursor,_.eq_s_b(1,"e")&&(_.bra=_.cursor,b()&&(e=_.limit-_.cursor,_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-e,_.slice_del(),u=!0,g())))}function k(){var e;b()&&(e=_.limit-_.cursor,_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-e,_.eq_s_b(3,"gem")||(_.cursor=_.limit-e,_.slice_del(),g())))}this.setCurrent=function(e){_.setCurrent(e)},this.getCurrent=function(){return _.getCurrent()},this.stem=function(){var e=_.cursor;return function(){for(var e,r,i,n=_.cursor;;){if(_.bra=_.cursor,e=_.find_among(o,11))switch(_.ket=_.cursor,e){case 1:_.slice_from("a");continue;case 2:_.slice_from("e");continue;case 3:_.slice_from("i");continue;case 4:_.slice_from("o");continue;case 5:_.slice_from("u");continue;case 6:if(_.cursor>=_.limit)break;_.cursor++;continue}break}for(_.cursor=n,_.bra=n,_.eq_s(1,"y")?(_.ket=_.cursor,_.slice_from("Y")):_.cursor=n;;)if(r=_.cursor,_.in_grouping(m,97,232)){if(i=_.cursor,_.bra=i,_.eq_s(1,"i"))_.ket=_.cursor,_.in_grouping(m,97,232)&&(_.slice_from("I"),_.cursor=r);else if(_.cursor=i,_.eq_s(1,"y"))_.ket=_.cursor,_.slice_from("Y"),_.cursor=r;else if(s(r))break}else if(s(r))break}(),_.cursor=e,i=_.limit,r=i,w()||((i=_.cursor)<3&&(i=3),w()||(r=_.cursor)),_.limit_backward=e,_.cursor=_.limit,function(){var e,r,i,n,o,t,s=_.limit-_.cursor;if(_.ket=_.cursor,e=_.find_among_b(c,5))switch(_.bra=_.cursor,e){case 1:b()&&_.slice_from("heid");break;case 2:k();break;case 3:b()&&_.out_grouping_b(f,97,232)&&_.slice_del()}if(_.cursor=_.limit-s,h(),_.cursor=_.limit-s,_.ket=_.cursor,_.eq_s_b(4,"heid")&&(_.bra=_.cursor,p()&&(r=_.limit-_.cursor,_.eq_s_b(1,"c")||(_.cursor=_.limit-r,_.slice_del(),_.ket=_.cursor,_.eq_s_b(2,"en")&&(_.bra=_.cursor,k())))),_.cursor=_.limit-s,_.ket=_.cursor,e=_.find_among_b(a,6))switch(_.bra=_.cursor,e){case 1:if(p()){if(_.slice_del(),i=_.limit-_.cursor,_.ket=_.cursor,_.eq_s_b(2,"ig")&&(_.bra=_.cursor,p()&&(n=_.limit-_.cursor,!_.eq_s_b(1,"e")))){_.cursor=_.limit-n,_.slice_del();break}_.cursor=_.limit-i,g()}break;case 2:p()&&(o=_.limit-_.cursor,_.eq_s_b(1,"e")||(_.cursor=_.limit-o,_.slice_del()));break;case 3:p()&&(_.slice_del(),h());break;case 4:p()&&_.slice_del();break;case 5:p()&&u&&_.slice_del()}_.cursor=_.limit-s,_.out_grouping_b(d,73,232)&&(t=_.limit-_.cursor,_.find_among_b(l,4)&&_.out_grouping_b(m,97,232)&&(_.cursor=_.limit-t,_.ket=_.cursor,_.cursor>_.limit_backward&&(_.cursor--,_.bra=_.cursor,_.slice_del())))}(),_.cursor=_.limit_backward,function(){for(var e;;)if(_.bra=_.cursor,e=_.find_among(n,3))switch(_.ket=_.cursor,e){case 1:_.slice_from("y");break;case 2:_.slice_from("i");break;case 3:if(_.cursor>=_.limit)return;_.cursor++}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return r.setCurrent(e),r.stem(),r.getCurrent()}):(r.setCurrent(e),r.stem(),r.getCurrent())}),e.Pipeline.registerFunction(e.du.stemmer,"stemmer-du"),e.du.stopWordFilter=e.generateStopWordFilter(" aan al alles als altijd andere ben bij daar dan dat de der deze die dit doch doen door dus een eens en er ge geen geweest haar had heb hebben heeft hem het hier hij hoe hun iemand iets ik in is ja je kan kon kunnen maar me meer men met mij mijn moet na naar niet niets nog nu of om omdat onder ons ook op over reeds te tegen toch toen tot u uit uw van veel voor want waren was wat werd wezen wie wil worden wordt zal ze zelf zich zij zijn zo zonder zou".split(" ")),e.Pipeline.registerFunction(e.du.stopWordFilter,"stopWordFilter-du")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/modernizr.1f0bcf2b.js: -------------------------------------------------------------------------------- 1 | !function(e,t){for(var n in t)e[n]=t[n]}(window,function(n){var r={};function o(e){if(r[e])return r[e].exports;var t=r[e]={i:e,l:!1,exports:{}};return n[e].call(t.exports,t,t.exports,o),t.l=!0,t.exports}return o.m=n,o.c=r,o.d=function(e,t,n){o.o(e,t)||Object.defineProperty(e,t,{enumerable:!0,get:n})},o.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},o.t=function(t,e){if(1&e&&(t=o(t)),8&e)return t;if(4&e&&"object"==typeof t&&t&&t.__esModule)return t;var n=Object.create(null);if(o.r(n),Object.defineProperty(n,"default",{enumerable:!0,value:t}),2&e&&"string"!=typeof t)for(var r in t)o.d(n,r,function(e){return t[e]}.bind(null,r));return n},o.n=function(e){var t=e&&e.__esModule?function(){return e.default}:function(){return e};return o.d(t,"a",t),t},o.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)},o.p="",o(o.s=12)}({12:function(e,t,n){"use strict";n.r(t);n(13)},13:function(e,t){var n;(function(i,d,p){function y(e,t){return typeof e===t}function s(e){var t=S.className,n=c._config.classPrefix||"";if(b&&(t=t.baseVal),c._config.enableJSClass){var r=new RegExp("(^|\\s)"+n+"no-js(\\s|$)");t=t.replace(r,"$1"+n+"js$2")}c._config.enableClasses&&(t+=" "+n+e.join(" "+n),b?S.className.baseVal=t:S.className=t)}function a(e,t){if("object"==typeof e)for(var n in e)v(e,n)&&a(n,e[n]);else{var r=(e=e.toLowerCase()).split("."),o=c[r[0]];if(2==r.length&&(o=o[r[1]]),void 0!==o)return c;t="function"==typeof t?t():t,1==r.length?c[r[0]]=t:(!c[r[0]]||c[r[0]]instanceof Boolean||(c[r[0]]=new Boolean(c[r[0]])),c[r[0]][r[1]]=t),s([(t&&0!=t?"":"no-")+r.join("-")]),c._trigger(e,t)}return c}function m(){return"function"!=typeof d.createElement?d.createElement(arguments[0]):b?d.createElementNS.call(d,"http://www.w3.org/2000/svg",arguments[0]):d.createElement.apply(d,arguments)}function o(e,t,n,r){var o,i,s,a,l,u="modernizr",f=m("div"),c=((l=d.body)||((l=m(b?"svg":"body")).fake=!0),l);if(parseInt(n,10))for(;n--;)(s=m("div")).id=r?r[n]:u+(n+1),f.appendChild(s);return(o=m("style")).type="text/css",o.id="s"+u,(c.fake?c:f).appendChild(o),c.appendChild(f),o.styleSheet?o.styleSheet.cssText=e:o.appendChild(d.createTextNode(e)),f.id=u,c.fake&&(c.style.background="",c.style.overflow="hidden",a=S.style.overflow,S.style.overflow="hidden",S.appendChild(c)),i=t(f,e),c.fake?(c.parentNode.removeChild(c),S.style.overflow=a,S.offsetHeight):f.parentNode.removeChild(f),!!i}function l(e){return e.replace(/([A-Z])/g,function(e,t){return"-"+t.toLowerCase()}).replace(/^ms-/,"-ms-")}function h(e,t){var n=e.length;if("CSS"in i&&"supports"in i.CSS){for(;n--;)if(i.CSS.supports(l(e[n]),t))return!0;return!1}if("CSSSupportsRule"in i){for(var r=[];n--;)r.push("("+l(e[n])+":"+t+")");return o("@supports ("+(r=r.join(" or "))+") { #modernizr { position: absolute; } }",function(e){return"absolute"==function(e,t,n){var r;if("getComputedStyle"in i){r=getComputedStyle.call(i,e,t);var o=i.console;null!==r?n&&(r=r.getPropertyValue(n)):o&&o[o.error?"error":"log"].call(o,"getComputedStyle returning null, its possible modernizr test results are inaccurate")}else r=!t&&e.currentStyle&&e.currentStyle[n];return r}(e,null,"position")})}return p}function u(e,t){return function(){return e.apply(t,arguments)}}function r(e,t,n,r,o){var i=e.charAt(0).toUpperCase()+e.slice(1),s=(e+" "+w.join(i+" ")+i).split(" ");return y(t,"string")||y(t,"undefined")?function(e,t,n,r){function o(){s&&(delete T.style,delete T.modElem)}if(r=!y(r,"undefined")&&r,!y(n,"undefined")){var i=h(e,n);if(!y(i,"undefined"))return i}for(var s,a,l,u,f,c=["modernizr","tspan","samp"];!T.style&&c.length;)s=!0,T.modElem=m(c.shift()),T.style=T.modElem.style;for(l=e.length,a=0;a=l.limit)return!1;l.cursor++}return!0}function a(){for(;!l.out_grouping(m,1072,1103);){if(l.cursor>=l.limit)return!1;l.cursor++}return!0}function p(e,n){var r,t;if(l.ket=l.cursor,r=l.find_among_b(e,n)){switch(l.bra=l.cursor,r){case 1:if(t=l.limit-l.cursor,!l.eq_s_b(1,"а")&&(l.cursor=l.limit-t,!l.eq_s_b(1,"я")))return!1;case 2:l.slice_del()}return!0}return!1}function d(e,n){var r;return l.ket=l.cursor,!!(r=l.find_among_b(e,n))&&(l.bra=l.cursor,1==r&&l.slice_del(),!0)}function _(){return!!d(t,26)&&(p(w,8),!0)}function b(){var e;l.ket=l.cursor,(e=l.find_among_b(o,2))&&(l.bra=l.cursor,n<=l.cursor&&1==e&&l.slice_del())}this.setCurrent=function(e){l.setCurrent(e)},this.getCurrent=function(){return l.getCurrent()},this.stem=function(){return e=l.limit,n=e,f()&&(e=l.cursor,a()&&f()&&a()&&(n=l.cursor)),l.cursor=l.limit,!(l.cursor=h.limit)return!0;h.cursor++}for(h.cursor=i;!h.out_grouping(d,97,246);){if(h.cursor>=h.limit)return!0;h.cursor++}return!1}function g(){var i,e;if(h.cursor>=o)if(e=h.limit_backward,h.limit_backward=o,h.ket=h.cursor,i=h.find_among_b(r,10)){switch(h.bra=h.cursor,h.limit_backward=e,i){case 1:if(!h.in_grouping_b(f,97,246))return;break;case 2:if(!(l<=h.cursor))return}h.slice_del()}else h.limit_backward=e}function j(){return h.find_among_b(i,7)}function q(){return h.eq_s_b(1,"i")&&h.in_grouping_b(e,97,246)}this.setCurrent=function(i){h.setCurrent(i)},this.getCurrent=function(){return h.getCurrent()},this.stem=function(){var i,e=h.cursor;return o=h.limit,l=o,p()||(o=h.cursor,p()||(l=h.cursor)),n=!1,h.limit_backward=e,h.cursor=h.limit,g(),h.cursor=h.limit,function(){var i,e,r;if(h.cursor>=o)if(e=h.limit_backward,h.limit_backward=o,h.ket=h.cursor,i=h.find_among_b(c,9))switch(h.bra=h.cursor,h.limit_backward=e,i){case 1:r=h.limit-h.cursor,h.eq_s_b(1,"k")||(h.cursor=h.limit-r,h.slice_del());break;case 2:h.slice_del(),h.ket=h.cursor,h.eq_s_b(3,"kse")&&(h.bra=h.cursor,h.slice_from("ksi"));break;case 3:h.slice_del();break;case 4:h.find_among_b(s,6)&&h.slice_del();break;case 5:h.find_among_b(a,6)&&h.slice_del();break;case 6:h.find_among_b(u,2)&&h.slice_del()}else h.limit_backward=e}(),h.cursor=h.limit,function(){var i,e,r;if(h.cursor>=o)if(e=h.limit_backward,h.limit_backward=o,h.ket=h.cursor,i=h.find_among_b(m,30)){switch(h.bra=h.cursor,h.limit_backward=e,i){case 1:if(!h.eq_s_b(1,"a"))return;break;case 2:case 9:if(!h.eq_s_b(1,"e"))return;break;case 3:if(!h.eq_s_b(1,"i"))return;break;case 4:if(!h.eq_s_b(1,"o"))return;break;case 5:if(!h.eq_s_b(1,"ä"))return;break;case 6:if(!h.eq_s_b(1,"ö"))return;break;case 7:if(r=h.limit-h.cursor,!j()&&(h.cursor=h.limit-r,!h.eq_s_b(2,"ie"))){h.cursor=h.limit-r;break}if(h.cursor=h.limit-r,h.cursor<=h.limit_backward){h.cursor=h.limit-r;break}h.cursor--,h.bra=h.cursor;break;case 8:if(!h.in_grouping_b(d,97,246)||!h.out_grouping_b(d,97,246))return}h.slice_del(),n=!0}else h.limit_backward=e}(),h.cursor=h.limit,function(){var i,e,r;if(h.cursor>=l)if(e=h.limit_backward,h.limit_backward=l,h.ket=h.cursor,i=h.find_among_b(w,14)){if(h.bra=h.cursor,h.limit_backward=e,1==i){if(r=h.limit-h.cursor,h.eq_s_b(2,"po"))return;h.cursor=h.limit-r}h.slice_del()}else h.limit_backward=e}(),h.cursor=h.limit,h.cursor=(n?h.cursor>=o&&(i=h.limit_backward,h.limit_backward=o,h.ket=h.cursor,h.find_among_b(_,2)?(h.bra=h.cursor,h.limit_backward=i,h.slice_del()):h.limit_backward=i):(h.cursor=h.limit,function(){var i,e,r,n,t,s;if(h.cursor>=o){if(e=h.limit_backward,h.limit_backward=o,h.ket=h.cursor,h.eq_s_b(1,"t")&&(h.bra=h.cursor,r=h.limit-h.cursor,h.in_grouping_b(d,97,246)&&(h.cursor=h.limit-r,h.slice_del(),h.limit_backward=e,n=h.limit-h.cursor,h.cursor>=l&&(h.cursor=l,t=h.limit_backward,h.limit_backward=h.cursor,h.cursor=h.limit-n,h.ket=h.cursor,i=h.find_among_b(k,2))))){if(h.bra=h.cursor,h.limit_backward=t,1==i){if(s=h.limit-h.cursor,h.eq_s_b(2,"po"))return;h.cursor=h.limit-s}return h.slice_del()}h.limit_backward=e}}()),h.limit),function(){var i,e,r,n;if(h.cursor>=o){for(i=h.limit_backward,h.limit_backward=o,e=h.limit-h.cursor,j()&&(h.cursor=h.limit-e,h.ket=h.cursor,h.cursor>h.limit_backward&&(h.cursor--,h.bra=h.cursor,h.slice_del())),h.cursor=h.limit-e,h.ket=h.cursor,h.in_grouping_b(b,97,228)&&(h.bra=h.cursor,h.out_grouping_b(d,97,246)&&h.slice_del()),h.cursor=h.limit-e,h.ket=h.cursor,h.eq_s_b(1,"j")&&(h.bra=h.cursor,r=h.limit-h.cursor,h.eq_s_b(1,"o")?h.slice_del():(h.cursor=h.limit-r,h.eq_s_b(1,"u")&&h.slice_del())),h.cursor=h.limit-e,h.ket=h.cursor,h.eq_s_b(1,"o")&&(h.bra=h.cursor,h.eq_s_b(1,"j")&&h.slice_del()),h.cursor=h.limit-e,h.limit_backward=i;;){if(n=h.limit-h.cursor,h.out_grouping_b(d,97,246)){h.cursor=h.limit-n;break}if(h.cursor=h.limit-n,h.cursor<=h.limit_backward)return;h.cursor--}h.ket=h.cursor,h.cursor>h.limit_backward&&(h.cursor--,h.bra=h.cursor,t=h.slice_to(),h.eq_v_b(t)&&h.slice_del())}}(),!0}},function(i){return"function"==typeof i.update?i.update(function(i){return e.setCurrent(i),e.stem(),e.getCurrent()}):(e.setCurrent(i),e.stem(),e.getCurrent())}),i.Pipeline.registerFunction(i.fi.stemmer,"stemmer-fi"),i.fi.stopWordFilter=i.generateStopWordFilter("ei eivät emme en et ette että he heidän heidät heihin heille heillä heiltä heissä heistä heitä hän häneen hänelle hänellä häneltä hänen hänessä hänestä hänet häntä itse ja johon joiden joihin joiksi joilla joille joilta joina joissa joista joita joka joksi jolla jolle jolta jona jonka jos jossa josta jota jotka kanssa keiden keihin keiksi keille keillä keiltä keinä keissä keistä keitä keneen keneksi kenelle kenellä keneltä kenen kenenä kenessä kenestä kenet ketkä ketkä ketä koska kuin kuka kun me meidän meidät meihin meille meillä meiltä meissä meistä meitä mihin miksi mikä mille millä miltä minkä minkä minua minulla minulle minulta minun minussa minusta minut minuun minä minä missä mistä mitkä mitä mukaan mutta ne niiden niihin niiksi niille niillä niiltä niin niin niinä niissä niistä niitä noiden noihin noiksi noilla noille noilta noin noina noissa noista noita nuo nyt näiden näihin näiksi näille näillä näiltä näinä näissä näistä näitä nämä ole olemme olen olet olette oli olimme olin olisi olisimme olisin olisit olisitte olisivat olit olitte olivat olla olleet ollut on ovat poikki se sekä sen siihen siinä siitä siksi sille sillä sillä siltä sinua sinulla sinulle sinulta sinun sinussa sinusta sinut sinuun sinä sinä sitä tai te teidän teidät teihin teille teillä teiltä teissä teistä teitä tuo tuohon tuoksi tuolla tuolle tuolta tuon tuona tuossa tuosta tuota tähän täksi tälle tällä tältä tämä tämän tänä tässä tästä tätä vaan vai vaikka yli".split(" ")),i.Pipeline.registerFunction(i.fi.stopWordFilter,"stopWordFilter-fi")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.hu.js: -------------------------------------------------------------------------------- 1 | !function(e,n){"function"==typeof define&&define.amd?define(n):"object"==typeof exports?module.exports=n():n()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var p,_,n;e.hu=function(){this.pipeline.reset(),this.pipeline.add(e.hu.trimmer,e.hu.stopWordFilter,e.hu.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hu.stemmer))},e.hu.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.hu.trimmer=e.trimmerSupport.generateTrimmer(e.hu.wordCharacters),e.Pipeline.registerFunction(e.hu.trimmer,"trimmer-hu"),e.hu.stemmer=(p=e.stemmerSupport.Among,_=e.stemmerSupport.SnowballProgram,n=new function(){var r,i=[new p("cs",-1,-1),new p("dzs",-1,-1),new p("gy",-1,-1),new p("ly",-1,-1),new p("ny",-1,-1),new p("sz",-1,-1),new p("ty",-1,-1),new p("zs",-1,-1)],n=[new p("á",-1,1),new p("é",-1,2)],a=[new p("bb",-1,-1),new p("cc",-1,-1),new p("dd",-1,-1),new p("ff",-1,-1),new p("gg",-1,-1),new p("jj",-1,-1),new p("kk",-1,-1),new p("ll",-1,-1),new p("mm",-1,-1),new p("nn",-1,-1),new p("pp",-1,-1),new p("rr",-1,-1),new p("ccs",-1,-1),new p("ss",-1,-1),new p("zzs",-1,-1),new p("tt",-1,-1),new p("vv",-1,-1),new p("ggy",-1,-1),new p("lly",-1,-1),new p("nny",-1,-1),new p("tty",-1,-1),new p("ssz",-1,-1),new p("zz",-1,-1)],t=[new p("al",-1,1),new p("el",-1,2)],e=[new p("ba",-1,-1),new p("ra",-1,-1),new p("be",-1,-1),new p("re",-1,-1),new p("ig",-1,-1),new p("nak",-1,-1),new p("nek",-1,-1),new p("val",-1,-1),new p("vel",-1,-1),new p("ul",-1,-1),new p("nál",-1,-1),new p("nél",-1,-1),new p("ból",-1,-1),new p("ról",-1,-1),new p("tól",-1,-1),new p("bõl",-1,-1),new p("rõl",-1,-1),new p("tõl",-1,-1),new p("ül",-1,-1),new p("n",-1,-1),new p("an",19,-1),new p("ban",20,-1),new p("en",19,-1),new p("ben",22,-1),new p("képpen",22,-1),new p("on",19,-1),new p("ön",19,-1),new p("képp",-1,-1),new p("kor",-1,-1),new p("t",-1,-1),new p("at",29,-1),new p("et",29,-1),new p("ként",29,-1),new p("anként",32,-1),new p("enként",32,-1),new p("onként",32,-1),new p("ot",29,-1),new p("ért",29,-1),new p("öt",29,-1),new p("hez",-1,-1),new p("hoz",-1,-1),new p("höz",-1,-1),new p("vá",-1,-1),new p("vé",-1,-1)],s=[new p("án",-1,2),new p("én",-1,1),new p("ánként",-1,3)],c=[new p("stul",-1,2),new p("astul",0,1),new p("ástul",0,3),new p("stül",-1,2),new p("estül",3,1),new p("éstül",3,4)],w=[new p("á",-1,1),new p("é",-1,2)],o=[new p("k",-1,7),new p("ak",0,4),new p("ek",0,6),new p("ok",0,5),new p("ák",0,1),new p("ék",0,2),new p("ök",0,3)],l=[new p("éi",-1,7),new p("áéi",0,6),new p("ééi",0,5),new p("é",-1,9),new p("ké",3,4),new p("aké",4,1),new p("eké",4,1),new p("oké",4,1),new p("áké",4,3),new p("éké",4,2),new p("öké",4,1),new p("éé",3,8)],u=[new p("a",-1,18),new p("ja",0,17),new p("d",-1,16),new p("ad",2,13),new p("ed",2,13),new p("od",2,13),new p("ád",2,14),new p("éd",2,15),new p("öd",2,13),new p("e",-1,18),new p("je",9,17),new p("nk",-1,4),new p("unk",11,1),new p("ánk",11,2),new p("énk",11,3),new p("ünk",11,1),new p("uk",-1,8),new p("juk",16,7),new p("ájuk",17,5),new p("ük",-1,8),new p("jük",19,7),new p("éjük",20,6),new p("m",-1,12),new p("am",22,9),new p("em",22,9),new p("om",22,9),new p("ám",22,10),new p("ém",22,11),new p("o",-1,18),new p("á",-1,19),new p("é",-1,20)],m=[new p("id",-1,10),new p("aid",0,9),new p("jaid",1,6),new p("eid",0,9),new p("jeid",3,6),new p("áid",0,7),new p("éid",0,8),new p("i",-1,15),new p("ai",7,14),new p("jai",8,11),new p("ei",7,14),new p("jei",10,11),new p("ái",7,12),new p("éi",7,13),new p("itek",-1,24),new p("eitek",14,21),new p("jeitek",15,20),new p("éitek",14,23),new p("ik",-1,29),new p("aik",18,26),new p("jaik",19,25),new p("eik",18,26),new p("jeik",21,25),new p("áik",18,27),new p("éik",18,28),new p("ink",-1,20),new p("aink",25,17),new p("jaink",26,16),new p("eink",25,17),new p("jeink",28,16),new p("áink",25,18),new p("éink",25,19),new p("aitok",-1,21),new p("jaitok",32,20),new p("áitok",-1,22),new p("im",-1,5),new p("aim",35,4),new p("jaim",36,1),new p("eim",35,4),new p("jeim",38,1),new p("áim",35,2),new p("éim",35,3)],k=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,1,17,52,14],f=new _;function b(){return r<=f.cursor}function d(){var e=f.limit-f.cursor;return!!f.find_among_b(a,23)&&(f.cursor=f.limit-e,!0)}function g(){if(f.cursor>f.limit_backward){f.cursor--,f.ket=f.cursor;var e=f.cursor-1;f.limit_backward<=e&&e<=f.limit&&(f.cursor=e,f.bra=e,f.slice_del())}}function h(){f.ket=f.cursor,f.find_among_b(e,44)&&(f.bra=f.cursor,b()&&(f.slice_del(),function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(n,2))&&(f.bra=f.cursor,b()))switch(e){case 1:f.slice_from("a");break;case 2:f.slice_from("e")}}()))}this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var e=f.cursor;return function(){var e,n=f.cursor;if(r=f.limit,f.in_grouping(k,97,252))for(;;){if(e=f.cursor,f.out_grouping(k,97,252))return f.cursor=e,f.find_among(i,8)||(f.cursor=e)=f.limit)return r=e;f.cursor++}if(f.cursor=n,f.out_grouping(k,97,252)){for(;!f.in_grouping(k,97,252);){if(f.cursor>=f.limit)return;f.cursor++}r=f.cursor}}(),f.limit_backward=e,f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(t,2))&&(f.bra=f.cursor,b())){if((1==e||2==e)&&!d())return;f.slice_del(),g()}}(),f.cursor=f.limit,h(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(s,3))&&(f.bra=f.cursor,b()))switch(e){case 1:f.slice_from("e");break;case 2:case 3:f.slice_from("a")}}(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(c,6))&&(f.bra=f.cursor,b()))switch(e){case 1:case 2:f.slice_del();break;case 3:f.slice_from("a");break;case 4:f.slice_from("e")}}(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(w,2))&&(f.bra=f.cursor,b())){if((1==e||2==e)&&!d())return;f.slice_del(),g()}}(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(l,12))&&(f.bra=f.cursor,b()))switch(e){case 1:case 4:case 7:case 9:f.slice_del();break;case 2:case 5:case 8:f.slice_from("e");break;case 3:case 6:f.slice_from("a")}}(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(u,31))&&(f.bra=f.cursor,b()))switch(e){case 1:case 4:case 7:case 8:case 9:case 12:case 13:case 16:case 17:case 18:f.slice_del();break;case 2:case 5:case 10:case 14:case 19:f.slice_from("a");break;case 3:case 6:case 11:case 15:case 20:f.slice_from("e")}}(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(m,42))&&(f.bra=f.cursor,b()))switch(e){case 1:case 4:case 5:case 6:case 9:case 10:case 11:case 14:case 15:case 16:case 17:case 20:case 21:case 24:case 25:case 26:case 29:f.slice_del();break;case 2:case 7:case 12:case 18:case 22:case 27:f.slice_from("a");break;case 3:case 8:case 13:case 19:case 23:case 28:f.slice_from("e")}}(),f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,(e=f.find_among_b(o,7))&&(f.bra=f.cursor,b()))switch(e){case 1:f.slice_from("a");break;case 2:f.slice_from("e");break;case 3:case 4:case 5:case 6:case 7:f.slice_del()}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}),e.Pipeline.registerFunction(e.hu.stemmer,"stemmer-hu"),e.hu.stopWordFilter=e.generateStopWordFilter("a abban ahhoz ahogy ahol aki akik akkor alatt amely amelyek amelyekben amelyeket amelyet amelynek ami amikor amit amolyan amíg annak arra arról az azok azon azonban azt aztán azután azzal azért be belül benne bár cikk cikkek cikkeket csak de e ebben eddig egy egyes egyetlen egyik egyre egyéb egész ehhez ekkor el ellen elsõ elég elõ elõször elõtt emilyen ennek erre ez ezek ezen ezt ezzel ezért fel felé hanem hiszen hogy hogyan igen ill ill. illetve ilyen ilyenkor ismét ison itt jobban jó jól kell kellett keressünk keresztül ki kívül között közül legalább legyen lehet lehetett lenne lenni lesz lett maga magát majd majd meg mellett mely melyek mert mi mikor milyen minden mindenki mindent mindig mint mintha mit mivel miért most már más másik még míg nagy nagyobb nagyon ne nekem neki nem nincs néha néhány nélkül olyan ott pedig persze rá s saját sem semmi sok sokat sokkal szemben szerint szinte számára talán tehát teljes tovább továbbá több ugyanis utolsó után utána vagy vagyis vagyok valaki valami valamint való van vannak vele vissza viszont volna volt voltak voltam voltunk által általában át én éppen és így õ õk õket össze úgy új újabb újra".split(" ")),e.Pipeline.registerFunction(e.hu.stopWordFilter,"stopWordFilter-hu")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.pt.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var j,C,r;e.pt=function(){this.pipeline.reset(),this.pipeline.add(e.pt.trimmer,e.pt.stopWordFilter,e.pt.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.pt.stemmer))},e.pt.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.pt.trimmer=e.trimmerSupport.generateTrimmer(e.pt.wordCharacters),e.Pipeline.registerFunction(e.pt.trimmer,"trimmer-pt"),e.pt.stemmer=(j=e.stemmerSupport.Among,C=e.stemmerSupport.SnowballProgram,r=new function(){var s,n,i,o=[new j("",-1,3),new j("ã",0,1),new j("õ",0,2)],a=[new j("",-1,3),new j("a~",0,1),new j("o~",0,2)],r=[new j("ic",-1,-1),new j("ad",-1,-1),new j("os",-1,-1),new j("iv",-1,1)],t=[new j("ante",-1,1),new j("avel",-1,1),new j("ível",-1,1)],u=[new j("ic",-1,1),new j("abil",-1,1),new j("iv",-1,1)],w=[new j("ica",-1,1),new j("ância",-1,1),new j("ência",-1,4),new j("ira",-1,9),new j("adora",-1,1),new j("osa",-1,1),new j("ista",-1,1),new j("iva",-1,8),new j("eza",-1,1),new j("logía",-1,2),new j("idade",-1,7),new j("ante",-1,1),new j("mente",-1,6),new j("amente",12,5),new j("ável",-1,1),new j("ível",-1,1),new j("ución",-1,3),new j("ico",-1,1),new j("ismo",-1,1),new j("oso",-1,1),new j("amento",-1,1),new j("imento",-1,1),new j("ivo",-1,8),new j("aça~o",-1,1),new j("ador",-1,1),new j("icas",-1,1),new j("ências",-1,4),new j("iras",-1,9),new j("adoras",-1,1),new j("osas",-1,1),new j("istas",-1,1),new j("ivas",-1,8),new j("ezas",-1,1),new j("logías",-1,2),new j("idades",-1,7),new j("uciones",-1,3),new j("adores",-1,1),new j("antes",-1,1),new j("aço~es",-1,1),new j("icos",-1,1),new j("ismos",-1,1),new j("osos",-1,1),new j("amentos",-1,1),new j("imentos",-1,1),new j("ivos",-1,8)],m=[new j("ada",-1,1),new j("ida",-1,1),new j("ia",-1,1),new j("aria",2,1),new j("eria",2,1),new j("iria",2,1),new j("ara",-1,1),new j("era",-1,1),new j("ira",-1,1),new j("ava",-1,1),new j("asse",-1,1),new j("esse",-1,1),new j("isse",-1,1),new j("aste",-1,1),new j("este",-1,1),new j("iste",-1,1),new j("ei",-1,1),new j("arei",16,1),new j("erei",16,1),new j("irei",16,1),new j("am",-1,1),new j("iam",20,1),new j("ariam",21,1),new j("eriam",21,1),new j("iriam",21,1),new j("aram",20,1),new j("eram",20,1),new j("iram",20,1),new j("avam",20,1),new j("em",-1,1),new j("arem",29,1),new j("erem",29,1),new j("irem",29,1),new j("assem",29,1),new j("essem",29,1),new j("issem",29,1),new j("ado",-1,1),new j("ido",-1,1),new j("ando",-1,1),new j("endo",-1,1),new j("indo",-1,1),new j("ara~o",-1,1),new j("era~o",-1,1),new j("ira~o",-1,1),new j("ar",-1,1),new j("er",-1,1),new j("ir",-1,1),new j("as",-1,1),new j("adas",47,1),new j("idas",47,1),new j("ias",47,1),new j("arias",50,1),new j("erias",50,1),new j("irias",50,1),new j("aras",47,1),new j("eras",47,1),new j("iras",47,1),new j("avas",47,1),new j("es",-1,1),new j("ardes",58,1),new j("erdes",58,1),new j("irdes",58,1),new j("ares",58,1),new j("eres",58,1),new j("ires",58,1),new j("asses",58,1),new j("esses",58,1),new j("isses",58,1),new j("astes",58,1),new j("estes",58,1),new j("istes",58,1),new j("is",-1,1),new j("ais",71,1),new j("eis",71,1),new j("areis",73,1),new j("ereis",73,1),new j("ireis",73,1),new j("áreis",73,1),new j("éreis",73,1),new j("íreis",73,1),new j("ásseis",73,1),new j("ésseis",73,1),new j("ísseis",73,1),new j("áveis",73,1),new j("íeis",73,1),new j("aríeis",84,1),new j("eríeis",84,1),new j("iríeis",84,1),new j("ados",-1,1),new j("idos",-1,1),new j("amos",-1,1),new j("áramos",90,1),new j("éramos",90,1),new j("íramos",90,1),new j("ávamos",90,1),new j("íamos",90,1),new j("aríamos",95,1),new j("eríamos",95,1),new j("iríamos",95,1),new j("emos",-1,1),new j("aremos",99,1),new j("eremos",99,1),new j("iremos",99,1),new j("ássemos",99,1),new j("êssemos",99,1),new j("íssemos",99,1),new j("imos",-1,1),new j("armos",-1,1),new j("ermos",-1,1),new j("irmos",-1,1),new j("ámos",-1,1),new j("arás",-1,1),new j("erás",-1,1),new j("irás",-1,1),new j("eu",-1,1),new j("iu",-1,1),new j("ou",-1,1),new j("ará",-1,1),new j("erá",-1,1),new j("irá",-1,1)],c=[new j("a",-1,1),new j("i",-1,1),new j("o",-1,1),new j("os",-1,1),new j("á",-1,1),new j("í",-1,1),new j("ó",-1,1)],l=[new j("e",-1,1),new j("ç",-1,2),new j("é",-1,1),new j("ê",-1,1)],f=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,3,19,12,2],d=new C;function v(){if(d.out_grouping(f,97,250)){for(;!d.in_grouping(f,97,250);){if(d.cursor>=d.limit)return!0;d.cursor++}return!1}return!0}function p(){var e,r,s=d.cursor;if(d.in_grouping(f,97,250))if(e=d.cursor,v()){if(d.cursor=e,function(){if(d.in_grouping(f,97,250))for(;!d.out_grouping(f,97,250);){if(d.cursor>=d.limit)return!1;d.cursor++}return i=d.cursor,!0}())return}else i=d.cursor;if(d.cursor=s,d.out_grouping(f,97,250)){if(r=d.cursor,v()){if(d.cursor=r,!d.in_grouping(f,97,250)||d.cursor>=d.limit)return;d.cursor++}i=d.cursor}}function _(){for(;!d.in_grouping(f,97,250);){if(d.cursor>=d.limit)return!1;d.cursor++}for(;!d.out_grouping(f,97,250);){if(d.cursor>=d.limit)return!1;d.cursor++}return!0}function h(){return i<=d.cursor}function b(){return s<=d.cursor}function g(){var e;if(d.ket=d.cursor,!(e=d.find_among_b(w,45)))return!1;switch(d.bra=d.cursor,e){case 1:if(!b())return!1;d.slice_del();break;case 2:if(!b())return!1;d.slice_from("log");break;case 3:if(!b())return!1;d.slice_from("u");break;case 4:if(!b())return!1;d.slice_from("ente");break;case 5:if(!(n<=d.cursor))return!1;d.slice_del(),d.ket=d.cursor,(e=d.find_among_b(r,4))&&(d.bra=d.cursor,b()&&(d.slice_del(),1==e&&(d.ket=d.cursor,d.eq_s_b(2,"at")&&(d.bra=d.cursor,b()&&d.slice_del()))));break;case 6:if(!b())return!1;d.slice_del(),d.ket=d.cursor,(e=d.find_among_b(t,3))&&(d.bra=d.cursor,1==e&&b()&&d.slice_del());break;case 7:if(!b())return!1;d.slice_del(),d.ket=d.cursor,(e=d.find_among_b(u,3))&&(d.bra=d.cursor,1==e&&b()&&d.slice_del());break;case 8:if(!b())return!1;d.slice_del(),d.ket=d.cursor,d.eq_s_b(2,"at")&&(d.bra=d.cursor,b()&&d.slice_del());break;case 9:if(!h()||!d.eq_s_b(1,"e"))return!1;d.slice_from("ir")}return!0}function k(e,r){if(d.eq_s_b(1,e)){d.bra=d.cursor;var s=d.limit-d.cursor;if(d.eq_s_b(1,r))return d.cursor=d.limit-s,h()&&d.slice_del(),!1}return!0}function q(){if(!g()&&(d.cursor=d.limit,!function(){var e,r;if(d.cursor>=i){if(r=d.limit_backward,d.limit_backward=i,d.ket=d.cursor,e=d.find_among_b(m,120))return d.bra=d.cursor,1==e&&d.slice_del(),d.limit_backward=r,!0;d.limit_backward=r}return!1}()))return d.cursor=d.limit,d.ket=d.cursor,void((e=d.find_among_b(c,7))&&(d.bra=d.cursor,1==e&&h()&&d.slice_del()));var e;d.cursor=d.limit,d.ket=d.cursor,d.eq_s_b(1,"i")&&(d.bra=d.cursor,d.eq_s_b(1,"c")&&(d.cursor=d.limit,h()&&d.slice_del()))}this.setCurrent=function(e){d.setCurrent(e)},this.getCurrent=function(){return d.getCurrent()},this.stem=function(){var e,r=d.cursor;return function(){for(var e;;){if(d.bra=d.cursor,e=d.find_among(o,3))switch(d.ket=d.cursor,e){case 1:d.slice_from("a~");continue;case 2:d.slice_from("o~");continue;case 3:if(d.cursor>=d.limit)break;d.cursor++;continue}break}}(),d.cursor=r,e=d.cursor,i=d.limit,s=n=i,p(),d.cursor=e,_()&&(n=d.cursor,_()&&(s=d.cursor)),d.limit_backward=r,d.cursor=d.limit,q(),d.cursor=d.limit,function(){var e;if(d.ket=d.cursor,e=d.find_among_b(l,4))switch(d.bra=d.cursor,e){case 1:h()&&(d.slice_del(),d.ket=d.cursor,d.limit,d.cursor,k("u","g")&&k("i","c"));break;case 2:d.slice_from("c")}}(),d.cursor=d.limit_backward,function(){for(var e;;){if(d.bra=d.cursor,e=d.find_among(a,3))switch(d.ket=d.cursor,e){case 1:d.slice_from("ã");continue;case 2:d.slice_from("õ");continue;case 3:if(d.cursor>=d.limit)break;d.cursor++;continue}break}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return r.setCurrent(e),r.stem(),r.getCurrent()}):(r.setCurrent(e),r.stem(),r.getCurrent())}),e.Pipeline.registerFunction(e.pt.stemmer,"stemmer-pt"),e.pt.stopWordFilter=e.generateStopWordFilter("a ao aos aquela aquelas aquele aqueles aquilo as até com como da das de dela delas dele deles depois do dos e ela elas ele eles em entre era eram essa essas esse esses esta estamos estas estava estavam este esteja estejam estejamos estes esteve estive estivemos estiver estivera estiveram estiverem estivermos estivesse estivessem estivéramos estivéssemos estou está estávamos estão eu foi fomos for fora foram forem formos fosse fossem fui fôramos fôssemos haja hajam hajamos havemos hei houve houvemos houver houvera houveram houverei houverem houveremos houveria houveriam houvermos houverá houverão houveríamos houvesse houvessem houvéramos houvéssemos há hão isso isto já lhe lhes mais mas me mesmo meu meus minha minhas muito na nas nem no nos nossa nossas nosso nossos num numa não nós o os ou para pela pelas pelo pelos por qual quando que quem se seja sejam sejamos sem serei seremos seria seriam será serão seríamos seu seus somos sou sua suas são só também te tem temos tenha tenham tenhamos tenho terei teremos teria teriam terá terão teríamos teu teus teve tinha tinham tive tivemos tiver tivera tiveram tiverem tivermos tivesse tivessem tivéramos tivéssemos tu tua tuas tém tínhamos um uma você vocês vos à às éramos".split(" ")),e.Pipeline.registerFunction(e.pt.stopWordFilter,"stopWordFilter-pt")}}); -------------------------------------------------------------------------------- /nlpdocument/docs/dl_framework.md: -------------------------------------------------------------------------------- 1 | # Deep Learning Framework Programming 2 | 3 | 4 | ## Programming in Tensorflow 5 | ### tf.variable_scope / tf.name_scope 6 | Both scopes have the same effect on all operations as well as variables, but *name scope* is ignored by ```:::python tf.get_variable```. Suggest use ```:::python tf.variable_scope``` in most cases. 7 | 8 | !!! info "Ref" 9 | The difference between name scope and variable scope in tensorflow at [stackoverflow](https://stackoverflow.com/questions/35919020/whats-the-difference-of-name-scope-and-a-variable-scope-in-tensorflow). 10 | 11 | ### Model Save / Restore 12 | Usually, we create a helper ```saver = tf.train.Saver()``` to save and restore the whole model. However, if we want to use pre-trained model for fine-tuning or transfer learning, there are 2 ways: (1) Create the network by writing code to create each and every layer manually as the original model, and then use ```tf.train.Saver()``` to restore pre-trained model's checkpoint file. (2) Use ```.meta``` file and create the helper as ```saver = tf.train.import_meta_graph('xxx_model-xxx.meta')``` and then restore the pre-trained model. 13 | 14 | !!! info "Ref" 15 | More details are in this [tutorial](https://cv-tricks.com/tensorflow-tutorial/save-restore-tensorflow-models-quick-complete-tutorial/). 16 | 17 | ### Multi-graph / Multi-session 18 | Sometimes we need to build more than one tensorflow graph, e.g., we need to transfer the outputs of one model into another model for further training, then we usually build 2 different graphs, each represents a model. It should be noted that all the operations of each model must be specified under its corresponding graph. A simple example is illustrated as follows: 19 | ``` python 20 | # define one graph named 'kbqa_graph' 21 | kbqa_graph = tf.Graph() 22 | with kbqa_graph.as_default(): 23 | kbqa_model = KbqaModel(**kbqa_model_config) 24 | kbqa_saver = tf.train.Saver() 25 | 26 | # define one session named 'kbqa_sess' for loading pre-trained KBQA model 27 | kbqa_sess = tf.Session(config=config, graph=kbqa_graph) 28 | model_path = '%s/model_best/best.model' % args.model_dir 29 | kbqa_saver.restore(kbqa_sess, save_path=model_path) 30 | 31 | # define another graph named 'main_graph' and the session 'main_sess' 32 | main_graph = tf.Graph() 33 | with main_graph.as_default(): 34 | main_model = MainModel(**model_config) 35 | main_server = tf.train.Server(max_to_keep=5) 36 | main_sess = tf.Session(config=config, graph=main_graph) 37 | 38 | ... 39 | 40 | # note: the operations of 'main_model' should be specified under the 'main_graph' domain 41 | with main_graph.as_default(): 42 | main_sess.run(tf.global_variables_initializer()) 43 | main_model.set_vocabs(sess, word_vocab, kd_vocab) 44 | ``` 45 | 46 | 47 | ## Programming in PyTorch 48 | ### CUDA out of memory 49 | When ```RuntimeError: CUDA out of memory``` occurs, usually (1) check if exists too large tensors in computation graph; (2) downsize the batch size; (3) or use multiple GPUs to train. Note to split batch size when using ```nn.DataParallel```. 50 | 51 | !!! info "Ref" 52 | Some other details are in this [debug log](https://docs.google.com/document/d/1Cpxs-aZcydqCzTEvfW-62ja6ZDhx2QEXR-f5HKmbeig/edit?usp=sharing). 53 | 54 | ## Online Tensorflow Serving 55 | TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments. Servables are the central abstraction in TensorFlow Serving. Servables are the underlying objects that clients use to perform computation (for example, a lookup or inference). 56 | 57 | The size and granularity of a Servable is flexible. A single Servable might include anything from a single shard of a lookup table to a single model to a tuple of inference models. Servables can be of any type and interface, enabling flexibility and future improvements such as: streaming results, experimental APIs, asynchronous modes of operation. 58 | 59 | ### How to deploy 60 | Tensorflow Serving follows the server-client architecture. While training a specific model, save it in the mode that can be used by tensorflow-serving. Deploy your model on a running docker to provide service. For clients, request server for prediction results of given data instances. The following is an example of the deployment procedure. 61 | 62 | **Save a trained model** 63 | 64 | The common way to save model in tensorflow looks like, 65 | 66 | ``` 67 | saver.save(session, checkpoint_prefix, global_step=current_step) 68 | tf.train.write_graph(sess.graph.as_graph_def(), checkpoint_prefix, "graph"+str(nn)+".pb", as_text=False) 69 | ``` 70 | For tensorflow serving, we save like, 71 | ``` python 72 | ### define input&output signature 73 | signature = tf.saved_model.signature_def_utils.build_signature_def( 74 | inputs={ 75 | 'input_x1': tf.saved_model.utils.build_tensor_info(self.input_x1), 76 | 'input_x2': tf.saved_model.utils.build_tensor_info(self.input_x2), 77 | 'ent_x1': tf.saved_model.utils.build_tensor_info(self.ent_x1), 78 | 'ent_x2': tf.saved_model.utils.build_tensor_info(self.ent_x2), 79 | 'input_y': tf.saved_model.utils.build_tensor_info(self.input_y), 80 | 'add_fea': tf.saved_model.utils.build_tensor_info(self.add_fea), 81 | 'dropout_keep_prob': tf.saved_model.utils.build_tensor_info(self.dropout_keep_prob) 82 | }, 83 | outputs={ 84 | 'output': tf.saved_model.utils.build_tensor_info(self.soft_prob) 85 | }, 86 | method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME 87 | ) 88 | ### saving 89 | builder = tf.saved_model.builder.SavedModelBuilder(graph_save_dir) 90 | builder.add_meta_graph_and_variables(sess, [tf.saved_model.tag_constants.SERVING], {tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature}) 91 | builder.save() 92 | ``` 93 | Define your own inputs and outputs according to the task. 94 | 95 | **Run a serving docker and deploy your model** 96 | 97 | ``` 98 | # pull a tensorflow-serving image 99 | $ sudo docker pull tensorflow/serving:latest-devel 100 | # run the serving docker 101 | $ sudo docker run -it -p 8500:8500 tensorflow/serving:latest-devel 102 | # copy your model file to the running docker (change the docker ID and your model path) 103 | $ sudo docker cp /data/huangweiyi/qaModel/code/kaaqa/runs/model 6d7d70e27ecc:/online_qa_model 104 | ``` 105 | 106 | !!! Note 107 | You need to create different version of your model for tensorflow-serving (refer to https://stackoverflow.com/questions/45544928/tensorflow-serving-no-versions-of-servable-model-found-under-base-path) 108 | 109 | 110 | ### Deploy your model in the running docker 111 | ``` 112 | $ tensorflow_model_server --port=8500 --model_name=qa --model_base_path=/online_qa_model 113 | ``` 114 | 115 | **Request the server for prediction results** 116 | 117 | ``` python 118 | hostport = '172.17.0.2:8500' 119 | channel = grpc.insecure_channel(hostport) 120 | stub = prediction_service_pb2_grpc.PredictionServiceStub(channel) 121 | 122 | request = predict_pb2.PredictRequest() 123 | request.model_spec.name = 'qa' 124 | 125 | inpH = InputHelper() 126 | x1_test, x2_test, ent_x1_test, ent_x2_test, y_test, x1_temp, x2_temp, add_fea_test = inpH.getTestSample(question, candidates) 127 | batches = inpH.batch_iter(list(zip(x1_test, x2_test, ent_x1_test, ent_x2_test, y_test, add_fea_test)), 10000, 1, shuffle=False) 128 | for db in batches: 129 | x1_dev_b, x2_dev_b, ent_x1_dev_b, ent_x2_dev_b, y_dev_b, add_fea_dev_b = zip(*db) 130 | for idx in range(len(x1_dev_b)): 131 | feature_dict = { 132 | "input_x1": x1_dev_b, 133 | "input_x2": x2_dev_b, 134 | "ent_x1": ent_x1_dev_b, 135 | "ent_x2": ent_x2_dev_b, 136 | "input_y": y_dev_b, 137 | "add_fea": add_fea_dev_b, 138 | "dropout_keep_prob": 1, 139 | } 140 | for key in ['input_x1', 'input_x2', 'ent_x1', 'ent_x2']: 141 | value = feature_dict.get(key)[idx].astype(np.int32) 142 | request.inputs[key].CopyFrom(tf.contrib.util.make_tensor_proto(value, shape=[1, value.size])) 143 | request.inputs['dropout_keep_prob'].CopyFrom(tf.contrib.util.make_tensor_proto(1.0, shape=[1])) 144 | request.inputs['input_y'].CopyFrom(tf.contrib.util.make_tensor_proto(1, shape=[1], dtype=np.int64)) 145 | value = add_fea_dev_b[0].astype(np.float32) 146 | request.inputs['add_fea'].CopyFrom(tf.contrib.util.make_tensor_proto(value, shape=[1, value.size])) 147 | 148 | result_future = stub.Predict.future(request, 3.0) 149 | score = np.array(result_future.result().outputs['output'].float_val)[1] 150 | ``` 151 | 152 | Modify "feature_dict" according to your input variables. The variable "score" is the model output for your request instance. For more details, please refer to http://210.75.252.89:3000/hweiyi/aiLawAssistant/src/branch/master/ranking/client.py for all the codes. 153 | 154 | !!! info "Ref" 155 | A detailed illustration of saving model for tensorflow-serving (https://zhuanlan.zhihu.com/p/40226973) 156 | 157 | !!! info "Ref" 158 | Documents of tensorflow-serving (https://bookdown.org/leovan/TensorFlow-Learning-Notes/4-5-deploy-tensorflow-serving.html#using-tensorflow-serving-via-docker--docker--tensorflow-serving) 159 | 160 | !!! info "Ref" 161 | An officail example (https://github.com/tensorflow/serving/tree/master/tensorflow_serving/example) 162 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.fr.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r,y,s;e.fr=function(){this.pipeline.reset(),this.pipeline.add(e.fr.trimmer,e.fr.stopWordFilter,e.fr.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.fr.stemmer))},e.fr.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.fr.trimmer=e.trimmerSupport.generateTrimmer(e.fr.wordCharacters),e.Pipeline.registerFunction(e.fr.trimmer,"trimmer-fr"),e.fr.stemmer=(r=e.stemmerSupport.Among,y=e.stemmerSupport.SnowballProgram,s=new function(){var s,i,t,n=[new r("col",-1,-1),new r("par",-1,-1),new r("tap",-1,-1)],u=[new r("",-1,4),new r("I",0,1),new r("U",0,2),new r("Y",0,3)],o=[new r("iqU",-1,3),new r("abl",-1,3),new r("Ièr",-1,4),new r("ièr",-1,4),new r("eus",-1,2),new r("iv",-1,1)],c=[new r("ic",-1,2),new r("abil",-1,1),new r("iv",-1,3)],a=[new r("iqUe",-1,1),new r("atrice",-1,2),new r("ance",-1,1),new r("ence",-1,5),new r("logie",-1,3),new r("able",-1,1),new r("isme",-1,1),new r("euse",-1,11),new r("iste",-1,1),new r("ive",-1,8),new r("if",-1,8),new r("usion",-1,4),new r("ation",-1,2),new r("ution",-1,4),new r("ateur",-1,2),new r("iqUes",-1,1),new r("atrices",-1,2),new r("ances",-1,1),new r("ences",-1,5),new r("logies",-1,3),new r("ables",-1,1),new r("ismes",-1,1),new r("euses",-1,11),new r("istes",-1,1),new r("ives",-1,8),new r("ifs",-1,8),new r("usions",-1,4),new r("ations",-1,2),new r("utions",-1,4),new r("ateurs",-1,2),new r("ments",-1,15),new r("ements",30,6),new r("issements",31,12),new r("ités",-1,7),new r("ment",-1,15),new r("ement",34,6),new r("issement",35,12),new r("amment",34,13),new r("emment",34,14),new r("aux",-1,10),new r("eaux",39,9),new r("eux",-1,1),new r("ité",-1,7)],l=[new r("ira",-1,1),new r("ie",-1,1),new r("isse",-1,1),new r("issante",-1,1),new r("i",-1,1),new r("irai",4,1),new r("ir",-1,1),new r("iras",-1,1),new r("ies",-1,1),new r("îmes",-1,1),new r("isses",-1,1),new r("issantes",-1,1),new r("îtes",-1,1),new r("is",-1,1),new r("irais",13,1),new r("issais",13,1),new r("irions",-1,1),new r("issions",-1,1),new r("irons",-1,1),new r("issons",-1,1),new r("issants",-1,1),new r("it",-1,1),new r("irait",21,1),new r("issait",21,1),new r("issant",-1,1),new r("iraIent",-1,1),new r("issaIent",-1,1),new r("irent",-1,1),new r("issent",-1,1),new r("iront",-1,1),new r("ît",-1,1),new r("iriez",-1,1),new r("issiez",-1,1),new r("irez",-1,1),new r("issez",-1,1)],w=[new r("a",-1,3),new r("era",0,2),new r("asse",-1,3),new r("ante",-1,3),new r("ée",-1,2),new r("ai",-1,3),new r("erai",5,2),new r("er",-1,2),new r("as",-1,3),new r("eras",8,2),new r("âmes",-1,3),new r("asses",-1,3),new r("antes",-1,3),new r("âtes",-1,3),new r("ées",-1,2),new r("ais",-1,3),new r("erais",15,2),new r("ions",-1,1),new r("erions",17,2),new r("assions",17,3),new r("erons",-1,2),new r("ants",-1,3),new r("és",-1,2),new r("ait",-1,3),new r("erait",23,2),new r("ant",-1,3),new r("aIent",-1,3),new r("eraIent",26,2),new r("èrent",-1,2),new r("assent",-1,3),new r("eront",-1,2),new r("ât",-1,3),new r("ez",-1,2),new r("iez",32,2),new r("eriez",33,2),new r("assiez",33,3),new r("erez",32,2),new r("é",-1,2)],f=[new r("e",-1,3),new r("Ière",0,2),new r("ière",0,2),new r("ion",-1,1),new r("Ier",-1,2),new r("ier",-1,2),new r("ë",-1,4)],m=[new r("ell",-1,-1),new r("eill",-1,-1),new r("enn",-1,-1),new r("onn",-1,-1),new r("ett",-1,-1)],_=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,128,130,103,8,5],b=[1,65,20,0,0,0,0,0,0,0,0,0,0,0,0,0,128],d=new y;function k(e,r,s){return!(!d.eq_s(1,e)||(d.ket=d.cursor,!d.in_grouping(_,97,251)))&&(d.slice_from(r),d.cursor=s,!0)}function p(e,r,s){return!!d.eq_s(1,e)&&(d.ket=d.cursor,d.slice_from(r),d.cursor=s,!0)}function g(){for(;!d.in_grouping(_,97,251);){if(d.cursor>=d.limit)return!0;d.cursor++}for(;!d.out_grouping(_,97,251);){if(d.cursor>=d.limit)return!0;d.cursor++}return!1}function q(){return t<=d.cursor}function v(){return i<=d.cursor}function h(){return s<=d.cursor}function z(){if(!function(){var e,r;if(d.ket=d.cursor,e=d.find_among_b(a,43)){switch(d.bra=d.cursor,e){case 1:if(!h())return!1;d.slice_del();break;case 2:if(!h())return!1;d.slice_del(),d.ket=d.cursor,d.eq_s_b(2,"ic")&&(d.bra=d.cursor,h()?d.slice_del():d.slice_from("iqU"));break;case 3:if(!h())return!1;d.slice_from("log");break;case 4:if(!h())return!1;d.slice_from("u");break;case 5:if(!h())return!1;d.slice_from("ent");break;case 6:if(!q())return!1;if(d.slice_del(),d.ket=d.cursor,e=d.find_among_b(o,6))switch(d.bra=d.cursor,e){case 1:h()&&(d.slice_del(),d.ket=d.cursor,d.eq_s_b(2,"at")&&(d.bra=d.cursor,h()&&d.slice_del()));break;case 2:h()?d.slice_del():v()&&d.slice_from("eux");break;case 3:h()&&d.slice_del();break;case 4:q()&&d.slice_from("i")}break;case 7:if(!h())return!1;if(d.slice_del(),d.ket=d.cursor,e=d.find_among_b(c,3))switch(d.bra=d.cursor,e){case 1:h()?d.slice_del():d.slice_from("abl");break;case 2:h()?d.slice_del():d.slice_from("iqU");break;case 3:h()&&d.slice_del()}break;case 8:if(!h())return!1;if(d.slice_del(),d.ket=d.cursor,d.eq_s_b(2,"at")&&(d.bra=d.cursor,h()&&(d.slice_del(),d.ket=d.cursor,d.eq_s_b(2,"ic")))){d.bra=d.cursor,h()?d.slice_del():d.slice_from("iqU");break}break;case 9:d.slice_from("eau");break;case 10:if(!v())return!1;d.slice_from("al");break;case 11:if(h())d.slice_del();else{if(!v())return!1;d.slice_from("eux")}break;case 12:if(!v()||!d.out_grouping_b(_,97,251))return!1;d.slice_del();break;case 13:return q()&&d.slice_from("ant"),!1;case 14:return q()&&d.slice_from("ent"),!1;case 15:return r=d.limit-d.cursor,d.in_grouping_b(_,97,251)&&q()&&(d.cursor=d.limit-r,d.slice_del()),!1}return!0}return!1}()&&(d.cursor=d.limit,!function(){var e,r;if(d.cursor=t){if(s=d.limit_backward,d.limit_backward=t,d.ket=d.cursor,e=d.find_among_b(f,7))switch(d.bra=d.cursor,e){case 1:if(h()){if(i=d.limit-d.cursor,!d.eq_s_b(1,"s")&&(d.cursor=d.limit-i,!d.eq_s_b(1,"t")))break;d.slice_del()}break;case 2:d.slice_from("i");break;case 3:d.slice_del();break;case 4:d.eq_s_b(2,"gu")&&d.slice_del()}d.limit_backward=s}}();d.cursor=d.limit,d.ket=d.cursor,d.eq_s_b(1,"Y")?(d.bra=d.cursor,d.slice_from("i")):(d.cursor=d.limit,d.eq_s_b(1,"ç")&&(d.bra=d.cursor,d.slice_from("c")))}this.setCurrent=function(e){d.setCurrent(e)},this.getCurrent=function(){return d.getCurrent()},this.stem=function(){var e,r=d.cursor;return function(){for(var e,r;;){if(e=d.cursor,d.in_grouping(_,97,251)){if(d.bra=d.cursor,r=d.cursor,k("u","U",e))continue;if(d.cursor=r,k("i","I",e))continue;if(d.cursor=r,p("y","Y",e))continue}if(d.cursor=e,!k("y","Y",d.bra=e)){if(d.cursor=e,d.eq_s(1,"q")&&(d.bra=d.cursor,p("u","U",e)))continue;if((d.cursor=e)>=d.limit)return;d.cursor++}}}(),d.cursor=r,function(){var e=d.cursor;if(t=d.limit,s=i=t,d.in_grouping(_,97,251)&&d.in_grouping(_,97,251)&&d.cursor=d.limit){d.cursor=t;break}d.cursor++}while(!d.in_grouping(_,97,251))}t=d.cursor,d.cursor=e,g()||(i=d.cursor,g()||(s=d.cursor))}(),d.limit_backward=r,d.cursor=d.limit,z(),d.cursor=d.limit,e=d.limit-d.cursor,d.find_among_b(m,5)&&(d.cursor=d.limit-e,d.ket=d.cursor,d.cursor>d.limit_backward&&(d.cursor--,d.bra=d.cursor,d.slice_del())),d.cursor=d.limit,function(){for(var e,r=1;d.out_grouping_b(_,97,251);)r--;if(r<=0){if(d.ket=d.cursor,e=d.limit-d.cursor,!d.eq_s_b(1,"é")&&(d.cursor=d.limit-e,!d.eq_s_b(1,"è")))return;d.bra=d.cursor,d.slice_from("e")}}(),d.cursor=d.limit_backward,function(){for(var e,r;r=d.cursor,d.bra=r,e=d.find_among(u,4);)switch(d.ket=d.cursor,e){case 1:d.slice_from("i");break;case 2:d.slice_from("u");break;case 3:d.slice_from("y");break;case 4:if(d.cursor>=d.limit)return;d.cursor++}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return s.setCurrent(e),s.stem(),s.getCurrent()}):(s.setCurrent(e),s.stem(),s.getCurrent())}),e.Pipeline.registerFunction(e.fr.stemmer,"stemmer-fr"),e.fr.stopWordFilter=e.generateStopWordFilter("ai aie aient aies ait as au aura aurai auraient aurais aurait auras aurez auriez aurions aurons auront aux avaient avais avait avec avez aviez avions avons ayant ayez ayons c ce ceci celà ces cet cette d dans de des du elle en es est et eu eue eues eurent eus eusse eussent eusses eussiez eussions eut eux eûmes eût eûtes furent fus fusse fussent fusses fussiez fussions fut fûmes fût fûtes ici il ils j je l la le les leur leurs lui m ma mais me mes moi mon même n ne nos notre nous on ont ou par pas pour qu que quel quelle quelles quels qui s sa sans se sera serai seraient serais serait seras serez seriez serions serons seront ses soi soient sois soit sommes son sont soyez soyons suis sur t ta te tes toi ton tu un une vos votre vous y à étaient étais était étant étiez étions été étée étées étés êtes".split(" ")),e.Pipeline.registerFunction(e.fr.stopWordFilter,"stopWordFilter-fr")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.ro.js: -------------------------------------------------------------------------------- 1 | !function(e,i){"function"==typeof define&&define.amd?define(i):"object"==typeof exports?module.exports=i():i()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var h,z,i;e.ro=function(){this.pipeline.reset(),this.pipeline.add(e.ro.trimmer,e.ro.stopWordFilter,e.ro.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ro.stemmer))},e.ro.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.ro.trimmer=e.trimmerSupport.generateTrimmer(e.ro.wordCharacters),e.Pipeline.registerFunction(e.ro.trimmer,"trimmer-ro"),e.ro.stemmer=(h=e.stemmerSupport.Among,z=e.stemmerSupport.SnowballProgram,i=new function(){var r,n,t,a,o=[new h("",-1,3),new h("I",0,1),new h("U",0,2)],s=[new h("ea",-1,3),new h("aţia",-1,7),new h("aua",-1,2),new h("iua",-1,4),new h("aţie",-1,7),new h("ele",-1,3),new h("ile",-1,5),new h("iile",6,4),new h("iei",-1,4),new h("atei",-1,6),new h("ii",-1,4),new h("ului",-1,1),new h("ul",-1,1),new h("elor",-1,3),new h("ilor",-1,4),new h("iilor",14,4)],c=[new h("icala",-1,4),new h("iciva",-1,4),new h("ativa",-1,5),new h("itiva",-1,6),new h("icale",-1,4),new h("aţiune",-1,5),new h("iţiune",-1,6),new h("atoare",-1,5),new h("itoare",-1,6),new h("ătoare",-1,5),new h("icitate",-1,4),new h("abilitate",-1,1),new h("ibilitate",-1,2),new h("ivitate",-1,3),new h("icive",-1,4),new h("ative",-1,5),new h("itive",-1,6),new h("icali",-1,4),new h("atori",-1,5),new h("icatori",18,4),new h("itori",-1,6),new h("ători",-1,5),new h("icitati",-1,4),new h("abilitati",-1,1),new h("ivitati",-1,3),new h("icivi",-1,4),new h("ativi",-1,5),new h("itivi",-1,6),new h("icităi",-1,4),new h("abilităi",-1,1),new h("ivităi",-1,3),new h("icităţi",-1,4),new h("abilităţi",-1,1),new h("ivităţi",-1,3),new h("ical",-1,4),new h("ator",-1,5),new h("icator",35,4),new h("itor",-1,6),new h("ător",-1,5),new h("iciv",-1,4),new h("ativ",-1,5),new h("itiv",-1,6),new h("icală",-1,4),new h("icivă",-1,4),new h("ativă",-1,5),new h("itivă",-1,6)],u=[new h("ica",-1,1),new h("abila",-1,1),new h("ibila",-1,1),new h("oasa",-1,1),new h("ata",-1,1),new h("ita",-1,1),new h("anta",-1,1),new h("ista",-1,3),new h("uta",-1,1),new h("iva",-1,1),new h("ic",-1,1),new h("ice",-1,1),new h("abile",-1,1),new h("ibile",-1,1),new h("isme",-1,3),new h("iune",-1,2),new h("oase",-1,1),new h("ate",-1,1),new h("itate",17,1),new h("ite",-1,1),new h("ante",-1,1),new h("iste",-1,3),new h("ute",-1,1),new h("ive",-1,1),new h("ici",-1,1),new h("abili",-1,1),new h("ibili",-1,1),new h("iuni",-1,2),new h("atori",-1,1),new h("osi",-1,1),new h("ati",-1,1),new h("itati",30,1),new h("iti",-1,1),new h("anti",-1,1),new h("isti",-1,3),new h("uti",-1,1),new h("işti",-1,3),new h("ivi",-1,1),new h("ităi",-1,1),new h("oşi",-1,1),new h("ităţi",-1,1),new h("abil",-1,1),new h("ibil",-1,1),new h("ism",-1,3),new h("ator",-1,1),new h("os",-1,1),new h("at",-1,1),new h("it",-1,1),new h("ant",-1,1),new h("ist",-1,3),new h("ut",-1,1),new h("iv",-1,1),new h("ică",-1,1),new h("abilă",-1,1),new h("ibilă",-1,1),new h("oasă",-1,1),new h("ată",-1,1),new h("ită",-1,1),new h("antă",-1,1),new h("istă",-1,3),new h("ută",-1,1),new h("ivă",-1,1)],w=[new h("ea",-1,1),new h("ia",-1,1),new h("esc",-1,1),new h("ăsc",-1,1),new h("ind",-1,1),new h("ând",-1,1),new h("are",-1,1),new h("ere",-1,1),new h("ire",-1,1),new h("âre",-1,1),new h("se",-1,2),new h("ase",10,1),new h("sese",10,2),new h("ise",10,1),new h("use",10,1),new h("âse",10,1),new h("eşte",-1,1),new h("ăşte",-1,1),new h("eze",-1,1),new h("ai",-1,1),new h("eai",19,1),new h("iai",19,1),new h("sei",-1,2),new h("eşti",-1,1),new h("ăşti",-1,1),new h("ui",-1,1),new h("ezi",-1,1),new h("âi",-1,1),new h("aşi",-1,1),new h("seşi",-1,2),new h("aseşi",29,1),new h("seseşi",29,2),new h("iseşi",29,1),new h("useşi",29,1),new h("âseşi",29,1),new h("işi",-1,1),new h("uşi",-1,1),new h("âşi",-1,1),new h("aţi",-1,2),new h("eaţi",38,1),new h("iaţi",38,1),new h("eţi",-1,2),new h("iţi",-1,2),new h("âţi",-1,2),new h("arăţi",-1,1),new h("serăţi",-1,2),new h("aserăţi",45,1),new h("seserăţi",45,2),new h("iserăţi",45,1),new h("userăţi",45,1),new h("âserăţi",45,1),new h("irăţi",-1,1),new h("urăţi",-1,1),new h("ârăţi",-1,1),new h("am",-1,1),new h("eam",54,1),new h("iam",54,1),new h("em",-1,2),new h("asem",57,1),new h("sesem",57,2),new h("isem",57,1),new h("usem",57,1),new h("âsem",57,1),new h("im",-1,2),new h("âm",-1,2),new h("ăm",-1,2),new h("arăm",65,1),new h("serăm",65,2),new h("aserăm",67,1),new h("seserăm",67,2),new h("iserăm",67,1),new h("userăm",67,1),new h("âserăm",67,1),new h("irăm",65,1),new h("urăm",65,1),new h("ârăm",65,1),new h("au",-1,1),new h("eau",76,1),new h("iau",76,1),new h("indu",-1,1),new h("ându",-1,1),new h("ez",-1,1),new h("ească",-1,1),new h("ară",-1,1),new h("seră",-1,2),new h("aseră",84,1),new h("seseră",84,2),new h("iseră",84,1),new h("useră",84,1),new h("âseră",84,1),new h("iră",-1,1),new h("ură",-1,1),new h("âră",-1,1),new h("ează",-1,1)],i=[new h("a",-1,1),new h("e",-1,1),new h("ie",1,1),new h("i",-1,1),new h("ă",-1,1)],m=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,2,32,0,0,4],l=new z;function f(e,i){l.eq_s(1,e)&&(l.ket=l.cursor,l.in_grouping(m,97,259)&&l.slice_from(i))}function p(){if(l.out_grouping(m,97,259)){for(;!l.in_grouping(m,97,259);){if(l.cursor>=l.limit)return!0;l.cursor++}return!1}return!0}function d(){var e,i,r=l.cursor;if(l.in_grouping(m,97,259)){if(e=l.cursor,!p())return void(a=l.cursor);if(l.cursor=e,!function(){if(l.in_grouping(m,97,259))for(;!l.out_grouping(m,97,259);){if(l.cursor>=l.limit)return!0;l.cursor++}return!1}())return void(a=l.cursor)}l.cursor=r,l.out_grouping(m,97,259)&&(i=l.cursor,p()&&(l.cursor=i,l.in_grouping(m,97,259)&&l.cursor=l.limit)return!1;l.cursor++}for(;!l.out_grouping(m,97,259);){if(l.cursor>=l.limit)return!1;l.cursor++}return!0}function v(){return t<=l.cursor}function _(){var e,i=l.limit-l.cursor;if(l.ket=l.cursor,(e=l.find_among_b(c,46))&&(l.bra=l.cursor,v())){switch(e){case 1:l.slice_from("abil");break;case 2:l.slice_from("ibil");break;case 3:l.slice_from("iv");break;case 4:l.slice_from("ic");break;case 5:l.slice_from("at");break;case 6:l.slice_from("it")}return r=!0,l.cursor=l.limit-i,!0}return!1}function g(){var e,i;for(r=!1;;)if(i=l.limit-l.cursor,!_()){l.cursor=l.limit-i;break}if(l.ket=l.cursor,(e=l.find_among_b(u,62))&&(l.bra=l.cursor,n<=l.cursor)){switch(e){case 1:l.slice_del();break;case 2:l.eq_s_b(1,"ţ")&&(l.bra=l.cursor,l.slice_from("t"));break;case 3:l.slice_from("ist")}r=!0}}function k(){var e;l.ket=l.cursor,(e=l.find_among_b(i,5))&&(l.bra=l.cursor,a<=l.cursor&&1==e&&l.slice_del())}this.setCurrent=function(e){l.setCurrent(e)},this.getCurrent=function(){return l.getCurrent()},this.stem=function(){var e,i=l.cursor;return function(){for(var e,i;e=l.cursor,l.in_grouping(m,97,259)&&(i=l.cursor,l.bra=i,f("u","U"),l.cursor=i,f("i","I")),l.cursor=e,!(l.cursor>=l.limit);)l.cursor++}(),l.cursor=i,e=l.cursor,a=l.limit,n=t=a,d(),l.cursor=e,b()&&(t=l.cursor,b()&&(n=l.cursor)),l.limit_backward=i,l.cursor=l.limit,function(){var e,i;if(l.ket=l.cursor,(e=l.find_among_b(s,16))&&(l.bra=l.cursor,v()))switch(e){case 1:l.slice_del();break;case 2:l.slice_from("a");break;case 3:l.slice_from("e");break;case 4:l.slice_from("i");break;case 5:i=l.limit-l.cursor,l.eq_s_b(2,"ab")||(l.cursor=l.limit-i,l.slice_from("i"));break;case 6:l.slice_from("at");break;case 7:l.slice_from("aţi")}}(),l.cursor=l.limit,g(),l.cursor=l.limit,r||(l.cursor=l.limit,function(){var e,i,r;if(l.cursor>=a){if(i=l.limit_backward,l.limit_backward=a,l.ket=l.cursor,e=l.find_among_b(w,94))switch(l.bra=l.cursor,e){case 1:if(r=l.limit-l.cursor,!l.out_grouping_b(m,97,259)&&(l.cursor=l.limit-r,!l.eq_s_b(1,"u")))break;case 2:l.slice_del()}l.limit_backward=i}}(),l.cursor=l.limit),k(),l.cursor=l.limit_backward,function(){for(var e;;){if(l.bra=l.cursor,e=l.find_among(o,3))switch(l.ket=l.cursor,e){case 1:l.slice_from("i");continue;case 2:l.slice_from("u");continue;case 3:if(l.cursor>=l.limit)break;l.cursor++;continue}break}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}),e.Pipeline.registerFunction(e.ro.stemmer,"stemmer-ro"),e.ro.stopWordFilter=e.generateStopWordFilter("acea aceasta această aceea acei aceia acel acela acele acelea acest acesta aceste acestea aceşti aceştia acolo acord acum ai aia aibă aici al ale alea altceva altcineva am ar are asemenea asta astea astăzi asupra au avea avem aveţi azi aş aşadar aţi bine bucur bună ca care caut ce cel ceva chiar cinci cine cineva contra cu cum cumva curând curînd când cât câte câtva câţi cînd cît cîte cîtva cîţi că căci cărei căror cărui către da dacă dar datorită dată dau de deci deja deoarece departe deşi din dinaintea dintr- dintre doi doilea două drept după dă ea ei el ele eram este eu eşti face fata fi fie fiecare fii fim fiu fiţi frumos fără graţie halbă iar ieri la le li lor lui lângă lîngă mai mea mei mele mereu meu mi mie mine mult multă mulţi mulţumesc mâine mîine mă ne nevoie nici nicăieri nimeni nimeri nimic nişte noastre noastră noi noroc nostru nouă noştri nu opt ori oricare orice oricine oricum oricând oricât oricînd oricît oriunde patra patru patrulea pe pentru peste pic poate pot prea prima primul prin puţin puţina puţină până pînă rog sa sale sau se spate spre sub sunt suntem sunteţi sută sînt sîntem sînteţi să săi său ta tale te timp tine toate toată tot totuşi toţi trei treia treilea tu tăi tău un una unde undeva unei uneia unele uneori unii unor unora unu unui unuia unul vi voastre voastră voi vostru vouă voştri vreme vreo vreun vă zece zero zi zice îi îl îmi împotriva în înainte înaintea încotro încât încît între întrucât întrucît îţi ăla ălea ăsta ăstea ăştia şapte şase şi ştiu ţi ţie".split(" ")),e.Pipeline.registerFunction(e.ro.stopWordFilter,"stopWordFilter-ro")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.it.js: -------------------------------------------------------------------------------- 1 | !function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var z,P,r;e.it=function(){this.pipeline.reset(),this.pipeline.add(e.it.trimmer,e.it.stopWordFilter,e.it.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.it.stemmer))},e.it.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.it.trimmer=e.trimmerSupport.generateTrimmer(e.it.wordCharacters),e.Pipeline.registerFunction(e.it.trimmer,"trimmer-it"),e.it.stemmer=(z=e.stemmerSupport.Among,P=e.stemmerSupport.SnowballProgram,r=new function(){var o,t,s,a=[new z("",-1,7),new z("qu",0,6),new z("á",0,1),new z("é",0,2),new z("í",0,3),new z("ó",0,4),new z("ú",0,5)],u=[new z("",-1,3),new z("I",0,1),new z("U",0,2)],c=[new z("la",-1,-1),new z("cela",0,-1),new z("gliela",0,-1),new z("mela",0,-1),new z("tela",0,-1),new z("vela",0,-1),new z("le",-1,-1),new z("cele",6,-1),new z("gliele",6,-1),new z("mele",6,-1),new z("tele",6,-1),new z("vele",6,-1),new z("ne",-1,-1),new z("cene",12,-1),new z("gliene",12,-1),new z("mene",12,-1),new z("sene",12,-1),new z("tene",12,-1),new z("vene",12,-1),new z("ci",-1,-1),new z("li",-1,-1),new z("celi",20,-1),new z("glieli",20,-1),new z("meli",20,-1),new z("teli",20,-1),new z("veli",20,-1),new z("gli",20,-1),new z("mi",-1,-1),new z("si",-1,-1),new z("ti",-1,-1),new z("vi",-1,-1),new z("lo",-1,-1),new z("celo",31,-1),new z("glielo",31,-1),new z("melo",31,-1),new z("telo",31,-1),new z("velo",31,-1)],w=[new z("ando",-1,1),new z("endo",-1,1),new z("ar",-1,2),new z("er",-1,2),new z("ir",-1,2)],r=[new z("ic",-1,-1),new z("abil",-1,-1),new z("os",-1,-1),new z("iv",-1,1)],n=[new z("ic",-1,1),new z("abil",-1,1),new z("iv",-1,1)],i=[new z("ica",-1,1),new z("logia",-1,3),new z("osa",-1,1),new z("ista",-1,1),new z("iva",-1,9),new z("anza",-1,1),new z("enza",-1,5),new z("ice",-1,1),new z("atrice",7,1),new z("iche",-1,1),new z("logie",-1,3),new z("abile",-1,1),new z("ibile",-1,1),new z("usione",-1,4),new z("azione",-1,2),new z("uzione",-1,4),new z("atore",-1,2),new z("ose",-1,1),new z("ante",-1,1),new z("mente",-1,1),new z("amente",19,7),new z("iste",-1,1),new z("ive",-1,9),new z("anze",-1,1),new z("enze",-1,5),new z("ici",-1,1),new z("atrici",25,1),new z("ichi",-1,1),new z("abili",-1,1),new z("ibili",-1,1),new z("ismi",-1,1),new z("usioni",-1,4),new z("azioni",-1,2),new z("uzioni",-1,4),new z("atori",-1,2),new z("osi",-1,1),new z("anti",-1,1),new z("amenti",-1,6),new z("imenti",-1,6),new z("isti",-1,1),new z("ivi",-1,9),new z("ico",-1,1),new z("ismo",-1,1),new z("oso",-1,1),new z("amento",-1,6),new z("imento",-1,6),new z("ivo",-1,9),new z("ità",-1,8),new z("istà",-1,1),new z("istè",-1,1),new z("istì",-1,1)],l=[new z("isca",-1,1),new z("enda",-1,1),new z("ata",-1,1),new z("ita",-1,1),new z("uta",-1,1),new z("ava",-1,1),new z("eva",-1,1),new z("iva",-1,1),new z("erebbe",-1,1),new z("irebbe",-1,1),new z("isce",-1,1),new z("ende",-1,1),new z("are",-1,1),new z("ere",-1,1),new z("ire",-1,1),new z("asse",-1,1),new z("ate",-1,1),new z("avate",16,1),new z("evate",16,1),new z("ivate",16,1),new z("ete",-1,1),new z("erete",20,1),new z("irete",20,1),new z("ite",-1,1),new z("ereste",-1,1),new z("ireste",-1,1),new z("ute",-1,1),new z("erai",-1,1),new z("irai",-1,1),new z("isci",-1,1),new z("endi",-1,1),new z("erei",-1,1),new z("irei",-1,1),new z("assi",-1,1),new z("ati",-1,1),new z("iti",-1,1),new z("eresti",-1,1),new z("iresti",-1,1),new z("uti",-1,1),new z("avi",-1,1),new z("evi",-1,1),new z("ivi",-1,1),new z("isco",-1,1),new z("ando",-1,1),new z("endo",-1,1),new z("Yamo",-1,1),new z("iamo",-1,1),new z("avamo",-1,1),new z("evamo",-1,1),new z("ivamo",-1,1),new z("eremo",-1,1),new z("iremo",-1,1),new z("assimo",-1,1),new z("ammo",-1,1),new z("emmo",-1,1),new z("eremmo",54,1),new z("iremmo",54,1),new z("immo",-1,1),new z("ano",-1,1),new z("iscano",58,1),new z("avano",58,1),new z("evano",58,1),new z("ivano",58,1),new z("eranno",-1,1),new z("iranno",-1,1),new z("ono",-1,1),new z("iscono",65,1),new z("arono",65,1),new z("erono",65,1),new z("irono",65,1),new z("erebbero",-1,1),new z("irebbero",-1,1),new z("assero",-1,1),new z("essero",-1,1),new z("issero",-1,1),new z("ato",-1,1),new z("ito",-1,1),new z("uto",-1,1),new z("avo",-1,1),new z("evo",-1,1),new z("ivo",-1,1),new z("ar",-1,1),new z("ir",-1,1),new z("erà",-1,1),new z("irà",-1,1),new z("erò",-1,1),new z("irò",-1,1)],m=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,128,128,8,2,1],f=[17,65,0,0,0,0,0,0,0,0,0,0,0,0,0,128,128,8,2],v=[17],b=new P;function d(e,r,n){return!(!b.eq_s(1,e)||(b.ket=b.cursor,!b.in_grouping(m,97,249)))&&(b.slice_from(r),b.cursor=n,!0)}function _(e){if(b.cursor=e,!b.in_grouping(m,97,249))return!1;for(;!b.out_grouping(m,97,249);){if(b.cursor>=b.limit)return!1;b.cursor++}return!0}function g(){var e,r=b.cursor;if(!function(){if(b.in_grouping(m,97,249)){var e=b.cursor;if(b.out_grouping(m,97,249)){for(;!b.in_grouping(m,97,249);){if(b.cursor>=b.limit)return _(e);b.cursor++}return!0}return _(e)}return!1}()){if(b.cursor=r,!b.out_grouping(m,97,249))return;if(e=b.cursor,b.out_grouping(m,97,249)){for(;!b.in_grouping(m,97,249);){if(b.cursor>=b.limit)return b.cursor=e,void(b.in_grouping(m,97,249)&&b.cursor=b.limit)return;b.cursor++}s=b.cursor}function p(){for(;!b.in_grouping(m,97,249);){if(b.cursor>=b.limit)return!1;b.cursor++}for(;!b.out_grouping(m,97,249);){if(b.cursor>=b.limit)return!1;b.cursor++}return!0}function k(){return s<=b.cursor}function h(){return o<=b.cursor}function q(){var e;if(b.ket=b.cursor,!(e=b.find_among_b(i,51)))return!1;switch(b.bra=b.cursor,e){case 1:if(!h())return!1;b.slice_del();break;case 2:if(!h())return!1;b.slice_del(),b.ket=b.cursor,b.eq_s_b(2,"ic")&&(b.bra=b.cursor,h()&&b.slice_del());break;case 3:if(!h())return!1;b.slice_from("log");break;case 4:if(!h())return!1;b.slice_from("u");break;case 5:if(!h())return!1;b.slice_from("ente");break;case 6:if(!k())return!1;b.slice_del();break;case 7:if(!(t<=b.cursor))return!1;b.slice_del(),b.ket=b.cursor,(e=b.find_among_b(r,4))&&(b.bra=b.cursor,h()&&(b.slice_del(),1==e&&(b.ket=b.cursor,b.eq_s_b(2,"at")&&(b.bra=b.cursor,h()&&b.slice_del()))));break;case 8:if(!h())return!1;b.slice_del(),b.ket=b.cursor,(e=b.find_among_b(n,3))&&(b.bra=b.cursor,1==e&&h()&&b.slice_del());break;case 9:if(!h())return!1;b.slice_del(),b.ket=b.cursor,b.eq_s_b(2,"at")&&(b.bra=b.cursor,h()&&(b.slice_del(),b.ket=b.cursor,b.eq_s_b(2,"ic")&&(b.bra=b.cursor,h()&&b.slice_del())))}return!0}function C(){var e;e=b.limit-b.cursor,b.ket=b.cursor,b.in_grouping_b(f,97,242)&&(b.bra=b.cursor,k()&&(b.slice_del(),b.ket=b.cursor,b.eq_s_b(1,"i")&&(b.bra=b.cursor,k())))?b.slice_del():b.cursor=b.limit-e,b.ket=b.cursor,b.eq_s_b(1,"h")&&(b.bra=b.cursor,b.in_grouping_b(v,99,103)&&k()&&b.slice_del())}this.setCurrent=function(e){b.setCurrent(e)},this.getCurrent=function(){return b.getCurrent()},this.stem=function(){var e,r,n,i=b.cursor;return function(){for(var e,r,n,i,o=b.cursor;;){if(b.bra=b.cursor,e=b.find_among(a,7))switch(b.ket=b.cursor,e){case 1:b.slice_from("à");continue;case 2:b.slice_from("è");continue;case 3:b.slice_from("ì");continue;case 4:b.slice_from("ò");continue;case 5:b.slice_from("ù");continue;case 6:b.slice_from("qU");continue;case 7:if(b.cursor>=b.limit)break;b.cursor++;continue}break}for(b.cursor=o;;)for(r=b.cursor;;){if(n=b.cursor,b.in_grouping(m,97,249)){if(b.bra=b.cursor,i=b.cursor,d("u","U",n))break;if(b.cursor=i,d("i","I",n))break}if(b.cursor=n,b.cursor>=b.limit)return b.cursor=r;b.cursor++}}(),b.cursor=i,e=b.cursor,s=b.limit,o=t=s,g(),b.cursor=e,p()&&(t=b.cursor,p()&&(o=b.cursor)),b.limit_backward=i,b.cursor=b.limit,function(){var e;if(b.ket=b.cursor,b.find_among_b(c,37)&&(b.bra=b.cursor,(e=b.find_among_b(w,5))&&k()))switch(e){case 1:b.slice_del();break;case 2:b.slice_from("e")}}(),b.cursor=b.limit,q()||(b.cursor=b.limit,b.cursor>=s&&(n=b.limit_backward,b.limit_backward=s,b.ket=b.cursor,(r=b.find_among_b(l,87))&&(b.bra=b.cursor,1==r&&b.slice_del()),b.limit_backward=n)),b.cursor=b.limit,C(),b.cursor=b.limit_backward,function(){for(var e;b.bra=b.cursor,e=b.find_among(u,3);)switch(b.ket=b.cursor,e){case 1:b.slice_from("i");break;case 2:b.slice_from("u");break;case 3:if(b.cursor>=b.limit)return;b.cursor++}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return r.setCurrent(e),r.stem(),r.getCurrent()}):(r.setCurrent(e),r.stem(),r.getCurrent())}),e.Pipeline.registerFunction(e.it.stemmer,"stemmer-it"),e.it.stopWordFilter=e.generateStopWordFilter("a abbia abbiamo abbiano abbiate ad agl agli ai al all alla alle allo anche avemmo avendo avesse avessero avessi avessimo aveste avesti avete aveva avevamo avevano avevate avevi avevo avrai avranno avrebbe avrebbero avrei avremmo avremo avreste avresti avrete avrà avrò avuta avute avuti avuto c che chi ci coi col come con contro cui da dagl dagli dai dal dall dalla dalle dallo degl degli dei del dell della delle dello di dov dove e ebbe ebbero ebbi ed era erano eravamo eravate eri ero essendo faccia facciamo facciano facciate faccio facemmo facendo facesse facessero facessi facessimo faceste facesti faceva facevamo facevano facevate facevi facevo fai fanno farai faranno farebbe farebbero farei faremmo faremo fareste faresti farete farà farò fece fecero feci fosse fossero fossi fossimo foste fosti fu fui fummo furono gli ha hai hanno ho i il in io l la le lei li lo loro lui ma mi mia mie miei mio ne negl negli nei nel nell nella nelle nello noi non nostra nostre nostri nostro o per perché più quale quanta quante quanti quanto quella quelle quelli quello questa queste questi questo sarai saranno sarebbe sarebbero sarei saremmo saremo sareste saresti sarete sarà sarò se sei si sia siamo siano siate siete sono sta stai stando stanno starai staranno starebbe starebbero starei staremmo staremo stareste staresti starete starà starò stava stavamo stavano stavate stavi stavo stemmo stesse stessero stessi stessimo steste stesti stette stettero stetti stia stiamo stiano stiate sto su sua sue sugl sugli sui sul sull sulla sulle sullo suo suoi ti tra tu tua tue tuo tuoi tutti tutto un una uno vi voi vostra vostre vostri vostro è".split(" ")),e.Pipeline.registerFunction(e.it.stopWordFilter,"stopWordFilter-it")}}); -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.es.js: -------------------------------------------------------------------------------- 1 | !function(e,s){"function"==typeof define&&define.amd?define(s):"object"==typeof exports?module.exports=s():s()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var C,P,s;e.es=function(){this.pipeline.reset(),this.pipeline.add(e.es.trimmer,e.es.stopWordFilter,e.es.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.es.stemmer))},e.es.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.es.trimmer=e.trimmerSupport.generateTrimmer(e.es.wordCharacters),e.Pipeline.registerFunction(e.es.trimmer,"trimmer-es"),e.es.stemmer=(C=e.stemmerSupport.Among,P=e.stemmerSupport.SnowballProgram,s=new function(){var r,n,i,a=[new C("",-1,6),new C("á",0,1),new C("é",0,2),new C("í",0,3),new C("ó",0,4),new C("ú",0,5)],t=[new C("la",-1,-1),new C("sela",0,-1),new C("le",-1,-1),new C("me",-1,-1),new C("se",-1,-1),new C("lo",-1,-1),new C("selo",5,-1),new C("las",-1,-1),new C("selas",7,-1),new C("les",-1,-1),new C("los",-1,-1),new C("selos",10,-1),new C("nos",-1,-1)],o=[new C("ando",-1,6),new C("iendo",-1,6),new C("yendo",-1,7),new C("ándo",-1,2),new C("iéndo",-1,1),new C("ar",-1,6),new C("er",-1,6),new C("ir",-1,6),new C("ár",-1,3),new C("ér",-1,4),new C("ír",-1,5)],s=[new C("ic",-1,-1),new C("ad",-1,-1),new C("os",-1,-1),new C("iv",-1,1)],u=[new C("able",-1,1),new C("ible",-1,1),new C("ante",-1,1)],w=[new C("ic",-1,1),new C("abil",-1,1),new C("iv",-1,1)],c=[new C("ica",-1,1),new C("ancia",-1,2),new C("encia",-1,5),new C("adora",-1,2),new C("osa",-1,1),new C("ista",-1,1),new C("iva",-1,9),new C("anza",-1,1),new C("logía",-1,3),new C("idad",-1,8),new C("able",-1,1),new C("ible",-1,1),new C("ante",-1,2),new C("mente",-1,7),new C("amente",13,6),new C("ación",-1,2),new C("ución",-1,4),new C("ico",-1,1),new C("ismo",-1,1),new C("oso",-1,1),new C("amiento",-1,1),new C("imiento",-1,1),new C("ivo",-1,9),new C("ador",-1,2),new C("icas",-1,1),new C("ancias",-1,2),new C("encias",-1,5),new C("adoras",-1,2),new C("osas",-1,1),new C("istas",-1,1),new C("ivas",-1,9),new C("anzas",-1,1),new C("logías",-1,3),new C("idades",-1,8),new C("ables",-1,1),new C("ibles",-1,1),new C("aciones",-1,2),new C("uciones",-1,4),new C("adores",-1,2),new C("antes",-1,2),new C("icos",-1,1),new C("ismos",-1,1),new C("osos",-1,1),new C("amientos",-1,1),new C("imientos",-1,1),new C("ivos",-1,9)],m=[new C("ya",-1,1),new C("ye",-1,1),new C("yan",-1,1),new C("yen",-1,1),new C("yeron",-1,1),new C("yendo",-1,1),new C("yo",-1,1),new C("yas",-1,1),new C("yes",-1,1),new C("yais",-1,1),new C("yamos",-1,1),new C("yó",-1,1)],l=[new C("aba",-1,2),new C("ada",-1,2),new C("ida",-1,2),new C("ara",-1,2),new C("iera",-1,2),new C("ía",-1,2),new C("aría",5,2),new C("ería",5,2),new C("iría",5,2),new C("ad",-1,2),new C("ed",-1,2),new C("id",-1,2),new C("ase",-1,2),new C("iese",-1,2),new C("aste",-1,2),new C("iste",-1,2),new C("an",-1,2),new C("aban",16,2),new C("aran",16,2),new C("ieran",16,2),new C("ían",16,2),new C("arían",20,2),new C("erían",20,2),new C("irían",20,2),new C("en",-1,1),new C("asen",24,2),new C("iesen",24,2),new C("aron",-1,2),new C("ieron",-1,2),new C("arán",-1,2),new C("erán",-1,2),new C("irán",-1,2),new C("ado",-1,2),new C("ido",-1,2),new C("ando",-1,2),new C("iendo",-1,2),new C("ar",-1,2),new C("er",-1,2),new C("ir",-1,2),new C("as",-1,2),new C("abas",39,2),new C("adas",39,2),new C("idas",39,2),new C("aras",39,2),new C("ieras",39,2),new C("ías",39,2),new C("arías",45,2),new C("erías",45,2),new C("irías",45,2),new C("es",-1,1),new C("ases",49,2),new C("ieses",49,2),new C("abais",-1,2),new C("arais",-1,2),new C("ierais",-1,2),new C("íais",-1,2),new C("aríais",55,2),new C("eríais",55,2),new C("iríais",55,2),new C("aseis",-1,2),new C("ieseis",-1,2),new C("asteis",-1,2),new C("isteis",-1,2),new C("áis",-1,2),new C("éis",-1,1),new C("aréis",64,2),new C("eréis",64,2),new C("iréis",64,2),new C("ados",-1,2),new C("idos",-1,2),new C("amos",-1,2),new C("ábamos",70,2),new C("áramos",70,2),new C("iéramos",70,2),new C("íamos",70,2),new C("aríamos",74,2),new C("eríamos",74,2),new C("iríamos",74,2),new C("emos",-1,1),new C("aremos",78,2),new C("eremos",78,2),new C("iremos",78,2),new C("ásemos",78,2),new C("iésemos",78,2),new C("imos",-1,2),new C("arás",-1,2),new C("erás",-1,2),new C("irás",-1,2),new C("ís",-1,2),new C("ará",-1,2),new C("erá",-1,2),new C("irá",-1,2),new C("aré",-1,2),new C("eré",-1,2),new C("iré",-1,2),new C("ió",-1,2)],d=[new C("a",-1,1),new C("e",-1,2),new C("o",-1,1),new C("os",-1,1),new C("á",-1,1),new C("é",-1,2),new C("í",-1,1),new C("ó",-1,1)],b=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,1,17,4,10],f=new P;function _(){if(f.out_grouping(b,97,252)){for(;!f.in_grouping(b,97,252);){if(f.cursor>=f.limit)return!0;f.cursor++}return!1}return!0}function h(){var e,s=f.cursor;if(function(){if(f.in_grouping(b,97,252)){var e=f.cursor;if(_()){if(f.cursor=e,!f.in_grouping(b,97,252))return!0;for(;!f.out_grouping(b,97,252);){if(f.cursor>=f.limit)return!0;f.cursor++}}return!1}return!0}()){if(f.cursor=s,!f.out_grouping(b,97,252))return;if(e=f.cursor,_()){if(f.cursor=e,!f.in_grouping(b,97,252)||f.cursor>=f.limit)return;f.cursor++}}i=f.cursor}function v(){for(;!f.in_grouping(b,97,252);){if(f.cursor>=f.limit)return!1;f.cursor++}for(;!f.out_grouping(b,97,252);){if(f.cursor>=f.limit)return!1;f.cursor++}return!0}function p(){return i<=f.cursor}function g(){return r<=f.cursor}function k(e,s){if(!g())return!0;f.slice_del(),f.ket=f.cursor;var r=f.find_among_b(e,s);return r&&(f.bra=f.cursor,1==r&&g()&&f.slice_del()),!1}function y(e){return!g()||(f.slice_del(),f.ket=f.cursor,f.eq_s_b(2,e)&&(f.bra=f.cursor,g()&&f.slice_del()),!1)}function q(){var e;if(f.ket=f.cursor,e=f.find_among_b(c,46)){switch(f.bra=f.cursor,e){case 1:if(!g())return!1;f.slice_del();break;case 2:if(y("ic"))return!1;break;case 3:if(!g())return!1;f.slice_from("log");break;case 4:if(!g())return!1;f.slice_from("u");break;case 5:if(!g())return!1;f.slice_from("ente");break;case 6:if(!(n<=f.cursor))return!1;f.slice_del(),f.ket=f.cursor,(e=f.find_among_b(s,4))&&(f.bra=f.cursor,g()&&(f.slice_del(),1==e&&(f.ket=f.cursor,f.eq_s_b(2,"at")&&(f.bra=f.cursor,g()&&f.slice_del()))));break;case 7:if(k(u,3))return!1;break;case 8:if(k(w,3))return!1;break;case 9:if(y("at"))return!1}return!0}return!1}this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var e,s=f.cursor;return e=f.cursor,i=f.limit,r=n=i,h(),f.cursor=e,v()&&(n=f.cursor,v()&&(r=f.cursor)),f.limit_backward=s,f.cursor=f.limit,function(){var e;if(f.ket=f.cursor,f.find_among_b(t,13)&&(f.bra=f.cursor,(e=f.find_among_b(o,11))&&p()))switch(e){case 1:f.bra=f.cursor,f.slice_from("iendo");break;case 2:f.bra=f.cursor,f.slice_from("ando");break;case 3:f.bra=f.cursor,f.slice_from("ar");break;case 4:f.bra=f.cursor,f.slice_from("er");break;case 5:f.bra=f.cursor,f.slice_from("ir");break;case 6:f.slice_del();break;case 7:f.eq_s_b(1,"u")&&f.slice_del()}}(),f.cursor=f.limit,q()||(f.cursor=f.limit,function(){var e,s;if(f.cursor>=i&&(s=f.limit_backward,f.limit_backward=i,f.ket=f.cursor,e=f.find_among_b(m,12),f.limit_backward=s,e)){if(f.bra=f.cursor,1==e){if(!f.eq_s_b(1,"u"))return!1;f.slice_del()}return!0}return!1}()||(f.cursor=f.limit,function(){var e,s,r,n;if(f.cursor>=i&&(s=f.limit_backward,f.limit_backward=i,f.ket=f.cursor,e=f.find_among_b(l,96),f.limit_backward=s,e))switch(f.bra=f.cursor,e){case 1:r=f.limit-f.cursor,f.eq_s_b(1,"u")?(n=f.limit-f.cursor,f.eq_s_b(1,"g")?f.cursor=f.limit-n:f.cursor=f.limit-r):f.cursor=f.limit-r,f.bra=f.cursor;case 2:f.slice_del()}}())),f.cursor=f.limit,function(){var e,s;if(f.ket=f.cursor,e=f.find_among_b(d,8))switch(f.bra=f.cursor,e){case 1:p()&&f.slice_del();break;case 2:p()&&(f.slice_del(),f.ket=f.cursor,f.eq_s_b(1,"u")&&(f.bra=f.cursor,s=f.limit-f.cursor,f.eq_s_b(1,"g")&&(f.cursor=f.limit-s,p()&&f.slice_del())))}}(),f.cursor=f.limit_backward,function(){for(var e;;){if(f.bra=f.cursor,e=f.find_among(a,6))switch(f.ket=f.cursor,e){case 1:f.slice_from("a");continue;case 2:f.slice_from("e");continue;case 3:f.slice_from("i");continue;case 4:f.slice_from("o");continue;case 5:f.slice_from("u");continue;case 6:if(f.cursor>=f.limit)break;f.cursor++;continue}break}}(),!0}},function(e){return"function"==typeof e.update?e.update(function(e){return s.setCurrent(e),s.stem(),s.getCurrent()}):(s.setCurrent(e),s.stem(),s.getCurrent())}),e.Pipeline.registerFunction(e.es.stemmer,"stemmer-es"),e.es.stopWordFilter=e.generateStopWordFilter("a al algo algunas algunos ante antes como con contra cual cuando de del desde donde durante e el ella ellas ellos en entre era erais eran eras eres es esa esas ese eso esos esta estaba estabais estaban estabas estad estada estadas estado estados estamos estando estar estaremos estará estarán estarás estaré estaréis estaría estaríais estaríamos estarían estarías estas este estemos esto estos estoy estuve estuviera estuvierais estuvieran estuvieras estuvieron estuviese estuvieseis estuviesen estuvieses estuvimos estuviste estuvisteis estuviéramos estuviésemos estuvo está estábamos estáis están estás esté estéis estén estés fue fuera fuerais fueran fueras fueron fuese fueseis fuesen fueses fui fuimos fuiste fuisteis fuéramos fuésemos ha habida habidas habido habidos habiendo habremos habrá habrán habrás habré habréis habría habríais habríamos habrían habrías habéis había habíais habíamos habían habías han has hasta hay haya hayamos hayan hayas hayáis he hemos hube hubiera hubierais hubieran hubieras hubieron hubiese hubieseis hubiesen hubieses hubimos hubiste hubisteis hubiéramos hubiésemos hubo la las le les lo los me mi mis mucho muchos muy más mí mía mías mío míos nada ni no nos nosotras nosotros nuestra nuestras nuestro nuestros o os otra otras otro otros para pero poco por porque que quien quienes qué se sea seamos sean seas seremos será serán serás seré seréis sería seríais seríamos serían serías seáis sido siendo sin sobre sois somos son soy su sus suya suyas suyo suyos sí también tanto te tendremos tendrá tendrán tendrás tendré tendréis tendría tendríais tendríamos tendrían tendrías tened tenemos tenga tengamos tengan tengas tengo tengáis tenida tenidas tenido tenidos teniendo tenéis tenía teníais teníamos tenían tenías ti tiene tienen tienes todo todos tu tus tuve tuviera tuvierais tuvieran tuvieras tuvieron tuviese tuvieseis tuviesen tuvieses tuvimos tuviste tuvisteis tuviéramos tuviésemos tuvo tuya tuyas tuyo tuyos tú un una uno unos vosotras vosotros vuestra vuestras vuestro vuestros y ya yo él éramos".split(" ")),e.Pipeline.registerFunction(e.es.stopWordFilter,"stopWordFilter-es")}}); -------------------------------------------------------------------------------- /nlpdocument/docs/index.md: -------------------------------------------------------------------------------- 1 | # NLP and Deep Learning Tricks 2 | This repository aims to keep track of some practical and theoretical tricks in natural language processing (NLP) / deep learning / machine learning, etc. Most of these tricks are summarized by members of our group, while some others are borrowed from open-source sites. 3 | 4 | 5 | ## Data processing 6 | ### Data check 7 | Do remember that carefully checking all the data is the most important preliminary step before building your model. Generally in NLP, we would check following things: 8 | * whether dirty data exixts (e.g., unreadable characters, incomplete key-value pairs) 9 | * what the max/min/avg lengths of input texts and output texts are 10 | * whether the vocabulary file is correcltly built 11 | * whether the data format at the last step before inputting into the model is actually as you what expect (**very important!**) 12 | 13 | ### Data chunking 14 | If the data is too large so that it cannot be loaded into the memory at once, we need to chunk the data at this time. Concretely, we split the whole data into certain number of chunk files and store on disk, then maintain a queue for reading chunk files into memory. The dequeue operation takes out certain number of chunk data for building data batches, the enqueue operation reads chunk files into memory one by one from disk to keep the queue full. Here, we provide a class ```DataBatcher``` as a possible implementation: 15 | ``` python 16 | # -*- coding: utf-8 -*- 17 | import numpy as np 18 | import pickle 19 | import time 20 | from queue import Queue 21 | from threading import Thread 22 | from data_loader import DataLoader 23 | 24 | # max number of chunk files being loaded into memory 25 | CHUNK_NUM = 20 26 | 27 | 28 | class DataBatcher(object): 29 | """ 30 | Data batcher with queue for loading big dataset 31 | """ 32 | 33 | def __init__(self, data_dir, file_list, batch_size, num_epoch, shuffle=False): 34 | self.data_dir = data_dir 35 | self.file_list = file_list 36 | self.batch_size = batch_size 37 | self.num_epoch = num_epoch 38 | self.shuffle = shuffle 39 | 40 | self.cur_epoch = 0 41 | self.loader_queue = Queue(maxsize=CHUNK_NUM) 42 | self.loader_queue_size = 0 43 | self.batch_iter = self.batch_generator() 44 | self.input_gen = self.loader_generator() 45 | 46 | # Start the threads that load the queues 47 | self.loader_q_thread = Thread(target=self.fill_loader_queue) 48 | self.loader_q_thread.setDaemon(True) 49 | self.loader_q_thread.start() 50 | 51 | # Start a thread that watches the other threads and restarts them if they're dead 52 | self.watch_thread = Thread(target=self.monitor_threads) 53 | self.watch_thread.setDaemon(True) 54 | self.watch_thread.start() 55 | 56 | def get_batch(self): 57 | try: 58 | batch_data, local_size = next(self.batch_iter) 59 | except StopIteration: 60 | batch_data = None 61 | local_size = 0 62 | return batch_data, local_size 63 | 64 | def get_epoch(self): 65 | return self.cur_epoch 66 | 67 | def full(self): 68 | if self.loader_queue_size == CHUNK_NUM: 69 | return True 70 | else: 71 | return False 72 | 73 | def batch_generator(self): 74 | while self.loader_queue_size > 0: 75 | data_loader = self.loader_queue.get() 76 | n_batch = data_loader.n_batch 77 | self.loader_queue_size -= 1 78 | for batch_idx in range(n_batch): 79 | batch_data, local_size = data_loader.get_batch(batch_idx=batch_idx) 80 | yield batch_data, local_size 81 | 82 | def loader_generator(self): 83 | for epoch in range(self.num_epoch): 84 | self.cur_epoch = epoch 85 | if self.shuffle: 86 | np.random.shuffle(self.file_list) 87 | for idx, f in enumerate(self.file_list): 88 | # here, the file reading process may vary from your task 89 | reader = open("%s/%s" % (self.data_dir, f), 'br') 90 | chunk_data = pickle.load(reader) 91 | # here, DataLoader is a self-defined class for data batching, similar to DataLoader in PyTorch 92 | data_loader = DataLoader(data=chunk_data, batch_size=self.batch_size) 93 | yield data_loader 94 | 95 | def fill_loader_queue(self): 96 | while True: 97 | if self.loader_queue_size <= CHUNK_NUM: 98 | try: 99 | data_loader = next(self.input_gen) 100 | self.loader_queue.put(data_loader) 101 | self.loader_queue_size += 1 102 | except StopIteration: 103 | break 104 | 105 | def monitor_threads(self): 106 | """Watch loader queue thread and restart if dead.""" 107 | while True: 108 | time.sleep(60) 109 | if not self.loader_q_thread.is_alive(): # if the thread is dead 110 | print('Found loader queue thread dead. Restarting.') 111 | new_t = Thread(target=self.fill_loader_queue) 112 | self.loader_q_thread = new_t 113 | new_t.daemon = True 114 | new_t.start() 115 | ``` 116 | 117 | 118 | ## Network architecture 119 | ### Seq2Seq 120 | Some tricks to train RNN and seq2seq models: 121 | 122 | * Embedding size: 1024 or 512. Lower dimensionality like 256 can also lead to good performances. Higher does not necessarily lead to better performances. 123 | * For the decoder: LSTM > GRU > Vanilla-RNN 124 | * 2-4 layers seems generally enough. Deeper models with residual connections seems more difficult to converge (high variance). More tricks needs to be discovered. 125 | * ResD (dense residual connections) > Res (only connected to previous layer) > no residual connections 126 | * For encoder: Bidirectional > Unidirectional (reversed input) > Unidirectional 127 | * Attention (additive) > Attention (multiplicative) > No attention. Authors suggest that attention act more as a skip connection mechanism than as a memory for the decoder. 128 | 129 | !!! info "Ref" 130 | [Massive Exploration of Neural Machine Translation Architectures](https://arxiv.org/abs/1703.03906), Denny Britz, Anna Goldie et al. 131 | 132 | For seq2seq, reverse the order of the input sequence (\['I', 'am', 'hungry'\] becomes \['hungry', 'am', 'I'\]). Keep the target sequence intact. 133 | 134 | !!! question "Why" 135 | From the authors: "*This way, [...] that makes it easy for SGD to “establish communication” between the input and the output. We found this simple data transformation to greatly improve the performance of the LSTM.*" 136 | 137 | !!! info "Ref" 138 | [Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215), Ilya Sutskever et al. 139 | 140 | 141 | ### Char-RNN 142 | By training in an unsupervised way a network to predict the next character of a text (char-RNN), the network will learn a representation which can then be used for a supervised task (here sentiment analysis). 143 | 144 | !!! info "Ref" 145 | [Learning to Generate Reviews and Discovering Sentiment](https://arxiv.org/abs/1704.01444), Ilya Sutskever et al. 146 | 147 | 148 | ## Parameters 149 | ### Learning rate 150 | The learning rate can be usually initialized as 0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1(3x growing up). A strategy used to select the hyperparameters is to randomly sample them (uniformly or logscale) and see the testing error after a few epoch. 151 | 152 | ### Beam size 153 | Usually set from 2 to 10. The larger beam size, the higher computational cost. 154 | 155 | ## Regularization 156 | ### Dropout 157 | To make Dropout works with RNN, it should only be applied on non-recurrent connections (between layers among a same timestep) [1]. Some more recent paper propose some tricks to make dropout works for recurrent connections[2]. 158 | 159 | !!! info "Ref" 160 | [1]. [Recurrent Neural Network Regularization](https://arxiv.org/abs/1409.2329), Wojciech Zaremba et al.
161 | [2]. [Recurrent Dropout without Memory Loss](https://arxiv.org/abs/1603.05118), Stanislau Semeniuta et al. 162 | 163 | ### Batch normalization 164 | adding a new normalization layer. Some additional tricks for accelerating BN Networks: 165 | * Increase the learning rate 166 | * Remove/reduce Dropout: speeds up training, without increasing overfitting 167 | * Remove/Reduce the L2 weight regularization 168 | * Accelerate the learning rate decay: because the network trains faster 169 | * Remove Local Response Normalization 170 | * Shuffle training examples more thoroughly: prevents the same examples from always appearing in a mini-batch together. (The authors speak about 1% improvements in the validation) 171 | * Reduce the photometric distortions 172 | 173 | !!! question "Why" 174 | Some good explanation at [Quora](https://www.quora.com/Why-does-batch-normalization-help). 175 | 176 | 177 | 178 | ## Reinforcement learning 179 | ### Asynchronous 180 | Train simultaneously multiple agents with different exploration policies (e.g., E-greedy with different values of epsilon) improve the robustness. 181 | 182 | !!! info "Ref" 183 | [Asynchronous Methods for Deep Reinforcement Learning](https://arxiv.org/abs/1602.01783), V. Mnih. 184 | 185 | ### Skip frame 186 | Compute the action every 4 frames instead of every frames. For the other frames, repeat the action. 187 | 188 | !!! question "Why" 189 | Works well on Atari games, when the player reactivity doesn't need to be frame perfect. Using this trick allows to greatly speed up the training (About x4). 190 | 191 | !!! info "Ref" 192 | [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/abs/1312.5602), V. Mnih. 193 | 194 | ### History 195 | Instead of only taking the current frame as input, stack the last frames together on a single input (size (h, w, c) with 1 grayscale frame by channel). Combined with a skip frame (repeat action) of 4, that means we would stack the frames t, t-4, t-8 and t-12. 196 | 197 | !!! question "Why" 198 | This allows the network to have some momentum information. 199 | 200 | !!! info "Ref" 201 | [Deep Reinforcement Learning with Double Q-learning](https://arxiv.org/abs/1509.06461), V. Mnih. 202 | 203 | ### Experience Replay 204 | Instead of updating every frames as the agent plays, to avoid correlations between the frames, it's better to sample a batch in the history of the transition taken (state, actionTaken, reward, nextState). This is basically the same idea as shuffling the dataset before training for supervised tasks. Some strategies exist to sample batches which contain more information (in the sense predicted reward different from real reward). 205 | 206 | !!! info "Ref" 207 | [Prioritized Experience Replay](https://arxiv.org/abs/1511.05952), Tom Schaul et al. 208 | 209 | ### PAAC (Parallel Advantage Actor Critic) 210 | It's possible to simplify the the A3C algorithm by batching the agent experiences and using a single model with synchronous updates. 211 | 212 | !!! info "Ref" 213 | [Efficient Parallel Methods for Deep Reinforcement Learning](https://arxiv.org/abs/1705.04862v2), Alfredo V. Clemente et al. 214 | -------------------------------------------------------------------------------- /nlpdocument/site/assets/javascripts/lunr/lunr.tr.js: -------------------------------------------------------------------------------- 1 | !function(r,i){"function"==typeof define&&define.amd?define(i):"object"==typeof exports?module.exports=i():i()(r.lunr)}(this,function(){return function(r){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var mr,dr,i;r.tr=function(){this.pipeline.reset(),this.pipeline.add(r.tr.trimmer,r.tr.stopWordFilter,r.tr.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(r.tr.stemmer))},r.tr.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",r.tr.trimmer=r.trimmerSupport.generateTrimmer(r.tr.wordCharacters),r.Pipeline.registerFunction(r.tr.trimmer,"trimmer-tr"),r.tr.stemmer=(mr=r.stemmerSupport.Among,dr=r.stemmerSupport.SnowballProgram,i=new function(){var t,r=[new mr("m",-1,-1),new mr("n",-1,-1),new mr("miz",-1,-1),new mr("niz",-1,-1),new mr("muz",-1,-1),new mr("nuz",-1,-1),new mr("müz",-1,-1),new mr("nüz",-1,-1),new mr("mız",-1,-1),new mr("nız",-1,-1)],i=[new mr("leri",-1,-1),new mr("ları",-1,-1)],e=[new mr("ni",-1,-1),new mr("nu",-1,-1),new mr("nü",-1,-1),new mr("nı",-1,-1)],n=[new mr("in",-1,-1),new mr("un",-1,-1),new mr("ün",-1,-1),new mr("ın",-1,-1)],u=[new mr("a",-1,-1),new mr("e",-1,-1)],o=[new mr("na",-1,-1),new mr("ne",-1,-1)],s=[new mr("da",-1,-1),new mr("ta",-1,-1),new mr("de",-1,-1),new mr("te",-1,-1)],c=[new mr("nda",-1,-1),new mr("nde",-1,-1)],l=[new mr("dan",-1,-1),new mr("tan",-1,-1),new mr("den",-1,-1),new mr("ten",-1,-1)],a=[new mr("ndan",-1,-1),new mr("nden",-1,-1)],m=[new mr("la",-1,-1),new mr("le",-1,-1)],d=[new mr("ca",-1,-1),new mr("ce",-1,-1)],f=[new mr("im",-1,-1),new mr("um",-1,-1),new mr("üm",-1,-1),new mr("ım",-1,-1)],b=[new mr("sin",-1,-1),new mr("sun",-1,-1),new mr("sün",-1,-1),new mr("sın",-1,-1)],w=[new mr("iz",-1,-1),new mr("uz",-1,-1),new mr("üz",-1,-1),new mr("ız",-1,-1)],_=[new mr("siniz",-1,-1),new mr("sunuz",-1,-1),new mr("sünüz",-1,-1),new mr("sınız",-1,-1)],k=[new mr("lar",-1,-1),new mr("ler",-1,-1)],p=[new mr("niz",-1,-1),new mr("nuz",-1,-1),new mr("nüz",-1,-1),new mr("nız",-1,-1)],g=[new mr("dir",-1,-1),new mr("tir",-1,-1),new mr("dur",-1,-1),new mr("tur",-1,-1),new mr("dür",-1,-1),new mr("tür",-1,-1),new mr("dır",-1,-1),new mr("tır",-1,-1)],y=[new mr("casına",-1,-1),new mr("cesine",-1,-1)],z=[new mr("di",-1,-1),new mr("ti",-1,-1),new mr("dik",-1,-1),new mr("tik",-1,-1),new mr("duk",-1,-1),new mr("tuk",-1,-1),new mr("dük",-1,-1),new mr("tük",-1,-1),new mr("dık",-1,-1),new mr("tık",-1,-1),new mr("dim",-1,-1),new mr("tim",-1,-1),new mr("dum",-1,-1),new mr("tum",-1,-1),new mr("düm",-1,-1),new mr("tüm",-1,-1),new mr("dım",-1,-1),new mr("tım",-1,-1),new mr("din",-1,-1),new mr("tin",-1,-1),new mr("dun",-1,-1),new mr("tun",-1,-1),new mr("dün",-1,-1),new mr("tün",-1,-1),new mr("dın",-1,-1),new mr("tın",-1,-1),new mr("du",-1,-1),new mr("tu",-1,-1),new mr("dü",-1,-1),new mr("tü",-1,-1),new mr("dı",-1,-1),new mr("tı",-1,-1)],h=[new mr("sa",-1,-1),new mr("se",-1,-1),new mr("sak",-1,-1),new mr("sek",-1,-1),new mr("sam",-1,-1),new mr("sem",-1,-1),new mr("san",-1,-1),new mr("sen",-1,-1)],v=[new mr("miş",-1,-1),new mr("muş",-1,-1),new mr("müş",-1,-1),new mr("mış",-1,-1)],q=[new mr("b",-1,1),new mr("c",-1,2),new mr("d",-1,3),new mr("ğ",-1,4)],C=[17,65,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,8,0,0,0,0,0,0,1],P=[1,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,1],F=[65],S=[65],W=[["a",[1,64,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],97,305],["e",[17,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,130],101,252],["ı",[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],97,305],["i",[17],101,105],["o",F,111,117],["ö",S,246,252],["u",F,111,117]],L=new dr;function x(r,i,e){for(;;){var n=L.limit-L.cursor;if(L.in_grouping_b(r,i,e)){L.cursor=L.limit-n;break}if(L.cursor=L.limit-n,L.cursor<=L.limit_backward)return!1;L.cursor--}return!0}function A(){var r,i;r=L.limit-L.cursor,x(C,97,305);for(var e=0;eL.limit_backward&&(L.cursor--,e=L.limit-L.cursor,i()))?(L.cursor=L.limit-e,!0):(L.cursor=L.limit-n,r()?(L.cursor=L.limit-n,!1):(L.cursor=L.limit-n,!(L.cursor<=L.limit_backward)&&(L.cursor--,!!i()&&(L.cursor=L.limit-n,!0))))}function j(r){return E(r,function(){return L.in_grouping_b(C,97,305)})}function T(){return j(function(){return L.eq_s_b(1,"n")})}function Z(){return j(function(){return L.eq_s_b(1,"y")})}function B(){return L.find_among_b(r,10)&&E(function(){return L.in_grouping_b(P,105,305)},function(){return L.out_grouping_b(C,97,305)})}function D(){return A()&&L.in_grouping_b(P,105,305)&&j(function(){return L.eq_s_b(1,"s")})}function G(){return L.find_among_b(i,2)}function H(){return A()&&L.find_among_b(n,4)&&T()}function I(){return A()&&L.find_among_b(s,4)}function J(){return A()&&L.find_among_b(c,2)}function K(){return A()&&L.find_among_b(f,4)&&Z()}function M(){return A()&&L.find_among_b(b,4)}function N(){return A()&&L.find_among_b(w,4)&&Z()}function O(){return L.find_among_b(_,4)}function Q(){return A()&&L.find_among_b(k,2)}function R(){return A()&&L.find_among_b(g,8)}function U(){return A()&&L.find_among_b(z,32)&&Z()}function V(){return L.find_among_b(h,8)&&Z()}function X(){return A()&&L.find_among_b(v,4)&&Z()}function Y(){var r=L.limit-L.cursor;return!(X()||(L.cursor=L.limit-r,U()||(L.cursor=L.limit-r,V()||(L.cursor=L.limit-r,L.eq_s_b(3,"ken")&&Z()))))}function $(){if(L.find_among_b(y,2)){var r=L.limit-L.cursor;if(O()||(L.cursor=L.limit-r,Q()||(L.cursor=L.limit-r,K()||(L.cursor=L.limit-r,M()||(L.cursor=L.limit-r,N()||(L.cursor=L.limit-r))))),X())return!1}return!0}function rr(){if(!A()||!L.find_among_b(p,4))return!0;var r=L.limit-L.cursor;return!U()&&(L.cursor=L.limit-r,!V())}function ir(){var r,i,e,n=L.limit-L.cursor;if(L.ket=L.cursor,t=!0,Y()&&(L.cursor=L.limit-n,$()&&(L.cursor=L.limit-n,function(){if(Q()){L.bra=L.cursor,L.slice_del();var r=L.limit-L.cursor;return L.ket=L.cursor,R()||(L.cursor=L.limit-r,U()||(L.cursor=L.limit-r,V()||(L.cursor=L.limit-r,X()||(L.cursor=L.limit-r)))),t=!1}return!0}()&&(L.cursor=L.limit-n,rr()&&(L.cursor=L.limit-n,e=L.limit-L.cursor,!(O()||(L.cursor=L.limit-e,N()||(L.cursor=L.limit-e,M()||(L.cursor=L.limit-e,K()))))||(L.bra=L.cursor,L.slice_del(),i=L.limit-L.cursor,L.ket=L.cursor,X()||(L.cursor=L.limit-i),0)))))){if(L.cursor=L.limit-n,!R())return;L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,r=L.limit-L.cursor,O()||(L.cursor=L.limit-r,Q()||(L.cursor=L.limit-r,K()||(L.cursor=L.limit-r,M()||(L.cursor=L.limit-r,N()||(L.cursor=L.limit-r))))),X()||(L.cursor=L.limit-r)}L.bra=L.cursor,L.slice_del()}function er(){var r,i,e,n;if(L.ket=L.cursor,L.eq_s_b(2,"ki")){if(r=L.limit-L.cursor,I())return L.bra=L.cursor,L.slice_del(),i=L.limit-L.cursor,L.ket=L.cursor,Q()?(L.bra=L.cursor,L.slice_del(),er()):(L.cursor=L.limit-i,B()&&(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er()))),!0;if(L.cursor=L.limit-r,H()){if(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,e=L.limit-L.cursor,G())L.bra=L.cursor,L.slice_del();else{if(L.cursor=L.limit-e,L.ket=L.cursor,!B()&&(L.cursor=L.limit-e,!D()&&(L.cursor=L.limit-e,!er())))return!0;L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er())}return!0}if(L.cursor=L.limit-r,J()){if(n=L.limit-L.cursor,G())L.bra=L.cursor,L.slice_del();else if(L.cursor=L.limit-n,D())L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er());else if(L.cursor=L.limit-n,!er())return!1;return!0}}return!1}function nr(r){if(L.ket=L.cursor,!J()&&(L.cursor=L.limit-r,!A()||!L.find_among_b(o,2)))return!1;var i=L.limit-L.cursor;if(G())L.bra=L.cursor,L.slice_del();else if(L.cursor=L.limit-i,D())L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er());else if(L.cursor=L.limit-i,!er())return!1;return!0}function tr(r){if(L.ket=L.cursor,!(A()&&L.find_among_b(a,2)||(L.cursor=L.limit-r,A()&&L.find_among_b(e,4))))return!1;var i=L.limit-L.cursor;return!(!D()&&(L.cursor=L.limit-i,!G()))&&(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er()),!0)}function ur(){var r,i=L.limit-L.cursor;return L.ket=L.cursor,!!(H()||(L.cursor=L.limit-i,A()&&L.find_among_b(m,2)&&Z()))&&(L.bra=L.cursor,L.slice_del(),r=L.limit-L.cursor,L.ket=L.cursor,!(!Q()||(L.bra=L.cursor,L.slice_del(),!er()))||(L.cursor=L.limit-r,L.ket=L.cursor,(B()||(L.cursor=L.limit-r,D()||(L.cursor=L.limit-r,er())))&&(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er())),!0))}function or(){var r,i,e=L.limit-L.cursor;if(L.ket=L.cursor,!(I()||(L.cursor=L.limit-e,A()&&L.in_grouping_b(P,105,305)&&Z()||(L.cursor=L.limit-e,A()&&L.find_among_b(u,2)&&Z()))))return!1;if(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,r=L.limit-L.cursor,B())L.bra=L.cursor,L.slice_del(),i=L.limit-L.cursor,L.ket=L.cursor,Q()||(L.cursor=L.limit-i);else if(L.cursor=L.limit-r,!Q())return!0;return L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,er(),!0}function sr(){var r,i,e=L.limit-L.cursor;if(L.ket=L.cursor,Q())return L.bra=L.cursor,L.slice_del(),void er();if(L.cursor=L.limit-e,L.ket=L.cursor,A()&&L.find_among_b(d,2)&&T())if(L.bra=L.cursor,L.slice_del(),r=L.limit-L.cursor,L.ket=L.cursor,G())L.bra=L.cursor,L.slice_del();else{if(L.cursor=L.limit-r,L.ket=L.cursor,!B()&&(L.cursor=L.limit-r,!D())){if(L.cursor=L.limit-r,L.ket=L.cursor,!Q())return;if(L.bra=L.cursor,L.slice_del(),!er())return}L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er())}else if(L.cursor=L.limit-e,!nr(e)&&(L.cursor=L.limit-e,!tr(e))){if(L.cursor=L.limit-e,L.ket=L.cursor,A()&&L.find_among_b(l,4))return L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,i=L.limit-L.cursor,void(B()?(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er())):(L.cursor=L.limit-i,Q()?(L.bra=L.cursor,L.slice_del()):L.cursor=L.limit-i,er()));if(L.cursor=L.limit-e,!ur()){if(L.cursor=L.limit-e,G())return L.bra=L.cursor,void L.slice_del();L.cursor=L.limit-e,er()||(L.cursor=L.limit-e,or()||(L.cursor=L.limit-e,L.ket=L.cursor,(B()||(L.cursor=L.limit-e,D()))&&(L.bra=L.cursor,L.slice_del(),L.ket=L.cursor,Q()&&(L.bra=L.cursor,L.slice_del(),er()))))}}}function cr(r,i,e){if(L.cursor=L.limit-r,function(){for(;;){var r=L.limit-L.cursor;if(L.in_grouping_b(C,97,305)){L.cursor=L.limit-r;break}if(L.cursor=L.limit-r,L.cursor<=L.limit_backward)return!1;L.cursor--}return!0}()){var n=L.limit-L.cursor;if(!L.eq_s_b(1,i)&&(L.cursor=L.limit-n,!L.eq_s_b(1,e)))return!0;L.cursor=L.limit-r;var t=L.cursor;return L.insert(L.cursor,L.cursor,e),L.cursor=t,!1}return!0}function lr(r,i,e){for(;!L.eq_s(i,e);){if(L.cursor>=L.limit)return!0;L.cursor++}return i!=L.limit||(L.cursor=r,!1)}function ar(){var r,i,e=L.cursor;return!(!lr(r=L.cursor,2,"ad")||!lr(L.cursor=r,5,"soyad"))&&(L.limit_backward=e,L.cursor=L.limit,i=L.limit-L.cursor,(L.eq_s_b(1,"d")||(L.cursor=L.limit-i,L.eq_s_b(1,"g")))&&cr(i,"a","ı")&&cr(i,"e","i")&&cr(i,"o","u")&&cr(i,"ö","ü"),L.cursor=L.limit,function(){var r;if(L.ket=L.cursor,r=L.find_among_b(q,4))switch(L.bra=L.cursor,r){case 1:L.slice_from("p");break;case 2:L.slice_from("ç");break;case 3:L.slice_from("t");break;case 4:L.slice_from("k")}}(),!0)}this.setCurrent=function(r){L.setCurrent(r)},this.getCurrent=function(){return L.getCurrent()},this.stem=function(){return!!(function(){for(var r,i=L.cursor,e=2;;){for(r=L.cursor;!L.in_grouping(C,97,305);){if(L.cursor>=L.limit)return L.cursor=r,!(0 | 26 | | Pairwise word interaction modeling with deep neural networks for semantic similarity measurement | Hua He1 and Jimmy Lin | 2016 | 0.7588 | 0.8219 | | | | | 0.709 | 0.7234 | | | | | | 27 | | Sentence similarity learning by lexical decomposition and composition | ZhiguoWang and Haitao Mi and Abraham Ittycheriah | 2016 | | | | | | | 0.7058 | 0.7226 | | | | | | 28 | | A compare aggregate model for matching text sequences | ShuohangWang | 2017 | | | | | | | 0.7433 | 0.7545 | **0.756** | **0.734** | | | https://github.com/shuohangwang/SeqMatchSeq | 29 | | A compare aggregate model with dynamic-clip attention for answer selection | Weijie Bian, Si Li, Zhao Yang, Guang Chen, Zhiqing Lin | 2017 | | | 0.821 | 0.899 | | | 0.754 | 0.764 | | | | | https://github.com/wjbianjason/Dynamic-Clip-Attention | 30 | | A Hybrid Framework for Text Modeling with Convolutional RNN | ChenglongWang, Feijun Jiang, Hongxia Yang | 2017 | | | | | | | 0.7427 | 0.7504 | 0.714 | 0.683 | | | | 31 | | Bilateral multi-perspective matching for natural language sentences | ZhiguoWang,Wael Hamza, Radu Florian | 2017 | | | 0.802 | 0.875 | | | 0.718 | 0.731 | | | | | https://github.com/zhiguowang/BiMPM | 32 | | Enhancing Recurrent Neural Networks with Positional Attention for Question Answering | Qin Chen1, Qinmin Hu1, Jimmy Xiangji Huang2, Liang He1,3 andWeijie An | 2017 | | | 0.7814 | 0.8513 | | | 0.7212 | 0.7312 | | | | | | 33 | | Inter-weighted alignment network for sentence pair modeling | Gehui Shen Yunlun Yang Zhi-Hong Deng | 2017 | | | 0.822 | 0.889 | | | 0.733 | 0.75 | | | | | | 34 | | Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2017 | 0.7499 | 0.8153 | | | 0.752 | 0.8146 | | | | | | | | 35 | | On the Benefit of Incorporating External Features in a Neural Architecture for Answer Sentence Selection | Ruey-Cheng Chen, Evi Yulianti, Mark Sanderson,W. Bruce Croſt | 2017 | 0.782 | 0.837 | | | | | 0.701 | 0.718 | | | | | | 36 | | Ranking Kernels for Structures and Embeddings : A Hybrid Preference and Classification Model | Kateryna Tymoshenko† and Daniele Bonadiman† and Alessandro Moschitti | 2017 | | | | | | | 0.7219 | 0.7408 | | | 0.771 | 0.8345 | | 37 | | A Multi-View Fusion Neural Network for Answer Selection | Lei Sha,∗ Xiaodong Zhang,∗ Feng Qian, Baobao Chang, Zhifang Sui | 2018 | | | | | | | 0.7462 | 0.7576 | | | **0.8005** | **0.8718** | | 38 | | CA-RNN : Using Context-Aligned Recurrent Neural Networks for Modeling Sentence Similarity | Qin Chen,1 Qinmin Hu,1 Jimmy Xiangji Huang,2 Liang He | 2018 | | | 0.8227 | 0.8886 | | | 0.7358 | 0.745 | | | | | | 39 | | CAN : Enhancing Sentence Similarity Modeling with Collaborative and Adversarial Network | Qin Chen1, Qinmin Hu, Jimmy Xiangji Huang3 and Liang He | 2018 | | | 0.841 | 0.9168 | | | 0.7303 | 0.7431 | | | | | | 40 | | Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | | | **0.854** | **0.935** | | | | | | | | | | 41 | | Context-Aware Answer Sentence Selection With Hierarchical Gated Recurrent Neural Networks | Chuanqi Tan , FuruWei, Qingyu Zhou, Nan Yang, Bowen Du ,Weifeng Lv, and Ming Zhou | 2018 | | | | | | | **0.7638** | **0.7825** | | | | | | 42 | | Cross Temporal Recurrent Networks for Ranking Question Answer Pairs | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | 0.7712 | 0.8384 | | | **0.7582** | 0.8233 | | | | | | | | 43 | | End-to-End Quantum-like Language Models with Application to Question Answering | Peng Zhang1,∗, Jiabin Niu1, Zhan Su1, BenyouWang2, Liqun Ma3, Dawei Song | 2018 | 0.7589 | 0.8254 | | | | | 0.6496 | 0.6594 | | | | | | 44 | | Hermitian Co-Attention Networks for Text Matching in Asymmetrical Domains | Yi Tay1, Anh Tuan Luu2, Siu Cheung Hui | 2018 | | | 0.784 | 0.895 | | | 0.743 | 0.756 | | | | | | 45 | | Hyperbolic representation learning for fast and efficient neural question answering | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | 0.77 | 0.825 | 0.784 | 0.865 | | | 0.712 | 0.727 | | | 0.795 | null | https://github.com/vanzytay/WSDM2018_HyperQA | 46 | | Knowledge as A Bridge: Improving Cross-domain Answer Selection with External Knowledge | Yang Deng1, Ying Shen1,∗, Min Yang2, Yaliang Li3, Nan Du3,Wei Fan3, Kai Lei1 | 2018 | 0.797 | 0.85 | | | | | | | | | | | | 47 | | Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs | Ying Shen1, Yang Deng1, Min Yang2, Yaliang Li3, Nan Du3,Wei Fan3, Kai Lei | 2018 | 0.7921 | 0.8444 | 0.8038 | 0.8846 | | | 0.7323 | 0.7494 | | | | | | 48 | | Multihop Attention Networks for Question Answer Matching | Nam Khanh Tran, Claudia Niederée | 2018 | | | 0.813 | 0.893 | | | 0.722 | 0.738 | 0.705 | 0.669 | | | https://github.com/namkhanhtran/nn4nqa | 49 | | Recurrently Controlled Recurrent Networks | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | | | 0.779 | 0.882 | | | 0.724 | 0.737 | | | | | https://github.com/vanzytay/NIPS2018_RCRN | 50 | | Self-Training for Jointly Learning to Ask and Answer Questions | Mrinmaya Sachan | 2018 | **0.798** | **0.854** | | | | | 0.754 | 0.753 | | | | | | 51 | | Semantic Linking in Convolutional Neural Networks for Answer Sentence Selection | Massimo Nicosia∗ and Alessandro Moschitti† | 2018 | | | 0.7793 | 0.8489 | | | 0.7224 | 0.7391 | | | | | | 52 | -------------------------------------------------------------------------------- /nlpdocument/site/404.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | NLP and DL Docs 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 84 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 |
109 | 186 |
187 | 188 |
189 | 190 | 191 | 192 | 193 | 194 | 195 | 241 | 242 |
243 |
244 | 245 | 246 |
247 |
248 |
249 | 499 |
500 |
501 |
502 | 503 | 504 | 505 |
506 |
507 | 508 |

404 - Not found

509 | 510 | 511 | 512 | 513 | 514 | 515 |
516 |
517 |
518 |
519 | 520 | 521 |
522 | 523 |
524 |
525 |
526 | 527 |
528 | Built by SIAT NLP Group 529 |
530 | 531 | powered by 532 | MkDocs 533 | and 534 | 535 | Material for MkDocs 536 |
537 | 538 |
539 | 540 | 541 | 542 | 543 | 544 | 545 |
546 | 547 |
548 |
549 |
550 | 551 |
552 | 553 | 554 | 555 | 556 | 557 | 558 | 559 | 560 | 561 | --------------------------------------------------------------------------------