├── .gitignore ├── .python-version ├── LICENSE ├── Procfile ├── README.md ├── manage.py ├── morphgnt_api ├── __init__.py ├── management │ ├── __init__.py │ └── commands │ │ ├── __init__.py │ │ └── load_morphgnt.py ├── migrations │ ├── 0001_initial.py │ ├── 0002_auto_20150703_1225.py │ └── __init__.py ├── models.py ├── ref.py ├── settings.py ├── urls.py ├── views.py └── wsgi.py ├── requirements.txt ├── runtime.txt └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | morphgnt-api 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2015 James Tauber. 2 | 3 | Permission is hereby granted, free of charge, to any person 4 | obtaining a copy of this software and associated documentation 5 | files (the "Software"), to deal in the Software without 6 | restriction, including without limitation the rights to use, 7 | copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | copies of the Software, and to permit persons to whom the 9 | Software is furnished to do so, subject to the following 10 | conditions: 11 | 12 | The above copyright notice and this permission notice shall be 13 | included in all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 16 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES 17 | OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 18 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 19 | HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 20 | WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 21 | FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | -------------------------------------------------------------------------------- /Procfile: -------------------------------------------------------------------------------- 1 | web: gunicorn --log-file - morphgnt_api.wsgi 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # morphgnt-api 2 | 3 | an experimental REST API for MorphGNT 4 | 5 | This is available on . 6 | 7 | Note that the `/v0/` prefix is used because there is no commitment to keep 8 | this API. It is subject to rapid change at the moment. 9 | 10 | The URI patterns are: 11 | 12 | ``` 13 | /v0/root.json 14 | /v0/book/{osis_id}.json 15 | /v0/paragraph/{paragraph_id}.json 16 | /v0/sentence/{sentence_id}.json 17 | /v0/verse/{verse_id}.json 18 | /v0/word/{word_id}.json 19 | /v0/verse-lookup/?{verse-reference} 20 | /v0/frequency/ 21 | ``` 22 | 23 | A word (currently) looks something like this: 24 | 25 | ``` 26 | { 27 | "@id": "/v0/word/64001001005.json", 28 | "@type": "word", 29 | "verse_id": "/v0/verse/640101.json", 30 | "sentence_id": "/v0/sentence/640001.json", 31 | "paragraph_id": "/v0/paragraph/64001.json", 32 | "crit_text": "λόγος,", 33 | "text": "λόγος,", 34 | "word": "λόγος", 35 | "norm": "λόγος", 36 | "lemma": "λόγος", 37 | "pos": "N", 38 | "case": "N", 39 | "number": "S", 40 | "gender": "M", 41 | "dep_type": "S", 42 | "head": "/v0/word/64001001002.json" 43 | } 44 | ``` 45 | 46 | 47 | A verse (currently) looks something like this: 48 | 49 | ``` 50 | { 51 | "@id": "/v0/verse/640101.json", 52 | "@type": "verse", 53 | "title": "John 1.1", 54 | "prev": null, 55 | "next": "/v0/verse/640102.json", 56 | "book": "/v0/book/John.json", 57 | "words": [...] 58 | } 59 | ``` 60 | 61 | where `words` is a list of objects like the word above. 62 | 63 | A paragraph and sentence are very similar to a verse (with an `@id`, `@type`, 64 | `prev`, `next`, `book` and `words` list). 65 | 66 | A book (currently) looks something like this: 67 | 68 | ``` 69 | { 70 | "@id": "/v0/book/1Cor.json", 71 | "@type": "book", 72 | "name": "1 Corinthians", 73 | root: "/v0/root.js", 74 | "first_paragraph": "/v0/paragraph/67001.json", 75 | "first_verse": "/v0/verse/670101.json", 76 | "first_sentence": "/v0/sentence/670001.json" 77 | } 78 | ``` 79 | 80 | `/v0/verse-lookup/?{verse-reference}` parses the verse reference and returns 81 | a JSON object indicating the URL of the verse resource. 82 | 83 | e.g. `/v0/verse-lookup/?Jn+3.16` will return `{"verse_id": "/v0/verse/640316.json"}` 84 | 85 | A variety of book name abbreviations are supported. 86 | 87 | `/v0/frequency/` takes a body of a JSON object containing an `input` property 88 | whose value consists of a list of JSON objects each with an `id` and `lemma` 89 | property. The response is a JSON object with an `output` property whose value 90 | consists of a list of JSON objects each with an `id` and `count` property. The 91 | `count` property gives the lemma count of the input `lemma`. The output counts 92 | are correlated to the input lemmas via matching `id` values. 93 | 94 | Feedback is greatly appreciated to make this whole API more useful. 95 | -------------------------------------------------------------------------------- /manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import os 3 | import sys 4 | 5 | if __name__ == "__main__": 6 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "morphgnt_api.settings") 7 | 8 | from django.core.management import execute_from_command_line 9 | 10 | execute_from_command_line(sys.argv) 11 | -------------------------------------------------------------------------------- /morphgnt_api/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/morphgnt/morphgnt-api/f53c7b4243f9da964f4dd2901544265f3ba8f465/morphgnt_api/__init__.py -------------------------------------------------------------------------------- /morphgnt_api/management/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/morphgnt/morphgnt-api/f53c7b4243f9da964f4dd2901544265f3ba8f465/morphgnt_api/management/__init__.py -------------------------------------------------------------------------------- /morphgnt_api/management/commands/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/morphgnt/morphgnt-api/f53c7b4243f9da964f4dd2901544265f3ba8f465/morphgnt_api/management/commands/__init__.py -------------------------------------------------------------------------------- /morphgnt_api/management/commands/load_morphgnt.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | from django.core.management.base import BaseCommand 4 | 5 | from morphgnt_api.models import Word, Book, Paragraph, Sentence, Verse 6 | 7 | 8 | def u(s): 9 | return s.decode("utf-8") 10 | 11 | 12 | books_by_osis_id = { 13 | "Matt": ("Matthew", 61), 14 | "Mark": ("Mark", 62), 15 | "Luke": ("Luke", 63), 16 | "John": ("John", 64), 17 | "Acts": ("Acts", 65), 18 | "Rom": ("Romans", 66), 19 | "1Cor": ("1 Corinthians", 67), 20 | "2Cor": ("2 Corinthians", 68), 21 | "Gal": ("Galatians", 69), 22 | "Eph": ("Ephesians", 70), 23 | "Phil": ("Philippians", 71), 24 | "Col": ("Colossians", 72), 25 | "1Thess": ("1 Thessalonians", 73), 26 | "2Thess": ("2 Thessalonians", 74), 27 | "1Tim": ("1 Timothy", 75), 28 | "2Tim": ("2 Timothy", 76), 29 | "Titus": ("Titus", 77), 30 | "Phlm": ("Philemon", 78), 31 | "Heb": ("Hebrews", 79), 32 | "Jas": ("James", 80), 33 | "1Pet": ("1 Peter", 81), 34 | "2Pet": ("2 Peter", 82), 35 | "1John": ("1 John", 83), 36 | "2John": ("2 John", 84), 37 | "3John": ("3 John", 85), 38 | "Jude": ("Jude", 86), 39 | "Rev": ("Revelation", 87), 40 | } 41 | 42 | 43 | books_by_sblgnt_id = { 44 | v[1]: k 45 | for k, v in books_by_osis_id.items() 46 | } 47 | 48 | 49 | class Command(BaseCommand): 50 | 51 | def add_arguments(self, parser): 52 | parser.add_argument("filename", help="filename to import") 53 | 54 | def handle(self, *args, **options): 55 | filename = options["filename"] 56 | 57 | for k, v in books_by_osis_id.items(): 58 | Book( 59 | book_osis_id=k, 60 | name=v[0], 61 | sblgnt_id=v[1], 62 | ).save() 63 | 64 | if filename == "-": 65 | self.handle_stream(sys.stdin) 66 | else: 67 | with open(filename, "rb") as f: 68 | self.handle_stream(f) 69 | 70 | def handle_stream(self, f): 71 | prev_verse = None 72 | prev_sentence = None 73 | prev_paragraph = None 74 | 75 | for line in f: 76 | word_id, verse_id, paragraph_id, sentence_id, \ 77 | pos, parse, \ 78 | crit_text, text, word, norm, lemma, \ 79 | dep_type, head = line.strip().split() 80 | 81 | Word( 82 | word_id=word_id, 83 | verse_id=verse_id, 84 | paragraph_id=paragraph_id, 85 | sentence_id=sentence_id, 86 | pos=pos, 87 | parse=parse, 88 | crit_text=u(crit_text), 89 | text=u(text), 90 | word=u(word), 91 | norm=u(norm), 92 | lemma=u(lemma), 93 | dep_type=dep_type, 94 | head=head if head != "None" else None, 95 | ).save() 96 | 97 | if prev_verse is None: 98 | prev_verse = Verse( 99 | verse_id=verse_id, 100 | book_osis_id=books_by_sblgnt_id[int(verse_id[:2])], 101 | prev_verse=None, 102 | next_verse=None, 103 | ) 104 | prev_verse.save() 105 | elif prev_verse.verse_id != verse_id: 106 | if prev_verse.verse_id[:2] == verse_id[:2]: 107 | prev_verse.next_verse = verse_id 108 | prev_verse.save() 109 | prev_verse = Verse( 110 | verse_id=verse_id, 111 | book_osis_id=books_by_sblgnt_id[int(verse_id[:2])], 112 | prev_verse=prev_verse.verse_id, 113 | next_verse=None, 114 | ) 115 | prev_verse.save() 116 | 117 | if prev_sentence is None: 118 | prev_sentence = Sentence( 119 | sentence_id=sentence_id, 120 | book_osis_id=books_by_sblgnt_id[int(sentence_id[:2])], 121 | prev_sentence=None, 122 | next_sentence=None, 123 | ) 124 | prev_sentence.save() 125 | elif prev_sentence.sentence_id != sentence_id: 126 | if prev_sentence.sentence_id[:2] == sentence_id[:2]: 127 | prev_sentence.next_sentence = sentence_id 128 | prev_sentence.save() 129 | prev_sentence = Sentence( 130 | sentence_id=sentence_id, 131 | book_osis_id=books_by_sblgnt_id[int(sentence_id[:2])], 132 | prev_sentence=prev_sentence.sentence_id, 133 | next_sentence=None, 134 | ) 135 | prev_sentence.save() 136 | 137 | if prev_paragraph is None: 138 | prev_paragraph = Paragraph( 139 | paragraph_id=paragraph_id, 140 | book_osis_id=books_by_sblgnt_id[int(paragraph_id[:2])], 141 | prev_paragraph=None, 142 | next_paragraph=None, 143 | ) 144 | prev_paragraph.save() 145 | elif prev_paragraph.paragraph_id != paragraph_id: 146 | if prev_paragraph.paragraph_id[:2] == paragraph_id[:2]: 147 | prev_paragraph.next_paragraph = paragraph_id 148 | prev_paragraph.save() 149 | prev_paragraph = Paragraph( 150 | paragraph_id=paragraph_id, 151 | book_osis_id=books_by_sblgnt_id[int(paragraph_id[:2])], 152 | prev_paragraph=prev_paragraph.paragraph_id, 153 | next_paragraph=None, 154 | ) 155 | prev_paragraph.save() 156 | -------------------------------------------------------------------------------- /morphgnt_api/migrations/0001_initial.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import unicode_literals 3 | 4 | from django.db import models, migrations 5 | 6 | 7 | class Migration(migrations.Migration): 8 | 9 | dependencies = [ 10 | ] 11 | 12 | operations = [ 13 | migrations.CreateModel( 14 | name='Book', 15 | fields=[ 16 | ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), 17 | ('book_osis_id', models.CharField(max_length=6)), 18 | ('name', models.CharField(max_length=20)), 19 | ('sblgnt_id', models.CharField(max_length=2)), 20 | ], 21 | ), 22 | migrations.CreateModel( 23 | name='Paragraph', 24 | fields=[ 25 | ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), 26 | ('paragraph_id', models.CharField(max_length=5)), 27 | ('book_osis_id', models.CharField(max_length=6)), 28 | ('prev_paragraph', models.CharField(max_length=5, null=True)), 29 | ('next_paragraph', models.CharField(max_length=5, null=True)), 30 | ], 31 | ), 32 | migrations.CreateModel( 33 | name='Sentence', 34 | fields=[ 35 | ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), 36 | ('sentence_id', models.CharField(max_length=6)), 37 | ('book_osis_id', models.CharField(max_length=6)), 38 | ('prev_sentence', models.CharField(max_length=6, null=True)), 39 | ('next_sentence', models.CharField(max_length=6, null=True)), 40 | ], 41 | ), 42 | migrations.CreateModel( 43 | name='Verse', 44 | fields=[ 45 | ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), 46 | ('verse_id', models.CharField(max_length=6)), 47 | ('book_osis_id', models.CharField(max_length=6)), 48 | ('prev_verse', models.CharField(max_length=6, null=True)), 49 | ('next_verse', models.CharField(max_length=6, null=True)), 50 | ], 51 | ), 52 | migrations.CreateModel( 53 | name='Word', 54 | fields=[ 55 | ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), 56 | ('word_id', models.CharField(max_length=11)), 57 | ('verse_id', models.CharField(max_length=6)), 58 | ('paragraph_id', models.CharField(max_length=5)), 59 | ('sentence_id', models.CharField(max_length=6)), 60 | ('pos', models.CharField(max_length=2)), 61 | ('parse', models.CharField(max_length=8)), 62 | ('crit_text', models.CharField(max_length=50)), 63 | ('text', models.CharField(max_length=50)), 64 | ('word', models.CharField(max_length=50)), 65 | ('norm', models.CharField(max_length=50)), 66 | ('lemma', models.CharField(max_length=50)), 67 | ('dep_type', models.CharField(max_length=4)), 68 | ('head', models.CharField(max_length=11, null=True)), 69 | ], 70 | ), 71 | ] 72 | -------------------------------------------------------------------------------- /morphgnt_api/migrations/0002_auto_20150703_1225.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import unicode_literals 3 | 4 | from django.db import models, migrations 5 | 6 | 7 | class Migration(migrations.Migration): 8 | 9 | dependencies = [ 10 | ('morphgnt_api', '0001_initial'), 11 | ] 12 | 13 | operations = [ 14 | migrations.AlterModelOptions( 15 | name='book', 16 | options={'ordering': ['sblgnt_id']}, 17 | ), 18 | migrations.AlterModelOptions( 19 | name='paragraph', 20 | options={'ordering': ['paragraph_id']}, 21 | ), 22 | migrations.AlterModelOptions( 23 | name='sentence', 24 | options={'ordering': ['sentence_id']}, 25 | ), 26 | migrations.AlterModelOptions( 27 | name='verse', 28 | options={'ordering': ['verse_id']}, 29 | ), 30 | migrations.AlterModelOptions( 31 | name='word', 32 | options={'ordering': ['word_id']}, 33 | ), 34 | migrations.AlterField( 35 | model_name='book', 36 | name='book_osis_id', 37 | field=models.CharField(max_length=6, db_index=True), 38 | ), 39 | migrations.AlterField( 40 | model_name='paragraph', 41 | name='book_osis_id', 42 | field=models.CharField(max_length=6, db_index=True), 43 | ), 44 | migrations.AlterField( 45 | model_name='paragraph', 46 | name='paragraph_id', 47 | field=models.CharField(max_length=5, db_index=True), 48 | ), 49 | migrations.AlterField( 50 | model_name='sentence', 51 | name='book_osis_id', 52 | field=models.CharField(max_length=6, db_index=True), 53 | ), 54 | migrations.AlterField( 55 | model_name='sentence', 56 | name='sentence_id', 57 | field=models.CharField(max_length=6, db_index=True), 58 | ), 59 | migrations.AlterField( 60 | model_name='verse', 61 | name='book_osis_id', 62 | field=models.CharField(max_length=6, db_index=True), 63 | ), 64 | migrations.AlterField( 65 | model_name='verse', 66 | name='verse_id', 67 | field=models.CharField(max_length=6, db_index=True), 68 | ), 69 | migrations.AlterField( 70 | model_name='word', 71 | name='paragraph_id', 72 | field=models.CharField(max_length=5, db_index=True), 73 | ), 74 | migrations.AlterField( 75 | model_name='word', 76 | name='sentence_id', 77 | field=models.CharField(max_length=6, db_index=True), 78 | ), 79 | migrations.AlterField( 80 | model_name='word', 81 | name='verse_id', 82 | field=models.CharField(max_length=6, db_index=True), 83 | ), 84 | migrations.AlterField( 85 | model_name='word', 86 | name='word_id', 87 | field=models.CharField(max_length=11, db_index=True), 88 | ), 89 | ] 90 | -------------------------------------------------------------------------------- /morphgnt_api/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/morphgnt/morphgnt-api/f53c7b4243f9da964f4dd2901544265f3ba8f465/morphgnt_api/migrations/__init__.py -------------------------------------------------------------------------------- /morphgnt_api/models.py: -------------------------------------------------------------------------------- 1 | from django.core.urlresolvers import reverse 2 | from django.db import models 3 | 4 | from .ref import verse_from_bcv, verse_range_title 5 | 6 | 7 | def parse_as_dict(parse): 8 | return { 9 | label: parse[i] 10 | for i, label in enumerate([ 11 | "person", "tense", "voice", "mood", "case", "number", "gender", "degree" 12 | ]) 13 | if parse[i] != "-" 14 | } 15 | 16 | 17 | class Word(models.Model): 18 | 19 | word_id = models.CharField(max_length=11, db_index=True) 20 | verse_id = models.CharField(max_length=6, db_index=True) 21 | paragraph_id = models.CharField(max_length=5, db_index=True) 22 | sentence_id = models.CharField(max_length=6, db_index=True) 23 | 24 | pos = models.CharField(max_length=2) 25 | parse = models.CharField(max_length=8) 26 | 27 | crit_text = models.CharField(max_length=50) 28 | text = models.CharField(max_length=50) 29 | word = models.CharField(max_length=50) 30 | norm = models.CharField(max_length=50) 31 | lemma = models.CharField(max_length=50) 32 | 33 | dep_type = models.CharField(max_length=4) 34 | head = models.CharField(max_length=11, null=True) 35 | 36 | class Meta: 37 | ordering = ["word_id"] 38 | 39 | @staticmethod 40 | def get_full_id(word_id): 41 | return reverse("word", args=[word_id]) if word_id else None 42 | 43 | def to_dict(self): 44 | d = { 45 | "@id": Word.get_full_id(self.word_id), 46 | "@type": "word", 47 | "verse_id": Verse.get_full_id(self.verse_id), 48 | "paragraph_id": Paragraph.get_full_id(self.paragraph_id), 49 | "sentence_id": Sentence.get_full_id(self.sentence_id), 50 | "pos": self.pos.strip("-"), 51 | "crit_text": self.crit_text, 52 | "text": self.text, 53 | "word": self.word, 54 | "norm": self.norm, 55 | "lemma": self.lemma, 56 | "dep_type": self.dep_type, 57 | "head": Word.get_full_id(self.head), 58 | } 59 | d.update(parse_as_dict(self.parse)) 60 | return d 61 | 62 | 63 | class Paragraph(models.Model): 64 | 65 | paragraph_id = models.CharField(max_length=5, db_index=True) 66 | book_osis_id = models.CharField(max_length=6, db_index=True) 67 | 68 | prev_paragraph = models.CharField(max_length=5, null=True) 69 | next_paragraph = models.CharField(max_length=5, null=True) 70 | 71 | class Meta: 72 | ordering = ["paragraph_id"] 73 | 74 | @staticmethod 75 | def get_full_id(paragraph_id): 76 | return reverse("paragraph", args=[paragraph_id]) if paragraph_id else None 77 | 78 | def words(self): 79 | return Word.objects.filter(paragraph_id=self.paragraph_id) 80 | 81 | def to_dict(self): 82 | words = list(self.words()) 83 | first_verse_id = words[0].verse_id 84 | last_verse_id = words[-1].verse_id 85 | return { 86 | "@id": Paragraph.get_full_id(self.paragraph_id), 87 | "title": verse_range_title(first_verse_id, last_verse_id), 88 | "@type": "paragraph", 89 | "prev": Paragraph.get_full_id(self.prev_paragraph), 90 | "next": Paragraph.get_full_id(self.next_paragraph), 91 | "book": Book.get_full_id(self.book_osis_id), 92 | "words": [w.to_dict() for w in words], 93 | } 94 | 95 | 96 | class Sentence(models.Model): 97 | 98 | sentence_id = models.CharField(max_length=6, db_index=True) 99 | book_osis_id = models.CharField(max_length=6, db_index=True) 100 | 101 | prev_sentence = models.CharField(max_length=6, null=True) 102 | next_sentence = models.CharField(max_length=6, null=True) 103 | 104 | class Meta: 105 | ordering = ["sentence_id"] 106 | 107 | @staticmethod 108 | def get_full_id(sentence_id): 109 | return reverse("sentence", args=[sentence_id]) if sentence_id else None 110 | 111 | def words(self): 112 | return Word.objects.filter(sentence_id=self.sentence_id) 113 | 114 | def to_dict(self): 115 | words = list(self.words()) 116 | first_verse_id = words[0].verse_id 117 | last_verse_id = words[-1].verse_id 118 | return { 119 | "@id": Sentence.get_full_id(self.sentence_id), 120 | "title": verse_range_title(first_verse_id, last_verse_id), 121 | "@type": "sentence", 122 | "prev": Sentence.get_full_id(self.prev_sentence), 123 | "next": Sentence.get_full_id(self.next_sentence), 124 | "book": Book.get_full_id(self.book_osis_id), 125 | "words": [w.to_dict() for w in words], 126 | } 127 | 128 | 129 | class Verse(models.Model): 130 | 131 | verse_id = models.CharField(max_length=6, db_index=True) 132 | book_osis_id = models.CharField(max_length=6, db_index=True) 133 | 134 | prev_verse = models.CharField(max_length=6, null=True) 135 | next_verse = models.CharField(max_length=6, null=True) 136 | 137 | class Meta: 138 | ordering = ["verse_id"] 139 | 140 | @staticmethod 141 | def get_full_id(verse_id): 142 | return reverse("verse", args=[verse_id]) if verse_id else None 143 | 144 | def words(self): 145 | return Word.objects.filter(verse_id=self.verse_id) 146 | 147 | def to_dict(self): 148 | return { 149 | "@id": reverse("verse", args=[self.verse_id]), 150 | "title": verse_from_bcv(self.verse_id).title, 151 | "@type": "verse", 152 | "book": reverse("book", args=[self.book_osis_id]), 153 | "prev": Verse.get_full_id(self.prev_verse), 154 | "next": Verse.get_full_id(self.next_verse), 155 | "words": [w.to_dict() for w in self.words()], 156 | } 157 | 158 | 159 | class Book(models.Model): 160 | 161 | book_osis_id = models.CharField(max_length=6, db_index=True) 162 | 163 | name = models.CharField(max_length=20) 164 | sblgnt_id = models.CharField(max_length=2) 165 | 166 | class Meta: 167 | ordering = ["sblgnt_id"] 168 | 169 | @staticmethod 170 | def get_full_id(book_osis_id): 171 | return reverse("book", args=[book_osis_id]) if book_osis_id else None 172 | 173 | def first_verse(self): 174 | return Verse.objects.filter(book_osis_id=self.book_osis_id)[0] 175 | 176 | def first_sentence(self): 177 | return Sentence.objects.filter(book_osis_id=self.book_osis_id)[0] 178 | 179 | def first_paragraph(self): 180 | return Paragraph.objects.filter(book_osis_id=self.book_osis_id)[0] 181 | 182 | def to_dict(self): 183 | return { 184 | "@id": Book.get_full_id(self.book_osis_id), 185 | "@type": "book", 186 | "name": self.name, 187 | "root": reverse("root"), 188 | "first_verse": Verse.get_full_id(self.first_verse().verse_id), 189 | "first_sentence": Sentence.get_full_id(self.first_sentence().sentence_id), 190 | "first_paragraph": Paragraph.get_full_id(self.first_paragraph().paragraph_id), 191 | } 192 | -------------------------------------------------------------------------------- /morphgnt_api/ref.py: -------------------------------------------------------------------------------- 1 | import re 2 | 3 | BOOK_NAMES = [ 4 | # sblgnt file, paratext, osis, full 5 | ("Mt", "MAT", "Matt", "Matthew"), 6 | ("Mk", "MRK", "Mark", "Mark"), 7 | ("Lk", "LUK", "Luke", "Luke"), 8 | ("Jn", "JHN", "John", "John"), 9 | ("Ac", "ACT", "Acts", "Acts"), 10 | ("Ro", "ROM", "Rom", "Romans"), 11 | ("1Co", "1CO", "1Cor", "1 Corinthians"), 12 | ("2Co", "2CO", "2Cor", "2 Corinthians"), 13 | ("Ga", "GAL", "Gal", "Galatians"), 14 | ("Eph", "EPH", "Eph", "Ephesians"), 15 | ("Php", "PHP", "Phil", "Philippians"), 16 | ("Col", "COL", "Col", "Colossians"), 17 | ("1Th", "1TH", "1Thess", "1 Thessalonians"), 18 | ("2Th", "2TH", "2Thess", "2 Thessalonians"), 19 | ("1Ti", "1TI", "1Tim", "1 Timothy"), 20 | ("2Ti", "2TI", "2Tim", "2 Timothy"), 21 | ("Tit", "TIT", "Titus", "Titus"), 22 | ("Phm", "PHM", "Phlm", "Philemon"), 23 | ("Heb", "HEB", "Heb", "Hebrews"), 24 | ("Jas", "JAS", "Jas", "James"), 25 | ("1Pe", "1PE", "1Pet", "1 Peter"), 26 | ("2Pe", "2PE", "2Pet", "2 Peter"), 27 | ("1Jn", "1JN", "1John", "1 John"), 28 | ("2Jn", "2JN", "2John", "2 John"), 29 | ("3Jn", "3JN", "3John", "3 John"), 30 | ("Jud", "JUD", "Jude", "Jude"), 31 | ("Re", "REV", "Rev", "Revelation"), 32 | ] 33 | 34 | BOOK_NAME_MAPPINGS = { 35 | name: i 36 | for i, name_set in enumerate(BOOK_NAMES, 1) 37 | for name in set(name_set) 38 | } 39 | 40 | BOOK_NAME_RE = "|".join(BOOK_NAME_MAPPINGS) 41 | 42 | OSIS_NAME = [None, *[row[2] for row in BOOK_NAMES]] 43 | FULL_NAME = [None, *[row[3] for row in BOOK_NAMES]] 44 | 45 | 46 | class Verse: 47 | def __init__(self, book_num, chapter_num, verse_num): 48 | self.book_num = book_num 49 | self.chapter_num = chapter_num 50 | self.verse_num = verse_num 51 | 52 | @property 53 | def bcv(self): 54 | "returns a BBCCVV string for the verse" 55 | return f"{self.book_num:02d}{self.chapter_num:02d}{self.verse_num:02d}" 56 | 57 | @property 58 | def tup(self): 59 | "returns a (B, C, V) tuple of integers for the verse" 60 | return (self.book_num, self.chapter_num, self.verse_num) 61 | 62 | @property 63 | def title(self): 64 | "returns {FULL_NAME} {CHAPTER}.{VERSE} for the verse" 65 | book_name = FULL_NAME[self.book_num] 66 | return f"{book_name} {self.chapter_num}.{self.verse_num}" 67 | 68 | def scoped_title(self, other_verse): 69 | if self.book_num == other_verse.book_num: 70 | if self.chapter_num == other_verse.chapter_num: 71 | return f"{self.verse_num}" 72 | else: 73 | return f"{self.chapter_num}.{self.verse_num}" 74 | else: 75 | return self.title 76 | 77 | 78 | VERSE_REF_RE = re.compile( 79 | rf"(?P({BOOK_NAME_RE}))\s(?P\d+)(:|.)(?P\d+)$" 80 | ) 81 | 82 | 83 | def verse(txt): 84 | m = VERSE_REF_RE.match(txt) 85 | if not m: 86 | raise ValueError("can't parse verse reference") 87 | 88 | return Verse( 89 | BOOK_NAME_MAPPINGS[m["book"]], 90 | int(m["chapter"]), 91 | int(m["verse"]) 92 | ) 93 | 94 | 95 | class Chapter: 96 | def __init__(self, book_num, chapter_num): 97 | self.book_num = book_num 98 | self.chapter_num = chapter_num 99 | 100 | @property 101 | def bc(self): 102 | "returns a BBCC string for the chapter" 103 | return f"{self.book_num:02d}{self.chapter_num:02d}" 104 | 105 | @property 106 | def tup(self): 107 | "returns a (B, C) tuple of integers for the chapter" 108 | return (self.book_num, self.chapter_num) 109 | 110 | @property 111 | def title(self): 112 | "returns {FULL_NAME} {CHAPTER} for the chapter" 113 | book_name = FULL_NAME[self.book_num] 114 | return f"{book_name} {self.chapter_num}" 115 | 116 | @property 117 | def filename(self): 118 | "returns filename-compatible string for chapter" 119 | book_name = OSIS_NAME[self.book_num] 120 | return book_name.replace(" ", "").lower() + f"_{self.chapter_num:02d}" 121 | 122 | def verse(self, verse_num): 123 | return Verse(self.book_num, self.chapter_num, verse_num) 124 | 125 | 126 | CHAPTER_REF_RE = re.compile( 127 | rf"(?P({BOOK_NAME_RE}))\s(?P\d+)$" 128 | ) 129 | 130 | 131 | def chapter(txt): 132 | m = CHAPTER_REF_RE.match(txt) 133 | if not m: 134 | raise ValueError("can't parse chapter reference") 135 | 136 | return Chapter( 137 | BOOK_NAME_MAPPINGS[m["book"]], 138 | int(m["chapter"]) 139 | ) 140 | 141 | 142 | class Book: 143 | def __init__(self, num, osis, title): 144 | self.num = num 145 | self.osis = osis 146 | self.title = title 147 | 148 | @property 149 | def filename(self): 150 | "returns filename-compatible string for book" 151 | return self.osis.replace(" ", "").lower() 152 | 153 | 154 | NT_BOOKS = [ 155 | Book(index, osis, full) 156 | for index, (sblgnt, paratext, osis, full) in enumerate(BOOK_NAMES, 1) 157 | ] 158 | 159 | 160 | def verse_from_bcv(bcv): 161 | b = int(bcv[0:2]) - 60 162 | c = int(bcv[2:4]) 163 | v = int(bcv[4:6]) 164 | return Verse(b, c, v) 165 | 166 | 167 | def verse_range_title(bcv1, bcv2): 168 | v1 = verse_from_bcv(bcv1) 169 | if bcv1 == bcv2: 170 | return v1.title 171 | v2 = verse_from_bcv(bcv2) 172 | return f"{v1.title}–{v2.scoped_title(v1)}" 173 | 174 | -------------------------------------------------------------------------------- /morphgnt_api/settings.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | import dj_database_url 4 | 5 | # SECURITY WARNING: keep the secret key used in production secret! 6 | SECRET_KEY = 'g7j+tn)pjz_sw7j3$z_c*^6gwv43b7!%1&#!nt2)q2u&6i#@qx' 7 | 8 | # SECURITY WARNING: don't run with debug turned on in production! 9 | DEBUG = bool(int(os.environ.get("DEBUG", "1"))) 10 | 11 | ALLOWED_HOSTS = [ 12 | "localhost", 13 | "api.morphgnt.org", 14 | "*.herokuapp.com" 15 | ] 16 | 17 | 18 | # Application definition 19 | 20 | INSTALLED_APPS = [ 21 | "morphgnt_api", 22 | "corsheaders", 23 | ] 24 | MIDDLEWARE = [ 25 | "corsheaders.middleware.CorsMiddleware", 26 | ] 27 | 28 | ROOT_URLCONF = "morphgnt_api.urls" 29 | WSGI_APPLICATION = "morphgnt_api.wsgi.application" 30 | 31 | 32 | # Database 33 | 34 | DATABASES = { 35 | "default": dj_database_url.config(default="postgres://localhost/morphgnt_api") 36 | } 37 | 38 | CORS_ORIGIN_ALLOW_ALL = True 39 | -------------------------------------------------------------------------------- /morphgnt_api/urls.py: -------------------------------------------------------------------------------- 1 | from django.conf.urls import include, url 2 | 3 | from . import views 4 | 5 | 6 | urlpatterns = [ 7 | url(r"^v0/word/(?P\d{11}).json$", views.word, name="word"), 8 | 9 | url(r"^v0/paragraph/(?P\d{5}).json$", views.paragraph, name="paragraph"), 10 | url(r"^v0/sentence/(?P\d{6}).json$", views.sentence, name="sentence"), 11 | url(r"^v0/verse/(?P\d{6}).json$", views.verse, name="verse"), 12 | 13 | url(r"^v0/book/(?P\w+).json$", views.book, name="book"), 14 | 15 | url(r"^v0/verse-lookup/$", views.verse_lookup, name="verse_lookup"), 16 | url(r"^v0/frequency/$", views.frequency, name="frequency"), 17 | url(r"^v0/kwic/$", views.kwic, name="kwic"), 18 | 19 | url(r"^v0/root.json$", views.root, name="root"), 20 | 21 | url(r"^loaderio-97c892ba1d7ce63eb626ca62a7a0a961/$", views.loader_verification), 22 | 23 | url(r"^$", views.home, name="home"), 24 | ] 25 | -------------------------------------------------------------------------------- /morphgnt_api/views.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | from django.core.urlresolvers import reverse 4 | from django.db import connection 5 | from django.http import HttpResponse, JsonResponse 6 | from django.shortcuts import get_object_or_404 7 | 8 | from morphgnt_api.models import Word, Book, Verse, Paragraph, Sentence 9 | 10 | from . import ref 11 | 12 | 13 | def resource_view(model): 14 | def _(request, **kwargs): 15 | response = JsonResponse(get_object_or_404(model, **kwargs).to_dict()) 16 | return response 17 | return _ 18 | 19 | 20 | word = resource_view(Word) 21 | paragraph = resource_view(Paragraph) 22 | sentence = resource_view(Sentence) 23 | verse = resource_view(Verse) 24 | book = resource_view(Book) 25 | 26 | 27 | def root(request): 28 | response = JsonResponse({ 29 | "books": [ 30 | { 31 | "@id": reverse("book", args=[book.book_osis_id]), 32 | "osis": book.book_osis_id, 33 | "name": book.name, 34 | } 35 | for book in Book.objects.order_by("sblgnt_id") 36 | ] 37 | }) 38 | return response 39 | 40 | 41 | def verse_lookup(request): 42 | try: 43 | verse = ref.verse(list(request.GET.keys())[0]) 44 | except ValueError as e: 45 | response = JsonResponse({"message": str(e)}, status=400) 46 | else: 47 | verse_id = f"{verse.book_num+60:02d}{verse.chapter_num:02d}{verse.verse_num:02d}" 48 | response = JsonResponse({ 49 | "verse_id": reverse("verse", args=[verse_id]), 50 | }) 51 | return response 52 | 53 | 54 | def frequency(request): 55 | 56 | payload = json.loads(request.body) 57 | output = [] 58 | 59 | lemmas = [item["lemma"] for item in payload["input"]] 60 | 61 | cursor = connection.cursor() 62 | cursor.execute("select lemma, count(*) from morphgnt_api_word where lemma IN %s group by lemma", (tuple(lemmas),)) 63 | counts = dict(cursor.fetchall()) 64 | 65 | for item in payload["input"]: 66 | id_, lemma = item["id"], item["lemma"] 67 | lemma_count = counts[lemma] 68 | output.append({"id": id_, "count": lemma_count}) 69 | 70 | return JsonResponse({"output": output}) 71 | 72 | 73 | def kwic(request): 74 | try: 75 | word = list(request.GET.keys())[0] 76 | except ValueError as e: 77 | response = JsonResponse({"message": str(e)}, status=400) 78 | else: 79 | words = list(Word.objects.filter( 80 | verse_id__in=Word.objects.filter(word=word).values("verse_id") 81 | ).values_list("text", "word", "verse_id")) 82 | 83 | results = [] 84 | 85 | for index, item in enumerate(words): 86 | if item[1] == word: 87 | pre = [] 88 | post = [] 89 | for i in range(max(0, index - 5), index): 90 | if words[i][2] == item[2]: 91 | pre.append(words[i][0]) 92 | for i in range(index + 1, min(len(words), index + 6)): 93 | if words[i][2] == item[2]: 94 | post.append(words[i][0]) 95 | 96 | results.append({ 97 | "verse_id": reverse("verse", args=[item[2]]), 98 | "title": ref.verse_from_bcv(item[2]).title, 99 | "pre": " ".join(pre), 100 | "keyword": item[0], 101 | "post": " ".join(post), 102 | }) 103 | 104 | response = JsonResponse({ 105 | "word": word, 106 | "results": results, 107 | }) 108 | return response 109 | 110 | 111 | def home(request): 112 | return HttpResponse("Go to {link} for API. See https://github.com/morphgnt/morphgnt-api for documentation.".format(link=reverse("root")), content_type="text/html") 113 | 114 | 115 | def loader_verification(request): 116 | return HttpResponse("loaderio-97c892ba1d7ce63eb626ca62a7a0a961") 117 | -------------------------------------------------------------------------------- /morphgnt_api/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for morphgnt_api project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/1.8/howto/deployment/wsgi/ 8 | """ 9 | 10 | import os 11 | 12 | from django.core.wsgi import get_wsgi_application 13 | 14 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "morphgnt_api.settings") 15 | 16 | application = get_wsgi_application() 17 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Django==1.11 2 | 3 | psycopg2==2.7.7 4 | gunicorn==19.7.1 5 | 6 | dj-database-url==0.4.2 7 | 8 | django-cors-headers==2.0.2 9 | -------------------------------------------------------------------------------- /runtime.txt: -------------------------------------------------------------------------------- 1 | python-3.6.8 2 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [flake8] 2 | ignore = E501 3 | --------------------------------------------------------------------------------