├── LICENSE
├── README.md
└── bible_gateway.py
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2024 Jaden Zaleski
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # BibleTranslations
2 |
3 | 
4 | 
5 | 
6 |
7 |
8 | Here you can generate a formatted version of all the Holy Bible translations that are availabe at Bible Gateway, in JSON and SQL format.
9 | Text is downloaded with the help of the meaningless package. Run the bible_gateway.py script to get started. Feel free to create any issues or pull requests as needed.
10 | I will add more versions as they are supported by the meaningless package.
11 |
12 | > [!WARNING]
13 | > Due to copyright issues, all formatted bible text has been removed from the repository. If you want to use the
14 | > formatted files, you will have to generate them yourself with the script.
15 |
16 | > [!IMPORTANT]
17 | > This repository is **actively under development**. You can follow along in the bible-translations project. You’re welcome to open issues for feature ideas or bug reports.
18 |
19 | Available Translations:
20 |
21 | * **AMP**
22 | * **ASV**
23 | * **AKJV**
24 | * **BRG**
25 | * **CSB**
26 | * **EHV**
27 | * **ESV**
28 | * **ESVUK**
29 | * **GNV**
30 | * **GW**
31 | * **ISV**
32 | * **JUB**
33 | * **KJV**
34 | * **KJ21**
35 | * **LEB**
36 | * **MEV**
37 | * **NASB**
38 | * **NASB1995**
39 | * **NET**
40 | * **NIV**
41 | * **NIVUK**
42 | * **NKJV**
43 | * **NLT**
44 | * **NLV**
45 | * **NMB\***
46 | * **NOG**
47 | * **NRSV**
48 | * **NRSVUE**
49 | * **WEB**
50 | * **YLT**
51 | * **RVA\***
52 | > [!IMPORTANT]
53 | > \* Both the **NMB** and **RVA** translations are not generated by the script since complete versions of them are not available on [Bible Gateway](https://www.biblegateway.com) yet. The [meaningless](https://github.com/daniel-tran/meaningless) package still works for those translations, just not the entire Bible.
54 |
55 | If you would like more resources, I have found that the
56 | [Holy-Bible-XML-Format](https://github.com/Beblia/Holy-Bible-XML-Format) is a great repo if you are willing to use XML.
57 |
58 | # Disclaimer
59 |
60 | The Bible text formatted by the script(s) in this repository is for educational, personal, non-commercial, and reference
61 | purposes only.
62 |
63 | Many of the Bible translations listed here are protected by copyright and may not be legally redistributed or used in
64 | other projects without explicit permission from their respective copyright holders.
65 |
66 | Bible text is retrieved via the [meaningless](https://github.com/daniel-tran/meaningless) package, which sources content
67 | from [BibleGateway.com](https://www.biblegateway.com). Use of this content may be subject to BibleGateway's Terms of
68 | Service and the individual licenses for each Bible translation. Some translations (such as KJV, ASV, YLT, and WEB) are
69 | in the public domain and may be freely used. Others (e.g., ESV, NIV, NLT, NASB, etc.) are licensed, and require
70 | permission for redistribution or certain uses.
71 |
72 | You are solely responsible for ensuring you have the proper rights or licenses before using, distributing, or publishing
73 | any of these translations.
74 |
75 | If you fork this repository, you must maintain compliance with all applicable copyright laws and licensing requirements.
76 | Forking does not grant you any additional rights to distribute or use copyrighted Bible translations beyond what is
77 | explicitly permitted by their respective copyright holders.
78 |
79 | This project does not claim ownership of any Bible text and is not affiliated with BibleGateway or any copyright holder.
80 |
--------------------------------------------------------------------------------
/bible_gateway.py:
--------------------------------------------------------------------------------
1 | import json
2 | import os
3 | import re
4 | import sys
5 |
6 | import meaningless.utilities.common as common
7 | from meaningless import JSONDownloader
8 | from meaningless.utilities.common import BIBLE_TRANSLATIONS
9 |
10 |
11 | # Replacing the function with a new version to allow for proper download of all verses in a chapter
12 | def custom_get_capped_integer(number, min_value=1, max_value=200):
13 | return min(max(int(number), int(min_value)), int(max_value))
14 |
15 |
16 | # Override the original function with the custom version
17 | common.get_capped_integer = custom_get_capped_integer
18 | # books of the bible in order
19 | books = ["Genesis", "Exodus", "Leviticus", "Numbers", "Deuteronomy", "Joshua", "Judges", "Ruth", "1 Samuel",
20 | "2 Samuel", "1 Kings", "2 Kings", "1 Chronicles", "2 Chronicles", "Ezra", "Nehemiah", "Esther", "Job",
21 | "Psalm", "Proverbs", "Ecclesiastes", "Song Of Solomon", "Isaiah", "Jeremiah", "Lamentations", "Ezekiel",
22 | "Daniel", "Hosea", "Joel", "Amos", "Obadiah", "Jonah", "Micah", "Nahum", "Habakkuk", "Zephaniah", "Haggai",
23 | "Zechariah", "Malachi", "Matthew", "Mark", "Luke", "John", "Acts", "Romans", "1 Corinthians",
24 | "2 Corinthians", "Galatians", "Ephesians", "Philippians", "Colossians", "1 Thessalonians",
25 | "2 Thessalonians", "1 Timothy", "2 Timothy", "Titus", "Philemon", "Hebrews", "James", "1 Peter", "2 Peter",
26 | "1 John", "2 John", "3 John", "Jude", "Revelation"]
27 |
28 | COUNT = 0
29 | TOTAL = 0
30 |
31 |
32 | # download all the books
33 | def download(book_name, folder, v):
34 | all_clear = True
35 | downloader = JSONDownloader(translation=v, show_passage_numbers=False, strip_excess_whitespace=True)
36 |
37 | if not downloader.download_book(book_name, folder + "/" + book_name + ".json") == 1:
38 | all_clear = False
39 |
40 | return all_clear
41 |
42 |
43 | # combine all the books into one json file
44 | def combine(folder, n):
45 | combined_data = {}
46 |
47 | # Iterate through all files in the folder
48 | for file_name in os.listdir(folder):
49 | if file_name.endswith('.json'):
50 | fp = os.path.join(folder, file_name)
51 |
52 | try:
53 | with open(fp, 'r') as f:
54 | data = json.load(f)
55 | # Exclude the "Info" section if present
56 | if "Info" in data:
57 | del data["Info"]
58 | # Remove extra whitespace characters from verse content
59 | for book, chapters in data.items():
60 | for chapter, verses in chapters.items():
61 | for verse_num, verse_content in verses.items():
62 | # Replace newline characters and excess spaces with a single space
63 | data[book][chapter][verse_num] = verse_content.strip()
64 |
65 | combined_data.update(data)
66 | except json.JSONDecodeError as e:
67 | print(f"Error parsing {file_name}: {e}")
68 | continue
69 |
70 | # Write the combined data to the output file in order
71 | ordered_data = {book: combined_data[book] for book in books if book in combined_data}
72 |
73 | with open(n, 'w') as out_file:
74 | json.dump(ordered_data, out_file, indent=4)
75 |
76 |
77 | # a text progress bar
78 | def generate_progress_bar(progress, total, length=20):
79 | progress_ratio = min(progress / total, 1)
80 | progress_bar_length = int(progress_ratio * length)
81 | progress_bar = "#" * progress_bar_length + "-" * (length - progress_bar_length)
82 | return f"[{progress_bar}] {progress:2d}/{total}"
83 |
84 |
85 | def generate_bible(bible_translation, show_progress=True):
86 | # root
87 | if not os.path.exists(bible_translation):
88 | os.makedirs(bible_translation)
89 |
90 | root = bible_translation + "/"
91 | path = root + bible_translation + "_books"
92 | if not os.path.exists(path):
93 | os.makedirs(path)
94 |
95 | files = os.listdir(path)
96 | total_files = len(files)
97 | # delete all files in folder
98 | for i, file in enumerate(files):
99 | file_path = os.path.join(path, file)
100 | os.remove(file_path)
101 | if show_progress:
102 | print("\rDeleted " + str(total_files) + " files.")
103 | # download all files
104 | flag = ""
105 | for i, book in enumerate(books):
106 | global COUNT, TOTAL
107 | COUNT += 1
108 | if not show_progress:
109 | print(f"\r[+] Downloading {bible_translation[:8]:<8} ({generate_progress_bar(COUNT, TOTAL, 40)})"
110 | f" ({round((COUNT / TOTAL) * 100)}%)", end="")
111 | if not download(books[i], path, bible_translation):
112 | flag = book
113 | break
114 | if show_progress:
115 | print(
116 | f"\r[+] Downloading book: {book[:15]:<15} ({generate_progress_bar(i + 1, len(books), 30)})", end="")
117 |
118 | if flag != "":
119 | if show_progress:
120 | print("\r[+] ERROR: " + flag + " failed to download.")
121 | else:
122 | if show_progress:
123 | print("\r[+] Download complete.")
124 |
125 | # combine all books
126 | combine(path, root + bible_translation + "_bible.json")
127 | if show_progress:
128 | print("[+] All books combined into: " + root + bible_translation + "_bible.json")
129 | # generate sql
130 | out_name = root + bible_translation + "_bible.sql"
131 | in_name = root + bible_translation + "_bible.json"
132 | with open(out_name, 'w') as output_file:
133 | with open(in_name, 'r') as input_file:
134 | output_file.write(
135 | "create table " + bible_translation.lower() + "(book_id int not null, book varchar(255) not null, "
136 | "chapter "
137 | "int not null, verse int not null, text varchar(1000) not "
138 | "null, primary key (book_id, chapter, verse));\n\n")
139 |
140 | cd = json.load(input_file)
141 |
142 | for book, chapters in cd.items():
143 | for chapter, verses in chapters.items():
144 | output_file.write(
145 | "INSERT INTO " + bible_translation.lower() + "(book_id, book, chapter, verse, text) "
146 | "VALUES\n")
147 | for verse_num, verse_content in verses.items():
148 | book_id = books.index(book) + 1
149 | verse_content = re.sub(r'\s+', ' ', verse_content)
150 | verse_content = verse_content.replace("'", "''")
151 | output_file.write("(" + str(
152 | book_id) + ",'" + book + "'," + chapter + "," + verse_num + ",'" + verse_content + "')")
153 | # Check if it's the last line
154 | if verse_num == list(verses.keys())[-1]:
155 | output_file.write(";\n")
156 | else:
157 | output_file.write(",\n")
158 |
159 | if show_progress:
160 | print("[+] SQL file created: " + out_name)
161 |
162 |
163 | if __name__ == '__main__':
164 | print("[+] Available Translations: ")
165 | for bt in BIBLE_TRANSLATIONS.keys():
166 | sys.stdout.write(bt + " ")
167 | TOTAL += 66
168 |
169 | download_all = input("\n[+] Download all translations (Y/N): ").upper()
170 | if download_all == "Y":
171 | bibles_trans = list(BIBLE_TRANSLATIONS.keys())
172 | # remove NMB since it's not complete
173 | bibles_trans.remove("NMB")
174 | bibles_trans.remove("RVA")
175 | bibles_trans.sort()
176 | TOTAL -= 66 * 2
177 | for t in bibles_trans:
178 | generate_bible(t, show_progress=False)
179 | print("\n[+] All translations downloaded!")
180 |
181 | else:
182 | translation = input("[+] Translation: ").upper()
183 | generate_bible(translation)
184 |
--------------------------------------------------------------------------------