├── .github └── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md ├── .gitignore ├── CONTRIBUTORS.txt ├── COPYING.txt ├── DONORS.txt ├── README.rst ├── gui ├── __init__.py ├── about.py ├── backups.py ├── help.py ├── main.py ├── starter.py └── version.py ├── icon.ico ├── mutf8 ├── LICENSE ├── README.md ├── __init__.py ├── cmutf8.c └── mutf8.py ├── nbt ├── CONTRIBUTORS.txt ├── LICENSE.txt ├── MANIFEST.in ├── README.md ├── README.txt ├── __init__.py ├── chunk.py ├── nbt.py ├── region.py └── world.py ├── progressbar ├── __init__.py ├── compat.py ├── progressbar.py └── widgets.py ├── regionfixer.py ├── regionfixer_core ├── bug_reporter.py ├── constants.py ├── interactive.py ├── progressbar.py ├── scan.py ├── util.py ├── version.py └── world.py ├── regionfixer_gui.py └── setup.py /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: Bug 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **Full copied text from the MS-DOS view** 14 | Please, include the command used to run regionfixer. 15 | 16 | **Expected behavior** 17 | A clear and concise description of what you expected to happen. 18 | 19 | **Screenshots** 20 | If applicable, add screenshots to help explain your problem. 21 | 22 | **Files that would help solving the issue** 23 | If possible, the world/files that triggers the error. 24 | 25 | **Desktop (please complete the following information):** 26 | - OS: [e.g. iOS] 27 | - Python version: [e.g. 2.7] 28 | - Region Fixer Version [e.g. 2.0.1] 29 | 30 | **Additional context** 31 | Add any other context about the problem here. 32 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: Feature request 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe how the solution would be implemented** 17 | If possible, describe how the solution would be implemented. 18 | 19 | **Describe alternatives you've considered** 20 | A clear and concise description of any alternative solutions or features you've considered. 21 | 22 | **Additional context** 23 | Add any other context or screenshots about the feature request here. 24 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # bytecompiled python files 2 | *.pyc 3 | -------------------------------------------------------------------------------- /CONTRIBUTORS.txt: -------------------------------------------------------------------------------- 1 | Original author: 2 | 3 | Fenixin (Alejandro Aguilera) - Main developer 4 | 5 | Contributors (in no particular order): 6 | 7 | aheadley (Alex Headley) - First multiprocessing version of Region Fixer. 8 | 734F96 (Lavander) - Update RegionFixer for Minecraft 1.18 9 | sleiss (Simon Leiß) - Fix typos 10 | kbn (Kristian Berge) - Small fixes 11 | KasperFranz (Kasper Sanguesa-Franz) - Fix typo in readme 12 | macfreek (Freek Dijkstra) - Fixes and lots of help 13 | Pisich (carloser) - Changes to the readme 14 | carlallen (Carl Allen) - Fix problem in MacOS 15 | charlyhue (Charly Hue) - Fix logging with onliners 16 | andm (andm) - Fix typos 17 | sandtechnology (sandtechnology) - Fix problem scanning old worlds 18 | -------------------------------------------------------------------------------- /DONORS.txt: -------------------------------------------------------------------------------- 1 | Donors (random order): 2 | Crappotron (Minecraft forums) 3 | Travis Wicks 4 | Nico van Duuren (Knights and Merchants) 5 | Diana Rotter 6 | Biocraft 7 | Andrew Van Hise 8 | Eugene Sterner 9 | Udell Ross Burton 10 | Powercraft Network 11 | David Wilczewski 12 | 13 | Sponsors: 14 | Initial development was sponsored by: NITRADO Servers (http://nitrado.net) 15 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ====================== 2 | Minecraft Region Fixer 3 | ====================== 4 | 5 | By Alejandro Aguilera (Fenixin) 6 | 7 | Locates problems and tries to fix Minecraft worlds (or region files). 8 | 9 | Minecraft Region Fixer tries to fix corrupted chunks in region files using old backup copies 10 | of the Minecraft world. If you don't have a copy, you can eliminate the 11 | corrupted chunks making Minecraft regenerate them. 12 | 13 | It also scans the 'level.dat' file and the player '\*.dat' and tries to 14 | read them. If there are any problems it prints warnings. At the moment 15 | it doesn't fix any problem in these files. 16 | 17 | Web page: 18 | https://github.com/Fenixin/Minecraft-Region-Fixer 19 | 20 | Mincraft forums posts: 21 | https://www.minecraftforum.net/forums/support/server-support-and/1903200-minecraft-region-fixer 22 | https://www.minecraftforum.net/forums/mapping-and-modding-java-edition/minecraft-tools/1261480-minecraft-region-fixer 23 | 24 | Supported platforms 25 | =================== 26 | This program only works with Python 3.x, and DOESN'T work with 27 | python 2.x. There was a Windows .exe for older versions, but right 28 | now you need to install the python interpreter to run this 29 | program. 30 | 31 | Notes 32 | ===== 33 | Older versions of Minecraft had big problems when loading broken 34 | worlds. Newer versions of Minecraft are improving the way 35 | they deal with corruption and other things. 36 | 37 | Region-Fixer still is useful for replacing chunks/regions with a 38 | backup, removing entities, or trying to see what's going wrong 39 | with your world. 40 | 41 | 42 | Usage 43 | ===== 44 | You can read the program help running: "python regionfixer.py --help" 45 | 46 | For usage examples and more info visit the wiki: 47 | 48 | https://github.com/Fenixin/Minecraft-Region-Fixer/wiki/Usage 49 | 50 | 51 | Bugs, suggestions, feedback, questions 52 | ====================================== 53 | Suggestions and bugs should go to the github page: 54 | 55 | https://github.com/Fenixin/Minecraft-Region-Fixer 56 | 57 | Feedback and questions should preferably go to these forums posts: 58 | 59 | (server administration) 60 | https://www.minecraftforum.net/forums/support/server-support-and/1903200-minecraft-region-fixer 61 | 62 | (mapping and modding) 63 | https://www.minecraftforum.net/forums/mapping-and-modding-java-edition/minecraft-tools/1261480-minecraft-region-fixer 64 | 65 | 66 | Donations and sponsors 67 | ====================== 68 | Region-Fixer was created thanks to sponsors and donations. You can find 69 | information about that in DONORS.txt 70 | 71 | 72 | Contributors 73 | ============ 74 | See CONTRIBUTORS.txt 75 | 76 | 77 | Warning 78 | ======= 79 | This program has been tested with a lot of worlds, but there may exist 80 | bugs, so please, MAKE A BACKUP OF YOUR WORLD BEFORE RUNNING it, 81 | I'M NOT RESPONSIBLE OF WHAT HAPPENS TO YOUR WORLD. Other way to say it 82 | is USE THIS TOOL AT YOUR OWN RISK. 83 | 84 | Think that you are playing with your precious saved games :P . 85 | 86 | Good luck! :) 87 | -------------------------------------------------------------------------------- /gui/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | from .main import MainWindow 5 | from .backups import BackupsWindow 6 | from .starter import Starter 7 | -------------------------------------------------------------------------------- /gui/about.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import wx 5 | 6 | from regionfixer_core.version import version_string as rf_ver 7 | from gui.version import version_string as gui_ver 8 | 9 | 10 | class AboutWindow(wx.Frame): 11 | def __init__(self, parent, title="About"): 12 | wx.Frame.__init__(self, parent, title=title, 13 | style=wx.CLOSE_BOX | wx.RESIZE_BORDER | wx.CAPTION) 14 | # Every windows should use panel as parent. Not doing so will 15 | # make the windows look non-native (very ugly) 16 | panel = wx.Panel(self) 17 | 18 | self.about1 = wx.StaticText(panel, style=wx.ALIGN_CENTER, 19 | label="Minecraft Region-Fixer (GUI) (ver. {0})\n(using Region-Fixer ver. {1})".format(gui_ver,rf_ver)) 20 | self.about2 = wx.StaticText(panel, style=wx.ALIGN_CENTER, 21 | label="Fix problems in Minecraft worlds.") 22 | self.about3 = wx.StaticText(panel, style=wx.ALIGN_CENTER, 23 | label="Official-web:") 24 | self.link_github = wx.HyperlinkCtrl(panel, wx.ID_ABOUT, 25 | "https://github.com/Fenixin/Minecraft-Region-Fixer", 26 | "https://github.com/Fenixin/Minecraft-Region-Fixer", 27 | style=wx.ALIGN_CENTER) 28 | self.about4 = wx.StaticText(panel, 29 | style=wx.TE_MULTILINE | wx.ALIGN_CENTER, 30 | label="Minecraft forums post:") 31 | self.link_minecraft_forums = wx.HyperlinkCtrl(panel, wx.ID_ABOUT, 32 | "http://www.minecraftforum.net/topic/302380-minecraft-region-fixer/", 33 | "http://www.minecraftforum.net/topic/302380-minecraft-region-fixer/", 34 | style=wx.ALIGN_CENTER) 35 | 36 | self.close_button = wx.Button(panel, wx.ID_CLOSE) 37 | 38 | self.sizer = wx.BoxSizer(wx.VERTICAL) 39 | self.sizer.Add(self.about1, 0, wx.ALIGN_CENTER | wx.TOP, 10) 40 | self.sizer.Add(self.about2, 0, wx.ALIGN_CENTER| wx.TOP, 20) 41 | self.sizer.Add(self.about3, 0, wx.ALIGN_CENTER | wx.TOP, 20) 42 | self.sizer.Add(self.link_github, 0, wx.ALIGN_CENTER | wx.ALL, 5) 43 | self.sizer.Add(self.about4, 0, wx.ALIGN_CENTER | wx.TOP, 20) 44 | self.sizer.Add(self.link_minecraft_forums, 0,wx.ALIGN_CENTER | wx.ALL, 5) 45 | self.sizer.Add(self.close_button, 0, wx.ALIGN_CENTER | wx.ALL, 20) 46 | 47 | # Fit sizers and make the windows not resizable 48 | panel.SetSizerAndFit(self.sizer) 49 | self.sizer.Fit(self) 50 | size = self.GetSize() 51 | self.SetMinSize(size) 52 | self.SetMaxSize(size) 53 | 54 | self.Bind(wx.EVT_BUTTON, self.OnClose, self.close_button) 55 | 56 | def OnClose(self, e): 57 | self.Show(False) 58 | -------------------------------------------------------------------------------- /gui/backups.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import wx 5 | import os 6 | 7 | # TODO: just copied this file to this module, is a cutre solution 8 | # improve it! See Importing python modules from relative paths, or 9 | # order this in a better way 10 | from regionfixer_core.world import World 11 | 12 | 13 | class BackupsWindow(wx.Frame): 14 | def __init__(self, parent, title): 15 | wx.Frame.__init__(self, parent, title=title) 16 | # Every windows should use panel as parent. Not doing so will 17 | # make the windows look non-native (very ugly) 18 | panel = wx.Panel(self) 19 | 20 | # Sizer with all the elements in the window 21 | self.all_sizer = wx.BoxSizer(wx.VERTICAL) 22 | 23 | # Text with help in the top 24 | self.help_text = wx.StaticText(panel, style=wx.TE_MULTILINE, 25 | label=("Region-Fixer will use the worlds in\n" 26 | "this list in top-down order.")) 27 | 28 | # List of worlds to use as backups 29 | self.world_list_box = wx.ListBox(panel, size=(180, 100)) 30 | test_list = [] 31 | self.world_list_box.Set(test_list) 32 | # Here will be the worlds to use as backup 33 | self.world_list = test_list[:] 34 | self.world_list_text = test_list[:] 35 | # Last path we used in the file dialog 36 | self.last_path = "" 37 | 38 | # Buttons 39 | self.buttons_sizer = wx.BoxSizer(wx.HORIZONTAL) 40 | self.add = wx.Button(panel, label="Add") 41 | self.move_up = wx.Button(panel, label="Move up") 42 | self.move_down = wx.Button(panel, label="Move down") 43 | self.buttons_sizer.Add(self.add, 0, 0) 44 | self.buttons_sizer.Add(self.move_up, 0, 0) 45 | self.buttons_sizer.Add(self.move_down, 0, 0) 46 | 47 | # Add things to the general sizer 48 | self.all_sizer.Add(self.help_text, proportion=0, 49 | flag=wx.GROW | wx.ALL, border=10) 50 | self.all_sizer.Add(self.world_list_box, proportion=1, 51 | flag=wx.EXPAND | wx.ALL, border=10) 52 | self.all_sizer.Add(self.buttons_sizer, proportion=0, 53 | flag=wx.ALIGN_CENTER | wx.ALL, border=10) 54 | 55 | # Layout sizers 56 | panel.SetSizerAndFit(self.all_sizer) 57 | 58 | # Bindings 59 | self.Bind(wx.EVT_CLOSE, self.OnClose) 60 | self.Bind(wx.EVT_BUTTON, self.OnAddWorld, self.add) 61 | self.Bind(wx.EVT_BUTTON, self.OnMoveUp, self.move_up) 62 | self.Bind(wx.EVT_BUTTON, self.OnMoveDown, self.move_down) 63 | 64 | # Show the window, usually False, True for fast testing 65 | self.Show(False) 66 | 67 | def get_dirs(self, list_dirs): 68 | """ From a list of paths return only the directories. """ 69 | 70 | tmp = [] 71 | for p in self.dirnames: 72 | if os.path.isdir(p): 73 | tmp.append(p) 74 | return tmp 75 | 76 | def are_there_files(self, list_dirs): 77 | """ Given a list of paths return True if there are any files. """ 78 | 79 | for d in list_dirs: 80 | if not os.path.isdir(d): 81 | return True 82 | return False 83 | 84 | def OnAddWorld(self, e): 85 | """ Called when the buttom Add is clicked. """ 86 | 87 | dlg = wx.DirDialog(self, "Choose a Minecraft world folder") 88 | # Set the last path used 89 | dlg.SetPath(self.last_path) 90 | if dlg.ShowModal() == wx.ID_OK: 91 | self.dirname = dlg.GetPath() 92 | # Check if it's a minecraft world 93 | w = World(self.dirname) 94 | if not w.isworld: 95 | error = wx.MessageDialog(self, "This directory doesn't look like a Minecraft world", "Error", wx.ICON_EXCLAMATION) 96 | error.ShowModal() 97 | error.Destroy() 98 | else: 99 | # Insert it in the ListBox 100 | self.world_list.append(w) 101 | index = self.world_list.index(w) 102 | # TODO check if it's a minecraft world 103 | self.world_list_box.InsertItems([w.name], pos = index) 104 | 105 | # Properly recover the last path used 106 | self.last_path = os.path.split(dlg.GetPath())[0] 107 | dlg.Destroy() 108 | 109 | def get_selected_index(self, list_box): 110 | """ Returns the index of the selected item in a list_box. """ 111 | 112 | index = None 113 | for i in range(len(self.world_list)): 114 | if list_box.IsSelected(i): 115 | index = i 116 | return index 117 | 118 | def move_left_inlist(self, l, index): 119 | """ Move the element in the list with index to the left. 120 | 121 | Return the index where the moved element is. 122 | 123 | """ 124 | 125 | tmp = l.pop(index) 126 | index = index - 1 if index != 0 else 0 127 | l.insert(index, tmp) 128 | 129 | return index 130 | 131 | def move_right_inlist(self, l, index): 132 | """ Move the element in the list with index to the right. 133 | 134 | Return the index where the moved element is. 135 | 136 | """ 137 | 138 | len_l = len(l) 139 | tmp = l.pop(index) 140 | index = index + 1 141 | if index == len_l: 142 | l.append(tmp) 143 | index = len_l - 1 144 | else: 145 | l.insert(index, tmp) 146 | 147 | return index 148 | 149 | def get_names_from_worlds(self, world_list): 150 | """ Return a list of names from a list of worlds in order. """ 151 | 152 | t = [] 153 | for i in world_list: 154 | t.append(i.name) 155 | return t 156 | 157 | def OnMoveUp(self, e): 158 | """ Move up in the world list the selected item. """ 159 | 160 | index = self.get_selected_index(self.world_list_box) 161 | 162 | if index is not None: 163 | index = self.move_left_inlist(self.world_list, index) 164 | #~ self.world_list_box.Set(self.world_list) 165 | self.world_list_box.Set(self.get_names_from_worlds(self.world_list)) 166 | self.world_list_box.Select(index) 167 | 168 | def OnMoveDown(self, e): 169 | """ Move down in the world list the selected item. """ 170 | 171 | index = self.get_selected_index(self.world_list_box) 172 | len_world_list = len(self.world_list) 173 | 174 | if index is not None: 175 | index = self.move_right_inlist(self.world_list, index) 176 | self.world_list_box.Set(self.get_names_from_worlds(self.world_list)) 177 | #~ self.world_list_box.Set(self.world_list) 178 | self.world_list_box.Select(index) 179 | 180 | def OnClose(self, e): 181 | """ Ran when the user closes this window. """ 182 | self.Show(False) 183 | -------------------------------------------------------------------------------- /gui/help.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import wx 5 | 6 | class HelpWindow(wx.Frame): 7 | def __init__(self, parent, title="Help"): 8 | wx.Frame.__init__(self, parent, title=title, 9 | style=wx.CLOSE_BOX | wx.RESIZE_BORDER | wx.CAPTION) 10 | # Every windows should use panel as parent. Not doing so will 11 | # make the windows look non-native (very ugly) 12 | panel = wx.Panel(self) 13 | 14 | self.help1 = wx.StaticText(panel, style=wx.ALIGN_CENTER, 15 | label="If you need help you can give a look to the wiki:") 16 | self.link_github = wx.HyperlinkCtrl(panel, wx.ID_ABOUT, 17 | "https://github.com/Fenixin/Minecraft-Region-Fixer/wiki", 18 | style=wx.ALIGN_CENTER, 19 | url="https://github.com/Fenixin/Minecraft-Region-Fixer/wiki") 20 | self.help2 = wx.StaticText(panel, 21 | style=wx.TE_MULTILINE | wx.ALIGN_CENTER, 22 | label="Or ask in the minecraft forums:") 23 | self.link_minecraft_forums = wx.HyperlinkCtrl(panel, wx.ID_ABOUT, 24 | "http://www.minecraftforum.net/topic/302380-minecraft-region-fixer/", 25 | "http://www.minecraftforum.net/topic/302380-minecraft-region-fixer/", 26 | style=wx.ALIGN_CENTER) 27 | 28 | self.close_button = wx.Button(panel, wx.ID_CLOSE) 29 | 30 | self.sizer = wx.BoxSizer(wx.VERTICAL) 31 | self.sizer.Add(self.help1, 0, wx.ALIGN_CENTER | wx.TOP, 10) 32 | self.sizer.Add(self.link_github, 0, wx.ALIGN_CENTER | wx.ALL, 5) 33 | self.sizer.Add(self.help2, 0, wx.ALIGN_CENTER | wx.TOP, 20) 34 | self.sizer.Add(self.link_minecraft_forums, 0, wx.ALIGN_CENTER | wx.ALL, 5) 35 | self.sizer.Add(self.close_button, 0, wx.ALIGN_CENTER | wx.ALL, 20) 36 | 37 | # Fit sizers and make the windows not resizable 38 | panel.SetSizerAndFit(self.sizer) 39 | self.sizer.Fit(self) 40 | size = self.GetSize() 41 | self.SetMinSize(size) 42 | self.SetMaxSize(size) 43 | 44 | self.Bind(wx.EVT_BUTTON, self.OnClose, self.close_button) 45 | 46 | def OnClose(self, e): 47 | self.Show(False) 48 | -------------------------------------------------------------------------------- /gui/main.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import wx 5 | from time import sleep 6 | from os.path import split, abspath 7 | from os import name as os_name 8 | 9 | from .backups import BackupsWindow 10 | from regionfixer_core.scan import AsyncWorldRegionScanner, AsyncDataScanner,\ 11 | ChildProcessException 12 | from regionfixer_core import world 13 | from regionfixer_core.world import World 14 | 15 | if os_name == 'nt': 16 | # Proper way to set an icon in windows 7 and above 17 | # Thanks to http://stackoverflow.com/a/15923439 18 | import ctypes 19 | myappid = 'Fenixin.region-fixer.gui.100' # arbitrary string 20 | ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(myappid) 21 | 22 | 23 | class MainWindow(wx.Frame): 24 | def __init__(self, parent, title, backups=None): 25 | wx.Frame.__init__(self, parent, title=title, size=(300, 400)) 26 | # Every windows should use panel as parent. Not doing so will 27 | # make the windows look non-native (very ugly) 28 | panel = wx.Panel(self) 29 | 30 | self.backups = backups 31 | 32 | # Icon 33 | ico = wx.Icon('icon.ico', wx.BITMAP_TYPE_ICO) 34 | self.SetIcon(ico) 35 | 36 | # Open world stuff 37 | self.last_path = "" # Last path opened 38 | self.world = None # World to scan 39 | 40 | # Status bar 41 | self.CreateStatusBar() 42 | 43 | # Create menu 44 | filemenu = wx.Menu() 45 | windowsmenu = wx.Menu() 46 | helpmenu = wx.Menu() 47 | 48 | # Add elements to filemenu 49 | menuOpen = filemenu.Append(wx.ID_OPEN, "&Open", "Open a Minecraft world") 50 | filemenu.AppendSeparator() 51 | menuExit = filemenu.Append(wx.ID_EXIT, "E&xit","Terminate program") 52 | 53 | # Add elements to helpmenu 54 | menuHelp = helpmenu.Append(wx.ID_HELP, "&Help", "Where to find help") 55 | helpmenu.AppendSeparator() 56 | menuAbout = helpmenu.Append(wx.ID_ABOUT, "&About", "Information about this program") 57 | 58 | # Add elements to windowsmenu 59 | menuBackups = windowsmenu.Append(-1, "&Backups", "Manage list of backups") 60 | # menuAdvanced = windowsmenu.Append(-1, "A&dvanced actions", "Manage list of backups") 61 | 62 | # Create a menu bar 63 | menuBar = wx.MenuBar() 64 | menuBar.Append(filemenu,"&File") 65 | menuBar.Append(windowsmenu,"&View") 66 | menuBar.Append(helpmenu,"&Help") 67 | self.SetMenuBar(menuBar) 68 | 69 | # Create elements in the window 70 | # First row: 71 | self.status_text = wx.StaticText(panel, style=wx.TE_MULTILINE, label="No world loaded") 72 | self.open_button = wx.Button(panel, label="Open") 73 | self.scan_button = wx.Button(panel, label="Scan") 74 | self.scan_button.Disable() 75 | self.firstrow_sizer = wx.BoxSizer(wx.HORIZONTAL) 76 | self.firstrow_sizer.Add(self.status_text, 1, wx.ALIGN_CENTER) 77 | self.firstrow_sizer.Add(self.open_button, 0, wx.EXPAND) 78 | self.firstrow_sizer.Add(self.scan_button, 0, wx.EXPAND) 79 | self.firstrow_static_box = wx.StaticBox(panel, label="World loaded") 80 | self.firstrow_static_box_sizer = wx.StaticBoxSizer(self.firstrow_static_box) 81 | self.firstrow_static_box_sizer.Add(self.firstrow_sizer, 1, wx.EXPAND) 82 | 83 | # Second row: 84 | self.proc_info_text = wx.StaticText(panel, label="Processes to use: ") 85 | self.proc_text = wx.TextCtrl(panel, value="1", size=(30, 24), style=wx.TE_CENTER) 86 | self.el_info_text = wx.StaticText(panel, label="Entity limit: " ) 87 | self.el_text = wx.TextCtrl(panel, value="150", size=(50, 24), style=wx.TE_CENTER) 88 | self.secondrow_sizer = wx.BoxSizer(wx.HORIZONTAL) 89 | self.secondrow_sizer.Add(self.proc_info_text, flag=wx.ALIGN_CENTER) 90 | self.secondrow_sizer.Add(self.proc_text, 0, flag=wx.RIGHT | wx.ALIGN_LEFT, border=15) 91 | self.secondrow_sizer.Add(self.el_info_text, 0, wx.ALIGN_CENTER) 92 | self.secondrow_sizer.Add(self.el_text, 0, wx.ALIGN_RIGHT) 93 | self.secondrow_static_box_sizer = wx.StaticBoxSizer(wx.StaticBox(panel, label="Scan options")) 94 | self.secondrow_static_box_sizer.Add(self.secondrow_sizer, 1, flag=wx.EXPAND) 95 | 96 | # Third row: 97 | # Note: In order to use a static box add it directly to a 98 | # static box sizer and add to the same sizer it's contents 99 | self.results_text = wx.TextCtrl(panel, style=wx.TE_READONLY | wx.TE_MULTILINE, value="Scan the world to get results", size = (500,200)) 100 | # Lets try to create a monospaced font: 101 | ffont = wx.Font(9, wx.FONTFAMILY_MODERN, wx.FONTSTYLE_NORMAL, wx.FONTWEIGHT_NORMAL) 102 | # print ffont.IsFixedWidth() 103 | textattr = wx.TextAttr(font = ffont) 104 | self.results_text.SetFont(ffont) 105 | self.results_text_box = wx.StaticBox(panel, label="Results", size = (100,100)) 106 | self.results_text_box_sizer = wx.StaticBoxSizer(self.results_text_box) 107 | self.results_text_box_sizer.Add(self.results_text, 1, wx.EXPAND) 108 | 109 | self.delete_all_chunks_button = wx.Button(panel, label = "Delete all bad chunks") 110 | self.replace_all_chunks_button = wx.Button(panel, label = "Replace all bad chunks (using backups)") 111 | self.delete_all_regions_button = wx.Button(panel, label = "Delete all bad regions") 112 | self.replace_all_regions_button = wx.Button(panel, label = "Replace all bad regions (using backups)") 113 | self.update_delete_buttons_status(False) 114 | self.update_replace_buttons_status(False) 115 | 116 | self.thirdrow_sizer = wx.BoxSizer(wx.HORIZONTAL) 117 | self.thirdrow_actions_box = wx.StaticBox(panel, label="Actions", size = (-1,-1)) 118 | self.thirdrow_buttons_box_sizer = wx.StaticBoxSizer(self.thirdrow_actions_box) 119 | self.thirdrow_buttons_sizer = wx.BoxSizer(wx.VERTICAL) 120 | self.thirdrow_buttons_sizer.Add(self.delete_all_chunks_button, 1, wx.EXPAND) 121 | self.thirdrow_buttons_sizer.Add(self.replace_all_chunks_button, 1, wx.EXPAND) 122 | self.thirdrow_buttons_sizer.Add(self.delete_all_regions_button, 1, wx.EXPAND) 123 | self.thirdrow_buttons_sizer.Add(self.replace_all_regions_button, 1, wx.EXPAND) 124 | self.thirdrow_buttons_box_sizer.Add(self.thirdrow_buttons_sizer, 1, wx.EXPAND) 125 | self.thirdrow_sizer.Add(self.results_text_box_sizer, 1, wx.EXPAND) 126 | self.thirdrow_sizer.Add(self.thirdrow_buttons_box_sizer, 0, wx.EXPAND) 127 | 128 | # All together now 129 | self.frame_sizer = wx.BoxSizer(wx.VERTICAL) 130 | self.frame_sizer.Add(self.firstrow_static_box_sizer, 0, wx.EXPAND) 131 | self.frame_sizer.Add(self.secondrow_static_box_sizer, 0, wx.EXPAND) 132 | self.frame_sizer.Add(self.thirdrow_sizer, 1, wx.EXPAND) 133 | 134 | # Layout sizers 135 | panel.SetSizerAndFit(self.frame_sizer) 136 | 137 | self.frame_sizer.Fit(self) 138 | 139 | # Bindings 140 | self.Bind(wx.EVT_MENU, self.OnAbout, menuAbout) 141 | self.Bind(wx.EVT_MENU, self.OnHelp, menuHelp) 142 | self.Bind(wx.EVT_MENU, self.OnOpen, menuOpen) 143 | self.Bind(wx.EVT_MENU, self.OnBackups, menuBackups) 144 | self.Bind(wx.EVT_MENU, self.OnExit, menuExit) 145 | self.Bind(wx.EVT_BUTTON, self.OnScan, self.scan_button) 146 | self.Bind(wx.EVT_BUTTON, self.OnOpen, self.open_button) 147 | self.Bind(wx.EVT_BUTTON, self.OnDeleteChunks, self.delete_all_chunks_button) 148 | self.Bind(wx.EVT_BUTTON, self.OnReplaceChunks, self.replace_all_chunks_button) 149 | self.Bind(wx.EVT_BUTTON, self.OnDeleteRegions, self.delete_all_regions_button) 150 | self.Bind(wx.EVT_BUTTON, self.OnReplaceRegions, self.replace_all_regions_button) 151 | 152 | self.Show(True) 153 | 154 | def OnExit(self, e): 155 | self.Close(True) 156 | 157 | def OnBackups(self, e): 158 | self.backups.Show(True) 159 | 160 | def OnAbout(self, e): 161 | self.about.Show(True) 162 | 163 | def OnHelp(self, e): 164 | self.help.Show(True) 165 | 166 | def OnOpen(self, e): 167 | """ Called when the open world button is pressed. """ 168 | dlg = wx.DirDialog(self, "Choose a Minecraft world folder") 169 | # Set the last path used 170 | dlg.SetPath(self.last_path) 171 | if dlg.ShowModal() == wx.ID_OK: 172 | self.dirname = dlg.GetPath() 173 | # Check if it's a minecraft world 174 | w = World(self.dirname) 175 | if not w.isworld: 176 | error = wx.MessageDialog(self, "This directory doesn't look like a Minecraft world", "Error", wx.ICON_EXCLAMATION) 177 | error.ShowModal() 178 | error.Destroy() 179 | else: 180 | # Insert it in the ListBox 181 | self.world = w 182 | self.update_world_status(self.world) 183 | 184 | # Properly recover the last path used 185 | self.last_path = split(dlg.GetPath())[0] 186 | dlg.Destroy() 187 | 188 | # Rest the results textctrl 189 | self.results_text.SetValue("") 190 | 191 | 192 | def OnScan(self, e): 193 | """ Called when the scan button is pressed. """ 194 | processes = int(self.proc_text.GetValue()) 195 | entity_limit = int(self.el_text.GetValue()) 196 | delete_entities = False 197 | 198 | ps = AsyncDataScanner(self.world.players, processes) 199 | ops = AsyncDataScanner(self.world.old_players, processes) 200 | ds = AsyncDataScanner(self.world.data_files, processes) 201 | ws = AsyncWorldRegionScanner(self.world, processes, entity_limit, 202 | delete_entities) 203 | 204 | things_to_scan = [ws, ops, ps, ds] 205 | dialog_texts = ["Scanning region files", 206 | "Scanning old format player files", 207 | "Scanning players", 208 | "Scanning data files"] 209 | try: 210 | for scanner, dialog_title in zip(things_to_scan, dialog_texts): 211 | progressdlg = wx.ProgressDialog( 212 | dialog_title, 213 | "Last scanned:\n starting...", 214 | len(scanner), self, 215 | style=wx.PD_ELAPSED_TIME | wx.PD_ESTIMATED_TIME | 216 | wx.PD_REMAINING_TIME | wx.PD_CAN_ABORT | 217 | wx.PD_AUTO_HIDE | wx.PD_SMOOTH) 218 | scanner.scan() 219 | counter = 0 220 | # NOTE TO SELF: ShowModal behaves different in windows and Linux! 221 | # Use it with care. 222 | progressdlg.Show() 223 | while not scanner.finished: 224 | sleep(0.001) 225 | result = scanner.get_last_result() 226 | 227 | if result: 228 | counter += 1 229 | not_cancelled, not_skipped = progressdlg.Update(counter, 230 | "Last scanned:\n" + scanner.str_last_scanned) 231 | if not not_cancelled: 232 | # User pressed cancel 233 | scanner.terminate() 234 | break 235 | progressdlg.Destroy() 236 | if not not_cancelled: 237 | break 238 | else: 239 | # The scan finished successfully 240 | self.world.scanned = True 241 | self.results_text.SetValue(self.world.generate_report(True)) 242 | self.update_delete_buttons_status(True) 243 | self.update_replace_buttons_status(True) 244 | except ChildProcessException as e: 245 | # Will be handled in starter.py by _excepthook() 246 | scanner.terminate() 247 | progressdlg.Destroy() 248 | raise e 249 | #=================================================================== 250 | # error_log_path = e.save_error_log() 251 | # filename = e.scanned_file.filename 252 | # scanner.terminate() 253 | # progressdlg.Destroy() 254 | # error = wx.MessageDialog(self, 255 | # ("Something went really wrong scanning {0}\n\n" 256 | # "This is probably an error in the code. Please, " 257 | # "if you have the time report it. " 258 | # "I have saved all the error information in:\n\n" 259 | # "{1}").format(filename, error_log_path), 260 | # "Error", 261 | # wx.ICON_ERROR) 262 | # error.ShowModal() 263 | #=================================================================== 264 | 265 | def OnDeleteChunks(self, e): 266 | progressdlg = wx.ProgressDialog("Removing chunks", "This may take a while", 267 | self.world.count_regions(), self, 268 | style=wx.PD_ELAPSED_TIME | 269 | wx.PD_ESTIMATED_TIME | 270 | wx.PD_REMAINING_TIME | 271 | wx.PD_CAN_SKIP | 272 | wx.PD_CAN_ABORT | 273 | wx.PD_AUTO_HIDE | 274 | wx.PD_SMOOTH 275 | ) 276 | progressdlg = progressdlg 277 | progressdlg.Pulse() 278 | remove_chunks = self.world.remove_problematic_chunks 279 | for problem in world.CHUNK_PROBLEMS: 280 | progressdlg.Pulse("Removing chunks with problem: {}".format(world.CHUNK_STATUS_TEXT[problem])) 281 | remove_chunks(problem) 282 | progressdlg.Destroy() 283 | progressdlg.Destroy() 284 | 285 | self.results_text.SetValue("Scan again the world for results.") 286 | self.update_delete_buttons_status(False) 287 | self.update_delete_buttons_status(False) 288 | 289 | def OnDeleteRegions(self, e): 290 | progressdlg = wx.ProgressDialog("Removing regions", "This may take a while...", 291 | self.world.count_regions(), self, 292 | style=wx.PD_ELAPSED_TIME | 293 | wx.PD_ESTIMATED_TIME | 294 | wx.PD_REMAINING_TIME | 295 | wx.PD_AUTO_HIDE | 296 | wx.PD_SMOOTH 297 | ) 298 | progressdlg = progressdlg 299 | progressdlg.Pulse() 300 | remove_regions = self.world.remove_problematic_regions 301 | for problem in world.REGION_PROBLEMS: 302 | progressdlg.Pulse("Removing regions with problem: {}".format(world.REGION_STATUS_TEXT[problem])) 303 | remove_regions(problem) 304 | progressdlg.Destroy() 305 | 306 | self.results_text.SetValue("Scan again the world for results.") 307 | self.update_delete_buttons_status(False) 308 | self.update_replace_buttons_status(False) 309 | 310 | def OnReplaceChunks(self, e): 311 | # Get options 312 | entity_limit = int(self.el_text.GetValue()) 313 | delete_entities = False 314 | 315 | progressdlg = wx.ProgressDialog("Removing chunks", "Removing...", 316 | self.world.count_regions(), self, 317 | style=wx.PD_ELAPSED_TIME | 318 | wx.PD_ESTIMATED_TIME | 319 | wx.PD_REMAINING_TIME | 320 | wx.PD_AUTO_HIDE | 321 | wx.PD_SMOOTH 322 | ) 323 | progressdlg = progressdlg 324 | backups = self.backups.world_list 325 | progressdlg.Pulse() 326 | replace_chunks = self.world.replace_problematic_chunks 327 | for problem in world.CHUNK_PROBLEMS: 328 | progressdlg.Pulse("Replacing chunks with problem: {}".format(world.CHUNK_STATUS_TEXT[problem])) 329 | replace_chunks(backups, problem, entity_limit, delete_entities) 330 | progressdlg.Destroy() 331 | 332 | self.results_text.SetValue("Scan again the world for results.") 333 | self.update_delete_buttons_status(False) 334 | self.update_replace_buttons_status(False) 335 | 336 | def OnReplaceRegions(self, e): 337 | # Get options 338 | entity_limit = int(self.el_text.GetValue()) 339 | delete_entities = False 340 | progressdlg = wx.ProgressDialog("Removing regions", "Removing...", 341 | self.world.count_regions(), self, 342 | style = wx.PD_ELAPSED_TIME | 343 | wx.PD_ESTIMATED_TIME | 344 | wx.PD_REMAINING_TIME | 345 | #~ wx.PD_CAN_SKIP | 346 | #~ wx.PD_CAN_ABORT | 347 | wx.PD_AUTO_HIDE | 348 | wx.PD_SMOOTH 349 | ) 350 | progressdlg = progressdlg 351 | backups = self.backups.world_list 352 | progressdlg.Pulse() 353 | replace_regions = self.world.replace_problematic_regions 354 | for problem in world.REGION_PROBLEMS: 355 | progressdlg.Pulse("Replacing regions with problem: {}".format(world.REGION_STATUS_TEXT[problem])) 356 | replace_regions(backups, problem, entity_limit, delete_entities) 357 | progressdlg.Destroy() 358 | 359 | self.results_text.SetValue("Scan again the world for results.") 360 | self.update_delete_buttons_status(False) 361 | self.update_replace_buttons_status(False) 362 | 363 | def update_delete_buttons_status(self, status): 364 | 365 | if status: 366 | self.delete_all_chunks_button.Enable() 367 | self.delete_all_regions_button.Enable() 368 | else: 369 | self.delete_all_chunks_button.Disable() 370 | self.delete_all_regions_button.Disable() 371 | 372 | def update_replace_buttons_status(self, status): 373 | 374 | if status: 375 | self.replace_all_chunks_button.Enable() 376 | self.replace_all_regions_button.Enable() 377 | else: 378 | self.replace_all_chunks_button.Disable() 379 | self.replace_all_regions_button.Disable() 380 | 381 | def update_world_status(self, world): 382 | self.status_text.SetLabel(world.path) 383 | self.scan_button.Enable() 384 | -------------------------------------------------------------------------------- /gui/starter.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import wx 5 | import sys 6 | import traceback 7 | from io import StringIO 8 | 9 | from .main import MainWindow 10 | from .backups import BackupsWindow 11 | from .about import AboutWindow 12 | from .help import HelpWindow 13 | 14 | from regionfixer_core.scan import ChildProcessException 15 | from regionfixer_core.bug_reporter import BugReporter 16 | from regionfixer_core.util import get_str_from_traceback 17 | 18 | ERROR_MSG = "\n\nOps! Something went really wrong and regionfixer crashed.\n\nI can try to send an automatic bug rerpot if you wish.\n" 19 | QUESTION_TEXT = ('Do you want to send an anonymous bug report to the region fixer ftp?\n' 20 | '(Answering no will print the bug report)') 21 | 22 | # Thanks to: 23 | # http://wxpython-users.1045709.n5.nabble.com/Exception-handling-strategies-td2369185.html 24 | # For a way to handle exceptions 25 | class MyApp(wx.App): 26 | def OnInit(self): 27 | sys.excepthook = self._excepthook 28 | return True 29 | 30 | def _excepthook(self, etype, value, tb): 31 | if isinstance(etype, ChildProcessException): 32 | s = "Using GUI:\n\n" + value.printable_traceback 33 | else: 34 | s = "Using GUI:\n\n" + get_str_from_traceback(etype, value, tb) 35 | # bug - display a dialog with the entire exception and traceback printed out 36 | traceback.print_tb(tb) 37 | dlg = wx.MessageDialog(self.main_window, 38 | ERROR_MSG + "\n" + QUESTION_TEXT, 39 | style=wx.ICON_ERROR | wx.YES_NO) 40 | # Get a string with the traceback and send it 41 | 42 | answer = dlg.ShowModal() 43 | if answer == wx.ID_YES: 44 | print("Sending bug report!") 45 | bugsender = BugReporter(error_str=s) 46 | success = bugsender.send() 47 | # Dialog with success or not of the ftp uploading 48 | if success: 49 | msg = "The bug report was successfully uploaded." 50 | style = 0 51 | else: 52 | msg = "Couldn't upload the bug report!\n\nPlease, try again later." 53 | style = wx.ICON_ERROR 54 | dlg = wx.MessageDialog(self.main_window, msg, style=style) 55 | dlg.ShowModal() 56 | else: 57 | dlg = wx.MessageDialog(self.main_window, "Error msg:\n\n" + s, 58 | style=wx.ICON_ERROR) 59 | dlg.ShowModal() 60 | 61 | 62 | class Starter(object): 63 | def __init__(self): 64 | """ Create the windows and set some variables. """ 65 | 66 | self.app = MyApp(False) 67 | 68 | self.frame = MainWindow(None, "Region-Fixer-GUI") 69 | # NOTE: It's very important that the MainWindow is parent of all others windows 70 | self.backups = BackupsWindow(self.frame, "Backups") 71 | self.about = AboutWindow(self.frame, "About") 72 | self.frame.backups = self.backups 73 | self.frame.about = self.about 74 | self.frame.help = HelpWindow(self.frame, "Help") 75 | # self.frame.error = ErrorWindow(self.frame, "Error") 76 | 77 | self.app.main_window = self.frame 78 | 79 | def run(self): 80 | """ Run the app main loop. """ 81 | 82 | self.app.MainLoop() 83 | -------------------------------------------------------------------------------- /gui/version.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Created on 24/06/2014 3 | 4 | @author: Alejandro 5 | ''' 6 | 7 | version_string = "0.0.1" 8 | version_numbers = version_string.split(".") 9 | -------------------------------------------------------------------------------- /icon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Fenixin/Minecraft-Region-Fixer/d890085536649fdc6ee6e3053e01f89bf16ea4e0/icon.ico -------------------------------------------------------------------------------- /mutf8/LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2012-2015 Tyler Kennedy . All rights reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a 4 | copy of this software and associated documentation files (the "Software"), 5 | to deal in the Software without restriction, including without limitation 6 | the rights to use, copy, modify, merge, publish, distribute, sublicense, 7 | and/or sell copies of the Software, and to permit persons to whom the 8 | Software is furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 18 | FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 19 | DEALINGS IN THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /mutf8/README.md: -------------------------------------------------------------------------------- 1 | ![Tests](https://github.com/TkTech/mutf8/workflows/Tests/badge.svg?branch=master) 2 | 3 | # mutf-8 4 | 5 | This package contains simple pure-python as well as C encoders and decoders for 6 | the MUTF-8 character encoding. In most cases, you can also parse the even-rarer 7 | CESU-8. 8 | 9 | These days, you'll most likely encounter MUTF-8 when working on files or 10 | protocols related to the JVM. Strings in a Java `.class` file are encoded using 11 | MUTF-8, strings passed by the JNI, as well as strings exported by the object 12 | serializer. 13 | 14 | This library was extracted from [Lawu][], a Python library for working with JVM 15 | class files. 16 | 17 | ## 🎉 Installation 18 | 19 | Install the package from PyPi: 20 | 21 | ``` 22 | pip install mutf8 23 | ``` 24 | 25 | Binary wheels are available for the following: 26 | 27 | | | py3.6 | py3.7 | py3.8 | py3.9 | 28 | | ---------------- | ----- | ----- | ----- | ----- | 29 | | OS X (x86_64) | y | y | y | y | 30 | | Windows (x86_64) | y | y | y | y | 31 | | Linux (x86_64) | y | y | y | y | 32 | 33 | If binary wheels are not available, it will attempt to build the C extension 34 | from source with any C99 compiler. If it could not build, it will fall back 35 | to a pure-python version. 36 | 37 | ## Usage 38 | 39 | Encoding and decoding is simple: 40 | 41 | ```python 42 | from mutf8 import encode_modified_utf8, decode_modified_utf8 43 | 44 | unicode = decode_modified_utf8(byte_like_object) 45 | bytes = encode_modified_utf8(unicode) 46 | ``` 47 | 48 | This module *does not* register itself globally as a codec, since importing 49 | should be side-effect-free. 50 | 51 | ## 📈 Benchmarks 52 | 53 | The C extension is significantly faster - often 20x to 40x faster. 54 | 55 | 56 | 57 | ### MUTF-8 Decoding 58 | | Name | Min (μs) | Max (μs) | StdDev | Ops | 59 | |------------------------------|------------|------------|----------|---------------| 60 | | cmutf8-decode_modified_utf8 | 0.00009 | 0.00080 | 0.00000 | 9957678.56358 | 61 | | pymutf8-decode_modified_utf8 | 0.00190 | 0.06040 | 0.00000 | 450455.96019 | 62 | 63 | ### MUTF-8 Encoding 64 | | Name | Min (μs) | Max (μs) | StdDev | Ops | 65 | |------------------------------|------------|------------|----------|----------------| 66 | | cmutf8-encode_modified_utf8 | 0.00008 | 0.00151 | 0.00000 | 11897361.05101 | 67 | | pymutf8-encode_modified_utf8 | 0.00180 | 0.16650 | 0.00000 | 474390.98091 | 68 | 69 | 70 | ## C Extension 71 | 72 | The C extension is optional. If a binary package is not available, or a C 73 | compiler is not present, the pure-python version will be used instead. If you 74 | want to ensure you're using the C version, import it directly: 75 | 76 | ```python 77 | from mutf8.cmutf8 import decode_modified_utf8 78 | 79 | decode_modified_utf(b'\xED\xA1\x80\xED\xB0\x80') 80 | ``` 81 | 82 | [Lawu]: https://github.com/tktech/lawu 83 | -------------------------------------------------------------------------------- /mutf8/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Utility methods for handling oddities in character encoding encountered 3 | when parsing and writing JVM ClassFiles or object serialization archives. 4 | 5 | MUTF-8 is the same as CESU-8, but with different encoding for 0x00 bytes. 6 | 7 | .. note:: 8 | 9 | http://bugs.python.org/issue2857 was an attempt in 2008 to get support 10 | for MUTF-8/CESU-8 into the python core. 11 | """ 12 | 13 | 14 | try: 15 | from mutf8.cmutf8 import decode_modified_utf8, encode_modified_utf8 16 | except ImportError: 17 | from mutf8.mutf8 import decode_modified_utf8, encode_modified_utf8 18 | 19 | 20 | # Shut up linters. 21 | ALL_IMPORTS = [decode_modified_utf8, encode_modified_utf8] 22 | -------------------------------------------------------------------------------- /mutf8/cmutf8.c: -------------------------------------------------------------------------------- 1 | #define PY_SSIZE_T_CLEAN 2 | #include 3 | #include 4 | 5 | PyDoc_STRVAR(decode_doc, 6 | "Decodes a bytestring containing MUTF-8 as defined in section\n" 7 | "4.4.7 of the JVM specification.\n\n" 8 | ":param s: A byte/buffer-like to be converted.\n" 9 | ":returns: A unicode representation of the original string."); 10 | static PyObject * 11 | decode_modified_utf8(PyObject *self, PyObject *args) 12 | { 13 | #define return_err(_msg) \ 14 | do { \ 15 | PyObject *exc = PyObject_CallFunction(PyExc_UnicodeDecodeError, \ 16 | "sy#nns", "mutf-8", view.buf, \ 17 | view.len, ix, ix + 1, _msg); \ 18 | if (exc != NULL) { \ 19 | PyCodec_StrictErrors(exc); \ 20 | Py_DECREF(exc); \ 21 | } \ 22 | PyMem_Free(cp_out); \ 23 | PyBuffer_Release(&view); \ 24 | return NULL; \ 25 | } while (0) 26 | 27 | Py_buffer view; 28 | 29 | if (!PyArg_ParseTuple(args, "y*", &view)) { 30 | return NULL; 31 | } 32 | 33 | // MUTF-8 input. 34 | uint8_t *buf = (uint8_t *)view.buf; 35 | // Array of temporary UCS-4 codepoints. 36 | // There's no point using PyUnicode_new and _WriteChar, because 37 | // it requires us to have iterated the string to get the maximum unicode 38 | // codepoint and count anyways. 39 | Py_UCS4 *cp_out = PyMem_Calloc(view.len, sizeof(Py_UCS4)); 40 | if (!cp_out) { 41 | return PyErr_NoMemory(); 42 | } 43 | 44 | // # of codepoints we found & current index into cp_out. 45 | Py_ssize_t cp_count = 0; 46 | 47 | for (Py_ssize_t ix = 0; ix < view.len; ix++) { 48 | Py_UCS4 x = buf[ix]; 49 | 50 | if (x == 0) { 51 | return_err("Embedded NULL byte in input."); 52 | } 53 | else if (x < 0x80) { 54 | // ASCII/one-byte codepoint. 55 | x &= 0x7F; 56 | } 57 | else if ((x & 0xE0) == 0xC0) { 58 | // Two-byte codepoint. 59 | if (ix + 1 >= view.len) { 60 | return_err( 61 | "2-byte codepoint started, but input too short" 62 | " to finish."); 63 | } 64 | x = ((x & 0x1F) << 0x06 | (buf[ix + 1] & 0x3F)); 65 | ix++; 66 | } 67 | else if ((x & 0xF0) == 0xE0) { 68 | // Three-byte codepoint. 69 | if (ix + 2 >= view.len) { 70 | return_err( 71 | "3-byte or 6-byte codepoint started, but input too short" 72 | " to finish."); 73 | } 74 | uint8_t b2 = buf[ix + 1]; 75 | uint8_t b3 = buf[ix + 2]; 76 | 77 | if (x == 0xED && (b2 & 0xF0) == 0xA0) { 78 | if (ix + 5 >= view.len) { 79 | return_err( 80 | "6-byte codepoint started, but input too short" 81 | " to finish."); 82 | } 83 | 84 | // Possible six-byte codepoint. 85 | uint8_t b4 = buf[ix + 3]; 86 | uint8_t b5 = buf[ix + 4]; 87 | uint8_t b6 = buf[ix + 5]; 88 | 89 | if (b4 == 0xED && (b5 & 0xF0) == 0xB0) { 90 | // Definite six-byte codepoint. 91 | x = ( 92 | 0x10000 | 93 | (b2 & 0x0F) << 0x10 | 94 | (b3 & 0x3F) << 0x0A | 95 | (b5 & 0x0F) << 0x06 | 96 | (b6 & 0x3F) 97 | ); 98 | ix += 5; 99 | cp_out[cp_count++] = x; 100 | continue; 101 | } 102 | } 103 | 104 | x = ( 105 | (x & 0x0F) << 0x0C | 106 | (b2 & 0x3F) << 0x06 | 107 | (b3 & 0x3F) 108 | ); 109 | 110 | ix += 2; 111 | } 112 | cp_out[cp_count++] = x; 113 | } 114 | 115 | PyObject *out = 116 | PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, cp_out, cp_count); 117 | 118 | PyMem_Free(cp_out); 119 | PyBuffer_Release(&view); 120 | return out; 121 | #undef return_err 122 | } 123 | 124 | inline Py_ssize_t _encoded_size(void *data, Py_ssize_t length, int kind) { 125 | Py_ssize_t byte_count = 0; 126 | 127 | for (Py_ssize_t i = 0; i < length; i++) { 128 | Py_UCS4 cp = PyUnicode_READ(kind, data, i); 129 | if (cp == 0x00) { 130 | // NULLs will get encoded as C0 80. 131 | byte_count += 2; 132 | } else if (cp <= 0x7F) { 133 | byte_count++; 134 | } else if (cp <= 0x7FF) { 135 | byte_count += 2; 136 | } else if (cp <= 0xFFFF) { 137 | byte_count += 3; 138 | } else { 139 | byte_count += 6; 140 | } 141 | } 142 | 143 | return byte_count; 144 | } 145 | 146 | PyDoc_STRVAR(encoded_size_doc, 147 | "Returns the number of bytes required to store the given\n" 148 | "unicode string when encoded as MUTF-8.\n\n" 149 | ":param u: Unicode string to be converted.\n" 150 | ":returns: The number of bytes required."); 151 | static PyObject * 152 | encoded_size(PyObject *self, PyObject *args) 153 | { 154 | PyObject *src = NULL; 155 | 156 | if (!PyArg_ParseTuple(args, "U", &src)) { 157 | return NULL; 158 | } 159 | 160 | return PyLong_FromSsize_t( 161 | _encoded_size( 162 | PyUnicode_DATA(src), 163 | PyUnicode_GET_LENGTH(src), 164 | PyUnicode_KIND(src) 165 | ) 166 | ); 167 | } 168 | 169 | PyDoc_STRVAR(encode_doc, 170 | "Encodes a unicode string as MUTF-8 as defined in section\n" 171 | "4.4.7 of the JVM specification.\n\n" 172 | ":param u: Unicode string to be converted.\n" 173 | ":returns: The encoded string as a `bytes` object."); 174 | static PyObject * 175 | encode_modified_utf8(PyObject *self, PyObject *args) 176 | { 177 | PyObject *src = NULL; 178 | 179 | if (!PyArg_ParseTuple(args, "U", &src)) { 180 | return NULL; 181 | } 182 | 183 | void *data = PyUnicode_DATA(src); 184 | Py_ssize_t length = PyUnicode_GET_LENGTH(src); 185 | int kind = PyUnicode_KIND(src); 186 | char *byte_out = PyMem_Calloc(_encoded_size(data, length, kind), 1); 187 | 188 | if (!byte_out) { 189 | return PyErr_NoMemory(); 190 | } 191 | 192 | Py_ssize_t byte_count = 0; 193 | 194 | for (Py_ssize_t i = 0; i < length; i++) { 195 | Py_UCS4 cp = PyUnicode_READ(kind, data, i); 196 | if (cp == 0x00) { 197 | // NULL byte encoding shortcircuit. 198 | byte_out[byte_count++] = 0xC0; 199 | byte_out[byte_count++] = 0x80; 200 | } 201 | else if (cp <= 0x7F) { 202 | // ASCII 203 | byte_out[byte_count++] = cp; 204 | } 205 | else if (cp <= 0x7FF) { 206 | // Two-byte codepoint. 207 | byte_out[byte_count++] = (0xC0 | (0x1F & (cp >> 0x06))); 208 | byte_out[byte_count++] = (0x80 | (0x3F & cp)); 209 | } 210 | else if (cp <= 0xFFFF) { 211 | // Three-byte codepoint 212 | byte_out[byte_count++] = (0xE0 | (0x0F & (cp >> 0x0C))); 213 | byte_out[byte_count++] = (0x80 | (0x3F & (cp >> 0x06))); 214 | byte_out[byte_count++] = (0x80 | (0x3F & cp)); 215 | } 216 | else { 217 | // "Two-times-three" byte codepoint. 218 | byte_out[byte_count++] = 0xED; 219 | byte_out[byte_count++] = 0xA0 | ((cp >> 0x10) & 0x0F); 220 | byte_out[byte_count++] = 0x80 | ((cp >> 0x0A) & 0x3F); 221 | byte_out[byte_count++] = 0xED; 222 | byte_out[byte_count++] = 0xB0 | ((cp >> 0x06) & 0x0F); 223 | byte_out[byte_count++] = 0x80 | (cp & 0x3F); 224 | } 225 | } 226 | 227 | PyObject *out = PyBytes_FromStringAndSize(byte_out, byte_count); 228 | PyMem_Free(byte_out); 229 | return out; 230 | } 231 | 232 | static PyMethodDef module_methods[] = { 233 | {"decode_modified_utf8", decode_modified_utf8, METH_VARARGS, decode_doc}, 234 | {"encode_modified_utf8", encode_modified_utf8, METH_VARARGS, encode_doc}, 235 | {"encoded_size", encoded_size, METH_VARARGS, encoded_size_doc}, 236 | {NULL, NULL, 0, NULL}}; 237 | 238 | static struct PyModuleDef cmutf8_module = { 239 | PyModuleDef_HEAD_INIT, 240 | "mutf8.cmutf8", 241 | PyDoc_STR("Encoders and decoders for the MUTF-8 encoding."), 242 | -1, 243 | module_methods, 244 | }; 245 | 246 | PyMODINIT_FUNC 247 | PyInit_cmutf8(void) 248 | { 249 | PyObject *m; 250 | 251 | m = PyModule_Create(&cmutf8_module); 252 | if (m == NULL) 253 | return NULL; 254 | 255 | return m; 256 | } 257 | -------------------------------------------------------------------------------- /mutf8/mutf8.py: -------------------------------------------------------------------------------- 1 | def decode_modified_utf8(s: bytes) -> str: 2 | """ 3 | Decodes a bytestring containing modified UTF-8 as defined in section 4 | 4.4.7 of the JVM specification. 5 | 6 | :param s: bytestring to be converted. 7 | :returns: A unicode representation of the original string. 8 | """ 9 | s_out = [] 10 | s_len = len(s) 11 | s_ix = 0 12 | 13 | while s_ix < s_len: 14 | b1 = s[s_ix] 15 | s_ix += 1 16 | 17 | if b1 == 0: 18 | raise UnicodeDecodeError( 19 | 'mutf-8', 20 | s, 21 | s_ix - 1, 22 | s_ix, 23 | 'Embedded NULL byte in input.' 24 | ) 25 | if b1 < 0x80: 26 | # ASCII/one-byte codepoint. 27 | s_out.append(chr(b1)) 28 | elif (b1 & 0xE0) == 0xC0: 29 | # Two-byte codepoint. 30 | if s_ix >= s_len: 31 | raise UnicodeDecodeError( 32 | 'mutf-8', 33 | s, 34 | s_ix - 1, 35 | s_ix, 36 | '2-byte codepoint started, but input too short to' 37 | ' finish.' 38 | ) 39 | 40 | s_out.append( 41 | chr( 42 | (b1 & 0x1F) << 0x06 | 43 | (s[s_ix] & 0x3F) 44 | ) 45 | ) 46 | s_ix += 1 47 | elif (b1 & 0xF0) == 0xE0: 48 | # Three-byte codepoint. 49 | if s_ix + 1 >= s_len: 50 | raise UnicodeDecodeError( 51 | 'mutf-8', 52 | s, 53 | s_ix - 1, 54 | s_ix, 55 | '3-byte or 6-byte codepoint started, but input too' 56 | ' short to finish.' 57 | ) 58 | 59 | b2 = s[s_ix] 60 | b3 = s[s_ix + 1] 61 | 62 | if b1 == 0xED and (b2 & 0xF0) == 0xA0: 63 | # Possible six-byte codepoint. 64 | if s_ix + 4 >= s_len: 65 | raise UnicodeDecodeError( 66 | 'mutf-8', 67 | s, 68 | s_ix - 1, 69 | s_ix, 70 | '3-byte or 6-byte codepoint started, but input too' 71 | ' short to finish.' 72 | ) 73 | 74 | b4 = s[s_ix + 2] 75 | b5 = s[s_ix + 3] 76 | b6 = s[s_ix + 4] 77 | 78 | if b4 == 0xED and (b5 & 0xF0) == 0xB0: 79 | # Definite six-byte codepoint. 80 | s_out.append( 81 | chr( 82 | 0x10000 | 83 | (b2 & 0x0F) << 0x10 | 84 | (b3 & 0x3F) << 0x0A | 85 | (b5 & 0x0F) << 0x06 | 86 | (b6 & 0x3F) 87 | ) 88 | ) 89 | s_ix += 5 90 | continue 91 | 92 | s_out.append( 93 | chr( 94 | (b1 & 0x0F) << 0x0C | 95 | (b2 & 0x3F) << 0x06 | 96 | (b3 & 0x3F) 97 | ) 98 | ) 99 | s_ix += 2 100 | else: 101 | raise RuntimeError 102 | 103 | return u''.join(s_out) 104 | 105 | 106 | def encode_modified_utf8(u: str) -> bytes: 107 | """ 108 | Encodes a unicode string as modified UTF-8 as defined in section 4.4.7 109 | of the JVM specification. 110 | 111 | :param u: unicode string to be converted. 112 | :returns: A decoded bytearray. 113 | """ 114 | final_string = bytearray() 115 | 116 | for c in (ord(char) for char in u): 117 | if c == 0x00: 118 | # NULL byte encoding shortcircuit. 119 | final_string.extend([0xC0, 0x80]) 120 | elif c <= 0x7F: 121 | # ASCII 122 | final_string.append(c) 123 | elif c <= 0x7FF: 124 | # Two-byte codepoint. 125 | final_string.extend([ 126 | (0xC0 | (0x1F & (c >> 0x06))), 127 | (0x80 | (0x3F & c)) 128 | ]) 129 | elif c <= 0xFFFF: 130 | # Three-byte codepoint. 131 | final_string.extend([ 132 | (0xE0 | (0x0F & (c >> 0x0C))), 133 | (0x80 | (0x3F & (c >> 0x06))), 134 | (0x80 | (0x3F & c)) 135 | ]) 136 | else: 137 | # Six-byte codepoint. 138 | final_string.extend([ 139 | 0xED, 140 | 0xA0 | ((c >> 0x10) & 0x0F), 141 | 0x80 | ((c >> 0x0A) & 0x3f), 142 | 0xED, 143 | 0xb0 | ((c >> 0x06) & 0x0f), 144 | 0x80 | (c & 0x3f) 145 | ]) 146 | 147 | return bytes(final_string) 148 | -------------------------------------------------------------------------------- /nbt/CONTRIBUTORS.txt: -------------------------------------------------------------------------------- 1 | d0sboots (David Walker) 2 | dtrauma (Thomas Roesner) 3 | Fenixin (Alejandro Aguilera) 4 | fwaggle (Jamie Fraser) 5 | jlsajfj (Joseph) 6 | k1988 (Terry Zhao) 7 | kamyu2 8 | MacFreek (Freek Dijkstra) 9 | MFLD.fr 10 | MidnightLightning (Brooks Boyd) 11 | MostAwesomeDude (Corbin Simpson) 12 | psolyca (Damien) 13 | s-leroux (Sylvain Leroux) 14 | SBliven (Spencer Bliven) 15 | steffen-kiess (Steffen Kieß) 16 | Stumpylog (Trenton Holmes) 17 | suresttexas00 (Surest Texas) 18 | tWoolie (Thomas Woolford) 19 | underscoren (Marius Steffens) 20 | Xgkkp 21 | Zachy (Zachary Howard) 22 | -------------------------------------------------------------------------------- /nbt/LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2010-2013 Thomas Woolford and contributors 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /nbt/MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.txt 2 | include LICENSE.txt 3 | include CONTRIBUTORS.txt 4 | include tests/*.py 5 | include tests/bigtest.nbt 6 | include tests/regiontest.mca 7 | include examples/*.py 8 | include doc/*.rst -------------------------------------------------------------------------------- /nbt/README.md: -------------------------------------------------------------------------------- 1 | This is a Named Binary Tag parser based upon the specification by Markus Persson. 2 | 3 | From The spec: 4 | "NBT (Named Binary Tag) is a tag based binary format designed to carry large 5 | amounts of binary data with smaller amounts of additional data. 6 | An NBT file consists of a single GZIPped Named Tag of type TAG_Compound." 7 | 8 | read the full spec at http://www.minecraft.net/docs/NBT.txt 9 | 10 | [![Build Status](https://secure.travis-ci.org/twoolie/NBT.png?branch=master)](http://travis-ci.org/#!/twoolie/NBT) 11 | [![Test Coverage Status](https://coveralls.io/repos/twoolie/NBT/badge.svg)](https://coveralls.io/r/twoolie/NBT) 12 | 13 | Usage: 14 | 1) Reading files. 15 | 16 | The easiest way to read an nbt file is to instantiate an NBTFile object e.g. 17 | 18 | >>> import nbt 19 | >>> nbtfile = nbt.NBTFile("bigtest.nbt",'rb') 20 | >>> nbtfile.name 21 | u'Level' 22 | >>> nbtfile["nested compound test"].tag_info() 23 | TAG_Compound("nested compound test"): 2 Entries 24 | >>> for tag in nbtfile["nested compound test"]["ham"].tags: 25 | ... print(tag.tag_info()) 26 | ... 27 | TAG_String("name"): Hampus 28 | TAG_Float("value"): 0.75 29 | >>> [tag.value for tag in nbtfile["listTest (long)"].value] 30 | [11, 12, 13, 14, 15] 31 | 32 | Files can also be read from a fileobj (file-like object that contains a compressed 33 | stream) or a buffer (file-like object that contains an uncompressed stream of NBT 34 | Tags) which can be accomplished thusly: 35 | 36 | >>> import nbt 37 | >>> nbtfile = NBTFile(fileobj=previously_opened_file) 38 | # or.... 39 | >>> nbtfile = NBTFile(buffer=net_socket.makefile()) 40 | 41 | 2) Writing files. 42 | 43 | Writing files is easy too! if you have a NBTFile object, simply call it's 44 | write_file() method. If the NBTFile was instantiated with a filename, then 45 | write_file needs no extra arguments. It just works. If however you created a new 46 | file object from scratch (or even if you just want to save it somewhere else) 47 | call write_file('path\to\new\file.nbt') 48 | 49 | >>> import nbt 50 | >>> nbtfile = nbt.NBTFile("bigtest.nbt",'rb') 51 | >>> nbtfile["listTest (compound)"].tags[0]["name"].value = "Different name" 52 | >>> nbtfile.write_file("newnbtfile.nbt") 53 | 54 | It is also possible to write to a buffer or fileobj using the same keyword args. 55 | 56 | >>> nbtfile.write_file(fileobj = my_file) #compressed 57 | >>> nbtfile.write_file(buffer = sock.makefile()) #uncompressed 58 | 59 | 3) Creating files 60 | 61 | Creating files is trickier but ultimately should give you no issue, as long as 62 | you have read the NBT spec (hint.. it's very short). Also be sure to note that 63 | the NBTFile object is actually a TAG_Compound with some wrapper features, so 64 | you can use all the standard tag features 65 | 66 | >>> from nbt import * 67 | >>> nbtfile = NBTFile() 68 | 69 | first, don't forget to name the top level tag 70 | 71 | >>> nbtfile.name = "My Top Level Tag" 72 | >>> nbtfile.tags.append(TAG_Float(name="My Float Name", value=3.152987593947)) 73 | >>> mylist = TAG_List(name="TestList", type=TAG_Long) #type needs to be pre-declared! 74 | >>> mylist.tags.append(TAG_Long(100)) 75 | >>> mylist.tags.extend([TAG_Long(120),TAG_Long(320),TAG_Long(19)]) 76 | >>> nbtfile.tags.append(mylist) 77 | >>> print(nbtfile.pretty_tree()) 78 | TAG_Compound("My Top Level Tag"): 2 Entries 79 | { 80 | TAG_Float("My Float Name"): 3.15298759395 81 | TAG_List("TestList"): 4 entries of type TAG_Long 82 | { 83 | TAG_Long: 100 84 | TAG_Long: 120 85 | TAG_Long: 320 86 | TAG_Long: 19 87 | } 88 | } 89 | >>> nbtfile["TestList"].tags.sort(key = lambda tag: tag.value) 90 | >>> print(nbtfile.pretty_tree()) 91 | TAG_Compound("My Top Level Tag"): 2 Entries 92 | { 93 | TAG_Float("My FloatName"): 3.15298759395 94 | TAG_List("TestList"): 4 entries of type TAG_Long 95 | { 96 | TAG_Long: 19 97 | TAG_Long: 100 98 | TAG_Long: 120 99 | TAG_Long: 320 100 | } 101 | } 102 | >>> nbtfile.write_file("mynbt.dat") 103 | -------------------------------------------------------------------------------- /nbt/README.txt: -------------------------------------------------------------------------------- 1 | ========================== 2 | The NBT library for Python 3 | ========================== 4 | 5 | Forewords 6 | ========= 7 | 8 | This is mainly a `Named Binary Tag` parser & writer library. 9 | 10 | From the initial specification by Markus Persson:: 11 | 12 | NBT (Named Binary Tag) is a tag based binary format designed to carry large 13 | amounts of binary data with smaller amounts of additional data. 14 | An NBT file consists of a single GZIPped Named Tag of type TAG_Compound. 15 | 16 | Current specification is on the official [Minecraft Wiki](https://minecraft.gamepedia.com/NBT_format). 17 | 18 | This library is very suited to inspect & edit the Minecraft data files. Provided 19 | examples demonstrate how to: 20 | - get player and world statistics, 21 | - list mobs, chest contents, biomes, 22 | - draw a simple world map, 23 | - etc. 24 | 25 | .. image:: world.png 26 | 27 | *Note: Examples are just here to help using and testing the library. 28 | Developing Minecraft tools is out of the scope of this project.* 29 | 30 | 31 | Status 32 | ====== 33 | 34 | The library supports all the currently known tag types (including the arrays 35 | of 'Integer' and 'Long'), and the examples work with the McRegion, 36 | pre-"flattened" and "flattened" Anvil formats. 37 | 38 | Last update was tested on Minecraft version **1.13.2**. 39 | 40 | 41 | Dependencies 42 | ============ 43 | 44 | The library, the tests and the examples are only using the Python core library, 45 | except `curl` for downloading some test reference data and `PIL` (Python 46 | Imaging Library) for the `map` example. 47 | 48 | Supported Python releases: 2.7, 3.4 to 3.7 49 | 50 | 51 | Usage 52 | ===== 53 | 54 | Reading files 55 | ------------- 56 | 57 | The easiest way to read an nbt file is to instantiate an NBTFile object e.g.:: 58 | 59 | >>> from nbt import nbt 60 | >>> nbtfile = nbt.NBTFile("bigtest.nbt",'rb') 61 | >>> nbtfile.name 62 | u'Level' 63 | >>> nbtfile["nested compound test"].tag_info() 64 | TAG_Compound("nested compound test"): 2 Entries 65 | >>> for tag in nbtfile["nested compound test"]["ham"].tags: 66 | ... print(tag.tag_info()) 67 | ... 68 | TAG_String("name"): Hampus 69 | TAG_Float("value"): 0.75 70 | >>> [tag.value for tag in nbtfile["listTest (long)"].value] 71 | [11, 12, 13, 14, 15] 72 | 73 | Files can also be read from a fileobj (file-like object that contains a compressed 74 | stream) or a buffer (file-like object that contains an uncompressed stream of NBT 75 | Tags) which can be accomplished thusly:: 76 | 77 | >>> from nbt.nbt import * 78 | >>> nbtfile = NBTFile(fileobj=previously_opened_file) 79 | # or.... 80 | >>> nbtfile = NBTFile(buffer=net_socket.makefile()) 81 | 82 | 83 | Writing files 84 | ------------- 85 | 86 | Writing files is easy too! if you have a NBTFile object, simply call it's 87 | write_file() method. If the NBTFile was instantiated with a filename, then 88 | write_file needs no extra arguments. It just works. If however you created a new 89 | file object from scratch (or even if you just want to save it somewhere else) 90 | call write_file('path\to\new\file.nbt'):: 91 | 92 | >>> from nbt import nbt 93 | >>> nbtfile = nbt.NBTFile("bigtest.nbt",'rb') 94 | >>> nbtfile["listTest (compound)"].tags[0]["name"].value = "Different name" 95 | >>> nbtfile.write_file("newnbtfile.nbt") 96 | 97 | It is also possible to write to a buffer or fileobj using the same keyword args:: 98 | 99 | >>> nbtfile.write_file(fileobj = my_file) #compressed 100 | >>> nbtfile.write_file(buffer = sock.makefile()) #uncompressed 101 | 102 | 103 | Creating files 104 | -------------- 105 | 106 | Creating files is trickier but ultimately should give you no issue, as long as 107 | you have read the NBT spec (hint.. it's very short). Also be sure to note that 108 | the NBTFile object is actually a TAG_Compound with some wrapper features, so 109 | you can use all the standard tag features:: 110 | 111 | >>> from nbt.nbt import * 112 | >>> nbtfile = NBTFile() 113 | 114 | 115 | First, don't forget to name the top level tag:: 116 | 117 | >>> nbtfile.name = "My Top Level Tag" 118 | >>> nbtfile.tags.append(TAG_Float(name="My Float Name", value=3.152987593947)) 119 | >>> mylist = TAG_List(name="TestList", type=TAG_Long) #type needs to be pre-declared! 120 | >>> mylist.tags.append(TAG_Long(100)) 121 | >>> mylist.tags.extend([TAG_Long(120),TAG_Long(320),TAG_Long(19)]) 122 | >>> nbtfile.tags.append(mylist) 123 | >>> print(nbtfile.pretty_tree()) 124 | TAG_Compound("My Top Level Tag"): 2 Entries 125 | { 126 | TAG_Float("My Float Name"): 3.15298759395 127 | TAG_List("TestList"): 4 entries of type TAG_Long 128 | { 129 | TAG_Long: 100 130 | TAG_Long: 120 131 | TAG_Long: 320 132 | TAG_Long: 19 133 | } 134 | } 135 | >>> nbtfile["TestList"].tags.sort(key = lambda tag: tag.value) 136 | >>> print(nbtfile.pretty_tree()) 137 | TAG_Compound("My Top Level Tag"): 2 Entries 138 | { 139 | TAG_Float("My FloatName"): 3.15298759395 140 | TAG_List("TestList"): 4 entries of type TAG_Long 141 | { 142 | TAG_Long: 19 143 | TAG_Long: 100 144 | TAG_Long: 120 145 | TAG_Long: 320 146 | } 147 | } 148 | >>> nbtfile.write_file("mynbt.dat") 149 | -------------------------------------------------------------------------------- /nbt/__init__.py: -------------------------------------------------------------------------------- 1 | __all__ = ["nbt", "world", "region", "chunk"] 2 | from . import * 3 | 4 | # Documentation only automatically includes functions specified in __all__. 5 | # If you add more functions, please manually include them in doc/index.rst. 6 | 7 | VERSION = (1, 5, 1) 8 | """NBT version as tuple. Note that the major and minor revision number are 9 | always present, but the patch identifier (the 3rd number) is only used in 1.4.""" 10 | 11 | def _get_version(): 12 | """Return the NBT version as string.""" 13 | return ".".join([str(v) for v in VERSION]) 14 | -------------------------------------------------------------------------------- /nbt/chunk.py: -------------------------------------------------------------------------------- 1 | """ 2 | Handles a single chunk of data (16x16x128 blocks) from a Minecraft save. 3 | 4 | For more information about the chunck format: 5 | https://minecraft.gamepedia.com/Chunk_format 6 | """ 7 | 8 | from io import BytesIO 9 | from struct import pack 10 | from math import ceil 11 | import array 12 | 13 | 14 | # Legacy numeric block identifiers 15 | # mapped to alpha identifiers in best effort 16 | # See https://minecraft.gamepedia.com/Java_Edition_data_values/Pre-flattening 17 | # TODO: move this map into a separate file 18 | 19 | block_ids = { 20 | 0: 'air', 21 | 1: 'stone', 22 | 2: 'grass_block', 23 | 3: 'dirt', 24 | 4: 'cobblestone', 25 | 5: 'oak_planks', 26 | 6: 'sapling', 27 | 7: 'bedrock', 28 | 8: 'flowing_water', 29 | 9: 'water', 30 | 10: 'flowing_lava', 31 | 11: 'lava', 32 | 12: 'sand', 33 | 13: 'gravel', 34 | 14: 'gold_ore', 35 | 15: 'iron_ore', 36 | 16: 'coal_ore', 37 | 17: 'oak_log', 38 | 18: 'oak_leaves', 39 | 19: 'sponge', 40 | 20: 'glass', 41 | 21: 'lapis_ore', 42 | 24: 'sandstone', 43 | 30: 'cobweb', 44 | 31: 'grass', 45 | 32: 'dead_bush', 46 | 35: 'white_wool', 47 | 37: 'dandelion', 48 | 38: 'poppy', 49 | 39: 'brown_mushroom', 50 | 40: 'red_mushroom', 51 | 43: 'stone_slab', 52 | 44: 'stone_slab', 53 | 47: 'bookshelf', 54 | 48: 'mossy_cobblestone', 55 | 49: 'obsidian', 56 | 50: 'torch', 57 | 51: 'fire', 58 | 52: 'spawner', 59 | 53: 'oak_stairs', 60 | 54: 'chest', 61 | 56: 'diamond_ore', 62 | 58: 'crafting_table', 63 | 59: 'wheat', 64 | 60: 'farmland', 65 | 61: 'furnace', 66 | 62: 'furnace', 67 | 63: 'sign', # will change to oak_sign in 1.14 68 | 64: 'oak_door', 69 | 65: 'ladder', 70 | 66: 'rail', 71 | 67: 'cobblestone_stairs', 72 | 72: 'oak_pressure_plate', 73 | 73: 'redstone_ore', 74 | 74: 'redstone_ore', 75 | 78: 'snow', 76 | 79: 'ice', 77 | 81: 'cactus', 78 | 82: 'clay', 79 | 83: 'sugar_cane', 80 | 85: 'oak_fence', 81 | 86: 'pumpkin', 82 | 91: 'lit_pumpkin', 83 | 101: 'iron_bars', 84 | 102: 'glass_pane', 85 | } 86 | 87 | 88 | def block_id_to_name(bid): 89 | try: 90 | name = block_ids[bid] 91 | except KeyError: 92 | name = 'unknown_%d' % (bid,) 93 | print("warning: unknown block id %i" % bid) 94 | print("hint: add that block to the 'block_ids' map") 95 | return name 96 | 97 | 98 | # Generic Chunk 99 | 100 | class Chunk(object): 101 | """Class for representing a single chunk.""" 102 | def __init__(self, nbt): 103 | self.chunk_data = nbt['Level'] 104 | self.coords = self.chunk_data['xPos'],self.chunk_data['zPos'] 105 | 106 | def get_coords(self): 107 | """Return the coordinates of this chunk.""" 108 | return (self.coords[0].value,self.coords[1].value) 109 | 110 | def __repr__(self): 111 | """Return a representation of this Chunk.""" 112 | return "Chunk("+str(self.coords[0])+","+str(self.coords[1])+")" 113 | 114 | 115 | # Chunk in Region old format 116 | 117 | class McRegionChunk(Chunk): 118 | 119 | def __init__(self, nbt): 120 | Chunk.__init__(self, nbt) 121 | self.blocks = BlockArray(self.chunk_data['Blocks'].value, self.chunk_data['Data'].value) 122 | 123 | def get_max_height(self): 124 | return 127 125 | 126 | def get_block(self, x, y, z): 127 | name = block_id_to_name(self.blocks.get_block(x, y, z)) 128 | return name 129 | 130 | def iter_block(self): 131 | for y in range(0, 128): 132 | for z in range(0, 16): 133 | for x in range(0, 16): 134 | yield self.get_block(x, y, z) 135 | 136 | 137 | # Section in Anvil new format 138 | 139 | class AnvilSection(object): 140 | 141 | def __init__(self, nbt, version): 142 | self.names = [] 143 | self.indexes = [] 144 | 145 | # Is the section flattened ? 146 | # See https://minecraft.gamepedia.com/1.13/Flattening 147 | 148 | if version == 0 or version == 1343: # 1343 = MC 1.12.2 149 | self._init_array(nbt) 150 | elif version >= 1631 and version <= 2230: # MC 1.13 to MC 1.15.2 151 | self._init_index_unpadded(nbt) 152 | elif version >= 2566 and version <= 2730: # MC 1.16.0 to MC 1.17.2 (latest tested version) 153 | self._init_index_padded(nbt) 154 | else: 155 | raise NotImplementedError() 156 | 157 | # Section contains 4096 blocks whatever data version 158 | 159 | assert len(self.indexes) == 4096 160 | 161 | 162 | # Decode legacy section 163 | # Contains an array of block numeric identifiers 164 | 165 | def _init_array(self, nbt): 166 | bids = [] 167 | for bid in nbt['Blocks'].value: 168 | try: 169 | i = bids.index(bid) 170 | except ValueError: 171 | bids.append(bid) 172 | i = len(bids) - 1 173 | self.indexes.append(i) 174 | 175 | for bid in bids: 176 | bname = block_id_to_name(bid) 177 | self.names.append(bname) 178 | 179 | 180 | # Decode modern section 181 | # Contains palette of block names and indexes packed with run-on between elements (pre 1.16 format) 182 | 183 | def _init_index_unpadded(self, nbt): 184 | 185 | for p in nbt['Palette']: 186 | name = p['Name'].value 187 | self.names.append(name) 188 | 189 | states = nbt['BlockStates'].value 190 | 191 | # Block states are packed into an array of longs 192 | # with variable number of bits per block (min: 4) 193 | 194 | num_bits = (len(self.names) - 1).bit_length() 195 | if num_bits < 4: num_bits = 4 196 | assert num_bits == len(states) * 64 / 4096 197 | mask = pow(2, num_bits) - 1 198 | 199 | i = 0 200 | bits_left = 64 201 | curr_long = states[0] 202 | 203 | for _ in range(0,4096): 204 | if bits_left == 0: 205 | i = i + 1 206 | curr_long = states[i] 207 | bits_left = 64 208 | 209 | if num_bits <= bits_left: 210 | self.indexes.append(curr_long & mask) 211 | curr_long = curr_long >> num_bits 212 | bits_left = bits_left - num_bits 213 | else: 214 | i = i + 1 215 | next_long = states[i] 216 | remaining_bits = num_bits - bits_left 217 | 218 | next_long = (next_long & (pow(2, remaining_bits) - 1)) << bits_left 219 | curr_long = (curr_long & (pow(2, bits_left) - 1)) 220 | self.indexes.append(next_long | curr_long) 221 | 222 | curr_long = states[i] 223 | curr_long = curr_long >> remaining_bits 224 | bits_left = 64 - remaining_bits 225 | 226 | 227 | # Decode modern section 228 | # Contains palette of block names and indexes packed with padding if elements don't fit (post 1.16 format) 229 | 230 | def _init_index_padded(self, nbt): 231 | 232 | for p in nbt['Palette']: 233 | name = p['Name'].value 234 | self.names.append(name) 235 | 236 | states = nbt['BlockStates'].value 237 | num_bits = (len(self.names) - 1).bit_length() 238 | if num_bits < 4: num_bits = 4 239 | mask = 2**num_bits - 1 240 | 241 | indexes_per_element = 64 // num_bits 242 | last_state_elements = 4096 % indexes_per_element 243 | if last_state_elements == 0: last_state_elements = indexes_per_element 244 | 245 | assert len(states) == ceil(4096 / indexes_per_element) 246 | 247 | for i in range(len(states)-1): 248 | long = states[i] 249 | 250 | for _ in range(indexes_per_element): 251 | self.indexes.append(long & mask) 252 | long = long >> num_bits 253 | 254 | 255 | long = states[-1] 256 | for _ in range(last_state_elements): 257 | self.indexes.append(long & mask) 258 | long = long >> num_bits 259 | 260 | 261 | 262 | def get_block(self, x, y, z): 263 | # Blocks are stored in YZX order 264 | i = y * 256 + z * 16 + x 265 | p = self.indexes[i] 266 | return self.names[p] 267 | 268 | 269 | def iter_block(self): 270 | for i in range(0, 4096): 271 | p = self.indexes[i] 272 | yield self.names[p] 273 | 274 | 275 | # Chunck in Anvil new format 276 | 277 | class AnvilChunk(Chunk): 278 | 279 | def __init__(self, nbt): 280 | Chunk.__init__(self, nbt) 281 | 282 | # Started to work on this class with MC version 1.13.2 283 | # so with the chunk data version 1631 284 | # Backported to first Anvil version (= 0) from examples 285 | # Could work with other versions, but has to be tested first 286 | 287 | try: 288 | version = nbt['DataVersion'].value 289 | if version != 1343 and not (version >= 1631 or version <= 2730): 290 | raise NotImplementedError('DataVersion %d not implemented' % (version,)) 291 | except KeyError: 292 | version = 0 293 | 294 | # Load all sections 295 | 296 | self.sections = {} 297 | if 'Sections' in self.chunk_data: 298 | for s in self.chunk_data['Sections']: 299 | if "BlockStates" in s.keys(): # sections may only contain lighting information 300 | self.sections[s['Y'].value] = AnvilSection(s, version) 301 | 302 | 303 | def get_section(self, y): 304 | """Get a section from Y index.""" 305 | if y in self.sections: 306 | return self.sections[y] 307 | 308 | return None 309 | 310 | 311 | def get_max_height(self): 312 | ymax = 0 313 | for y in self.sections.keys(): 314 | if y > ymax: ymax = y 315 | return ymax * 16 + 15 316 | 317 | 318 | def get_block(self, x, y, z): 319 | """Get a block from relative x,y,z.""" 320 | sy,by = divmod(y, 16) 321 | section = self.get_section(sy) 322 | if section == None: 323 | return None 324 | 325 | return section.get_block(x, by, z) 326 | 327 | 328 | def iter_block(self): 329 | for s in self.sections.values(): 330 | for b in s.iter_block(): 331 | yield b 332 | 333 | 334 | class BlockArray(object): 335 | """Convenience class for dealing with a Block/data byte array.""" 336 | def __init__(self, blocksBytes=None, dataBytes=None): 337 | """Create a new BlockArray, defaulting to no block or data bytes.""" 338 | if isinstance(blocksBytes, (bytearray, array.array)): 339 | self.blocksList = list(blocksBytes) 340 | else: 341 | self.blocksList = [0]*32768 # Create an empty block list (32768 entries of zero (air)) 342 | 343 | if isinstance(dataBytes, (bytearray, array.array)): 344 | self.dataList = list(dataBytes) 345 | else: 346 | self.dataList = [0]*16384 # Create an empty data list (32768 4-bit entries of zero make 16384 byte entries) 347 | 348 | def get_blocks_struct(self): 349 | """Return a dictionary with block ids keyed to (x, y, z).""" 350 | cur_x = 0 351 | cur_y = 0 352 | cur_z = 0 353 | blocks = {} 354 | for block_id in self.blocksList: 355 | blocks[(cur_x,cur_y,cur_z)] = block_id 356 | cur_y += 1 357 | if (cur_y > 127): 358 | cur_y = 0 359 | cur_z += 1 360 | if (cur_z > 15): 361 | cur_z = 0 362 | cur_x += 1 363 | return blocks 364 | 365 | # Give blockList back as a byte array 366 | def get_blocks_byte_array(self, buffer=False): 367 | """Return a list of all blocks in this chunk.""" 368 | if buffer: 369 | length = len(self.blocksList) 370 | return BytesIO(pack(">i", length)+self.get_blocks_byte_array()) 371 | else: 372 | return array.array('B', self.blocksList).tostring() 373 | 374 | def get_data_byte_array(self, buffer=False): 375 | """Return a list of data for all blocks in this chunk.""" 376 | if buffer: 377 | length = len(self.dataList) 378 | return BytesIO(pack(">i", length)+self.get_data_byte_array()) 379 | else: 380 | return array.array('B', self.dataList).tostring() 381 | 382 | def generate_heightmap(self, buffer=False, as_array=False): 383 | """Return a heightmap, representing the highest solid blocks in this chunk.""" 384 | non_solids = [0, 8, 9, 10, 11, 38, 37, 32, 31] 385 | if buffer: 386 | return BytesIO(pack(">i", 256)+self.generate_heightmap()) # Length + Heightmap, ready for insertion into Chunk NBT 387 | else: 388 | bytes = [] 389 | for z in range(16): 390 | for x in range(16): 391 | for y in range(127, -1, -1): 392 | offset = y + z*128 + x*128*16 393 | if (self.blocksList[offset] not in non_solids or y == 0): 394 | bytes.append(y+1) 395 | break 396 | if (as_array): 397 | return bytes 398 | else: 399 | return array.array('B', bytes).tostring() 400 | 401 | def set_blocks(self, list=None, dict=None, fill_air=False): 402 | """ 403 | Sets all blocks in this chunk, using either a list or dictionary. 404 | Blocks not explicitly set can be filled to air by setting fill_air to True. 405 | """ 406 | if list: 407 | # Inputting a list like self.blocksList 408 | self.blocksList = list 409 | elif dict: 410 | # Inputting a dictionary like result of self.get_blocks_struct() 411 | list = [] 412 | for x in range(16): 413 | for z in range(16): 414 | for y in range(128): 415 | coord = x,y,z 416 | offset = y + z*128 + x*128*16 417 | if (coord in dict): 418 | list.append(dict[coord]) 419 | else: 420 | if (self.blocksList[offset] and not fill_air): 421 | list.append(self.blocksList[offset]) 422 | else: 423 | list.append(0) # Air 424 | self.blocksList = list 425 | else: 426 | # None of the above... 427 | return False 428 | return True 429 | 430 | def set_block(self, x,y,z, id, data=0): 431 | """Sets the block a x, y, z to the specified id, and optionally data.""" 432 | offset = y + z*128 + x*128*16 433 | self.blocksList[offset] = id 434 | if (offset % 2 == 1): 435 | # offset is odd 436 | index = (offset-1)//2 437 | b = self.dataList[index] 438 | self.dataList[index] = (b & 240) + (data & 15) # modify lower bits, leaving higher bits in place 439 | else: 440 | # offset is even 441 | index = offset//2 442 | b = self.dataList[index] 443 | self.dataList[index] = (b & 15) + (data << 4 & 240) # modify ligher bits, leaving lower bits in place 444 | 445 | # Get a given X,Y,Z or a tuple of three coordinates 446 | def get_block(self, x,y,z, coord=False): 447 | """Return the id of the block at x, y, z.""" 448 | """ 449 | Laid out like: 450 | (0,0,0), (0,1,0), (0,2,0) ... (0,127,0), (0,0,1), (0,1,1), (0,2,1) ... (0,127,1), (0,0,2) ... (0,127,15), (1,0,0), (1,1,0) ... (15,127,15) 451 | 452 | :: 453 | 454 | blocks = [] 455 | for x in range(15): 456 | for z in range(15): 457 | for y in range(127): 458 | blocks.append(Block(x,y,z)) 459 | """ 460 | 461 | offset = y + z*128 + x*128*16 if (coord == False) else coord[1] + coord[2]*128 + coord[0]*128*16 462 | return self.blocksList[offset] 463 | -------------------------------------------------------------------------------- /nbt/nbt.py: -------------------------------------------------------------------------------- 1 | """ 2 | Handle the NBT (Named Binary Tag) data format 3 | 4 | For more information about the NBT format: 5 | https://minecraft.gamepedia.com/NBT_format 6 | """ 7 | 8 | from struct import Struct, error as StructError 9 | from gzip import GzipFile 10 | 11 | from mutf8 import encode_modified_utf8, decode_modified_utf8 12 | 13 | try: 14 | from collections.abc import MutableMapping, MutableSequence, Sequence 15 | except ImportError: # for Python 2.7 16 | from collections import MutableMapping, MutableSequence, Sequence 17 | import sys 18 | 19 | _PY3 = sys.version_info >= (3,) 20 | if _PY3: 21 | unicode = str 22 | basestring = str 23 | else: 24 | range = xrange 25 | 26 | TAG_END = 0 27 | TAG_BYTE = 1 28 | TAG_SHORT = 2 29 | TAG_INT = 3 30 | TAG_LONG = 4 31 | TAG_FLOAT = 5 32 | TAG_DOUBLE = 6 33 | TAG_BYTE_ARRAY = 7 34 | TAG_STRING = 8 35 | TAG_LIST = 9 36 | TAG_COMPOUND = 10 37 | TAG_INT_ARRAY = 11 38 | TAG_LONG_ARRAY = 12 39 | 40 | 41 | class MalformedFileError(Exception): 42 | """Exception raised on parse error.""" 43 | pass 44 | 45 | 46 | class TAG(object): 47 | """TAG, a variable with an intrinsic name.""" 48 | id = None 49 | 50 | def __init__(self, value=None, name=None): 51 | self.name = name 52 | self.value = value 53 | 54 | # Parsers and Generators 55 | def _parse_buffer(self, buffer): 56 | raise NotImplementedError(self.__class__.__name__) 57 | 58 | def _render_buffer(self, buffer): 59 | raise NotImplementedError(self.__class__.__name__) 60 | 61 | # Printing and Formatting of tree 62 | def tag_info(self): 63 | """Return Unicode string with class, name and unnested value.""" 64 | return self.__class__.__name__ + ( 65 | '(%r)' % self.name if self.name 66 | else "") + ": " + self.valuestr() 67 | 68 | def valuestr(self): 69 | """Return Unicode string of unnested value. For iterators, this 70 | returns a summary.""" 71 | return unicode(self.value) 72 | 73 | def namestr(self): 74 | """Return Unicode string of tag name.""" 75 | return unicode(self.name) 76 | 77 | def pretty_tree(self, indent=0): 78 | """Return formated Unicode string of self, where iterable items are 79 | recursively listed in detail.""" 80 | return ("\t" * indent) + self.tag_info() 81 | 82 | # Python 2 compatibility; Python 3 uses __str__ instead. 83 | def __unicode__(self): 84 | """Return a unicode string with the result in human readable format. 85 | Unlike valuestr(), the result is recursive for iterators till at least 86 | one level deep.""" 87 | return unicode(self.value) 88 | 89 | def __str__(self): 90 | """Return a string (ascii formated for Python 2, unicode for Python 3) 91 | with the result in human readable format. Unlike valuestr(), the result 92 | is recursive for iterators till at least one level deep.""" 93 | return str(self.value) 94 | 95 | # Unlike regular iterators, __repr__() is not recursive. 96 | # Use pretty_tree for recursive results. 97 | # iterators should use __repr__ or tag_info for each item, like 98 | # regular iterators 99 | def __repr__(self): 100 | """Return a string (ascii formated for Python 2, unicode for Python 3) 101 | describing the class, name and id for debugging purposes.""" 102 | return "<%s(%r) at 0x%x>" % ( 103 | self.__class__.__name__, self.name, id(self)) 104 | 105 | 106 | class _TAG_Numeric(TAG): 107 | """_TAG_Numeric, comparable to int with an intrinsic name""" 108 | 109 | def __init__(self, value=None, name=None, buffer=None): 110 | super(_TAG_Numeric, self).__init__(value, name) 111 | if buffer: 112 | self._parse_buffer(buffer) 113 | 114 | # Parsers and Generators 115 | def _parse_buffer(self, buffer): 116 | # Note: buffer.read() may raise an IOError, for example if buffer is a 117 | # corrupt gzip.GzipFile 118 | self.value = self.fmt.unpack(buffer.read(self.fmt.size))[0] 119 | 120 | def _render_buffer(self, buffer): 121 | buffer.write(self.fmt.pack(self.value)) 122 | 123 | 124 | class _TAG_End(TAG): 125 | id = TAG_END 126 | fmt = Struct(">b") 127 | 128 | def _parse_buffer(self, buffer): 129 | # Note: buffer.read() may raise an IOError, for example if buffer is a 130 | # corrupt gzip.GzipFile 131 | value = self.fmt.unpack(buffer.read(1))[0] 132 | if value != 0: 133 | raise ValueError( 134 | "A Tag End must be rendered as '0', not as '%d'." % value) 135 | 136 | def _render_buffer(self, buffer): 137 | buffer.write(b'\x00') 138 | 139 | 140 | # == Value Tags ==# 141 | class TAG_Byte(_TAG_Numeric): 142 | """Represent a single tag storing 1 byte.""" 143 | id = TAG_BYTE 144 | fmt = Struct(">b") 145 | 146 | 147 | class TAG_Short(_TAG_Numeric): 148 | """Represent a single tag storing 1 short.""" 149 | id = TAG_SHORT 150 | fmt = Struct(">h") 151 | 152 | 153 | class TAG_Int(_TAG_Numeric): 154 | """Represent a single tag storing 1 int.""" 155 | id = TAG_INT 156 | fmt = Struct(">i") 157 | """Struct(">i"), 32-bits integer, big-endian""" 158 | 159 | 160 | class TAG_Long(_TAG_Numeric): 161 | """Represent a single tag storing 1 long.""" 162 | id = TAG_LONG 163 | fmt = Struct(">q") 164 | 165 | 166 | class TAG_Float(_TAG_Numeric): 167 | """Represent a single tag storing 1 IEEE-754 floating point number.""" 168 | id = TAG_FLOAT 169 | fmt = Struct(">f") 170 | 171 | 172 | class TAG_Double(_TAG_Numeric): 173 | """Represent a single tag storing 1 IEEE-754 double precision floating 174 | point number.""" 175 | id = TAG_DOUBLE 176 | fmt = Struct(">d") 177 | 178 | 179 | class TAG_Byte_Array(TAG, MutableSequence): 180 | """ 181 | TAG_Byte_Array, comparable to a collections.UserList with 182 | an intrinsic name whose values must be bytes 183 | """ 184 | id = TAG_BYTE_ARRAY 185 | 186 | def __init__(self, name=None, buffer=None): 187 | # TODO: add a value parameter as well 188 | super(TAG_Byte_Array, self).__init__(name=name) 189 | if buffer: 190 | self._parse_buffer(buffer) 191 | 192 | # Parsers and Generators 193 | def _parse_buffer(self, buffer): 194 | length = TAG_Int(buffer=buffer) 195 | self.value = bytearray(buffer.read(length.value)) 196 | 197 | def _render_buffer(self, buffer): 198 | length = TAG_Int(len(self.value)) 199 | length._render_buffer(buffer) 200 | buffer.write(bytes(self.value)) 201 | 202 | # Mixin methods 203 | def __len__(self): 204 | return len(self.value) 205 | 206 | def __iter__(self): 207 | return iter(self.value) 208 | 209 | def __contains__(self, item): 210 | return item in self.value 211 | 212 | def __getitem__(self, key): 213 | return self.value[key] 214 | 215 | def __setitem__(self, key, value): 216 | # TODO: check type of value 217 | self.value[key] = value 218 | 219 | def __delitem__(self, key): 220 | del (self.value[key]) 221 | 222 | def insert(self, key, value): 223 | # TODO: check type of value, or is this done by self.value already? 224 | self.value.insert(key, value) 225 | 226 | # Printing and Formatting of tree 227 | def valuestr(self): 228 | return "[%i byte(s)]" % len(self.value) 229 | 230 | def __unicode__(self): 231 | return '[' + ",".join([unicode(x) for x in self.value]) + ']' 232 | 233 | def __str__(self): 234 | return '[' + ",".join([str(x) for x in self.value]) + ']' 235 | 236 | 237 | class TAG_Int_Array(TAG, MutableSequence): 238 | """ 239 | TAG_Int_Array, comparable to a collections.UserList with 240 | an intrinsic name whose values must be integers 241 | """ 242 | id = TAG_INT_ARRAY 243 | 244 | def __init__(self, name=None, buffer=None): 245 | # TODO: add a value parameter as well 246 | super(TAG_Int_Array, self).__init__(name=name) 247 | if buffer: 248 | self._parse_buffer(buffer) 249 | 250 | def update_fmt(self, length): 251 | """ Adjust struct format description to length given """ 252 | self.fmt = Struct(">" + str(length) + "i") 253 | 254 | # Parsers and Generators 255 | def _parse_buffer(self, buffer): 256 | length = TAG_Int(buffer=buffer).value 257 | self.update_fmt(length) 258 | self.value = list(self.fmt.unpack(buffer.read(self.fmt.size))) 259 | 260 | def _render_buffer(self, buffer): 261 | length = len(self.value) 262 | self.update_fmt(length) 263 | TAG_Int(length)._render_buffer(buffer) 264 | buffer.write(self.fmt.pack(*self.value)) 265 | 266 | # Mixin methods 267 | def __len__(self): 268 | return len(self.value) 269 | 270 | def __iter__(self): 271 | return iter(self.value) 272 | 273 | def __contains__(self, item): 274 | return item in self.value 275 | 276 | def __getitem__(self, key): 277 | return self.value[key] 278 | 279 | def __setitem__(self, key, value): 280 | self.value[key] = value 281 | 282 | def __delitem__(self, key): 283 | del (self.value[key]) 284 | 285 | def insert(self, key, value): 286 | self.value.insert(key, value) 287 | 288 | # Printing and Formatting of tree 289 | def valuestr(self): 290 | return "[%i int(s)]" % len(self.value) 291 | 292 | 293 | class TAG_Long_Array(TAG, MutableSequence): 294 | """ 295 | TAG_Long_Array, comparable to a collections.UserList with 296 | an intrinsic name whose values must be integers 297 | """ 298 | id = TAG_LONG_ARRAY 299 | 300 | def __init__(self, name=None, buffer=None): 301 | super(TAG_Long_Array, self).__init__(name=name) 302 | if buffer: 303 | self._parse_buffer(buffer) 304 | 305 | def update_fmt(self, length): 306 | """ Adjust struct format description to length given """ 307 | self.fmt = Struct(">" + str(length) + "q") 308 | 309 | # Parsers and Generators 310 | def _parse_buffer(self, buffer): 311 | length = TAG_Int(buffer=buffer).value 312 | self.update_fmt(length) 313 | self.value = list(self.fmt.unpack(buffer.read(self.fmt.size))) 314 | 315 | def _render_buffer(self, buffer): 316 | length = len(self.value) 317 | self.update_fmt(length) 318 | TAG_Int(length)._render_buffer(buffer) 319 | buffer.write(self.fmt.pack(*self.value)) 320 | 321 | # Mixin methods 322 | def __len__(self): 323 | return len(self.value) 324 | 325 | def __iter__(self): 326 | return iter(self.value) 327 | 328 | def __contains__(self, item): 329 | return item in self.value 330 | 331 | def __getitem__(self, key): 332 | return self.value[key] 333 | 334 | def __setitem__(self, key, value): 335 | self.value[key] = value 336 | 337 | def __delitem__(self, key): 338 | del (self.value[key]) 339 | 340 | def insert(self, key, value): 341 | self.value.insert(key, value) 342 | 343 | # Printing and Formatting of tree 344 | def valuestr(self): 345 | return "[%i long(s)]" % len(self.value) 346 | 347 | 348 | class TAG_String(TAG, Sequence): 349 | """ 350 | TAG_String, comparable to a collections.UserString with an 351 | intrinsic name 352 | """ 353 | id = TAG_STRING 354 | 355 | def __init__(self, value=None, name=None, buffer=None): 356 | super(TAG_String, self).__init__(value, name) 357 | if buffer: 358 | self._parse_buffer(buffer) 359 | 360 | # Parsers and Generators 361 | def _parse_buffer(self, buffer): 362 | length = TAG_Short(buffer=buffer) 363 | read = buffer.read(length.value) 364 | if len(read) != length.value: 365 | raise StructError() 366 | #self.value = read.decode("utf-8") 367 | self.value = decode_modified_utf8(read) 368 | 369 | def _render_buffer(self, buffer): 370 | #save_val = self.value.encode("utf-8") 371 | save_val = encode_modified_utf8(self.value) 372 | length = TAG_Short(len(save_val)) 373 | length._render_buffer(buffer) 374 | buffer.write(save_val) 375 | 376 | # Mixin methods 377 | def __len__(self): 378 | return len(self.value) 379 | 380 | def __iter__(self): 381 | return iter(self.value) 382 | 383 | def __contains__(self, item): 384 | return item in self.value 385 | 386 | def __getitem__(self, key): 387 | return self.value[key] 388 | 389 | # Printing and Formatting of tree 390 | def __repr__(self): 391 | return self.value 392 | 393 | 394 | # == Collection Tags ==# 395 | class TAG_List(TAG, MutableSequence): 396 | """ 397 | TAG_List, comparable to a collections.UserList with an intrinsic name 398 | """ 399 | id = TAG_LIST 400 | 401 | def __init__(self, type=None, value=None, name=None, buffer=None): 402 | super(TAG_List, self).__init__(value, name) 403 | if type: 404 | self.tagID = type.id 405 | else: 406 | self.tagID = None 407 | self.tags = [] 408 | if buffer: 409 | self._parse_buffer(buffer) 410 | # if self.tagID == None: 411 | # raise ValueError("No type specified for list: %s" % (name)) 412 | 413 | # Parsers and Generators 414 | def _parse_buffer(self, buffer): 415 | self.tagID = TAG_Byte(buffer=buffer).value 416 | self.tags = [] 417 | length = TAG_Int(buffer=buffer) 418 | for x in range(length.value): 419 | self.tags.append(TAGLIST[self.tagID](buffer=buffer)) 420 | 421 | def _render_buffer(self, buffer): 422 | TAG_Byte(self.tagID)._render_buffer(buffer) 423 | length = TAG_Int(len(self.tags)) 424 | length._render_buffer(buffer) 425 | for i, tag in enumerate(self.tags): 426 | if tag.id != self.tagID: 427 | raise ValueError( 428 | "List element %d(%s) has type %d != container type %d" % 429 | (i, tag, tag.id, self.tagID)) 430 | tag._render_buffer(buffer) 431 | 432 | # Mixin methods 433 | def __len__(self): 434 | return len(self.tags) 435 | 436 | def __iter__(self): 437 | return iter(self.tags) 438 | 439 | def __contains__(self, item): 440 | return item in self.tags 441 | 442 | def __getitem__(self, key): 443 | return self.tags[key] 444 | 445 | def __setitem__(self, key, value): 446 | self.tags[key] = value 447 | 448 | def __delitem__(self, key): 449 | del (self.tags[key]) 450 | 451 | def insert(self, key, value): 452 | self.tags.insert(key, value) 453 | 454 | # Printing and Formatting of tree 455 | def __repr__(self): 456 | return "%i entries of type %s" % ( 457 | len(self.tags), TAGLIST[self.tagID].__name__) 458 | 459 | # Printing and Formatting of tree 460 | def valuestr(self): 461 | return "[%i %s(s)]" % (len(self.tags), TAGLIST[self.tagID].__name__) 462 | 463 | def __unicode__(self): 464 | return "[" + ", ".join([tag.tag_info() for tag in self.tags]) + "]" 465 | 466 | def __str__(self): 467 | return "[" + ", ".join([tag.tag_info() for tag in self.tags]) + "]" 468 | 469 | def pretty_tree(self, indent=0): 470 | output = [super(TAG_List, self).pretty_tree(indent)] 471 | if len(self.tags): 472 | output.append(("\t" * indent) + "{") 473 | output.extend([tag.pretty_tree(indent + 1) for tag in self.tags]) 474 | output.append(("\t" * indent) + "}") 475 | return '\n'.join(output) 476 | 477 | 478 | class TAG_Compound(TAG, MutableMapping): 479 | """ 480 | TAG_Compound, comparable to a collections.OrderedDict with an 481 | intrinsic name 482 | """ 483 | id = TAG_COMPOUND 484 | 485 | def __init__(self, buffer=None, name=None): 486 | # TODO: add a value parameter as well 487 | super(TAG_Compound, self).__init__() 488 | self.tags = [] 489 | if name: 490 | self.name = name 491 | else: 492 | self.name = "" 493 | if buffer: 494 | self._parse_buffer(buffer) 495 | 496 | # Parsers and Generators 497 | def _parse_buffer(self, buffer): 498 | while True: 499 | type = TAG_Byte(buffer=buffer) 500 | if type.value == TAG_END: 501 | # print("found tag_end") 502 | break 503 | else: 504 | name = TAG_String(buffer=buffer).value 505 | try: 506 | tag = TAGLIST[type.value]() 507 | except KeyError: 508 | raise ValueError("Unrecognised tag type %d" % type.value) 509 | tag.name = name 510 | self.tags.append(tag) 511 | tag._parse_buffer(buffer) 512 | 513 | def _render_buffer(self, buffer): 514 | for tag in self.tags: 515 | TAG_Byte(tag.id)._render_buffer(buffer) 516 | TAG_String(tag.name)._render_buffer(buffer) 517 | tag._render_buffer(buffer) 518 | buffer.write(b'\x00') # write TAG_END 519 | 520 | # Mixin methods 521 | def __len__(self): 522 | return len(self.tags) 523 | 524 | def __iter__(self): 525 | for key in self.tags: 526 | yield key.name 527 | 528 | def __contains__(self, key): 529 | if isinstance(key, int): 530 | return key <= len(self.tags) 531 | elif isinstance(key, basestring): 532 | for tag in self.tags: 533 | if tag.name == key: 534 | return True 535 | return False 536 | elif isinstance(key, TAG): 537 | return key in self.tags 538 | return False 539 | 540 | def __getitem__(self, key): 541 | if isinstance(key, int): 542 | return self.tags[key] 543 | elif isinstance(key, basestring): 544 | for tag in self.tags: 545 | if tag.name == key: 546 | return tag 547 | else: 548 | raise KeyError("Tag %s does not exist" % key) 549 | else: 550 | raise TypeError( 551 | "key needs to be either name of tag, or index of tag, " 552 | "not a %s" % type(key).__name__) 553 | 554 | def __setitem__(self, key, value): 555 | assert isinstance(value, TAG), "value must be an nbt.TAG" 556 | if isinstance(key, int): 557 | # Just try it. The proper error will be raised if it doesn't work. 558 | self.tags[key] = value 559 | elif isinstance(key, basestring): 560 | value.name = key 561 | for i, tag in enumerate(self.tags): 562 | if tag.name == key: 563 | self.tags[i] = value 564 | return 565 | self.tags.append(value) 566 | 567 | def __delitem__(self, key): 568 | if isinstance(key, int): 569 | del (self.tags[key]) 570 | elif isinstance(key, basestring): 571 | self.tags.remove(self.__getitem__(key)) 572 | else: 573 | raise ValueError( 574 | "key needs to be either name of tag, or index of tag") 575 | 576 | def keys(self): 577 | return [tag.name for tag in self.tags] 578 | 579 | def iteritems(self): 580 | for tag in self.tags: 581 | yield (tag.name, tag) 582 | 583 | # Printing and Formatting of tree 584 | def __unicode__(self): 585 | return "{" + ", ".join([tag.tag_info() for tag in self.tags]) + "}" 586 | 587 | def __str__(self): 588 | return "{" + ", ".join([tag.tag_info() for tag in self.tags]) + "}" 589 | 590 | def valuestr(self): 591 | return '{%i Entries}' % len(self.tags) 592 | 593 | def pretty_tree(self, indent=0): 594 | output = [super(TAG_Compound, self).pretty_tree(indent)] 595 | if len(self.tags): 596 | output.append(("\t" * indent) + "{") 597 | output.extend([tag.pretty_tree(indent + 1) for tag in self.tags]) 598 | output.append(("\t" * indent) + "}") 599 | return '\n'.join(output) 600 | 601 | 602 | TAGLIST = {TAG_END: _TAG_End, TAG_BYTE: TAG_Byte, TAG_SHORT: TAG_Short, 603 | TAG_INT: TAG_Int, TAG_LONG: TAG_Long, TAG_FLOAT: TAG_Float, 604 | TAG_DOUBLE: TAG_Double, TAG_BYTE_ARRAY: TAG_Byte_Array, 605 | TAG_STRING: TAG_String, TAG_LIST: TAG_List, 606 | TAG_COMPOUND: TAG_Compound, TAG_INT_ARRAY: TAG_Int_Array, 607 | TAG_LONG_ARRAY: TAG_Long_Array} 608 | 609 | 610 | class NBTFile(TAG_Compound): 611 | """Represent an NBT file object.""" 612 | 613 | def __init__(self, filename=None, buffer=None, fileobj=None): 614 | """ 615 | Create a new NBTFile object. 616 | Specify either a filename, file object or data buffer. 617 | If filename of file object is specified, data should be GZip-compressed. 618 | If a data buffer is specified, it is assumed to be uncompressed. 619 | 620 | If filename is specified, the file is closed after reading and writing. 621 | If file object is specified, the caller is responsible for closing the 622 | file. 623 | """ 624 | super(NBTFile, self).__init__() 625 | self.filename = filename 626 | self.type = TAG_Byte(self.id) 627 | closefile = True 628 | # make a file object 629 | if filename: 630 | self.filename = filename 631 | self.file = GzipFile(filename, 'rb') 632 | elif buffer: 633 | if hasattr(buffer, 'name'): 634 | self.filename = buffer.name 635 | self.file = buffer 636 | closefile = False 637 | elif fileobj: 638 | if hasattr(fileobj, 'name'): 639 | self.filename = fileobj.name 640 | self.file = GzipFile(fileobj=fileobj) 641 | else: 642 | self.file = None 643 | closefile = False 644 | # parse the file given initially 645 | if self.file: 646 | self.parse_file() 647 | if closefile: 648 | # Note: GzipFile().close() does NOT close the fileobj, 649 | # So we are still responsible for closing that. 650 | try: 651 | self.file.close() 652 | except (AttributeError, IOError): 653 | pass 654 | self.file = None 655 | 656 | def parse_file(self, filename=None, buffer=None, fileobj=None): 657 | """Completely parse a file, extracting all tags.""" 658 | closefile = True 659 | if filename: 660 | self.file = GzipFile(filename, 'rb') 661 | elif buffer: 662 | if hasattr(buffer, 'name'): 663 | self.filename = buffer.name 664 | self.file = buffer 665 | closefile = False 666 | elif fileobj: 667 | if hasattr(fileobj, 'name'): 668 | self.filename = fileobj.name 669 | self.file = GzipFile(fileobj=fileobj) 670 | if self.file: 671 | try: 672 | type = TAG_Byte(buffer=self.file) 673 | if type.value == self.id: 674 | name = TAG_String(buffer=self.file).value 675 | self._parse_buffer(self.file) 676 | self.name = name 677 | if closefile: 678 | self.file.close() 679 | else: 680 | raise MalformedFileError( 681 | "First record is not a Compound Tag") 682 | except StructError as e: 683 | raise MalformedFileError( 684 | "Partial File Parse: file possibly truncated.") 685 | else: 686 | raise ValueError( 687 | "NBTFile.parse_file(): Need to specify either a " 688 | "filename or a file object" 689 | ) 690 | 691 | def write_file(self, filename=None, buffer=None, fileobj=None): 692 | """Write this NBT file to a file.""" 693 | closefile = True 694 | if buffer: 695 | self.filename = None 696 | self.file = buffer 697 | closefile = False 698 | elif filename: 699 | self.filename = filename 700 | self.file = GzipFile(filename, "wb") 701 | elif fileobj: 702 | self.filename = None 703 | self.file = GzipFile(fileobj=fileobj, mode="wb") 704 | elif self.filename: 705 | self.file = GzipFile(self.filename, "wb") 706 | elif not self.file: 707 | raise ValueError( 708 | "NBTFile.write_file(): Need to specify either a " 709 | "filename or a file object" 710 | ) 711 | # Render tree to file 712 | TAG_Byte(self.id)._render_buffer(self.file) 713 | TAG_String(self.name)._render_buffer(self.file) 714 | self._render_buffer(self.file) 715 | # make sure the file is complete 716 | try: 717 | self.file.flush() 718 | except (AttributeError, IOError): 719 | pass 720 | if closefile: 721 | try: 722 | self.file.close() 723 | except (AttributeError, IOError): 724 | pass 725 | 726 | def __repr__(self): 727 | """ 728 | Return a string (ascii formated for Python 2, unicode 729 | for Python 3) describing the class, name and id for 730 | debugging purposes. 731 | """ 732 | if self.filename: 733 | return "<%s(%r) with %s(%r) at 0x%x>" % ( 734 | self.__class__.__name__, self.filename, 735 | TAG_Compound.__name__, self.name, id(self) 736 | ) 737 | else: 738 | return "<%s with %s(%r) at 0x%x>" % ( 739 | self.__class__.__name__, TAG_Compound.__name__, 740 | self.name, id(self) 741 | ) 742 | -------------------------------------------------------------------------------- /nbt/world.py: -------------------------------------------------------------------------------- 1 | """ 2 | Handles a Minecraft world save using either the Anvil or McRegion format. 3 | 4 | For more information about the world format: 5 | https://minecraft.gamepedia.com/Level_format 6 | """ 7 | 8 | import os, glob, re 9 | from . import region 10 | from . import chunk 11 | from .region import InconceivedChunk, Location 12 | 13 | class UnknownWorldFormat(Exception): 14 | """Unknown or invalid world folder.""" 15 | def __init__(self, msg=""): 16 | self.msg = msg 17 | 18 | 19 | class _BaseWorldFolder(object): 20 | """ 21 | Abstract class, representing either a McRegion or Anvil world folder. 22 | This class will use either Anvil or McRegion, with Anvil the preferred format. 23 | Simply calling WorldFolder() will do this automatically. 24 | """ 25 | type = "Generic" 26 | extension = '' 27 | chunkclass = chunk.Chunk 28 | 29 | def __init__(self, world_folder): 30 | """Initialize a WorldFolder.""" 31 | self.worldfolder = world_folder 32 | self.regionfiles = {} 33 | self.regions = {} 34 | self.chunks = None 35 | # os.listdir triggers an OSError for non-existant directories or permission errors. 36 | # This is needed, because glob.glob silently returns no files. 37 | os.listdir(world_folder) 38 | self.set_regionfiles(self.get_filenames()) 39 | 40 | def get_filenames(self): 41 | """Find all matching file names in the world folder. 42 | 43 | This method is private, and it's use it deprecated. Use get_regionfiles() instead.""" 44 | # Warning: glob returns a empty list if the directory is unreadable, without raising an Exception 45 | return list(glob.glob(os.path.join(self.worldfolder,'region','r.*.*.'+self.extension))) 46 | 47 | def set_regionfiles(self, filenames): 48 | """ 49 | This method directly sets the region files for this instance to use. 50 | It assumes the filenames are in the form r... 51 | """ 52 | for filename in filenames: 53 | # Assume that filenames have the name r... 54 | m = re.match(r"r.(\-?\d+).(\-?\d+)."+self.extension, os.path.basename(filename)) 55 | if m: 56 | x = int(m.group(1)) 57 | z = int(m.group(2)) 58 | else: 59 | # Only raised if a .mca of .mcr file exists which does not comply to the 60 | # r... filename format. This may raise false 61 | # errors if a copy is made, e.g. "r.0.-1 copy.mca". If this is an issue, override 62 | # get_filenames(). In most cases, it is an error, and we like to raise that. 63 | # Changed, no longer raise error, because we want to continue the loop. 64 | # raise UnknownWorldFormat("Unrecognized filename format %s" % os.path.basename(filename)) 65 | # TODO: log to stderr using logging facility. 66 | pass 67 | self.regionfiles[(x,z)] = filename 68 | 69 | def get_regionfiles(self): 70 | """Return a list of full path of all region files.""" 71 | return list(self.regionfiles.values()) 72 | 73 | def nonempty(self): 74 | """Return True is the world is non-empty.""" 75 | return len(self.regionfiles) > 0 76 | 77 | def get_region(self, x,z): 78 | """Get a region using x,z coordinates of a region. Cache results.""" 79 | if (x,z) not in self.regions or self.regions[x,z].closed: 80 | if (x,z) in self.regionfiles: 81 | self.regions[(x,z)] = region.RegionFile(self.regionfiles[(x,z)]) 82 | else: 83 | # Return an empty RegionFile object 84 | # TODO: this does not yet allow for saving of the region file 85 | # TODO: this currently fails with a ValueError! 86 | # TODO: generate the correct name, and create the file 87 | # and add the fie to self.regionfiles 88 | self.regions[(x,z)] = region.RegionFile() 89 | self.regions[(x,z)].loc = Location(x=x,z=z) 90 | return self.regions[(x,z)] 91 | 92 | def iter_regions(self): 93 | """ 94 | Return an iterable list of all region files. Use this function if you only 95 | want to loop through each region files once, and do not want to cache the results. 96 | """ 97 | # TODO: Implement BoundingBox 98 | # TODO: Implement sort order 99 | for x,z in self.regionfiles.keys(): 100 | close_after_use = False 101 | if (x,z) in self.regions: 102 | regionfile = self.regions[(x,z)] 103 | else: 104 | # It is not yet cached. 105 | # Get file, but do not cache later. 106 | regionfile = region.RegionFile(self.regionfiles[(x,z)], chunkclass = self.chunkclass) 107 | regionfile.loc = Location(x=x,z=z) 108 | close_after_use = True 109 | try: 110 | yield regionfile 111 | finally: 112 | if close_after_use: 113 | regionfile.close() 114 | 115 | def call_for_each_region(self, callback_function, boundingbox=None): 116 | """ 117 | Return an iterable that calls callback_function for each region file 118 | in the world. This is equivalent to: 119 | ``` 120 | for the_region in iter_regions(): 121 | yield callback_function(the_region) 122 | ```` 123 | 124 | This function is threaded. It uses pickle to pass values between threads. 125 | See [What can be pickled and unpickled?](https://docs.python.org/library/pickle.html#what-can-be-pickled-and-unpickled) in the Python documentation 126 | for limitation on the output of `callback_function()`. 127 | """ 128 | raise NotImplementedError() 129 | 130 | def get_nbt(self,x,z): 131 | """ 132 | Return a NBT specified by the chunk coordinates x,z. Raise InconceivedChunk 133 | if the NBT file is not yet generated. To get a Chunk object, use get_chunk. 134 | """ 135 | rx,cx = divmod(x,32) 136 | rz,cz = divmod(z,32) 137 | if (rx,rz) not in self.regions and (rx,rz) not in self.regionfiles: 138 | raise InconceivedChunk("Chunk %s,%s is not present in world" % (x,z)) 139 | nbt = self.get_region(rx,rz).get_nbt(cx,cz) 140 | assert nbt != None 141 | return nbt 142 | 143 | def set_nbt(self,x,z,nbt): 144 | """ 145 | Set a chunk. Overrides the NBT if it already existed. If the NBT did not exists, 146 | adds it to the Regionfile. May create a new Regionfile if that did not exist yet. 147 | nbt must be a nbt.NBTFile instance, not a Chunk or regular TAG_Compound object. 148 | """ 149 | raise NotImplementedError() 150 | # TODO: implement 151 | 152 | def iter_nbt(self): 153 | """ 154 | Return an iterable list of all NBT. Use this function if you only 155 | want to loop through the chunks once, and don't need the block or data arrays. 156 | """ 157 | # TODO: Implement BoundingBox 158 | # TODO: Implement sort order 159 | for region in self.iter_regions(): 160 | for c in region.iter_chunks(): 161 | yield c 162 | 163 | def call_for_each_nbt(self, callback_function, boundingbox=None): 164 | """ 165 | Return an iterable that calls callback_function for each NBT structure 166 | in the world. This is equivalent to: 167 | ``` 168 | for the_nbt in iter_nbt(): 169 | yield callback_function(the_nbt) 170 | ```` 171 | 172 | This function is threaded. It uses pickle to pass values between threads. 173 | See [What can be pickled and unpickled?](https://docs.python.org/library/pickle.html#what-can-be-pickled-and-unpickled) in the Python documentation 174 | for limitation on the output of `callback_function()`. 175 | """ 176 | raise NotImplementedError() 177 | 178 | def get_chunk(self,x,z): 179 | """ 180 | Return a chunk specified by the chunk coordinates x,z. Raise InconceivedChunk 181 | if the chunk is not yet generated. To get the raw NBT data, use get_nbt. 182 | """ 183 | return self.chunkclass(self.get_nbt(x, z)) 184 | 185 | def get_chunks(self, boundingbox=None): 186 | """ 187 | Return a list of all chunks. Use this function if you access the chunk 188 | list frequently and want to cache the result. 189 | Use iter_chunks() if you only want to loop through the chunks once or have a 190 | very large world. 191 | """ 192 | if self.chunks == None: 193 | self.chunks = list(self.iter_chunks()) 194 | return self.chunks 195 | 196 | def iter_chunks(self): 197 | """ 198 | Return an iterable list of all chunks. Use this function if you only 199 | want to loop through the chunks once or have a very large world. 200 | Use get_chunks() if you access the chunk list frequently and want to cache 201 | the results. Use iter_nbt() if you are concerned about speed and don't want 202 | to parse the block data. 203 | """ 204 | # TODO: Implement BoundingBox 205 | # TODO: Implement sort order 206 | for c in self.iter_nbt(): 207 | yield self.chunkclass(c) 208 | 209 | def chunk_count(self): 210 | """Return a count of the chunks in this world folder.""" 211 | c = 0 212 | for r in self.iter_regions(): 213 | c += r.chunk_count() 214 | return c 215 | 216 | def get_boundingbox(self): 217 | """ 218 | Return minimum and maximum x and z coordinates of the chunks that 219 | make up this world save 220 | """ 221 | b = BoundingBox() 222 | for rx,rz in self.regionfiles.keys(): 223 | region = self.get_region(rx,rz) 224 | rx,rz = 32*rx,32*rz 225 | for cc in region.get_chunk_coords(): 226 | x,z = (rx+cc['x'],rz+cc['z']) 227 | b.expand(x,None,z) 228 | return b 229 | 230 | def __repr__(self): 231 | return "%s(%r)" % (self.__class__.__name__,self.worldfolder) 232 | 233 | 234 | class McRegionWorldFolder(_BaseWorldFolder): 235 | """Represents a world save using the old McRegion format.""" 236 | type = "McRegion" 237 | extension = 'mcr' 238 | chunkclass = chunk.McRegionChunk 239 | 240 | 241 | class AnvilWorldFolder(_BaseWorldFolder): 242 | """Represents a world save using the new Anvil format.""" 243 | type = "Anvil" 244 | extension = 'mca' 245 | chunkclass = chunk.AnvilChunk 246 | 247 | 248 | class _WorldFolderFactory(object): 249 | """Factory class: instantiate the subclassses in order, and the first instance 250 | whose nonempty() method returns True is returned. If no nonempty() returns True, 251 | a UnknownWorldFormat exception is raised.""" 252 | def __init__(self, subclasses): 253 | self.subclasses = subclasses 254 | def __call__(self, *args, **kwargs): 255 | for cls in self.subclasses: 256 | wf = cls(*args, **kwargs) 257 | if wf.nonempty(): # Check if the world is non-empty 258 | return wf 259 | raise UnknownWorldFormat("Empty world or unknown format") 260 | 261 | WorldFolder = _WorldFolderFactory([AnvilWorldFolder, McRegionWorldFolder]) 262 | """ 263 | Factory instance that returns a AnvilWorldFolder or McRegionWorldFolder 264 | instance, or raise a UnknownWorldFormat. 265 | """ 266 | 267 | 268 | 269 | class BoundingBox(object): 270 | """A bounding box of x,y,z coordinates.""" 271 | def __init__(self, minx=None, maxx=None, miny=None, maxy=None, minz=None, maxz=None): 272 | self.minx,self.maxx = minx, maxx 273 | self.miny,self.maxy = miny, maxy 274 | self.minz,self.maxz = minz, maxz 275 | def expand(self,x,y,z): 276 | """ 277 | Expands the bounding 278 | """ 279 | if x != None: 280 | if self.minx is None or x < self.minx: 281 | self.minx = x 282 | if self.maxx is None or x > self.maxx: 283 | self.maxx = x 284 | if y != None: 285 | if self.miny is None or y < self.miny: 286 | self.miny = y 287 | if self.maxy is None or y > self.maxy: 288 | self.maxy = y 289 | if z != None: 290 | if self.minz is None or z < self.minz: 291 | self.minz = z 292 | if self.maxz is None or z > self.maxz: 293 | self.maxz = z 294 | def lenx(self): 295 | if self.maxx is None or self.minx is None: 296 | return 0 297 | return self.maxx-self.minx+1 298 | def leny(self): 299 | if self.maxy is None or self.miny is None: 300 | return 0 301 | return self.maxy-self.miny+1 302 | def lenz(self): 303 | if self.maxz is None or self.minz is None: 304 | return 0 305 | return self.maxz-self.minz+1 306 | def __repr__(self): 307 | return "%s(%s,%s,%s,%s,%s,%s)" % (self.__class__.__name__,self.minx,self.maxx, 308 | self.miny,self.maxy,self.minz,self.maxz) 309 | -------------------------------------------------------------------------------- /progressbar/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | # 4 | # progressbar - Text progress bar library for Python. 5 | # Copyright (c) 2005 Nilton Volpato 6 | # 7 | # This library is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU Lesser General Public 9 | # License as published by the Free Software Foundation; either 10 | # version 2.1 of the License, or (at your option) any later version. 11 | # 12 | # This library is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 15 | # Lesser General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU Lesser General Public 18 | # License along with this library; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 20 | 21 | """Text progress bar library for Python. 22 | 23 | A text progress bar is typically used to display the progress of a long 24 | running operation, providing a visual cue that processing is underway. 25 | 26 | The ProgressBar class manages the current progress, and the format of the line 27 | is given by a number of widgets. A widget is an object that may display 28 | differently depending on the state of the progress bar. There are three types 29 | of widgets: 30 | - a string, which always shows itself 31 | 32 | - a ProgressBarWidget, which may return a different value every time its 33 | update method is called 34 | 35 | - a ProgressBarWidgetHFill, which is like ProgressBarWidget, except it 36 | expands to fill the remaining width of the line. 37 | 38 | The progressbar module is very easy to use, yet very powerful. It will also 39 | automatically enable features like auto-resizing when the system supports it. 40 | """ 41 | 42 | __author__ = 'Nilton Volpato' 43 | __author_email__ = 'nilton.volpato@gmail.com' 44 | __date__ = '2011-05-14' 45 | __version__ = '2.5' 46 | 47 | from .compat import * 48 | from .widgets import * 49 | from .progressbar import * 50 | -------------------------------------------------------------------------------- /progressbar/compat.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # progressbar - Text progress bar library for Python. 4 | # Copyright (c) 2005 Nilton Volpato 5 | # 6 | # This library is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU Lesser General Public 8 | # License as published by the Free Software Foundation; either 9 | # version 2.1 of the License, or (at your option) any later version. 10 | # 11 | # This library is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 14 | # Lesser General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU Lesser General Public 17 | # License along with this library; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 19 | 20 | """Compatibility methods and classes for the progressbar module.""" 21 | 22 | 23 | # Python 3.x (and backports) use a modified iterator syntax 24 | # This will allow 2.x to behave with 3.x iterators 25 | try: 26 | next 27 | except NameError: 28 | def next(iter): 29 | try: 30 | # Try new style iterators 31 | return iter.__next__() 32 | except AttributeError: 33 | # Fallback in case of a "native" iterator 34 | return iter.next() 35 | 36 | 37 | # Python < 2.5 does not have "any" 38 | try: 39 | any 40 | except NameError: 41 | def any(iterator): 42 | for item in iterator: 43 | if item: return True 44 | return False 45 | -------------------------------------------------------------------------------- /progressbar/progressbar.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # progressbar - Text progress bar library for Python. 4 | # Copyright (c) 2005 Nilton Volpato 5 | # 6 | # This library is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU Lesser General Public 8 | # License as published by the Free Software Foundation; either 9 | # version 2.1 of the License, or (at your option) any later version. 10 | # 11 | # This library is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 14 | # Lesser General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU Lesser General Public 17 | # License along with this library; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 19 | 20 | """Main ProgressBar class.""" 21 | 22 | from __future__ import division 23 | 24 | import math 25 | import os 26 | import signal 27 | import sys 28 | import time 29 | 30 | try: 31 | from fcntl import ioctl 32 | from array import array 33 | import termios 34 | except ImportError: 35 | pass 36 | 37 | from .compat import * # for: any, next 38 | from . import widgets 39 | 40 | 41 | class ProgressBar(object): 42 | """The ProgressBar class which updates and prints the bar. 43 | 44 | A common way of using it is like: 45 | >>> pbar = ProgressBar().start() 46 | >>> for i in range(100): 47 | ... # do something 48 | ... pbar.update(i+1) 49 | ... 50 | >>> pbar.finish() 51 | 52 | You can also use a ProgressBar as an iterator: 53 | >>> progress = ProgressBar() 54 | >>> for i in progress(some_iterable): 55 | ... # do something 56 | ... 57 | 58 | Since the progress bar is incredibly customizable you can specify 59 | different widgets of any type in any order. You can even write your own 60 | widgets! However, since there are already a good number of widgets you 61 | should probably play around with them before moving on to create your own 62 | widgets. 63 | 64 | The term_width parameter represents the current terminal width. If the 65 | parameter is set to an integer then the progress bar will use that, 66 | otherwise it will attempt to determine the terminal width falling back to 67 | 80 columns if the width cannot be determined. 68 | 69 | When implementing a widget's update method you are passed a reference to 70 | the current progress bar. As a result, you have access to the 71 | ProgressBar's methods and attributes. Although there is nothing preventing 72 | you from changing the ProgressBar you should treat it as read only. 73 | 74 | Useful methods and attributes include (Public API): 75 | - currval: current progress (0 <= currval <= maxval) 76 | - maxval: maximum (and final) value 77 | - finished: True if the bar has finished (reached 100%) 78 | - start_time: the time when start() method of ProgressBar was called 79 | - seconds_elapsed: seconds elapsed since start_time and last call to 80 | update 81 | - percentage(): progress in percent [0..100] 82 | """ 83 | 84 | __slots__ = ('currval', 'fd', 'finished', 'last_update_time', 85 | 'left_justify', 'maxval', 'next_update', 'num_intervals', 86 | 'poll', 'seconds_elapsed', 'signal_set', 'start_time', 87 | 'term_width', 'update_interval', 'widgets', '_time_sensitive', 88 | '__iterable') 89 | 90 | _DEFAULT_MAXVAL = 100 91 | _DEFAULT_TERMSIZE = 80 92 | _DEFAULT_WIDGETS = [widgets.Percentage(), ' ', widgets.Bar()] 93 | 94 | def __init__(self, maxval=None, widgets=None, term_width=None, poll=1, 95 | left_justify=True, fd=None): 96 | """Initializes a progress bar with sane defaults.""" 97 | 98 | # Don't share a reference with any other progress bars 99 | if widgets is None: 100 | widgets = list(self._DEFAULT_WIDGETS) 101 | 102 | self.maxval = maxval 103 | self.widgets = widgets 104 | self.fd = fd if fd is not None else sys.stderr 105 | self.left_justify = left_justify 106 | 107 | self.signal_set = False 108 | if term_width is not None: 109 | self.term_width = term_width 110 | else: 111 | try: 112 | self._handle_resize() 113 | signal.signal(signal.SIGWINCH, self._handle_resize) 114 | self.signal_set = True 115 | except (SystemExit, KeyboardInterrupt): raise 116 | except: 117 | self.term_width = self._env_size() 118 | 119 | self.__iterable = None 120 | self._update_widgets() 121 | self.currval = 0 122 | self.finished = False 123 | self.last_update_time = None 124 | self.poll = poll 125 | self.seconds_elapsed = 0 126 | self.start_time = None 127 | self.update_interval = 1 128 | self.next_update = 0 129 | 130 | 131 | def __call__(self, iterable): 132 | """Use a ProgressBar to iterate through an iterable.""" 133 | 134 | try: 135 | self.maxval = len(iterable) 136 | except: 137 | if self.maxval is None: 138 | self.maxval = widgets.UnknownLength 139 | 140 | self.__iterable = iter(iterable) 141 | return self 142 | 143 | 144 | def __iter__(self): 145 | return self 146 | 147 | 148 | def __next__(self): 149 | try: 150 | value = next(self.__iterable) 151 | if self.start_time is None: 152 | self.start() 153 | else: 154 | self.update(self.currval + 1) 155 | return value 156 | except StopIteration: 157 | if self.start_time is None: 158 | self.start() 159 | self.finish() 160 | raise 161 | 162 | 163 | # Create an alias so that Python 2.x won't complain about not being 164 | # an iterator. 165 | next = __next__ 166 | 167 | 168 | def _env_size(self): 169 | """Tries to find the term_width from the environment.""" 170 | 171 | return int(os.environ.get('COLUMNS', self._DEFAULT_TERMSIZE)) - 1 172 | 173 | 174 | def _handle_resize(self, signum=None, frame=None): 175 | """Tries to catch resize signals sent from the terminal.""" 176 | 177 | h, w = array('h', ioctl(self.fd, termios.TIOCGWINSZ, '\0' * 8))[:2] 178 | self.term_width = w 179 | 180 | 181 | def percentage(self): 182 | """Returns the progress as a percentage.""" 183 | if self.maxval is widgets.UnknownLength: 184 | return float("NaN") 185 | if self.currval >= self.maxval: 186 | return 100.0 187 | return (self.currval * 100.0 / self.maxval) if self.maxval else 100.00 188 | 189 | percent = property(percentage) 190 | 191 | 192 | def _format_widgets(self): 193 | result = [] 194 | expanding = [] 195 | width = self.term_width 196 | 197 | for index, widget in enumerate(self.widgets): 198 | if isinstance(widget, widgets.WidgetHFill): 199 | result.append(widget) 200 | expanding.insert(0, index) 201 | else: 202 | widget = widgets.format_updatable(widget, self) 203 | result.append(widget) 204 | width -= len(widget) 205 | 206 | count = len(expanding) 207 | while count: 208 | portion = max(int(math.ceil(width * 1. / count)), 0) 209 | index = expanding.pop() 210 | count -= 1 211 | 212 | widget = result[index].update(self, portion) 213 | width -= len(widget) 214 | result[index] = widget 215 | 216 | return result 217 | 218 | 219 | def _format_line(self): 220 | """Joins the widgets and justifies the line.""" 221 | 222 | widgets = ''.join(self._format_widgets()) 223 | 224 | if self.left_justify: return widgets.ljust(self.term_width) 225 | else: return widgets.rjust(self.term_width) 226 | 227 | 228 | def _need_update(self): 229 | """Returns whether the ProgressBar should redraw the line.""" 230 | if self.currval >= self.next_update or self.finished: return True 231 | 232 | delta = time.time() - self.last_update_time 233 | return self._time_sensitive and delta > self.poll 234 | 235 | 236 | def _update_widgets(self): 237 | """Checks all widgets for the time sensitive bit.""" 238 | 239 | self._time_sensitive = any(getattr(w, 'TIME_SENSITIVE', False) 240 | for w in self.widgets) 241 | 242 | 243 | def update(self, value=None): 244 | """Updates the ProgressBar to a new value.""" 245 | 246 | if value is not None and value is not widgets.UnknownLength: 247 | if (self.maxval is not widgets.UnknownLength 248 | and not 0 <= value <= self.maxval): 249 | 250 | raise ValueError('Value out of range') 251 | 252 | self.currval = value 253 | 254 | 255 | if not self._need_update(): return 256 | if self.start_time is None: 257 | raise RuntimeError('You must call "start" before calling "update"') 258 | 259 | now = time.time() 260 | self.seconds_elapsed = now - self.start_time 261 | self.next_update = self.currval + self.update_interval 262 | self.fd.write(self._format_line() + '\r') 263 | self.fd.flush() 264 | self.last_update_time = now 265 | 266 | 267 | def start(self): 268 | """Starts measuring time, and prints the bar at 0%. 269 | 270 | It returns self so you can use it like this: 271 | >>> pbar = ProgressBar().start() 272 | >>> for i in range(100): 273 | ... # do something 274 | ... pbar.update(i+1) 275 | ... 276 | >>> pbar.finish() 277 | """ 278 | 279 | if self.maxval is None: 280 | self.maxval = self._DEFAULT_MAXVAL 281 | 282 | self.num_intervals = max(100, self.term_width) 283 | self.next_update = 0 284 | 285 | if self.maxval is not widgets.UnknownLength: 286 | if self.maxval < 0: raise ValueError('Value out of range') 287 | self.update_interval = self.maxval / self.num_intervals 288 | 289 | 290 | self.start_time = self.last_update_time = time.time() 291 | self.update(0) 292 | 293 | return self 294 | 295 | 296 | def finish(self): 297 | """Puts the ProgressBar bar in the finished state.""" 298 | 299 | if self.finished: 300 | return 301 | self.finished = True 302 | self.update(self.maxval) 303 | self.fd.write('\n') 304 | if self.signal_set: 305 | signal.signal(signal.SIGWINCH, signal.SIG_DFL) 306 | -------------------------------------------------------------------------------- /progressbar/widgets.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # progressbar - Text progress bar library for Python. 4 | # Copyright (c) 2005 Nilton Volpato 5 | # 6 | # This library is free software; you can redistribute it and/or 7 | # modify it under the terms of the GNU Lesser General Public 8 | # License as published by the Free Software Foundation; either 9 | # version 2.1 of the License, or (at your option) any later version. 10 | # 11 | # This library is distributed in the hope that it will be useful, 12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 14 | # Lesser General Public License for more details. 15 | # 16 | # You should have received a copy of the GNU Lesser General Public 17 | # License along with this library; if not, write to the Free Software 18 | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 19 | 20 | """Default ProgressBar widgets.""" 21 | 22 | from __future__ import division 23 | 24 | import datetime 25 | import math 26 | 27 | try: 28 | from abc import ABCMeta, abstractmethod 29 | except ImportError: 30 | AbstractWidget = object 31 | abstractmethod = lambda fn: fn 32 | else: 33 | AbstractWidget = ABCMeta('AbstractWidget', (object,), {}) 34 | 35 | class UnknownLength: 36 | pass 37 | 38 | def format_updatable(updatable, pbar): 39 | if hasattr(updatable, 'update'): return updatable.update(pbar) 40 | else: return updatable 41 | 42 | 43 | class Widget(AbstractWidget): 44 | """The base class for all widgets. 45 | 46 | The ProgressBar will call the widget's update value when the widget should 47 | be updated. The widget's size may change between calls, but the widget may 48 | display incorrectly if the size changes drastically and repeatedly. 49 | 50 | The boolean TIME_SENSITIVE informs the ProgressBar that it should be 51 | updated more often because it is time sensitive. 52 | """ 53 | 54 | TIME_SENSITIVE = False 55 | __slots__ = () 56 | 57 | @abstractmethod 58 | def update(self, pbar): 59 | """Updates the widget. 60 | 61 | pbar - a reference to the calling ProgressBar 62 | """ 63 | 64 | 65 | class WidgetHFill(Widget): 66 | """The base class for all variable width widgets. 67 | 68 | This widget is much like the \\hfill command in TeX, it will expand to 69 | fill the line. You can use more than one in the same line, and they will 70 | all have the same width, and together will fill the line. 71 | """ 72 | 73 | @abstractmethod 74 | def update(self, pbar, width): 75 | """Updates the widget providing the total width the widget must fill. 76 | 77 | pbar - a reference to the calling ProgressBar 78 | width - The total width the widget must fill 79 | """ 80 | 81 | 82 | class Timer(Widget): 83 | """Widget which displays the elapsed seconds.""" 84 | 85 | __slots__ = ('format_string',) 86 | TIME_SENSITIVE = True 87 | 88 | def __init__(self, format='Elapsed Time: %s'): 89 | self.format_string = format 90 | 91 | @staticmethod 92 | def format_time(seconds): 93 | """Formats time as the string "HH:MM:SS".""" 94 | 95 | return str(datetime.timedelta(seconds=int(seconds))) 96 | 97 | 98 | def update(self, pbar): 99 | """Updates the widget to show the elapsed time.""" 100 | 101 | return self.format_string % self.format_time(pbar.seconds_elapsed) 102 | 103 | 104 | class ETA(Timer): 105 | """Widget which attempts to estimate the time of arrival.""" 106 | 107 | TIME_SENSITIVE = True 108 | 109 | def update(self, pbar): 110 | """Updates the widget to show the ETA or total time when finished.""" 111 | 112 | if pbar.maxval is UnknownLength or pbar.currval == 0: 113 | return 'ETA: --:--:--' 114 | elif pbar.finished: 115 | return 'Time: %s' % self.format_time(pbar.seconds_elapsed) 116 | else: 117 | elapsed = pbar.seconds_elapsed 118 | eta = elapsed * pbar.maxval / pbar.currval - elapsed 119 | return 'ETA: %s' % self.format_time(eta) 120 | 121 | 122 | class AdaptiveETA(Timer): 123 | """Widget which attempts to estimate the time of arrival. 124 | 125 | Uses a weighted average of two estimates: 126 | 1) ETA based on the total progress and time elapsed so far 127 | 2) ETA based on the progress as per the last 10 update reports 128 | 129 | The weight depends on the current progress so that to begin with the 130 | total progress is used and at the end only the most recent progress is 131 | used. 132 | """ 133 | 134 | TIME_SENSITIVE = True 135 | NUM_SAMPLES = 10 136 | 137 | def _update_samples(self, currval, elapsed): 138 | sample = (currval, elapsed) 139 | if not hasattr(self, 'samples'): 140 | self.samples = [sample] * (self.NUM_SAMPLES + 1) 141 | else: 142 | self.samples.append(sample) 143 | return self.samples.pop(0) 144 | 145 | def _eta(self, maxval, currval, elapsed): 146 | return elapsed * maxval / float(currval) - elapsed 147 | 148 | def update(self, pbar): 149 | """Updates the widget to show the ETA or total time when finished.""" 150 | if pbar.maxval is UnknownLength or pbar.currval == 0: 151 | return 'ETA: --:--:--' 152 | elif pbar.finished: 153 | return 'Time: %s' % self.format_time(pbar.seconds_elapsed) 154 | else: 155 | elapsed = pbar.seconds_elapsed 156 | currval1, elapsed1 = self._update_samples(pbar.currval, elapsed) 157 | eta = self._eta(pbar.maxval, pbar.currval, elapsed) 158 | if pbar.currval > currval1: 159 | etasamp = self._eta(pbar.maxval - currval1, 160 | pbar.currval - currval1, 161 | elapsed - elapsed1) 162 | weight = (pbar.currval / float(pbar.maxval)) ** 0.5 163 | eta = (1 - weight) * eta + weight * etasamp 164 | return 'ETA: %s' % self.format_time(eta) 165 | 166 | 167 | class FileTransferSpeed(Widget): 168 | """Widget for showing the transfer speed (useful for file transfers).""" 169 | 170 | FMT = '%6.2f %s%s/s' 171 | PREFIXES = ' kMGTPEZY' 172 | __slots__ = ('unit',) 173 | 174 | def __init__(self, unit='B'): 175 | self.unit = unit 176 | 177 | def update(self, pbar): 178 | """Updates the widget with the current SI prefixed speed.""" 179 | 180 | if pbar.seconds_elapsed < 2e-6 or pbar.currval < 2e-6: # =~ 0 181 | scaled = power = 0 182 | else: 183 | speed = pbar.currval / pbar.seconds_elapsed 184 | power = int(math.log(speed, 1000)) 185 | scaled = speed / 1000.**power 186 | 187 | return self.FMT % (scaled, self.PREFIXES[power], self.unit) 188 | 189 | 190 | class AnimatedMarker(Widget): 191 | """An animated marker for the progress bar which defaults to appear as if 192 | it were rotating. 193 | """ 194 | 195 | __slots__ = ('markers', 'curmark') 196 | 197 | def __init__(self, markers='|/-\\'): 198 | self.markers = markers 199 | self.curmark = -1 200 | 201 | def update(self, pbar): 202 | """Updates the widget to show the next marker or the first marker when 203 | finished""" 204 | 205 | if pbar.finished: return self.markers[0] 206 | 207 | self.curmark = (self.curmark + 1) % len(self.markers) 208 | return self.markers[self.curmark] 209 | 210 | # Alias for backwards compatibility 211 | RotatingMarker = AnimatedMarker 212 | 213 | 214 | class Counter(Widget): 215 | """Displays the current count.""" 216 | 217 | __slots__ = ('format_string',) 218 | 219 | def __init__(self, format='%d'): 220 | self.format_string = format 221 | 222 | def update(self, pbar): 223 | return self.format_string % pbar.currval 224 | 225 | 226 | class Percentage(Widget): 227 | """Displays the current percentage as a number with a percent sign.""" 228 | 229 | def update(self, pbar): 230 | return '%3.0f%%' % pbar.percentage() 231 | 232 | 233 | class FormatLabel(Timer): 234 | """Displays a formatted label.""" 235 | 236 | mapping = { 237 | 'elapsed': ('seconds_elapsed', Timer.format_time), 238 | 'finished': ('finished', None), 239 | 'last_update': ('last_update_time', None), 240 | 'max': ('maxval', None), 241 | 'seconds': ('seconds_elapsed', None), 242 | 'start': ('start_time', None), 243 | 'value': ('currval', None) 244 | } 245 | 246 | __slots__ = ('format_string',) 247 | def __init__(self, format): 248 | self.format_string = format 249 | 250 | def update(self, pbar): 251 | context = {} 252 | for name, (key, transform) in self.mapping.items(): 253 | try: 254 | value = getattr(pbar, key) 255 | 256 | if transform is None: 257 | context[name] = value 258 | else: 259 | context[name] = transform(value) 260 | except: pass 261 | 262 | return self.format_string % context 263 | 264 | 265 | class SimpleProgress(Widget): 266 | """Returns progress as a count of the total (e.g.: "5 of 47").""" 267 | 268 | __slots__ = ('sep',) 269 | 270 | def __init__(self, sep=' of '): 271 | self.sep = sep 272 | 273 | def update(self, pbar): 274 | if pbar.maxval is UnknownLength: 275 | return '%d%s?' % (pbar.currval, self.sep) 276 | return '%d%s%s' % (pbar.currval, self.sep, pbar.maxval) 277 | 278 | 279 | class Bar(WidgetHFill): 280 | """A progress bar which stretches to fill the line.""" 281 | 282 | __slots__ = ('marker', 'left', 'right', 'fill', 'fill_left') 283 | 284 | def __init__(self, marker='#', left='|', right='|', fill=' ', 285 | fill_left=True): 286 | """Creates a customizable progress bar. 287 | 288 | marker - string or updatable object to use as a marker 289 | left - string or updatable object to use as a left border 290 | right - string or updatable object to use as a right border 291 | fill - character to use for the empty part of the progress bar 292 | fill_left - whether to fill from the left or the right 293 | """ 294 | self.marker = marker 295 | self.left = left 296 | self.right = right 297 | self.fill = fill 298 | self.fill_left = fill_left 299 | 300 | 301 | def update(self, pbar, width): 302 | """Updates the progress bar and its subcomponents.""" 303 | 304 | left, marked, right = (format_updatable(i, pbar) for i in 305 | (self.left, self.marker, self.right)) 306 | 307 | width -= len(left) + len(right) 308 | # Marked must *always* have length of 1 309 | if pbar.maxval is not UnknownLength and pbar.maxval: 310 | marked *= int(pbar.currval / pbar.maxval * width) 311 | else: 312 | marked = '' 313 | 314 | if self.fill_left: 315 | return '%s%s%s' % (left, marked.ljust(width, self.fill), right) 316 | else: 317 | return '%s%s%s' % (left, marked.rjust(width, self.fill), right) 318 | 319 | 320 | class ReverseBar(Bar): 321 | """A bar which has a marker which bounces from side to side.""" 322 | 323 | def __init__(self, marker='#', left='|', right='|', fill=' ', 324 | fill_left=False): 325 | """Creates a customizable progress bar. 326 | 327 | marker - string or updatable object to use as a marker 328 | left - string or updatable object to use as a left border 329 | right - string or updatable object to use as a right border 330 | fill - character to use for the empty part of the progress bar 331 | fill_left - whether to fill from the left or the right 332 | """ 333 | self.marker = marker 334 | self.left = left 335 | self.right = right 336 | self.fill = fill 337 | self.fill_left = fill_left 338 | 339 | 340 | class BouncingBar(Bar): 341 | def update(self, pbar, width): 342 | """Updates the progress bar and its subcomponents.""" 343 | 344 | left, marker, right = (format_updatable(i, pbar) for i in 345 | (self.left, self.marker, self.right)) 346 | 347 | width -= len(left) + len(right) 348 | 349 | if pbar.finished: return '%s%s%s' % (left, width * marker, right) 350 | 351 | position = int(pbar.currval % (width * 2 - 1)) 352 | if position > width: position = width * 2 - position 353 | lpad = self.fill * (position - 1) 354 | rpad = self.fill * (width - len(marker) - len(lpad)) 355 | 356 | # Swap if we want to bounce the other way 357 | if not self.fill_left: rpad, lpad = lpad, rpad 358 | 359 | return '%s%s%s%s%s' % (left, lpad, marker, rpad, right) 360 | -------------------------------------------------------------------------------- /regionfixer_core/bug_reporter.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Created on 16/09/2014 3 | 4 | @author: Alejandro 5 | ''' 6 | 7 | import sys 8 | import ftplib 9 | import datetime 10 | from io import StringIO 11 | from .util import query_yes_no, get_str_from_traceback 12 | 13 | 14 | SERVER = 'regionfixer.no-ip.org' 15 | USER = 'regionfixer_bugreporter' 16 | PASSWORD = 'supersecretpassword' 17 | BUGREPORTS_DIR = 'bugreports' 18 | 19 | 20 | class BugReporter(object): 21 | ''' 22 | Class to report bugs to region fixer ftp. 23 | 24 | You can init it without arguments and it will extract the traceback 25 | directly from sys.exc_info(). The traceback will be formated and 26 | uploaded as a text file. 27 | Or you can init it using an error string (error_str). The string 28 | will be uploaded as a text file. 29 | ''' 30 | 31 | def __init__(self, error_str=None, server=SERVER, 32 | user=USER, password=PASSWORD): 33 | ''' 34 | Constructor 35 | ''' 36 | if error_str: 37 | self.error_file_obj = self._get_fileobj_from_str(error_str) 38 | else: 39 | (ty, value, tb) = sys.exc_info() 40 | self.error_file_obj = self._get_fileobj_from_tb(ty, value, tb) 41 | self.server = server 42 | self.user = user 43 | self.password = password 44 | 45 | self._exception = None 46 | 47 | def _get_fileobj_from_tb(self, ty, value, tb): 48 | ''' Return a file obj from a traceback object. ''' 49 | f = StringIO(get_str_from_traceback(ty, value, tb)) 50 | f.seek(0) 51 | return f 52 | 53 | def _get_fileobj_from_str(self, error_str): 54 | ''' Return a file object from a string. ''' 55 | f = StringIO(error_str) 56 | f.seek(0) 57 | return f 58 | 59 | @property 60 | def error_str(self): 61 | ''' Return the string that is currently ready for upload. ''' 62 | self.error_file_obj.seek(0) 63 | s = self.error_file_obj.read() 64 | self.error_file_obj.seek(0) 65 | return s 66 | 67 | @property 68 | def exception_str(self): 69 | ''' Return the exception caused by uploading the file. ''' 70 | return self._exception.message 71 | 72 | def ask_and_send(self, question_text): 73 | ''' Query the user yes/no to send the file and send it. ''' 74 | if query_yes_no(question_text): 75 | return self.send() 76 | 77 | def send(self): 78 | ''' Send the file to the ftp. 79 | 80 | If an exception is thrown, you can retrieve it at 81 | exception_str. 82 | ''' 83 | try: 84 | s = ftplib.FTP(self.server, self.user, 85 | self.password) 86 | 87 | s.cwd(BUGREPORTS_DIR) 88 | 89 | error_name = str(datetime.datetime.now()) 90 | 91 | s.storlines("STOR " + error_name, self.error_file_obj) 92 | s.quit() 93 | return True 94 | except Exception as e: 95 | # TODO: prints shouldn't be here! 96 | print("Couldn't send the bug report!") 97 | self._exception = e 98 | print(e) 99 | return False 100 | -------------------------------------------------------------------------------- /regionfixer_core/constants.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | # 5 | # Region Fixer. 6 | # Fix your region files with a backup copy of your Minecraft world. 7 | # Copyright (C) 2020 Alejandro Aguilera (Fenixin) 8 | # https://github.com/Fenixin/Minecraft-Region-Fixer 9 | # 10 | # This program is free software: you can redistribute it and/or modify 11 | # it under the terms of the GNU General Public License as published by 12 | # the Free Software Foundation, either version 3 of the License, or 13 | # (at your option) any later version. 14 | # 15 | # This program is distributed in the hope that it will be useful, 16 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 17 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 18 | # GNU General Public License for more details. 19 | # 20 | # You should have received a copy of the GNU General Public License 21 | # along with this program. If not, see . 22 | # 23 | 24 | 25 | 26 | ################ 27 | # Return values 28 | ################ 29 | 30 | RV_OK = 0 # world scanned and no problems found 31 | RV_CRASH = 1 # crash or end unexpectedly 32 | RV_NOTHING_TO_SCAN = 20 # no files/worlds to scan 33 | # RV_WRONG_COMMAND = 2 # the command line used is wrong and region fixer didn't execute. argparse uses this value by default 34 | RV_BAD_WORLD = 3 # scan completed successfully but problems have been found in the scan 35 | 36 | 37 | 38 | 39 | # -------------- 40 | # Chunk related: 41 | # -------------- 42 | # Used to mark the status of chunks: 43 | CHUNK_NOT_CREATED = -1 44 | CHUNK_OK = 0 45 | CHUNK_CORRUPTED = 1 46 | CHUNK_WRONG_LOCATED = 2 47 | CHUNK_TOO_MANY_ENTITIES = 3 48 | CHUNK_SHARED_OFFSET = 4 49 | CHUNK_MISSING_ENTITIES_TAG = 5 50 | 51 | # Chunk statuses 52 | CHUNK_STATUSES = [CHUNK_NOT_CREATED, 53 | CHUNK_OK, 54 | CHUNK_CORRUPTED, 55 | CHUNK_WRONG_LOCATED, 56 | CHUNK_TOO_MANY_ENTITIES, 57 | CHUNK_SHARED_OFFSET, 58 | CHUNK_MISSING_ENTITIES_TAG] 59 | 60 | # Status that are considered problems 61 | CHUNK_PROBLEMS = [CHUNK_CORRUPTED, 62 | CHUNK_WRONG_LOCATED, 63 | CHUNK_TOO_MANY_ENTITIES, 64 | CHUNK_SHARED_OFFSET, 65 | CHUNK_MISSING_ENTITIES_TAG] 66 | 67 | # Text describing each chunk status 68 | CHUNK_STATUS_TEXT = {CHUNK_NOT_CREATED: "Not created", 69 | CHUNK_OK: "OK", 70 | CHUNK_CORRUPTED: "Corrupted", 71 | CHUNK_WRONG_LOCATED: "Wrong located", 72 | CHUNK_TOO_MANY_ENTITIES: "Too many entities", 73 | CHUNK_SHARED_OFFSET: "Sharing offset", 74 | CHUNK_MISSING_ENTITIES_TAG: "Missing Entities tag" 75 | } 76 | 77 | # arguments used in the options 78 | CHUNK_PROBLEMS_ARGS = {CHUNK_CORRUPTED: 'corrupted', 79 | CHUNK_WRONG_LOCATED: 'wrong-located', 80 | CHUNK_TOO_MANY_ENTITIES: 'entities', 81 | CHUNK_SHARED_OFFSET: 'shared-offset', 82 | CHUNK_MISSING_ENTITIES_TAG: 'missing_tag' 83 | } 84 | 85 | # used in some places where there is less space 86 | CHUNK_PROBLEMS_ABBR = {CHUNK_CORRUPTED: 'c', 87 | CHUNK_WRONG_LOCATED: 'w', 88 | CHUNK_TOO_MANY_ENTITIES: 'tme', 89 | CHUNK_SHARED_OFFSET: 'so', 90 | CHUNK_MISSING_ENTITIES_TAG: 'mt' 91 | } 92 | 93 | # Dictionary with possible solutions for the chunks problems, 94 | # used to create options dynamically 95 | # The possible solutions right now are: 96 | CHUNK_SOLUTION_REMOVE = 51 97 | CHUNK_SOLUTION_REPLACE = 52 98 | CHUNK_SOLUTION_REMOVE_ENTITIES = 53 99 | CHUNK_SOLUTION_RELOCATE_USING_DATA = 54 100 | 101 | CHUNK_PROBLEMS_SOLUTIONS = {CHUNK_CORRUPTED: [CHUNK_SOLUTION_REMOVE, CHUNK_SOLUTION_REPLACE], 102 | CHUNK_WRONG_LOCATED: [CHUNK_SOLUTION_REMOVE, CHUNK_SOLUTION_REPLACE, CHUNK_SOLUTION_RELOCATE_USING_DATA], 103 | CHUNK_TOO_MANY_ENTITIES: [CHUNK_SOLUTION_REMOVE_ENTITIES, CHUNK_SOLUTION_REPLACE], 104 | CHUNK_SHARED_OFFSET: [CHUNK_SOLUTION_REMOVE, CHUNK_SOLUTION_REPLACE], 105 | CHUNK_MISSING_ENTITIES_TAG: [CHUNK_SOLUTION_REMOVE, CHUNK_SOLUTION_REPLACE]} 106 | 107 | # chunk problems that can be fixed (so they don't need to be removed or replaced) 108 | FIXABLE_CHUNK_PROBLEMS = [CHUNK_CORRUPTED, CHUNK_MISSING_ENTITIES_TAG, CHUNK_WRONG_LOCATED] 109 | 110 | # list with problem, status-text, problem arg tuples 111 | CHUNK_PROBLEMS_ITERATOR = [] 112 | for problem in CHUNK_PROBLEMS: 113 | CHUNK_PROBLEMS_ITERATOR.append((problem, 114 | CHUNK_STATUS_TEXT[problem], 115 | CHUNK_PROBLEMS_ARGS[problem])) 116 | 117 | # Used to know where to look in a chunk status tuple 118 | TUPLE_NUM_ENTITIES = 0 119 | TUPLE_STATUS = 1 120 | 121 | 122 | 123 | 124 | # --------------- 125 | # Region related: 126 | # --------------- 127 | # Used to mark the status of region files: 128 | REGION_OK = 100 129 | REGION_TOO_SMALL = 101 130 | REGION_UNREADABLE = 102 131 | REGION_UNREADABLE_PERMISSION_ERROR = 103 132 | 133 | # Region statuses 134 | REGION_STATUSES = [REGION_OK, 135 | REGION_TOO_SMALL, 136 | REGION_UNREADABLE, 137 | REGION_UNREADABLE_PERMISSION_ERROR] 138 | 139 | # Text describing each region status used to list all the problem at the end of the scan 140 | REGION_STATUS_TEXT = {REGION_OK: "OK", 141 | REGION_TOO_SMALL: "Too small", 142 | REGION_UNREADABLE: "Unreadable IOError", 143 | # This status differentiates IOError from a file that you don't have permission to access 144 | # TODO: It would be better to open region files only in write mode when needed 145 | REGION_UNREADABLE_PERMISSION_ERROR: "Permission error" 146 | } 147 | 148 | # Status that are considered problems 149 | REGION_PROBLEMS = [REGION_TOO_SMALL, 150 | REGION_UNREADABLE, 151 | REGION_UNREADABLE_PERMISSION_ERROR] 152 | 153 | # arguments used in the options 154 | REGION_PROBLEMS_ARGS = {REGION_TOO_SMALL: 'too_small', 155 | REGION_UNREADABLE: 'unreadable', 156 | REGION_UNREADABLE_PERMISSION_ERROR: 'permission_error' 157 | } 158 | 159 | # used in some places where there is less space 160 | REGION_PROBLEMS_ABBR = {REGION_TOO_SMALL: 'ts', 161 | REGION_UNREADABLE: 'ur', 162 | REGION_UNREADABLE_PERMISSION_ERROR: 'pe' 163 | } 164 | 165 | # Dictionary with possible solutions for the region problems, 166 | # used to create options dynamically 167 | # The possible solutions right now are: 168 | REGION_SOLUTION_REMOVE = 151 169 | REGION_SOLUTION_REPLACE = 152 170 | 171 | REGION_PROBLEMS_SOLUTIONS = {REGION_TOO_SMALL: [REGION_SOLUTION_REMOVE, REGION_SOLUTION_REPLACE]} 172 | 173 | # list with problem, status-text, problem arg tuples 174 | REGION_PROBLEMS_ITERATOR = [] 175 | for problem in REGION_PROBLEMS: 176 | try: 177 | REGION_PROBLEMS_ITERATOR.append((problem, 178 | REGION_STATUS_TEXT[problem], 179 | REGION_PROBLEMS_ARGS[problem])) 180 | except KeyError: 181 | pass 182 | 183 | 184 | 185 | # ------------------ 186 | # Data file related: 187 | # ------------------ 188 | # Used to mark the status of data files: 189 | DATAFILE_OK = 200 190 | DATAFILE_UNREADABLE = 201 191 | 192 | # Data files statuses 193 | DATAFILE_STATUSES = [DATAFILE_OK, 194 | DATAFILE_UNREADABLE] 195 | 196 | # Status that are considered problems 197 | DATAFILE_PROBLEMS = [DATAFILE_UNREADABLE] 198 | 199 | # Text describing each chunk status 200 | DATAFILE_STATUS_TEXT = {DATAFILE_OK: "OK", 201 | DATAFILE_UNREADABLE: "The data file cannot be read" 202 | } 203 | 204 | # arguments used in the options 205 | DATAFILE_PROBLEMS_ARGS = {DATAFILE_OK: 'OK', 206 | DATAFILE_UNREADABLE: 'unreadable' 207 | } 208 | 209 | # used in some places where there is less space 210 | DATAFILE_PROBLEM_ABBR = {DATAFILE_OK: 'ok', 211 | DATAFILE_UNREADABLE: 'ur' 212 | } 213 | 214 | # Dictionary with possible solutions for the chunks problems, 215 | # used to create options dynamically 216 | # The possible solutions right now are: 217 | DATAFILE_SOLUTION_REMOVE = 251 218 | 219 | DATAFILE_PROBLEMS_SOLUTIONS = {DATAFILE_UNREADABLE: [DATAFILE_SOLUTION_REMOVE]} 220 | 221 | # list with problem, status-text, problem arg tuples 222 | DATAFILE_PROBLEMS_ITERATOR = [] 223 | for problem in DATAFILE_PROBLEMS: 224 | DATAFILE_PROBLEMS_ITERATOR.append((problem, 225 | DATAFILE_STATUS_TEXT[problem], 226 | DATAFILE_PROBLEMS_ARGS[problem])) 227 | 228 | CHUNK_PROBLEMS_ITERATOR = [] 229 | for problem in CHUNK_PROBLEMS: 230 | CHUNK_PROBLEMS_ITERATOR.append((problem, 231 | CHUNK_STATUS_TEXT[problem], 232 | CHUNK_PROBLEMS_ARGS[problem])) 233 | 234 | # Dimension names: 235 | DIMENSION_NAMES = {"": "Overworld", 236 | "DIM1": "The End", 237 | "DIM-1": "Nether" 238 | } 239 | 240 | # Region files types 241 | LEVEL_DIR = "region" 242 | POI_DIR = "poi" 243 | ENTITIES_DIR = "entities" 244 | REGION_TYPES_NAMES = {LEVEL_DIR: ("level/region", "Level/Region"), 245 | POI_DIR: ("POIs", "POIs"), 246 | ENTITIES_DIR: ("entities", "Entities" ) 247 | } 248 | -------------------------------------------------------------------------------- /regionfixer_core/interactive.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | # 5 | # Region Fixer. 6 | # Fix your region files with a backup copy of your Minecraft world. 7 | # Copyright (C) 2020 Alejandro Aguilera (Fenixin) 8 | # https://github.com/Fenixin/Minecraft-Region-Fixer 9 | # 10 | # This program is free software: you can redistribute it and/or modify 11 | # it under the terms of the GNU General Public License as published by 12 | # the Free Software Foundation, either version 3 of the License, or 13 | # (at your option) any later version. 14 | # 15 | # This program is distributed in the hope that it will be useful, 16 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 17 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 18 | # GNU General Public License for more details. 19 | # 20 | # You should have received a copy of the GNU General Public License 21 | # along with this program. If not, see . 22 | # 23 | 24 | 25 | from cmd import Cmd 26 | 27 | import regionfixer_core.constants as c 28 | from regionfixer_core import world 29 | from regionfixer_core.scan import console_scan_world, console_scan_regionset 30 | 31 | 32 | class InteractiveLoop(Cmd): 33 | def __init__(self, world_list, regionset, options, backup_worlds): 34 | Cmd.__init__(self) 35 | self.world_list = world_list 36 | self.regionset = regionset 37 | self.world_names = [str(i.name) for i in self.world_list] 38 | # if there's only one world use it 39 | if len(self.world_list) == 1 and len(self.regionset) == 0: 40 | self.current = world_list[0] 41 | elif len(self.world_list) == 0 and len(self.regionset) > 0: 42 | self.current = self.regionset 43 | else: 44 | self.current = None 45 | self.options = options 46 | self.backup_worlds = backup_worlds 47 | self.prompt = "#-> " 48 | self.intro = ("Minecraft Region-Fixer interactive mode.\n(Use tab to " 49 | "autocomplete. Type help for a list of commands.)\n") 50 | 51 | # Possible args for chunks stuff 52 | possible_args = "" 53 | first = True 54 | for i in list(c.CHUNK_PROBLEMS_ARGS.values()) + ['all']: 55 | if not first: 56 | possible_args += ", " 57 | possible_args += i 58 | first = False 59 | self.possible_chunk_args_text = possible_args 60 | 61 | # Possible args for region stuff 62 | possible_args = "" 63 | first = True 64 | for i in list(c.REGION_PROBLEMS_ARGS.values()) + ['all']: 65 | if not first: 66 | possible_args += ", " 67 | possible_args += i 68 | first = False 69 | self.possible_region_args_text = possible_args 70 | 71 | ################################################# 72 | # Do methods 73 | ################################################# 74 | def do_set(self, arg): 75 | """ Command to change some options and variables in interactive 76 | mode """ 77 | args = arg.split() 78 | if len(args) > 2: 79 | print("Error: too many parameters.") 80 | elif len(args) == 0: 81 | print("Write \'help set\' to see a list of all possible variables") 82 | else: 83 | if args[0] == "entity-limit": 84 | if len(args) == 1: 85 | print("entity-limit = {0}".format(self.options.entity_limit)) 86 | else: 87 | try: 88 | if int(args[1]) >= 0: 89 | self.options.entity_limit = int(args[1]) 90 | print("entity-limit = {0}".format(args[1])) 91 | print("Updating chunk status...") 92 | self.current.rescan_entities(self.options) 93 | else: 94 | print("Invalid value. Valid values are positive integers and zero") 95 | except ValueError: 96 | print("Invalid value. Valid values are positive integers and zero") 97 | 98 | elif args[0] == "workload": 99 | 100 | if len(args) == 1: 101 | if self.current: 102 | print("Current workload:\n{0}\n".format(self.current.__str__())) 103 | print("List of possible worlds and region-sets (determined by the command used to run region-fixer):") 104 | number = 1 105 | for w in self.world_list: 106 | print(" ### world{0} ###".format(number)) 107 | number += 1 108 | # add a tab and print 109 | for i in w.__str__().split("\n"): 110 | print("\t" + i) 111 | print() 112 | print(" ### regionset ###") 113 | for i in self.regionset.__str__().split("\n"): 114 | print("\t" + i) 115 | print("\n(Use \"set workload world1\" or name_of_the_world or regionset to choose one)") 116 | 117 | else: 118 | a = args[1] 119 | if len(a) == 6 and a[:5] == "world" and int(a[-1]) >= 1: 120 | # get the number and choos the correct world from the list 121 | number = int(args[1][-1]) - 1 122 | try: 123 | self.current = self.world_list[number] 124 | print("workload = {0}".format(self.current.world_path)) 125 | except IndexError: 126 | print("This world is not in the list!") 127 | elif a in self.world_names: 128 | for w in self.world_list: 129 | if w.name == args[1]: 130 | self.current = w 131 | print("workload = {0}".format(self.current.world_path)) 132 | break 133 | else: 134 | print("This world name is not on the list!") 135 | elif args[1] == "regionset": 136 | if len(self.regionset): 137 | self.current = self.regionset 138 | print("workload = set of region files") 139 | else: 140 | print("The region set is empty!") 141 | else: 142 | print("Invalid world number, world name or regionset.") 143 | 144 | elif args[0] == "processes": 145 | if len(args) == 1: 146 | print("processes = {0}".format(self.options.processes)) 147 | else: 148 | try: 149 | if int(args[1]) > 0: 150 | self.options.processes = int(args[1]) 151 | print("processes = {0}".format(args[1])) 152 | else: 153 | print("Invalid value. Valid values are positive integers.") 154 | except ValueError: 155 | print("Invalid value. Valid values are positive integers.") 156 | 157 | elif args[0] == "verbose": 158 | if len(args) == 1: 159 | print("verbose = {0}".format(str(self.options.verbose))) 160 | else: 161 | if args[1] == "True": 162 | self.options.verbose = True 163 | print("verbose = {0}".format(args[1])) 164 | elif args[1] == "False": 165 | self.options.verbose = False 166 | print("verbose = {0}".format(args[1])) 167 | else: 168 | print("Invalid value. Valid values are True and False.") 169 | else: 170 | print("Invalid argument! Write \'help set\' to see a list of valid variables.") 171 | 172 | def do_summary(self, arg): 173 | """ Prints a summary of all the problems found in the region 174 | files. """ 175 | if len(arg) == 0: 176 | if self.current: 177 | if self.current.scanned: 178 | text = self.current.generate_report(True) 179 | if text: 180 | print(text) 181 | else: 182 | print("No problems found!") 183 | else: 184 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 185 | else: 186 | print("No world/region-set is set! Use \'set workload\' to set a world/regionset to work with.") 187 | else: 188 | print("This command doesn't use any arguments.") 189 | 190 | def do_current_workload(self, arg): 191 | """ Prints the info of the current workload """ 192 | if len(arg) == 0: 193 | if self.current: 194 | print(self.current) 195 | else: 196 | print("No world/region-set is set! Use \'set workload\' to set a world/regionset to work with.") 197 | else: 198 | print("This command doesn't use any arguments.") 199 | 200 | def do_scan(self, arg): 201 | """ Scans the current workload. """ 202 | # TODO: what about scanning while deleting entities as done in non-interactive mode? 203 | # this would need an option to choose which of the two methods use 204 | o = self.options 205 | if len(arg.split()) > 0: 206 | print("Error: too many parameters.") 207 | else: 208 | if self.current: 209 | if isinstance(self.current, world.World): 210 | self.current = world.World(self.current.path) 211 | console_scan_world(self.current, o.processes, 212 | o.entity_limit, o.delete_entities, 213 | o.verbose) 214 | elif isinstance(self.current, world.RegionSet): 215 | print("\n{0:-^60}".format(' Scanning region files ')) 216 | console_scan_regionset(self.current, o.processes, 217 | o.entity_limit, o.delete_entities, 218 | o.verbose) 219 | else: 220 | print("No world set! Use \'set workload\'") 221 | 222 | def do_count_chunks(self, arg): 223 | """ Counts the number of chunks with the given problem and 224 | prints the result """ 225 | if self.current and self.current.scanned: 226 | if len(arg.split()) == 0: 227 | print("Possible counters are: {0}".format(self.possible_chunk_args_text)) 228 | elif len(arg.split()) > 1: 229 | print("Error: too many parameters.") 230 | else: 231 | if arg in list(c.CHUNK_PROBLEMS_ARGS.values()) or arg == 'all': 232 | total = self.current.count_chunks(None) 233 | for problem, status_text, a in c.CHUNK_PROBLEMS_ITERATOR: 234 | if arg == 'all' or arg == a: 235 | n = self.current.count_chunks(problem) 236 | print("Chunks with status \'{0}\': {1}".format(status_text, n)) 237 | print("Total chunks: {0}".format(total)) 238 | else: 239 | print("Unknown counter.") 240 | else: 241 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 242 | 243 | def do_count_regions(self, arg): 244 | """ Counts the number of regions with the given problem and 245 | prints the result """ 246 | if self.current and self.current.scanned: 247 | if len(arg.split()) == 0: 248 | print("Possible counters are: {0}".format(self.possible_region_args_text)) 249 | elif len(arg.split()) > 1: 250 | print("Error: too many parameters.") 251 | else: 252 | if arg in list(c.REGION_PROBLEMS_ARGS.values()) or arg == 'all': 253 | total = self.current.count_regions(None) 254 | for problem, status_text, a in c.REGION_PROBLEMS_ITERATOR: 255 | if arg == 'all' or arg == a: 256 | n = self.current.count_regions(problem) 257 | print("Regions with status \'{0}\': {1}".format(status_text, n)) 258 | print("Total regions: {0}".format(total)) 259 | else: 260 | print("Unknown counter.") 261 | else: 262 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 263 | 264 | def do_count_all(self, arg): 265 | """ Print all the counters for chunks and regions. """ 266 | if self.current and self.current.scanned: 267 | if len(arg.split()) > 0: 268 | print("This command doesn't requiere any arguments") 269 | else: 270 | print("{0:#^60}".format("Chunk problems:")) 271 | self.do_count_chunks('all') 272 | print("\n") 273 | print("{0:#^60}".format("Region problems:")) 274 | self.do_count_regions('all') 275 | else: 276 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 277 | 278 | def do_remove_entities(self, arg): 279 | if self.current and self.current.scanned: 280 | if len(arg.split()) > 0: 281 | print("Error: too many parameters.") 282 | else: 283 | print("WARNING: This will delete all the entities in the chunks that have more entities than entity-limit, make sure you know what entities are!.\nAre you sure you want to continue? (yes/no):") 284 | answer = input() 285 | if answer == 'yes': 286 | counter = self.current.remove_entities() 287 | print("Deleted {0} entities.".format(counter)) 288 | if counter: 289 | self.current.scanned = False 290 | self.current.rescan_entities(self.options) 291 | elif answer == 'no': 292 | print("Ok!") 293 | else: 294 | print("Invalid answer, use \'yes\' or \'no\' the next time!.") 295 | else: 296 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 297 | 298 | def do_remove_chunks(self, arg): 299 | if self.current and self.current.scanned: 300 | if len(arg.split()) == 0: 301 | print("Possible arguments are: {0}".format(self.possible_chunk_args_text)) 302 | elif len(arg.split()) > 1: 303 | print("Error: too many parameters.") 304 | else: 305 | if arg in list(c.CHUNK_PROBLEMS_ARGS.values()) or arg == 'all': 306 | for problem, status_text, a in c.CHUNK_PROBLEMS_ITERATOR: 307 | if arg == 'all' or arg == a: 308 | n = self.current.remove_problematic_chunks(problem) 309 | if n: 310 | self.current.scanned = False 311 | print("Removed {0} chunks with status \'{1}\'.\n".format(n, status_text)) 312 | else: 313 | print("Unknown argument.") 314 | else: 315 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 316 | 317 | def do_replace_chunks(self, arg): 318 | el = self.options.entity_limit 319 | de = self.options.delete_entities 320 | if self.current and self.current.scanned: 321 | if len(arg.split()) == 0: 322 | print("Possible arguments are: {0}".format(self.possible_chunk_args_text)) 323 | elif len(arg.split()) > 1: 324 | print("Error: too many parameters.") 325 | else: 326 | if arg in list(c.CHUNK_PROBLEMS_ARGS.values()) or arg == 'all': 327 | for problem, status_text, a in c.CHUNK_PROBLEMS_ITERATOR: 328 | if arg == 'all' or arg == a: 329 | n = self.current.replace_problematic_chunks(self.backup_worlds, problem, el, de) 330 | if n: 331 | self.current.scanned = False 332 | print("\nReplaced {0} chunks with status \'{1}\'.".format(n, status_text)) 333 | else: 334 | print("Unknown argument.") 335 | else: 336 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 337 | 338 | def do_replace_regions(self, arg): 339 | el = self.options.entity_limit 340 | de = self.options.delete_entities 341 | if self.current and self.current.scanned: 342 | if len(arg.split()) == 0: 343 | print("Possible arguments are: {0}".format(self.possible_region_args_text)) 344 | elif len(arg.split()) > 1: 345 | print("Error: too many parameters.") 346 | else: 347 | if arg in list(c.REGION_PROBLEMS_ARGS.values()) or arg == 'all': 348 | for problem, status_text, a in c.REGION_PROBLEMS_ITERATOR: 349 | if arg == 'all' or arg == a: 350 | n = self.current.replace_problematic_regions(self.backup_worlds, problem, el, de) 351 | if n: 352 | self.current.scanned = False 353 | print("\nReplaced {0} regions with status \'{1}\'.".format(n, status_text)) 354 | else: 355 | print("Unknown argument.") 356 | else: 357 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 358 | 359 | def do_remove_regions(self, arg): 360 | if self.current and self.current.scanned: 361 | if len(arg.split()) == 0: 362 | print("Possible arguments are: {0}".format(self.possible_region_args_text)) 363 | elif len(arg.split()) > 1: 364 | print("Error: too many parameters.") 365 | else: 366 | if arg in list(c.REGION_PROBLEMS_ARGS.values()) or arg == 'all': 367 | for problem, status_text, a in c.REGION_PROBLEMS_ITERATOR: 368 | if arg == 'all' or arg == a: 369 | n = self.current.remove_problematic_regions(problem) 370 | if n: 371 | self.current.scanned = False 372 | print("\nRemoved {0} regions with status \'{1}\'.".format(n, status_text)) 373 | else: 374 | print("Unknown argument.") 375 | else: 376 | print("The world hasn't be scanned (or it needs a rescan). Use \'scan\' to scan it.") 377 | 378 | def do_quit(self, arg): 379 | print("Quitting.") 380 | return True 381 | 382 | def do_exit(self, arg): 383 | print("Exiting.") 384 | return True 385 | 386 | def do_EOF(self, arg): 387 | print("Quitting.") 388 | return True 389 | 390 | ################################################# 391 | # Complete methods 392 | ################################################# 393 | def complete_arg(self, text, possible_args): 394 | l = [] 395 | for arg in possible_args: 396 | if text in arg and arg.find(text) == 0: 397 | l.append(arg + " ") 398 | return l 399 | 400 | def complete_set(self, text, line, begidx, endidx): 401 | if "workload " in line: 402 | # return the list of world names plus 'regionset' plus a list of world1, world2... 403 | possible_args = tuple(self.world_names) + ('regionset',) + tuple(['world' + str(i + 1) for i in range(len(self.world_names))]) 404 | elif 'verbose ' in line: 405 | possible_args = ('True', 'False') 406 | else: 407 | possible_args = ('entity-limit', 'verbose', 'processes', 'workload') 408 | return self.complete_arg(text, possible_args) 409 | 410 | def complete_count_chunks(self, text, line, begidx, endidx): 411 | possible_args = list(c.CHUNK_PROBLEMS_ARGS.values()) + ['all'] 412 | return self.complete_arg(text, possible_args) 413 | 414 | def complete_remove_chunks(self, text, line, begidx, endidx): 415 | possible_args = list(c.CHUNK_PROBLEMS_ARGS.values()) + ['all'] 416 | return self.complete_arg(text, possible_args) 417 | 418 | def complete_replace_chunks(self, text, line, begidx, endidx): 419 | possible_args = list(c.CHUNK_PROBLEMS_ARGS.values()) + ['all'] 420 | return self.complete_arg(text, possible_args) 421 | 422 | def complete_count_regions(self, text, line, begidx, endidx): 423 | possible_args = list(c.REGION_PROBLEMS_ARGS.values()) + ['all'] 424 | return self.complete_arg(text, possible_args) 425 | 426 | def complete_remove_regions(self, text, line, begidx, endidx): 427 | possible_args = list(c.REGION_PROBLEMS_ARGS.values()) + ['all'] 428 | return self.complete_arg(text, possible_args) 429 | 430 | def complete_replace_regions(self, text, line, begidx, endidx): 431 | possible_args = list(c.REGION_PROBLEMS_ARGS.values()) + ['all'] 432 | return self.complete_arg(text, possible_args) 433 | 434 | ################################################# 435 | # Help methods 436 | ################################################# 437 | # TODO sería una buena idea poner un artículo de ayuda de como usar el programa en un caso típico. 438 | # TODO: the help texts need a normalize 439 | def help_set(self): 440 | print("\nSets some variables used for the scan in interactive mode. " 441 | "If you run this command without an argument for a variable " 442 | "you can see the current state of the variable. You can set:\n" 443 | " verbose\n" 444 | "If True prints a line per scanned region file instead of " 445 | "showing a progress bar.\n" 446 | " entity-limit\n" 447 | "If a chunk has more than this number of entities it will be " 448 | "added to the list of chunks with too many entities problem.\n" 449 | " processes" 450 | "Number of cores used while scanning the world.\n" 451 | " workload\n" 452 | "If you input a few worlds you can choose wich one will be " 453 | "scanned using this command.\n") 454 | 455 | def help_current_workload(self): 456 | print("\nPrints information of the current region-set/world. This will be the region-set/world to scan and fix.\n") 457 | 458 | def help_scan(self): 459 | print("\nScans the current world set or the region set.\n") 460 | 461 | def help_count_chunks(self): 462 | print("\n Prints out the number of chunks with the given status. For example") 463 | print("\'count corrupted\' prints the number of corrupted chunks in the world.") 464 | print() 465 | print("Possible status are: {0}\n".format(self.possible_chunk_args_text)) 466 | 467 | def help_remove_entities(self): 468 | print("\nRemove all the entities in chunks that have more than entity-limit entities.") 469 | print() 470 | print("This chunks are the ones with status \'too many entities\'.\n") 471 | 472 | def help_remove_chunks(self): 473 | print("\nRemoves bad chunks with the given problem.") 474 | print() 475 | print("Please, be careful, when used with the status too-many-entities this will") 476 | print("REMOVE THE CHUNKS with too many entities problems, not the entities.") 477 | print("To remove only the entities see the command remove_entities.") 478 | print() 479 | print("For example \'remove_chunks corrupted\' this will remove corrupted chunks.") 480 | print() 481 | print("Possible status are: {0}\n".format(self.possible_chunk_args_text)) 482 | print() 483 | 484 | def help_replace_chunks(self): 485 | print("\nReplaces bad chunks with the given status using the backups directories.") 486 | print() 487 | print("Exampe: \"replace_chunks corrupted\"") 488 | print() 489 | print("this will replace the corrupted chunks with the given backups.") 490 | print() 491 | print("Possible status are: {0}\n".format(self.possible_chunk_args_text)) 492 | print() 493 | print("Note: after replacing any chunks you have to rescan the world.\n") 494 | 495 | def help_count_regions(self): 496 | print("\n Prints out the number of regions with the given status. For example ") 497 | print("\'count_regions too-small\' prints the number of region with \'too-small\' status.") 498 | print() 499 | print("Possible status are: {0}\n".format(self.possible_region_args_text)) 500 | 501 | def help_remove_regions(self): 502 | print("\nRemoves regions with the given status.") 503 | print() 504 | print("Example: \'remove_regions too-small\'") 505 | print() 506 | print("this will remove the region files with status \'too-small\'.") 507 | print() 508 | print("Possible status are: {0}".format(self.possible_region_args_text)) 509 | print() 510 | print("Note: after removing any regions you have to rescan the world.\n") 511 | 512 | def help_replace_regions(self): 513 | print("\nReplaces regions with the given status.") 514 | print() 515 | print("Example: \"replace_regions too-small\"") 516 | print() 517 | print("this will try to replace the region files with status \'too-small\'") 518 | print("with the given backups.") 519 | print() 520 | print("Possible status are: {0}".format(self.possible_region_args_text)) 521 | print() 522 | print("Note: after replacing any regions you have to rescan the world.\n") 523 | 524 | def help_summary(self): 525 | print("\nPrints a summary of all the problems found in the current workload.\n") 526 | 527 | def help_quit(self): 528 | print("\nQuits interactive mode, exits region-fixer. Same as \'EOF\' and \'exit\' commands.\n") 529 | 530 | def help_EOF(self): 531 | print("\nQuits interactive mode, exits region-fixer. Same as \'quit\' and \'exit\' commands\n") 532 | 533 | def help_exit(self): 534 | print("\nQuits interactive mode, exits region-fixer. Same as \'quit\' and \'EOF\' commands\n") 535 | 536 | def help_help(self): 537 | print("Prints help help.") 538 | -------------------------------------------------------------------------------- /regionfixer_core/progressbar.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: iso-8859-1 -*- 3 | # 4 | # progressbar - Text progressbar library for python. 5 | # Copyright (c) 2005 Nilton Volpato 6 | # 7 | # This library is free software; you can redistribute it and/or 8 | # modify it under the terms of the GNU Lesser General Public 9 | # License as published by the Free Software Foundation; either 10 | # version 2.1 of the License, or (at your option) any later version. 11 | # 12 | # This library is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 15 | # Lesser General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU Lesser General Public 18 | # License along with this library; if not, write to the Free Software 19 | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 20 | 21 | 22 | """Text progressbar library for python. 23 | 24 | This library provides a text mode progressbar. This is tipically used 25 | to display the progress of a long running operation, providing a 26 | visual clue that processing is underway. 27 | 28 | The ProgressBar class manages the progress, and the format of the line 29 | is given by a number of widgets. A widget is an object that may 30 | display diferently depending on the state of the progress. There are 31 | three types of widget: 32 | - a string, which always shows itself; 33 | - a ProgressBarWidget, which may return a diferent value every time 34 | it's update method is called; and 35 | - a ProgressBarWidgetHFill, which is like ProgressBarWidget, except it 36 | expands to fill the remaining width of the line. 37 | 38 | The progressbar module is very easy to use, yet very powerful. And 39 | automatically supports features like auto-resizing when available. 40 | """ 41 | 42 | __author__ = "Nilton Volpato" 43 | __author_email__ = "first-name dot last-name @ gmail.com" 44 | __date__ = "2006-05-07" 45 | __version__ = "2.2" 46 | 47 | # Changelog 48 | # 49 | # 2006-05-07: v2.2 fixed bug in windows 50 | # 2005-12-04: v2.1 autodetect terminal width, added start method 51 | # 2005-12-04: v2.0 everything is now a widget (wow!) 52 | # 2005-12-03: v1.0 rewrite using widgets 53 | # 2005-06-02: v0.5 rewrite 54 | # 2004-??-??: v0.1 first version 55 | 56 | 57 | import sys, time 58 | from array import array 59 | try: 60 | from fcntl import ioctl 61 | import termios 62 | except ImportError: 63 | pass 64 | import signal 65 | 66 | class ProgressBarWidget(object): 67 | """This is an element of ProgressBar formatting. 68 | 69 | The ProgressBar object will call it's update value when an update 70 | is needed. It's size may change between call, but the results will 71 | not be good if the size changes drastically and repeatedly. 72 | """ 73 | def update(self, pbar): 74 | """Returns the string representing the widget. 75 | 76 | The parameter pbar is a reference to the calling ProgressBar, 77 | where one can access attributes of the class for knowing how 78 | the update must be made. 79 | 80 | At least this function must be overriden.""" 81 | pass 82 | 83 | class ProgressBarWidgetHFill(object): 84 | """This is a variable width element of ProgressBar formatting. 85 | 86 | The ProgressBar object will call it's update value, informing the 87 | width this object must the made. This is like TeX \\hfill, it will 88 | expand to fill the line. You can use more than one in the same 89 | line, and they will all have the same width, and together will 90 | fill the line. 91 | """ 92 | def update(self, pbar, width): 93 | """Returns the string representing the widget. 94 | 95 | The parameter pbar is a reference to the calling ProgressBar, 96 | where one can access attributes of the class for knowing how 97 | the update must be made. The parameter width is the total 98 | horizontal width the widget must have. 99 | 100 | At least this function must be overriden.""" 101 | pass 102 | 103 | 104 | class ETA(ProgressBarWidget): 105 | "Widget for the Estimated Time of Arrival" 106 | def format_time(self, seconds): 107 | return time.strftime('%H:%M:%S', time.gmtime(seconds)) 108 | def update(self, pbar): 109 | if pbar.currval == 0: 110 | return 'ETA: --:--:--' 111 | elif pbar.finished: 112 | return 'Time: %s' % self.format_time(pbar.seconds_elapsed) 113 | else: 114 | elapsed = pbar.seconds_elapsed 115 | eta = elapsed * pbar.maxval / pbar.currval - elapsed 116 | return 'ETA: %s' % self.format_time(eta) 117 | 118 | class FileTransferSpeed(ProgressBarWidget): 119 | "Widget for showing the transfer speed (useful for file transfers)." 120 | def __init__(self): 121 | self.fmt = '%6.2f %s' 122 | self.units = ['B','K','M','G','T','P'] 123 | def update(self, pbar): 124 | if pbar.seconds_elapsed < 2e-6:#== 0: 125 | bps = 0.0 126 | else: 127 | bps = float(pbar.currval) / pbar.seconds_elapsed 128 | spd = bps 129 | for u in self.units: 130 | if spd < 1000: 131 | break 132 | spd /= 1000 133 | return self.fmt % (spd, u+'/s') 134 | 135 | class RotatingMarker(ProgressBarWidget): 136 | "A rotating marker for filling the bar of progress." 137 | def __init__(self, markers='|/-\\'): 138 | self.markers = markers 139 | self.curmark = -1 140 | def update(self, pbar): 141 | if pbar.finished: 142 | return self.markers[0] 143 | self.curmark = (self.curmark + 1)%len(self.markers) 144 | return self.markers[self.curmark] 145 | 146 | class Percentage(ProgressBarWidget): 147 | "Just the percentage done." 148 | def update(self, pbar): 149 | return '%3d%%' % pbar.percentage() 150 | 151 | class Bar(ProgressBarWidgetHFill): 152 | "The bar of progress. It will strech to fill the line." 153 | def __init__(self, marker='#', left='|', right='|'): 154 | self.marker = marker 155 | self.left = left 156 | self.right = right 157 | def _format_marker(self, pbar): 158 | if isinstance(self.marker, (str, unicode)): 159 | return self.marker 160 | else: 161 | return self.marker.update(pbar) 162 | def update(self, pbar, width): 163 | percent = pbar.percentage() 164 | cwidth = width - len(self.left) - len(self.right) 165 | marked_width = int(percent * cwidth / 100) 166 | m = self._format_marker(pbar) 167 | bar = (self.left + (m*marked_width).ljust(cwidth) + self.right) 168 | return bar 169 | 170 | class ReverseBar(Bar): 171 | "The reverse bar of progress, or bar of regress. :)" 172 | def update(self, pbar, width): 173 | percent = pbar.percentage() 174 | cwidth = width - len(self.left) - len(self.right) 175 | marked_width = int(percent * cwidth / 100) 176 | m = self._format_marker(pbar) 177 | bar = (self.left + (m*marked_width).rjust(cwidth) + self.right) 178 | return bar 179 | 180 | default_widgets = [Percentage(), ' ', Bar()] 181 | class ProgressBar(object): 182 | """This is the ProgressBar class, it updates and prints the bar. 183 | 184 | The term_width parameter may be an integer. Or None, in which case 185 | it will try to guess it, if it fails it will default to 80 columns. 186 | 187 | The simple use is like this: 188 | >>> pbar = ProgressBar().start() 189 | >>> for i in xrange(100): 190 | ... # do something 191 | ... pbar.update(i+1) 192 | ... 193 | >>> pbar.finish() 194 | 195 | But anything you want to do is possible (well, almost anything). 196 | You can supply different widgets of any type in any order. And you 197 | can even write your own widgets! There are many widgets already 198 | shipped and you should experiment with them. 199 | 200 | When implementing a widget update method you may access any 201 | attribute or function of the ProgressBar object calling the 202 | widget's update method. The most important attributes you would 203 | like to access are: 204 | - currval: current value of the progress, 0 <= currval <= maxval 205 | - maxval: maximum (and final) value of the progress 206 | - finished: True if the bar is have finished (reached 100%), False o/w 207 | - start_time: first time update() method of ProgressBar was called 208 | - seconds_elapsed: seconds elapsed since start_time 209 | - percentage(): percentage of the progress (this is a method) 210 | """ 211 | def __init__(self, maxval=100, widgets=default_widgets, term_width=None, 212 | fd=sys.stderr): 213 | assert maxval > 0 214 | self.maxval = maxval 215 | self.widgets = widgets 216 | self.fd = fd 217 | self.signal_set = False 218 | if term_width is None: 219 | try: 220 | self.handle_resize(None,None) 221 | signal.signal(signal.SIGWINCH, self.handle_resize) 222 | self.signal_set = True 223 | except: 224 | self.term_width = 79 225 | else: 226 | self.term_width = term_width 227 | 228 | self.currval = 0 229 | self.finished = False 230 | self.prev_percentage = -1 231 | self.start_time = None 232 | self.seconds_elapsed = 0 233 | 234 | def handle_resize(self, signum, frame): 235 | h,w=array('h', ioctl(self.fd,termios.TIOCGWINSZ,'\0'*8))[:2] 236 | self.term_width = w 237 | 238 | def percentage(self): 239 | "Returns the percentage of the progress." 240 | return self.currval*100.0 / self.maxval 241 | 242 | def _format_widgets(self): 243 | r = [] 244 | hfill_inds = [] 245 | num_hfill = 0 246 | currwidth = 0 247 | for i, w in enumerate(self.widgets): 248 | if isinstance(w, ProgressBarWidgetHFill): 249 | r.append(w) 250 | hfill_inds.append(i) 251 | num_hfill += 1 252 | elif isinstance(w, (str, unicode)): 253 | r.append(w) 254 | currwidth += len(w) 255 | else: 256 | weval = w.update(self) 257 | currwidth += len(weval) 258 | r.append(weval) 259 | for iw in hfill_inds: 260 | r[iw] = r[iw].update(self, (self.term_width-currwidth)/num_hfill) 261 | return r 262 | 263 | def _format_line(self): 264 | return ''.join(self._format_widgets()).ljust(self.term_width) 265 | 266 | def _need_update(self): 267 | return int(self.percentage()) != int(self.prev_percentage) 268 | 269 | def update(self, value): 270 | "Updates the progress bar to a new value." 271 | assert 0 <= value <= self.maxval 272 | self.currval = value 273 | if not self._need_update() or self.finished: 274 | return 275 | if not self.start_time: 276 | self.start_time = time.time() 277 | self.seconds_elapsed = time.time() - self.start_time 278 | self.prev_percentage = self.percentage() 279 | if value != self.maxval: 280 | self.fd.write(self._format_line() + '\r') 281 | else: 282 | self.finished = True 283 | self.fd.write(self._format_line() + '\n') 284 | 285 | def start(self): 286 | """Start measuring time, and prints the bar at 0%. 287 | 288 | It returns self so you can use it like this: 289 | >>> pbar = ProgressBar().start() 290 | >>> for i in xrange(100): 291 | ... # do something 292 | ... pbar.update(i+1) 293 | ... 294 | >>> pbar.finish() 295 | """ 296 | self.update(0) 297 | return self 298 | 299 | def finish(self): 300 | """Used to tell the progress is finished.""" 301 | self.update(self.maxval) 302 | if self.signal_set: 303 | signal.signal(signal.SIGWINCH, signal.SIG_DFL) 304 | 305 | 306 | 307 | 308 | 309 | 310 | if __name__=='__main__': 311 | import os 312 | 313 | def example1(): 314 | widgets = ['Test: ', Percentage(), ' ', Bar(marker=RotatingMarker()), 315 | ' ', ETA(), ' ', FileTransferSpeed()] 316 | pbar = ProgressBar(widgets=widgets, maxval=10000000).start() 317 | for i in range(1000000): 318 | # do something 319 | pbar.update(10*i+1) 320 | pbar.finish() 321 | print 322 | 323 | def example2(): 324 | class CrazyFileTransferSpeed(FileTransferSpeed): 325 | "It's bigger between 45 and 80 percent" 326 | def update(self, pbar): 327 | if 45 < pbar.percentage() < 80: 328 | return 'Bigger Now ' + FileTransferSpeed.update(self,pbar) 329 | else: 330 | return FileTransferSpeed.update(self,pbar) 331 | 332 | widgets = [CrazyFileTransferSpeed(),' <<<', Bar(), '>>> ', Percentage(),' ', ETA()] 333 | pbar = ProgressBar(widgets=widgets, maxval=10000000) 334 | # maybe do something 335 | pbar.start() 336 | for i in range(2000000): 337 | # do something 338 | pbar.update(5*i+1) 339 | pbar.finish() 340 | print 341 | 342 | def example3(): 343 | widgets = [Bar('>'), ' ', ETA(), ' ', ReverseBar('<')] 344 | pbar = ProgressBar(widgets=widgets, maxval=10000000).start() 345 | for i in range(1000000): 346 | # do something 347 | pbar.update(10*i+1) 348 | pbar.finish() 349 | print 350 | 351 | def example4(): 352 | widgets = ['Test: ', Percentage(), ' ', 353 | Bar(marker='0',left='[',right=']'), 354 | ' ', ETA(), ' ', FileTransferSpeed()] 355 | pbar = ProgressBar(widgets=widgets, maxval=500) 356 | pbar.start() 357 | for i in range(100,500+1,50): 358 | time.sleep(0.2) 359 | pbar.update(i) 360 | pbar.finish() 361 | print 362 | 363 | 364 | example1() 365 | example2() 366 | example3() 367 | example4() 368 | 369 | -------------------------------------------------------------------------------- /regionfixer_core/util.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | # 5 | # Region Fixer. 6 | # Fix your region files with a backup copy of your Minecraft world. 7 | # Copyright (C) 2020 Alejandro Aguilera (Fenixin) 8 | # https://github.com/Fenixin/Minecraft-Region-Fixer 9 | # 10 | # This program is free software: you can redistribute it and/or modify 11 | # it under the terms of the GNU General Public License as published by 12 | # the Free Software Foundation, either version 3 of the License, or 13 | # (at your option) any later version. 14 | # 15 | # This program is distributed in the hope that it will be useful, 16 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 17 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 18 | # GNU General Public License for more details. 19 | # 20 | # You should have received a copy of the GNU General Public License 21 | # along with this program. If not, see . 22 | # 23 | 24 | import platform 25 | import sys 26 | import traceback 27 | 28 | 29 | def get_str_from_traceback(ty, value, tb): 30 | """ Return a string from a traceback plus exception. 31 | 32 | Inputs: 33 | - ty -- Exception type 34 | - value -- value of the traceback 35 | - tb -- Traceback 36 | 37 | """ 38 | 39 | t = traceback.format_exception(ty, value, tb) 40 | s = str(ty) + "\n" 41 | for i in t: 42 | s += i 43 | return s 44 | 45 | 46 | # Stolen from: 47 | # http://stackoverflow.com/questions/3041986/python-command-line-yes-no-input 48 | def query_yes_no(question, default="yes"): 49 | """Ask a yes/no question via raw_input() and return their answer. 50 | 51 | "question" is a string that is presented to the user. 52 | "default" is the presumed answer if the user just hits . 53 | It must be "yes" (the default), "no" or None (meaning 54 | an answer is required of the user). 55 | 56 | The "answer" return value is one of "yes" or "no". 57 | """ 58 | valid = {"yes": True, "y": True, "ye": True, 59 | "no": False, "n": False 60 | } 61 | if default is None: 62 | prompt = " [y/n] " 63 | elif default == "yes": 64 | prompt = " [Y/n] " 65 | elif default == "no": 66 | prompt = " [y/N] " 67 | else: 68 | raise ValueError("invalid default answer: '%s'" % default) 69 | 70 | while True: 71 | sys.stdout.write(question + prompt) 72 | choice = input().lower() 73 | if default is not None and choice == '': 74 | return valid[default] 75 | elif choice in valid: 76 | return valid[choice] 77 | else: 78 | sys.stdout.write("Please respond with 'yes' or 'no' " 79 | "(or 'y' or 'n').\n") 80 | 81 | 82 | # stolen from minecraft overviewer 83 | # https://github.com/overviewer/Minecraft-Overviewer/ 84 | def is_bare_console(): 85 | """Returns true if the python script is running in a bare console 86 | 87 | In Windows, that is, if the script wasn't started in a cmd.exe 88 | session. 89 | 90 | """ 91 | 92 | if platform.system() == 'Windows': 93 | try: 94 | import ctypes 95 | GetConsoleProcessList = ctypes.windll.kernel32.GetConsoleProcessList 96 | num = GetConsoleProcessList(ctypes.byref(ctypes.c_int(0)), ctypes.c_int(1)) 97 | if num == 1: 98 | return True 99 | 100 | except Exception: 101 | pass 102 | return False 103 | 104 | 105 | def entitle(text, level=0): 106 | """ Put the text in a title with lot's of hashes around it. """ 107 | 108 | t = '' 109 | if level == 0: 110 | t += "\n" 111 | t += "{0:#^60}\n".format('') 112 | t += "{0:#^60}\n".format(' ' + text + ' ') 113 | t += "{0:#^60}\n".format('') 114 | return t 115 | 116 | 117 | def table(columns): 118 | """ Generates a text containing a pretty table. 119 | 120 | Input: 121 | - columns -- A list containing lists in which each one of the is a column 122 | of the table. 123 | 124 | """ 125 | 126 | def get_max_len(l): 127 | """ Takes a list of strings and returns the length of the biggest string """ 128 | m = 0 129 | for e in l: 130 | if len(str(e)) > m: 131 | m = len(str(e)) 132 | return m 133 | 134 | text = "" 135 | # stores the size of the biggest element in that column 136 | ml = [] 137 | # fill up ml 138 | for c in columns: 139 | m = 0 140 | t = get_max_len(c) 141 | if t > m: 142 | m = t 143 | ml.append(m) 144 | # get the total width of the table: 145 | ml_total = 0 146 | for i in range(len(ml)): 147 | ml_total += ml[i] + 2 # size of each word + 2 spaces 148 | ml_total += 1 + 2 # +1 for the separator | and +2 for the borders 149 | text += "-" * ml_total + "\n" 150 | # all the columns have the same number of rows 151 | row = len(columns[0]) 152 | for r in range(row): 153 | line = "|" 154 | # put all the elements in this row together with spaces 155 | for i in range(len(columns)): 156 | line += "{0: ^{width}}".format(columns[i][r], width=ml[i] + 2) 157 | # add a separator for the first column 158 | if i == 0: 159 | line += "|" 160 | 161 | text += line + "|" + "\n" 162 | if r == 0: 163 | text += "-" * ml_total + "\n" 164 | text += "-" * ml_total 165 | return text 166 | 167 | -------------------------------------------------------------------------------- /regionfixer_core/version.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | # 5 | # Region Fixer. 6 | # Fix your region files with a backup copy of your Minecraft world. 7 | # Copyright (C) 2020 Alejandro Aguilera (Fenixin) 8 | # https://github.com/Fenixin/Minecraft-Region-Fixer 9 | # 10 | # This program is free software: you can redistribute it and/or modify 11 | # it under the terms of the GNU General Public License as published by 12 | # the Free Software Foundation, either version 3 of the License, or 13 | # (at your option) any later version. 14 | # 15 | # This program is distributed in the hope that it will be useful, 16 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 17 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 18 | # GNU General Public License for more details. 19 | # 20 | # You should have received a copy of the GNU General Public License 21 | # along with this program. If not, see . 22 | # 23 | 24 | version_string = "0.3.6" 25 | version_numbers = version_string.split('.') 26 | -------------------------------------------------------------------------------- /regionfixer_gui.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | from multiprocessing import freeze_support 5 | import sys 6 | 7 | # Needed for the gui 8 | import regionfixer_core 9 | import nbt 10 | 11 | from gui import Starter 12 | if __name__ == '__main__': 13 | freeze_support() 14 | s = Starter() 15 | value = s.run() 16 | sys.exit(value) 17 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # taken from: http://www.wiki.wxpython.org/py2exe-python26 2 | 3 | # ======================================================# 4 | # File automagically generated by GUI2Exe version 0.3 5 | # Andrea Gavana, 01 April 2007 6 | # ======================================================# 7 | 8 | # Let's start with some default (for me) imports... 9 | 10 | from distutils.core import setup 11 | import py2exe 12 | import glob 13 | import os 14 | import zlib 15 | import shutil 16 | 17 | from regionfixer_core import version as cli_version 18 | #=============================================================================== 19 | # from gui import version as gui_version 20 | #=============================================================================== 21 | 22 | 23 | # Remove the build folder 24 | shutil.rmtree("build", ignore_errors=True) 25 | 26 | # do the same for dist folder 27 | shutil.rmtree("dist", ignore_errors=True) 28 | 29 | MANIFEST_TEMPLATE = """ 30 | 31 | 32 | 38 | %(prog)s 39 | 40 | 41 | 42 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 57 | 58 | 59 | 60 | 61 | 62 | 70 | 71 | 72 | 73 | """ 74 | 75 | class Target(object): 76 | """ A simple class that holds information on our executable file. """ 77 | def __init__(self, **kw): 78 | """ Default class constructor. Update as you need. """ 79 | self.__dict__.update(kw) 80 | 81 | 82 | # Ok, let's explain why I am doing that. 83 | # Often, data_files, excludes and dll_excludes (but also resources) 84 | # can be very long list of things, and this will clutter too much 85 | # the setup call at the end of this file. So, I put all the big lists 86 | # here and I wrap them using the textwrap module. 87 | 88 | data_files = ['COPYING.txt', 'README.rst', 'CONTRIBUTORS.txt', 'DONORS.txt', 'icon.ico'] 89 | 90 | includes = [] 91 | excludes = ['_gtkagg', '_tkagg', 'bsddb', 'curses', 'email', 'pywin.debugger', 92 | 'pywin.debugger.dbgcon', 'pywin.dialogs', 'tcl', 93 | 'Tkconstants', 'Tkinter'] 94 | packages = [] 95 | dll_excludes = ['libgdk-win32-2.0-0.dll', 'libgobject-2.0-0.dll', 'tcl84.dll', 96 | 'tk84.dll', 97 | 'MSVCP90.dll', 'mswsock.dll', 'powrprof.dll'] 98 | icon_resources = [(1, 'icon.ico')] 99 | bitmap_resources = [] 100 | other_resources = [] 101 | other_resources = [(24, 1, MANIFEST_TEMPLATE % dict(prog="MyAppName"))] 102 | 103 | 104 | # This is a place where the user custom code may go. You can do almost 105 | # whatever you want, even modify the data_files, includes and friends 106 | # here as long as they have the same variable name that the setup call 107 | # below is expecting. 108 | 109 | 110 | # 111 | # The following will copy the MSVC run time dll's 112 | # (msvcm90.dll, msvcp90.dll and msvcr90.dll) and 113 | # the Microsoft.VC90.CRT.manifest which I keep in the 114 | # "Py26MSdlls" folder to the dist folder 115 | # 116 | # depending on wx widgets you use, you might need to add 117 | # gdiplus.dll to the above collection 118 | 119 | py26MSdll = glob.glob(r"c:\Dev\Py26MSdlls-9.0.21022.8\msvc\*.*") 120 | 121 | # install the MSVC 9 runtime dll's into the application folder 122 | data_files += [("", py26MSdll),] 123 | 124 | # I found on some systems one has to put them into sub-folders. 125 | ##data_files += [("Microsoft.VC90.CRT", py26MSdll), 126 | ## ("lib\Microsoft.VC90.CRT", py26MSdll)] 127 | 128 | 129 | 130 | # Ok, now we are going to build our target class. 131 | # I chose this building strategy as it works perfectly for me :-D 132 | 133 | #=============================================================================== 134 | # GUI_Target = Target( 135 | # # what to build 136 | # script = "regionfixer_gui.py", 137 | # icon_resources = icon_resources, 138 | # bitmap_resources = bitmap_resources, 139 | # other_resources = other_resources, 140 | # dest_base = "regionfixer_gui", 141 | # version = gui_version.version_string, 142 | # company_name = "No Company", 143 | # copyright = "Copyright (C) 2020 Alejandro Aguilera", 144 | # name = "Region Fixer GUI" 145 | # ) 146 | #=============================================================================== 147 | 148 | CLI_Target = Target( 149 | # what to build 150 | script = "regionfixer.py", 151 | icon_resources = icon_resources, 152 | bitmap_resources = bitmap_resources, 153 | other_resources = other_resources, 154 | dest_base = "regionfixer", 155 | version = cli_version.version_string, 156 | company_name = "No Company", 157 | copyright = "Copyright (C) 2019 Alejandro Aguilera", 158 | name = "Region Fixer" 159 | ) 160 | 161 | 162 | # That's serious now: we have all (or almost all) the options py2exe 163 | # supports. I put them all even if some of them are usually defaulted 164 | # and not used. Some of them I didn't even know about. 165 | 166 | setup( 167 | 168 | data_files = data_files, 169 | 170 | options = {"py2exe": {"compressed": 2, 171 | "optimize": 2, 172 | "includes": includes, 173 | "excludes": excludes, 174 | "packages": packages, 175 | "dll_excludes": dll_excludes, 176 | "bundle_files": 2, 177 | "dist_dir": "dist", 178 | "xref": False, 179 | "skip_archive": False, 180 | "ascii": False, 181 | "custom_boot_script": '', 182 | } 183 | }, 184 | 185 | zipfile = "lib\library.zip", 186 | console = [CLI_Target] 187 | #windows = [GUI_Target] 188 | ) 189 | 190 | # This is a place where any post-compile code may go. 191 | # You can add as much code as you want, which can be used, for example, 192 | # to clean up your folders or to do some particular post-compilation 193 | # actions. 194 | 195 | # And we are done. That's a setup script :-D 196 | 197 | --------------------------------------------------------------------------------