├── .gitignore ├── extraction ├── out.png ├── README.md └── extract.py ├── visualization ├── out.jpg ├── README.md └── visualization.py ├── coco-format-conversion ├── README.md ├── manga109.py └── coco_format_conversion.py ├── LICENSE └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | __pycache__ 3 | .vscode 4 | -------------------------------------------------------------------------------- /extraction/out.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/manga109/manga109-demos/HEAD/extraction/out.png -------------------------------------------------------------------------------- /visualization/out.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/manga109/manga109-demos/HEAD/visualization/out.jpg -------------------------------------------------------------------------------- /visualization/README.md: -------------------------------------------------------------------------------- 1 | # Visualization Demo 2 | 3 | Code referenced from the [manga109api](https://github.com/manga109/manga109api) repository. 4 | 5 | ## Usage 6 | Executing `python visualization.py` will output the image shown in the Output section. 7 | 8 | When using the code, please specify your dataset path. For example, if you would like to visualize the page 6 of "ARMS", please run the following command : 9 | 10 | ```bash 11 | python visualization.py --manga109_root_dir YOUR_DIR/Manga109_YYYY_MM_DD --book ARMS --page_index 6 12 | ``` 13 | 14 | ## Output 15 | This demo outputs the following image: 16 | 17 | ![](./out.jpg) 18 | 19 | ARMS, (c) Masaki Kato -------------------------------------------------------------------------------- /extraction/README.md: -------------------------------------------------------------------------------- 1 | # Extraction Demo 2 | 3 | Code referenced from the [manga109api](https://github.com/manga109/manga109api) repository. 4 | 5 | ## Usage 6 | Executing `python extract.py` will output a cropped image/s based on the specified annotation provided. 7 | 8 | For a usage example, if you want to extract a faces from 5 pages of YumeNoKayoiji's book, please run the following command : 9 | 10 | ```bash 11 | python extract.py --book YumeNoKayoiji --annotation face --pages 5 --preprocess --size 256 12 | ``` 13 | 14 | ## Output 15 | This will result in a folder being created called 'manga109extracted' and will be filled with face images from the 5 pages in the YumeNoKayoiji's book. 16 | If you run the same script on a different book, it will be added to the same folder with filenames matching the book. 17 | 18 | ![](./out.png) 19 | 20 | Yume no Kayoiji, (c) Yasuyuki Oono 21 | -------------------------------------------------------------------------------- /coco-format-conversion/README.md: -------------------------------------------------------------------------------- 1 | # COCO Format Conversion 2 | 3 | This script converts the Manga109 XML annotations to MSCOCO's JSON format. 4 | 5 | ## Usage 6 | Run the following command to execute the conversion. For the `--manga109-root-dir` option, specify the path to the Manga109 dataset root directory (not the "annotations" directory). 7 | 8 | ```bash 9 | python coco_format_conversion.py YOUR_DIR/Manga109_YYYY_MM_DD --outpath [out-dir] --train-val-cutoff [cutoff_ratio] --val-test-cutoff [cutoff_ratio] 10 | ``` 11 | 12 | The training-validation cutoff and validation-testing cutoff are set to 0.8 and 0.9 by default, repsectively. 13 | 14 | ## Output 15 | The annotation will be outputted as three JSON files, each for training, validation, and the testing dataset, together containing all of the annotations in the provided Manga109 dataset. 16 | 17 | The `file_name` attribute of each image will be set as `[title]/[page_index].jpg`, so that the `images` directory can be directly specified as the image locations when loaded as an MSCOCO dataset. 18 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Aizawa Yamasaki Matsui Laboratory 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /visualization/visualization.py: -------------------------------------------------------------------------------- 1 | import manga109api 2 | from PIL import Image, ImageDraw 3 | import argparse 4 | 5 | def draw_rectangle(img, x0, y0, x1, y1, annotation_type): 6 | assert annotation_type in ["body", "face", "frame", "text"] 7 | color = {"body": "#258039", "face": "#f5be41", 8 | "frame": "#31a9b8", "text": "#cf3721"}[annotation_type] 9 | draw = ImageDraw.Draw(img) 10 | draw.rectangle([x0, y0, x1, y1], outline=color, width=10) 11 | 12 | if __name__ == "__main__": 13 | parser = argparse.ArgumentParser() 14 | parser.add_argument("--manga109_root_dir") 15 | parser.add_argument("--book", default="ARMS") 16 | parser.add_argument("--page_index", type=int, default=6) 17 | args = parser.parse_args() 18 | 19 | manga109_root_dir = args.manga109_root_dir 20 | book = args.book 21 | page_index = args.page_index 22 | 23 | p = manga109api.Parser(root_dir=manga109_root_dir) 24 | annotation = p.get_annotation(book=book) 25 | img = Image.open(p.img_path(book=book, index=page_index)) 26 | 27 | for annotation_type in ["body", "face", "frame", "text"]: 28 | rois = annotation["page"][page_index][annotation_type] 29 | for roi in rois: 30 | draw_rectangle(img, roi["@xmin"], roi["@ymin"], roi["@xmax"], roi["@ymax"], annotation_type) 31 | 32 | img.save("out.jpg") -------------------------------------------------------------------------------- /coco-format-conversion/manga109.py: -------------------------------------------------------------------------------- 1 | import manga109api 2 | 3 | bb_categories = ["frame", "text", "face", "body"] 4 | 5 | 6 | def bb_iter(title, manga109_root_dir, categories=bb_categories): 7 | for i_page, page in enumerate( 8 | Book(title, manga109_root_dir).get_page_iter()): 9 | for i_category, bbtype in enumerate(categories): 10 | for bb in page.get_bbs()[bbtype]: 11 | yield i_page, i_category, bb 12 | 13 | 14 | class Page(object): 15 | def __init__(self, book, page_index): 16 | self.book = book 17 | self.page_index = page_index 18 | 19 | self.width = self.book.annotations["page"][self.page_index]["@width"] 20 | self.height = self.book.annotations["page"][self.page_index]["@height"] 21 | 22 | def get_bbs(self): 23 | bb_dict = dict([(a,[d for d in self.book.annotations["page"][self.page_index][a]]) for a in bb_categories]) 24 | return bb_dict 25 | 26 | 27 | class Book(object): 28 | def __init__(self, book, manga109_root_dir): 29 | self.book = book 30 | self.parser = manga109api.Parser(manga109_root_dir) 31 | self.annotations = self.parser.get_annotation(book=book) 32 | self.n_pages = len(self.annotations["page"]) 33 | 34 | def get_page_iter(self): 35 | for page_index in range(self.n_pages): 36 | yield Page(self, page_index) 37 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Manga109 Demos 2 | 3 | This repository is a collection of demo codes for the [Manga109 dataset](http://www.manga109.org/en/). 4 | 5 | Manga109 is the largest dataset for manga (Japanese comic) images, 6 | that is made publicly available for academic research purposes with proper copyright notation. 7 | 8 | To download images/annotations of Manga109, please visit [this website](http://www.manga109.org/en/download.html) and send an application via the form. 9 | You will then receive a password for downloading the images (109 titles of manga 10 | as jpeg files) 11 | and their annotations (bounding box coordinates of face, body, frame, and speech balloon with texts, 12 | in the form of XML). 13 | 14 | Please see [manga109api](https://github.com/manga109/manga109api) as well when using the dataset. 15 | 16 | ## Contents 17 | 18 | - Annotation visualization demo - `./visualization` 19 | - Bounding box cropping demo - `./extraction` 20 | - MSCOCO format conversion - `./coco-format-conversion` 21 | 22 | 23 | ## Citations 24 | When you make use of images in Manga109, please cite the following paper: 25 | 26 | @article{mtap_matsui_2017, 27 | author={Yusuke Matsui and Kota Ito and Yuji Aramaki and Azuma Fujimoto and Toru Ogawa and Toshihiko Yamasaki and Kiyoharu Aizawa}, 28 | title={Sketch-based Manga Retrieval using Manga109 Dataset}, 29 | journal={Multimedia Tools and Applications}, 30 | volume={76}, 31 | number={20}, 32 | pages={21811--21838}, 33 | doi={10.1007/s11042-016-4020-z}, 34 | year={2017} 35 | } 36 | 37 | When you use annotation data of Manga109, please cite the following paper: 38 | 39 | @article{multimedia_aizawa_2020, 40 | author={Kiyoharu Aizawa and Azuma Fujimoto and Atsushi Otsubo and Toru Ogawa and Yusuke Matsui and Koki Tsubota and Hikaru Ikuta}, 41 | title={Building a Manga Dataset ``Manga109'' with Annotations for Multimedia Applications}, 42 | journal={IEEE MultiMedia}, 43 | volume={27}, 44 | number={2}, 45 | pages={8--18}, 46 | doi={10.1109/mmul.2020.2987895}, 47 | year={2020} 48 | } 49 | -------------------------------------------------------------------------------- /extraction/extract.py: -------------------------------------------------------------------------------- 1 | import manga109api 2 | import argparse 3 | import os 4 | import glob 5 | from PIL import Image 6 | 7 | 8 | def args_parser(): 9 | """ 10 | :return: This function returns the manual input of book, annotation_type, and page count. 11 | """ 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument('--book', type=str, help='Name of book to annotate from.') 14 | parser.add_argument('--annotation', type=str, help='Type of annotation: "body", "face", "frame", "text".') 15 | parser.add_argument('--pages', type=int, default=1, help='Number of pages to annotate.') 16 | parser.add_argument('--preprocess', action='store_true', help='Preprocess the extracted images to have a uniform size.') 17 | parser.add_argument('--size', type=int, default=128, help='The uniform size if using preprocessing.') 18 | args = parser.parse_args() 19 | return args 20 | 21 | 22 | if __name__ == "__main__": 23 | ap = args_parser() 24 | manga109_root_dir = "manga109extracted" 25 | if not os.path.exists(manga109_root_dir): 26 | os.makedirs(manga109_root_dir) 27 | book = ap.book 28 | page_count = ap.pages 29 | file_count = [glob.glob(os.path.join(manga109_root_dir, '**', '*.*'), recursive=True)] 30 | count = len(file_count[0]) 31 | 32 | for page_index in range(page_count): 33 | tracker = 0 34 | p = manga109api.Parser(root_dir="Manga109s_data") 35 | annotation = p.get_annotation(book=book) 36 | img = Image.open(p.img_path(book=book, index=page_index)) 37 | for annotation_type in [ap.annotation]: 38 | rois = annotation["page"][page_index][annotation_type] 39 | for roi in rois: 40 | cropped = img.crop((roi["@xmin"], roi["@ymin"], roi["@xmax"], roi["@ymax"])) 41 | image_x_dim, image_y_dim = cropped.size 42 | if ap.preprocess: 43 | cropped = cropped.resize((ap.size, ap.size), Image.ANTIALIAS) 44 | if image_x_dim >= (ap.size / 2) and image_y_dim >= (ap.size / 2): 45 | cropped.save("manga109extracted/%s_%d.jpg" % (ap.book, count)) 46 | count += 1 47 | tracker += 1 48 | print("Extracted %d %s images from page %d of %s's book." % (tracker, ap.annotation, page_index + 1, ap.book)) 49 | -------------------------------------------------------------------------------- /coco-format-conversion/coco_format_conversion.py: -------------------------------------------------------------------------------- 1 | import os 2 | import pathlib 3 | import json 4 | from collections import OrderedDict 5 | import argparse 6 | 7 | from manga109 import * 8 | 9 | d_info = OrderedDict([ 10 | ("description", "Manga109"), 11 | ("url", "https://manga109.org"), 12 | ]) 13 | 14 | d_licenses = [OrderedDict([ 15 | ("id", 0), 16 | ("url", "http://www.manga109.org/en/download.html"), 17 | ("name", "Manga109"), 18 | ])] 19 | 20 | d_categories = [ 21 | OrderedDict([ 22 | ("id", 0), 23 | ("name", "frame"), 24 | ("supercategory", "frame"), 25 | ]), 26 | OrderedDict([ 27 | ("id", 1), 28 | ("name", "text"), 29 | ("supercategory", "text"), 30 | ]), 31 | OrderedDict([ 32 | ("id", 2), 33 | ("name", "face"), 34 | ("supercategory", "character"), 35 | ]), 36 | OrderedDict([ 37 | ("id", 3), 38 | ("name", "body"), 39 | ("supercategory", "character"), 40 | ]), 41 | ] 42 | 43 | 44 | def images_annotations(manga109_root_dir, train_val_cutoff, val_test_cutoff): 45 | images_path = pathlib.Path(manga109_root_dir) / "images" 46 | ret_images = [] 47 | ret_annotations = [] 48 | 49 | ret_images_val = [] 50 | ret_annotations_val = [] 51 | 52 | ret_images_test = [] 53 | ret_annotations_test = [] 54 | 55 | image_id_base = 0 56 | annotation_id_base = 0 57 | 58 | 59 | titlelist = manga109api.Parser(root_dir=manga109_root_dir).books 60 | for i_title, title in enumerate(titlelist): 61 | print("{}...".format(title), flush=True) 62 | titlepath = images_path / title 63 | 64 | # Images 65 | filelist = tuple(sorted(os.listdir(titlepath))) 66 | for i_file, (filename, page) in enumerate(zip(filelist, Book(title, manga109_root_dir).get_page_iter())): 67 | width, height = page.width, page.height 68 | id_int = i_file + image_id_base 69 | image = OrderedDict([ 70 | ("license", 0), 71 | ("id", id_int), 72 | ("width", width), 73 | ("height", height), 74 | ("file_name", str(pathlib.Path(title) / filename)), 75 | ]) 76 | (ret_images if i_title < len(titlelist) * train_val_cutoff else 77 | ret_images_val if i_title < len(titlelist) * 78 | val_test_cutoff else ret_images_test).append(image) 79 | 80 | # Annotations 81 | bb_list = tuple(bb_iter(title, manga109_root_dir, categories=[d["name"] for d in d_categories])) 82 | for i_id, (i_page, i_category, bb) in enumerate(bb_list): 83 | bb_id = i_id + annotation_id_base 84 | image_id = i_page + image_id_base 85 | p1 = bb["@xmin"], bb["@ymin"] 86 | p2 = bb["@xmin"], bb["@ymax"] 87 | p3 = bb["@xmax"], bb["@ymax"] 88 | p4 = bb["@xmax"], bb["@ymin"] 89 | segmentation = [p1, p2, p3, p4] 90 | width = (bb["@xmax"] - bb["@xmin"]) 91 | height = (bb["@ymax"] - bb["@ymin"]) 92 | area = width * height 93 | annotation = OrderedDict([ 94 | ("id", bb_id), 95 | ("image_id", image_id), 96 | ("category_id", i_category), 97 | ("segmentation", segmentation), 98 | ("area", area), 99 | ("bbox", [bb["@xmin"], bb["@ymin"], width, height]), 100 | ("iscrowd", 0), 101 | ]) 102 | (ret_annotations if i_title < len(titlelist) * train_val_cutoff 103 | else ret_annotations_val if i_title < len(titlelist) * 104 | val_test_cutoff else ret_annotations_test).append(annotation) 105 | 106 | image_id_base += i_file + 1 107 | annotation_id_base += i_id + 1 108 | return ret_images, ret_annotations, ret_images_val, ret_annotations_val, ret_images_test, ret_annotations_test 109 | 110 | def getcoco(manga109_root_dir, train_val_cutoff, val_test_cutoff): 111 | (ret_images, ret_annotations, 112 | ret_images_val, ret_annotations_val, 113 | ret_images_test, ret_annotations_test) = images_annotations( 114 | manga109_root_dir, train_val_cutoff, val_test_cutoff) 115 | 116 | data_training = OrderedDict([ 117 | ("info", d_info), 118 | ("licenses", d_licenses), 119 | ("images", ret_images), 120 | ("annotations", ret_annotations), 121 | ("categories", d_categories) 122 | ]) 123 | 124 | data_val = OrderedDict([ 125 | ("info", d_info), 126 | ("licenses", d_licenses), 127 | ("images", ret_images_val), 128 | ("annotations", ret_annotations_val), 129 | ("categories", d_categories) 130 | ]) 131 | 132 | data_test = OrderedDict([ 133 | ("info", d_info), 134 | ("licenses", d_licenses), 135 | ("images", ret_images_test), 136 | ("annotations", ret_annotations_test), 137 | ("categories", d_categories) 138 | ]) 139 | 140 | return data_training, data_val, data_test 141 | 142 | 143 | if __name__ == "__main__": 144 | parser = argparse.ArgumentParser(description="Convert the Manga109 dataset XML to MSCOCO format.") 145 | parser.add_argument("manga109_root_dir", 146 | type=str, 147 | help="The root directory of the Manga109 dataset.") 148 | parser.add_argument("--outpath", 149 | type=pathlib.Path, 150 | default=pathlib.Path("./out/"), 151 | help="The output directory.") 152 | parser.add_argument("--train-val-cutoff", 153 | type=float, 154 | default=0.8, 155 | help="Cutoff ratio between the training and the validation datasets.") 156 | parser.add_argument("--val-test-cutoff", 157 | type=float, 158 | default=0.9, 159 | help="Cutoff ratio between the validation and the testing datasets.") 160 | args = parser.parse_args() 161 | 162 | manga109_root_dir = pathlib.Path(args.manga109_root_dir) 163 | 164 | data_training, data_val, data_test = getcoco(manga109_root_dir, args.train_val_cutoff, args.val_test_cutoff) 165 | 166 | args.outpath.mkdir(parents=True, exist_ok=True) 167 | 168 | for outname, data_dict in zip(["train", "val", "test"], [data_training, data_val, data_test]): 169 | coco_json = json.dumps(data_dict) 170 | print(f"Writing data for {outname}...") 171 | with (args.outpath / "instances_{}.json".format(outname)).open("wt") as f: 172 | f.write(coco_json) 173 | print("Done.") 174 | --------------------------------------------------------------------------------