├── requirements.txt ├── assets ├── teaser.png ├── diagram1.png ├── diagram2.png ├── diagram3.png ├── diagram1_plan.png ├── diagram2_plan.png ├── diagram3_plan.png ├── ppt_diagram_1.png ├── ppt_diagram_2.png ├── inkscape_diagram_1.png └── inkscape_diagram_2.png ├── LICENSE ├── src ├── custom_main.py ├── openai_handler.py └── in_ctx_prompts.py └── README.md /requirements.txt: -------------------------------------------------------------------------------- 1 | openai==0.27.8 2 | python-dotenv==1.0.0 3 | tqdm==4.65.0 -------------------------------------------------------------------------------- /assets/teaser.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/teaser.png -------------------------------------------------------------------------------- /assets/diagram1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/diagram1.png -------------------------------------------------------------------------------- /assets/diagram2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/diagram2.png -------------------------------------------------------------------------------- /assets/diagram3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/diagram3.png -------------------------------------------------------------------------------- /assets/diagram1_plan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/diagram1_plan.png -------------------------------------------------------------------------------- /assets/diagram2_plan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/diagram2_plan.png -------------------------------------------------------------------------------- /assets/diagram3_plan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/diagram3_plan.png -------------------------------------------------------------------------------- /assets/ppt_diagram_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/ppt_diagram_1.png -------------------------------------------------------------------------------- /assets/ppt_diagram_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/ppt_diagram_2.png -------------------------------------------------------------------------------- /assets/inkscape_diagram_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/inkscape_diagram_1.png -------------------------------------------------------------------------------- /assets/inkscape_diagram_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aszala/DiagrammerGPT/HEAD/assets/inkscape_diagram_2.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Abhay Zala 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /src/custom_main.py: -------------------------------------------------------------------------------- 1 | import argparse # type: ignore 2 | import json 3 | from tqdm import tqdm 4 | from openai_handler import generate_use_all_icl 5 | 6 | if __name__ == "__main__": 7 | parser = argparse.ArgumentParser() 8 | 9 | parser.add_argument("--test_file", type=str, default="custom_prompts.json") 10 | parser.add_argument("--save_file", type=str, default="generated_diagram_plans.json") 11 | 12 | args = parser.parse_args() 13 | 14 | save_file = f"{args.save_file}" 15 | 16 | out = { } 17 | 18 | with open(f"{args.test_file}", "r") as f: 19 | data = json.load(f) 20 | 21 | test_data = [] 22 | 23 | for x in data: 24 | test_data.append({ 25 | "image": x["image"], 26 | "caption": x["caption"], 27 | "topic": x["topic"] 28 | }) 29 | 30 | with open(f"{args.test_file}", "r") as f: 31 | test_images = json.load(f) 32 | 33 | for sample in tqdm(test_data, desc="Generating Diagram Plans"): 34 | diagram, corrections, attempts = generate_use_all_icl(sample["caption"], sample["topic"]) 35 | 36 | out[sample["image"]] = { 37 | "image": sample["image"], 38 | "caption": sample["caption"], 39 | "diagram": diagram, 40 | "corrections": corrections, 41 | "attempts": attempts, 42 | } 43 | 44 | with open(save_file, "w") as f: 45 | json.dump(out, f, indent=2) 46 | 47 | with open(save_file, "w") as f: 48 | json.dump(out, f, indent=2) -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # [DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning](https://diagrammerGPT.github.io/) 2 | 3 | Official implementation of **DiagrammerGPT**, a novel two-stage text-to-diagram generation framework that leverages the layout guidance capabilities of LLMs to generate more accurate open-domain, open-platform diagrams. 4 | 5 | 6 | [![arXiv](https://img.shields.io/badge/Arxiv-DiagrammerGPT-orange)](https://arxiv.org/abs/2310.12128) [![ProjectPage](https://img.shields.io/badge/ProjectPage-DiagrammerGPT-green)](https://diagrammerGPT.github.io/) [![Dataset](https://img.shields.io/badge/Dataset-AI2DCaption-yellow)](https://huggingface.co/datasets/abhayzala/AI2D-Caption) 7 | 8 | 9 | 10 | [Abhay Zala](https://aszala.com/), 11 | [Han Lin](https://hl-hanlin.github.io/), 12 | [Jaemin Cho](https://j-min.io), 13 | [Mohit Bansal](https://www.cs.unc.edu/~mbansal/) 14 | 15 | 16 | ### Code Release Todo List 17 | - [x] Diagram Plan Generation Source Code 18 | - [x] AI2D-Caption Dataset Release 19 | - [ ] Diagram Generation Source Code 20 | 21 |
22 | 23 |
24 | 25 | An overview of DiagrammerGPT, our two-stage framework for open-domain, open platform diagram generation. 26 | - In the first diagram planning stage, given a prompt, our LLM (GPT-4) generates a diagram plan, which consists of dense entities (objects and text labels), fine-grained relationships (between the entities), and precise layouts (2D bounding boxes of entities). Then, the LLM iteratively refines the diagram plan (i.e., updating the plan to better align with the input prompts). 27 | - In the second diagram generation stage, our DiagramGLIGEN outputs the diagram given the diagram plan, then, we render the text labels on the diagram. 28 | 29 |
30 | 31 | 32 | # Generated Examples 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 |
Input PromptDiagram PlanGenerated Diagram
A diagram showing the layers of the earth. It includes the inner and outer cores, the mantle, and the crust.
A diagram showing the Earth's position in four phases as it revolves around the sun.
A diagram showing three rows of rocks. Each row has 5 rocks. The first row shows different types of igneous rocks, including granite, diorite, felsite, basalt, and obsidian. The second row shows different types of sedimentary rocks, including conglomerate, sandstone, shale, limestone, and dolomite. The third row shows different types of metamorphic rocks, including slate, schist, serpentine, quartzite, and marble. Include a label for the type of rock each row shows and each rock.
56 | 57 | # Examples Rendered with Other Platforms 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 78 | 79 | 80 | 81 | 82 | 89 | 90 | 91 | 92 |
Input PromptRendered with Microsoft PowerPointRendered with Inkscape
A diagram showing two 68 | food chains. The left food 69 | chain, starting from the 70 | bottom, goes from lichen, 71 | to slug, to toad, to snake, 72 | to eagle. The right food 73 | chain, starting from the 74 | bottom, goes from algae, to 75 | snail, to crayfish, to fish, to 76 | alligator. 77 |
A diagram showing the 83 | eight phases of the moon 84 | with labels as it revolves 85 | around Earth. It also 86 | indicates the direction of 87 | the sunlight. 88 |
93 | 94 | # Citation 95 | 96 | If you find our project useful in your research, please cite the following paper: 97 | 98 | ```bibtex 99 | @inproceedings{Zala2024DiagrammerGPT, 100 | author = {Abhay Zala and Han Lin and Jaemin Cho and Mohit Bansal}, 101 | title = {DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning}, 102 | year = {2024}, 103 | booktitle = {COLM}, 104 | } 105 | ``` 106 | -------------------------------------------------------------------------------- /src/openai_handler.py: -------------------------------------------------------------------------------- 1 | import openai 2 | from in_ctx_prompts import prompts 3 | import copy 4 | 5 | from dotenv import dotenv_values 6 | config = dotenv_values(".env") 7 | 8 | openai.api_key = config['API_KEY'] 9 | openai.api_base = config['API_BASE'] 10 | openai.api_type = config['API_TYPE'] 11 | openai.api_version = config['API_VERSION'] 12 | 13 | def call_api(log, engine="gpt-4", use_azure=True): 14 | if use_azure: 15 | resp = openai.ChatCompletion.create( 16 | engine=engine, 17 | messages=log, 18 | temperature=0 19 | ) 20 | else: 21 | resp = openai.ChatCompletion.create( 22 | model=engine, 23 | messages=log, 24 | temperature=0 25 | ) 26 | 27 | resp = resp['choices'][0]['message']['content'] 28 | 29 | return resp 30 | 31 | def generate_use_all_icl(caption, topic): 32 | diagram_prompt_keys = [x for x in prompts.keys() if x.startswith("diagram_")] 33 | auditor_prompt_keys = [x for x in prompts.keys() if x.startswith("auditor_")] 34 | 35 | prompt_preamble = prompts[diagram_prompt_keys[0]].split("Here are some examples:")[0] + "Here are some examples:" 36 | prompt_examples = "" 37 | 38 | for key in diagram_prompt_keys: 39 | topic_icl = prompts[key].split("Here are some examples:")[1].split("\n") 40 | topic_icl = '\n'.join(topic_icl[:-2]) 41 | 42 | prompt_examples += f"{topic_icl}\n" 43 | 44 | diagram_prompt = f"{prompt_preamble}{prompt_examples}\nCaption:\n{caption.lower()}\nTopic:\n{topic}\n" 45 | 46 | 47 | auditor_preamble = prompts[auditor_prompt_keys[0]].split("Here are some examples:")[0] + "Here are some examples:\n" 48 | auditor_examples = "" 49 | 50 | for key in auditor_prompt_keys: 51 | topic_icl = prompts[key].split("Here are some examples:")[1].split("\n") 52 | topic_icl = '\n'.join(topic_icl[:-2]) 53 | 54 | auditor_examples += f"{topic_icl}\n" 55 | 56 | auditor_prompt = f"{auditor_preamble}{auditor_examples}\nCaption:\n{caption.lower()}\nTopic:\n{topic}" 57 | 58 | diagram_chatlog = [ 59 | {"role": "user", "content": diagram_prompt}, 60 | ] 61 | 62 | parsed_resp = None 63 | corrections = [] 64 | diagram_attempts = [] 65 | 66 | for i in range(5): 67 | diagram = call_api(diagram_chatlog, engine="gpt-4-32k") 68 | 69 | parsed_resp = parse_diagram(diagram, caption, topic, load=parsed_resp) 70 | diagram_attempts.append(copy.deepcopy(parsed_resp)) 71 | 72 | if i == 4: 73 | continue 74 | 75 | correction_chatlog = [ 76 | {"role": "user", "content": f"{auditor_prompt}\n{diagram}\nWhat is wrong with this diagram?\n"}, 77 | ] 78 | 79 | correction = call_api(correction_chatlog, engine="gpt-4-32k") 80 | corrections.append(correction) 81 | 82 | words = correction.lower().replace(".", "").split(" ") 83 | 84 | if "impossible" in words or "nothing" in words or ("correct" in words and not "not correct" in correction.lower()): 85 | break 86 | 87 | correction = parse_correction(correction) 88 | 89 | diagram_chatlog.append({"role": "assistant", "content": diagram}) 90 | diagram_chatlog.append({"role": "user", "content": correction}) 91 | 92 | tokens = 0 93 | for x in diagram_chatlog: 94 | tokens += len(x["content"].split(" ")) * 1.333 95 | 96 | if tokens > 32000: 97 | break 98 | 99 | return parsed_resp, corrections, diagram_attempts 100 | 101 | def parse_correction(correction): 102 | return f"{correction} Can you please fix it?" 103 | 104 | def parse_diagram(diagram, caption, topic, load=None): 105 | lines = diagram.split('\n') 106 | lines = [x.strip() for x in lines] 107 | 108 | layout = "" 109 | entity_bounds = { } 110 | entity_relatonships = [ ] 111 | 112 | if load is not None: 113 | layout = load["layout"] 114 | entity_bounds = load["entity_bounds"] 115 | entity_relatonships = load["entity_relationships"] 116 | 117 | entity_index = -100000 118 | relationship_index = -100000 119 | location_index = -100000 120 | 121 | save_values = [] 122 | 123 | for i, line in enumerate(lines): 124 | if line.startswith("Diagram Layout:"): 125 | layout = lines[i+1] 126 | if line.startswith("Required Entities:"): 127 | entity_index = i 128 | if line.startswith("Entity Relationships:"): 129 | relationship_index = i 130 | save_values.append("entity_bounds") 131 | if line.startswith("Entity Locations"): 132 | location_index = i 133 | save_values.append("entity_relationships") 134 | 135 | if entity_index > -1: 136 | x = relationship_index 137 | 138 | if x < 0: 139 | x = location_index 140 | 141 | if x < 0: 142 | x = len(lines) 143 | 144 | for line in lines[entity_index+1:x]: 145 | if len(line.strip()) == 0: 146 | continue 147 | 148 | if "image" in line: 149 | words = line.split(" image ") 150 | 151 | id = line.split("(")[1] 152 | value = words[0] 153 | type = "image" 154 | 155 | elif "text" in line: 156 | words = line.split(" text label ") 157 | 158 | id = line.split("(")[1] 159 | value = words[0].replace("\"", "") 160 | type = "text" 161 | 162 | id = id.replace("(", "").replace(")", "") 163 | entity_bounds[id] = { 164 | "id": id, 165 | "type": type, 166 | "value": value, 167 | "bounds": [] 168 | } 169 | 170 | if relationship_index > -1: 171 | x = location_index 172 | 173 | if x < 0: 174 | x = len(lines) 175 | 176 | for line in lines[relationship_index+1:x]: 177 | ids = line.split(" ") 178 | 179 | passed = len(line.strip()) > 0 180 | count = 0 181 | for i, word in enumerate(ids): 182 | if word.startswith("T") or word.startswith("I"): 183 | if not (word in entity_bounds): 184 | passed = False 185 | break 186 | count += 1 187 | 188 | if count < 2: 189 | passed = False 190 | 191 | if not passed: 192 | print("Failed to parse relationship: ", line) 193 | continue 194 | entity_relatonships.append(line) 195 | 196 | if location_index > -1: 197 | for line in lines[location_index+1:]: 198 | words = line.split(" is located at ") 199 | 200 | try: 201 | id = words[0] 202 | words[1] = words[1][:words[1].index("]")] 203 | value = words[1].replace("[", "").replace("]", "").replace(", ", ",") 204 | except: 205 | print("Failed to parse location: ", line) 206 | continue 207 | 208 | if not (id in entity_bounds): 209 | print("Failed to parse location: ", line) 210 | continue 211 | 212 | bounds = value.split(",") 213 | bounds = [int(x.strip()) for x in bounds] 214 | 215 | entity_bounds[id]["bounds"] = bounds 216 | 217 | bad_ids = [] 218 | for id in entity_bounds: 219 | if len(entity_bounds[id]["bounds"]) == 0: 220 | bad_ids.append(id) 221 | 222 | for id in bad_ids: 223 | del entity_bounds[id] 224 | 225 | r = { 226 | "caption": caption, 227 | "topic": topic, 228 | "layout": layout, 229 | "entity_bounds": entity_bounds, 230 | "entity_relationships": list(set(entity_relatonships)) 231 | } 232 | 233 | return r -------------------------------------------------------------------------------- /src/in_ctx_prompts.py: -------------------------------------------------------------------------------- 1 | DIAGRAM_PROMPT_astronomy = """Given a caption of a diagram and topic, generate the diagram layout, and then a list of required entities that would be needed to create the described diagram. Then generate a list of the relationships between the entities (i.e. which ones are connected or labeling each other). Finally, generate the location of each entity. 2 | An entity can be an image or text. Entity locations should be generated in [x, y, width, height] format, where 0,0 is the top left corner and 100,100 is max image size. 3 | 4 | Think step-by-step, break each caption into parts and generate the required entities, relationships, and locations for each part. 5 | 6 | Here are some rules to follow: 7 | All numbers should be positive, do not generate negative numbers. 8 | Please always generate a list of entities, even if the list is empty. 9 | Entities should not be outside the bounds [0, 0, 100, 100]. 10 | The x coordinate + the width of an entity should not exceed 100. 11 | The y coordinate + the height of an entity should not exceed 100. 12 | Entities of the same type should not overlap. 13 | 14 | Here are some examples: 15 | 16 | Caption: 17 | A diagram showing the Earth revolving around the sun in four different phases representing each solstice or equinox. 18 | Topic: 19 | astronomy 20 | Diagram Layout: 21 | circular 22 | Required Entities: 23 | earth image (I0) 24 | earth image (I1) 25 | earth image (I2) 26 | earth image (I3) 27 | sun image (I4) 28 | Entity Relationships: 29 | I1 has an arrow to I0 30 | I2 has an arrow to I1 31 | I0 has an arrow to I3 32 | I3 has an arrow to I2 33 | Entity Locations: 34 | I0 is located at [8, 37, 17, 28] 35 | I1 is located at [43, 17, 14, 20] 36 | I2 is located at [75, 37, 17, 26] 37 | I3 is located at [40, 57, 20, 30] 38 | I4 is located at [48, 47, 3, 6] 39 | 40 | Caption: 41 | A diagram showing the positions of the Earth, the Moon, and the Sun during a lunar eclipse. The Earth is directly between the sun and the moon. The moon is in the earth's shadow. 42 | Topic: 43 | astronomy 44 | Diagram Layout: 45 | abstract 46 | Required Entities: 47 | sun image (I0) 48 | earth image (I1) 49 | moon image (I2) 50 | "Sun" text label (T0) 51 | "Earth" text label (T1) 52 | "Moon" text label (T2) 53 | "Shadow" text label (T3) 54 | Entity Relationships: 55 | T1 labels I1 56 | I1 is between I0 and I2 57 | T0 labels I0 58 | T2 labels I2 59 | Entity Locations: 60 | I0 is located at [8, 56, 13, 28] 61 | I1 is located at [58, 23, 10, 18] 62 | I2 is located at [77, 12, 4, 7] 63 | T0 is located at [10, 86, 9, 7] 64 | T1 is located at [46, 20, 11, 9] 65 | T2 is located at [85, 14, 10, 9] 66 | T3 is located at [66, 19, 12, 8] 67 | 68 | Caption: 69 | A diagram showing the cross section of a sun with all parts labeled. The labeled parts include eruptive prominences, coronal loops, chromosphere, sunspots, the core, radiative zone, convective zone, and the corona. 70 | Topic: 71 | astronomy 72 | Diagram Layout: 73 | abstract 74 | Required Entities: 75 | cross section of a sun image (I0) 76 | sunspots image (I1) 77 | large eruptive prominence image (I2) 78 | coronal loops image (I3) 79 | large eruptive prominence image (I4) 80 | "Sunspots" text label (T0) 81 | "Large Eruptive Prominence" text label (T1) 82 | "Coronal Loops" text label (T2) 83 | "Large Eruptive Prominence" text label (T3) 84 | "Chromosphere" text label (T4) 85 | "Solar Flares" text label (T5) 86 | "Photosphere" text label (T6) 87 | "Prominence" text label (T7) 88 | "Core (Nuclear Fusion)" text label (T8) 89 | "Radiative Zone" text label (T9) 90 | "Convective Zone" text label (T10) 91 | "Corona" text label (T11) 92 | Entity Relationships: 93 | T2 labels I3 94 | T0 labels I1 95 | T1 labels I2 96 | T3 labels I5 97 | Entity Locations: 98 | I0 is located at [13, 14, 74, 73] 99 | I1 is located at [37, 34, 10, 7] 100 | I2 is located at [5, 56, 22, 26] 101 | I3 is located at [35, 84, 8, 4] 102 | I4 is located at [66, 71, 19, 28] 103 | T0 is located at [37, 31, 9, 3] 104 | T1 is located at [1, 72, 19, 5] 105 | T2 is located at [32, 88, 14, 3] 106 | T3 is located at [75, 82, 19, 4] 107 | T4 is located at [36, 60, 14, 3] 108 | T5 is located at [59, 65, 11, 3] 109 | T6 is located at [31, 24, 12, 2] 110 | T7 is located at [25, 13, 12, 3] 111 | T8 is located at [49, 46, 15, 5] 112 | T9 is located at [61, 38, 10, 5] 113 | T10 is located at [68, 30, 12, 6] 114 | T11 is located at [81, 21, 7, 4] 115 | 116 | Caption: 117 | A diagram showing the Earth going around the sun. It indicates when each equinox or solstice occurs. It also says what circle of latitude the sun is over at each point. 118 | Topic: 119 | astronomy 120 | Diagram Layout: 121 | circular 122 | Required Entities: 123 | "Earth + Revolution = Seasons" text label (T0) 124 | "Vernal Equinox Sun overhead at Equator" text label (T1) 125 | "Summer Solstice Sun overhead at Tropic of Cancer" text label (T2) 126 | "Autumnal Equinox Sun overhead at Equator" text label (T3) 127 | "Winter Solstice Sun overhead at Tropic of Capricorn" text label (T4) 128 | earth image (I0) 129 | earth image (I1) 130 | earth image (I2) 131 | earth image (I3) 132 | sun image (I4) 133 | Entity Relationships: 134 | I4 is between I1 and I3 135 | I1 connects to I2 136 | T2 labels I1 137 | I2 connects to I3 138 | T3 labels I2 139 | T1 labels I0 140 | I3 connects to I0 141 | I0 connects to I1 142 | I4 is between I0 and I2 143 | T4 labels I3 144 | Entity Locations: 145 | T0 is located at [12, 5, 71, 8] 146 | T1 is located at [41, 17, 18, 10] 147 | T2 is located at [1, 48, 20, 10] 148 | T3 is located at [38, 85, 22, 9] 149 | T4 is located at [79, 48, 18, 11] 150 | I0 is located at [45, 28, 9, 10] 151 | I1 is located at [21, 50, 6, 10] 152 | I2 is located at [46, 73, 7, 11] 153 | I3 is located at [72, 50, 7, 10] 154 | I4 is located at [45, 50, 8, 12] 155 | 156 | Caption: 157 | A diagram showing the phases of the moon with the earth in the center, including the full moon, new moon, first quarter, last quarter, and the crescent and gibbous phases. The sun is included on the side of the new moon. 158 | Topic: 159 | astronomy 160 | Diagram Layout: 161 | circular 162 | Required Entities: 163 | moon image (I0) 164 | moon image (I1) 165 | moon image (I2) 166 | moon image (I3) 167 | moon image (I4) 168 | moon image (I5) 169 | moon image (I6) 170 | moon image (I7) 171 | sun image (I8) 172 | "Waning Crescent" text label (T0) 173 | "Last Quarter" text label (T1) 174 | "Waning Gibbous" text label (T2) 175 | "Full Moon" text label (T3) 176 | "Waxing GIbbous" text label (T4) 177 | "First Quarter" text label (T5) 178 | "Waxing Crescent" text label (T6) 179 | "New Moon" text label (T7) 180 | Entity Relationships: 181 | T0 labels I7 182 | I8 left I5 183 | T7 labels I6 184 | I8 left I4 185 | I8 left I2 186 | T2 labels I1 187 | I8 left I1 188 | T3 labels I2 189 | T1 labels I0 190 | T5 labels I4 191 | I8 left I0 192 | I8 left I7 193 | T6 labels I5 194 | I8 left I3 195 | I8 left I6 196 | T4 labels I3 197 | Entity Locations: 198 | I0 is located at [47, 14, 12, 15] 199 | I1 is located at [68, 21, 11, 16] 200 | I2 is located at [79, 42, 12, 15] 201 | I3 is located at [68, 66, 12, 15] 202 | I4 is located at [47, 71, 12, 15] 203 | I5 is located at [27, 66, 12, 14] 204 | I6 is located at [16, 42, 11, 15] 205 | I7 is located at [27, 21, 12, 15] 206 | I8 is located at [0, 0, 9, 100] 207 | T0 is located at [18, 10, 11, 7] 208 | T1 is located at [45, 8, 15, 4] 209 | T2 is located at [77, 10, 9, 8] 210 | T3 is located at [86, 31, 12, 4] 211 | T4 is located at [83, 68, 10, 8] 212 | T5 is located at [45, 88, 15, 4] 213 | T6 is located at [14, 69, 12, 6] 214 | T7 is located at [9, 31, 13, 5] 215 | 216 | Caption: 217 | A diagram showing the Perigee and Apogee phases of the moon as it orbits around Earth. 218 | Topic: 219 | astronomy 220 | Diagram Layout: 221 | circular 222 | Required Entities: 223 | moon image (I0) 224 | moon image (I1) 225 | "Moon" text label (T0) 226 | "Earth" text label (T1) 227 | earth image (I2) 228 | "Moon's Orbit" text label (T2) 229 | "Apogee" text label (T3) 230 | "Pergiee" text label (T4) 231 | Entity Relationships: 232 | T0 labels I1 233 | T4 labels I0 234 | T0 labels I0 235 | T3 labels I1 236 | T1 labels I2 237 | I2 is between I0 and I1 238 | I1 has an arrow to I0 239 | I0 has an arrow to I1 240 | Entity Locations: 241 | I0 is located at [4, 40, 8, 14] 242 | I1 is located at [87, 38, 9, 16] 243 | T0 is located at [80, 29, 15, 8] 244 | T1 is located at [29, 61, 16, 7] 245 | I2 is located at [29, 32, 16, 27] 246 | T2 is located at [57, 80, 25, 16] 247 | T3 is located at [74, 51, 14, 9] 248 | T4 is located at [12, 36, 14, 8] 249 | 250 | Caption: 251 | A diagram showing the Earth revolving around the sun at four points. 252 | Topic: 253 | astronomy 254 | Diagram Layout: 255 | circular 256 | Required Entities: 257 | earth image (I0) 258 | earth image (I1) 259 | earth image (I2) 260 | earth image (I3) 261 | sun image (I4) 262 | "Sun" text label (T0) 263 | Entity Relationships: 264 | T0 labels I4 265 | I4 is between I1 and I3 266 | I4 is between I0 and I2 267 | Entity Locations: 268 | I0 is located at [44, 18, 9, 11] 269 | I1 is located at [65, 42, 9, 12] 270 | I2 is located at [44, 68, 9, 12] 271 | I3 is located at [22, 44, 10, 11] 272 | I4 is located at [41, 38, 17, 21] 273 | T0 is located at [45, 45, 7, 5] 274 | 275 | Caption: 276 | A diagram showing the Earth revolving around the sun. The solstices and equinoxes are labeled at each Earth phase in the rotation. There are also labels indicating which hemispheres get the most solar energy at each Earth phase. 277 | Topic: 278 | astronomy 279 | Diagram Layout: 280 | circular 281 | Required Entities: 282 | earth image (I0) 283 | earth image (I1) 284 | earth image (I2) 285 | earth image (I3) 286 | "Vernal Equinox March 21-22 Incoming solar energy equal in both hemispheres" text label (T0) 287 | "Summer Solstice June 21-22 Incoming solar energy greatest in Northern Hemisphere" text label (T1) 288 | "Autumnal Equinox September 22-23 Incoming solar energy equal in both hemispheres" text label (T2) 289 | "Winter Solstice December 21-22 Incoming solar energy greatest in Southern Hemisphere" text label (T3) 290 | "Orbit" text label (T4) 291 | "Sun" text label (T5) 292 | earth image (I4) 293 | Entity Relationships: 294 | I1 has an arrow to I2 295 | I4 is between I1 and I3 296 | T3 labels I3 297 | I3 has an arrow to I0 298 | T0 labels I0 299 | T2 labels I2 300 | T5 labels I4 301 | T1 labels I1 302 | I4 is between I0 and I2 303 | I0 has an arrow to I1 304 | I2 has an arrow to I3 305 | Entity Locations: 306 | I0 is located at [39, 11, 17, 21] 307 | I1 is located at [3, 41, 18, 21] 308 | I2 is located at [38, 74, 18, 22] 309 | I3 is located at [78, 41, 17, 20] 310 | T0 is located at [57, 1, 22, 22] 311 | T1 is located at [0, 65, 19, 25] 312 | T2 is located at [58, 82, 29, 18] 313 | T3 is located at [80, 65, 19, 24] 314 | T4 is located at [24, 71, 7, 3] 315 | T5 is located at [45, 50, 6, 3] 316 | I4 is located at [40, 42, 16, 18] 317 | 318 | Caption: 319 | A diagram showing the relative the Sun, Moon, and Earth during a total lunar eclipse. It also has the umbra and penumbra labels. 320 | Topic: 321 | astronomy 322 | Diagram Layout: 323 | rows 324 | Required Entities: 325 | "Sun" text label (T0) 326 | "Earth" text label (T1) 327 | "Moon" text label (T2) 328 | "Total Lunar Eclipse" text label (T3) 329 | "Umbra" text label (T4) 330 | "Penumbra" text label (T5) 331 | sun image (I0) 332 | earth image (I1) 333 | moon image (I2) 334 | Entity Relationships: 335 | T0 labels I0 336 | I1 is between I0 and I2 337 | T2 labels I2 338 | T1 label I1 339 | Entity Locations: 340 | T0 is located at [13, 92, 4, 6] 341 | T1 is located at [66, 91, 6, 6] 342 | T2 is located at [79, 92, 7, 7] 343 | T3 is located at [40, 3, 19, 7] 344 | T4 is located at [70, 10, 7, 6] 345 | T5 is located at [88, 25, 11, 7] 346 | I0 is located at [1, 14, 29, 69] 347 | I1 is located at [64, 38, 9, 22] 348 | I2 is located at [80, 44, 4, 10] 349 | 350 | Caption: 351 | A diagram showing the alignment of the Sun, the Moon, and the Earth at various points through out the moon's rotation around Earth. 352 | Topic: 353 | astronomy 354 | Diagram Layout: 355 | tree 356 | Required Entities: 357 | sun image (I0) 358 | earth image (I1) 359 | moon image (I2) 360 | earth image (I3) 361 | moon image (I4) 362 | earth image (I5) 363 | moon image (I6) 364 | earth image (I7) 365 | moon image (I8) 366 | "Similarly, the alignment of the Sun, Moon, and Earth during a new moon results in spring tides." text label (T0) 367 | "New moon" text label (T1) 368 | "Gravitational pull of Sun and Moon" text label (T2) 369 | "Spring tide" text label (T3) 370 | Entity Relationships: 371 | I5 has an arrow to I0 372 | I0 left I1 373 | I0 has an arrow to I5 374 | I0 has an arrow to I3 375 | I8 below I7 376 | I3 has an arrow to I0 377 | I0 has an arrow to I7 378 | I0 left I7 379 | I0 left I3 380 | I2 left I1 381 | I6 right I5 382 | I4 above I3 383 | I1 has an arrow to I0 384 | I0 has an arrow to I1 385 | I7 has an arrow to I0 386 | I0 left I5 387 | Entity Locations: 388 | I0 is located at [16, 11, 25, 62] 389 | I1 is located at [72, 3, 13, 14] 390 | I2 is located at [69, 11, 4, 7] 391 | I3 is located at [78, 22, 11, 20] 392 | I4 is located at [78, 17, 6, 5] 393 | I5 is located at [77, 45, 14, 13] 394 | I6 is located at [91, 50, 4, 7] 395 | I7 is located at [75, 61, 11, 16] 396 | I8 is located at [75, 79, 6, 5] 397 | T0 is located at [5, 86, 92, 12] 398 | T1 is located at [57, 26, 12, 5] 399 | T2 is located at [16, 4, 17, 5] 400 | T3 is located at [34, 14, 18, 7] 401 | 402 | Caption: 403 | """ 404 | 405 | AUDITOR_PROMPT_astronomy = """Given a caption and a layout of a diagram, you should determine if something is wrong in the diagram based on the caption. You should explain your answer. Think step-by-step as to why the diagram is correct or not. 406 | The diagram will be described in terms of entities in the diagram, the relationships between the entities, and the location of each entity. 407 | An entity can be an image or text. Entity locations will be in [x, y, width, height] format, where 0,0 is the top left corner and 100,100 is max image size. 408 | 409 | Here are some rules the diagrams should follow: 410 | All numbers should be positive, no negative numbers. 411 | Entities should not be outside the bounds [0, 0, 100, 100]. 412 | The x coordinate + the width of an entity should not exceed 100. 413 | The y coordinate + the height of an entity should not exceed 100. 414 | Entities of the same type should not overlap. 415 | 416 | Here are some examples: 417 | Caption: 418 | A diagram showing where the tides occur on Earth and the angle to the moon. This angle is indicated by the declination of Moon. 419 | Topic: 420 | astronomy 421 | Diagram Layout: 422 | abstract 423 | Required Entities: 424 | earth image (I0) 425 | moon image (I1) 426 | "High Tide" text label (T0) 427 | "Low Tide" text label (T1) 428 | "Declination of Moon" text label (T2) 429 | "Angle" text label (T3) 430 | Entity Relationships: 431 | T3 labels I0 and I1 432 | T1 labels I0 433 | T0 labels I0 434 | T2 labels I1 435 | Entity Locations: 436 | I0 is located at [20, 30, 30, 40] 437 | I1 is located at [70, 30, 20, 20] 438 | T0 is located at [10, 20, 10, 5] 439 | T1 is located at [10, 75, 10, 5] 440 | T2 is located at [70, 10, 20, 5] 441 | T3 is located at [50, 50, 10, 5] 442 | What is wrong with this diagram? 443 | The declination of the moon text should be inbetween the moon and the earth. The high tide text should be on the side of earth that is facing the moon and low tide should be on the other side. The tidal bulge location should also be marked. 444 | 445 | Caption: 446 | A diagram showing the phases of the moon as it revolves around Earth, including the first quarter, waxing crescent, waning crescent, full moon, new moon, wanning crescent, and waxing crescent. It also indicates where the sunlight is coming from. 447 | Topic: 448 | astronomy 449 | Diagram Layout: 450 | circular 451 | Required Entities: 452 | moon image (I0) 453 | moon image (I1) 454 | moon image (I2) 455 | moon image (I3) 456 | moon image (I4) 457 | moon image (I5) 458 | moon image (I6) 459 | moon image (I7) 460 | "First Quarter" text label (T0) 461 | "Waxing Crescent" text label (T1) 462 | "Waning Crescent" text label (T2) 463 | "Full Moon" text label (T3) 464 | "New Moon" text label (T4) 465 | "Waning Crescent" text label (T5) 466 | "Waxing Crescent" text label (T6) 467 | "Sunlight" text label (T7) 468 | Entity Relationships: 469 | T5 labels I5 470 | T6 labels I6 471 | T2 labels I2 472 | T7 points to I7 473 | T1 labels I1 474 | T3 labels I3 475 | T4 labels I4 476 | T0 labels I0 477 | Entity Locations: 478 | I0 is located at [20, 50, 10, 15] 479 | I1 is located at [35, 30, 10, 15] 480 | I2 is located at [50, 20, 10, 15] 481 | I3 is located at [65, 30, 10, 15] 482 | I4 is located at [80, 50, 10, 15] 483 | I5 is located at [65, 70, 10, 15] 484 | I6 is located at [50, 80, 10, 15] 485 | I7 is located at [35, 70, 10, 15] 486 | T0 is located at [15, 45, 15, 5] 487 | T1 is located at [30, 25, 15, 5] 488 | T2 is located at [45, 15, 15, 5] 489 | T3 is located at [60, 25, 15, 5] 490 | T4 is located at [75, 45, 15, 5] 491 | T5 is located at [60, 65, 15, 5] 492 | T6 is located at [45, 85, 15, 5] 493 | T7 is located at [30, 65, 15, 5] 494 | What is wrong with this diagram? 495 | There should be an earth in the center of the moons. The moon phases are out of order, it should go from new moon to waxing crescent to first quarter to waxing gibbous to full moon to waning gibbous to last quarter to waning crescent. The sunlight label should be to the side of the diagram where the new moon is. 496 | 497 | Caption: 498 | A diagram showing the phases of the moon as it revolves around the Earth, including the full moon, new moon, first quarter, last quarter, waxing crescent, waning crescent, waxing gibbous, and waning gibbous. It also indicates which day of the month each moon phase occurs. 499 | Topic: 500 | astronomy 501 | Diagram Layout: 502 | circular 503 | Required Entities: 504 | moon image (I0) 505 | moon image (I1) 506 | moon image (I2) 507 | moon image (I3) 508 | moon image (I4) 509 | moon image (I5) 510 | moon image (I6) 511 | moon image (I7) 512 | earth image (I8) 513 | "Full Moon (Day 15)" text label (T0) 514 | "New Moon (Day 1)" text label (T1) 515 | "First Quarter (Day 7)" text label (T2) 516 | "Last Quarter (Day 22)" text label (T3) 517 | "Waxing Crescent (Day 3)" text label (T4) 518 | "Waning Crescent (Day 27)" text label (T5) 519 | "Waxing Gibbous (Day 11)" text label (T6) 520 | "Waning Gibbous (Day 18)" text label (T7) 521 | Entity Relationships: 522 | T5 labels I5 523 | T6 labels I6 524 | T2 labels I2 525 | T7 labels I7 526 | T1 labels I1 527 | T3 labels I3 528 | T4 labels I4 529 | T0 labels I0 530 | Entity Locations: 531 | I0 is located at [50, 50, 10, 10] 532 | I1 is located at [75, 50, 10, 10] 533 | I2 is located at [50, 75, 10, 10] 534 | I3 is located at [25, 50, 10, 10] 535 | I4 is located at [65, 65, 10, 10] 536 | I5 is located at [35, 35, 10, 10] 537 | I6 is located at [65, 35, 10, 10] 538 | I7 is located at [35, 65, 10, 10] 539 | I8 is located at [50, 50, 20, 20] 540 | T0 is located at [52, 52, 15, 5] 541 | T1 is located at [77, 52, 15, 5] 542 | T2 is located at [52, 77, 15, 5] 543 | T3 is located at [27, 52, 15, 5] 544 | T4 is located at [67, 67, 15, 5] 545 | T5 is located at [37, 37, 15, 5] 546 | T6 is located at [67, 37, 15, 5] 547 | T7 is located at [37, 67, 15, 5] 548 | What is wrong with this diagram? 549 | The moon phases are out of order, it should go from new moon to waxing crescent to first quarter to waxing gibbous to full moon to waning gibbous to last quarter to waning crescent. The text labels should be further away from their images so there do not overlap. The only object in the center should be an image of the earth, there should not be any moons in the center. 550 | 551 | Caption: 552 | A diagram showing the rotation axis of the Earth, marking the positions of the North Pole and the South Pole. It indicates where on earth long days, short days, equal days occur if the sunlight is coming in from the side. It also indicates where on earth the sun never sets and where the sun never rises. 553 | Topic: 554 | astronomy 555 | Diagram Layout: 556 | abstract 557 | Required Entities: 558 | earth image (I0) 559 | "North Pole" text label (T1) 560 | "South Pole" text label (T2) 561 | "Long Days" text label (T3) 562 | "Short Days" text label (T4) 563 | "Equal Days" text label (T5) 564 | "Sun Never Sets" text label (T6) 565 | "Sun Never Rises" text label (T7) 566 | "Sunlight" text label (T8) 567 | "Rotation Axis" text label (T9) 568 | Entity Relationships: 569 | T3 labels I0 570 | T9 labels I0 571 | T4 labels I0 572 | T6 labels I0 573 | T7 labels I0 574 | T1 labels I0 575 | T5 labels I0 576 | T2 labels I0 577 | T8 labels I0 578 | Entity Locations: 579 | I0 is located at [30, 20, 40, 60] 580 | T1 is located at [35, 10, 10, 5] 581 | T2 is located at [35, 80, 10, 5] 582 | T3 is located at [10, 30, 10, 5] 583 | T4 is located at [80, 30, 10, 5] 584 | T5 is located at [45, 10, 10, 5] 585 | T6 is located at [10, 60, 10, 5] 586 | T7 is located at [80, 60, 10, 5] 587 | T8 is located at [45, 80, 10, 5] 588 | T9 is located at [30, 50, 10, 5] 589 | What is wrong with this diagram? 590 | The sunlight should either be on the right or left side of the diagram, not in the bottom. Short days and longs should be on the same left or right side, but opposite top and bottom sides. 591 | 592 | Caption: 593 | A diagram showing all eight planets in the solar system in order, starting with Mercury and ending with Neptune. It also indicates the axial tilt of each planet. 594 | Topic: 595 | astronomy 596 | Diagram Layout: 597 | linear 598 | Required Entities: 599 | "Mercury" text label (T0) 600 | "Venus" text label (T1) 601 | "Earth" text label (T2) 602 | "Mars" text label (T3) 603 | "Jupiter" text label (T4) 604 | "Saturn" text label (T5) 605 | "Uranus" text label (T6) 606 | "Neptune" text label (T7) 607 | mercury image (I0) 608 | venus image (I1) 609 | earth image (I2) 610 | mars image (I3) 611 | jupiter image (I4) 612 | saturn image (I5) 613 | uranus image (I6) 614 | neptune image (I7) 615 | "axial tilt" text label (T8) 616 | Entity Relationships: 617 | T5 labels I5 618 | T6 labels I6 619 | T2 labels I2 620 | T7 labels I7 621 | T1 labels I1 622 | T3 labels I3 623 | T4 labels I4 624 | T0 labels I0 625 | Entity Locations: 626 | T0 is located at [5, 5, 10, 5] 627 | T1 is located at [20, 5, 10, 5] 628 | T2 is located at [35, 5, 10, 5] 629 | T3 is located at [50, 5, 10, 5] 630 | T4 is located at [65, 5, 10, 5] 631 | T5 is located at [80, 5, 10, 5] 632 | T6 is located at [95, 5, 10, 5] 633 | T7 is located at [110, 5, 10, 5] 634 | I0 is located at [5, 15, 10, 10] 635 | I1 is located at [20, 15, 10, 10] 636 | I2 is located at [35, 15, 10, 10] 637 | I3 is located at [50, 15, 10, 10] 638 | I4 is located at [65, 15, 10, 10] 639 | I5 is located at [80, 15, 10, 10] 640 | I6 is located at [95, 15, 10, 10] 641 | I7 is located at [110, 15, 10, 10] 642 | T8 is located at [60, 30, 20, 5] 643 | What is wrong with this diagram? 644 | The axial tilt of each planet should be shown in the diagram. For example, mercury has an axial tilt of 0.01 and neptune has an axial tilt of 28.32. 645 | 646 | Caption: 647 | A diagram showing the four phases of the moon side by side along with the dates in January when each phase occurs. It shows waxing crescent, waxing gibbous, new moon, and full moon. 648 | Topic: 649 | astronomy 650 | Diagram Layout: 651 | rows 652 | Required Entities: 653 | moon image (I0) 654 | moon image (I1) 655 | moon image (I2) 656 | moon image (I3) 657 | moon image (I4) 658 | moon image (I5) 659 | moon image (I6) 660 | moon image (I7) 661 | "New Moon January 1" text label (T0) 662 | "Waxing Crescent January 5" text label (T1) 663 | "First Quarter January 9" text label (T2) 664 | "Waxing Gibbous January 13" text label (T3) 665 | "Full Moon January 17" text label (T4) 666 | "Waning Gibbous January 21" text label (T5) 667 | "Last Quarter January 25" text label (T6) 668 | "Waning Crescent January 29" text label (T7) 669 | Entity Relationships: 670 | T5 labels I5 671 | T6 labels I6 672 | T2 labels I2 673 | T7 labels I7 674 | T1 labels I1 675 | T3 labels I3 676 | T4 labels I4 677 | T0 labels I0 678 | Entity Locations: 679 | I0 is located at [1, 10, 12, 20] 680 | I1 is located at [15, 10, 12, 20] 681 | I2 is located at [29, 10, 12, 20] 682 | I3 is located at [43, 10, 12, 20] 683 | I4 is located at [57, 10, 12, 20] 684 | I5 is located at [71, 10, 12, 20] 685 | I6 is located at [85, 10, 12, 20] 686 | I7 is located at [99, 10, 12, 20] 687 | T0 is located at [1, 32, 12, 5] 688 | T1 is located at [15, 32, 18, 5] 689 | T2 is located at [29, 32, 16, 5] 690 | T3 is located at [43, 32, 18, 5] 691 | T4 is located at [57, 32, 12, 5] 692 | T5 is located at [71, 32, 18, 5] 693 | T6 is located at [85, 32, 14, 5] 694 | T7 is located at [99, 32, 18, 5] 695 | What is wrong with this diagram? 696 | The order of the moons should be waxing crescent, waxing gibbous, then new moon, and then full moon. Other moon phases should be skipped. 697 | 698 | Caption: 699 | A diagram showing the phases of the moon as it revolves around Earth, including waxing, full, waning, and quarter moons. It also has the sun on the side of the diagram to indicate where the light is coming from. 700 | Topic: 701 | astronomy 702 | Diagram Layout: 703 | circular 704 | Required Entities: 705 | moon image (I0) 706 | moon image (I1) 707 | moon image (I2) 708 | moon image (I3) 709 | moon image (I4) 710 | moon image (I5) 711 | moon image (I6) 712 | moon image (I7) 713 | sun image (I8) 714 | "Waxing Crescent" text label (T0) 715 | "First Quarter" text label (T1) 716 | "Waxing Gibbous" text label (T2) 717 | "Full Moon" text label (T3) 718 | "Waning Gibbous" text label (T4) 719 | "Last Quarter" text label (T5) 720 | "Waning Crescent" text label (T6) 721 | "New Moon" text label (T7) 722 | Entity Relationships: 723 | T5 labels I4 724 | T3 labels I2 725 | T0 labels I7 726 | T2 labels I1 727 | T1 labels I0 728 | T6 labels I5 729 | T4 labels I3 730 | T7 labels I6 731 | Entity Locations: 732 | I0 is located at [47, 14, 12, 15] 733 | I1 is located at [68, 21, 11, 16] 734 | I2 is located at [79, 42, 12, 15] 735 | I3 is located at [68, 66, 12, 15] 736 | I4 is located at [47, 71, 12, 15] 737 | I5 is located at [27, 66, 12, 14] 738 | I6 is located at [16, 42, 11, 15] 739 | I7 is located at [27, 21, 12, 15] 740 | I8 is located at [0, 0, 9, 100] 741 | T0 is located at [18, 10, 11, 7] 742 | T1 is located at [45, 8, 15, 4] 743 | T2 is located at [77, 10, 9, 8] 744 | T3 is located at [86, 31, 12, 4] 745 | T4 is located at [83, 68, 10, 8] 746 | T5 is located at [45, 88, 15, 4] 747 | T6 is located at [14, 69, 12, 6] 748 | T7 is located at [9, 31, 13, 5] 749 | What is wrong with this diagram? 750 | There should be an earth in the center of the moons. There should also be a sun on the side of the diagram where the new moon is, to indicated where the sunlight is coming from. 751 | 752 | Caption: 753 | A diagram showing the positions of the Earth as it orbits the sun during different seasons. It also indicates if there is an equinox or solstice at each point and which day in the year it occurs. 754 | Topic: 755 | astronomy 756 | Diagram Layout: 757 | circular 758 | Required Entities: 759 | earth image (I0) 760 | earth image (I1) 761 | earth image (I2) 762 | earth image (I3) 763 | sun image (I4) 764 | "Vernal Equinox March 20" text label (T0) 765 | "Summer Solstice June 21" text label (T1) 766 | "Autumnal Equinox September 22" text label (T2) 767 | "Winter Solstice December 21" text label (T3) 768 | Entity Relationships: 769 | T2 labels I2 770 | I4 is between I0 and I2 771 | T1 labels I1 772 | T3 labels I3 773 | T0 labels I0 774 | I4 is between I1 and I3 775 | Entity Locations: 776 | I0 is located at [45, 18, 9, 11] 777 | I1 is located at [65, 42, 9, 12] 778 | I2 is located at [44, 68, 9, 12] 779 | I3 is located at [22, 44, 10, 11] 780 | I4 is located at [41, 38, 17, 21] 781 | T0 is located at [45, 10, 15, 5] 782 | T1 is located at [70, 40, 15, 5] 783 | T2 is located at [45, 80, 20, 5] 784 | T3 is located at [15, 40, 15, 5] 785 | What is wrong with this diagram? 786 | The text should be placed a little further away from the images they label so there is no overlap. The text labels should also be slightly larger. The earth images should also have arrows pointing between them in the order of the months. 787 | 788 | Caption: 789 | A diagram showing the seasons as the Earth orbits around the Sun. It illustrates the different seasons in each hemisphere at each equinox and solstice. 790 | Topic: 791 | astronomy 792 | Diagram Layout: 793 | circular 794 | Required Entities: 795 | earth image (I0) 796 | earth image (I1) 797 | earth image (I2) 798 | earth image (I3) 799 | sun image (I4) 800 | "Spring Equinox" text label (T0) 801 | "Summer Solstice" text label (T1) 802 | "Autumn Equinox" text label (T2) 803 | "Winter Solstice" text label (T3) 804 | "Northern Hemisphere" text label (T4) 805 | "Southern Hemisphere" text label (T5) 806 | Entity Relationships: 807 | I1 has an arrow to I0 808 | I3 has an arrow to I2 809 | I0 has an arrow to I3 810 | T2 labels I2 811 | T5 labels I2 and I3 812 | I2 has an arrow to I1 813 | T1 labels I1 814 | T3 labels I3 815 | T4 labels I0 and I1 816 | T0 labels I0 817 | Entity Locations: 818 | I0 is located at [8, 37, 17, 28] 819 | I1 is located at [43, 17, 14, 20] 820 | I2 is located at [75, 37, 17, 26] 821 | I3 is located at [40, 57, 20, 30] 822 | I4 is located at [48, 47, 3, 6] 823 | T0 is located at [8, 30, 17, 7] 824 | T1 is located at [43, 10, 14, 7] 825 | T2 is located at [75, 30, 17, 7] 826 | T3 is located at [40, 50, 20, 7] 827 | T4 is located at [25, 5, 20, 7] 828 | T5 is located at [25, 88, 20, 7] 829 | What is wrong with this diagram? 830 | The earths should be the same size. The sun should be a little bigger. The seasons should be labled in each hemisphere for each earth image. For example, in the bottom earth that covers winter solstice, it should say northern winter and southern summer to indicate the season in each hemisphere. 831 | 832 | """ 833 | 834 | DIAGRAM_PROMPT_biology = """Given a caption of a diagram and topic, generate the diagram layout, and then a list of required entities that would be needed to create the described diagram. Then generate a list of the relationships between the entities (i.e. which ones are connected or labeling each other). Finally, generate the location of each entity. 835 | An entity can be an image or text. Entity locations should be generated in [x, y, width, height] format, where 0,0 is the top left corner and 100,100 is max image size. 836 | 837 | Think step-by-step, break each caption into parts and generate the required entities, relationships, and locations for each part. 838 | 839 | Here are some rules to follow: 840 | All numbers should be positive, do not generate negative numbers. 841 | Please always generate a list of entities, even if the list is empty. 842 | Entities should not be outside the bounds [0, 0, 100, 100]. 843 | The x coordinate + the width of an entity should not exceed 100. 844 | The y coordinate + the height of an entity should not exceed 100. 845 | Entities of the same type should not overlap. 846 | 847 | Here are some examples: 848 | 849 | Caption: 850 | A diagram showing a food chain, illustrating the relationships among various animals and their connections to each other through feeding habits. It includes dragonfly, frog, snake, buzzard, fox, ladybird, butterfly, greenfly, titmouse, mouse, rabbit, plantain, grasshopper, and berries. 851 | Topic: 852 | biology 853 | Diagram Layout: 854 | tree 855 | Required Entities: 856 | dragonfly image (I0) 857 | buzzard image (I1) 858 | fox image (I2) 859 | frog image (I3) 860 | snake image (I4) 861 | ladybird image (I5) 862 | mouse image (I6) 863 | titmouse image (I7) 864 | greenfly image (I8) 865 | butterfly image (I9) 866 | rabbit image (I10) 867 | grasshopper image (I11) 868 | berries image (I12) 869 | plantain image (I13) 870 | "dragonfly" text label (T0) 871 | "buzzard" text label (T1) 872 | "fox" text label (T2) 873 | "frog" text label (T3) 874 | "snake" text label (T4) 875 | "ladybird" text label (T5) 876 | "mouse" text label (T6) 877 | "titmouse" text label (T7) 878 | "greenfly" text label (T8) 879 | "butterfly" text label (T9) 880 | "rabbit" text label (T10) 881 | "grasshopper" text label (T11) 882 | "berries" text label (T12) 883 | "plantain" text label (T13) 884 | Entity Relationships: 885 | T13 labels I13 886 | I13 has an arrow to I11 887 | I3 has an arrow to I4 888 | T0 labels I0 889 | I4 has an arrow to I1 890 | T1 labels I1 891 | T2 labels I2 892 | I6 has an arrow to I1 893 | T5 labels I5 894 | T12 labels I12 895 | I7 has an arrow to I4 896 | I13 has an arrow to I10 897 | I11 has an arrow to I3 898 | I7 has an arrow to I2 899 | I12 has an arrow to I7 900 | T6 labels I6 901 | T11 labels I11 902 | I12 has an arrow to I8 903 | I5 has an arrow to I0 904 | T9 labels I9 905 | I10 has an arrow to I2 906 | I6 has an arrow to I2 907 | T4 labels I4 908 | T7 labels I7 909 | I12 has an arrow to I11 910 | I5 has an arrow to I7 911 | I7 has an arrow to I1 912 | T10 labels I10 913 | I9 has an arrow to I3 914 | I13 has an arrow to I6 915 | I8 has an arrow to I5 916 | T8 labels I8 917 | I10 has an arrow to I1 918 | I12 has an arrow to I9 919 | I0 has an arrow to I3 920 | T3 labels I3 921 | Entity Locations: 922 | I0 is located at [9, 7, 13, 10] 923 | I1 is located at [60, 0, 10, 19] 924 | I2 is located at [75, 5, 24, 15] 925 | I3 is located at [24, 20, 13, 12] 926 | I4 is located at [43, 27, 12, 13] 927 | I5 is located at [9, 35, 10, 9] 928 | I6 is located at [67, 39, 13, 16] 929 | I7 is located at [45, 46, 14, 11] 930 | I8 is located at [7, 56, 12, 9] 931 | I9 is located at [24, 53, 17, 10] 932 | I10 is located at [83, 48, 13, 14] 933 | I11 is located at [35, 69, 12, 8] 934 | I12 is located at [9, 84, 16, 11] 935 | I13 is located at [59, 75, 11, 21] 936 | T0 is located at [8, 2, 13, 6] 937 | T1 is located at [50, 2, 11, 5] 938 | T2 is located at [85, 2, 9, 3] 939 | T3 is located at [29, 16, 8, 4] 940 | T4 is located at [42, 23, 9, 4] 941 | T5 is located at [0, 30, 12, 6] 942 | T6 is located at [60, 47, 9, 4] 943 | T7 is located at [46, 58, 14, 5] 944 | T8 is located at [1, 64, 12, 5] 945 | T9 is located at [17, 50, 11, 4] 946 | T10 is located at [88, 61, 9, 4] 947 | T11 is located at [32, 78, 17, 5] 948 | T12 is located at [12, 96, 11, 4] 949 | T13 is located at [69, 92, 11, 4] 950 | 951 | Caption: 952 | A diagram showing the life cycle of a butterfly, going from the stage of an egg, to a caterpillar, to a cocoon, to an adult butterfly. 953 | Topic: 954 | biology 955 | Diagram Layout: 956 | circular 957 | Required Entities: 958 | adult butterfly image (I0) 959 | egg image (I1) 960 | caterpillar image (I2) 961 | chrysalis image (I3) 962 | "adult" text label (T0) 963 | "egg" text label (T1) 964 | "caterpillar" text label (T2) 965 | "chrysalis" text label (T3) 966 | Entity Relationships: 967 | T2 labels I2 968 | T1 labels I1 969 | T0 labels I0 970 | T3 labels I3 971 | Entity Locations: 972 | I0 is located at [43, 5, 22, 30] 973 | I1 is located at [72, 54, 9, 6] 974 | I2 is located at [50, 82, 26, 8] 975 | I3 is located at [24, 46, 10, 29] 976 | T0 is located at [68, 16, 18, 9] 977 | T1 is located at [82, 54, 12, 6] 978 | T2 is located at [52, 91, 22, 7] 979 | T3 is located at [3, 77, 31, 9] 980 | 981 | Caption: 982 | A diagram showing the cycle of a Take-all Gaeumannomyces graminis infection a plant. It first causes a severe infection which results in stunted patches and a rat tail appearance of roots and whiteheads. Then it overwinters as mycelium on roots and stem bases of infected plants. Then it makes a runner hyphase on roots. Then the cycle repeats. 983 | Topic: 984 | biology 985 | Diagram Layout: 986 | circular 987 | Required Entities: 988 | "take-all gaeumannomyces graminis" text label (T0) 989 | "severe infection results in stunted patches, 'rat tail' appearance of roots and whiteheads" text label (T1) 990 | "whitehead" text label (T2) 991 | "fungus overwinters as mycelium on roots and stem bases of infected plants, spreading to volunteers and autumn sown crops." text label (T3) 992 | "runner hyphae on roots" text label (T4) 993 | plant with roots image (I0) 994 | roots image (I1) 995 | roots image (I2) 996 | whitehead plant image (I3) 997 | Entity Relationships: 998 | T3 has an arrow to I2 999 | I1 has an arrow to I0 1000 | T1 has an arrow to T3 1001 | T2 labels I3 1002 | I0 has an arrow to T1 1003 | T3 has an arrow to I1 1004 | T4 labels I1 1005 | Entity Locations: 1006 | T0 is located at [20, 0, 52, 13] 1007 | T1 is located at [39, 19, 61, 10] 1008 | T2 is located at [81, 66, 15, 6] 1009 | T3 is located at [39, 81, 61, 16] 1010 | T4 is located at [0, 87, 33, 5] 1011 | I0 is located at [9, 28, 16, 31] 1012 | I1 is located at [4, 60, 13, 25] 1013 | I2 is located at [48, 45, 14, 26] 1014 | I3 is located at [87, 33, 6, 30] 1015 | 1016 | Caption: 1017 | A diagram showing the life cycle of a cockroach, illustrating the different stages from egg to nymphs to adult. 1018 | Topic: 1019 | biology 1020 | Diagram Layout: 1021 | circular 1022 | Required Entities: 1023 | adult bug image (I0) 1024 | egg image (I1) 1025 | nymph bug image (I2) 1026 | nymph bug image (I3) 1027 | "adult 10-20 days" text label (T0) 1028 | "egg 1-8 days" text label (T1) 1029 | "nymphs 14 days" text label (T2) 1030 | Entity Relationships: 1031 | T0 labels I0 1032 | I2 has an arrow to I3 1033 | T2 labels I2 1034 | I0 has an arrow to I1 1035 | I3 has an arrow to I0 1036 | T2 labels I3 1037 | T1 labels I1 1038 | I1 has an arrow to I2 1039 | Entity Locations: 1040 | I0 is located at [35, 0, 40, 41] 1041 | I1 is located at [94, 54, 6, 14] 1042 | I2 is located at [46, 80, 20, 20] 1043 | I3 is located at [0, 44, 28, 28] 1044 | T0 is located at [49, 51, 14, 11] 1045 | T1 is located at [84, 57, 9, 11] 1046 | T2 is located at [29, 71, 20, 9] 1047 | 1048 | Caption: 1049 | A diagram showing the life cycle of a grasshopper, including the wingless nymph, eggs in ground, and winged adult stages. 1050 | Topic: 1051 | biology 1052 | Diagram Layout: 1053 | circular 1054 | Required Entities: 1055 | "wingless nymph" text label (T0) 1056 | "winged adult" text label (T1) 1057 | "eggs in ground" text label (T2) 1058 | "grasshopper lifecycle" text label (T3) 1059 | nymph grasshopper image (I0) 1060 | adult grasshopper image (I1) 1061 | egg image (I2) 1062 | Entity Relationships: 1063 | T2 labels I2 1064 | T1 labels I1 1065 | T0 labels I0 1066 | Entity Locations: 1067 | T0 is located at [45, 3, 20, 9] 1068 | T1 is located at [74, 73, 26, 8] 1069 | T2 is located at [0, 43, 15, 13] 1070 | T3 is located at [22, 92, 49, 8] 1071 | I0 is located at [32, 13, 38, 26] 1072 | I1 is located at [29, 51, 45, 37] 1073 | I2 is located at [16, 39, 12, 20] 1074 | 1075 | Caption: 1076 | A diagram showing a sweetpotato whitefly, greenhouse whitefly, and a bandedwing whitefly. 1077 | Topic: 1078 | biology 1079 | Diagram Layout: 1080 | rows 1081 | Required Entities: 1082 | "sweetpotato whitefly" text label (T0) 1083 | "greenhouse whitefly" text label (T1) 1084 | "bandedwing whitefly" text label (T2) 1085 | fly image (I0) 1086 | fly image (I1) 1087 | fly image (I2) 1088 | Entity Relationships: 1089 | T2 labels I2 1090 | T1 labels I1 1091 | T0 labels I0 1092 | Entity Locations: 1093 | T0 is located at [13, 69, 23, 11] 1094 | T1 is located at [38, 69, 22, 11] 1095 | T2 is located at [64, 69, 21, 11] 1096 | I0 is located at [17, 18, 20, 48] 1097 | I1 is located at [37, 17, 22, 50] 1098 | I2 is located at [60, 17, 24, 49] 1099 | 1100 | Caption: 1101 | A diagram showing the Ladybug Life Cycle from an egg, to a larva, to a pupa, and then an adult. 1102 | Topic: 1103 | biology 1104 | Diagram Layout: 1105 | circular 1106 | Required Entities: 1107 | egg image (I0) 1108 | larva image (I1) 1109 | pupa image (I2) 1110 | adult ladybug image (I3) 1111 | "egg" text label (T0) 1112 | "larva" text label (T1) 1113 | "pupa" text label (T2) 1114 | "adult" text label (T3) 1115 | "ladybug life cycle" text label (T4) 1116 | Entity Relationships: 1117 | T2 labels I2 1118 | T1 labels I1 1119 | T0 labels I0 1120 | T3 labels I3 1121 | Entity Locations: 1122 | I0 is located at [46, 23, 6, 6] 1123 | I1 is located at [71, 39, 16, 20] 1124 | I2 is located at [39, 67, 24, 11] 1125 | I3 is located at [8, 37, 26, 25] 1126 | T0 is located at [43, 31, 10, 4] 1127 | T1 is located at [59, 44, 14, 4] 1128 | T2 is located at [44, 63, 15, 4] 1129 | T3 is located at [34, 46, 13, 4] 1130 | T4 is located at [23, 4, 52, 6] 1131 | 1132 | Caption: 1133 | A diagram showing the food web, which is a series of interconnected relationships between various animals and plants. The web includes, grass, two different plants, a grasshopper, a mouse, a butterfly, a hawk, a snake, a deer, and a wolf. 1134 | Topic: 1135 | biology 1136 | Diagram Layout: 1137 | tree 1138 | Required Entities: 1139 | "food web" text label (T0) 1140 | wolf image (I0) 1141 | deer image (I1) 1142 | snake image (I2) 1143 | mouse image (I3) 1144 | bird image (I4) 1145 | grasshopper image (I5) 1146 | butterfly image (I6) 1147 | grass image (I7) 1148 | plant image (I8) 1149 | plant image (I9) 1150 | Entity Relationships: 1151 | I5 connects to I8 1152 | I6 connects to I9 1153 | I0 connects to I1 1154 | I1 connects to I7 1155 | I3 connects to I7 1156 | I2 connects to I3 1157 | I3 connects to I9 1158 | I2 connects to I5 1159 | I3 connects to I4 1160 | I0 connects to I3 1161 | I5 connects to I7 1162 | I3 connects to I8 1163 | Entity Locations: 1164 | T0 is located at [60, 10, 16, 5] 1165 | I0 is located at [31, 2, 22, 26] 1166 | I1 is located at [1, 13, 19, 39] 1167 | I2 is located at [22, 30, 21, 15] 1168 | I3 is located at [35, 44, 21, 14] 1169 | I4 is located at [61, 23, 18, 18] 1170 | I5 is located at [14, 56, 20, 12] 1171 | I6 is located at [79, 41, 20, 19] 1172 | I7 is located at [7, 74, 23, 23] 1173 | I8 is located at [42, 67, 15, 29] 1174 | I9 is located at [66, 59, 12, 33] 1175 | 1176 | Caption: 1177 | A diagram showing the relationships between different organisms in a food chain. It includes, kingfisher, fish, water flea, algae, water boatman, and dragonfly nymph. 1178 | Topic: 1179 | biology 1180 | Diagram Layout: 1181 | tree 1182 | Required Entities: 1183 | "Kingfisher" text label (T0) 1184 | "Fish" text label (T1) 1185 | "Water flea" text label (T2) 1186 | "Algae" text label (T3) 1187 | "Water boatman" text label (T4) 1188 | "Dragonfly nymph" text label (T5) 1189 | Entity Relationships: 1190 | T1 has an arrow to T5 1191 | T3 has an arrow to T1 1192 | T1 has an arrow to T0 1193 | T2 has an arrow to T4 1194 | T3 has an arrow to T2 1195 | T3 has an arrow to T4 1196 | T2 has an arrow to T1 1197 | T4 has an arrow to T5 1198 | Entity Locations: 1199 | T0 is located at [16, 3, 22, 6] 1200 | T1 is located at [8, 27, 10, 6] 1201 | T2 is located at [50, 26, 23, 6] 1202 | T3 is located at [32, 51, 14, 7] 1203 | T4 is located at [74, 50, 16, 8] 1204 | T5 is located at [51, 88, 19, 9] 1205 | 1206 | Caption: 1207 | A diagram showing the food web of Onondaga Lake, which includes fish eating animals, plants and algae, macroinvertebrates, juveniles, planktivores, and zooplankton. It also indicates how other environmental factors affect the web. 1208 | Topic: 1209 | biology 1210 | Diagram Layout: 1211 | tree 1212 | Required Entities: 1213 | fish-eating animals image (I0) 1214 | game fish image (I1) 1215 | plants and algae image (I2) 1216 | juvenile fish image (I3) 1217 | macroinvertebrates image (I4) 1218 | planktivores image (I5) 1219 | zooplankton image (I6) 1220 | "fish-eating animals" text label (T0) 1221 | "game fish" text label (T1) 1222 | "plants and algae" text label (T2) 1223 | "macroinvertebrates (insects, worms, and snails that live in the bottom sediments)" text label (T3) 1224 | "juveniles" text label (T4) 1225 | "planktivores (fish that eat plankton)" text label (T5) 1226 | "zooplankton" text label (T6) 1227 | "nutrients (such as phosphorus and nitrogen)" text label (T7) 1228 | "sources: urban & rural runoff" text label (T8) 1229 | "sunlight" text label (T9) 1230 | "carbon dioxide (c02)" text label (T10) 1231 | "onondaga lake food web" text label (T11) 1232 | Entity Relationships: 1233 | T0 labels I0 1234 | I6 has an arrow to I5 1235 | T6 labels I6 1236 | T8 has an arrow to T7 1237 | T1 labels I1 1238 | T3 labels I3 1239 | I3 has an arrow to I1 1240 | T5 labels I5 1241 | I2 has an arrow to I3 1242 | T2 labels I2 1243 | I2 has an arrow to I5 1244 | I4 has an arrow to I1 1245 | I2 has an arrow to I6 1246 | I6 has an arrow to I4 1247 | T4 labels I4 1248 | I3 has an arrow to I4 1249 | T7 has an arrow to I2 1250 | I1 has an arrow to I0 1251 | T10 has an arrow to I2 1252 | T9 has an arrow to I2 1253 | Entity Locations: 1254 | I0 is located at [1, 11, 21, 12] 1255 | I1 is located at [2, 35, 31, 14] 1256 | I2 is located at [38, 31, 23, 14] 1257 | I3 is located at [23, 59, 18, 9] 1258 | I4 is located at [13, 72, 12, 17] 1259 | I5 is located at [47, 58, 11, 17] 1260 | I6 is located at [65, 55, 9, 15] 1261 | T0 is located at [22, 14, 14, 4] 1262 | T1 is located at [15, 50, 8, 4] 1263 | T2 is located at [41, 46, 13, 3] 1264 | T3 is located at [7, 89, 19, 9] 1265 | T4 is located at [32, 68, 11, 4] 1266 | T5 is located at [46, 76, 14, 6] 1267 | T6 is located at [71, 70, 15, 9] 1268 | T7 is located at [75, 49, 22, 7] 1269 | T8 is located at [73, 30, 25, 10] 1270 | T9 is located at [58, 24, 10, 5] 1271 | T10 is located at [28, 22, 12, 5] 1272 | T11 is located at [18, 4, 66, 9] 1273 | 1274 | Caption: 1275 | """ 1276 | 1277 | AUDITOR_PROMPT_biology = """Given a caption and a layout of a diagram, you should determine if something is wrong in the diagram based on the caption. You should explain your answer. Think step-by-step as to why the diagram is correct or not. 1278 | The diagram will be described in terms of entities in the diagram, the relationships between the entities, and the location of each entity. 1279 | An entity can be an image or text. Entity locations will be in [x, y, width, height] format, where 0,0 is the top left corner and 100,100 is max image size. 1280 | 1281 | Here are some rules the diagrams should follow: 1282 | All numbers should be positive, no negative numbers. 1283 | Entities should not be outside the bounds [0, 0, 100, 100]. 1284 | The x coordinate + the width of an entity should not exceed 100. 1285 | The y coordinate + the height of an entity should not exceed 100. 1286 | Entities of the same type should not overlap. 1287 | 1288 | Here are some examples: 1289 | Caption: 1290 | A diagram showing the food chain of an owl, starting from a plant (producer), to insects (herbivore/consumer), to a mouse (omnivore/consumer), and finally to an owl (carnivore/consumer). 1291 | Topic: 1292 | biology 1293 | Diagram Layout: 1294 | tree 1295 | Required Entities: 1296 | plant image (I0) 1297 | insect image (I1) 1298 | mouse image (I2) 1299 | owl image (I3) 1300 | "plant (producer)" text label (T0) 1301 | "insects (herbivore/consumer)" text label (T1) 1302 | "mouse (omnivore/consumer)" text label (T2) 1303 | "owl (carnivore/consumer)" text label (T3) 1304 | Entity Relationships: 1305 | I2 has an arrow to I3 1306 | T2 labels I2 1307 | I1 has an arrow to I2 1308 | T1 labels I1 1309 | T3 labels I3 1310 | I0 has an arrow to I1 1311 | T0 labels I0 1312 | Entity Locations: 1313 | I0 is located at [10, 10, 20, 20] 1314 | I1 is located at [40, 30, 20, 20] 1315 | I2 is located at [70, 50, 20, 20] 1316 | I3 is located at [100, 70, 20, 20] 1317 | T0 is located at [5, 5, 30, 5] 1318 | T1 is located at [35, 25, 30, 5] 1319 | T2 is located at [65, 45, 30, 5] 1320 | T3 is located at [95, 65, 30, 5] 1321 | What is wrong with this diagram? 1322 | The chain should be in a straight line. 1323 | 1324 | Caption: 1325 | A diagram showing the food chain in a mountainous area, illustrating the relationships among different animals. It includes a mountain lion, a hawk, a frog, a deer, a snake, a rabbit, a cricket, a mouse, grass, and trees. The arrows in the diagram indicate the flow of energy and nutrients. It also indicates that the snake is a king cobra. 1326 | Topic: 1327 | biology 1328 | Diagram Layout: 1329 | tree 1330 | Required Entities: 1331 | mountain lion image (I0) 1332 | hawk image (I1) 1333 | frog image (I2) 1334 | deer image (I3) 1335 | snake image (I4) 1336 | rabbit image (I5) 1337 | cricket image (I6) 1338 | mouse image (I7) 1339 | grass image (I8) 1340 | tree image (I9) 1341 | "mountain lion" text label (T0) 1342 | "hawk" text label (T1) 1343 | "frog" text label (T2) 1344 | "deer" text label (T3) 1345 | "snake" text label (T4) 1346 | "rabbit" text label (T5) 1347 | "cricket" text label (T6) 1348 | "mouse" text label (T7) 1349 | "grass" text label (T8) 1350 | "trees" text label (T9) 1351 | "king cobra" text label (T10) 1352 | Entity Relationships: 1353 | T2 labels I2 1354 | T4 labels I4 1355 | I6 has an arrow to I2 1356 | T7 labels I7 1357 | T0 labels I0 1358 | T5 labels I5 1359 | I7 has an arrow to I5 1360 | T8 labels I8 1361 | T10 labels I4 1362 | I4 has an arrow to I0 1363 | I9 has an arrow to I6 1364 | I5 has an arrow to I4 1365 | I8 has an arrow to I7 1366 | I3 has an arrow to I0 1367 | T6 labels I6 1368 | T9 labels I9 1369 | I2 has an arrow to I1 1370 | T1 labels I1 1371 | T3 labels I3 1372 | Entity Locations: 1373 | I0 is located at [80, 0, 20, 20] 1374 | I1 is located at [60, 20, 20, 20] 1375 | I2 is located at [40, 40, 20, 20] 1376 | I3 is located at [20, 20, 20, 20] 1377 | I4 is located at [60, 60, 20, 20] 1378 | I5 is located at [40, 80, 20, 20] 1379 | I6 is located at [20, 60, 20, 20] 1380 | I7 is located at [0, 80, 20, 20] 1381 | I8 is located at [0, 40, 20, 20] 1382 | I9 is located at [0, 0, 20, 20] 1383 | T0 is located at [85, 5, 15, 5] 1384 | T1 is located at [65, 25, 10, 5] 1385 | T2 is located at [45, 45, 10, 5] 1386 | T3 is located at [25, 25, 10, 5] 1387 | T4 is located at [65, 65, 10, 5] 1388 | T5 is located at [45, 85, 10, 5] 1389 | T6 is located at [25, 65, 10, 5] 1390 | T7 is located at [5, 85, 10, 5] 1391 | T8 is located at [5, 45, 10, 5] 1392 | T9 is located at [5, 5, 10, 5] 1393 | T10 is located at [65, 75, 15, 5] 1394 | What is wrong with this diagram? 1395 | The snake should have an arrow the to the mountain lion and the king cobra text. The grass and tree images should have an arrow to the deer. The rabbit should have an arrow to the hawk and the mountain lion. The grass should have an arrow the cricket. The mouse should not have an arrow the rabbit, and should instead have an arrow the hawk. 1396 | 1397 | Caption: 1398 | A diagram showing the food chain in a mountainous area, illustrating the relationships among different animals. It includes a mountain lion, a hawk, a frog, a deer, a snake, a rabbit, a cricket, a mouse, grass, and trees. The arrows in the diagram indicate the flow of energy and nutrients. 1399 | Topic: 1400 | biology 1401 | Diagram Layout: 1402 | tree 1403 | Required Entities: 1404 | mountain lion image (I0) 1405 | hawk image (I1) 1406 | frog image (I2) 1407 | deer image (I3) 1408 | snake image (I4) 1409 | rabbit image (I5) 1410 | cricket image (I6) 1411 | mouse image (I7) 1412 | grass image (I8) 1413 | tree image (I9) 1414 | "mountain lion" text label (T0) 1415 | "hawk" text label (T1) 1416 | "frog" text label (T2) 1417 | "deer" text label (T3) 1418 | "snake" text label (T4) 1419 | "rabbit" text label (T5) 1420 | "cricket" text label (T6) 1421 | "mouse" text label (T7) 1422 | "grass" text label (T8) 1423 | "trees" text label (T9) 1424 | Entity Relationships: 1425 | I1 has an arrow to I0 1426 | T2 labels I2 1427 | T4 labels I4 1428 | I4 has an arrow to I1 1429 | I6 has an arrow to I2 1430 | T7 labels I7 1431 | T0 labels I0 1432 | T5 labels I5 1433 | I7 has an arrow to I5 1434 | T8 labels I8 1435 | I5 has an arrow to I3 1436 | I9 has an arrow to I6 1437 | I8 has an arrow to I7 1438 | I3 has an arrow to I0 1439 | I2 has an arrow to I4 1440 | T6 labels I6 1441 | T9 labels I9 1442 | T1 labels I1 1443 | T3 labels I3 1444 | Entity Locations: 1445 | I0 is located at [80, 0, 20, 20] 1446 | I1 is located at [60, 20, 20, 20] 1447 | I2 is located at [40, 40, 20, 20] 1448 | I3 is located at [20, 60, 20, 20] 1449 | I4 is located at [60, 60, 20, 20] 1450 | I5 is located at [0, 80, 20, 20] 1451 | I6 is located at [40, 80, 20, 20] 1452 | I7 is located at [80, 80, 20, 20] 1453 | I8 is located at [20, 100, 20, 20] 1454 | I9 is located at [60, 100, 20, 20] 1455 | T0 is located at [85, 5, 15, 5] 1456 | T1 is located at [65, 25, 15, 5] 1457 | T2 is located at [45, 45, 15, 5] 1458 | T3 is located at [25, 65, 15, 5] 1459 | T4 is located at [65, 65, 15, 5] 1460 | T5 is located at [5, 85, 15, 5] 1461 | T6 is located at [45, 85, 15, 5] 1462 | T7 is located at [85, 85, 15, 5] 1463 | T8 is located at [25, 105, 15, 5] 1464 | T9 is located at [65, 105, 15, 5] 1465 | What is wrong with this diagram? 1466 | The hawk does not need an arrow to the mountain lion. The mouse and the frog need an arrow to the hawk. The trees and the grass need an arrow to the deer. The rabbit needs an arrow to the hawk and the mountain lion, not the mouse. 1467 | 1468 | Caption: 1469 | A diagram showing a simplified illustration of the relationships between different biological organisms, starting with a plant, then a caterpillar, mouse, squirrel, chicken, organism Q and P, snake, organism R and S, and finally, hawk. It also includes arrows, indicating the connections between these organisms. 1470 | Topic: 1471 | biology 1472 | Diagram Layout: 1473 | tree 1474 | Required Entities: 1475 | plant image (I0) 1476 | caterpillar image (I1) 1477 | mouse image (I2) 1478 | squirrel image (I3) 1479 | chicken image (I4) 1480 | organism q image (I5) 1481 | organism p image (I6) 1482 | snake image (I7) 1483 | organism r image (I8) 1484 | organism s image (I9) 1485 | hawk image (I10) 1486 | "plant" text label (T0) 1487 | "caterpillar" text label (T1) 1488 | "mouse" text label (T2) 1489 | "squirrel" text label (T3) 1490 | "chicken" text label (T4) 1491 | "organism q" text label (T5) 1492 | "organism p" text label (T6) 1493 | "snake" text label (T7) 1494 | "organism r" text label (T8) 1495 | "organism s" text label (T9) 1496 | "hawk" text label (T10) 1497 | Entity Relationships: 1498 | T10 labels I10 1499 | T2 labels I2 1500 | I1 has an arrow to I2 1501 | T4 labels I4 1502 | I9 has an arrow to I10 1503 | I0 has an arrow to I1 1504 | I5 has an arrow to I6 1505 | T7 labels I7 1506 | I7 has an arrow to I8 1507 | T0 labels I0 1508 | T5 labels I5 1509 | I2 has an arrow to I3 1510 | T8 labels I8 1511 | T6 labels I6 1512 | I6 has an arrow to I7 1513 | T9 labels I9 1514 | I8 has an arrow to I9 1515 | I3 has an arrow to I4 1516 | T1 labels I1 1517 | T3 labels I3 1518 | I4 has an arrow to I5 1519 | Entity Locations: 1520 | I0 is located at [0, 0, 10, 10] 1521 | I1 is located at [15, 10, 10, 10] 1522 | I2 is located at [30, 20, 10, 10] 1523 | I3 is located at [45, 30, 10, 10] 1524 | I4 is located at [60, 40, 10, 10] 1525 | I5 is located at [75, 50, 10, 10] 1526 | I6 is located at [90, 60, 10, 10] 1527 | I7 is located at [75, 70, 10, 10] 1528 | I8 is located at [60, 80, 10, 10] 1529 | I9 is located at [45, 90, 10, 10] 1530 | I10 is located at [30, 100, 10, 10] 1531 | T0 is located at [0, 15, 10, 5] 1532 | T1 is located at [15, 25, 10, 5] 1533 | T2 is located at [30, 35, 10, 5] 1534 | T3 is located at [45, 45, 10, 5] 1535 | T4 is located at [60, 55, 10, 5] 1536 | T5 is located at [75, 65, 10, 5] 1537 | T6 is located at [90, 75, 10, 5] 1538 | T7 is located at [75, 85, 10, 5] 1539 | T8 is located at [60, 95, 10, 5] 1540 | T9 is located at [45, 105, 10, 5] 1541 | T10 is located at [30, 115, 10, 5] 1542 | What is wrong with this diagram? 1543 | The plant needs arrows to the mouse and squirrel. The mouse should not have an arrow the squirrel. Squirrel needs arrow to the hawk. The chicken needs an arrow to the snake. The caterpillar needs an arrow to the chicken. 1544 | 1545 | Caption: 1546 | A diagram showing the life cycle of a water flea, through four different stages: egg, larva, pupa, and adult. 1547 | Topic: 1548 | biology 1549 | Diagram Layout: 1550 | circular 1551 | Required Entities: 1552 | egg image (I0) 1553 | larva image (I1) 1554 | pupa image (I2) 1555 | adult water flea image (I3) 1556 | "egg" text label (T0) 1557 | "larva" text label (T1) 1558 | "pupa" text label (T2) 1559 | "adult" text label (T3) 1560 | Entity Relationships: 1561 | T2 labels I2 1562 | T1 labels I1 1563 | T0 labels I0 1564 | T3 labels I3 1565 | Entity Locations: 1566 | I0 is located at [10, 50, 20, 20] 1567 | I1 is located at [40, 10, 20, 20] 1568 | I2 is located at [70, 50, 20, 20] 1569 | I3 is located at [40, 90, 20, 20] 1570 | T0 is located at [15, 45, 10, 5] 1571 | T1 is located at [45, 5, 10, 5] 1572 | T2 is located at [75, 45, 10, 5] 1573 | T3 is located at [45, 85, 10, 5] 1574 | What is wrong with this diagram? 1575 | There should be arrows between each phase. The adult water flea is also out of bounds, it should slightly more up so it is not outside the diagram. 1576 | 1577 | Caption: 1578 | A diagram showing a consumption relationship between different types of organisms, including man, birds, duck, chicken, beetle, worm, corn, guava, pechay, and bacteria/fungi. 1579 | Topic: 1580 | biology 1581 | Diagram Layout: 1582 | tree 1583 | Required Entities: 1584 | man image (I0) 1585 | bird image (I1) 1586 | duck image (I2) 1587 | chicken image (I3) 1588 | beetle image (I4) 1589 | worm image (I5) 1590 | corn image (I6) 1591 | guava image (I7) 1592 | pechay image (I8) 1593 | bacteria/fungi image (I9) 1594 | "man" text label (T0) 1595 | "birds" text label (T1) 1596 | "duck" text label (T2) 1597 | "chicken" text label (T3) 1598 | "beetle" text label (T4) 1599 | "worm" text label (T5) 1600 | "corn" text label (T6) 1601 | "guava" text label (T7) 1602 | "pechay" text label (T8) 1603 | "bacteria/fungi" text label (T9) 1604 | Entity Relationships: 1605 | I1 has an arrow to I0 1606 | T2 labels I2 1607 | T4 labels I4 1608 | I6 has an arrow to I5 1609 | I4 has an arrow to I1 1610 | T7 labels I7 1611 | I9 has an arrow to I7 1612 | I2 has an arrow to I0 1613 | T0 labels I0 1614 | T5 labels I5 1615 | I7 has an arrow to I5 1616 | T8 labels I8 1617 | I9 has an arrow to I8 1618 | I9 has an arrow to I6 1619 | I5 has an arrow to I4 1620 | I3 has an arrow to I0 1621 | I8 has an arrow to I5 1622 | T6 labels I6 1623 | T9 labels I9 1624 | T1 labels I1 1625 | T3 labels I3 1626 | Entity Locations: 1627 | I0 is located at [50, 0, 20, 20] 1628 | I1 is located at [30, 20, 15, 15] 1629 | I2 is located at [50, 20, 15, 15] 1630 | I3 is located at [70, 20, 15, 15] 1631 | I4 is located at [20, 40, 15, 15] 1632 | I5 is located at [50, 40, 15, 15] 1633 | I6 is located at [20, 60, 15, 15] 1634 | I7 is located at [50, 60, 15, 15] 1635 | I8 is located at [80, 60, 15, 15] 1636 | I9 is located at [50, 80, 15, 15] 1637 | T0 is located at [55, 5, 10, 5] 1638 | T1 is located at [32, 35, 10, 5] 1639 | T2 is located at [52, 35, 10, 5] 1640 | T3 is located at [72, 35, 10, 5] 1641 | T4 is located at [22, 55, 10, 5] 1642 | T5 is located at [52, 55, 10, 5] 1643 | T6 is located at [22, 75, 10, 5] 1644 | T7 is located at [52, 75, 10, 5] 1645 | T8 is located at [82, 75, 10, 5] 1646 | T9 is located at [52, 95, 15, 5] 1647 | What is wrong with this diagram? 1648 | Worm needs an arrow to the chicken and birds. Corn needs an arrow to the duck, the chicken, and the birds. The guava needs an arrow to the worm and the beetle. 1649 | 1650 | Caption: 1651 | A diagram showing three food chains. The first chain is flowers and grass being consumed by a rabbit and then the rabbit being consumed by a fox. The second is wheat going to field mice going to a hawk. The third one is fruit trees going to deer which goes to a bear. 1652 | Topic: 1653 | biology 1654 | Diagram Layout: 1655 | tree 1656 | Required Entities: 1657 | "flowers and grass" text label (T0) 1658 | "rabbit" text label (T1) 1659 | "fox" text label (T2) 1660 | "wheat" text label (T3) 1661 | "field mice" text label (T4) 1662 | "hawk" text label (T5) 1663 | "fruit trees" text label (T6) 1664 | "deer" text label (T7) 1665 | "bear" text label (T8) 1666 | rabbit image (I0) 1667 | fox image (I1) 1668 | field mice image (I2) 1669 | hawk image (I3) 1670 | deer image (I4) 1671 | bear image (I5) 1672 | Entity Relationships: 1673 | I2 has an arrow to I3 1674 | T5 labels I3 1675 | I4 has an arrow to I5 1676 | T8 labels I5 1677 | T7 labels I4 1678 | T4 labels I2 1679 | T2 labels I1 1680 | T1 labels I0 1681 | T3 has an arrow to I2 1682 | T0 has an arrow to I0 1683 | T6 has an arrow to I4 1684 | I0 has an arrow to I1 1685 | Entity Locations: 1686 | T0 is located at [5, 10, 20, 5] 1687 | T1 is located at [30, 20, 10, 5] 1688 | T2 is located at [50, 30, 10, 5] 1689 | T3 is located at [5, 50, 10, 5] 1690 | T4 is located at [30, 60, 15, 5] 1691 | T5 is located at [50, 70, 10, 5] 1692 | T6 is located at [5, 90, 15, 5] 1693 | T7 is located at [30, 80, 10, 5] 1694 | T8 is located at [50, 90, 10, 5] 1695 | I0 is located at [25, 15, 15, 10] 1696 | I1 is located at [45, 25, 15, 10] 1697 | I2 is located at [25, 55, 15, 10] 1698 | I3 is located at [45, 65, 15, 10] 1699 | I4 is located at [25, 85, 15, 10] 1700 | I5 is located at [45, 95, 15, 10] 1701 | What is wrong with this diagram? 1702 | Each food chain should be in a straight line. The base of each food chain should also have an image. Flowers and grass should be seperated into their own entities. 1703 | 1704 | """ 1705 | 1706 | DIAGRAM_PROMPT_engineering = """Given a caption of a diagram and topic, generate the diagram layout, and then a list of required entities that would be needed to create the described diagram. Then generate a list of the relationships between the entities (i.e. which ones are connected or labeling each other). Finally, generate the location of each entity. 1707 | An entity can be an image or text. Entity locations should be generated in [x, y, width, height] format, where 0,0 is the top left corner and 100,100 is max image size. 1708 | 1709 | Think step-by-step, break each caption into parts and generate the required entities, relationships, and locations for each part. 1710 | 1711 | Here are some rules to follow: 1712 | All numbers should be positive, do not generate negative numbers. 1713 | Please always generate a list of entities, even if the list is empty. 1714 | Entities should not be outside the bounds [0, 0, 100, 100]. 1715 | The x coordinate + the width of an entity should not exceed 100. 1716 | The y coordinate + the height of an entity should not exceed 100. 1717 | Entities of the same type should not overlap. 1718 | 1719 | Here are some examples: 1720 | 1721 | Caption: 1722 | A diagram showing two simple circuits with a battery and three lights. One circuit has a switch to control the flow of electricity. Both circuits are connects in series. 1723 | Topic: 1724 | engineering 1725 | Diagram Layout: 1726 | circular 1727 | Required Entities: 1728 | battery image (I0) 1729 | "battery" text label (T0) 1730 | "switch" text label (T1) 1731 | switch image (I1) 1732 | light bulb image (I2) 1733 | light bulb image (I3) 1734 | light bulb image (I4) 1735 | battery image (I5) 1736 | light bulb image (I6) 1737 | light bulb image (I7) 1738 | light bulb image (I8) 1739 | Entity Relationships: 1740 | I6 connects to I7 1741 | T1 labels I1 1742 | I0 connects to I1 1743 | I3 connects to I4 1744 | I5 connects to I6 1745 | I7 connects to I8 1746 | I8 connects to I5 1747 | I4 connects to I0 1748 | I2 connects to I3 1749 | T0 labels I0 1750 | I1 connects to I2 1751 | Entity Locations: 1752 | I0 is located at [0, 62, 7, 20] 1753 | T0 is located at [10, 69, 11, 7] 1754 | T1 is located at [10, 9, 10, 6] 1755 | I1 is located at [9, 17, 16, 14] 1756 | I2 is located at [27, 3, 14, 27] 1757 | I3 is located at [27, 35, 15, 26] 1758 | I4 is located at [26, 65, 15, 27] 1759 | I5 is located at [58, 33, 9, 28] 1760 | I6 is located at [73, 7, 7, 20] 1761 | I7 is located at [88, 43, 10, 12] 1762 | I8 is located at [74, 56, 7, 20] 1763 | 1764 | Caption: 1765 | A diagram showing two circuits. In both a circuits, there are two light bulbs connected to a battery. The first circuit is in series and the second one is in parallel. 1766 | Topic: 1767 | engineering 1768 | Diagram Layout: 1769 | circular 1770 | Required Entities: 1771 | battery image (I0) 1772 | light bulb image (I1) 1773 | light bulb image (I2) 1774 | light bulb image (I3) 1775 | light bulb image (I4) 1776 | battery image (I5) 1777 | Entity Relationships: 1778 | I4 connects to I5 1779 | I0 connects to I1 1780 | I4 connects to I3 1781 | I3 connects to I4 1782 | I5 connects to I4 1783 | I1 connects to I2 1784 | Entity Locations: 1785 | I0 is located at [17, 18, 22, 13] 1786 | I1 is located at [13, 46, 13, 27] 1787 | I2 is located at [28, 47, 14, 24] 1788 | I3 is located at [66, 71, 16, 25] 1789 | I4 is located at [68, 30, 14, 26] 1790 | I5 is located at [64, 3, 21, 12] 1791 | 1792 | Caption: 1793 | A diagram showing a simple electric circuit with a battery, a light bulb, and a switch. 1794 | Topic: 1795 | engineering 1796 | Diagram Layout: 1797 | circular 1798 | Required Entities: 1799 | light bulb image (I0) 1800 | battery image (I1) 1801 | switch image (I2) 1802 | "simple electric circuit" text label (T0) 1803 | Entity Relationships: 1804 | I1 connects to I2 1805 | I0 connects to I1 1806 | I2 connects to I0 1807 | Entity Locations: 1808 | I0 is located at [18, 22, 17, 26] 1809 | I1 is located at [30, 74, 40, 12] 1810 | I2 is located at [66, 38, 10, 9] 1811 | T0 is located at [5, 5, 90, 7] 1812 | 1813 | Caption: 1814 | A diagram showing a simple circuit with a battery and three light bulbs. The lights and battery are connected in series. 1815 | Topic: 1816 | engineering 1817 | Diagram Layout: 1818 | circular 1819 | Required Entities: 1820 | battery image (I0) 1821 | light bulb image (I1) 1822 | light bulb image (I2) 1823 | light bulb image (I3) 1824 | Entity Relationships: 1825 | I1 connects to I2 1826 | I2 connects to I3 1827 | I0 connects to I1 1828 | I3 connects to I0 1829 | Entity Locations: 1830 | I0 is located at [10, 37, 12, 37] 1831 | I1 is located at [44, 61, 14, 29] 1832 | I2 is located at [77, 37, 14, 29] 1833 | I3 is located at [43, 3, 14, 28] 1834 | 1835 | Caption: 1836 | A diagram showing a circuit schematic. The 10V better is connected to a 1 ohm resistor directly. Then the 1 ohm resistor is connected to a 25 ohm resistor and 30 ohm resistor in parallel. Those resistors are connected to a 50 ohm and 55 ohm resistor respectively. Then the parallel circuit comes back together and connects to the battery. Both sides of the parallel circuit are connected via a 1 ohm resistor. 1837 | Topic: 1838 | engineering 1839 | Diagram Layout: 1840 | circular 1841 | Required Entities: 1842 | "10v" text label (T0) 1843 | "1 ohm" text label (T1) 1844 | "25 ohm" text label (T2) 1845 | "50 ohm" text label (T3) 1846 | "1 ohm" text label (T4) 1847 | "30 ohm" text label (T5) 1848 | "55 ohm" text label (T6) 1849 | battery schematic image (I0) 1850 | resistor schematic image (I1) 1851 | resistor schematic image (I2) 1852 | resistor schematic image (I3) 1853 | resistor schematic image (I4) 1854 | resistor schematic image (I5) 1855 | resistor schematic image (I6) 1856 | Entity Relationships: 1857 | T0 labels I0 1858 | T2 labels I2 1859 | T6 labels I6 1860 | T5 labels I5 1861 | T1 labels I1 1862 | T4 labels I4 1863 | T3 labels I3 1864 | Entity Locations: 1865 | T0 is located at [1, 40, 9, 8] 1866 | T1 is located at [26, 11, 8, 9] 1867 | T2 is located at [53, 26, 9, 8] 1868 | T3 is located at [38, 72, 10, 7] 1869 | T4 is located at [62, 58, 8, 7] 1870 | T5 is located at [90, 27, 9, 8] 1871 | T6 is located at [89, 71, 9, 8] 1872 | I0 is located at [8, 48, 9, 7] 1873 | I1 is located at [25, 2, 12, 9] 1874 | I2 is located at [48, 21, 5, 17] 1875 | I3 is located at [48, 66, 6, 18] 1876 | I4 is located at [62, 46, 12, 9] 1877 | I5 is located at [83, 21, 6, 19] 1878 | I6 is located at [83, 64, 6, 20] 1879 | 1880 | Caption: 1881 | A diagram showing a circuit with a 6V battery being connected to 2 light bulbs on opposite sides and then a switch in between them. The switch then connects to both light bulbs. 1882 | Topic: 1883 | engineering 1884 | Diagram Layout: 1885 | circular 1886 | Required Entities: 1887 | battery image (I0) 1888 | switch image (I1) 1889 | light bulb on image (I2) 1890 | light bulb off image (I3) 1891 | "6v" text label (T0) 1892 | Entity Relationships: 1893 | T0 labels I0 1894 | I2 connects to I1 1895 | I0 connects to I1 1896 | I3 connects to I1 1897 | I0 connects to I2 1898 | I0 connects to I3 1899 | Entity Locations: 1900 | I0 is located at [9, 40, 11, 21] 1901 | I1 is located at [30, 37, 66, 26] 1902 | I2 is located at [66, 0, 10, 20] 1903 | I3 is located at [66, 75, 10, 21] 1904 | T0 is located at [12, 35, 6, 5] 1905 | 1906 | Caption: 1907 | A diagram showing the components of a slipper spring. It includes a spring hanger, a spring bolt and locknut, a two rebound clips, a slipper, and a centre bolt. 1908 | Topic: 1909 | engineering 1910 | Diagram Layout: 1911 | abstract 1912 | Required Entities: 1913 | spring hanger image (I0) 1914 | spring bolt and locknut image (I1) 1915 | rebound clip image (I2) 1916 | rebound clip image (I3) 1917 | slipper image (I4) 1918 | "spring hanger" text label (T0) 1919 | "spring bolt & locknut" text label (T1) 1920 | "rebound clip" text label (T2) 1921 | "slipper" text label (T3) 1922 | "centre bolt" text label (T4) 1923 | centre bolt image (I5) 1924 | "eye/slipper spring components" text label (T5) 1925 | Entity Relationships: 1926 | T0 labels I0 1927 | T3 labels I4 1928 | T2 labels I2 1929 | T4 labels I5 1930 | T2 labels I3 1931 | T1 labels I1 1932 | Entity Locations: 1933 | I0 is located at [6, 18, 8, 24] 1934 | I1 is located at [7, 29, 4, 11] 1935 | I2 is located at [22, 56, 5, 18] 1936 | I3 is located at [65, 49, 6, 16] 1937 | I4 is located at [81, 17, 16, 16] 1938 | T0 is located at [16, 6, 15, 8] 1939 | T1 is located at [2, 72, 20, 7] 1940 | T2 is located at [75, 77, 13, 8] 1941 | T3 is located at [67, 2, 7, 6] 1942 | T4 is located at [42, 91, 11, 6] 1943 | I5 is located at [45, 64, 3, 15] 1944 | T5 is located at [61, 91, 33, 8] 1945 | 1946 | Caption: 1947 | A diagram showing a simple electronic circuit with a lamp and a battery. 1948 | Topic: 1949 | engineering 1950 | Diagram Layout: 1951 | circular 1952 | Required Entities: 1953 | lamp image (I0) 1954 | battery image (I1) 1955 | "lamp" text label (T0) 1956 | "battery" text label (T1) 1957 | Entity Relationships: 1958 | T1 labels I1 1959 | I0 connects to I1 1960 | I1 connects to I0 1961 | T0 labels I0 1962 | Entity Locations: 1963 | I0 is located at [32, 12, 28, 38] 1964 | I1 is located at [27, 65, 31, 22] 1965 | T0 is located at [31, 2, 14, 7] 1966 | T1 is located at [28, 90, 21, 7] 1967 | 1968 | Caption: 1969 | A diagram showing the flow of energy through various processes, including energy conversion, storage, and distribution. Each process is represented as a block. It also shows the sun as an energy source going into the energy conversion, and energy use as a light bulb and electric utility as powerlines coming out of the energy distribution. 1970 | Topic: 1971 | engineering 1972 | Diagram Layout: 1973 | tree 1974 | Required Entities: 1975 | sun image (I0) 1976 | metal plate image (I1) 1977 | block image (I2) 1978 | block image (I3) 1979 | light bulb image (I4) 1980 | block image (I5) 1981 | powerline image (I6) 1982 | "energy source" text label (T0) 1983 | "energy conversion" text label (T1) 1984 | "energy storage" text label (T2) 1985 | "energy inversion & conditioning" text label (T3) 1986 | "energy use" text label (T4) 1987 | "energy distribution" text label (T5) 1988 | "electric utility" text label (T6) 1989 | Entity Relationships: 1990 | I3 has an arrow to I2 1991 | T0 labels I0 1992 | I6 has an arrow to I5 1993 | I2 has an arrow to I3 1994 | T2 labels I2 1995 | I2 has an arrow to I5 1996 | I5 has an arrow to I6 1997 | T6 labels I6 1998 | I0 has an arrow to I1 1999 | I1 has an arrow to I2 2000 | T5 labels I5 2001 | I5 has an arrow to I4 2002 | T1 labels I1 2003 | T4 labels I4 2004 | T3 labels I3 2005 | Entity Locations: 2006 | I0 is located at [0, 2, 11, 20] 2007 | I1 is located at [9, 38, 20, 17] 2008 | I2 is located at [44, 28, 12, 35] 2009 | I3 is located at [43, 78, 16, 20] 2010 | I4 is located at [70, 2, 12, 20] 2011 | I5 is located at [72, 37, 10, 18] 2012 | I6 is located at [69, 71, 18, 28] 2013 | T0 is located at [1, 26, 10, 11] 2014 | T1 is located at [10, 57, 15, 11] 2015 | T2 is located at [30, 83, 12, 14] 2016 | T3 is located at [43, 11, 16, 18] 2017 | T4 is located at [83, 8, 11, 12] 2018 | T5 is located at [82, 39, 17, 14] 2019 | T6 is located at [87, 78, 10, 15] 2020 | 2021 | Caption: 2022 | A diagram showing a battery with a light bulb connected to it. 2023 | Topic: 2024 | engineering 2025 | Diagram Layout: 2026 | circular 2027 | Required Entities: 2028 | light bulb image (I0) 2029 | battery image (I1) 2030 | Entity Relationships: 2031 | I0 connects to I1 2032 | I1 connects to I0 2033 | Entity Locations: 2034 | I0 is located at [8, 4, 22, 41] 2035 | I1 is located at [66, 40, 27, 39] 2036 | 2037 | Caption: 2038 | """ 2039 | 2040 | AUDITOR_PROMPT_engineering = """Given a caption and a layout of a diagram, you should determine if something is wrong in the diagram based on the caption. You should explain your answer. Think step-by-step as to why the diagram is correct or not. 2041 | The diagram will be described in terms of entities in the diagram, the relationships between the entities, and the location of each entity. 2042 | An entity can be an image or text. Entity locations will be in [x, y, width, height] format, where 0,0 is the top left corner and 100,100 is max image size. 2043 | 2044 | Here are some rules the diagrams should follow: 2045 | All numbers should be positive, no negative numbers. 2046 | Entities should not be outside the bounds [0, 0, 100, 100]. 2047 | The x coordinate + the width of an entity should not exceed 100. 2048 | The y coordinate + the height of an entity should not exceed 100. 2049 | Entities of the same type should not overlap. 2050 | 2051 | Here are some examples: 2052 | Caption: 2053 | A diagram showing a simple electrical circuit schematic with a battery, a switch, and two light bulbs. 2054 | Topic: 2055 | engineering 2056 | Diagram Layout: 2057 | circular 2058 | Required Entities: 2059 | battery schematic image (I0) 2060 | light bulb schematic image (I1) 2061 | light bulb schematic image (I2) 2062 | "simple electrical circuit" text label (T0) 2063 | Entity Relationships: 2064 | I2 connects to I0 2065 | I1 connects to I2 2066 | I0 connects to I1 2067 | T0 labels I0 2068 | Entity Locations: 2069 | I0 is located at [10, 40, 20, 20] 2070 | I1 is located at [40, 10, 20, 20] 2071 | I2 is located at [70, 40, 20, 20] 2072 | T0 is located at [10, 70, 80, 10] 2073 | What is wrong with this diagram? 2074 | The switch is missing from the circuit in this diagram. 2075 | 2076 | Caption: 2077 | A diagram showing the cross-section of a cable with different components, including rubber insulator and copper conductor. 2078 | Topic: 2079 | engineering 2080 | Diagram Layout: 2081 | abstract 2082 | Required Entities: 2083 | cross-section of cable image (I0) 2084 | rubber insulator image (I1) 2085 | copper conductor image (I2) 2086 | "rubber insulator" text label (T0) 2087 | "copper conductor" text label (T1) 2088 | Entity Relationships: 2089 | T1 labels I2 2090 | T0 labels I1 2091 | Entity Locations: 2092 | I0 is located at [0, 0, 100, 100] 2093 | I1 is located at [20, 40, 20, 20] 2094 | I2 is located at [60, 40, 20, 20] 2095 | T0 is located at [20, 65, 20, 10] 2096 | T1 is located at [60, 65, 25, 10] 2097 | What is wrong with this diagram? 2098 | The copper conductor should be inside of the rubber insulator. The rubber insulator should be a bit bigger. 2099 | 2100 | Caption: 2101 | A diagram showing a simple electronic circuit schematic with a battery and a resistor. 2102 | Topic: 2103 | engineering 2104 | Diagram Layout: 2105 | circular 2106 | Required Entities: 2107 | battery schematic image (I0) 2108 | resistor schematic image (I1) 2109 | "battery" text label (T0) 2110 | "resistor" text label (T1) 2111 | Entity Relationships: 2112 | T1 labels I1 2113 | I0 connects to I1 2114 | T0 labels I0 2115 | Entity Locations: 2116 | I0 is located at [10, 40, 20, 20] 2117 | I1 is located at [60, 40, 20, 20] 2118 | T0 is located at [10, 65, 20, 10] 2119 | T1 is located at [60, 65, 20, 10] 2120 | What is wrong with this diagram? 2121 | There should a connection from I1 to I0 as well. 2122 | 2123 | Caption: 2124 | A diagram showing a circuit with two light bulbs and a battery connect in parallel. There is also a schematic diagram of the circuit. 2125 | Topic: 2126 | engineering 2127 | Diagram Layout: 2128 | circular 2129 | Required Entities: 2130 | light bulb image (I0) 2131 | light bulb image (I1) 2132 | battery image (I2) 2133 | circuit schematic image (I3) 2134 | Entity Relationships: 2135 | I2 connects to I0 2136 | I0 connects to I2 2137 | I1 connects to I2 2138 | I2 connects to I1 2139 | Entity Locations: 2140 | I0 is located at [10, 20, 20, 30] 2141 | I1 is located at [10, 60, 20, 30] 2142 | I2 is located at [40, 40, 20, 20] 2143 | I3 is located at [70, 40, 20, 20] 2144 | What is wrong with this diagram? 2145 | The light bulbs should also be connected to each other. The image of a circuit schematic should actually have 3 entities that have the same connections as the actual circuit. 2146 | 2147 | Caption: 2148 | A diagram showing a simple circuit schematic with a battery, light bulb, and a switch all connected in series. 2149 | Topic: 2150 | engineering 2151 | Diagram Layout: 2152 | circular 2153 | Required Entities: 2154 | battery image (I0) 2155 | light bulb image (I1) 2156 | switch image (I2) 2157 | "simple circuit schematic" text label (T0) 2158 | Entity Relationships: 2159 | I2 connects to I0 2160 | I1 connects to I2 2161 | I0 connects to I1 2162 | Entity Locations: 2163 | I0 is located at [10, 40, 20, 20] 2164 | I1 is located at [40, 40, 20, 20] 2165 | I2 is located at [70, 40, 20, 20] 2166 | T0 is located at [10, 10, 80, 10] 2167 | What is wrong with this diagram? 2168 | The components of the circuit should form a loop rather than straight line. 2169 | 2170 | Caption: 2171 | A diagram showing a simple circuit with two light bulbs and a battery. The light bulbs are connected parallel with the battery. 2172 | Topic: 2173 | engineering 2174 | Diagram Layout: 2175 | circular 2176 | Required Entities: 2177 | light bulb image (I0) 2178 | light bulb image (I1) 2179 | battery image (I2) 2180 | Entity Relationships: 2181 | I0 connects to I2 2182 | I1 connects to I2 2183 | Entity Locations: 2184 | I0 is located at [10, 30, 20, 30] 2185 | I1 is located at [70, 30, 20, 30] 2186 | I2 is located at [40, 50, 20, 30] 2187 | What is wrong with this diagram? 2188 | The battery should connect to both light bulbs and the light bulbs should connect to each other. 2189 | 2190 | Caption: 2191 | A diagram showing two circuits. The first is a open circuit with a light bulb connected to battery but the battery doesn't have a connection to the light bulb. The second is a closed circuit with a light bulb connected to a battery and the battery returns the connection. 2192 | Topic: 2193 | engineering 2194 | Diagram Layout: 2195 | circular 2196 | Required Entities: 2197 | light bulb image (I0) 2198 | battery image (I1) 2199 | light bulb image (I2) 2200 | battery image (I3) 2201 | "open circuit" text label (T0) 2202 | "closed circuit" text label (T1) 2203 | Entity Relationships: 2204 | I2 connects to I3 2205 | I1 connects to I0 2206 | I3 connects to I2 2207 | Entity Locations: 2208 | I0 is located at [10, 20, 15, 25] 2209 | I1 is located at [10, 60, 15, 25] 2210 | I2 is located at [60, 20, 15, 25] 2211 | I3 is located at [60, 60, 15, 25] 2212 | T0 is located at [30, 30, 20, 10] 2213 | T1 is located at [80, 70, 20, 10] 2214 | What is wrong with this diagram? 2215 | The open and closed circuit text labels should be above each circuit, not besides them. 2216 | 2217 | Caption: 2218 | A diagram showing a electrical circuit schematic. It has a 12V battery connected to a 1k resistor, a light bulb, and another 1k resistor. 2219 | Topic: 2220 | engineering 2221 | Diagram Layout: 2222 | circular 2223 | Required Entities: 2224 | "12v" text label (T0) 2225 | "1k" text label (T1) 2226 | "1k" text label (T2) 2227 | battery schematic image (I0) 2228 | resistor schematic image (I1) 2229 | light bulb schematic image (I2) 2230 | resistor schematic image (I3) 2231 | Entity Relationships: 2232 | I3 connects to I0 2233 | I1 connects to I2 2234 | I2 connects to I3 2235 | T1 labels I1 2236 | I0 connects to I1 2237 | T0 labels I0 2238 | T2 labels I3 2239 | Entity Locations: 2240 | T0 is located at [2, 40, 8, 7] 2241 | T1 is located at [20, 10, 8, 7] 2242 | T2 is located at [80, 10, 8, 7] 2243 | I0 is located at [10, 50, 15, 10] 2244 | I1 is located at [30, 20, 10, 10] 2245 | I2 is located at [50, 50, 10, 10] 2246 | I3 is located at [70, 20, 10, 10] 2247 | What is wrong with this diagram? 2248 | The light bulb schematic and the resistor schematic should swap places to make the diagram flow smoother. 2249 | 2250 | Caption: 2251 | A diagram showing the structure of an electric circuit with a light bulb, a switch, and a battery. 2252 | Topic: 2253 | engineering 2254 | Diagram Layout: 2255 | circular 2256 | Required Entities: 2257 | light bulb image (I0) 2258 | battery image (I1) 2259 | "electric circuit" text label (T0) 2260 | Entity Relationships: 2261 | T0 labels I0 and I1 2262 | I0 connects to I1 2263 | I1 connects to I0 2264 | Entity Locations: 2265 | I0 is located at [10, 30, 20, 40] 2266 | I1 is located at [70, 30, 20, 40] 2267 | T0 is located at [40, 50, 20, 10] 2268 | What is wrong with this diagram? 2269 | The switch is missing from the circuit in this diagram. 2270 | 2271 | """ 2272 | 2273 | prompts = {'diagram_astronomy': DIAGRAM_PROMPT_astronomy,'auditor_astronomy': AUDITOR_PROMPT_astronomy,'diagram_biology': DIAGRAM_PROMPT_biology,'auditor_biology': AUDITOR_PROMPT_biology, 'diagram_engineering': DIAGRAM_PROMPT_engineering,'auditor_engineering': AUDITOR_PROMPT_engineering,} --------------------------------------------------------------------------------