├── backend ├── core │ ├── __init__.py │ ├── llm.py │ ├── approximate_costs.py │ ├── analysis_utils.py │ ├── analysis_components.py │ ├── prompts.py │ └── analysis_groups.py ├── poetry.toml ├── pyproject.toml ├── test.py └── test_multi.py ├── image ├── exp1after.gif ├── exp1before.gif ├── exp2after.gif ├── exp2before.gif ├── exp3after.gif ├── exp3before.gif ├── exp4after.gif ├── exp4before.gif ├── exp5after.gif ├── exp5before.gif ├── overview.png ├── Fig.4 (a) Distribution by Component Type.png ├── Fig.8 (b) Score Comparisons Before vs After Repair.png ├── Fig.8 (a) Comparison of Scores Before and After Repair.png ├── Fig. 7 Frequency of Likert Scores Before and After Repair.png ├── Fig.5 Examples of Component and System Design Knowledge entities.png └── Fig.4 (b) Distribution by Component Design Aspects with SoftHard Constraint.png ├── scripts ├── System Level Knowledge Relationship Graph.png ├── create_knowledge_graph.md └── prepare_kb_dump.py ├── examples ├── test_projects.md └── example1.tsx ├── README.md ├── LICENSE └── library └── system_design_knowledge_base.csv /backend/core/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /backend/poetry.toml: -------------------------------------------------------------------------------- 1 | [virtualenvs] 2 | create = true 3 | -------------------------------------------------------------------------------- /image/exp1after.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp1after.gif -------------------------------------------------------------------------------- /image/exp1before.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp1before.gif -------------------------------------------------------------------------------- /image/exp2after.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp2after.gif -------------------------------------------------------------------------------- /image/exp2before.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp2before.gif -------------------------------------------------------------------------------- /image/exp3after.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp3after.gif -------------------------------------------------------------------------------- /image/exp3before.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp3before.gif -------------------------------------------------------------------------------- /image/exp4after.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp4after.gif -------------------------------------------------------------------------------- /image/exp4before.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp4before.gif -------------------------------------------------------------------------------- /image/exp5after.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp5after.gif -------------------------------------------------------------------------------- /image/exp5before.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/exp5before.gif -------------------------------------------------------------------------------- /image/overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/overview.png -------------------------------------------------------------------------------- /image/Fig.4 (a) Distribution by Component Type.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/Fig.4 (a) Distribution by Component Type.png -------------------------------------------------------------------------------- /scripts/System Level Knowledge Relationship Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/scripts/System Level Knowledge Relationship Graph.png -------------------------------------------------------------------------------- /image/Fig.8 (b) Score Comparisons Before vs After Repair.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/Fig.8 (b) Score Comparisons Before vs After Repair.png -------------------------------------------------------------------------------- /image/Fig.8 (a) Comparison of Scores Before and After Repair.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/Fig.8 (a) Comparison of Scores Before and After Repair.png -------------------------------------------------------------------------------- /image/Fig. 7 Frequency of Likert Scores Before and After Repair.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/Fig. 7 Frequency of Likert Scores Before and After Repair.png -------------------------------------------------------------------------------- /image/Fig.5 Examples of Component and System Design Knowledge entities.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/Fig.5 Examples of Component and System Design Knowledge entities.png -------------------------------------------------------------------------------- /image/Fig.4 (b) Distribution by Component Design Aspects with SoftHard Constraint.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UGAIForge/DesignRepair/HEAD/image/Fig.4 (b) Distribution by Component Design Aspects with SoftHard Constraint.png -------------------------------------------------------------------------------- /backend/pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.poetry] 2 | name = "backend" 3 | version = "0.1.0" 4 | description = "" 5 | authors = ["Abi Raja "] 6 | license = "MIT" 7 | 8 | [tool.poetry.dependencies] 9 | python = "^3.10" 10 | fastapi = "^0.95.0" 11 | uvicorn = "^0.25.0" 12 | websockets = "^12.0" 13 | openai = "^1.2.4" 14 | python-dotenv = "^1.0.0" 15 | beautifulsoup4 = "^4.12.2" 16 | httpx = "^0.25.1" 17 | jinja2 = "^3.1.3" 18 | tiktoken = "^0.5.2" 19 | 20 | [tool.poetry.group.dev.dependencies] 21 | pytest = "^7.4.3" 22 | pyright = "^1.1.345" 23 | 24 | [build-system] 25 | requires = ["poetry-core"] 26 | build-backend = "poetry.core.masonry.api" 27 | -------------------------------------------------------------------------------- /examples/test_projects.md: -------------------------------------------------------------------------------- 1 | ### Vercel's V0 projects 2 | 3 | | ID | url | 4 | | ---- | ---------------------------- | 5 | | 1 | https://v0.dev/t/0W13RkH | 6 | | 2 | https://v0.dev/t/CF6CtRGlgJi | 7 | | 3 | https://v0.dev/t/OKXb3ACML6t | 8 | | 4 | https://v0.dev/t/itf6aBV | 9 | | 5 | https://v0.dev/t/YKiZgaUISA3 | 10 | | 6 | https://v0.dev/t/zJO10z7wUTe | 11 | | 7 | https://v0.dev/t/x4SRZe6 | 12 | | 8 | https://v0.dev/t/9tk0WDvMrYm | 13 | | 9 | https://v0.dev/t/xiSjIAI | 14 | | 10 | https://v0.dev/t/W5ak7S2nJ9y | 15 | | 11 | https://v0.dev/t/gYyJyeY | 16 | | 12 | https://v0.dev/t/sAfLDnh | 17 | | 13 | https://v0.dev/t/Xx6DE3L | 18 | | 14 | https://v0.dev/t/LnxRCcq | 19 | | 15 | https://v0.dev/t/AfXYpLG | 20 | | 16 | https://v0.dev/t/y37cnlJ | 21 | | 17 | https://v0.dev/t/IIDP1z3aQjw | 22 | | 18 | https://v0.dev/t/fi5AQgx | 23 | | 19 | https://v0.dev/t/rRBlufM | 24 | | 20 | https://v0.dev/t/VT395Yf4lhX | 25 | 26 | 27 | ### Github projects 28 | 29 | | ID | url | 30 | | ---- | ------------------------------------------------------------ | 31 | | 1 | https://github.com/cypress-io/cypress-realworld-app | 32 | | 2 | https://github.com/gothinkster/react-redux-realworld-example-app | 33 | | 3 | https://github.com/jgudo/ecommerce-react?tab=readme-ov-file | 34 | | 4 | https://github.com/oldboyxx/jira_clone | 35 | | 5 | https://github.com/HospitalRun/hospitalrun-frontend | 36 | | 6 | https://github.com/bbc/simorgh | 37 | -------------------------------------------------------------------------------- /backend/core/llm.py: -------------------------------------------------------------------------------- 1 | from typing import Awaitable, Callable, List 2 | from openai import AsyncOpenAI 3 | from openai.types.chat import ChatCompletionMessageParam, ChatCompletionChunk 4 | 5 | MODEL_GPT_4_VISION = "gpt-4-vision-preview" 6 | MODEL_GPT_4 = "gpt-4-1106-preview" 7 | MODEL_GPT_4o = "gpt-4o-2024-05-13" 8 | MODEL_GPT_4_TURBO = "gpt-4-turbo-2024-04-09" 9 | 10 | async def stream_openai_response( 11 | messages: List[ChatCompletionMessageParam], 12 | api_key: str, 13 | temperature: float, 14 | functions: List[dict] | None, 15 | base_url: str | None, 16 | callback: Callable[[str], Awaitable[None]], 17 | model: str | None = None, 18 | ) -> str: 19 | client = AsyncOpenAI(api_key=api_key, base_url=base_url) 20 | 21 | if model is None: 22 | model = MODEL_GPT_4 23 | 24 | # Base parameters 25 | params = {"model": model, "messages": messages, "stream": True, "timeout": 600, "temperature": temperature} 26 | 27 | # Add 'max_tokens' only if the model is a GPT4 vision model 28 | if model == MODEL_GPT_4_VISION: 29 | params["max_tokens"] = 4096 30 | 31 | # Add function calling 32 | if functions is not None: 33 | params["functions"] = functions 34 | params["function_call"] = {"name": functions[0]["name"]} 35 | 36 | stream = await client.chat.completions.create(**params) # type: ignore 37 | full_response = "" 38 | async for chunk in stream: # type: ignore 39 | assert isinstance(chunk, ChatCompletionChunk) 40 | if functions is not None: 41 | if chunk.choices[0].delta.function_call: 42 | content = chunk.choices[0].delta.function_call.arguments or "" 43 | else: 44 | content = "" 45 | else: 46 | content = chunk.choices[0].delta.content or "" 47 | full_response += content 48 | # print(content) 49 | await callback(content) 50 | 51 | await client.close() 52 | 53 | return full_response -------------------------------------------------------------------------------- /backend/core/approximate_costs.py: -------------------------------------------------------------------------------- 1 | import tiktoken 2 | 3 | # TODO: use langchain if we will use it for future agents 4 | MODEL_COST_PER_1K_TOKENS = { 5 | # GPT-4 input 6 | "gpt-4": 0.03, 7 | "gpt-4-0314": 0.03, 8 | "gpt-4-0613": 0.03, 9 | "gpt-4-32k": 0.06, 10 | "gpt-4-32k-0314": 0.06, 11 | "gpt-4-32k-0613": 0.06, 12 | "gpt-4-vision-preview": 0.01, 13 | "gpt-4-1106-preview": 0.01, 14 | # GPT-4 output 15 | "gpt-4-completion": 0.06, 16 | "gpt-4-0314-completion": 0.06, 17 | "gpt-4-0613-completion": 0.06, 18 | "gpt-4-32k-completion": 0.12, 19 | "gpt-4-32k-0314-completion": 0.12, 20 | "gpt-4-32k-0613-completion": 0.12, 21 | "gpt-4-vision-preview-completion": 0.03, 22 | "gpt-4-1106-preview-completion": 0.03, 23 | # GPT-3.5 input 24 | "gpt-3.5-turbo": 0.0015, 25 | "gpt-3.5-turbo-0301": 0.0015, 26 | "gpt-3.5-turbo-0613": 0.0015, 27 | "gpt-3.5-turbo-1106": 0.001, 28 | "gpt-3.5-turbo-instruct": 0.0015, 29 | "gpt-3.5-turbo-16k": 0.003, 30 | "gpt-3.5-turbo-16k-0613": 0.003, 31 | # GPT-3.5 output 32 | "gpt-3.5-turbo-completion": 0.002, 33 | "gpt-3.5-turbo-0301-completion": 0.002, 34 | "gpt-3.5-turbo-0613-completion": 0.002, 35 | "gpt-3.5-turbo-1106-completion": 0.002, 36 | "gpt-3.5-turbo-instruct-completion": 0.002, 37 | "gpt-3.5-turbo-16k-completion": 0.004, 38 | "gpt-3.5-turbo-16k-0613-completion": 0.004, 39 | } 40 | 41 | 42 | def approximate_costs(fx_args, full_response): 43 | encoding = tiktoken.encoding_for_model(fx_args["model"]) 44 | input_tokens = 0 45 | for message in fx_args["messages"]: 46 | input_tokens += 3 + len(encoding.encode(message["content"])) 47 | output_tokens = 3 + len(encoding.encode(full_response)) 48 | total_cost = ( 49 | MODEL_COST_PER_1K_TOKENS[fx_args["model"]] * input_tokens / 1000 50 | + MODEL_COST_PER_1K_TOKENS[fx_args["model"] + "-completion"] 51 | * output_tokens 52 | / 1000 53 | ) 54 | return dict(total_cost=total_cost, total_tokens=input_tokens + output_tokens) 55 | -------------------------------------------------------------------------------- /backend/test.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | from pydantic import BaseModel 4 | from typing import Optional 5 | import asyncio 6 | import csv 7 | from core.analysis_components import analysis_components 8 | from core.analysis_groups import analysis_groups 9 | from core.analysis_utils import repair_to_full_code 10 | 11 | 12 | 13 | class FileContext(BaseModel): 14 | file_name: str 15 | file_dir: str 16 | file_content: str 17 | root_directory: str = None 18 | example_content: Optional[str] = None 19 | 20 | 21 | 22 | def process_file(file_dir, file_name, root_directory=None, example=None): 23 | """ 24 | processes files 25 | """ 26 | file_path = os.path.join(file_dir, file_name) 27 | with open(file_path, "r") as file: 28 | file_content = file.read() 29 | 30 | example_content = None 31 | if example: 32 | example_path = os.path.join(root_directory, "./" + example) 33 | try: 34 | with open(example_path, "r") as example_file: 35 | example_content = example_file.read() 36 | except FileNotFoundError: 37 | print(f"Could not find example file at {example_path}") 38 | return 39 | 40 | ctx = FileContext( 41 | file_name=file_name, 42 | file_dir=file_dir, 43 | file_content=file_content, 44 | root_directory=root_directory, 45 | example_content=example_content, 46 | ) 47 | return ctx 48 | 49 | 50 | 51 | def load_component_guidelines(file_path): 52 | with open(file_path, "r") as f: 53 | return json.load(f) 54 | 55 | 56 | 57 | def load_system_level_guidelines(file_path): 58 | data_by_property_and_constraint = {} # based on property constraint 59 | 60 | with open(file_path, 'r', newline='', encoding='utf-8') as csvfile: 61 | reader = csv.reader(csvfile) 62 | for row in reader: 63 | # print(row[0:5]) 64 | if row[0] == 'material_design_section': 65 | continue 66 | if len(row) != 6: 67 | print(f"Invalid row number: {row}") 68 | continue 69 | 70 | # check system_design_aspect 71 | if row[1] not in ["Structure","Flow","Layout","Implement","Label","Text","Color","Typography","Shape","Icon","Elevation"]: 72 | print(f"Invalid row High_Level_Subclasses: {row}") 73 | continue 74 | 75 | # check property 76 | if row[2] not in ["Group", "Clickable", "Spacing", "Platform","Label","Text","Color","all elements","Icon"]: 77 | print(f"Invalid row property: {row}") 78 | continue 79 | 80 | # check constraint 81 | if row[3] not in ["soft","hard"]: 82 | print(f"Invalid row constraint: {row}") 83 | continue 84 | 85 | # init 86 | if row[2] not in data_by_property_and_constraint: 87 | data_by_property_and_constraint[row[2]] = {"soft": [], "hard": []} 88 | 89 | # save by property constraint 90 | data_by_property_and_constraint[row[2]][row[3]].append(row) 91 | 92 | return data_by_property_and_constraint 93 | 94 | 95 | if __name__ == "__main__": 96 | file_dir = r"..\examples" # Your frontend code file folder 97 | file_name = "example1.tsx" # Your frontend code file 98 | 99 | # page to be tested 100 | pageurl = "http://localhost:3000" # Your rendered page url 101 | 102 | comp_guidelines_path = r"..\library\components_knowledge_base.json" # component guideline file 103 | system_level_guidelines_path = r"..\library\system_design_knowledge_base.csv" # system level guideline file 104 | 105 | ctx = process_file(file_dir, file_name) 106 | 107 | # make log folder 108 | folder_name = ctx.file_name.split(".")[0] 109 | if not os.path.exists(folder_name): 110 | os.mkdir(folder_name) 111 | 112 | # a.components detect and repair 113 | comp_guidelines = load_component_guidelines(comp_guidelines_path) 114 | comp_suggestions = asyncio.run(analysis_components(ctx, comp_guidelines)) 115 | asyncio.run(repair_to_full_code(ctx, comp_suggestions, "", folder_name, 'comp_')) 116 | 117 | # b.groups detect and repair 118 | system_level_guidelines = load_system_level_guidelines(system_level_guidelines_path) 119 | property_suggestions = asyncio.run(analysis_groups(ctx, system_level_guidelines, pageurl)) 120 | asyncio.run(repair_to_full_code(ctx, "", property_suggestions, folder_name, 'property_')) 121 | 122 | # c.repair 123 | asyncio.run(repair_to_full_code(ctx, comp_suggestions, property_suggestions, folder_name, '')) 124 | 125 | -------------------------------------------------------------------------------- /backend/test_multi.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | from pydantic import BaseModel 4 | from typing import Optional 5 | import asyncio 6 | import csv 7 | from core.analysis_components import analysis_components 8 | from core.analysis_groups import analysis_groups 9 | from core.analysis_utils import repair_to_full_code, repair_to_full_code_multi 10 | import datetime 11 | 12 | 13 | class FileContext(BaseModel): 14 | file_name: str 15 | file_dir: str 16 | file_content: str 17 | root_directory: str = None 18 | example_content: Optional[str] = None 19 | 20 | 21 | 22 | def process_file(file_dir, file_name, root_directory=None, example=None): 23 | """ 24 | processes files 25 | """ 26 | file_path = os.path.join(file_dir, file_name) 27 | with open(file_path, "r") as file: 28 | file_content = file.read() 29 | 30 | example_content = None 31 | if example: 32 | example_path = os.path.join(root_directory, "./" + example) 33 | try: 34 | with open(example_path, "r") as example_file: 35 | example_content = example_file.read() 36 | except FileNotFoundError: 37 | print(f"Could not find example file at {example_path}") 38 | return 39 | 40 | ctx = FileContext( 41 | file_name=file_name, 42 | file_dir=file_dir, 43 | file_content=file_content, 44 | root_directory=root_directory, 45 | example_content=example_content, 46 | ) 47 | return ctx 48 | 49 | 50 | def process_file_multi(file_dir, file_name_list, testname, root_directory=None, example=None): 51 | """ 52 | processes files 53 | """ 54 | file_content = "" 55 | for file_name in file_name_list: 56 | file_path = os.path.join(file_dir, file_name) 57 | with open(file_path, "r") as file: 58 | file_content_single = file.read() 59 | file_content += file_name + ":\n" 60 | file_content += file_content_single 61 | file_content += "\n" 62 | 63 | example_content = None 64 | if example: 65 | example_path = os.path.join(root_directory, "./" + example) 66 | try: 67 | with open(example_path, "r") as example_file: 68 | example_content = example_file.read() 69 | except FileNotFoundError: 70 | print(f"Could not find example file at {example_path}") 71 | return 72 | 73 | now = datetime.datetime.now() 74 | timestamp = now.strftime("%Y%m%d_%H%M%S") 75 | folder_name = f"log/{timestamp}_{testname}" 76 | 77 | ctx = FileContext( 78 | file_name= folder_name, 79 | load_files = str(file_name_list), 80 | file_dir=file_dir, 81 | file_content=file_content, 82 | root_directory=root_directory, 83 | example_content=example_content, 84 | ) 85 | return ctx 86 | 87 | def load_component_guidelines(file_path): 88 | with open(file_path, "r") as f: 89 | return json.load(f) 90 | 91 | 92 | 93 | def load_system_level_guidelines(file_path): 94 | data_by_property_and_constraint = {} # based on property constraint 95 | 96 | with open(file_path, 'r', newline='', encoding='utf-8') as csvfile: 97 | reader = csv.reader(csvfile) 98 | for row in reader: 99 | # print(row[0:5]) 100 | if row[0] == 'material_design_section': 101 | continue 102 | if len(row) != 6: 103 | print(f"Invalid row number: {row}") 104 | continue 105 | 106 | # check system_design_aspect 107 | if row[1] not in ["Structure","Flow","Layout","Implement","Label","Text","Color","Typography","Shape","Icon","Elevation"]: 108 | print(f"Invalid row High_Level_Subclasses: {row}") 109 | continue 110 | 111 | # check property 112 | if row[2] not in ["Group", "Clickable", "Spacing", "Platform","Label","Text","Color","all elements","Icon"]: 113 | print(f"Invalid row property: {row}") 114 | continue 115 | 116 | # check constraint 117 | if row[3] not in ["soft","hard"]: 118 | print(f"Invalid row constraint: {row}") 119 | continue 120 | 121 | # init 122 | if row[2] not in data_by_property_and_constraint: 123 | data_by_property_and_constraint[row[2]] = {"soft": [], "hard": []} 124 | 125 | # save by property constraint 126 | data_by_property_and_constraint[row[2]][row[3]].append(row) 127 | 128 | return data_by_property_and_constraint 129 | 130 | 131 | if __name__ == "__main__": 132 | file_dir = r"..\examples" # Your frontend code file folder 133 | file_name_list = ["example1.tsx"] # Your frontend code file 134 | testname = file_name_list[0].split(".")[0] 135 | 136 | # page to be tested 137 | pageurl = "http://localhost:3000" # Your rendered page url 138 | 139 | comp_guidelines_path = r"..\library\components_knowledge_base.json" # component guideline file 140 | system_level_guidelines_path = r"..\library\system_design_knowledge_base.csv" # system level guideline file 141 | 142 | ctx = process_file_multi(file_dir, file_name_list, testname) 143 | 144 | # make log folder 145 | if not os.path.exists(ctx.file_name): 146 | os.makedirs(ctx.file_name) 147 | 148 | # a.components detect and repair 149 | comp_guidelines = load_component_guidelines(comp_guidelines_path) 150 | comp_suggestions = asyncio.run(analysis_components(ctx, comp_guidelines)) 151 | 152 | # b.groups detect and repair 153 | system_level_guidelines = load_system_level_guidelines(system_level_guidelines_path) 154 | property_suggestions = asyncio.run(analysis_groups(ctx, system_level_guidelines, pageurl)) 155 | 156 | # c.repair 157 | asyncio.run(repair_to_full_code_multi(ctx, comp_suggestions, property_suggestions, ctx.file_name)) 158 | 159 | -------------------------------------------------------------------------------- /scripts/create_knowledge_graph.md: -------------------------------------------------------------------------------- 1 | # For neo4j 2 | 3 | ### OUR Dataset 4 | CREATE (n:Style { name: 'Color', level: 'High level' }) return n; 5 | CREATE (n:Style { name: 'Typography', level: 'High level' }) return n; 6 | CREATE (n:Style { name: 'Icon', level: 'High level' }) return n; 7 | CREATE (n:Style { name: 'Shape', level: 'High level' }) return n; 8 | CREATE (n:Style { name: 'Elevation', level: 'High level' }) return n; 9 | 10 | 11 | CREATE (n:Foundations { name: 'Color', level: 'High level', Subclasses: 'Accessibility'}) return n; 12 | CREATE (n:Foundations { name: 'Text', level: 'High level', Subclasses: 'Accessibility, Content'}) return n; 13 | CREATE (n:Foundations { name: 'Label', level: 'High level', Subclasses: 'Accessibility, Content'}) return n; 14 | CREATE (n:Foundations { name: 'Structure', level: 'High level', Subclasses: 'Accessibility'}) return n; 15 | CREATE (n:Foundations { name: 'Flow', level: 'High level', Subclasses: 'Accessibility'}) return n; 16 | CREATE (n:Foundations { name: 'Layout', level: 'High level', Subclasses: 'Accessibility, Layout'}) return n; 17 | CREATE (n:Foundations { name: 'Implement', level: 'High level', Subclasses: 'Accessibility'}) return n; 18 | 19 | 20 | 21 | CREATE (n:Property_Group { name: 'Color', level: 'Middle level' }) return n; 22 | match(a:Foundations {name:'Color'}),(b:Property_Group {name:'Color'}) create (b)<-[r:Accessibility_contrast]-(a) return r; 23 | match(a:Style {name:'Color'}),(b:Property_Group {name:'Color'}) create (b)<-[r:role_utilities]-(a) return r; 24 | 25 | CREATE (n:Property_Group { name: 'Text', level: 'Middle level' }) return n; 26 | match(a:Style {name:'Typography'}),(b:Property_Group {name:'Text'}) create (b)<-[r:font]-(a) return r; 27 | match(a:Foundations {name:'Text'}),(b:Property_Group {name:'Text'}) create (b)<-[r:Content_Accessibility_size]-(a) return r; 28 | match(a:Foundations {name:'Structure'}),(b:Property_Group {name:'Text'}) create (b)<-[r:Accessibility_heading]-(a) return r; 29 | 30 | CREATE (n:Property_Group { name: 'Label', level: 'Middle level' }) return n; 31 | match(a:Foundations {name:'Label'}),(b:Property_Group {name:'Label'}) create (b)<-[r:Content_Accessibility]-(a) return r; 32 | match(a:Foundations {name:'Structure'}),(b:Property_Group {name:'Label'}) create (b)<-[r:Accessibility_label]-(a) return r; 33 | 34 | CREATE (n:Property_Group { name: 'Group', level: 'Middle level' }) return n; 35 | match(a:Foundations {name:'Structure'}),(b:Property_Group {name:'Group'}) create (b)<-[r:Accessibility_landmark]-(a) return r; 36 | match(a:Foundations {name:'Flow'}),(b:Property_Group {name:'Group'}) create (b)<-[r:Accessibility_priority]-(a) return r; 37 | match(a:Foundations {name:'Layout'}),(b:Property_Group {name:'Group'}) create (b)<-[r:Layout]-(a) return r; 38 | 39 | CREATE (n:Property_Group { name: 'Clickable', level: 'Middle level' }) return n; 40 | match(a:Foundations {name:'Flow'}),(b:Property_Group {name:'Clickable'}) create (b)<-[r:Accessibility_focus_order]-(a) return r; 41 | match(a:Foundations {name:'Layout'}),(b:Property_Group {name:'Clickable'}) create (b)<-[r:Accessibility_touch_target_size]-(a) return r; 42 | match(a:Style {name:'Elevation'}),(b:Property_Group {name:'Clickable'}) create (b)<-[r:floating_element]-(a) return r; 43 | 44 | CREATE (n:Property_Group { name: 'Spacing', level: 'Middle level' }) return n; 45 | match(a:Foundations {name:'Layout'}),(b:Property_Group {name:'Spacing'}) create (b)<-[r:Layout]-(a) return r; 46 | 47 | CREATE (n:Property_Group { name: 'Platform', level: 'Middle level' }) return n; 48 | match(a:Foundations {name:'Implement'}),(b:Property_Group {name:'Platform'}) create (b)<-[r:Accessibility]-(a) return r; 49 | match(a:Foundations {name:'Layout'}),(b:Property_Group {name:'Platform'}) create (b)<-[r:Layout]-(a) return r; 50 | 51 | 52 | 53 | # Elements 54 | 57 | 58 | CREATE (n:Component { name: 'badges', level: 'Low level' }) RETURN n; 59 | CREATE (n:Component { name: 'bottom app bar', level: 'Low level' }) RETURN n; 60 | CREATE (n:Component { name: 'bottom sheets', level: 'Low level' }) RETURN n; 61 | CREATE (n:Component { name: 'common buttons', level: 'Low level' }) RETURN n; 62 | CREATE (n:Component { name: 'cards', level: 'Low level' }) RETURN n; 63 | CREATE (n:Component { name: 'carousel', level: 'Low level' }) RETURN n; 64 | CREATE (n:Component { name: 'checkbox', level: 'Low level' }) RETURN n; 65 | CREATE (n:Component { name: 'chips', level: 'Low level' }) RETURN n; 66 | CREATE (n:Component { name: 'date pickers', level: 'Low level' }) RETURN n; 67 | CREATE (n:Component { name: 'dialogs', level: 'Low level' }) RETURN n; 68 | CREATE (n:Component { name: 'divider', level: 'Low level' }) RETURN n; 69 | CREATE (n:Component { name: 'extended fab', level: 'Low level' }) RETURN n; 70 | CREATE (n:Component { name: 'floating action button', level: 'Low level' }) RETURN n; 71 | CREATE (n:Component { name: 'icon buttons', level: 'Low level' }) RETURN n; 72 | CREATE (n:Component { name: 'lists', level: 'Low level' }) RETURN n; 73 | CREATE (n:Component { name: 'menus', level: 'Low level' }) RETURN n; 74 | CREATE (n:Component { name: 'navigation bar', level: 'Low level' }) RETURN n; 75 | CREATE (n:Component { name: 'navigation drawer', level: 'Low level' }) RETURN n; 76 | CREATE (n:Component { name: 'navigation rail', level: 'Low level' }) RETURN n; 77 | CREATE (n:Component { name: 'progress indicators', level: 'Low level' }) RETURN n; 78 | CREATE (n:Component { name: 'radio button', level: 'Low level' }) RETURN n; 79 | CREATE (n:Component { name: 'search', level: 'Low level' }) RETURN n; 80 | CREATE (n:Component { name: 'segmented buttons', level: 'Low level' }) RETURN n; 81 | CREATE (n:Component { name: 'side sheets', level: 'Low level' }) RETURN n; 82 | CREATE (n:Component { name: 'sliders', level: 'Low level' }) RETURN n; 83 | CREATE (n:Component { name: 'snackbar', level: 'Low level' }) RETURN n; 84 | CREATE (n:Component { name: 'switch', level: 'Low level' }) RETURN n; 85 | CREATE (n:Component { name: 'tabs', level: 'Low level' }) RETURN n; 86 | CREATE (n:Component { name: 'text fields', level: 'Low level' }) RETURN n; 87 | CREATE (n:Component { name: 'time pickers', level: 'Low level' }) RETURN n; 88 | CREATE (n:Component { name: 'tooltips', level: 'Low level' }) RETURN n; 89 | CREATE (n:Component { name: 'top app bar', level: 'Low level' }) RETURN n; 90 | 91 | 92 | 93 | -------------------------------------------------------------------------------- /backend/core/analysis_utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import time 3 | from core.prompts import analysis_system 4 | from core.prompts import regenerate_file_content, merge_suggestions, regenerate_file_content_multi 5 | from core.llm import stream_openai_response 6 | from pydantic import BaseModel 7 | from typing import List 8 | import json 9 | from collections import defaultdict 10 | 11 | class RepairedCode(BaseModel): 12 | filename_with_path: str 13 | repaired_code_file: str 14 | 15 | 16 | class RepairedCodeSchema(BaseModel): 17 | repaired_code_files: List[RepairedCode] 18 | 19 | 20 | class AnalysisComponent(BaseModel): 21 | bad_design_code_filename: str 22 | bad_design_code: str 23 | detailed_reference_from_guidelines: str 24 | suggestion_fix_code: str 25 | 26 | 27 | class AnalysisComponentSchema(BaseModel): 28 | design_suggestion: List[AnalysisComponent] 29 | 30 | 31 | def generate_analysis_report(): 32 | pass 33 | 34 | async def get_response(prompt_messages, functions_schema=None, temperature=0.0, model=None): 35 | async def process_chunk(content: str): 36 | pass 37 | 38 | if not os.environ.get("OPENAI_API_KEY"): 39 | raise Exception("OpenAI API key not found") 40 | 41 | completion = await stream_openai_response( 42 | messages=prompt_messages, 43 | temperature=temperature, 44 | functions=functions_schema, 45 | api_key=os.environ.get("OPENAI_API_KEY"), 46 | callback=lambda x: process_chunk(x), 47 | base_url=None, 48 | model=model, 49 | ) 50 | return completion 51 | 52 | 53 | def assemble_regenerate_prompt(og_prompt, file_content, suggestions): 54 | 55 | regenerate_file_content_prompt = og_prompt.replace("{file_content}", file_content) 56 | regenerate_file_content_prompt = regenerate_file_content_prompt.replace("{suggestions}", suggestions) 57 | print ("------REGENERATE FILE-------") 58 | prompt = [ 59 | { 60 | "role": "system", 61 | "content": analysis_system, 62 | }, 63 | { 64 | "role": "user", 65 | "content": regenerate_file_content_prompt 66 | } 67 | ] 68 | return prompt 69 | 70 | 71 | def assemble_prompt_with_function(og_prompt, file_content, suggestions, function_schema=RepairedCodeSchema.schema()): 72 | 73 | regenerate_file_content_prompt = og_prompt.replace("{file_content}", file_content) 74 | regenerate_file_content_prompt = regenerate_file_content_prompt.replace("{suggestions}", suggestions) 75 | print ("------REGENERATE FILE-------") 76 | prompt = [ 77 | { 78 | "role": "system", 79 | "content": analysis_system, 80 | }, 81 | { 82 | "role": "user", 83 | "content": regenerate_file_content_prompt 84 | } 85 | ] 86 | 87 | functions=[ 88 | { 89 | "name": "repair_to_full_code", 90 | "description": "Repair content of files to full code", 91 | "parameters": function_schema 92 | }, 93 | ] 94 | return prompt, functions 95 | 96 | 97 | def save_code_to_file(code, filename): 98 | try: 99 | with open(filename, 'w') as file: 100 | file.write(code) 101 | print(f"Code saved to {filename} successfully.") 102 | except Exception as e: 103 | print(f"Error occurred while saving the code to {filename}: {e}") 104 | 105 | 106 | 107 | async def repair_to_full_code(ctx, comp_suggestions, property_suggestions, save_path, sub_name=None): 108 | 109 | ### regenerate code(refer aider) 110 | suggestions = str(comp_suggestions) + str(property_suggestions) 111 | 112 | regenerate_prompt = assemble_regenerate_prompt(regenerate_file_content, ctx.file_content, suggestions) 113 | print(regenerate_prompt) 114 | completion = await get_response(regenerate_prompt) 115 | # print(completion) 116 | 117 | # save code to file 118 | # save_dir = ctx.file_dir.replace("orginal", "generated") 119 | # if not os.path.exists(save_dir): 120 | # os.makedirs(save_dir) 121 | 122 | # timestamp = int(time.time() * 1000) 123 | # file_name = str(timestamp) + '.' + ctx.file_name.split(".")[1] 124 | # save_name = os.path.join(save_dir, file_name) 125 | if sub_name: 126 | save_name = sub_name + ctx.file_name 127 | else: 128 | save_name = (ctx.file_name) 129 | 130 | save_path = os.path.join(save_path, save_name) 131 | save_code_to_file(completion, save_path) 132 | print("-----Write to file: " + save_name + " -----") 133 | 134 | 135 | 136 | async def repair_to_full_code_multi_once(ctx, comp_suggestions, property_suggestions, save_path, sub_name=None): 137 | 138 | ### regenerate code(refer aider) 139 | suggestions = str(comp_suggestions) + str(property_suggestions) 140 | 141 | regenerate_prompt, regenerate_schema = assemble_prompt_with_function(regenerate_file_content_multi, ctx.file_content, suggestions) 142 | print(regenerate_prompt) 143 | completion = await get_response(regenerate_prompt, regenerate_schema) 144 | 145 | if "repaired_code_files" in completion: 146 | completion_json = json.loads(completion) 147 | repaired_code_files = completion_json['repaired_code_files'] # remove duplicates 148 | else: 149 | raise Exception("No components found in results") 150 | for code in repaired_code_files: 151 | print(code['filename_with_path']) 152 | 153 | for code in repaired_code_files: 154 | save_name = code['filename_with_path'] 155 | save_content = code['repaired_code_file'] 156 | save_path = os.path.join(save_path, save_name) 157 | save_dir= os.path.dirname(save_path) 158 | #save path, remove filename 159 | if not os.path.exists(save_dir): 160 | os.makedirs(save_dir) 161 | save_code_to_file(save_content, save_path) 162 | print("-----Write to file: " + save_name + " -----") 163 | 164 | 165 | 166 | async def repair_to_full_code_multi(ctx, comp_suggestions, property_suggestions, save_path, sub_name=None): 167 | 168 | ### regenerate code(refer aider) 169 | suggestions = str(comp_suggestions) + str(property_suggestions) 170 | 171 | merge_prompt, merge_schema = assemble_prompt_with_function(merge_suggestions, ctx.file_content, suggestions, AnalysisComponentSchema.schema()) 172 | completion = await get_response(merge_prompt, merge_schema) 173 | 174 | if "design_suggestion" in completion: 175 | completion_json = json.loads(completion) 176 | merged_suggestion = completion_json['design_suggestion'] 177 | else: 178 | raise Exception("No suggestions found in results") 179 | 180 | # log 181 | log_file = os.path.join(save_path, "repair_all_log.json") 182 | # with open(log_file, "a") as file: 183 | # file.write("merged suggestions:" + formatted_json + "\n\n") 184 | with open(log_file, 'w', encoding='utf-8') as f: 185 | json.dump(completion_json, f, indent=4, ensure_ascii=False) 186 | 187 | # refer filename merge suggestions 188 | suggestions_by_file = defaultdict(list) 189 | for suggestion in merged_suggestion: 190 | filename = suggestion['bad_design_code_filename'] 191 | suggestions_by_file[filename].append(suggestion) 192 | 193 | # split repair files 194 | for filename, suggestions in suggestions_by_file.items(): 195 | print(f"Suggestions for {filename}:") 196 | regenerate_prompt, regenerate_schema = assemble_prompt_with_function(regenerate_file_content_multi, ctx.file_content, str(suggestions), RepairedCode.schema()) 197 | completion = await get_response(regenerate_prompt, regenerate_schema) 198 | completion_json = json.loads(completion) 199 | 200 | save_name = completion_json['filename_with_path'] 201 | save_content = completion_json['repaired_code_file'] 202 | save_file = os.path.join(save_path, save_name) 203 | save_dir= os.path.dirname(save_file) 204 | #save path 205 | if not os.path.exists(save_dir): 206 | os.makedirs(save_dir) 207 | 208 | save_code_to_file(save_content, save_file) 209 | print("-----Write to file: " + save_name + " -----") -------------------------------------------------------------------------------- /examples/example1.tsx: -------------------------------------------------------------------------------- 1 | /** 2 | * v0 by Vercel. 3 | * @see https://v0.dev/t/0W13RkH 4 | * Documentation: https://v0.dev/docs#integrating-generated-code-into-your-nextjs-app 5 | */ 6 | export default function Component() { 7 | return ( 8 |
9 |
10 |
11 |
12 |
13 |

14 | Discover Our Unique Features 15 |

16 |

17 | Our features are designed to enhance your productivity and streamline your workflow. 18 |

19 |
20 |
21 |
22 |
23 |
24 | 25 |
26 |

Smart Inbox

27 |

28 | Our Smart Inbox feature helps you manage your emails efficiently by prioritizing important emails. 29 |

30 |
31 |
32 |
33 | 34 |
35 |

Seamless Integration

36 |

37 | Seamless Integration allows you to connect with your favorite apps and services without leaving your 38 | inbox. 39 |

40 |
41 |
42 |
43 | 44 |
45 |

Advanced Customization

46 |

47 | With Advanced Customization, you can personalize your email client to suit your preferences and work 48 | style. 49 |

50 |
51 |
52 |
53 | 54 |
55 |

Powerful Search

56 |

57 | Our Powerful Search feature allows you to find any email, contact, or file in seconds. 58 |

59 |
60 |
61 |
62 | 63 |
64 |

Reliable Security

65 |

66 | With Reliable Security, your data is always safe and protected. 67 |

68 |
69 |
70 |
71 | 72 |
73 |

Easy Collaboration

74 |

75 | Easy Collaboration allows you to share and edit documents with your team in real time. 76 |

77 |
78 |
79 |
80 |
81 |
82 |
83 |
84 | ) 85 | } 86 | 87 | function InboxIcon(props) { 88 | return ( 89 | 101 | 102 | 103 | 104 | ) 105 | } 106 | 107 | 108 | function LockIcon(props) { 109 | return ( 110 | 122 | 123 | 124 | 125 | ) 126 | } 127 | 128 | 129 | function MergeIcon(props) { 130 | return ( 131 | 143 | 144 | 145 | 146 | 147 | ) 148 | } 149 | 150 | 151 | function SearchIcon(props) { 152 | return ( 153 | 165 | 166 | 167 | 168 | ) 169 | } 170 | 171 | 172 | function SettingsIcon(props) { 173 | return ( 174 | 186 | 187 | 188 | 189 | ) 190 | } 191 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models 2 | 3 | 4 | 5 | **Authors:** Mingyue Yuan, Jieshan Chen\*, Zhenchang Xing, Aaron Quigley, Yuyu Luo, Tianqi Luo, Gelareh Mohammadi, Qinghua Lu, Liming Zhu 6 | 7 | DesignRepair is a **dual-stream, knowledge-driven** approach leveraging **Large Language Models (LLMs)** to detect and repair design quality issues in frontend code. It incorporates both **source code analysis** and **user-perceived rendered view analysis**, guided by **Material Design 3** guidelines. 8 | 9 | ### Key Features: 10 | 11 | - **Dual-Stream Analysis**: Simultaneously analysis the source code and its properties of rendered output to identify design issues. 12 | - **Knowledge-Driven Repair**: Integrates Material Design 3 guidelines to ensure repairs align with modern design standards. 13 | - **LLM-Powered Enhancements**: Utilizes the power of Large Language Models to detect and fix design issues effectively. 14 | 15 | For more details, read our [paper](https://arxiv.org/abs/2411.01606) on arXiv. 16 | 17 | ## Abstract 18 | 19 | The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's V0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities comprehensively. Moreover, none of them examine the quality of LLM-generated UI design. 20 | In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. 21 | Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics. 22 | 23 | ## Approach 24 | 25 |
26 | 27 |
28 |

Framework Overview of DesignRepair

29 |
30 | 31 | The overview illustrates the overview of our approach, DesignRepair, which consists of three phases, namely, an offline knowledge base construction, online page extraction and knowledge-driven repair phases. 32 | 33 | For the offline knowledge base construction phase A in picture, we built a two-part knowledge base (KB): a low-level Component Knowledge Base (KB-Comp) and a high-level System Design Knowledge Base (KB-System). This knowledge base functions as a domain expert, offering guidance for addressing potential UI issues. 34 | Given the frontend code and rendered page, we enter the second phase, where we use a parallel dual-pipe method to extract the used components and their corresponding property groups B in picture. 35 | Finally, we implement a knowledge-driven, LLM-based repair method enhanced with Retrieval-Augmented Generation (RAG) techniques (C in picture). This approach allows us to meticulously analyze and repair issues concurrently. By employing a divide and conquer strategy, we tackle each component/property group individually before synthesizing the repairs. This ensures a cohesive, optimized final output, achieved through a thorough and scrutinized repair process. 36 | 37 | ## Description 38 | 39 | | Section of Pipeline | File Path | Info | 40 | | --- | --- | --- | 41 | | Component Knowledge Base | [`.\library\component\knowledge_base.json`](https://github.com/UGAIForge/DesignRepair/blob/main/library/component_knowledge_base.json) | Extracted from https://m3.material.io/components of the Material Design official documents, it includes 24 component types, and their corresponding guidelines | 42 | | System Knowledge Base | [`.\library\component\system_base.csv`](https://github.com/UGAIForge/DesignRepair/blob/main/library/system_design_knowledge_base.csv) | Extracted from https://m3.material.io/foundations and https://m3.material.io/styles of the Material Design official documents, and formed into 7 types of Property Groups: Group, Clickable, Spacing, Platform, Label, Text, and Color. The mapping relationship can be visualized in Neo4j using .\scripts\create_knowledge_graph.md. | 43 | | Prompts | [`.\backend\core\prompts.py`](https://github.com/UGAIForge/DesignRepair/blob/main/backend/core/prompts.py) | All prompts, including P_comp_extra, P_map_kb, P_individual, P_all | 44 | | Knowledge Base Extraction | [`.\scripts\prepare_kb_dump.py`](https://github.com/UGAIForge/DesignRepair/blob/main/scripts/prepare_kb_dump.py) | Document processing script, structured into knowledge database | 45 | 46 | ### Prompts Detail 47 | 48 | The prompts' content is located in the file `.\backend\core\prompts.py`. 49 | 50 | The following table outlines the prompt names in the paper, their corresponding variable names in the code, and their respective information: 51 | 52 | | Prompt Name in Paper | Variable Name in Code | Information | 53 | | --- | --- | --- | 54 | | P_comp_extra | get_related_components_prompt_web_page_simpler, get_related_components_prompt_web_page_complex | For step B1, extract related components type list | 55 | | P_map_kb | get_related_components_prompt_library | For step C1, map page components list to library component name list | 56 | | P_individual_components | components_analysis_content | For step C2, analyze components-level design issues | 57 | | P_individual_property | property_analysis_content | For step C2, analyze system-level design issues | 58 | | P_all | regenerate_file_content or regenerate_file_content_multi | For step C3, summarize and analyze all design issues and generate fixed page code | 59 | 60 | 61 | ## Configuration 62 | 63 | ### Installation 64 | 65 | Follow these steps to install and setup the project: 66 | 67 | 1. Clone the repository: 68 | ```bash 69 | git clone https://github.com/UGAIForge/DesignRepair.git 70 | ``` 71 | 72 | 2. Navigate to the project directory: 73 | ```bash 74 | cd DesignRepair2024 75 | ``` 76 | 77 | 3. Install the required packages: 78 | ```bash 79 | cd backend 80 | poetry install 81 | poetry shell 82 | ``` 83 | 84 | ### add openai key 85 | create file `.envs` 86 | 87 | OPENAI_API_KEY="keyhere" 88 | 89 | ### Usage 90 | 91 | Update the content of `text.py` to configure your local environment to run the code: 92 | 93 | ```python 94 | if __name__ == "__main__": 95 | file_dir = r"..\examples" # Your frontend code file folder 96 | file_name = "example1.tsx" # Your frontend code file 97 | 98 | # page to be tested 99 | pageurl = "http://localhost:3000" # Your rendered page url 100 | ``` 101 | 102 | To run the project, execute the `test.py` script: 103 | 104 | ```bash 105 | python test.py 106 | ``` 107 | 108 | To test multiple files simultaneously, execute the `test_multi.py` script: 109 | 110 | ```bash 111 | python test_multi.py 112 | ``` 113 | 114 | 115 | 116 | ## Here are some compare results of our DesignRepair 117 | 118 | 119 | ### example1 before vs after 120 |
121 | 122 | 123 |
124 | 125 | ### example2 before vs after 126 |
127 | 128 | 129 |
130 | 131 | ### example3 before vs after 132 |
133 | 134 | 135 |
136 | 137 | ### example4 before vs after 138 |
139 | 140 | 141 |
142 | 143 | ### example5 before vs after 144 |
145 | 146 | 147 |
148 | 149 | 150 | ## Citation 151 | ``` 152 | @article{yuan2024designrepair, 153 | title={DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models}, 154 | author={Yuan, Mingyue and Chen, Jieshan and Xing, Zhenchang and Quigley, Aaron and Luo, Yuyu and Luo, Tianqi and Mohammadi, Gelareh and Lu, Qinghua and Zhu, Liming}, 155 | journal={arXiv preprint arXiv:2411.01606}, 156 | year={2024} 157 | } 158 | ``` 159 | 160 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /backend/core/analysis_components.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | import jinja2 4 | import json 5 | import asyncio 6 | import copy 7 | import time 8 | from openai import OpenAI 9 | from pydantic import BaseModel 10 | from typing import Optional, List 11 | 12 | # from core.approximate_costs import approximate_costs 13 | from core.prompts import analysis_system 14 | from core.prompts import get_related_components_prompt_web_page, get_related_components_prompt_web_page_simpler, get_related_components_prompt_web_page_complex 15 | from core.prompts import get_related_components_prompt_library 16 | from core.prompts import components_analysis_content 17 | from core.analysis_utils import get_response 18 | 19 | from dotenv import load_dotenv 20 | load_dotenv() 21 | 22 | client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY")) 23 | 24 | # Define the schema using 25 | class ComponentSchema(BaseModel): 26 | components: List[str] 27 | 28 | class AnalysisComponent(BaseModel): 29 | bad_design_code_filename: str 30 | bad_design_code: str 31 | detailed_reference_from_guidelines: str 32 | suggestion_fix_code: str 33 | 34 | class AnalysisComponentSchema(BaseModel): 35 | bad_component_design: List[AnalysisComponent] 36 | 37 | 38 | def extract_components_list(comp_guidelines): 39 | components_list = [] 40 | for comp in comp_guidelines: 41 | components_list.append(comp['component_type']) 42 | return components_list 43 | 44 | 45 | test_content = '''{ 46 | "bad_component_design": [ 47 | { 48 | "bad_design_code": "", 49 | "detailed_reference_from_guidelines": "Component: badges\nAnatomy#Container#Badges have fixed positions. Don't change the position of the badge arbitrarily or place the badge over the icon.", 50 | "suggestion_fix_code": "" 51 | }, 52 | { 53 | "bad_design_code": "{email.tags.join(', ')}", 54 | "detailed_reference_from_guidelines": "Component: badges\nAnatomy#Label text#Don't let the badge get cut off 55 | or collide with another element", 56 | "suggestion_fix_code": "{email.tags.join(', ')}" 57 | }, 58 | { 59 | "bad_design_code": "", 60 | "detailed_reference_from_guidelines": "Component: checkbox\nUsage#None#Checkboxes let users select one or more options from a list. A parent checkbox allows for easy selection or deselection of all items.", 61 | "suggestion_fix_code": "" 62 | }, 63 | { 64 | "bad_design_code": "", 65 | "detailed_reference_from_guidelines": "Component: icon buttons\nBehavior#Selection#Don't use toggle icon buttons for actions that don't have a selected state, such as an icon button for an overflow menu", 66 | "suggestion_fix_code": "" 67 | } 68 | ] 69 | }''' 70 | 71 | # TODO - Test segment code into chunks, and for each chunk find the relevant components in the library 72 | def assemble_get_web_components_prompt(file_content, sys_prompt): 73 | print ("------GET WEB COMPONENTS-------") 74 | prompt = [ 75 | { 76 | "role": "system", 77 | "content": analysis_system, 78 | }, 79 | { 80 | "role": "user", 81 | "content": sys_prompt.replace("{file_content}", file_content) 82 | } 83 | ] 84 | functions=[ 85 | { 86 | "name": "get_web_page_components", 87 | "description": "Get all the components name in the web page", 88 | "parameters": ComponentSchema.schema() 89 | }, 90 | ] 91 | return prompt, functions 92 | 93 | 94 | def assemble_get_library_components_prompt(page_response, components_list): 95 | print ("------GET LIBRARY COMPONENTS-------") 96 | prompt =[ 97 | { 98 | "role": "system", 99 | "content": analysis_system, 100 | }, 101 | { 102 | "role": "user", 103 | "content": get_related_components_prompt_library.replace("{components_list}", str(components_list)) 104 | }, 105 | { 106 | "role": "user", 107 | "content": "You have the following list of components in the web page:\n" 108 | + page_response + "\n" + 109 | "Get all the corresponding components name in library.\n" 110 | }] 111 | 112 | functions=[ 113 | { 114 | "name": "get_library_components", 115 | "description": "Get all the corresponding components name in library", 116 | "parameters": ComponentSchema.schema() 117 | }, 118 | ] 119 | return prompt, functions 120 | 121 | 122 | def assemble_components_analysis_prompt(file_content, related_guidelines): 123 | print ("-----ANALYZE COMPONENTS-------") 124 | def get_soft_constraints(related_guidelines): 125 | soft_constraints = "" 126 | for item in related_guidelines: 127 | # Extract the name of the component 128 | component_name = item.get('component_type', 'Unnamed Component') 129 | guidelines = item.get('guidelines', {}).get('soft', {}) 130 | 131 | # Append component name 132 | soft_constraints += f"Component: {component_name}\n" 133 | 134 | for section, content in guidelines.items(): 135 | soft_constraints += f"{section}:\n" 136 | for sub_section, sub_content in content.items(): 137 | soft_constraints += f"- {sub_section}: {sub_content}\n" 138 | 139 | soft_constraints += "\n\n" # Add a newline for separation between items 140 | 141 | return soft_constraints.strip() 142 | 143 | def get_hard_constraints(related_guidelines): 144 | hard_constraints = "" 145 | for item in related_guidelines: 146 | # Extract the name of the component 147 | component_name = item.get('component_type', 'Unnamed Component') 148 | 149 | constraints = item['guidelines'].get('hard', {}) 150 | do_constraints = constraints.get('do', []) 151 | dont_constraints = constraints.get('dont', []) 152 | 153 | # Append component name 154 | hard_constraints += f"Component: {component_name}\n" 155 | 156 | # Append 'Do' and 'Don't' constraints 157 | if do_constraints: 158 | hard_constraints += "Do:\n" + "\n".join(do_constraints) 159 | if dont_constraints: 160 | hard_constraints += "\nDon't:\n" + "\n".join(dont_constraints) 161 | 162 | hard_constraints += "\n\n" # Add extra newline for separation between components 163 | 164 | return hard_constraints.strip() 165 | 166 | components_analysis_content_prompt = components_analysis_content 167 | components_analysis_content_prompt = components_analysis_content_prompt.replace("{file_content}", file_content) 168 | # components_analysis_content_prompt = components_analysis_content_prompt.replace("{soft_constraints}", get_soft_constraints(related_guidelines)) 169 | components_analysis_content_prompt = components_analysis_content_prompt.replace("{hard_constraints}", get_hard_constraints(related_guidelines)) 170 | 171 | prompt =[ 172 | { 173 | "role": "system", 174 | "content": analysis_system, 175 | }, 176 | { 177 | "role": "user", 178 | "content": components_analysis_content_prompt 179 | }, 180 | { 181 | "role": "system", 182 | "content": "Please respond ONLY with valid json that conforms to this pydantic json_schema: {model_class.schema_json()}.\n" 183 | }] 184 | 185 | functions=[ 186 | { 187 | "name": "analyze_components", 188 | "description": "analyze ", 189 | "parameters": AnalysisComponentSchema.schema() 190 | }, 191 | ] 192 | return prompt, functions 193 | 194 | 195 | 196 | def get_related_comp_guidelines(need_list, comp_guidelines, components_list): 197 | related_guidelines = [] 198 | 199 | for needed_comp in need_list: 200 | if needed_comp.lower() in components_list: 201 | print("Found component in library: " + needed_comp) 202 | for single_guideline in comp_guidelines: 203 | if single_guideline['component_type'] == needed_comp: 204 | related_guidelines.append(single_guideline) 205 | break 206 | return related_guidelines 207 | 208 | 209 | 210 | async def analysis_components(ctx, comp_guidelines): 211 | components_list = extract_components_list(comp_guidelines) 212 | # print(str(components_list)) 213 | 214 | # define log file 215 | folder_name = ctx.file_name.split(".")[0] 216 | log_file = os.path.join(folder_name, "components.log") 217 | 218 | # get web simpler components 219 | get_web_components_prompt, web_components_schema = assemble_get_web_components_prompt(ctx.file_content, get_related_components_prompt_web_page_simpler) 220 | completion_simpler = await get_response(get_web_components_prompt, web_components_schema, model="gpt-4-turbo-2024-04-09") 221 | print(completion_simpler) 222 | completion_simpler_list = list(json.loads(completion_simpler)['components']) 223 | if len(completion_simpler_list) >= 10: 224 | get_web_components_prompt.append( 225 | { 226 | "role": "user", 227 | "content": "Don't include all the components! Only the most relevant ones in the webpage." 228 | } 229 | ) 230 | completion_simpler = await get_response(get_web_components_prompt, web_components_schema, model="gpt-4-turbo-2024-04-09") 231 | print("second try",completion_simpler) 232 | completion_simpler_list = list(json.loads(completion_simpler)['components']) 233 | with open(log_file, "a") as file: 234 | file.write("simpler components" + completion_simpler + "\n") 235 | 236 | 237 | get_web_components_prompt, web_components_schema = assemble_get_web_components_prompt(ctx.file_content, get_related_components_prompt_web_page_complex) 238 | completion_complex = await get_response(get_web_components_prompt, web_components_schema, model="gpt-4-turbo-2024-04-09") 239 | print(completion_complex) 240 | completion_complex_list = list(json.loads(completion_complex)['components']) 241 | if len(completion_complex_list) >= 5: 242 | get_web_components_prompt.append( 243 | { 244 | "role": "user", 245 | "content": "Don't include all the components! Only the most relevant ones." 246 | } 247 | ) 248 | completion_complex = await get_response(get_web_components_prompt, web_components_schema, model="gpt-4-turbo-2024-04-09") 249 | print("second try",completion_complex) 250 | completion_complex_list = list(json.loads(completion_complex)['components']) 251 | if len(completion_complex_list) >= 15: 252 | completion_complex_list = [] 253 | with open(log_file, "a") as file: 254 | file.write("complex components" + completion_complex + "\n") 255 | 256 | components_lists = completion_simpler_list + completion_complex_list 257 | 258 | # get library components 259 | get_library_components_prompt, lib_components_schema = assemble_get_library_components_prompt(str(components_lists), components_list) 260 | completion = await get_response(get_library_components_prompt, lib_components_schema, model="gpt-4-turbo-2024-04-09") 261 | print(completion) 262 | with open(log_file, "a") as file: 263 | file.write("library components" + completion + "\n") 264 | 265 | # extract components 266 | my_list = [] 267 | if "components" in completion: 268 | completion_json = json.loads(completion) 269 | my_list = list(set(completion_json['components'])) # remove duplicates 270 | else: 271 | raise Exception("No components found in results") 272 | print(my_list) 273 | with open(log_file, "a") as file: 274 | file.write("simpler components" + completion + "\n") 275 | 276 | # get related components guidelines 277 | related_guidelines = get_related_comp_guidelines(my_list, comp_guidelines, components_list) 278 | 279 | # get components analysis prompt 280 | components_analysis_prompt, components_analysis_schema = assemble_components_analysis_prompt(ctx.file_content, related_guidelines) 281 | # print(components_analysis_prompt) 282 | # completion = await get_response(components_analysis_prompt, components_analysis_schema, 0.2) 283 | completion = await get_response(components_analysis_prompt, components_analysis_schema) 284 | # print(completion) 285 | with open(log_file, "a") as file: 286 | file.write(completion + "\n") 287 | 288 | return completion -------------------------------------------------------------------------------- /backend/core/prompts.py: -------------------------------------------------------------------------------- 1 | 2 | analysis_system = """ 3 | You are an expert frontend developer. 4 | Your task is to analyze the web page and provide suggestions to fix the bad design. 5 | """ 6 | 7 | 8 | get_related_components_prompt_web_page = """ 9 | Here is the web page you need to analyze: 10 | '''{file_content}''' 11 | -Summarize the web page, break it down into smaller components. 12 | Extract all the components from the web page, save it in a list {"components"}. 13 | """ 14 | 15 | 16 | 17 | get_related_components_prompt_web_page_simpler = """ 18 | Here is the web page you need to analyze: 19 | '''{file_content}''' 20 | -Summarize the web page, break it down into smaller components. 21 | -Carefully analyze the Tags, import library and function (HTML elements like div, img, p, section, svg, and h1 (and comment containers) are not considered UI components, wheras other special HTML elements are considered UI components, like input). 22 | -Further more, You need to analyze is there any simpler UI components of the following web components: 23 | 24 | [badges: Small labels that display information or status. (Often implemented using HTML elements with CSS styling), 25 | buttons: Interactive elements that trigger actions. (Can be achieved using HTML button elements and CSS), 26 | checkbox: A UI element that allows users to select one or more options from a set. (Implemented with HTML checkbox elements and potentially CSS for styling), 27 | date pickers: Components that allow users to select a date. (Usually involve HTML input elements with specialized JavaScript libraries for date selection), 28 | text fields: Input areas for users to enter text. (Simply HTML input elements with type="text or input"), 29 | time pickers: Components that allow users to select a time. (Similar to date pickers, using HTML input elements and JavaScript libraries), 30 | sliders: Components that allow users to select a value from a range. (Often involve HTML input elements with type="range" and CSS), 31 | radio button: UI elements where only one option can be selected at a time from a group. (Implemented with HTML radio buttons and potentially CSS), 32 | icon buttons: Buttons that display an icon instead of text. (Can be achieved using HTML button elements, CSS for styling, and potentially Font Awesome or other icon libraries), 33 | extended fab: An extended floating action button with additional functionality. (Usually a custom component using HTML, CSS, and JavaScript), 34 | floating action button: A circular button that typically performs a primary action. (Can be implemented with HTML button), 35 | progress Indicators: Visual elements that show the status of an ongoing process. (Can be implemented with HTML, CSS, and JavaScript)] 36 | 37 | 38 | Extract all the components name from the web page, save it in a list {"components"}. 39 | """ 40 | 41 | get_related_components_prompt_web_page_complex = """ 42 | Here is the web page you need to analyze: 43 | '''{file_content}''' 44 | -You need to analyze is there any complex UI components of the following web components name in the web page: 45 | 46 | [bottom app bar: A bar at the bottom of the screen that provides essential actions for the current view. (Requires multiple HTML elements and CSS styles), 47 | bottom sheets: Modal sheets that slide up from the bottom of the screen. (Requires a combination of HTML, CSS, and potentially JavaScript for interactivity), 48 | cards: Informative elements that showcase content and actions. (Often involve various HTML elements and CSS for styling and layout), 49 | carousel: A component that automatically or manually cycles through a series of content items. (Involves HTML, CSS, and JavaScript for animation and interaction), 50 | chips: Compact components that represent a choice, attribute, or action. (Multiple HTML elements and CSS for styling), 51 | dialogs: Modal windows that provide focused content and require user interaction. (Usually involve multiple HTML elements, CSS for styling, and JavaScript for behavior), 52 | divider: A visual separator between sections of content. (Often implemented using HTML elements with CSS styling), 53 | lists: Ordered or unordered lists of items that can be interactive or non-interactive. (Can be achieved with HTML ul and li elements, potentially with CSS), 54 | menus: Overlays that display navigation or options. (Typically involve HTML, CSS, and JavaScript for dynamic behavior), 55 | navigation bar: A horizontal bar at the top or bottom of a web page that provides primary navigation links. (Requires HTML elements and CSS for styling), 56 | navigation drawer: A vertical panel that slides in from the side of the screen for navigation. (Usually involves HTML, CSS, and JavaScript for animation and interaction), 57 | navigation rail: A persistent navigation bar that appears on the side of the screen. (Requires HTML elements and CSS for styling), 58 | search: A component that allows users to search for content. (Often implemented using HTML input elements, potentially with JavaScript for advanced search functionality), 59 | segmented buttons: A group of buttons where only one can be selected at a time. (Requires a combination of HTML elements and CSS for styling), 60 | side sheets: Modal sheets that slide in from the side of the screen. (Similar to bottom sheets, involving HTML, CSS, and potentially JavaScript), 61 | snackbar: A brief temporary message that appears at the bottom of the screen. (Usually involves HTML, CSS animations, and potentially JavaScript for timing), 62 | switch: A toggle component that can be switched on or off. (Can be implemented with HTML checkboxes and CSS styling), 63 | tabs: A component that allows users to switch between different views. (Requires HTML elements, CSS for styling, and JavaScript for tab switching functionality), 64 | tooltips: Informative popups that appear on hover or click. (Involve HTML elements with CSS positioning and JavaScript for behavior), 65 | top app bar: A bar at the top of the screen that often includes the application title and actions. (Similar to bottom app bar, requiring HTML elements and CSS styles)] 66 | 67 | Only extract the most related complex components name from the web page, save it in a list {"components"}. 68 | """ 69 | 70 | 71 | get_related_components_prompt_library = """ 72 | Multiple component guidelines can be used while checking the component in order to help find the mistake and improve. 73 | You have the following list of [library component name]: 74 | '''{components_list}''' 75 | -Map web page components you got, to the corresponding library components. save it in a list {"components"}. 76 | 77 | Notice, the web page component name and the library component name do not correspond exactly. 78 | Tip: You can decompose components or combine components to find the corresponding components in the library. 79 | Please try your best to find it. 80 | Only save the exact library component name in the list {"components"} 81 | """ 82 | 83 | components_soft_analysis_content = """ 84 | 85 | #### 86 | Here is the current content of the files you need to analyze: 87 | '''{file_content}''' 88 | 89 | #### 90 | Multiple component guidelines can be used while checking the component in order to help find the mistake and improve. 91 | [Anatomy, Behavior, Placement, Responsive layout, Usage] are the most common component guidelines you need to check. 92 | 93 | Here are the related guidelines: 94 | 95 | Here are the general guidelines you can use, we name it "soft constraints", 96 | REMEMBER THIS IS NOT MANDATORY, REGARDED AS OPTIONAL. 97 | '''{soft_constraints}''' 98 | 99 | """ 100 | 101 | test = """ 102 | 103 | """ 104 | 105 | 106 | components_analysis_content = """ 107 | 108 | #### 109 | Here is the current content of the files you need to analyze: 110 | '''{file_content}''' 111 | 112 | #### 113 | Multiple component guidelines can be used while checking the component in order to help find the mistake and improve. 114 | [Anatomy, Behavior, Placement, Responsive layout, Usage] are the most common component guidelines you need to check. 115 | 116 | Here are the related guidelines: 117 | 118 | Here are the guidelines you must follow, we name it "hard constraints", 119 | REMEMBER THIS IS MANDATORY, ONCE YOU FIND A BAD DESIGN NOT FOLLOWING THE GUIDELINE, YOU MUST FIX IT. FIND AS MANY BAD DESIGN AS POSSIBLE. 120 | '''{hard_constraints}''' 121 | 122 | #### 123 | Use the above instructions to analyze the supplied files, and provide detailed suggestions to fix all the bad design. 124 | To suggest changes to a file you MUST return the bad design code snippet with its filename, detailed reference guidelines, and fix code suggestion. in the json format. 125 | '''{bad_design_code_filename}''' 126 | '''{bad_design_code}''' 127 | '''{detailed_reference_from_guidelines}''' 128 | '''{suggestion_fix_code}''' 129 | find as many bad design as possible. 130 | """ 131 | 132 | 133 | property_analysis_content = """ 134 | #### 135 | Here is the current content of the files you need to analyze: 136 | '''{file_content}''' 137 | 138 | #### 139 | Here is the detailed properties of '''{property_type}''' we extracted from the above web page you need to analyze: 140 | '''{properties}''' 141 | 142 | #### 143 | Multiple High level guidelines related to '''{property_type}''' properties can be used while checking in order to help find the mistake and improve. 144 | '''{property_type}''' is the most important aspect you need to check. 145 | 146 | Here are the related guidelines: 147 | 148 | Here are the guidelines you must follow, we name it "hard constraints", 149 | REMEMBER THIS IS MANDATORY, ONCE YOU FIND A BAD DESIGN NOT FOLLOWING THE GUIDELINE, YOU MUST FIX IT. FIND AS MANY BAD DESIGN AS POSSIBLE. 150 | '''{hard_constraints}''' 151 | 152 | #### 153 | Use the above instructions to analyze the supplied files, and provide detailed suggestions to fix all the bad design. 154 | To suggest changes to a file you MUST return the bad design code snippet, with its filename, detailed reference guidelines, and fix code suggestion. in the json format. 155 | '''{bad_design_code_filename}''' 156 | '''{bad_design_code}''' 157 | '''{detailed_reference_from_guidelines}''' 158 | '''{suggestion_fix_code}''' 159 | find as many bad design as possible. 160 | """ 161 | 162 | # - For images, use placeholder images from https://placehold.co and include a detailed description of the image in the alt text so that an image generation AI can generate the image later. 163 | regenerate_file_content = """ 164 | To suggest changes to a file you MUST return the entire content of the updated file. 165 | 166 | Here is the current content of the files you need to modify: 167 | '''{file_content}''' 168 | 169 | Here are the bad design code snippet, detailed reference guidelines, and fix code suggestion. 170 | '''{suggestions}''' 171 | 172 | - There may be conflicts between bad design code snippets. Please try your best to merge the conflicts to give the best improvement results. 173 | - Do not add comments in the code such as "" and "" in place of writing the full code. WRITE THE FULL CODE. 174 | - Repeat elements as needed. For example, if there are 15 items, the code should have 15 items. DO NOT LEAVE comments like "" or bad things will happen. 175 | 176 | Remember, You are an expert frontend developer, and frontend designer. 177 | Only focusing on the most important aspect to improve. 178 | You MUST make the color, layout in beatiful and consistent design, and readability!!! 179 | 180 | Return only the full code you improved. 181 | Do not include markdown "```" or "```jsx" at the start or end. 182 | """ 183 | 184 | merge_suggestions = """ 185 | Here is the current content of the files you need to modify: 186 | '''{file_content}''' 187 | 188 | Here are the bad design code snippet, detailed reference guidelines, and suggested fixes. 189 | '''{suggestions}''' 190 | 191 | - There may be conflicts between the bad design code snippets. Please try your best to resolve these conflicts and merge suggestions to provide the best improvement results. 192 | 193 | Review the suggestions and rethink how to fix the bad design using a consistent approach. 194 | To suggest changes to a file, you MUST return the bad design code snippet, with its filename, detailed reference guidelines, and suggested fix in JSON format. 195 | For code snippets being fixed in the same file, use the exact same filename and try to place the fixed code in adjacent order. 196 | '''{bad_design_code_filename}''' 197 | '''{bad_design_code}''' 198 | '''{detailed_reference_from_guidelines}''' 199 | '''{suggested_fix_code}''' 200 | """ 201 | 202 | # - check all the suggestions one by one and fix the file. 203 | regenerate_file_content_multi = """ 204 | To suggest changes to a file you MUST return the entire content of the updated file. 205 | 206 | Here is the current content of the files you need to modify: 207 | '''{file_content}''' 208 | 209 | Here are the bad design code snippet, detailed reference guidelines, and fix code suggestion. 210 | '''{suggestions}''' 211 | 212 | - Do not add comments in the code such as "" and "" in place of writing the full code. WRITE THE FULL CODE. 213 | - Repeat elements as needed. For example, if there are 15 items, the code should have 15 items. DO NOT LEAVE comments like "" or bad things will happen. 214 | 215 | To suggest changes to a file you MUST return the entire content of the updated file. 216 | Every file listing MUST use this format: with the filename with any originally provided path and entire content of the file in the json format. 217 | '''{filename_with_path}''' 218 | '''{repaired_code_file}''' 219 | """ 220 | -------------------------------------------------------------------------------- /scripts/prepare_kb_dump.py: -------------------------------------------------------------------------------- 1 | import re 2 | import os 3 | import json 4 | import copy 5 | import pandas as pd 6 | import numpy as np 7 | import matplotlib.pyplot as plt 8 | 9 | component_info = { 10 | "component_type": "", 11 | "description": "", 12 | "guidelines": { 13 | "docs_path": None, 14 | "hard": { 15 | "do": [], 16 | "dont": [] 17 | }, 18 | "soft": {} 19 | }, #-<> -![] 20 | # "accessbility": { 21 | # "docs_path": None, 22 | # "constraints": { 23 | # "do": [], 24 | # "dont": [] 25 | # }, 26 | # "general": {} 27 | # }, 28 | # "specs": { 29 | # "Measurements": "", 30 | # "docs_path": "" 31 | # }, 32 | } 33 | 34 | 35 | def find_matches(pattern, content, flags=0): 36 | return [(m.group(0), m.start(0)) for m in re.finditer(pattern, content, flags)] 37 | 38 | 39 | def extract_content(content): 40 | 41 | # Extracting title 42 | header_matches = find_matches(r'^(#### .*|### .*|## .*|# .*)$', content, re.MULTILINE) 43 | # Extracting bulleted lists 44 | bullet_list_matches = find_matches(r'^\*(?!\*).+', content, re.MULTILINE) 45 | # Extracting ![] content 46 | image_alt_matches = find_matches(r'!\[(?!img\])([^]]+)\]', content) 47 | # Extracting lines that start with specific patterns like a., b., 1., etc. 48 | numbered_lists_matches = find_matches(r'^\d+\.\s.*$', content, re.MULTILINE) 49 | # Extracting table rows 50 | table_matches_matches = find_matches(r'^\|.*\|$', content, re.MULTILINE) 51 | # Extracting 'checkDo' and the next relevant lines based on the given condition 52 | # checkdo_matches = find_matches(r'checkDo\s*\n(.*?)(?:\n|$)(?(1)(.*?\n|)|(?:\n(.*?\n.*?)\n|))', content) 53 | checkdo_matches = find_matches(r'checkDo\s*(.*?)(?:\n|$)(?(1)(.*?\n|)|(?:\n(.*?\n.*?)\n|))', content) 54 | # Extracting 'closeDon’t' and the next relevant lines based on the given condition 55 | closedont_matches = find_matches(r'closeDon’t\s*(.*?)(?:\n|$)(?(1)(.*?\n|)|(?:\n(.*?\n.*?)\n|))', content) 56 | 57 | # Merge all matched results and sort them based on their position in the original document 58 | all_matches = sorted(header_matches + bullet_list_matches + image_alt_matches + numbered_lists_matches + table_matches_matches + checkdo_matches + closedont_matches, key=lambda x: x[1]) 59 | 60 | # Extract the results after sorting 61 | sorted_results = [match[0] for match in all_matches] 62 | 63 | # Count the number of matches for each type 64 | list_count = len(bullet_list_matches) + len(numbered_lists_matches) 65 | image_alt_count = len(image_alt_matches) 66 | table_count = len(table_matches_matches) 67 | checkdo_count = len(checkdo_matches) 68 | closedont_count = len(closedont_matches) 69 | 70 | return sorted_results, list_count, image_alt_count, table_count, checkdo_count, closedont_count 71 | 72 | 73 | def process_comp_guidelines(file_path, component_info, docount, dontcount, guidelinecount): 74 | 75 | with open(file_path, 'r') as file: 76 | content = file.read() 77 | 78 | # Extracting all level headlines and their content 79 | pattern = r'(#+)\s*(.*?)\n(.*?)\n(?=\n#+ |\Z)' 80 | matches = re.findall(pattern, content, flags=re.DOTALL) 81 | 82 | parsed_content = {} 83 | 84 | current_h2_key = None 85 | current_h3_key = None 86 | 87 | for match in matches: 88 | level, heading, section_content = match 89 | heading_key = heading.strip() 90 | 91 | # remove ', '', section_content , flags=re.DOTALL) 93 | # remove link 94 | section_content = re.sub(r'\(https?://\S+\)', '', section_content ) 95 | 96 | # Extracting and removing "checkDo" and "closeDon’t" 97 | do_pattern = r'checkDo\s*(.*?)(?:\n|$)(?(1)(.*?\n|)|(?:\n(.*?\n.*?)\n|))' 98 | dont_pattern = r'closeDon’t\s*(.*?)(?:\n|$)(?(1)(.*?\n|)|(?:\n(.*?\n.*?)\n|))' 99 | 100 | do_content = re.findall(do_pattern, section_content, flags=re.DOTALL) 101 | dont_content = re.findall(dont_pattern, section_content, flags=re.DOTALL) 102 | 103 | section_content = re.sub(do_pattern, '', section_content, flags=re.DOTALL) 104 | section_content = re.sub(dont_pattern, '', section_content, flags=re.DOTALL) 105 | 106 | section_content = section_content.strip() 107 | 108 | if level == '##': 109 | current_h2_key = heading_key 110 | current_h3_key = None 111 | # use setdefault() keep staying in current_h2_key 112 | component_info["guidelines"]["soft"].setdefault(current_h2_key, {}) 113 | component_info["guidelines"]["soft"][current_h2_key].setdefault("general", {}) 114 | # 115 | component_info["guidelines"]["soft"][current_h2_key]["soft"] = {} 116 | component_info["guidelines"]["soft"][current_h2_key]["soft"] = section_content 117 | #save index 118 | # component_info["guidelines_indexs"].setdefault(current_h2_key, {}) 119 | # component_info["guidelines_indexs"][current_h2_key]["general"] = {} 120 | guidelinecount += 1 121 | 122 | elif level == '###' and current_h2_key is not None: 123 | current_h3_key = heading_key 124 | component_info["guidelines"]["soft"][current_h2_key][current_h3_key] = {} 125 | component_info["guidelines"]["soft"][current_h2_key][current_h3_key] = section_content 126 | #save index 127 | # component_info["guidelines_indexs"][current_h2_key][current_h3_key] = {} 128 | guidelinecount += 1 129 | 130 | elif level == '####' and current_h3_key is not None: 131 | component_info["guidelines"]["soft"][current_h2_key][current_h3_key] += level + ' ' + heading + '\n' + section_content + '\n' 132 | 133 | # Save the extracted content 134 | if do_content: 135 | for item in do_content: 136 | docount += 1 137 | concatenated_str = ''.join(item) 138 | if concatenated_str.endswith('\n'): 139 | concatenated_str = concatenated_str[:-1] 140 | combined_string = current_h2_key + "#" + str(current_h3_key) + "#" + concatenated_str 141 | component_info["guidelines"]["hard"]["do"].extend([combined_string]) 142 | 143 | # component_info["guidelines"]["constraints"]["do"].extend([current_h2_key, current_h3_key, do_content]) 144 | if dont_content: 145 | for item in dont_content: 146 | dontcount += 1 147 | concatenated_str = ''.join(item) 148 | if concatenated_str.endswith('\n'): 149 | concatenated_str = concatenated_str[:-1] 150 | combined_string = current_h2_key + "#" + str(current_h3_key) + "#" + concatenated_str 151 | component_info["guidelines"]["hard"]["dont"].extend([combined_string]) 152 | 153 | # component_info["guidelines"]["constraints"]["dont"].extend([current_h2_key, current_h3_key, dont_content]) 154 | 155 | return component_info, docount, dontcount, guidelinecount 156 | 157 | 158 | 159 | def analyze_guidelines(component_info): 160 | h2_keys = component_info["guidelines"]["soft"].keys() 161 | 162 | analysis_results = {} 163 | for h2_key in h2_keys: 164 | sub_items = component_info["guidelines"]["soft"][h2_key] 165 | analysis_results[h2_key] = { 166 | "sub_item_count": len(sub_items), 167 | "sub_item_lengths": {sub_item_key: len(sub_item_content) for sub_item_key, sub_item_content in sub_items.items()} 168 | } 169 | return analysis_results 170 | 171 | 172 | 173 | 174 | def plot_h2_key_analysis(aggregated_h2_data): 175 | data = { 176 | "h2_key": [], 177 | "total_sub_item_count": [], 178 | "average_length": [], 179 | "min_length": [], 180 | "max_length": [] 181 | } 182 | 183 | for h2_key, values in aggregated_h2_data.items(): 184 | lengths = [length for sublist in values["sub_item_lengths"].values() for length in sublist] 185 | average_length = sum(lengths) / len(lengths) if lengths else 0 186 | non_zero_lengths = [length for length in lengths if length > 0] 187 | min_length = min(non_zero_lengths) if non_zero_lengths else 0 188 | max_length = max(lengths) if lengths else 0 189 | 190 | data["h2_key"].append(h2_key) 191 | data["total_sub_item_count"].append(values["total_sub_item_count"]) 192 | data["average_length"].append(average_length) 193 | data["min_length"].append(min_length) 194 | data["max_length"].append(max_length) 195 | # print(f"H2 Key: {h2_key}, Total Sub Item Count: {values['total_sub_item_count']}, Average Length: {average_length}, Min Length: {min_length}, Max Length: {max_length}") 196 | 197 | df = pd.DataFrame(data) 198 | 199 | # max 5 200 | df_top5 = df.nlargest(5, 'total_sub_item_count') 201 | top5_keys = df_top5['h2_key'].tolist() 202 | print(top5_keys) 203 | 204 | # other 205 | other_data = df[~df['h2_key'].isin(top5_keys)].sum(numeric_only=True) 206 | other_data['h2_key'] = 'Other' 207 | # 使用 pd.concat 而不是 append 208 | df_top5_and_other = pd.concat([df_top5, pd.DataFrame([other_data])], ignore_index=True) 209 | 210 | plt.figure(figsize=(10, 6)) 211 | plt.title("General Component Aspects Count", fontsize=16, fontweight='bold') 212 | bars = plt.bar(df_top5_and_other['h2_key'], df_top5_and_other['total_sub_item_count'], color='skyblue') 213 | plt.ylabel('Total Guidelines Count', fontsize=12) 214 | # Adding gridlines for better readability 215 | plt.grid(True, which='both', linestyle='--', linewidth=0.5) 216 | for bar in bars: 217 | yval = bar.get_height() 218 | plt.text(bar.get_x() + bar.get_width()/2, yval, int(yval), va='bottom', ha='center', fontsize=10) 219 | plt.show() 220 | 221 | plt.figure(figsize=(10, 6)) 222 | plt.title("Average Guidelines Length", fontsize=16, fontweight='bold') 223 | x = range(len(df_top5)) 224 | 225 | err_lower = np.clip(df_top5['average_length'] - df_top5['min_length'], 0, None) 226 | err_upper = np.clip(df_top5['max_length'] - df_top5['average_length'], 0, None) 227 | err = [err_lower, err_upper] 228 | 229 | bars = plt.bar(x, df_top5['average_length'], yerr=err, align='center', alpha=0.4, ecolor='grey', capsize=5, color='coral') 230 | # plt.xlabel('H2 Key', fontsize=12) 231 | plt.ylabel('Length', fontsize=12) 232 | plt.xticks(x, df_top5['h2_key']) 233 | # Adding gridlines for better readability 234 | plt.grid(True, which='both', linestyle='--', linewidth=0.5) 235 | 236 | for bar in bars: 237 | yval = bar.get_height() 238 | plt.text(bar.get_x() + bar.get_width()/2, yval, round(yval, 2), va='bottom', ha='center', fontsize=10) 239 | 240 | plt.tight_layout() 241 | plt.legend(['Average Length with Min/Max Range']) 242 | plt.show() 243 | 244 | 245 | 246 | def plot_constraints_analysis(constraints_count): 247 | # Specific keys to consider 248 | specific_keys = ['Behavior', 'Usage', 'Anatomy', 'Placement', 'Responsive layout'] 249 | 250 | # specific_keys=['Interaction & style', 'Use cases', 'Keyboard navigation', 'Labeling elements', 'Initial focus', 'Visual indicators'] 251 | 252 | filtered_counts = {key: {'do': 0, 'dont': 0} for key in specific_keys} 253 | filtered_counts['Other'] = {'do': 0, 'dont': 0} # Additional category for 'Other' 254 | 255 | # Filtering and re-categorizing the data 256 | for key, value in constraints_count.items(): 257 | if key in specific_keys: 258 | filtered_counts[key]['do'] += value['do'] 259 | filtered_counts[key]['dont'] += value['dont'] 260 | else: 261 | filtered_counts['Other']['do'] += value['do'] 262 | filtered_counts['Other']['dont'] += value['dont'] 263 | 264 | labels = list(filtered_counts.keys()) 265 | do_counts = [value['do'] for value in filtered_counts.values()] 266 | dont_counts = [value['dont'] for value in filtered_counts.values()] 267 | 268 | # Adjusting the figure size 269 | plt.figure(figsize=(10, 6)) 270 | 271 | # New color scheme for publication 272 | # colors_do = ['#4daf4a', '#377eb8', '#ff7f00', '#984ea3', '#e41a1c', '#ffff33'] # Set of distinct colors for 'Do' 273 | # colors_dont = ['#a6cee3', '#1f78b4', '#fdbf6f', '#cab2d6', '#fb9a99', '#b2df8a'] # Different set for 'Don’t' 274 | colors_do = ['#ffff33'] 275 | colors_dont = ['#b2df8a'] 276 | 277 | x = range(len(labels)) 278 | bars_do = plt.bar(x, do_counts, width=0.4, color=colors_do, label='Do', align='center') 279 | bars_dont = plt.bar(x, dont_counts, width=0.4, color=colors_dont, label='Don’t', align='edge') 280 | 281 | # Adding values on top of the bars 282 | for bar in bars_do + bars_dont: 283 | height = bar.get_height() 284 | plt.text(bar.get_x() + bar.get_width() / 2, height, int(height), ha='center', va='bottom', fontsize=9) 285 | 286 | # Font size and style adjustments 287 | # plt.xlabel('Categories', fontsize=12) 288 | plt.ylabel('Count', fontsize=12) 289 | plt.title('Do and Don’t Constraints Count', fontsize=16, fontweight='bold') 290 | plt.xticks(x, labels, fontsize=10) 291 | plt.yticks(fontsize=10) 292 | 293 | # Adding gridlines for better readability 294 | plt.grid(True, which='both', linestyle='--', linewidth=0.5) 295 | # Adjusting the legend 296 | plt.legend(fontsize=12) 297 | # Ensuring the plot is spaced adequately 298 | plt.tight_layout() 299 | # Save the plot with high resolution 300 | plt.savefig('constraints_analysis_plot.png', dpi=300) 301 | plt.show() 302 | 303 | 304 | 305 | def get_description(file_path): 306 | descriptions = {} 307 | with open(file_path, 'r') as file: 308 | docs = file.read() 309 | sections = re.split(r'\n#+ ', docs) 310 | for section in sections: 311 | if not section.startswith(' \n'): 312 | # Split the section into the heading and the content 313 | heading, *content = section.split('\n\n', 1) 314 | # Remove "##" and strip whitespace from the heading 315 | key = heading.strip().lower() 316 | # Join the content back together if it was split 317 | content = '\n'.join(content).strip() 318 | # Add the heading and content to the dictionary 319 | descriptions[key] = content 320 | return descriptions 321 | 322 | 323 | 324 | 325 | 326 | def comp_main(file_path, description_path): 327 | directories = ["Components"] 328 | components_list = [] 329 | docount = 0 330 | dontcount = 0 331 | guidelinecount = 0 332 | aggregated_h2_data = {} 333 | constraints_count = {} 334 | 335 | descriptions = get_description(description_path) 336 | 337 | for directory in directories: 338 | for root, dirs, files in os.walk(os.path.join(file_path, directory)): 339 | for file in files: 340 | if file.endswith("_guidelines.md"): 341 | # if file.endswith("_accessibility.md"): 342 | # init 343 | new_component = copy.deepcopy(component_info) 344 | new_component["component_type"] = file.split("_")[0].replace("-", " ") 345 | new_component["guidelines"]["docs_path"] = file 346 | 347 | # get description 348 | if new_component["component_type"] in descriptions.keys(): 349 | new_component["description"] = descriptions[new_component["component_type"]] 350 | elif new_component["component_type"][:-1] in descriptions.keys(): # remove "s" 351 | new_component["description"] = descriptions[new_component["component_type"][:-1]] 352 | else: 353 | print(new_component["component_type"], "no description found") 354 | 355 | # structure guidelines 356 | new_component, docount, dontcount, guidelinecount = process_comp_guidelines(os.path.join(root, file), new_component, docount, dontcount, guidelinecount) 357 | components_list.append(new_component) 358 | 359 | # Analyze and aggregate guidelines 360 | analysis_results = analyze_guidelines(new_component) 361 | for h2_key, data in analysis_results.items(): 362 | if h2_key not in aggregated_h2_data: 363 | aggregated_h2_data[h2_key] = {"total_sub_item_count": 0, "sub_item_lengths": {}} 364 | aggregated_h2_data[h2_key]["total_sub_item_count"] += data["sub_item_count"] 365 | for sub_item_key, length in data["sub_item_lengths"].items(): 366 | aggregated_h2_data[h2_key]["sub_item_lengths"].setdefault(sub_item_key, []).append(length) 367 | 368 | # Analyze and aggregate constraints 369 | h2_keys = new_component["guidelines"]["soft"].keys() 370 | for h2_key in h2_keys: 371 | do_count = sum(h2_key in item for item in new_component["guidelines"]["hard"]["do"]) 372 | dont_count = sum(h2_key in item for item in new_component["guidelines"]["hard"]["dont"]) 373 | constraints_count[h2_key] = constraints_count.get(h2_key, {'do': 0, 'dont': 0}) 374 | constraints_count[h2_key]['do'] += do_count 375 | constraints_count[h2_key]['dont'] += dont_count 376 | 377 | 378 | plot_h2_key_analysis(aggregated_h2_data) 379 | plot_constraints_analysis(constraints_count) 380 | 381 | print(docount, dontcount, guidelinecount, len(components_list)) 382 | 383 | with open("components_knowledge_base.json", "w") as outfile: 384 | json.dump(components_list, outfile, indent=4) 385 | 386 | return 387 | 388 | 389 | 390 | if __name__ == "__main__": 391 | file_path = r"doc_path" 392 | description_path = r"description_path" 393 | comp_main(file_path, description_path) -------------------------------------------------------------------------------- /backend/core/analysis_groups.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import re 3 | from playwright.async_api import async_playwright 4 | from playwright.sync_api import Playwright, expect, sync_playwright 5 | 6 | import os 7 | import json 8 | import time 9 | from pydantic import BaseModel 10 | from typing import Optional, List 11 | from core.llm import stream_openai_response 12 | from core.prompts import analysis_system 13 | from core.prompts import property_analysis_content 14 | 15 | class AnalysisProperty(BaseModel): 16 | bad_design_code_filename: str 17 | bad_design_code: str 18 | detailed_reference_from_guidelines: str 19 | suggestion_fix_code: str 20 | 21 | class AnalysisPropertySchema(BaseModel): 22 | bad_property_design: List[AnalysisProperty] 23 | 24 | 25 | async def get_response(prompt_messages, functions_schema=None, temperature=0.0): 26 | async def process_chunk(content: str): 27 | pass 28 | 29 | if not os.environ.get("OPENAI_API_KEY"): 30 | raise Exception("OpenAI API key not found") 31 | 32 | completion = await stream_openai_response( 33 | messages=prompt_messages, 34 | temperature=temperature, 35 | functions=functions_schema, 36 | api_key=os.environ.get("OPENAI_API_KEY"), 37 | callback=lambda x: process_chunk(x), 38 | base_url=None, 39 | ) 40 | return completion 41 | 42 | async def load_page(playwright, pageurl=None, width = None, height = None): 43 | 44 | browser = await playwright.chromium.launch( 45 | traces_dir=None, 46 | headless=False, 47 | args=[ 48 | "--disable-blink-features=AutomationControlled", 49 | ]) 50 | page = await browser.new_page() 51 | 52 | if width and height: 53 | await page.set_viewport_size({"width": width, "height": height}) 54 | 55 | if pageurl: 56 | await page.goto(pageurl) 57 | else: 58 | await page.goto("https://www.google.com") 59 | print(await page.title()) 60 | await asyncio.sleep(3) 61 | 62 | await page.bring_to_front() 63 | return page 64 | 65 | 66 | 67 | def remove_extra_eol(text): 68 | # Replace EOL symbols 69 | text = text.replace('\n', ' ') 70 | return re.sub(r'\s{2,}', ' ', text) 71 | 72 | 73 | def get_first_line(s): 74 | first_line = s.split('\n')[0] 75 | tokens = first_line.split() 76 | if len(tokens) > 8: 77 | return ' '.join(tokens[:8]) + '...' 78 | else: 79 | return first_line 80 | 81 | 82 | async def get_element_description(element, tag_name, role_value, type_value): 83 | ''' 84 | Asynchronously generates a descriptive text for a web element based on its tag type. 85 | Handles various HTML elements like 'select', 'input', and 'textarea', extracting attributes and content relevant to accessibility and interaction. 86 | ''' 87 | 88 | salient_attributes = [ 89 | "alt", 90 | "aria-describedby", 91 | "aria-label", 92 | "aria-role", 93 | "input-checked", 94 | # "input-value", 95 | "label", 96 | "name", 97 | "option_selected", 98 | "placeholder", 99 | "readonly", 100 | "text-value", 101 | "title", 102 | "value", 103 | ] 104 | 105 | parent_value = "parent_node: " 106 | parent_locator = element.locator('xpath=..') 107 | num_parents = await parent_locator.count() 108 | if num_parents > 0: 109 | # only will be zero or one parent node 110 | parent_text = (await parent_locator.inner_text(timeout=0) or "").strip() 111 | if parent_text: 112 | parent_value += parent_text 113 | parent_value = remove_extra_eol(get_first_line(parent_value)).strip() 114 | if parent_value == "parent_node:": 115 | parent_value = "" 116 | else: 117 | parent_value += " " 118 | 119 | if tag_name == "select": 120 | text1 = "Selected Options: " 121 | text2 = "" 122 | text3 = " - Options: " 123 | text4 = "" 124 | 125 | text2 = await element.evaluate( 126 | "select => select.options[select.selectedIndex].textContent", timeout=0 127 | ) 128 | 129 | if text2: 130 | options = await element.evaluate("select => Array.from(select.options).map(option => option.text)", 131 | timeout=0) 132 | text4 = " | ".join(options) 133 | 134 | if not text4: 135 | text4 = await element.text_content(timeout=0) 136 | if not text4: 137 | text4 = await element.inner_text(timeout=0) 138 | 139 | return parent_value+text1 + remove_extra_eol(text2.strip()) + text3 + text4 140 | 141 | input_value = "" 142 | 143 | none_input_type = ["submit", "reset", "checkbox", "radio", "button", "file"] 144 | 145 | if tag_name == "input" or tag_name == "textarea": 146 | if role_value not in none_input_type and type_value not in none_input_type: 147 | text1 = "input value=" 148 | text2 = await element.input_value(timeout=0) 149 | if text2: 150 | input_value = text1 + "\"" + text2 + "\"" + " " 151 | 152 | text_content = await element.text_content(timeout=0) 153 | text = (text_content or '').strip() 154 | if text: 155 | text = remove_extra_eol(text) 156 | if len(text) > 80: 157 | text_content_in = await element.inner_text(timeout=0) 158 | text_in = (text_content_in or '').strip() 159 | if text_in: 160 | return input_value + remove_extra_eol(text_in) 161 | else: 162 | return input_value + text 163 | 164 | # get salient_attributes 165 | text1 = "" 166 | for attr in salient_attributes: 167 | attribute_value = await element.get_attribute(attr, timeout=0) 168 | if attribute_value: 169 | text1 += f"{attr}=" + "\"" + attribute_value.strip() + "\"" + " " 170 | 171 | text = (parent_value + text1).strip() 172 | if text: 173 | return input_value + remove_extra_eol(text.strip()) 174 | 175 | 176 | # try to get from the first child node 177 | first_child_locator = element.locator('xpath=./child::*[1]') 178 | 179 | num_childs = await first_child_locator.count() 180 | if num_childs>0: 181 | for attr in salient_attributes: 182 | attribute_value = await first_child_locator.get_attribute(attr, timeout=0) 183 | if attribute_value: 184 | text1 += f"{attr}=" + "\"" + attribute_value.strip() + "\"" + " " 185 | 186 | text = (parent_value + text1).strip() 187 | if text: 188 | return input_value + remove_extra_eol(text.strip()) 189 | 190 | return None 191 | 192 | 193 | async def get_element_data(element, tag_name): 194 | tag_name_list = ['a', 'button', 195 | 'input', 196 | 'select', 'textarea', 'adc-tab'] 197 | 198 | # await aprint(element,tag_name) 199 | if await element.is_hidden(timeout=0) or await element.is_disabled(timeout=0): 200 | return None 201 | 202 | tag_head = "" 203 | real_tag_name = "" 204 | if tag_name in tag_name_list: 205 | tag_head = tag_name 206 | real_tag_name = tag_name 207 | else: 208 | real_tag_name = await element.evaluate("element => element.tagName.toLowerCase()", timeout=0) 209 | if real_tag_name in tag_name_list: 210 | # already detected 211 | return None 212 | else: 213 | tag_head = real_tag_name 214 | 215 | role_value = await element.get_attribute('role', timeout=0) 216 | type_value = await element.get_attribute('type', timeout=0) 217 | # await aprint("start to get element description",element,tag_name ) 218 | description = await get_element_description(element, real_tag_name, role_value, type_value) 219 | if not description: 220 | return None 221 | 222 | rect = await element.bounding_box() or {'x': 0, 'y': 0, 'width': 0, 'height': 0} 223 | 224 | if role_value: 225 | tag_head += " role=" + "\"" + role_value + "\"" 226 | if type_value: 227 | tag_head += " type=" + "\"" + type_value + "\"" 228 | 229 | box_model = [rect['x'], rect['y'], rect['x'] + rect['width'], rect['y'] + rect['height']] 230 | center_point = ((box_model[0] + box_model[2]) / 2, (box_model[1] + box_model[3]) / 2) 231 | selector = element 232 | 233 | 234 | return [center_point, description, tag_head, box_model, selector, real_tag_name] 235 | 236 | 237 | async def get_text_from_element(element): 238 | """ 239 | Extracts text content from a web element. 240 | """ 241 | try: 242 | return await element.text_content() 243 | except Exception as e: 244 | return None 245 | 246 | 247 | async def get_groups_with_playwright(page): 248 | interactive_elements_selectors = [ 249 | 'a', 'button', 250 | 'input', 251 | 'select', 'textarea', 'adc-tab', '[role="button"]', '[role="radio"]', '[role="option"]', '[role="combobox"]', 252 | '[role="textbox"]', 253 | '[role="listbox"]', '[role="menu"]', 254 | '[type="button"]', '[type="radio"]', '[type="combobox"]', '[type="textbox"]', '[type="listbox"]', 255 | '[type="menu"]', 256 | '[tabindex]:not([tabindex="-1"])', '[contenteditable]:not([contenteditable="false"])', 257 | '[onclick]', '[onfocus]', '[onkeydown]', '[onkeypress]', '[onkeyup]', "[checkbox]", 258 | '[aria-disabled="false"],[data-link]' 259 | ] 260 | 261 | tasks = [] 262 | all_text_content = [] 263 | 264 | seen_elements = set() 265 | for selector in interactive_elements_selectors: 266 | locator = page.locator(selector) 267 | element_count = await locator.count() 268 | for index in range(element_count): 269 | element = locator.nth(index) 270 | tag_name = selector.replace(":not([tabindex=\"-1\"])", "") 271 | tag_name = tag_name.replace(":not([contenteditable=\"false\"])", "") 272 | task = get_element_data(element, tag_name) 273 | # task = get_text_from_element(element) 274 | tasks.append(task) 275 | 276 | results = await asyncio.gather(*tasks) 277 | # for text in results: 278 | # if text: 279 | # all_text_content.append(text.strip()) 280 | 281 | # print(results) 282 | return results 283 | 284 | def check_and_repair(ctx): 285 | pass 286 | 287 | 288 | async def get_all_xpaths(page): 289 | """ 290 | get_all_xpaths XPath 291 | """ 292 | script = """ 293 | () => { 294 | const allElements = document.querySelectorAll('*'); 295 | const getXPath = (element) => { 296 | if (element.id) return 'id(\"' + element.id + '\")'; 297 | if (element === document.body) return element.tagName; 298 | 299 | var ix = 0; 300 | const siblings = element.parentNode ? element.parentNode.childNodes : []; 301 | for (var i = 0; i < siblings.length; i++) { 302 | const sibling = siblings[i]; 303 | if (sibling === element) 304 | return getXPath(element.parentNode) + '/' + element.tagName + '[' + (ix + 1) + ']'; 305 | if (sibling.nodeType === 1 && sibling.tagName === element.tagName) ix++; 306 | } 307 | }; 308 | 309 | return Array.from(allElements).map(el => getXPath(el)).filter(x => x != null); 310 | } 311 | """ 312 | 313 | xpaths = await page.evaluate(script) 314 | return xpaths 315 | 316 | 317 | async def get_text_properties(page): 318 | """ 319 | Extract properties of text elements from the page, including their outer HTML and XPath, with deduplication based on XPath or content. 320 | """ 321 | script = """ 322 | () => { 323 | const getXPath = (element) => { 324 | if (element.id !== '') return 'id(\"' + element.id + '\")'; 325 | if (element === document.body) return element.tagName; 326 | let ix = 0; 327 | const siblings = element.parentNode.childNodes; 328 | for (let i = 0; i < siblings.length; i++) { 329 | const sibling = siblings[i]; 330 | if (sibling === element) 331 | return getXPath(element.parentNode) + '/' + element.tagName.toLowerCase() + '[' + (ix + 1) + ']'; 332 | if (sibling.nodeType === 1 && sibling.tagName === element.tagName) ix++; 333 | } 334 | return null; 335 | }; 336 | 337 | const getTextContent = (element) => { 338 | return element.childNodes.length === 1 && element.childNodes[0].nodeType === Node.TEXT_NODE 339 | ? element.textContent.trim() 340 | : Array.from(element.childNodes) 341 | .filter(e => e.nodeType === Node.TEXT_NODE) 342 | .map(e => e.textContent.trim()) 343 | .join(''); 344 | }; 345 | 346 | const allTextElements = Array.from(document.querySelectorAll('h1, h2, h3, h4, h5, h6, p, span, a, div')); 347 | const elementsData = []; 348 | 349 | allTextElements.forEach(element => { 350 | const computedStyle = window.getComputedStyle(element); 351 | const xpath = getXPath(element); 352 | const content = getTextContent(element); 353 | 354 | if (content) { 355 | elementsData.push({ 356 | outerHTML: element.outerHTML, 357 | xpath: xpath, 358 | content: content, 359 | fontSize: computedStyle.fontSize, 360 | fontFamily: computedStyle.fontFamily, 361 | fontWeight: computedStyle.fontWeight, 362 | color: computedStyle.color, 363 | backgroundColor: computedStyle.backgroundColor, 364 | tagName: element.tagName, 365 | class: element.className 366 | }); 367 | } 368 | }); 369 | 370 | return elementsData; 371 | } 372 | """ 373 | return await page.evaluate(script) 374 | 375 | # save uniqueElementsWithColor 376 | async def get_all_colors2(page): 377 | script = """ 378 | () => { 379 | const findColor = (element) => { 380 | let node = element; 381 | let color = null, backgroundColor = null; 382 | while (node && node instanceof Element) { 383 | const computedStyle = window.getComputedStyle(node); 384 | if (!color && computedStyle.color !== 'rgba(0, 0, 0, 0)' && computedStyle.color !== 'rgb(0, 0, 0)') { 385 | color = computedStyle.color; 386 | } 387 | if (!backgroundColor && computedStyle.backgroundColor !== 'rgba(0, 0, 0, 0)' && computedStyle.backgroundColor !== 'transparent') { 388 | backgroundColor = computedStyle.backgroundColor; 389 | } 390 | if (color && backgroundColor) break; 391 | node = node.parentNode; 392 | } 393 | return { color, backgroundColor }; 394 | }; 395 | 396 | const getXPath = (element) => { 397 | if (!element || !element.parentNode) { 398 | return null; 399 | } 400 | if (element.id) { 401 | return `id("${element.id}")`; 402 | } 403 | if (element === document.body) { 404 | return element.tagName.toLowerCase(); 405 | } 406 | let ix = 1; 407 | const siblings = element.parentNode.childNodes; 408 | for (let i = 0; i < siblings.length; i++) { 409 | const sibling = siblings[i]; 410 | if (sibling === element) { 411 | return `${getXPath(element.parentNode)}/${element.tagName.toLowerCase()}[${ix}]`; 412 | } 413 | if (sibling.nodeType === 1 && sibling.tagName === element.tagName) { 414 | ix++; 415 | } 416 | } 417 | return null; 418 | }; 419 | 420 | 421 | const isElementVisible = (element) => { 422 | return element.offsetWidth > 0 || element.offsetHeight > 0 || element.getClientRects().length > 0; 423 | }; 424 | 425 | const uniqueElementsWithColor = new Map(); 426 | const traverseDOM = (element, parentXPath = '') => { 427 | const xpath = parentXPath ? `${parentXPath}/${getXPath(element)}` : getXPath(element); 428 | if (isElementVisible(element)) { 429 | const { color, backgroundColor } = findColor(element); 430 | if (color || backgroundColor) { 431 | // If the parent element has the same color, don't add this element 432 | const parentXpath = parentXPath ? parentXPath : getXPath(element.parentNode); 433 | const parentData = uniqueElementsWithColor.get(parentXpath); 434 | if (!parentData || parentData.color !== color || parentData.backgroundColor !== backgroundColor) { 435 | uniqueElementsWithColor.set(xpath, { 436 | element: element.outerHTML, 437 | backgroundColor: backgroundColor, 438 | color: color 439 | }); 440 | } 441 | } 442 | } 443 | Array.from(element.children).forEach(child => { 444 | traverseDOM(child, xpath); 445 | }); 446 | }; 447 | 448 | traverseDOM(document.body); 449 | return Array.from(uniqueElementsWithColor.values()); 450 | } 451 | """ 452 | return await page.evaluate(script) 453 | 454 | # save all colors 455 | async def get_all_colors(page): 456 | script = """ 457 | () => { 458 | const findColor = (element) => { 459 | let node = element; 460 | let color = null, backgroundColor = null; 461 | while (node && node instanceof Element) { // Ensure node is a DOM element 462 | const computedStyle = window.getComputedStyle(node); 463 | if (!color && computedStyle.color !== 'rgba(0, 0, 0, 0)' && computedStyle.color !== 'rgb(0, 0, 0)') { 464 | color = computedStyle.color; 465 | } 466 | if (!backgroundColor && computedStyle.backgroundColor !== 'rgba(0, 0, 0, 0)' && computedStyle.backgroundColor !== 'transparent') { 467 | backgroundColor = computedStyle.backgroundColor; 468 | } 469 | if (color && backgroundColor) break; 470 | node = node.parentNode; 471 | } 472 | return { color, backgroundColor }; 473 | }; 474 | 475 | 476 | const getXPath = (element) => { 477 | if (element.id) { 478 | return `id("${element.id}")`; 479 | } 480 | if (element === document.body) { 481 | return element.tagName.toLowerCase(); 482 | } 483 | let ix = 1; 484 | const siblings = element.parentNode.childNodes; 485 | for (let i = 0; i < siblings.length; i++) { 486 | const sibling = siblings[i]; 487 | if (sibling === element) { 488 | return `${getXPath(element.parentNode)}/${element.tagName.toLowerCase()}[${ix}]`; 489 | } 490 | if (sibling.nodeType === 1 && sibling.tagName === element.tagName) { 491 | ix++; 492 | } 493 | } 494 | return null; 495 | }; 496 | 497 | const isElementVisible = (element) => { 498 | return element.offsetWidth > 0 || element.offsetHeight > 0 || element.getClientRects().length > 0; 499 | }; 500 | 501 | const traverseDOM = (element, parentXPath = '') => { 502 | const xpath = parentXPath ? `${parentXPath}/${getXPath(element)}` : getXPath(element); 503 | if (isElementVisible(element)) { 504 | const { color, backgroundColor } = findColor(element); 505 | if (color || backgroundColor) { 506 | uniqueElementsWithColor.set(xpath, { 507 | element: element.outerHTML, 508 | // xpath: xpath, 509 | backgroundColor: backgroundColor, 510 | color: color 511 | }); 512 | } 513 | } 514 | Array.from(element.children).forEach(child => { 515 | traverseDOM(child, xpath); 516 | }); 517 | }; 518 | 519 | const uniqueElementsWithColor = new Map(); 520 | traverseDOM(document.body); 521 | return Array.from(uniqueElementsWithColor.values()); 522 | } 523 | """ 524 | return await page.evaluate(script) 525 | 526 | 527 | async def get_label_properties(page): 528 | script = """ 529 | () => { 530 | const allElements = document.querySelectorAll('img, video, button, input, select, textarea, label'); 531 | const labelData = []; 532 | 533 | allElements.forEach(element => { 534 | let labelText = ''; 535 | if (element.tagName.toLowerCase() === 'img') { 536 | labelText = element.alt; // Alternative text for images 537 | } else if (element.tagName.toLowerCase() === 'video') { 538 | labelText = element.getAttribute('aria-label') || element.title; // ARIA label or title for videos 539 | } else if (element.tagName.toLowerCase() === 'button') { 540 | labelText = element.textContent.trim() || element.getAttribute('aria-label'); // Text content or ARIA label for buttons 541 | } else if (element.tagName.toLowerCase() === 'input' || element.tagName.toLowerCase() === 'select' || element.tagName.toLowerCase() === 'textarea') { 542 | const label = document.querySelector(`label[for='${element.id}']`); 543 | labelText = label ? label.textContent.trim() : ''; 544 | } 545 | 546 | if (labelText) { 547 | labelData.push({ 548 | element: element.outerHTML, 549 | label: labelText 550 | }); 551 | } 552 | }); 553 | 554 | return labelData; 555 | } 556 | """ 557 | return await page.evaluate(script) 558 | 559 | 560 | async def get_clickable_properties(page): 561 | script = """ 562 | () => { 563 | const allClickableElements = document.querySelectorAll('a, button, input[type="button"], input[type="submit"], [onclick], [role="button"]'); 564 | const clickableData = []; 565 | 566 | allClickableElements.forEach(element => { 567 | const computedStyle = window.getComputedStyle(element); 568 | const rect = element.getBoundingClientRect(); 569 | 570 | let isLargeEnough = false; 571 | const MIN_TOUCH_SIZE = 44; // 44x44 pixels is a common touch target size recommendation 572 | if (rect.width >= MIN_TOUCH_SIZE && rect.height >= MIN_TOUCH_SIZE) { 573 | isLargeEnough = true; 574 | } 575 | 576 | // Check for elevation (e.g., box-shadow or non-zero z-index) 577 | const hasElevation = computedStyle.boxShadow !== 'none' || parseInt(computedStyle.zIndex, 10) > 0; 578 | let elevationColor = null; 579 | if (hasElevation && computedStyle.boxShadow !== 'none') { 580 | elevationColor = computedStyle.boxShadow.split(' ')[3]; // Assuming the color is the 4th value in boxShadow 581 | } 582 | 583 | clickableData.push({ 584 | element: element.outerHTML, 585 | touchSize: { 586 | width: rect.width, 587 | height: rect.height, 588 | }, 589 | isLargeEnough: isLargeEnough, 590 | focusable: 'tabIndex' in element && element.tabIndex >= 0, 591 | hasElevation: hasElevation, 592 | elevationColor: elevationColor 593 | }); 594 | }); 595 | 596 | return clickableData; 597 | } 598 | """ 599 | return await page.evaluate(script) 600 | 601 | 602 | 603 | async def get_layout_properties(page): 604 | script = """ 605 | () => { 606 | const allElements = document.querySelectorAll('*'); 607 | const layoutData = new Map(); 608 | 609 | allElements.forEach(element => { 610 | const computedStyle = window.getComputedStyle(element); 611 | 612 | const layoutInfo = { 613 | margin: { 614 | top: computedStyle.marginTop, 615 | right: computedStyle.marginRight, 616 | bottom: computedStyle.marginBottom, 617 | left: computedStyle.marginLeft 618 | }, 619 | padding: { 620 | top: computedStyle.paddingTop, 621 | right: computedStyle.paddingRight, 622 | bottom: computedStyle.paddingBottom, 623 | left: computedStyle.paddingLeft 624 | }, 625 | border: { 626 | top: computedStyle.borderTopWidth, 627 | right: computedStyle.borderRightWidth, 628 | bottom: computedStyle.borderBottomWidth, 629 | left: computedStyle.borderLeftWidth 630 | } 631 | }; 632 | 633 | // Skip if all values are zero 634 | if (Object.values(layoutInfo.margin).every(val => val === '0px') && 635 | Object.values(layoutInfo.padding).every(val => val === '0px') && 636 | Object.values(layoutInfo.border).every(val => val === '0px')) { 637 | return; 638 | } 639 | 640 | const layoutString = JSON.stringify(layoutInfo); 641 | let isChildOfExistingElement = false; 642 | 643 | // Check if an element with the same layout already exists 644 | layoutData.forEach((data, key) => { 645 | if (key === layoutString) { 646 | const existingElement = document.createElement('div'); 647 | existingElement.innerHTML = data.element; 648 | if (existingElement.firstChild.contains(element)) { 649 | isChildOfExistingElement = true; 650 | } 651 | } 652 | }); 653 | 654 | if (!isChildOfExistingElement) { 655 | layoutData.set(layoutString, { 656 | element: element.outerHTML, 657 | layoutInfo 658 | }); 659 | } 660 | }); 661 | 662 | 663 | return Array.from(layoutData.values()); 664 | } 665 | """ 666 | return await page.evaluate(script) 667 | 668 | 669 | def assemble_analysis_property_prompt(file_content, related_guidelines, property_type, property_list): 670 | print (f"-----ANALYZE PROPERTIES : {property_type}-------") 671 | 672 | property_analysis_content_prompt = property_analysis_content 673 | property_analysis_content_prompt = property_analysis_content_prompt.replace("{file_content}", file_content) 674 | property_analysis_content_prompt = property_analysis_content_prompt.replace("{property_type}", property_type) 675 | property_analysis_content_prompt = property_analysis_content_prompt.replace("{properties}", str(property_list)) 676 | property_analysis_content_prompt = property_analysis_content_prompt.replace("{hard_constraints}", str(related_guidelines['hard'])) 677 | 678 | prompt =[ 679 | { 680 | "role": "system", 681 | "content": analysis_system, 682 | }, 683 | { 684 | "role": "user", 685 | "content": property_analysis_content_prompt 686 | }, 687 | { 688 | "role": "system", 689 | "content": "Please respond ONLY with valid json that conforms to this pydantic json_schema: {model_class.schema_json()}.\n" 690 | }] 691 | 692 | functions=[ 693 | { 694 | "name": "analyze_components", 695 | "description": "analyze ", 696 | "parameters": AnalysisPropertySchema.schema() 697 | }, 698 | ] 699 | return prompt, functions 700 | 701 | 702 | def is_json_file(file): 703 | try: 704 | json.loads(file) 705 | return True 706 | except ValueError: 707 | return False 708 | 709 | 710 | async def analysis_groups(ctx, high_level_guidelines, pageurl): 711 | result_data = [] 712 | 713 | # define log file 714 | folder_name = ctx.file_name.split(".")[0] 715 | log_file = os.path.join(folder_name, "property.log") 716 | 717 | async with async_playwright() as playwright: 718 | # playwright load page 719 | page = await load_page(playwright, pageurl) 720 | # time.sleep(30) 721 | 722 | property_results = {} 723 | 724 | # text property 725 | text_properties = await get_text_properties(page) 726 | # for item in text_properties: 727 | # print(item) 728 | # print(item['content']) 729 | print(len(text_properties)) 730 | property_results["Text"] = text_properties 731 | 732 | # color property 733 | all_colors = await get_all_colors(page) 734 | # for item in all_colors: 735 | # print(item) 736 | print(len(all_colors)) 737 | property_results["Color"] = all_colors 738 | 739 | # label property 740 | label_properties = await get_label_properties(page) 741 | # for item in label_properties: 742 | # print(item) 743 | print(len(label_properties)) 744 | property_results["Label"] = label_properties 745 | 746 | # clickable property 747 | clickable_properties = await get_clickable_properties(page) 748 | # for item in clickable_properties: 749 | # print(item) 750 | print(len(clickable_properties)) 751 | property_results["Clickable"] = clickable_properties 752 | 753 | # spacing property 754 | spacing_properties = await get_layout_properties(page) 755 | # for item in spacing_properties: 756 | # print(item) 757 | print(len(spacing_properties)) 758 | property_results["Spacing"] = spacing_properties 759 | 760 | # 3 import related guidelines 761 | for property in ["Group", "Clickable", "Spacing", "Platform","Label","Text","Color","Icon"]: 762 | if property in high_level_guidelines and property in property_results and len(property_results[property]) > 0: 763 | 764 | with open(log_file, "a") as file: 765 | guidelines_len = (len(high_level_guidelines[property]['soft'])+len(high_level_guidelines[property]['hard'])) 766 | file.write( property + ": " + str(len(property_results[property])) + " "+ str(guidelines_len) + "\n") 767 | 768 | # 4 analysis and repair 769 | analysis_property_prompt, property_schema = assemble_analysis_property_prompt(ctx.file_content, high_level_guidelines[property], property, property_results[property]) 770 | completion = await get_response(analysis_property_prompt, property_schema) 771 | # print(completion) 772 | 773 | # result validation 774 | max_attempts = 2 775 | for _ in range(max_attempts): 776 | if is_json_file(completion): 777 | break 778 | print("result has problem, regenerate") 779 | completion = await get_response(analysis_property_prompt, property_schema) 780 | 781 | if not is_json_file(completion): 782 | continue 783 | else: 784 | res_content = json.loads(completion) 785 | 786 | ##################### 787 | new_key = f"bad_{property}_property_design" 788 | res_content = res_content["bad_property_design"] 789 | res = {new_key: res_content} 790 | result_data.append(res) 791 | 792 | elif property == "Group" or property == "Icon": # or property == "Platform": 793 | # 4 analysis and repair 794 | analysis_property_prompt, property_schema = assemble_analysis_property_prompt(ctx.file_content, high_level_guidelines[property], property, ["analyze code directly"]) 795 | completion = await get_response(analysis_property_prompt, property_schema) 796 | # print(completion) 797 | 798 | res_content = json.loads(completion) 799 | new_key = f"bad_{property}_property_design" 800 | res_content = res_content["bad_property_design"] 801 | res = {new_key: res_content} 802 | result_data.append(res) 803 | 804 | await page.close() 805 | 806 | # print(result_data) 807 | return result_data 808 | 809 | -------------------------------------------------------------------------------- /library/system_design_knowledge_base.csv: -------------------------------------------------------------------------------- 1 | material_design_section,system_design_aspect,property,constraint,relation,content 2 | Foundations,Structure,Group,soft,Accessibility Hierarchy,"Types of feedback 3 | 4 | Visual feedback (such as labels, colors, and icons) and touch feedback show users what is available in the UI. 5 | 6 | Navigation 7 | 8 | Navigation can have clear task flows with minimal steps, easy-to-locate controls and clear labeling. Focus control, or the ability to control keyboard and reading focus, can be implemented for frequently used tasks. 9 | 10 | Every added button, image, and line of text increases the complexity of a UI. You can simplify how your UI is understood by using: 11 | 12 | Clearly visible elements 13 | Sufficient contrast and size 14 | A clear hierarchy of importance 15 | Key information that is discernable at a glance 16 | To convey an item’s relative level of importance: 17 | 18 | Place important actions at the top or bottom of the screen (reachable with shortcuts) 19 | Place related items of a similar hierarchy next to each other" 20 | Foundations,Layout,Clickable,soft,Accessibility Touch and pointer target sizes,"Touch targets are the parts of the screen that respond to user input, extending beyond the visual bounds of an element. For example, an icon may appear to be 24 x 24dp, but the padding surrounding it comprises the full 48 x 48dp touch target. 21 | For most platforms, consider making touch targets at least 48 x 48dp. A touch target this size results in a physical size of about 9mm, regardless of screen size. The recommended target size for touchscreen elements is 7-10mm. It may be appropriate to use larger touch targets to accommodate a larger spectrum of users. 22 | Note: iOS recommends 44 x 44dp targets. 23 | Pointer targets are similar to touch targets, but are implemented by motion-tracking pointer devices such as a mouse or a stylus. " 24 | Foundations,Layout,Spacing,soft,Accessibility Target spacing,"In most cases, targets separated by 8dp of space or more promote balanced information density and usability." 25 | Foundations,Label,Label,hard,Accessibility Alternative text,"Do,Alt text helps translate a visual UI into a text-based UI. Alt text is a short label (up to 125 characters) in the code that describes an image for users who are unable to see them. Since alt text is only for images, there is no need to add “image of” or “picture of” to the alt text. A screen reader will read the alt text aloud in place of the image. 26 | Use alt text to convey what the image is showing in an informative, short phrase. 27 | " 28 | Foundations,Label,Label,hard,Accessibility Captions,"Do,Captions are the text that appear below an asset. They explain an asset’s contextual information–the who, what, when, and where. Both sighted and screen reader users rely on captions for descriptions of assets. 29 | Captions are the text that appear below an asset. They explain an asset’s contextual information–the who, what, when, and where. Both sighted and screen reader users rely on captions for descriptions of assets. 30 | " 31 | Foundations,Color,Color,soft,Accessibility Essential and non-essential elements,"Informative images have essential and non-essential elements. Essential information should have a 3:1 color contrast ratio for large text and 4.5:1 for small text. 32 | The illustration contains both essential and non-essential information:Essential: The text meets all contrast ratios and size requirementsEssential: An illustrative visual representation of the instructions that follows color contrast guidelinesNon-essential: The decorative elements create background and personality for the illustration. They do not relay information and do not have to meet contrast requirements." 33 | Foundations,Implement,Platform,hard,Accessibility Implementing accessibility,"Do,Use native elements, such as the standard platform dialog" 34 | Foundations,Implement,Platform,hard,Accessibility Implementing accessibility,"Don’t,Be wary of using non-standard elements, such as a non-standard platform dialog to perform a standard dialog task. It requires extra testing to work well with assistive technology." 35 | Foundations,Color,Color,soft,Accessibility Contrast ratios," 36 | Color contrast is important for users to distinguish various text and non-text elements. Higher contrast makes the imagery easier to see, while low-contrast images may be difficult for some users to differentiate in bright or low light conditions, such as on a very sunny day or at night. 37 | Contrast ratios represent how different one color is from another color, commonly written as 1:1 or 21:1. The greater the difference is between the two numbers in the ratio, the greater the difference in relative luminance between the colors. The contrast ratio between a color and its background ranges from 1-21 based on its luminance (the intensity of light emitted) according to the World Wide Web Consortium (W3C). 38 | The W3C recommends the following contrast ratios for body text and image text 39 | | Text type | Color contrast ratio | 40 | | --------------------------------------------------------------------- | ---------------------------- | 41 | | Large text (at 14 pt bold/18 pt regular and up) and graphics`` | 3:1 against the background | 42 | | Small text`` | 4.5:1 against the background | 43 | " 44 | Foundations,Color,Color,soft,Accessibility Clustering elements," 45 | The contrast of the button container color against the background color falls below 3:1, failing the 3:1 contrast ratio standard 46 | The container color exceeds 3:1 contrast ratio against the background color 47 | Standalone components, such as FABs, don’t benefit from a 3:1 contrast ratio between the container and background colors 48 | When placing components together in a cluster, use components or types of components that each achieve at least 3:1 contrast between themselves and the background. 49 | " 50 | Foundations,Text,Text,soft,Accessibility Text resizing,"Most components behave the same when text is resized: 51 | 52 | Text and line height scale up proportionally, multiplied by scale value 53 | Padding remains constant at 1x the default size 54 | Spacing between elements in a component remain constant at 1x the default size 55 | 56 | Button text displayed at 1x, 1.3x, and 2x scales. All have top and bottom padding of 8dp. 57 | 58 | Button text displayed at 1x, 1.3x, and 2x scales. All have left and right padding of 24dp. 59 | 60 | When text resizing isn't controlled by the device OS, offer multipliers such as 1.5x or 2x to allow users to increase the text size. Using multipliers to scale text can result in values with decimals, but this approach is more feasible for implementation. 61 | 62 | To calculate a font's size using multipliers, take the default font size (density = 0) and multiply it by the scale value. 63 | 64 | Button with label text at 1x and 2x scale. 65 | 66 | Components that don't include text, like progress indicators, checkboxes, or radio buttons, aren't affected by text resizing. 67 | " 68 | Foundations,Text,Text,hard,Accessibility Text resizing,"Don’t,Icon button with the icon shown at 1x scale and incorrectly at 2x scale." 69 | Foundations,Text,Text,soft,Text resizing Text Increase container size,"Resizing containers can prevent text from overlapping, clipping, or truncating. 70 | Consider how text might reflow in a way that allows the eye to follow the end of one line to the beginning of the following line" 71 | Foundations,Text,Text,soft,Accessibility Text resizing Reflow the layout,"Consider reflowing the layout, especially when components grow very long. To accommodate larger text, components can be stacked on top of one another, rather than fixed side-by-side." 72 | Foundations,Text,Text,soft,Accessibility Text resizing Enable content to scroll,"When long strings of enlarged text don’t fit on one screen, consider adding a scrollbar to provide access to more content. 73 | Vertical scrolling is preferable to horizontal. Users should only be asked to scroll in one direction, rather than both vertically and horizontally." 74 | Foundations,Text,Text,soft,Accessibility Text resizing Use long press tooltips to provide enlarged labels,"Some components, such as app bars and navigation bars, position text in spaces with stricter space and character limits. In these situations, you can add a tooltip to display enlarged content in the UI. 75 | In this case, the text size in the component remains displayed at 1x while the scaled up text is displayed in a tooltip on long press. 76 | Tooltips are the best choice for displaying enlarged text in: Top app bar 77 | Navigation bar 78 | Navigation rail 79 | Tabs, when fixed to the top of a screen and don’t move off-screen upon scrolling 80 | Tooltip on navigation rail displays scaled up label text." 81 | Foundations,Text,Text,hard,Accessibility Text resizing Use long press tooltips to provide enlarged labels,"Do,Scale up text in an adjacent tooltip to maintain space in a UI for consuming content." 82 | Foundations,Structure,Group,soft,Accessibility Structure Web Landmarks,"For web only: Landmarks and headings help assistive-technology users orient themselves to a web page and allow for easy navigation and traversal across large sections of a document or page. 83 | 84 | By classifying and labeling sections of a page, structural information that is conveyed visually through layout design can also be represented in code. 85 | Landmarks are large blocks of content that establish the high-level structure of your layout. They're a set of ARIA roles that provide easy access to, and important meaning for, common content areas of a web page. 86 | 87 | There are eight landmark roles: navigation, search, main, banner, complementary, contentinfo, region, and form. 88 | 89 | The eight landmark roles in the W3C ARIA guidelines include: 90 | 91 | Navigation: Contains lists of navigation links (there can be multiple, in which case you should differentiate in label) 92 | Search: A search field 93 | Main: The main content area as defined by UX. There should be only one. 94 | Banner: Typically the header; content repeated from page to page, often contains navigation and toolbars. There should be only one. 95 | Complementary: A sidebar or aside to main content that can stand alone without the main content 96 | Contentinfo: Typically the footer; contains information describing the site and its content (for example, copyright). There should be only one. 97 | Region: Content regions are important content blocks. They can be nested inside the “main” landmark. Regions should be labeled with names that make the purpose of that region clear. 98 | Form: Takes and stores user info" 99 | Foundations,Label,Label,soft,Accessibility Structure Add accessibility labels,"Add clear and specific labels to any landmark roles that appear multiple times (regions or navigation typically). This will help users differentiate information. 100 | 101 | Labels should be added to all regions, as well as any landmark where a label will enhance meaning. For example, explaining the contents or purpose of a sidebar. 102 | 103 | Don't repeat the landmark role within a label." 104 | Foundations,Structure,Group,soft,Accessibility Structure Define headings,"Assistive technology users often navigate web pages with the help of headings. They create a clear hierarchy to help users navigate and take action. 105 | 106 | Identify headings based on content hierarchy, rather than visual styling 107 | Headings should not skip a level, for example, don't go from H2 to H4 without using an H3 108 | Map content on your pages to headings (H1–H6) in sequential order based on the hierarchy of your content 109 | A single H1 for the page title is recommended" 110 | Foundations,Flow,Clickable,soft,Accessibility Define initial focus and component-level focus,"Focus refers to which control is currently the active target of user interactions, such as mouse clicks or keyboard taps. Generally, the tab key moves focus between interactive elements. 111 | 112 | Define the initial focus when a user loads a screen, as well as initial focus for components with multiple interactive elements, like a complex card or a dialog. 113 | Focus is particularly important when an element is activated by the user or the user changes context. 114 | 115 | For example, when a dialog is triggered, check for the following: 116 | 117 | Focus is set to the dialog component, likely to a specific interactive element within the dialog such as a text input field or edit button 118 | When the user closes or cancels the dialog, focus returns to the interactive element that initiated the action" 119 | Foundations,Flow,Clickable,soft,Accessibility Define any atypical key traversal through the page and components,"Tab typically moves focus between interactive elements and is often used as primary navigation. Tab + Shift reverses direction. 120 | Arrow keys are typically used to navigate within components (for example, moving between cells in a form or traversing items in a menu.) 121 | Enter activates a link or button, or sends a form when a form item has focus." 122 | Foundations,Label,Label,soft,Accessibility Label elements,"Accessibility labels assist users who cannot rely on a product's visual interface. Thoughtful labels help make the text-based experience as usable as the visual experience. Labels should concisely describe an element's content, purpose and behavior. " 123 | Foundations,Label,Label,hard,Accessibility Add labels for meaningful images and interactive elements,"Do,Labels should be concise, descriptive, and convey the content and context of the image. Labels should be concise, descriptive, and convey the content and context of the image." 124 | Foundations,Label,Label,hard,Accessibility Add labels for meaningful images and interactive elements,"Don’t, Do not use an element role (button, menu, etc.) in your label. This identifier is automatically added when the element is assigned its proper role, typically by a developer." 125 | Foundations,Label,Label,hard,Accessibility Assign a role to interactive elements,"Don't,Don't include the control type in the label. Screen readers automatically add the control, so you’d be having it repeat (for example,ex. “Got it button button”)" 126 | Foundations,Label,Label,soft,Accessibility Assign a role to interactive elements,"ARIA roles apply to web apps and specify how to increase the accessibility of web pages on top of HTML. 127 | 128 | For web, assign ARIA roles for all interactive elements 129 | For non-web, assign roles based on your design system components (button, slider, menu, etc.) 130 | Assign ARIA roles (web) or component type (mobile) to communicate desired interaction patterns into engineering action. Note that some visual elements may look the same, but are intended to behave differently. 131 | 132 | Defining an interactive element's category by assigning it a role helps users of assistive technology establish expectations for how to interact with that element and anticipate what is likely to happen upon interaction." 133 | Foundations,Text,Text,hard,Content alt text,"Do, Write alternative text for images to provide context to screen reader users." 134 | Foundations,Text,Text,hard,Content alt text,"Don't,Avoid leaving the automatically generated file number as alt text." 135 | Foundations,Text,Text,hard,Content Focus on the meaning of the image,"Do,Describe the context and overall meaning, rather than focusing on the details. For example, in a shopping app, focus on the item for sale rather than the things around it. 136 | Alt text can also help improve SEO, but its primary purpose should be to make sites usable for all. 137 | Focus on the important part of the image." 138 | Foundations,Text,Text,hard,Content Focus on the meaning of the image,"Don't,Avoid detailed descriptions that don’t contribute to the image’s meaning." 139 | Foundations,Text,Text,hard,Content Keep it short,"Do,Write brief alt text." 140 | Foundations,Text,Text,hard,Content Keep it short,"Don't,Don’t write more than 125 characters of alt text. " 141 | Foundations,Text,Text,hard,Content Don’t start alt text with image of,"Do,Describe the image, rather than the format." 142 | Foundations,Text,Text,hard,Content Don’t start alt text with image of,"Don't,Avoid writing “image of. The screen reader will announce it’s an image." 143 | Foundations,Text,Text,hard,Content Context matters,"Do,Alt text should always relate to the context." 144 | Foundations,Text,Text,hard,Content Context matters,"Do,Change alt text for an image depending on where it’s used." 145 | Foundations,Text,Text,hard,Content Alt text is subjective,"Do,Make alt text consistent with the caption." 146 | Foundations,Text,Text,hard,Content Alt text is subjective,"Don’t,Don’t ignore the caption. Information in alt text should correspond to adjacent text." 147 | Foundations,Text,Text,hard,Content Alt text is subjective,"Do,Focus on the meaning of the image." 148 | Foundations,Text,Text,hard,Content Alt text is subjective,"Do,Use alt text to reinforce ideas in the caption." 149 | Foundations,Text,Text,hard,Content Alt text is subjective,"Don’t,Don’t write alt with details that aren’t relevant to the context of the image." 150 | Foundations,Text,Text,hard,Content Captions should benefit all users,"Do,Capture the meaning of the image in a few words." 151 | Foundations,Text,Text,hard,Content Captions should benefit all users,"Don’t,Don’t repeat the caption as the alt text" 152 | Foundations,Text,Text,hard,Content Types of imagery,"Caution, Be careful when using alt text that describes the type of image. 153 | Occasionally it benefits users to name the type of image. This can include: 154 | Chart 155 | Infographic 156 | Map 157 | Graph 158 | Screenshot 159 | Headshot 160 | Diagram" 161 | Foundations,Text,Text,hard,Content Charts and graphs,"Do,Capture the meaning of the chart." 162 | Foundations,Text,Text,hard,Content Charts and graphs,"Don’t,Avoid copying data points." 163 | Foundations,Text,Text,soft,Content Charts and graphs,"For a chart with a lot of data points, write alt text for the most important ones. For example, on a stock price graph users would want to know the prices for open, close, high and low. 164 | 165 | Ensure that the concepts shown in the chart or graph are explained fully in the body copy. 166 | 167 | A general formula would be: [chart type] of [data type] where [reason for showing the chart]." 168 | Foundations,Text,Text,hard,Content Word choice Use global examples & explain local references,"Do,Use generalized, global examples. Most countries and cultures have holidays." 169 | Foundations,Text,Text,hard,Content Word choice Use global examples & explain local references,"Don’t,Don’t call out a specific country or culture’s holiday" 170 | Foundations,Text,Text,hard,Content Word choice Use global examples & explain local references,"Caution, 171 | Help translators understand the context by adding message descriptions 172 | If it doesn’t make sense to use a global example, explain the reference in the message description so the translator can substitute a locale-specific example. Some instances where local references should be called out include: 173 | 174 | Locations 175 | Names (common first names and nicknames) 176 | Currencies 177 | Temperatures 178 | Date formats 179 | Providers (internet and cable)" 180 | Foundations,Text,Text,hard,Content Word choice Avoid abbreviations,"Do,Use clear names to refer to things" 181 | Foundations,Text,Text,hard,Content Word choice Avoid abbreviations,"Don’t,Avoid abbreviations" 182 | Foundations,Text,Text,hard,Content Word choice Clarify pronouns,"Do,Using nouns instead of, or in addition to, pronouns can help clarify future and past user actions" 183 | Foundations,Text,Text,hard,Content Word choice Clarify pronouns,"Don’t,Avoid using pronouns when it’s unclear what nouns they’re referring to, especially when explaining user actions" 184 | Foundations,Text,Text,hard,Content Word choice Clarify “this” and “that”,"Do,Make sure it’s clear who text is referring to" 185 | Foundations,Text,Text,hard,Content Word choice Clarify “this” and “that”,"Don’t,Avoid using “this” and “that”" 186 | Foundations,Text,Text,hard,Content Word choice Avoid idiomatic colloquial and polite expressions,"Do,Clear, everyday language can be used in an expressive and whimsical way when paired with imagery" 187 | Foundations,Text,Text,hard,Content Word choice Avoid idiomatic colloquial and polite expressions,"Don’t,Idiomatic phrases can be difficult for everyone to understand and for translators to localize" 188 | Foundations,Text,Text,hard,Content Word choice Reduce technical jargon,"Do,Plain language is easier for everyone to understand" 189 | Foundations,Text,Text,hard,Content Word choice Reduce technical jargon,"Don’t,Confusing language makes it difficult for people to understand the actions they’re taking" 190 | Foundations,Text,Text,soft,Content Word choice Clarify ambiguities,"Some words have multiple meanings. For example, “traffic,” “filter,” and “change” can all be used as either nouns or verbs. Avoid using both meanings of the word in the same string or body of text. If a word has the potential to be confusing, be sure to provide as much context as possible in the message description so the translation will be accurate." 191 | Foundations,Text,Text,hard,Content Style guide Explain consequences,"Do,Tell users what will happen if they take an action and how they can undo it" 192 | Foundations,Text,Text,hard,Content Style guide Explain consequences,"Don’t,Don’t misrepresent consequences or try to influence a user’s decision" 193 | Foundations,Text,Text,soft,Content Style guide Use scannable words and formats,"People scan UI text in search of the most meaningful content to them. Help by using specific titles and headings that clearly describe a topic. When users are skimming or hurrying through an action, this organization helps them avoid mistakes and unintentional actions. People scan UI text in search of the most meaningful content to them. Help by using specific titles and headings that clearly describe a topic. When users are skimming or hurrying through an action, this organization helps them avoid mistakes and unintentional actions." 194 | Foundations,Text,Text,hard,Content Style guide Use sentence case,"Do,Capitalize the first word of a sentence or phrase" 195 | Foundations,Text,Text,hard,Content Style guide Use sentence case,"Don’t,Capitalize the first word of a sentence or phrase" 196 | Foundations,Text,Text,hard,Content Style guide Use abbreviations sparingly,"Do,When an abbreviation is appropriate, make sure it’s formatted and spelled correctly to avoid confusion" 197 | Foundations,Text,Text,hard,Content Style guide Use abbreviations sparingly,"Don’t,Avoid using abbreviations when there’s space to spell out a word" 198 | Foundations,Text,Text,hard,Content Style guide Use second person pronouns (“you”),"Do,Write from the user’s point of view to help them take action" 199 | Foundations,Text,Text,hard,Content Style guide Use second person pronouns (“you”),"Don’t,Avoid writing that sounds impersonal and robotic" 200 | Foundations,Text,Text,hard,Content Style guide Don’t combine first and second person,"Do,Write from a user’s point of view by emphasizing their perspective with “you” and “your”" 201 | Foundations,Text,Text,hard,Content Style guide Don’t combine first and second person,"Don’t,Don’t mix different forms of address in the same screen. Instead, use “you” and “your” or get rid of the pronoun." 202 | Foundations,Text,Text,hard,Content Style guide Use caution with “I” and “we”,"Don’t,Don’t use first person pronouns to speak for the voice of Google" 203 | Foundations,Text,Text,hard,Content Style guide Use caution with “I” and “we”,"Don’t,Avoid using first person pronouns. Write from the user’s point of view by using second person pronouns or removing pronouns altogether." 204 | Foundations,Text,Text,hard,Content Style guide Use caution with “I” and “we”,"Do,First person pronouns can help users understand when they’re making impactful decisions" 205 | Foundations,Text,Text,hard,Content Style guide Use caution with “I” and “we”,"Caution,Use caution with “we” or “our.” Even when these pronouns represent real people employed by Google, seeing first person pronouns in UI text can be confusing or jarring." 206 | Foundations,Text,Text,hard,Content Style guide Skip periods and unnecessary punctuation,"Do,Omit punctuation on single-line sentences" 207 | Foundations,Text,Text,hard,Content Style guide Skip periods and unnecessary punctuation,"Don’t,Avoid using periods to end single sentences" 208 | Foundations,Text,Text,soft,Content Style guide Skip periods and unnecessary punctuation,"To help readers scan text, avoid using periods and other unnecessary punctuation. 209 | Avoid using periods to end single sentences, particularly in: 210 | 211 | Labels 212 | Tooltip text 213 | Bulleted lists 214 | Dialog body text 215 | Hyperlinked text 216 | Use periods on: 217 | 218 | Multiple sentences 219 | Long or complex sentences, if it suits the context 220 | Any sentence followed by a link" 221 | Foundations,Text,Text,hard,Content Style guide Use contractions,"Do,Avoid spelling out words that can be contractions" 222 | Foundations,Text,Text,hard,Content Style guide Use contractions,"Don’t,Phrases that aren’t contracted can feel stiff or overly formal" 223 | Foundations,Text,Text,hard,Content Style guide Use serial commas,"Do,Use a serial comma in lists of three or more items" 224 | Foundations,Text,Text,hard,Content Style guide Use serial commas,"Don’t,Don’t skip serial commas before “and”" 225 | Foundations,Text,Text,hard,Content Style guide Use exclamation points sparingly,"Do,Exclamation marks can be used to emphasize celebratory moments" 226 | Foundations,Text,Text,hard,Content Style guide Use exclamation points sparingly,"Don’t,Avoid using exclamation marks for empty states and common tasks. Save it for bigger accomplishments." 227 | Foundations,Text,Text,hard,Content Style guide Use ellipses sparingly,"Do,Ellipses show an action in progress" 228 | Foundations,Text,Text,hard,Content Style guide Use ellipses sparingly,"Don’t,Don’t use ellipses in buttons or menu items" 229 | Foundations,Text,Text,hard,Content Style guide Use parentheses to define terms,"Do,Use parentheses to define terms and jargon" 230 | Foundations,Text,Text,hard,Content Style guide Use parentheses to define terms,"Don’t,Don’t use parentheses to add extra thoughts. If information is needed, include it in the sentence without parentheses for easier scanning and improved comprehension." 231 | Foundations,Text,Text,hard,Content Style guide Skip ampersands in body text,"Do,Ampersands can be used in headlines" 232 | Foundations,Text,Text,hard,Content Style guide Skip ampersands in body text,"Don’t,Avoid ampersands in email subject lines" 233 | Foundations,Text,Text,soft,Content Style guide Use dashes with caution,"Dashes and hyphens can interrupt a sentence and lead to a fragmented experience, so they should be used with caution. There are three kinds of dashes: 234 | 235 | Em dash: — 236 | En dash: – 237 | Hyphen: - 238 | 239 | Em dashes are best avoided in UX writing, as they indicate a break in the flow of a sentence that could be simplified using a comma, period, or new sentence. 240 | 241 | Use an en dash without spaces to indicate a range, such as 9 AM–Noon." 242 | Foundations,Text,Text,soft,Content Style guide Use hyphens with care,"Hyphens can help readers better understand how words relate to each other by binding closely related words. They can also be used to represent negative numbers, such as -100. Spaces should never be used surrounding hyphens. 243 | 244 | Refer to the Associated Press (AP) style guidelines if you are unsure whether an adjective or noun phrase needs a hyphen. 245 | 246 | Rule Examples Why 247 | Hyphenate adjective phrases 248 | Case-by-case basis 249 | Best-in-class performance 250 | Once-in-a-lifetime opportunity When multiple words are used together as an adjective, they should be hyphenated. However, common, easily understood adjective phrases don't need to be, such as 'cell phone' or 'chocolate chip cookie'. 251 | Cell phone 252 | Chocolate chip cookie Except in the case of proper nouns and easily understood adjective phrases, which don't need to be 253 | Hyphenate noun phrases 254 | 255 | A noun phrase is two or more words acting as a noun. These phrases are hyphenated in certain cases: 256 | Sign-off 257 | Drive-through 258 | Go-ahead Hyphenate a noun phrase if it contains a verb followed by an adverb 259 | Higher-up 260 | Most-read Hyphenate an adjective phrase that is functioning as a noun 261 | Jack-of-all-trades 262 | Stick-in-the-mud Some noun phrases, especially long or complicated ones, are always hyphenated" 263 | Foundations,Layout,Group,soft,Layout Grouping,"Grouping is a method for connecting related elements that share a context, such as an image grouped with a caption. It visually relates elements and establishes boundaries to differentiate unrelated elements. By placing a caption under an image this composition shows an explicit group. 264 | Explicit grouping uses visual boundaries such as outlines, dividers, and shadows to group related elements in an enclosed area. Explicit grouping can also indicate that an item is interactive, such as list items contained between dividers, or a card displaying an image and its caption. 265 | Implicit grouping uses close proximity and open space (rather than lines and shadows) to group related items. For example, a headline closely followed by a subhead and thumbnail image are implicitly grouped together by proximity and separated from other headline-subhead-thumbnail groups by open space." 266 | Foundations,Layout,Spacing,soft,Layout Margins,"Margins are the spaces between the edge of a window area and the elements within that window area. 267 | 268 | Margin widths are defined using fixed or scaling values for each window size class. To better adapt to the window, the margin width can change at different breakpoints. Wider margins are more appropriate for larger screens, as they create more open space around the perimeter of content. 269 | 270 | See margin measurements for each window class: compact, medium, expanded, large, and extra-large. 271 | 272 | Screen highlighting vertical blue margin on left side of screen" 273 | Foundations,Layout,Spacing,soft,Layout Spacers,"A spacer refers to the space between two panes in a layout. Spacers measure 24dp wide." 274 | Foundations,Layout,Spacing,soft,Layout Padding,"Padding refers to the space between UI elements. Padding can be measured vertically and horizontally and does not need to span the entire height or width of a layout. Padding is measured in increments of 4dp." 275 | Foundations,Layout,Spacing,hard,Layout Information density,"Do,Consider using higher density information design when users need to scan lots of information on screen" 276 | Foundations,Layout,Spacing,hard,Layout Information density,"Caution,Avoid reducing selectable targets to less than 48x48dp" 277 | Foundations,Layout,Spacing,hard,Layout Information density,"Don't,Don’t increase density in UIs that involve focused tasks, such as interacting with a dropdown menu or picker as it reduces usability by limiting selectable space." 278 | Foundations,Layout,Spacing,hard,Layout Information density,"Don't,Don't increase the density in components that alert the user of changes, such as snackbars or dialogs." 279 | Foundations,Layout,Spacing,soft,Layout Density scale,"The density scale allows you to control the internal spacing of individual components when needed. 280 | 281 | The density scale is numbered, starting at 0 for a component’s default density. The scale moves to negative numbers (-1, -2, -3) as space decreases, creating higher density. Each increment represents a decrease in the height of a component by 4dp. 282 | 283 | Three buttons, with densities of +1, ), and -1 284 | Buttons are offered in multiple densities to best fit a needs of a design 285 | 286 | When multiple elements are stacked vertically within a component, use 4dp increments to separate them. Center the grouped element within the component container. 287 | 288 | Stacked element showing 20 dp between label and input 289 | The measurement between the label and input is 20dp 290 | 291 | Parent container showing label above input 292 | The label and input are centered within their parent container 293 | 294 | Don’t apply density to interaction targets. They should remain a minimum of 48x48dp, even if their visual element (such as an icon) is smaller. 295 | 296 | Settings button is 24 by 24 dp, but has interaction target of 48 x 48 dp. 297 | The interaction target for an icon button can be larger than its icon, as long as it meets the 48x48dp minimum size. 298 | 299 | Button with height of 36 dp and interaction target of 48 dp" 300 | Foundations,Layout,Group,soft,Layout Navigation region,"The navigation region holds primary navigation components and elements such as: 301 | 302 | Navigation drawer 303 | Navigation rail 304 | Navigation bar 305 | Elements in this section help people navigate between destinations in an app or to access important actions. 306 | 307 | Navigation drawer, navigation rail, and navigation bar each shown on a separate screen." 308 | Foundations,Layout,Group,soft,Layout Body region,"The body region contains most of the content in an app, including: 309 | 310 | Images 311 | Text 312 | Lists 313 | Cards 314 | Buttons 315 | App bar 316 | Search bar 317 | Content in the body region is grouped into one or more panes." 318 | Foundations,Layout,Group,soft,Layout Panes,"A pane refers to the grouping of content within the body region. All app content is inside one or more panes. 319 | 320 | The arrangement and relative size of the layout regions and panes will vary depending on the device’s window size class: 321 | 322 | Compact: 1 pane 323 | Medium: 1 (recommended) or 2 panes 324 | Expanded: 1 or 2 panes (recommended) 325 | Large: 1 or 2 panes (recommended) 326 | Extra-large: 1 to 2 panes (recommended) with standard side sheet serving as a potential additional 3rd pane. 327 | Two paned screen layout. 328 | First pane 329 | Second pane 330 | 331 | Pane types 332 | 333 | Fixed: fixed width (360 dp) 334 | Flexible: responsive to available space, can grow and shrink 335 | Fixed and flexible panes of a screen. 336 | Fixed pane 337 | Flexible pane 338 | 339 | A pane is different from a window, which refers to the frame containing an app. Multi-window views are a system UI feature used to display more than one app simultaneously. 340 | 341 | Multi-window support guide for Android 342 | 343 | Side-by-side windows with single taskbar below. 344 | Two windows shown next to one another with a taskbar underneath. 345 | 346 | Layouts 347 | Single-pane layouts use one flexible pane. 348 | 349 | A mobile screen with a single pane inside a window. 350 | A singular pane extending from one edge to another inside a window. 351 | 352 | Two-pane layouts typically use one fixed and one flexible pane, which can appear in whatever order is best for the content. 353 | 354 | Fixed and flexible panes arranged 2 different ways. 355 | Fixed pane 356 | Flexible pane 357 | 358 | An alternative two-pane layout, called split-pane, uses two flexible panes equal to 50% of the screen width. This layout best supports foldable devices. 359 | 360 | Two flexible panes layout. 361 | First pane 362 | Second pane 363 | 364 | Number of panes Pane type 365 | 1 Flexible 366 | 2 1 fixed, 1 flexible 367 | 2 (split-pane) 2 flexible" 368 | Foundations,Layout,Group,soft,Layout App bars,"Panes can include a top app bar and bottom app bar. 369 | 370 | Screen layout with app bar inside left pane. 371 | An example of an app bar inside a pane. 372 | 373 | Any nesting actions within the top app bar should be hidden or revealed based on available width. 374 | 375 | Top app bar of a compact window on a mobile device shows 2 items. 376 | A compact window with two actions revealed. 377 | 378 | Top app bar of expanded window shows 5 items. 379 | A expanded window class with five items revealed. 380 | 381 | When layouts transition from one to two panes, avoid shifting elements between panes. 382 | Columns 383 | Content in a pane can be displayed in multiple columns to segment and align content. 384 | 385 | Columns are exclusive to a pane and are not used at the window level. 386 | 387 | A single pane containing 3 columns." 388 | Foundations,Layout,Group,hard,Layout App bars,"Don’t,Don’t move elements to different UI objects when switching between window classes." 389 | Foundations,Layout,Platform,soft,Layout Display cutout,"A display cutout is an area on some devices that extends into the display surface. It allows for an edge-to-edge experience while providing space for important sensors on the screen of the device. 390 | 391 | Applications can extend around display cutouts or other features, but some parts of the UI might be obscured. 392 | 393 | Content safe area shown in portrait and in landscape mode." 394 | Foundations,Layout,Platform,soft,Layout Foldable devices,"Fold 395 | The fold of a foldable device divides the screen into two portions, either horizontally or vertically. The fold can be a flexible area of the screen or, on dual-screen devices, a hinge that separates two displays. 396 | 397 | A flexible fold is barely visible, although some users may feel a tactile difference on the screen surface. Content can flow over the fold fairly easily. 398 | 399 | Center fold of a foldable device layout. 400 | Folds are typically found in the center of the device screen and can present a seamless experience. 401 | 402 | On devices with a physical hinge, designing the screen as two distinct sections (separate window areas or panes) allows a composition to work well across the hinge and screens. 403 | 404 | Center fold on a foldable device with a physical hinge. 405 | A physical hinge separates two parts. There is no display hardware in this region. 406 | 407 | Device state 408 | Foldable devices can have several physical states: folded, open flat, and tabletop. 409 | 410 | Folded 411 | The folded state can include a front screen, which often fits in the compact window size class, just like a mobile phone in portrait orientation. 412 | 413 | Compact window of a folded device. 414 | The front screen of a foldable device 415 | 416 | Open flat 417 | An open flat state refers to the fully opened screen, which usually increases the window size class to medium or expanded. An open device can be used in landscape or portrait orientations. 418 | 419 | Open portrait state of a mobile device. 420 | In an open portrait state, the longer device edge is vertical while the shorter edge is horizontal. 421 | 422 | Open landscape state of a mobile device. 423 | In an open landscape state, the longer device edge is horizontal while the vertical edge is shorter. 424 | 425 | Tabletop 426 | Tabletop refers to a half-opened state forming a rough 90 degree angle, with one half of the device resting on a surface. This posture resembles a laptop. 427 | 428 | UI controls near the fold can be difficult for users to access, and text overlaying the fold can be hard to read. 429 | 430 | Tabletop state of a mobile device showing camera ;ems on the vertical plane. 431 | If camera hardware is present, a tabletop device is best positioned on a side without any protruding hardware elements." 432 | Foundations,Layout,Platform,soft,Layout App continuity,"When running on a foldable device, an app can transition from one screen to another automatically. After the transition, the app should resume in the same state and location, and the current task should continue seamlessly. 433 | 434 | A news app in compact mode compared to the open landscape state where the news app expands with a new column next to the compact news feed. 435 | A news app shows a feed in a compact and expanded window class when a foldable device switches device state." 436 | Foundations,Layout,Platform,soft,Layout Scrolling and multiple panes,"Depending on how your app uses panes, the scroll behavior of a folded design may change in the unfolded design. 437 | 438 | If you expand a pane, you can decide whether the whole window will scroll together or if each side (each pane) scrolls independently. 439 | 440 | A foldable device screen in open landscape mode with a single pane showing vertical scroll arrows. 441 | A single pane can scroll its inside content vertically and horizontally. 442 | 443 | If your design has multiple panes, each pane can operate as an independently scrollable area. 444 | 445 | A foldable device screen in open portrait mode with double panes each with a vertical scroll arrow. 446 | Multiple panes can scroll inside content independently of one another." 447 | Foundations,Layout,Platform,soft,Layout Multi-window mode,"Multi-window mode is an Android system feature for displaying multiple apps on the same screen. This can be especially useful for multi-tasking, or workflows that depend on comparing information. 448 | 449 | Note: This concept should not be confused with using multiple panes to display content from a single app. For more on that, see: Panes. 450 | 451 | 2 apps appear side-by-side with a task bar below spanning the width of the screen. 452 | Screen displaying an email app and a contacts app in multi-window mode" 453 | Foundations,Layout,Platform,soft,Layout User needs,"The ways that windows are created, arranged, and adjusted should feel straightforward for all users and across any window size class. Methods for seamless window management include: 454 | Apply smooth transitions as described in motion guidance 455 | Ensure that users can create multiple windows easily and move between them as needed 456 | Keep mental models and interaction patterns simple so that users aren’t required to think about which mode is appropriate for each task 457 | Design and implement window dynamics consistently across variations in foldable hardware, including those with a hinge that separates two displays" 458 | Foundations,Layout,Platform,soft,Layout Window creation and behavior,"Android provides several ways for users to create a multi-window view." 459 | Foundations,Layout,Platform,soft,Layout Taskbar,"The taskbar provides a launching point for pinned and suggested apps to easily become a separate window. 460 | To create a new window, a user selects and drags an app from the taskbar and moves the app icon to indicate where the new window should be displayed. 461 | The taskbar is positioned at the bottom of a screen. 462 | Android taskbar" 463 | Foundations,Layout,Platform,soft,Layout Context menu,"Users can also create multiple windows through the overview by the app context menu. 464 | 465 | 2 apps appear side-by-side with a task bar below spanning the width of the screen. 466 | Multi-window mode can have vertical positioning 467 | 468 | 2 apps are stacked in landscape mode with a task bar below spanning the width of the screen. 469 | Multi-window mode can have horizontal positioning" 470 | Foundations,Layout,Platform,soft,Layout Adjusting window sizes,"By default multiple windows are created as a 50/50 side-by-side split. 471 | 472 | The windows can be adjusted further to 1:3 or 2:3 proportions. These ratios provide a primary and secondary window dynamic, offering greater flexibility and allowing focus on one application as needed. 473 | 474 | When in a multi-window mode, the available screen area often changes from medium or expanded window class to compact. Layouts should adapt accordingly. 475 | 476 | 2 apps appear side-by-side with the left-side app using two-thirds of the screen, and the right app one-third. 477 | The screen handle can be dragged and released to create the desired window ratio. The handle automatically adjusts to the closest snap point." 478 | Styles,Typography,Text,hard,Font Adjustable axes,"Be careful of using excessively lighter weight type for body text. Lower resolution displays can struggle with rendering more delicate typography, especially at small sizes. Instead, consider lighter weights at larger font sizes, such as display type." 479 | Styles,Typography,Text,hard,Font Adjustable axes,"Excessive weight at smaller sizes may affect legibility" 480 | Styles,Typography,Text,hard,Font Width,"Do,A thinner width can allow for more characters to fit at small sizes, such as in a label" 481 | Styles,Typography,Text,hard,Font Width,"Don’t,Styles,Typography,Text,hard,Font Width, A thinner width can allow for more characters to fit at small sizes, such as in a label" 482 | Styles,Typography,Text,hard,Font Optical size,"Do,Use an optical size that matches your type size" 483 | Styles,Typography,Text,hard,Font Optical size,"Don’t,Don’t use large optical type sizes at small sizes. Instead use a smaller optical size, if available." 484 | Styles,Typography,Text,soft,Font Applying type Headline,"Headlines are best-suited for short, high-emphasis text on smaller screens. These styles can be good for marking primary passages of text or important regions of content. 485 | Headlines can also make use of expressive typefaces, provided that appropriate line height and letter spacing is also integrated to maintain readability. 486 | Example expressive headline typeface on phone screen 487 | Expressive typefaces can be used for headlines styles as well 488 | The name Ana Russo in headline style in a contact card 489 | Headline style used for short text on a small screen 490 | Headline style set above body text 491 | Dialog using a headline style" 492 | Styles,Typography,Text,soft,Font Applying type Title,"Titles are smaller than headline styles, and should be used for medium-emphasis text that remains relatively short. For example, consider using title styles to divide secondary passages of text or secondary regions of content. 493 | 494 | For titles, use caution when using expressive fonts, including display, handwritten, and script styles. 495 | 496 | An article card using title style for the article title 497 | A news article title using the title style to capture attention 498 | Top app bar using title style for the contact name, Aki Aro 499 | Top app bar using title style 500 | 501 | Title style stating ‘Top News’ above headlines 502 | Example of title style applied to a category header: Top News" 503 | Styles,Typography,Text,soft,Font Applying type Body,"Body styles are used for longer passages of text in your app. 504 | 505 | Use typefaces intended for body styles, which are readable at smaller sizes and can be comfortably read in longer passages. 506 | 507 | Avoid expressive or decorative fonts for body text because these can be harder to read at small sizes. 508 | 509 | Body-style typeface 510 | Body styles must be readable for long passages 511 | 512 | Body style text in article 513 | Body style used throughout an article about pesto 514 | 515 | Three stacked examples of body style in app setup flow 516 | Example of body style used throughout a setup flow" 517 | Styles,Typography,Text,soft,Font Applying type Body,"Label styles are smaller, utilitarian styles, used for things like the text inside components or for very small text in the content body, such as captions. 518 | 519 | Buttons, for example, use the label large style. 520 | 521 | Several label-style buttons on device screen. 522 | Label styles should enable quick reading at small sizes, such as in buttons 523 | Music player timecode featuring label style 524 | A music player using label style for the timecode 525 | 526 | Five labels set horizontally across the navigation bar 527 | A navigation bar using label style for the destination text" 528 | Styles,Typography,Text,soft,Font Typesetting Using padding and bounding boxes,"Use this method for web products, and iOS products, where applicable. Note that some design tools also use bounding boxes for typesetting, but their methods vary and will need to be reconciled with the engineering implementation. 529 | 530 | In web UIs, the line height and bounding box height are the same. Text is vertically centered within the bounding box, following the “half-leading” behavior established by CSS. 531 | 532 | The vertical position of the text isn’t controlled directly, but through the combination of the bounding box and font metrics. 533 | 534 | Diagram of bounding box equaling line height 535 | The bounding box height is defined by the line height specified, with equal space placed above and below the text 536 | 537 | Padding is the space between UI elements, such as between an image and a bounding box, or between the inner edge of the bounding box and the text. 538 | 539 | Diagram of 20 dp padding above text 540 | The padding surrounding the text bounding box 541 | 542 | Specify the distance of UI elements from fixed reference points, such as the container edge. For the web, automate this calculation using Sass or CSS." 543 | Styles,Typography,Text,hard,Font Typesetting Using padding and bounding boxes,"Do,Use line-height, padding, and container measurements for setting typography on the web and iOS" 544 | Styles,Typography,Text,soft,Font Typesetting Using padding and bounding boxes,"Vertical alignment using padding and bounding boxes: 545 | 546 | Line height 547 | Measure the height of the bounding box 548 | Centering 549 | Ensure equal top and bottom padding around the inner edge of the bounding box by using center align 550 | Spacing 551 | Use the height of the bounding box, and top and bottom padding to determine spacing 552 | Diagram of using padding and bounding boxes to measure line height, alignment, and spacing." 553 | Styles,Typography,Text,soft,Font Typesetting Using the baseline,"Vertical alignment using the baseline: 554 | Line height 555 | Measure distance from the text baseline of one line to the text baseline of the next line 556 | Centering 557 | Specify center alignment as a reference instead of measuring the distance to the text baseline 558 | Spacing 559 | Use the distance from a reference point to the text baseline" 560 | Styles,Typography,Text,hard,Font Typesetting Using the baseline,"Do,Android screens rely on distance to baselines for spacing" 561 | Styles,Typography,Text,soft,Font Ensuring readability,"Line height 562 | 563 | Line height is the space between each line of text and is directly connected to type size. 564 | 565 | Material’s type tokens are optimized for intended size and use. 566 | 567 | An example of larger type with a line height ratio of 1.2 568 | For larger type legibility using styles like title, headline, and display, we recommend a line height ratio of 1.2 times the type size 569 | 570 | An example of body type with a line height ratio of 1.5 571 | For smaller body copy using styles like body and label, we recommend a line height ratio around 1.5 times the type size. If your line height is too tight, you’ll undermine the flow of the text. Too loose, and the lines won’t feel cohesive. 572 | Tabular numbers 573 | 574 | Use tabular figures (also known as monospaced numbers) rather than proportional digits in tables or places where values may change often, such as clocks. 575 | 576 | Use monospaced tabular numbers to keep values optically aligned for better scanning. 577 | 578 | Vertical view showing the spacing variation in proportional numbers versus monospacing of tabular numbers 579 | Proportional numbers 580 | Monospaced tabular numbers 581 | Times displayed in tabular numbers 582 | Use tabular numbers to prevent layout shifting when values change, such as in a clock UI" 583 | Styles,Typography,Text,hard,Font Using Material Symbols with typography,"Do,Use the same size for your Material Symbols and text" 584 | Styles,Typography,Text,hard,Font Using Material Symbols with typography,"Don’t,Don’t mix the sizes of your symbol and text" 585 | Styles,Typography,Text,hard,Font Using Material Symbols with typography,"Do,Use the same weight for your symbol and text" 586 | Styles,Typography,Text,hard,Font Using Material Symbols with typography,"Don’tDon’t use the different weights for Material Symbols and text" 587 | Styles,Typography,Text,hard,Font Using Material Symbols with typography,"Do,Shift down the baseline of symbols to approximately 11.5% of the text size" 588 | Styles,Typography,Text,hard,Font Using Material Symbols with typography,"Don’t,Don’t use the same baseline for Material Symbols and text" 589 | Styles,Typography,Color,soft,Accessibility,"Color & contrast 590 | Support visual accessibility by choosing the appropriate color contrast between your product’s text and background. Contrast is the perceived difference between the lightness or darkness of two colors, and is quantified by a contrast ratio. Key contrast ratios indicate levels of contrast that are sufficient for accessibility. 591 | 592 | Learn more about contrast and accessibility 593 | 594 | Text should achieve sufficient contrast between its color and that of its background. Sufficient contrast depends on the size of the text and the level of accessibility required. 595 | 596 | Learn more about contrast requirements for text 597 | 598 | Large text should achieve a contrast ratio of 4.5:1 for AAA-level accessibility 599 | Small text should achieve a contrast ratio of 7:1 for AAA-level accessibility 600 | 601 | The default color for typography is on surface, although on surface variant is a strong alternative. 602 | 603 | Diagram showing the default color for text is 'on surface'. 604 | Default typography colors 605 | 606 | For hyperlinked text appearing on top of a surface color, use primary or tertiary. 607 | 608 | Hyperlinked text must also be underlined. 609 | 610 | Hyperlinks should be underlined and use primary or tertiary color" 611 | Styles,Elevation,Clickable,soft,Depicting elevation,"To successfully depict elevation, a surface must show: 612 | 613 | Surface edges, contrasting the surface from its surroundings 614 | Overlap with other surfaces, either at rest or in motion 615 | Distance from other surfaces 616 | 3 images. The first shows a purple square overlapping a white square. The second shows a purple square overlapping another purple square but with shadows beneath the top square. The third shows a purple square overlapping a dark gray square" 617 | Styles,Elevation,Clickable,soft,Tonal difference,"Elevation, scrim, and tonal differences used to indicate separation. 618 | A FAB's elevation helps separate it from body content 619 | A scrim appears below a modal to communicate importance 620 | Tonal differences between a top app bar and body content indicate separate surfaces" 621 | Styles,Elevation,Clickable,hard,Tonal difference,"Do,Ensure floating elements have sufficient contrast with surfaces beneath" 622 | Styles,Elevation,Clickable,hard,Tonal difference,"Don't,Don't use colors with insufficient contrast. The relationship between surfaces must be clear." 623 | Styles,Elevation,Clickable,soft,Shadows,"Podcast app with each show displayed as a card separated from the background using small dark shadows 624 | Smaller, sharper shadows indicate a surface’s close proximity to the surface behind it 625 | 626 | Podcast app with each show displayed as a card separated from the background using more fuzzy and diffused shadows 627 | Larger, softer shadows express more distance between a surface and the one behind it 628 | 629 | When it comes to applying shadows, less is more. The fewer levels in your UI, the more power they have to direct attention and action." 630 | Styles,Elevation,Clickable,soft,When to use visible shadows,"Protect elements 631 | 632 | When a background is patterned or visually busy, the hairline style might not provide sufficient protection. In these cases, use elevation to separate and emphasize elements such as cards, chips, or buttons, as in the image below. 633 | 634 | Buttons with shadows separating them from a background image 635 | Interactive elements are emphasized with elevation 636 | 637 | Encourage interaction 638 | 639 | Elements can temporarily lift on focus, selection, or another kind of interaction, like swipe. A raised element can also lower when a higher element appears" 640 | Styles,Color,Color,hard,Pairing and layering colors,"Do,Pair and layer color roles as intended to ensure expected visual results and accessibility. In this example, the two buttons mapped with (1) primary, (2) on primary, (3) secondary container, and (4) on secondary container stay legible as the contrast level changes." 641 | Styles,Color,Color,hard,Pairing and layering colors,"Don't,Improper color mappings can produce unintended visual results and break accessibility. In this example, the two buttons mapped with (1) primary, (2) primary container, (3) secondary container, and (4) on surface become illegible as the contrast level changes." 642 | Styles,Color,Color,soft,Primary,"Use primary roles for the most prominent components across the UI, such as the FAB, high-emphasis buttons, and active states. 643 | 644 | Primary – High-emphasis fills, texts, and icons against surface 645 | 646 | On primary – Text and icons against primary 647 | 648 | Primary container – Standout fill color against surface, for key components like FAB 649 | 650 | On primary container – Text and icons against primary container" 651 | Styles,Color,Color,soft,Secondary,"Use secondary roles for less prominent components in the UI such as filter chips. 652 | 653 | There are four secondary roles: 654 | 655 | Secondary – Less prominent fills, text, and icons against surface 656 | 657 | On secondary – Text and icons against secondary 658 | 659 | Secondary container – Less prominent fill color against surface, for recessive components like tonal buttons 660 | 661 | On secondary container – Text and icons against secondary container 662 | 4 color swatches: secondary, on secondary, secondary container, and on secondary container." 663 | Styles,Color,Color,soft,Tertiary,"Use tertiary roles for contrasting accents that balance primary and secondary colors or bring heightened attention to an element such as an input field. 664 | 665 | There are four tertiary roles: 666 | 667 | Tertiary – Complementary fills, text, and icons against surface 668 | 669 | On tertiary – Text and icons against tertiary 670 | 671 | Tertiary container – Complementary container color against surface, for components like input fields 672 | 673 | On tertiary container – Text and icons against tertiary container 674 | The tertiary color roles can be applied at the designer's discretion. They're intended to support broader color expression." 675 | Styles,Color,Color,soft,Error,"Use error roles to communicate error states, such as an incorrect password entered into a text field. 676 | 677 | There are four error roles: 678 | 679 | Error – Attention-grabbing color against surface for fills, icons, and text, indicating urgency 680 | 681 | On error – Text and icons against error 682 | 683 | Error container – Attention-grabbing fill color against surface 684 | 685 | On error container – Text and icons against error container 686 | Error is an example of a static color (it doesn't change even in dynamic color schemes). Error color roles are made static by default with any dynamic color scheme." 687 | Styles,Color,Color,soft,Surface,"Use surface roles for more neutral backgrounds, and container colors for components like cards, sheets, and dialogs. 688 | 689 | There are three surface roles: 690 | 691 | Surface – Default color for backgrounds 692 | 693 | On surface – Text and icons against any surface color 694 | 695 | On surface variant – Lower-emphasis color for text and icons against any surface color 696 | 3 color swatches: Surface, On surface and On surface variant. 697 | Surface and on surface roles in light theme 698 | 699 | There are also five surface container roles named based on their level of emphasis: 700 | 701 | Surface container lowest – Lowest-emphasis container color 702 | 703 | Surface container low – Low-emphasis container color 704 | 705 | Surface container – Default container color 706 | 707 | Surface container high – High-emphasis container color 708 | 709 | Surface container highest – Highest-emphasis container color 710 | Surface container is the default role, but the others are especially helpful for creating hierarchy and nested containers in layouts for expanded screens. 711 | 712 | 4 color swatches in light and dark theme: Surface container lowest, Surface container low, Surface container, Surface container high and Surface container highest. 713 | The five surface container roles, shown in light and dark theme 714 | 715 | The most common combination of surface roles uses surface for a background area and surface container for a navigation area. 716 | 717 | Email app using Surface for the main background color and Surface Container for the navigation bar background 718 | Surface 719 | Surface container 720 | 721 | All color mappings – but especially surface colors – should remain the same for layout regions across window size classes. For example, the body area will use the surface color and the navigation area will use the surface container color on both mobile and tablet. 722 | 723 | Mobile and tablet screens both using Surface for main background and surface container for navigation background. 724 | Surface 725 | Surface container 726 | 727 | Depending on necessary hierarchy, feature area, and design logic, you can use add-on surface colors in larger window class sizes as long as colors are consistently applied. 728 | 729 | Mobile, foldable, and tablet screens showing how the body and navigation regions have the same color roles across window sizes. 730 | In this example, the body and navigation regions have the same color roles across window size classes (surface and surface container, respectively) with the addition of other surface container colors at larger sizes. 731 | 732 | By default, neutral-colored components such as navigation bars, menus, or dialogs are mapped to specific surface container roles, but these roles can be remapped by makers to suit user needs. 733 | 734 | Surface container low applied to an elevated button and card, surface container applied to the top and bottom bar, surface container high applied to the FAB and basic dialog, surface container highest applied to an input label and off switch. 735 | Default surface container roles applied to components: 736 | 737 | Surface container low 738 | Surface container 739 | Surface container high 740 | Surface container highest" 741 | Styles,Color,Color,soft,Inverse colors,"Inverse roles are applied selectively to components to achieve colors that are the reverse of those in the surrounding UI, creating a contrasting effect. 742 | 743 | Inverse surface – Background fills for elements which contrast against surface 744 | 745 | Inverse on surface – Text and icons against inverse surface 746 | 747 | Inverse primary – Actionable elements, such as text buttons, against inverse surface 748 | 3 color swatches: Inverse surface, inverse on surface and inverse primary roles. 749 | Inverse surface, inverse on surface, and inverse primary roles in the color scheme, shown in light theme 750 | 751 | Snackbar component using inverse surface for its background, inverse on surface for its text and inverse primary of its text button, shown in light theme 752 | A snackbar which uses: 753 | 754 | Inverse surface for its background 755 | Inverse on surface for its text 756 | Inverse primary for its text button" 757 | Styles,Color,Color,soft,Outline,"There are two outline colors to be used against surface: 758 | 759 | Outline – Important boundaries, such as a text field outline 760 | 761 | Outline variant – Decorative elements, such as dividers 762 | 2 color swatches: Outline and outline variant 763 | Outline and outline variant roles in the color scheme, shown in light theme 764 | 765 | Diagram comparing used of outline and outline variant. 766 | A text field which uses outline for its container border 767 | A list item which uses outline variant for its divider line" 768 | Styles,Color,Color,hard,Outline,"Don’t,Don’t use the outline color for dividers since they have different contrast requirements. Instead, use outline variant." 769 | Styles,Color,Color,hard,Outline,"Don’t,Don’t use the outline color for components that contain multiple elements, such as cards. Instead, use outline variant." 770 | Styles,Color,Color,hard,Outline,"Don’t,Don’t use the outline variant color for clustered elements like chips, or other UI elements that are in close proximity to each other. Instead, use outline or another color providing 3:1 contrast with the surface color." 771 | Styles,Color,Color,hard,Outline,"Don’t,Don’t use the outline variant color to create visual hierarchy or define the visual boundary of targets. Instead, use the outline color or another color providing 3:1 contrast with the surface color." 772 | Styles,Color,Color,soft,Fixed accent colors,"Primary fixed, secondary fixed, and tertiary fixed are fill colors used against surface. These colors maintain the same tone in light and dark themes, as opposed to regular container colors, which change in tone between these themes. The fixed color role may be used instead of the equivalent container role in situations where such fixed behavior is desired. 773 | 774 | The primary fixed dim, secondary fixed dim, and tertiary fixed dim roles provide a stronger, more emphasized tone relative to the equivalent fixed color. They may be used where a deeper color but the same fixed behavior is desired. 775 | 776 | 6 color swatches: Primary, secondary and tertiary fixed swatches, along with their darker Dim counterparts, shown in both light and dark theme. 777 | Fixed and fixed dim color roles for the primary, secondary, and tertiary color groups, shown in both light and dark themes. Note how the colors stay the same between themes. 778 | 779 | FAB in light and dark theme, using the primary fixed role for its container fill color. 780 | A FAB which uses primary fixed (1) for its container fill color, shown in light and dark themes. Note how the container color stays the same between themes. 781 | 782 | FAB in light and dark theme, using the primary container role for its container fill color. 783 | For comparison, a FAB which uses primary container (2) for its container fill color, shown in light and dark themes. Note how the container color changes tone between themes. 784 | 785 | 2 email app screens using primary fixed and primary fixed dim. 786 | Examples of fixed and fixed dim colors in use: 787 | 788 | Primary fixed for a FAB container color 789 | Primary fixed dim for an icon button container" 790 | Styles,Color,Color,hard,Fixed accent colors,"Don’t,Similar to container roles, fixed colors do not pass minimum contrast requirements such as 3:1 for non-text contrast and 4.5:1 for text contrast. They should not be applied to elements requiring such contrast." 791 | Styles,Color,Color,hard,Fixed accent colors,"Do,Use the primary, secondary, and tertiary roles for accent colors where minimum contrast is required" 792 | Styles,Shape,all elements,soft,Shape scale,"The scale is a range of seven corner shape styles defining the amount of cut or roundedness: None, extra small, small, medium, large, extra large, and full. 793 | 794 | Each role in the scale defines two things: 795 | 796 | Shape family (rounded or cut) 797 | Value 798 | By default all roles use the rounded shape family. Most values are expressed in absolute dp measurements, except for the full style which is expressed in percentage. 799 | 800 | Apply shape styles using tokens 801 | 802 | Shape scale showing all 7 levels for rounded and cut corner styles 803 | None 804 | Extra small 805 | Small 806 | Medium 807 | Large 808 | Extra large 809 | Full" 810 | Styles,Shape,all elements,soft,Baseline shapes,"By default, every component is mapped to one of the seven shape styles. 811 | Shape style 812 | Component 813 | None 814 | Banners 815 | Bottom app bars 816 | Full-screen dialogs 817 | Lists 818 | Navigation bars 819 | Navigation rails 820 | Progress indicators 821 | Search view (full-screen) 822 | Side sheets (docked) 823 | Tabs 824 | Top app bars 825 | Extra small 826 | Autocomplete menu 827 | Select menu 828 | Snackbars 829 | Standard menu 830 | Text fields 831 | Small 832 | Chips 833 | Rich tooltip 834 | Medium 835 | Cards 836 | Small FABs 837 | Large 838 | Extended FABs 839 | FABs 840 | Navigation drawers 841 | Extra large 842 | Bottom sheets (docked) 843 | Dialogs 844 | Floating sheets 845 | Large FABs 846 | Search view (docked) 847 | Time picker 848 | Time input 849 | Full 850 | Badge 851 | Buttons 852 | Icon buttons 853 | Sliders 854 | Switches 855 | Search bar" 856 | Styles,Shape,all elements,soft,Symmetry,"Components can have either symmetric or asymmetric corner shapes. Symmetric shapes have the same values for all corners, while asymmetric shapes can have corners with different values. A shape token that ends with a “directional modifier” – such as top, bottom, start, or end – is always asymmetrical. 857 | 4 shapes illustrating symmetrical and asymmetrical styles 858 | Extra small top 859 | Large top 860 | Extra large top 861 | Large end 862 | Tokens for asymmetrical shapes include a “directional modifier” like top, bottom, start, or end." 863 | Styles,Shape,all elements,soft,Shape family,"Components can have one of two shape families: 864 | Rounded corners 865 | Cut corners 866 | rounded shape and cut corner shape 867 | Rounded corner 868 | Cut corner" 869 | Styles,Shape,all elements,soft,Customizing shapes For a style,"You can customize the shape family and size of any style level in the shape scale. Such changes affect all components mapped to that shape style, except for those with an override. 870 | Customizing the corner size of the Medium style applies the change to all components using this style, such as cards and small FABs" 871 | Styles,Shape,all elements,soft,For a specific component,"You can change the style level that a component is mapped to. 872 | For example, by default, buttons are mapped to the shape style “full.” If your product requires a smaller amount of roundedness, it can be remapped to the small or medium style in shape scale. 873 | Remapping the button shape to a different style applies the change to all buttons in the UI" 874 | Styles,Shape,all elements,hard,Shape family,"Caution,To avoid unintended results, consider the choice of shape family and size in tandem with a component's layout and content" 875 | Styles,Icon,Icon,hard,Design principles,"Do,Simplify icons for greater clarity and legibility" 876 | Styles,Icon,Icon,hard,Design principles,"Don’t,Don’t be overly literal. Avoid complex icons." 877 | Styles,Icon,Icon,hard,Design principles,"Do,Make icons graphic and bold" 878 | Styles,Icon,Icon,hard,Design principles,"Don’t,Don’t use delicate or loose organic shapes" 879 | Styles,Icon,Icon,hard,Design principles,"Do,Use and maintain a consistent visual style throughout one icon set" 880 | Styles,Icon,Icon,hard,Design principles,"Don’t,Avoid mixing styles for one icon set" 881 | Styles,Icon,Icon,hard,Design Icon sizes and layout,"Do,Icon content is limited to the 20dp x 20dp live area, with 2dp of padding around the perimeter" 882 | Styles,Icon,Icon,hard,Design Icon sizes and layout,"Caution,If additional visual weight is needed, content may extend into the padding between the live area and the trim area" 883 | Styles,Icon,Icon,hard,Design Icon sizes and layout,"Don’t,No parts of the icon should extend outside of the trim area" 884 | Styles,Icon,Icon,hard,Design Grid and keyline shapes,"Do,Position icons “on pixel” within the icon grid" 885 | Styles,Icon,Icon,hard,Design Grid and keyline shapes,"Don’t,Don’t place the icon on a coordinate that isn’t 'on pixel'" 886 | Styles,Icon,Icon,hard,Design Icon metrics,"Caution,Overly round corners reduces the symbol’s legibility" 887 | Styles,Icon,Icon,hard,Design Icon metrics,"Don’t,Don’t use inconsistent corner radii" 888 | Styles,Icon,Icon,hard,Design Weight and stroke,"Don’t,Don’t use inconsistent stroke weights or rounded stroke terminals" 889 | Styles,Icon,Icon,hard,Design Weight and stroke,"Do,Use consistent stroke weights and squared stroke terminals" 890 | Styles,Icon,Icon,hard,Design Complex icon shapes,"Don’t,Don’t tilt, rotate, or make icons appear dimensional" 891 | Styles,Icon,Icon,hard,Design Complex icon shapes,"Do,Make icons face forward" 892 | Styles,Icon,Icon,hard,Applying icons Customizing Symbols,"Do,Apply weights consistently" 893 | Styles,Icon,Icon,hard,Applying icons Customizing Symbols,"Don’t,Don't use the lightest weight for standard-size (24dp) icons. The minimum weight for the size is 200." 894 | Styles,Icon,Icon,hard,Applying icons Customizing Symbols,"Caution,Be careful using excessive weight for standard 24dp symbols" 895 | Styles,Icon,Icon,hard,Applying icons Customizing Symbols,"Don’t,Don’t mix different weights" 896 | Styles,Icon,Icon,hard,Applying icons Using symbols with typography,"Don’t,Don’t use the different weights for Material Symbols and text" 897 | Styles,Icon,Icon,hard,Applying icons Using symbols with typography,"Do,Use the same weight for Material Symbols and Google Sans family text" 898 | Styles,Icon,Icon,hard,Applying icons Using symbols with typography,"Don’t,Don’t use the same baseline for Material Symbols and text" 899 | Styles,Icon,Icon,hard,Applying icons Using symbols with typography,"Do,Shift down the baseline of symbols to approximately 11.5% of the type size" 900 | Styles,Icon,Icon,hard,Applying icons Accessibility,"Caution,Use caution if icons are displayed without labels. Icon meaning should always be unambiguous and accessible for all users. Text labels can be omitted in specific circumstances where reduced visual impact is necessary." 901 | Styles,Icon,Icon,hard,Applying icons Accessibility,"Do,Label text provides short descriptions, especially useful for navigation" 902 | --------------------------------------------------------------------------------