├── .dockerignore ├── .env.example ├── .gitignore ├── Dockerfile ├── LICENSE ├── Makefile ├── OAIWUI_GPT.py ├── OAIWUI_GPT_WebUI.py ├── OAIWUI_Images.py ├── OAIWUI_Images_WebUI.py ├── OAIWUI_WebUI.py ├── README.md ├── assets ├── Infotrend_Logo.png ├── Infotrend_LogoOnly.png ├── Screenshot-OAIWUI_WebUI_GPT.jpg ├── Screenshot-OAIWUI_WebUI_GPT_small.jpg ├── Screenshot-OAIWUI_WebUI_Image.jpg └── Screenshot-OAIWUI_WebUI_Image_small.jpg ├── common_functions.py ├── common_functions_WebUI.py ├── config.sh ├── entrypoint.sh ├── list_models.py ├── models.json ├── models.md ├── models.txt ├── ollama_helper.py ├── prompt_presets.example └── shakespeare.json ├── prompt_presets_settings-example.json ├── pyproject.toml └── unraid └── OpenAI_WebUI.xml /.dockerignore: -------------------------------------------------------------------------------- 1 | # Ignore everything by default 2 | * 3 | 4 | # Allow directories (required for Git to process directory structure) 5 | !*/ 6 | 7 | # Allow all Python files 8 | !*.py 9 | !pyproject.toml 10 | !*.sh 11 | -------------------------------------------------------------------------------- /.env.example: -------------------------------------------------------------------------------- 1 | # Uncomment if you have an OpenAI API key 2 | #OPENAI_API_KEY=Your OpenAI API key 3 | # as obtained from https://platform.openai.com/account/api-keys 4 | 5 | # Uncomment if you have a Perplexity API key 6 | #PERPLEXITY_API_KEY=Your Perlplexity API key 7 | # as obtained following instrutions from https://docs.perplexity.ai/guides/getting-started 8 | 9 | # Uncomment if you have a Gemini API key 10 | #GEMINI_API_KEY=Your Gemini API key 11 | # as obtained following instrutions from https://docs.perplexity.ai/guides/getting-started 12 | 13 | # Set the ULR to your Ollama server 14 | #OLLAMA_HOST=Your ollama server URL 15 | 16 | # You should have at least one API key enabled 17 | 18 | # Location to save content. 19 | # Make sure the base directory exists, otherswise no data will be stored 20 | OAIWUI_SAVEDIR=./savedir 21 | # recommended directory 22 | # for development: ./savedir 23 | # for container : /iti 24 | 25 | # If a .streamlit/secrets.toml file is present, a UI password will be required to access the WebUI 26 | # The file must contain a valid password in the form 27 | # password = "oaiwui1password4webui!" 28 | 29 | # Only show the GPT tab, or show both GPT and Image 30 | # Authorized value: True or False 31 | OAIWUI_GPT_ONLY=False 32 | 33 | # Models: 34 | # More details at https://github.com/Infotrend-Inc/OpenAI_WebUI 35 | # Recognized values can be seen in the models.md file 36 | # To get a valid list of models, use: python3 ./list_models.py 37 | 38 | # List which GPT models your API keys are authorized to use 39 | # The order below is the order that will be in the model dropdown 40 | # Note that we will not validate content, just match against the provided list. If your model does not appear, check here first 41 | OAIWUI_GPT_MODELS=gpt-4o-mini gpt-4o gpt-4 42 | 43 | # Disable the vision capabilities in the WebUI 44 | # Authorized value: True or False 45 | OAIWUI_GPT_VISION=True 46 | 47 | # List which Images models your API key is authorized to use 48 | # The order of this list will be the order in the dropdown 49 | OAIWUI_IMAGE_MODELS=dall-e-3 50 | 51 | # Default username (leave commented to be prompted -- default: mutli-user mode) 52 | #OAIWUI_USERNAME= 53 | 54 | # Prompt presets directory 55 | #OAIWUI_PROMPT_PRESETS_DIR=./prompt_presets.example 56 | # example of directory location 57 | # for development: ./prompt_presets.example 58 | # for container : /prompt_presets 59 | 60 | 61 | # Prompt presets only: disables the selection of model, tokens and temperature. Only the preset prompts selection is available. 62 | # Requires a valid OAIWUI_PROMPT_PRESETS_DIR and a JSON file with the presets set. 63 | #OAIWUI_PROMPT_PRESETS_ONLY=./prompt_presets_settings-example.json 64 | # example of file location 65 | # for development: ./prompt_presets_settings-example.json 66 | # for container : /prompt_presets.json -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __off 2 | __pycache__ 3 | venv 4 | .env 5 | .env.docker 6 | savedir 7 | savedir.docker 8 | *.svg 9 | .streamlit/secrets.toml 10 | poetry.lock 11 | prompt_presets 12 | uv.lock 13 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | ARG OAIWUI_BASE="ubuntu:24.04" 2 | FROM ${OAIWUI_BASE} 3 | 4 | ENV DEBIAN_FRONTEND=noninteractive 5 | RUN apt-get update -y --fix-missing\ 6 | && apt-get upgrade -y \ 7 | && apt-get install -y --no-install-recommends \ 8 | apt-utils locales wget curl ca-certificates build-essential sudo python3 python3-dev \ 9 | && apt-get clean 10 | 11 | # UTF-8 12 | RUN localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 13 | ENV LANG=en_US.utf8 \ 14 | LC_ALL=C \ 15 | PYTHONUNBUFFERED=1 \ 16 | PYTHONHASHSEED=random \ 17 | PIP_ROOT_USER_ACTION=ignore 18 | 19 | # Setup pip 20 | RUN mv /usr/lib/python3.12/EXTERNALLY-MANAGED /usr/lib/python3.12/EXTERNALLY-MANAGED.old \ 21 | && wget -q -O /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py \ 22 | && python3 /tmp/get-pip.py \ 23 | && pip3 install -U pip \ 24 | && rm -rf /tmp/get-pip.py /root/.cache/pip 25 | 26 | # Every sudo group user does not need a password 27 | RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers 28 | 29 | # Create a new group for the oaiwui user and the oaiwuitoo user 30 | RUN groupadd -g 1024 oaiwui \ 31 | && groupadd -g 1025 oaiwuitoo 32 | 33 | # The oaiwui user will have UID 1024, 34 | # be part of the oaiwui group and be sudo capable (passwordless) 35 | RUN useradd -u 1024 -d /home/oaiwui -g oaiwui -s /bin/bash -m oaiwui \ 36 | && usermod -aG sudo oaiwui 37 | # The oaiwuitoo user will have UID 1025 ... 38 | RUN useradd -u 1025 -d /home/oaiwuitoo -g oaiwuitoo -s /bin/bash -m oaiwuitoo \ 39 | && usermod -aG sudo oaiwuitoo 40 | 41 | # Setup uv as oaiwuitoo 42 | USER oaiwuitoo 43 | 44 | # Install uv 45 | # https://docs.astral.sh/uv/guides/integration/docker/#installing-uv 46 | RUN wget https://astral.sh/uv/install.sh -O /tmp/uv-installer.sh \ 47 | && sh /tmp/uv-installer.sh \ 48 | && rm /tmp/uv-installer.sh 49 | ENV PATH="/home/oaiwuitoo/.local/bin/:$PATH" 50 | 51 | # Verify that python3 and uv are installed 52 | RUN which python3 && python3 --version 53 | RUN which uv && uv --version 54 | 55 | # Get the source code (making sure the directories are owned by oaiwuitoo and the users group shared by oaiwui and oaiwuitoo) 56 | RUN sudo mkdir /app /app/.streamlit /app/assets /iti \ 57 | && sudo chown -R oaiwuitoo:oaiwuitoo /app /iti 58 | 59 | COPY --chown=oaiwuitoo:oaiwuitoo pyproject.toml OAIWUI_WebUI.py common_functions.py common_functions_WebUI.py OAIWUI_Images.py OAIWUI_Images_WebUI.py OAIWUI_GPT.py OAIWUI_GPT_WebUI.py ollama_helper.py models.json /app/ 60 | COPY --chown=oaiwuitoo:oaiwuitoo assets/Infotrend_Logo.png /app/assets/ 61 | 62 | # Sync the project into a new environment 63 | WORKDIR /app 64 | RUN uv sync 65 | # Check that the venv is created with the expected tools 66 | RUN test -d /app/.venv \ 67 | && test -x /app/.venv/bin/python3 \ 68 | && test -x /app/.venv/bin/streamlit 69 | 70 | EXPOSE 8501 71 | 72 | HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health 73 | 74 | # Final copies (as root, done at the end to avoid rebuilding previous steps) 75 | USER root 76 | 77 | COPY --chmod=644 config.sh /oaiwui_config.sh 78 | COPY --chmod=755 entrypoint.sh /entrypoint.sh 79 | 80 | # Run as oaiwuitoo 81 | USER oaiwuitoo 82 | 83 | # The entrypoint will enable us to switch to the oaiwui user and run the application 84 | ENTRYPOINT ["/entrypoint.sh"] 85 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Infotrend, Inc. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | SHELL := /bin/bash 2 | .PHONY: all 3 | 4 | # Base python image base on Debian 5 | OAIWUI_BASE="ubuntu:24.04" 6 | 7 | # Version infomation and container name 8 | OAIWUI_VERSION="0.9.11" 9 | 10 | OAIWUI_CONTAINER_NAME="openai_webui" 11 | 12 | # Default build tag 13 | OAIWUI_BUILD="${OAIWUI_CONTAINER_NAME}:${OAIWUI_VERSION}" 14 | OAIWUI_BUILD_LATEST="${OAIWUI_CONTAINER_NAME}:latest" 15 | 16 | OAIWUI_BUILDX="oaiwui" 17 | 18 | all: 19 | @echo "** Available Docker images to be built (make targets):" 20 | @echo "build: will build the ${OAIWUI_BUILD} image and tag it as latest as well" 21 | @echo "delete: will delete the main latest image" 22 | @echo "buildx_rm: will delete the buildx builder" 23 | @echo "" 24 | @echo "** Run the WebUI (must have uv installed):" 25 | @echo "uv_run: will run the WebUI using uv" 26 | 27 | ##### 28 | uv_run: 29 | uv tool run --with-requirements pyproject.toml streamlit run ./OAIWUI_WebUI.py --server.port=8501 --server.address=0.0.0.0 --server.headless=true --server.fileWatcherType=none --browser.gatherUsageStats=False --logger.level=info 30 | 31 | uv_run_debug: 32 | uv tool run --with-requirements pyproject.toml streamlit run ./OAIWUI_WebUI.py --server.port=8501 --server.address=0.0.0.0 --server.headless=true --browser.gatherUsageStats=False --logger.level=debug 33 | 34 | ##### 35 | 36 | buildx_prep: 37 | @docker buildx ls | grep -q ${OAIWUI_BUILDX} && echo \"builder already exists -- to delete it, use: docker buildx rm ${OAIWUI_BUILDX}\" || docker buildx create --name ${OAIWUI_BUILDX} --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=256000000 38 | @docker buildx use ${OAIWUI_BUILDX} || exit 1 39 | 40 | 41 | build: 42 | @make buildx_prep 43 | @BUILDX_EXPERIMENTAL=1 docker buildx debug --on=error build --progress plain --build-arg OAIWUI_BASE=${OAIWUI_BASE} -t ${OAIWUI_BUILD} --load -f Dockerfile . 44 | @docker tag ${OAIWUI_BUILD} ${OAIWUI_BUILD_LATEST} 45 | 46 | delete: 47 | @docker rmi ${OAIWUI_BUILD_LATEST} 48 | @docker rmi ${OAIWUI_BUILD} 49 | 50 | buildx_rm: 51 | @docker buildx ls | grep -q ${OAIWUI_BUILDX} || echo "builder does not exist" 52 | @echo "** About to delete buildx: ${OAIWUI_BUILDX}" 53 | @echo "Press Ctl+c within 5 seconds to cancel" 54 | @for i in 5 4 3 2 1; do echo -n "$$i "; sleep 1; done; echo "" 55 | @docker buildx rm ${OAIWUI_BUILDX} 56 | ## 57 | 58 | list_models: 59 | @python3 ./list_models.py > models.txt 60 | @python3 ./list_models.py --markdown > models.md 61 | 62 | docker_push: 63 | @echo "Creating docker hub tags -- Press Ctl+c within 5 seconds to cancel -- will only work for maintainers" 64 | @for i in 5 4 3 2 1; do echo -n "$$i "; sleep 1; done; echo "" 65 | @make build 66 | @docker tag ${OAIWUI_BUILD} infotrend/${OAIWUI_BUILD} 67 | @docker tag ${OAIWUI_BUILD_LATEST} infotrend/${OAIWUI_BUILD_LATEST} 68 | @echo "hub.docker.com upload -- Press Ctl+c within 5 seconds to cancel -- will only work for maintainers" 69 | @for i in 5 4 3 2 1; do echo -n "$$i "; sleep 1; done; echo "" 70 | @docker push infotrend/${OAIWUI_BUILD} 71 | @docker push infotrend/${OAIWUI_BUILD_LATEST} 72 | 73 | ## Maintainers: 74 | # - Create a new branch on GitHub that match the expected release tag, pull and checkout that branch 75 | # - update the version number if the following files (ex: "0.9.11"): 76 | # common_functions.py:iti_version="0.9.11" 77 | # Makefile:OAIWUI_VERSION="0.9.11" 78 | # pyproject.toml:version = "0.9.11" 79 | # README.md:Latest version: 0.9.11 80 | # - Local Test 81 | # % make uv_run_debug 82 | # - Build docker image after local testing 83 | # % make build 84 | # - Test in Docker then unraid 85 | # - Upload the images to docker hub 86 | # % make docker_push 87 | # - Generate the models md and txt files 88 | # % make list_models 89 | # - Update the README.md file's version + date + changelog 90 | # - Update the unraid/OpenAI_WebUI.xml file's and sections 91 | # - Commit and push the changes to GitHub (in the branch created at the beginning) 92 | # - On Github, "Open a pull request", 93 | # use the version for the release name 94 | # add PR modifications as a summary of the content of the commits, 95 | # create the PR, add a self-approve message, merge and delete the branch 96 | # - On the build system, checkout main and pull the changes 97 | # % git checkout main 98 | # % git pull 99 | # - Delete the temporary branch 100 | # % git branch -d branch_name 101 | # - Tag the release on GitHub 102 | # % git tag version_id 103 | # % git push origin version_id 104 | # - Create a release on GitHub using the version tag, add the release notes, and publish 105 | # - Delete the created docker builder 106 | # % make buildx_rm 107 | -------------------------------------------------------------------------------- /OAIWUI_GPT.py: -------------------------------------------------------------------------------- 1 | import openai 2 | from openai import OpenAI 3 | 4 | import json 5 | 6 | import os.path 7 | 8 | import copy 9 | 10 | import common_functions as cf 11 | 12 | 13 | ##### 14 | def simpler_gpt_call(apikey, messages, model_engine, base_url:str='', model_provider:str='OpenAI', resp_file:str='', **kwargs): 15 | client = None 16 | if cf.isNotBlank(base_url): 17 | client = OpenAI(api_key=apikey, base_url=base_url) 18 | else: 19 | client = OpenAI(api_key=apikey) 20 | if client is None: 21 | return("Unable to create an OpenAI API Compartible Client handler", "") 22 | 23 | # beta models limitation: https://platform.openai.com/docs/guides/reasoning 24 | # o1 will may not provide an answer if the max_completion_tokens is lower than 2000 25 | 26 | # Generate a response (20231108: Fixed for new API version) 27 | try: 28 | response = client.chat.completions.create( 29 | model=model_engine, 30 | messages = messages, 31 | **kwargs 32 | ) 33 | # using list from venv/lib/python3.11/site-packages/openai/_exceptions.py 34 | except openai.APIConnectionError as e: 35 | return(f"{model_provider} API request failed to connect: {e}", "") 36 | except openai.AuthenticationError as e: 37 | return(f"{model_provider} API request was not authorized: {e}", "") 38 | except openai.RateLimitError as e: 39 | return(f"{model_provider} API request exceeded rate limit: {e}", "") 40 | except openai.APIError as e: 41 | return(f"{model_provider} API returned an API Error: {e}", "") 42 | except openai.OpenAIError as e: 43 | return(f"{model_provider} API request failed: {e}", "") 44 | 45 | response_dict = {} 46 | # Convert response to dict using model_dump() for Pydantic models 47 | try: 48 | response_dict = response.model_dump() 49 | except AttributeError: 50 | # Fallback for objects that don't support model_dump 51 | response_dict = vars(response) 52 | 53 | if cf.isNotBlank(resp_file): 54 | with open(resp_file, 'w') as f: 55 | json.dump(response_dict, f, indent=4) 56 | 57 | response_text = response.choices[0].message.content 58 | 59 | # Add citations if the key is present in the response, irrelevant of the model provider 60 | citations_text = "" 61 | if 'citations' in response_dict: 62 | cf.logit("Found citations", "debug") 63 | citations_text += "\n\nCitations:\n" 64 | for i in range(len(response_dict['citations'])): 65 | citations_text += f"\n[{i+1}] {response_dict['citations'][i]}\n" 66 | response_text += citations_text 67 | 68 | 69 | return "", response_text 70 | 71 | ########## 72 | class OAIWUI_GPT: 73 | def __init__(self, base_save_location, username): 74 | cf.logit("---------- In OAIWUI_GPT __init__ ----------", "debug") 75 | 76 | if cf.isBlank(base_save_location): 77 | base_save_location = "savedir" 78 | if cf.isBlank(username): 79 | username = "test" 80 | 81 | self.apikeys = {} 82 | self.save_location = os.path.join(base_save_location, username, "gpt") 83 | err = cf.make_wdir_recursive(self.save_location) 84 | if cf.isNotBlank(err): 85 | cf.error_exit(err) # nothing else to do here 86 | self.last_runfile = os.path.join(self.save_location, "last_run.json") 87 | 88 | self.models = {} 89 | self.models_status = {} 90 | self.model_help = "" 91 | self.per_model_help = {} 92 | self.gpt_presets = {} 93 | self.gpt_presets_help = "" 94 | self.gpt_roles = {} 95 | self.gpt_roles_help = "" 96 | self.model_capability = {} 97 | 98 | self.beta_models = {} 99 | 100 | self.per_model_provider = {} 101 | self.per_model_url = {} 102 | self.per_model_meta = {} 103 | 104 | self.models_warning = {} 105 | self.known_models = {} 106 | 107 | self.last_dest_dir = None 108 | 109 | ##### 110 | def get_models(self): 111 | return self.models 112 | 113 | def get_models_status(self): 114 | return self.models_status 115 | 116 | def get_model_help(self): 117 | return self.model_help 118 | 119 | def get_model_capability(self): 120 | return self.model_capability 121 | 122 | def get_per_model_help(self): 123 | return self.per_model_help 124 | 125 | def get_per_model_provider(self): 126 | return self.per_model_provider 127 | 128 | def get_gpt_presets(self): 129 | return self.gpt_presets 130 | 131 | def get_gpt_presets_help(self): 132 | return self.gpt_presets_help 133 | 134 | def get_gpt_roles(self): 135 | return self.gpt_roles 136 | 137 | def get_gpt_roles_help(self): 138 | return self.gpt_roles_help 139 | 140 | def get_save_location(self): 141 | return self.save_location 142 | 143 | def get_beta_models(self): 144 | return self.beta_models 145 | 146 | def get_per_model_meta(self): 147 | return self.per_model_meta 148 | 149 | def get_models_warning(self): 150 | return self.models_warning 151 | 152 | def get_known_models(self): 153 | return self.known_models 154 | 155 | def check_apikeys(self, meta): 156 | if 'provider' in meta: 157 | provider = meta["provider"] 158 | else: 159 | return "Missing provider" 160 | 161 | if provider in self.apikeys: 162 | return "" # no need to continue, we have it 163 | 164 | warn, apikey = cf.check_apikeys(provider, meta) 165 | if cf.isNotBlank(warn): 166 | return warn 167 | 168 | self.apikeys[provider] = apikey 169 | return "" 170 | 171 | ##### 172 | def set_parameters(self, models_list, av_models_list): 173 | models = {} 174 | models_status = {} 175 | model_help = "" 176 | 177 | warning = "" 178 | 179 | s_models_list = [] 180 | t_models = models_list.replace(",", " ").split() 181 | for t_model in t_models: 182 | model = t_model.strip() 183 | if model in av_models_list: 184 | if "meta" in av_models_list[model]: 185 | err = self.check_apikeys(av_models_list[model]["meta"]) 186 | if cf.isNotBlank(err): 187 | warning += f"Discarding Model {model}: {err}. " 188 | s_models_list.append(model) 189 | self.per_model_meta[model] = av_models_list[model]["meta"] 190 | else: 191 | warning += f"Discarding Model {model}: Missing the meta information. " 192 | self.models_warning[model] = f"Discarding: Missing the meta information" 193 | else: 194 | warning += f"Unknown Model: {model}. " 195 | self.models_warning[model] = f"Requested, unavailable" 196 | 197 | known_models = list(av_models_list.keys()) 198 | for t_model in s_models_list: 199 | model = t_model.strip() 200 | if model in av_models_list: 201 | if av_models_list[model]["status"] == "deprecated": 202 | warning += f"Model {model} is deprecated (" + av_models_list[model]["status_details"] + "), discarding it. " 203 | self.models_warning[model] = f"deprecated (" + av_models_list[model]["status_details"] + ")" 204 | else: 205 | models[model] = dict(av_models_list[model]) 206 | if cf.isNotBlank(models[model]["status_details"]): 207 | models_status[model] = models[model]["status"] +" (" + models[model]["status_details"] + ")" 208 | else: 209 | warning += f"Unknown model: {model}." 210 | self.models_warning[model] = f"Unknown model" 211 | self.known_models = known_models 212 | 213 | model_help = "" 214 | for key in models: 215 | extra = "" 216 | if 'provider' in models[key]["meta"]: 217 | extra = f"provider: {models[key]['meta']['provider']}, " 218 | self.per_model_provider[key] = models[key]['meta']['provider'] 219 | per_model_help = f"{key} ({extra}" + models[key]["status"] + "):\n" 220 | per_model_help += models[key]["label"] + "\n" 221 | per_model_help += "[Data: " + models[key]["data"] + " | " 222 | per_model_help += "Tokens -- max: " + str(models[key]["max_token"]) + " / " 223 | per_model_help += "context: " + str(models[key]["context_token"]) + "]" 224 | if 'capability' in models[key]: 225 | capabilities = models[key]["capability"] 226 | self.model_capability[key] = capabilities 227 | per_model_help += " | Capability: " + ", ".join(capabilities) 228 | 229 | if 'beta_model' in models[key]["meta"]: 230 | self.beta_models[key] = models[key]['meta']['beta_model'] 231 | else: 232 | self.beta_models[key] = False 233 | 234 | if 'apiurl' in models[key]["meta"]: 235 | self.per_model_url[key] = models[key]['meta']['apiurl'] 236 | 237 | if cf.isNotBlank(models[key]["status_details"]): 238 | per_model_help += " NOTE: " + models[key]["status_details"] 239 | self.per_model_help[key] = per_model_help 240 | model_help += f"{per_model_help}\n\n" 241 | 242 | active_models = [x for x in av_models_list if av_models_list[x]["status"] == "active"] 243 | active_models_txt = ",".join(active_models) 244 | 245 | if len(models) == 0: 246 | return f"No models kept, unable to continue. Active models: {active_models_txt}", warning 247 | 248 | model_help += "For a list of available supported models, see https://github.com/Infotrend-Inc/OpenAI_WebUI/models.md\n\n" 249 | model_help += f"List of active models supported by this release: {active_models_txt}\n\n" 250 | 251 | self.models = models 252 | self.models_status = models_status 253 | self.model_help = model_help 254 | 255 | self.gpt_presets = { 256 | "None": { 257 | "pre": "", 258 | "post": "", 259 | "kwargs": {} 260 | }, 261 | "Keywords": { 262 | "pre": "Extract keywords from this text: ", 263 | "post": "", 264 | "kwargs": {"top_p": 1.0, "frequency_penalty": 0.8, "presence_penalty": 0.0} 265 | }, 266 | "Summarization": { 267 | "pre": "", 268 | "post": "Tl;dr", 269 | "kwargs": {"top_p": 1.0, "frequency_penalty": 0.0, "presence_penalty": 1} 270 | } 271 | } 272 | 273 | self.gpt_presets_help = "None: regular, no additonal parameters\n\nKeywords: Extract keywords from a block of text. At a lower temperature it picks keywords from the text. At a higher temperature it will generate related keywords which can be helpful for creating search indexes.\n\nSummarization: Summarize text." 274 | 275 | self.gpt_roles = { 276 | 'user': 'help instruct the assistant', 277 | 'system': 'helps set the behavior of the assistant (ex: "You are a helpful assistant. You also like to speak in the words of Shakespeare. Incorporate that into your responses.")', 278 | 'assistant': 'helps set the past conversations. This is relevant when you had a chat that went over the maximum number of tokens and need to start a new one: give the chat history some fresh context' 279 | } 280 | 281 | self.gpt_roles_help = "" 282 | for key in self.gpt_roles: 283 | self.gpt_roles_help += key + ":\n" + self.gpt_roles[key] + "\n\n" 284 | 285 | return "", warning 286 | 287 | ##### 288 | def get_rf_role_prompt_response(self, run_file): 289 | role = "" 290 | prompt = "" 291 | response = "" 292 | 293 | run_json = cf.get_run_file(run_file) 294 | if 'role' in run_json: 295 | role = run_json['role'] 296 | if 'prompt' in run_json: 297 | prompt = run_json['prompt'] 298 | if 'response' in run_json: 299 | response = run_json['response'] 300 | 301 | return (role, prompt, response) 302 | 303 | 304 | ##### 305 | def get_dest_dir(self): 306 | return os.path.join(self.save_location, cf.get_timeUTC()) 307 | 308 | 309 | ##### 310 | def check_msg_content(self, msg): 311 | if 'role' not in msg: 312 | return "role not found in message" 313 | if 'content' not in msg: 314 | return "content not found in message" 315 | 316 | return "" 317 | 318 | def chatgpt_it(self, model_engine, chat_messages, max_tokens, temperature, msg_extra=None, websearch_context_size="low", **kwargs): 319 | vision_capable = False 320 | openai_websearch_enabled = False 321 | if model_engine in self.model_capability: 322 | capability = self.model_capability[model_engine] 323 | if 'vision' in capability: 324 | vision_capable = True 325 | if 'websearch' in capability: 326 | provider = self.per_model_provider[model_engine] 327 | if provider == "OpenAI": 328 | openai_websearch_enabled = True 329 | 330 | beta_model = False 331 | if model_engine in self.beta_models: 332 | beta_model = self.beta_models[model_engine] 333 | 334 | last_runfile = self.last_runfile 335 | if cf.isNotBlank(last_runfile): 336 | err = cf.check_file_r(last_runfile) 337 | if cf.isBlank(err): 338 | last_run_json = cf.get_run_file(last_runfile) 339 | if 'last_destdir' in last_run_json: 340 | self.last_dest_dir = last_run_json['last_destdir'] 341 | 342 | dest_dir = self.last_dest_dir 343 | if len(chat_messages) < 2 or cf.isBlank(dest_dir): 344 | dest_dir = self.get_dest_dir() 345 | self.last_dest_dir = dest_dir 346 | 347 | err = cf.make_wdir_recursive(dest_dir) 348 | if cf.isNotBlank(err): 349 | return f"While checking {dest_dir}: {err}", "" 350 | slug = os.path.basename(dest_dir) 351 | 352 | err = cf.check_existing_dir_w(dest_dir) 353 | if cf.isNotBlank(err): 354 | return f"While checking {dest_dir}: {err}", "" 355 | 356 | apikey = self.apikeys[self.per_model_provider[model_engine]] 357 | provider = '' if model_engine not in self.per_model_provider else self.per_model_provider[model_engine] 358 | base_url = '' if model_engine not in self.per_model_url else self.per_model_url[model_engine] 359 | 360 | unformatted_messages = [] 361 | 362 | # load any oaiwui_skip messages from previous chat on disk 363 | if cf.isNotBlank(self.last_runfile): 364 | run_file = "" 365 | if cf.check_file_r(self.last_runfile) == "": 366 | tmp = cf.read_json(self.last_runfile) 367 | if 'last_runfile' in tmp: 368 | run_file = tmp['last_runfile'] 369 | if cf.isNotBlank(run_file): # We can only load previous messages if the file exists 370 | if cf.check_file_r(run_file) == "": 371 | old_run_json = cf.get_run_file(run_file) 372 | if 'messages' in old_run_json: 373 | for msg in old_run_json['messages']: 374 | if 'oaiwui_skip' in msg: 375 | unformatted_messages.append(copy.deepcopy(msg)) 376 | 377 | if msg_extra is not None: 378 | for msg in msg_extra: 379 | unformatted_messages.append(copy.deepcopy(msg)) 380 | 381 | if len(unformatted_messages) == 0: 382 | if 'init_msg' in self.per_model_meta[model_engine]: 383 | init_msg = self.per_model_meta[model_engine]['init_msg'] 384 | init_msg['oaiwui_skip'] = slug 385 | unformatted_messages.append(init_msg) 386 | 387 | for msg in chat_messages: 388 | unformatted_messages.append(copy.deepcopy(msg)) 389 | 390 | clean_messages = [] 391 | for msg in unformatted_messages: 392 | err = self.check_msg_content(msg) 393 | if cf.isNotBlank(err): 394 | return err, "" 395 | 396 | # skip vision messages when the model is not vision-capable 397 | if 'oaiwui_vision' in msg: 398 | if vision_capable is False: 399 | continue # skip this message 400 | clean_messages.append(msg) 401 | continue 402 | 403 | # skip messages with roles that are removed in beta models 404 | if beta_model is True: 405 | if msg['role'] in self.per_model_meta[model_engine]['removed_roles']: 406 | continue 407 | 408 | to_add = { 'role': msg['role'], 'content': [ {'type': 'text', 'text': msg['content']} ] } 409 | if 'msg_format' in self.per_model_meta[model_engine] and self.per_model_meta[model_engine]['msg_format'] == 'role_content': 410 | to_add = { 'role': msg['role'], 'content': msg['content'] } 411 | clean_messages.append(to_add) 412 | 413 | msg_file = f"{dest_dir}/msg.json" 414 | with open(msg_file, 'w') as f: 415 | json.dump(clean_messages, f, indent=4) 416 | 417 | # Use kwargs to hold max_tokens and temperature 418 | if self.beta_models[model_engine] is True: 419 | kwargs['max_completion_tokens'] = max_tokens 420 | elif openai_websearch_enabled is True: 421 | kwargs['response_format'] = { 'type': 'text'} 422 | kwargs['web_search_options'] = { 'search_context_size': websearch_context_size } 423 | else: 424 | kwargs['max_tokens'] = max_tokens 425 | kwargs['temperature'] = temperature 426 | 427 | resp_file = f"{dest_dir}/resp.json" 428 | err, response = simpler_gpt_call(apikey, clean_messages, model_engine, base_url, provider, resp_file, **kwargs) 429 | 430 | if cf.isNotBlank(err): 431 | return err, "" 432 | 433 | unformatted_messages.append({ 'role': 'assistant', 'content': response}) 434 | 435 | run_file = f"{dest_dir}/run.json" 436 | with open(run_file, 'w') as f: 437 | json.dump(unformatted_messages, f, indent=4) 438 | with open(self.last_runfile, 'w') as f: 439 | json.dump({'last_destdir': dest_dir, 'last_runfile': run_file}, f, indent=4) 440 | 441 | return "", run_file 442 | 443 | 444 | ##### 445 | def estimate_tokens(self, txt): 446 | # https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them 447 | word_count = len(txt.split()) 448 | char_count = len(txt) 449 | return max(int(word_count / 0.75), int(char_count / 4.00)) 450 | 451 | 452 | ##### 453 | def get_history(self): 454 | return cf.get_gpt_history(self.save_location) 455 | -------------------------------------------------------------------------------- /OAIWUI_GPT_WebUI.py: -------------------------------------------------------------------------------- 1 | import openai 2 | from openai import OpenAI 3 | 4 | import streamlit as st 5 | import extra_streamlit_components as stx 6 | from streamlit_extras.stoggle import stoggle 7 | 8 | from PIL import Image 9 | import base64 10 | import io 11 | import math 12 | 13 | import json 14 | import copy 15 | 16 | import os.path 17 | import tempfile 18 | 19 | import common_functions as cf 20 | import common_functions_WebUI as cfw 21 | 22 | import OAIWUI_GPT as OAIWUI_GPT 23 | 24 | ########## 25 | class OAIWUI_GPT_WebUI: 26 | def __init__(self, oaiwui_gpt: OAIWUI_GPT, enable_vision: bool = True, prompt_presets_dir: str = None, prompt_presets_file: str = None) -> None: 27 | self.last_gpt_query = "last_gpt_query" 28 | 29 | self.oaiwui_gpt = oaiwui_gpt 30 | self.save_location = oaiwui_gpt.get_save_location() 31 | self.models = oaiwui_gpt.get_models() 32 | self.model_help = oaiwui_gpt.get_model_help() 33 | self.models_status = oaiwui_gpt.get_models_status() 34 | self.model_capability = oaiwui_gpt.get_model_capability() 35 | self.gpt_roles = oaiwui_gpt.get_gpt_roles() 36 | self.gpt_roles_help = oaiwui_gpt.get_gpt_roles_help() 37 | self.gpt_presets = oaiwui_gpt.get_gpt_presets() 38 | self.gpt_presets_help = oaiwui_gpt.get_gpt_presets_help() 39 | 40 | self.per_model_help = oaiwui_gpt.get_per_model_help() 41 | self.beta_models = oaiwui_gpt.get_beta_models() 42 | self.per_model_provider = oaiwui_gpt.get_per_model_provider() 43 | self.per_model_meta = oaiwui_gpt.get_per_model_meta() 44 | 45 | self.enable_vision = enable_vision 46 | 47 | self.prompt_presets_dir = prompt_presets_dir 48 | self.prompt_presets = {} 49 | 50 | self.prompt_presets_file = prompt_presets_file 51 | self.prompt_presets_settings = {} 52 | 53 | self.models_warning = oaiwui_gpt.get_models_warning() 54 | self.known_models = oaiwui_gpt.get_known_models() 55 | 56 | err = self.load_prompt_presets() 57 | if cf.isNotBlank(err): 58 | st.error(err) 59 | cf.error_exit(err) 60 | 61 | 62 | def resize_rectangle(self, original_width, original_height, max_width, max_height): 63 | aspect_ratio = original_width / original_height 64 | max_area = max_width * max_height 65 | original_area = original_width * original_height 66 | 67 | # Calculate scaling factor for proportional fit 68 | scale_factor = math.sqrt(max_area / original_area) 69 | 70 | # Scale the dimensions 71 | new_width = original_width * scale_factor 72 | new_height = original_height * scale_factor 73 | 74 | # Check if resizing would make the rectangle larger 75 | if new_width >= original_width or new_height >= original_height: 76 | return original_width, original_height 77 | 78 | # Adjust if necessary to fit within max dimensions 79 | if new_width > max_width: 80 | new_width = max_width 81 | new_height = new_width / aspect_ratio 82 | 83 | if new_height > max_height: 84 | new_height = max_height 85 | new_width = new_height * aspect_ratio 86 | 87 | return new_width, new_height 88 | 89 | def img_resize_core(self, im, max_x, max_y): 90 | new_x, new_y = self.resize_rectangle(im.size[0], im.size[1], max_x, max_y) 91 | return int(new_x), int(new_y) 92 | 93 | def img_resize(self, tfilen, details_selection): 94 | with Image.open(tfilen) as im: 95 | new_x, new_y = im_x, im_y = im.size[0], im.size[1] 96 | if details_selection == "low": 97 | new_x, new_y = self.img_resize_core(im, 512, 512) 98 | else: 99 | new_x, new_y = self.img_resize_core(im, 2048, 2048) 100 | 101 | if new_x == im_x and new_y == im_y: 102 | return new_x, new_y 103 | 104 | im = im.resize((new_x, new_y)) 105 | im.save(tfilen, format="png") 106 | return new_x, new_y 107 | 108 | 109 | def image_uploader(self, details_selection): 110 | # image uploader: PNG (.png), JPEG (.jpeg and .jpg), WEBP (.webp), and non-animated GIF (.gif). 111 | uploaded_file = st.file_uploader("Upload a PNG/JPEG/WebP image (automatic resize to a value closer to the selected \"details\" selected, see its \"?\")", type=['png','jpg','jpeg','webp']) 112 | if uploaded_file is not None: 113 | placeholder = st.empty() 114 | tfile = tempfile.NamedTemporaryFile(delete=False) 115 | tfilen = str(tfile.name) 116 | with open(tfilen, "wb") as outfile: 117 | outfile.write(uploaded_file.getvalue()) 118 | 119 | # confirm it is a valid PNG, JPEG, WEBP image 120 | im_fmt = None 121 | im_det = None 122 | im_area = None 123 | try: 124 | with Image.open(tfilen) as im: 125 | im_fmt = im.format 126 | im_det = f"{im.size[0]}x{im.size[1]}" 127 | except OSError: 128 | pass 129 | 130 | if im_fmt == "PNG" or im_fmt == "JPEG" or im_fmt == "WEBP": 131 | im_x, im_y = self.img_resize(tfilen, details_selection) 132 | tk_cst = (170 * im_x * im_y) // (512*512) + 85 133 | tk_cst = 1105 if tk_cst > 1105 else tk_cst # following the details of "Calculating costs" 134 | n_img_det = f"{im_x}x{im_y}" 135 | res_txt = f"resized to: {n_img_det} " if n_img_det != im_det else "" 136 | tkn_txt = f"(est. token cost -- \"high\": {tk_cst} | \"low\": 85)" 137 | placeholder.info(f"Uploaded image: {im_fmt} orig size: {im_det} {res_txt} {tkn_txt}") 138 | return tfilen 139 | else: 140 | placeholder.error(f"Uploaded image ({im_fmt}) is not a valid/supported PNG, JPEG, or WEBP image") 141 | return None 142 | 143 | return None 144 | 145 | ##### 146 | 147 | def load_prompt_presets(self): 148 | if self.prompt_presets_dir is None: 149 | return "" 150 | 151 | prompt_presets = {} 152 | for file in os.listdir(self.prompt_presets_dir): 153 | if file.endswith(".json"): 154 | err = cf.check_file_r(os.path.join(self.prompt_presets_dir, file)) 155 | if cf.isNotBlank(err): 156 | return err 157 | with open(os.path.join(self.prompt_presets_dir, file), "r") as f: 158 | prompt_presets[file.split(".json")[0]] = json.load(f) 159 | 160 | self.prompt_presets = prompt_presets 161 | 162 | if self.prompt_presets_file is not None: 163 | err = cf.check_file_r(self.prompt_presets_file) 164 | if cf.isNotBlank(err): 165 | return err 166 | with open(self.prompt_presets_file, "r") as f: 167 | self.prompt_presets_settings = json.load(f) 168 | if 'model' not in self.prompt_presets_settings: 169 | return f"Could not find 'model' in {self.prompt_presets_file}" 170 | model = self.prompt_presets_settings['model'] 171 | if model not in self.models: 172 | return f"Could not find requested 'model' ({model}) in available models: {list(self.models.keys())} (from {self.prompt_presets_file})" 173 | if 'tokens' not in self.prompt_presets_settings: 174 | return f"Could not find 'tokens' in {self.prompt_presets_file}" 175 | tmp = self.prompt_presets_settings['tokens'] 176 | if tmp is None: 177 | return f"Invalid 'tokens' ({tmp}) in {self.prompt_presets_file}" 178 | if tmp <= 0: 179 | return f"Invalid 'tokens' ({tmp}) in {self.prompt_presets_file}" 180 | if tmp > self.models[model]['max_token']: 181 | return f"Requested 'tokens' ({tmp}) is greater than model's 'max_token' ({self.models[model]['max_token']}) in {self.prompt_presets_file}" 182 | if 'temperature' not in self.prompt_presets_settings: 183 | return f"Could not find 'temperature' in {self.prompt_presets_file}" 184 | tmp = self.prompt_presets_settings['temperature'] 185 | if tmp is None: 186 | return f"Invalid 'temperature' ({tmp}) in {self.prompt_presets_file}" 187 | if tmp < 0: 188 | return f"Invalid 'temperature' ({tmp}) in {self.prompt_presets_file}" 189 | if tmp > 1: 190 | return f"Invalid 'temperature' ({tmp}) in {self.prompt_presets_file}" 191 | 192 | return "" 193 | 194 | 195 | ##### 196 | def set_ui(self): 197 | st.sidebar.empty() 198 | vision_capable = False 199 | websearch_capable = False 200 | vision_mode = False 201 | disable_preset_prompts = False 202 | prompt_preset = None 203 | msg_extra = None 204 | beta_model = False 205 | 206 | temperature_selector = True 207 | tokens_selector = True 208 | max_tokens_selector = True 209 | role_selector = True 210 | preset_selector = False 211 | prompt_preset_selector = True 212 | 213 | # models used since last clear chat 214 | if 'model_used' not in st.session_state: 215 | st.session_state.model_used = [] 216 | 217 | base_model_list = list(self.models.keys()) 218 | model_list = [] 219 | for model_name in base_model_list: 220 | model_provider = self.per_model_provider[model_name] if model_name in self.per_model_provider else "Unknown" 221 | model_list.append(f"{model_name} ({model_provider})") 222 | 223 | if 'gpt_messages' not in st.session_state: 224 | st.session_state.gpt_messages = [] 225 | else: 226 | disable_preset_prompts = False 227 | 228 | with st.sidebar: 229 | st.text("Check the various ? for help", help=f"[Run Details]\n\nRunID: {cfw.get_runid()}\n\nSave location: {self.save_location}\n\nUTC time: {cf.get_timeUTC()}\n") 230 | 231 | if st.button("Clear Chat"): 232 | st.session_state.gpt_messages = [] 233 | disable_preset_prompts = False 234 | if self.last_gpt_query in st.session_state: 235 | del st.session_state[self.last_gpt_query] 236 | if 'gpt_msg_extra' in st.session_state: 237 | del st.session_state['gpt_msg_extra'] # only a clear will allow us to set msg_extra again 238 | st.session_state.model_used = [] 239 | 240 | # create a location placeholder for the prompt preset selector 241 | st_preset_placeholder = st.empty() 242 | 243 | if self.prompt_presets_settings == {}: 244 | # Only available if not in "preset only" mode 245 | tmp_model_name = st.selectbox("model", options=model_list, index=0, key="model_name", help=self.model_help) 246 | model_name = tmp_model_name.split(" ")[0] 247 | if model_name in self.models_status: 248 | st.info(f"{model_name}: {self.models_status[model_name]}") 249 | if "vision" in self.model_capability[model_name]: 250 | vision_capable = True 251 | if "websearch" in self.model_capability[model_name]: 252 | websearch_capable = True 253 | 254 | roles_toremove = [] 255 | if model_name in self.per_model_meta and "removed_roles" in self.per_model_meta[model_name]: 256 | roles_toremove = self.per_model_meta[model_name]["removed_roles"] 257 | 258 | if model_name in self.per_model_meta and "disabled_features" in self.per_model_meta[model_name]: 259 | disabled_features = self.per_model_meta[model_name]["disabled_features"] 260 | if "prompt_preset" in disabled_features: 261 | prompt_preset_selector = False 262 | if "preset" in disabled_features: 263 | preset_selector = False 264 | if "role" in disabled_features: 265 | role_selector = False 266 | if "temperature" in disabled_features: 267 | temperature_selector = False 268 | if "max_tokens" in disabled_features: 269 | max_tokens_selector = False 270 | if "tokens" in disabled_features: 271 | tokens_selector = False 272 | 273 | if model_name in self.beta_models and self.beta_models[model_name] is True: 274 | beta_model = True 275 | 276 | m_token = self.models[model_name]['max_token'] 277 | 278 | # vision mode bypass 279 | if self.enable_vision is False: 280 | vision_mode = False 281 | vision_capable = False 282 | 283 | if vision_capable: 284 | vision_mode = st.toggle(label="Vision", value=False, help="Enable the upload of an image. Vision's limitation and cost can be found at https://platform.openai.com/docs/guides/vision/limitations.\n\nDisables the role and presets selectors. Image(s) are resized when over the max of the \'details\' selected. Please be aware that each 512px x 512px title is expected to cost 170 tokens. Using this mode disables roles, presets and chat (the next prompt will not have knowledge of past thread of conversation)") 285 | 286 | if vision_mode: 287 | vision_details = st.selectbox("Vision Details", options=["auto", "low", "high"], index=0, key="vision_details", help="The model will use the auto setting which will look at the image input size and decide if it should use the low or high setting.\n\n- low: the model will receive a low-res 512px x 512px version of the image, and represent the image with a budget of 85 tokens. This allows the API to return faster responses and consume fewer input tokens for use cases that do not require high detail.\n\n- high will first allows the model to first see the low res image (using 85 tokens) and then creates detailed crops using 170 tokens for each 512px x 512px tile.\n\n\n\nImage inputs are metered and charged in tokens, just as text inputs are. The token cost of a given image is determined by two factors: its size, and the detail option on each image_url block. All images with detail: low cost 85 tokens each. detail: high images are first scaled to fit within a 2048 x 2048 square, maintaining their aspect ratio. Then, they are scaled such that the shortest side of the image is 768px long. Finally, a count of how many 512px squares the image consists of is performed. Each of those squares costs 170 tokens. Another 85 tokens are always added to the final total. More details at https://platform.openai.com/docs/guides/vision/calculating-costs") 288 | role_selector = False 289 | preset_selector = False 290 | 291 | img_file = None 292 | if vision_mode: 293 | img_file = self.image_uploader(vision_details) 294 | img_type = "png" # convert everything to PNG for processing 295 | if img_file is not None: 296 | img_b64 = None 297 | img_bytes = io.BytesIO() 298 | with Image.open(img_file) as image: 299 | image.save(img_bytes, format=img_type) 300 | img_b64 = base64.b64encode(img_bytes.getvalue()).decode('utf-8') 301 | if img_b64 is not None: 302 | img_str = f"data:image/{img_type};base64,{img_b64}" 303 | msg_extra = [ 304 | { 305 | "role": "user", 306 | "content": [ 307 | { 308 | "type": "image_url", 309 | "image_url": { 310 | "url": img_str, 311 | "details": vision_details 312 | } 313 | } 314 | ], 315 | "oaiwui_skip": True, 316 | "oaiwui_vision": True 317 | } 318 | ] 319 | if os.path.exists(img_file): 320 | os.remove(img_file) 321 | 322 | role = list(self.gpt_roles.keys())[0] 323 | if role_selector is True: 324 | tmp_roles = self.gpt_roles.copy() 325 | for r in roles_toremove: 326 | tmp_roles.pop(r) 327 | tmp_txt = "" if beta_model is False else "\n\nBeta models do not support the 'system' role" 328 | role = st.selectbox("Role", options=tmp_roles, index=0, key="input_role", help = "Role of the input text\n\n" + self.gpt_roles_help + tmp_txt) 329 | 330 | max_tokens = 1000 331 | if max_tokens_selector is True: 332 | if beta_model is False: 333 | max_tokens = st.slider('max_tokens', 0, m_token, 1000, 100, "%i", "max_tokens", "The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model\'s context length.") 334 | else: 335 | max_tokens = st.slider('max_completion_tokens', 0, m_token, 4000, 100, "%i", "max_completion_tokens", "For beta models: control the total number of tokens generated by the model, including both reasoning and visible completion tokens. If the token value is too low, the answer might be empty") 336 | 337 | temperature = 0.5 338 | if temperature_selector is True: 339 | temperature = st.slider('temperature', 0.0, 1.0, 0.5, 0.01, "%0.2f", "temperature", "The temperature of the model. Higher temperature results in more surprising text.") 340 | 341 | websearch = "low" 342 | if websearch_capable is True: 343 | websearch = st.selectbox('Search Context Size', options=["low", "medium", "high"], index=0, help="Controls how much context is retrieved from the web to help the tool formulate a response.", key="websearch") 344 | 345 | if preset_selector is True: 346 | presets = st.selectbox("GPT Task", options=list(self.gpt_presets.keys()), index=0, key="presets", help=self.gpt_presets_help) 347 | else: 348 | presets = list(self.gpt_presets.keys())[0] 349 | 350 | else: # "preset only" mode 351 | model_name = self.prompt_presets_settings['model'] 352 | max_tokens = self.prompt_presets_settings['tokens'] 353 | temperature = self.prompt_presets_settings['temperature'] 354 | presets = list(self.gpt_presets.keys())[0] 355 | role = list(self.gpt_roles.keys())[0] 356 | 357 | model_provider = self.per_model_provider[model_name] if model_name in self.per_model_provider else "Unknown" 358 | 359 | # use the location of the placeholder now that we have the vision settings 360 | if prompt_preset_selector is True: 361 | if self.prompt_presets_dir is not None: 362 | prompt_preset = st_preset_placeholder.selectbox("Prompt preset", options=list(self.prompt_presets.keys()), index=None, key="prompt_preset", help="Load a prompt preset. Can only be used with new chats.", disabled=disable_preset_prompts) 363 | if prompt_preset is not None: 364 | if prompt_preset not in self.prompt_presets: 365 | st_preset_placeholder.warning(f"Unkown {prompt_preset}") 366 | else: 367 | if 'messages' in self.prompt_presets[prompt_preset]: 368 | if 'gpt_msg_extra' not in st.session_state: 369 | msg_extra = self.prompt_presets[prompt_preset]["messages"] 370 | st.session_state['gpt_msg_extra'] = msg_extra 371 | # clear the chat history in the GPT call as well 372 | else: 373 | if 'gpt_msg_extra' in st.session_state: 374 | del st.session_state['gpt_msg_extra'] 375 | 376 | gpt_show_history = st.toggle(label='Show Prompt History', value=False, help="Show a list of prompts that you have used in the past (most recent first). Loading a selected prompt does not load the parameters used for the generation.", key="gpt_show_history") 377 | if gpt_show_history: 378 | gpt_allow_history_deletion = st.toggle('Allow Prompt History Deletion', value=False, help="This will allow you to delete a prompt from the history. This will delete the prompt and all its associated files. This cannot be undone.", key="gpt_allow_history_deletion") 379 | 380 | download_placeholder = st.empty() 381 | 382 | prompt_value=f"Model: {model_name} ({model_provider}) " 383 | prompt_value += f"[role: {role} " 384 | if tokens_selector is True: 385 | prompt_value += f"| max_tokens: {max_tokens} " 386 | if temperature_selector is True: 387 | prompt_value += f"| temperature: {temperature}" 388 | if vision_mode: 389 | prompt_value += f" | vision details: {vision_details}" 390 | if preset_selector is True: 391 | prompt_value += f" | preset: {presets}" 392 | if prompt_preset_selector is True: 393 | if prompt_preset is not None: 394 | prompt_value += f" | prompt preset: {prompt_preset}" 395 | if websearch_capable: 396 | prompt_value += f" | websearch: {websearch}" 397 | prompt_value += f" ]" 398 | st.text(prompt_value, help=f'GPT provides a simple but powerful interface to any models. You input some text as a prompt, and the model will generate a text completion that attempts to match whatever context or pattern you gave it:\n\n - The tool works on text to: answer questions, provide definitions, translate, summarize, and analyze sentiments.\n\n- Keep your prompts clear and specific. The tool works best when it has a clear understanding of what you\'re asking it, so try to avoid vague or open-ended prompts.\n\n- Use complete sentences and provide context or background information as needed.\n\n- Some presets are available in the sidebar, check their details for more information.\n\nA few example prompts (to use with "None" preset):\n\n- Create a list of 8 questions for a data science interview\n\n- Generate an outline for a blog post on MFT\n\n- Translate "bonjour comment allez vous" in 1. English 2. German 3. Japanese\n\n- write python code to display with an image selector from a local directory using OpenCV\n\n- Write a creative ad and find a name for a container to run machine learning and computer vision algorithms by providing access to many common ML frameworks\n\n- some models support "Chat" conversations. If you see the "Clear Chat" button, this will be one such model. They also support different max tokens, so adapt accordingly. The "Clear Chat" is here to allow you to start a new "Chat". Chat models can be given writing styles using the "system" "role"\n\nMore examples and hints can be found at https://platform.openai.com/examples') 399 | 400 | # Check for model warnings 401 | if list(self.models_warning.keys()) != []: 402 | warning_text = " - " +"\n - ".join ([f"{model}: {self.models_warning[model]}" for model in sorted(self.models_warning.keys())]) 403 | warning_text += f"\n\n\nKnown models: {self.known_models}" 404 | st.text("⚠️ Models warnings", help=f"{warning_text}") 405 | 406 | # Main window 407 | 408 | if gpt_show_history: 409 | err, hist = self.oaiwui_gpt.get_history() 410 | if cf.isNotBlank(err): 411 | st.error(err) 412 | cf.error_exit(err) 413 | if len(hist) == 0: 414 | st.warning("No prompt history found") 415 | else: 416 | cfw.show_gpt_history(hist, gpt_allow_history_deletion) 417 | 418 | for message in st.session_state.gpt_messages: 419 | with st.chat_message(message['role']): 420 | st.markdown(message["content"]) 421 | 422 | if prompt := st.chat_input("Enter your prompt"): 423 | # Display user message in chat message container 424 | with st.chat_message(role): 425 | st.markdown(prompt) 426 | 427 | if cf.isBlank(prompt) or len(prompt) < 5: 428 | st.error("Please provide a prompt of at least 5 characters before requesting an answer", icon="✋") 429 | return () 430 | 431 | prompt = self.gpt_presets[presets]["pre"] + prompt + self.gpt_presets[presets]["post"] 432 | 433 | prompt_token_count = self.oaiwui_gpt.estimate_tokens(prompt) 434 | requested_token_count = prompt_token_count + max_tokens 435 | if requested_token_count > self.models[model_name]["context_token"]: 436 | st.warning("You requested an estimated %i tokens, which might exceed the model's context window of %i tokens. We are still proceeding with the request, but an error return is possible." % (requested_token_count, self.models[model_name]["context_token"])) 437 | 438 | if max_tokens > 0: 439 | tmp_txt1 = "" if tokens_selector is False else f" for max_tokens: {max_tokens}" 440 | tmp_txt2 = "" if temperature_selector is False else f" (temperature: {temperature})" 441 | st.toast(f"Requesting {model_provider} with model: {model_name}{tmp_txt1}{tmp_txt2}") 442 | with st.spinner(f"Asking {model_provider} with model: {model_name} {tmp_txt1}{tmp_txt2}. Prompt est. tokens : {prompt_token_count}"): 443 | if msg_extra is None: 444 | if 'gpt_msg_extra' in st.session_state: 445 | msg_extra = st.session_state['gpt_msg_extra'] 446 | else: 447 | if 'gpt_msg_extra' in st.session_state: 448 | tmp = msg_extra 449 | msg_extra = copy.deepcopy(st.session_state['gpt_msg_extra']) 450 | msg_extra.append(tmp[0]) 451 | 452 | st.session_state.gpt_messages.append({"role": role, "content": prompt}) 453 | 454 | # only add the model to the list if it is different from the last one 455 | if len(st.session_state.model_used) == 0 or st.session_state.model_used[-1] != model_name: 456 | st.session_state.model_used.append(model_name) 457 | 458 | err, run_file = self.oaiwui_gpt.chatgpt_it(model_name, st.session_state.gpt_messages, max_tokens, temperature, msg_extra, websearch, **self.gpt_presets[presets]["kwargs"]) 459 | if cf.isNotBlank(err): 460 | st.error(err) 461 | if cf.isNotBlank(run_file): 462 | st.session_state[self.last_gpt_query] = run_file 463 | st.toast("Done") 464 | 465 | if len(st.session_state.model_used) > 0: 466 | cf.logit(f"ℹ️ Models prompted: {', '.join(st.session_state.model_used)}", "info") 467 | 468 | if self.last_gpt_query in st.session_state: 469 | run_file = st.session_state[self.last_gpt_query] 470 | run_json = cf.get_run_file(run_file) 471 | 472 | response = run_json[-1]["content"] 473 | st.chat_message("assistant").write(response) 474 | st.session_state.gpt_messages.append({"role": "assistant", "content": response}) 475 | del st.session_state[self.last_gpt_query] 476 | 477 | if download_placeholder is not None: 478 | chat_text = "" 479 | for msg in st.session_state.gpt_messages: 480 | chat_text += msg["role"] + ": " + msg["content"] + "\n" 481 | download_placeholder.download_button(label="Download Chat", data=chat_text) 482 | -------------------------------------------------------------------------------- /OAIWUI_Images.py: -------------------------------------------------------------------------------- 1 | import openai 2 | from openai import OpenAI 3 | 4 | import json 5 | 6 | import requests 7 | 8 | import os.path 9 | import pathlib 10 | import base64 11 | 12 | import common_functions as cf 13 | 14 | from datetime import datetime 15 | 16 | 17 | ###### 18 | # https://github.com/openai/openai-openapi/blob/master/openapi.yaml 19 | def images_call(apikey, model, prompt, img_size, img_count, resp_file:str='', **kwargs): 20 | client = OpenAI(api_key=apikey) 21 | 22 | # Generate a response 23 | try: 24 | response = client.images.generate( 25 | model=model, 26 | prompt=prompt, 27 | size=img_size, 28 | n=img_count, 29 | **kwargs 30 | ) 31 | # using list from venv/lib/python3.11/site-packages/openai/_exceptions.py 32 | except openai.APIConnectionError as e: 33 | return(f"OpenAI API request failed to connect: {e}", "") 34 | except openai.AuthenticationError as e: 35 | return(f"OpenAI API request was not authorized: {e}", "") 36 | except openai.RateLimitError as e: 37 | return(f"OpenAI API request exceeded rate limit: {e}", "") 38 | except openai.APIError as e: 39 | return(f"OpenAI API returned an API Error: {e}", "") 40 | except openai.OpenAIError as e: 41 | return(f"OpenAI API request failed: {e}", "") 42 | 43 | response_dict = {} 44 | # Convert response to dict using model_dump() for Pydantic models 45 | try: 46 | response_dict = response.model_dump() 47 | except AttributeError: 48 | # Fallback for objects that don't support model_dump 49 | response_dict = vars(response) 50 | 51 | if cf.isNotBlank(resp_file): 52 | with open(resp_file, 'w') as f: 53 | json.dump(response_dict, f, indent=4) 54 | 55 | return "", response 56 | 57 | 58 | ########## 59 | class OAIWUI_Images: 60 | def __init__(self, base_save_location, username): 61 | cf.logit("---------- In OAIWUI_Images __init__ ----------", "debug") 62 | 63 | self.last_images_query = 'last_images_query' 64 | 65 | self.apikeys = {} 66 | self.save_location = os.path.join(base_save_location, username) 67 | err = cf.make_wdir_recursive(self.save_location) 68 | if cf.isNotBlank(err): 69 | cf.error_exit(err) # nothing else to do here 70 | self.models = {} 71 | self.models_status = {} 72 | self.model_help = "" 73 | self.per_model_help = {} 74 | 75 | self.images_modes = { 76 | "Image": "The image generations endpoint allows you to create an original image given a text prompt. Generated images and maximum number of requested images depends on the model selected. Smaller sizes are faster to generate." 77 | } 78 | self.images_help = "" 79 | for key in self.images_modes: 80 | self.images_help += key + ":\n" 81 | self.images_help += self.images_modes[key] + "\n" 82 | 83 | self.per_model_provider = {} 84 | self.models_warning = {} 85 | self.known_models = {} 86 | 87 | 88 | ##### 89 | def check_apikeys(self, model, meta): 90 | if 'provider' in meta: 91 | provider = meta["provider"] 92 | else: 93 | return "Missing provider" 94 | 95 | if provider in self.apikeys: 96 | return "" # no need to continue, we have it 97 | 98 | warn, apikey = cf.check_apikeys(provider, meta) 99 | if cf.isNotBlank(warn): 100 | return warn 101 | 102 | self.apikeys[provider] = apikey 103 | return "" 104 | 105 | ##### 106 | def set_parameters(self, models_list, av_models_list): 107 | models = {} 108 | models_status = {} 109 | model_help = "" 110 | 111 | warning = "" 112 | 113 | t_models_list = models_list.replace(",", " ").split() 114 | known_models = list(av_models_list.keys()) 115 | for t_model in t_models_list: 116 | model = t_model.strip() 117 | if model in av_models_list: 118 | if "meta" in av_models_list[model]: 119 | err = self.check_apikeys(model, av_models_list[model]["meta"]) 120 | if cf.isNotBlank(err): 121 | warning += f"Discarding Model {model}: {err}. " 122 | self.models_warning[model] = f"Discarding: {err}" 123 | continue 124 | if 'provider' in av_models_list[model]["meta"]: 125 | self.per_model_provider[model] = av_models_list[model]["meta"]["provider"] 126 | else: 127 | warning += f"Discarding Model {model}: Missing the provider information. " 128 | self.models_warning[model] = f"Discarding: Missing the provider information" 129 | continue 130 | else: 131 | warning += f"Discarding Model {model}: Missing the meta information. " 132 | self.models_warning[model] = f"Discarding: Missing the meta information" 133 | continue 134 | 135 | if av_models_list[model]["status"] == "deprecated": 136 | warning += f"Model [{model}] is deprecated (" + av_models_list[model]['status_details'] + "), discarding it" 137 | self.models_warning[model] = f"deprecated (" + av_models_list[model]['status_details'] + ")" 138 | else: 139 | models[model] = dict(av_models_list[model]) 140 | if cf.isNotBlank(models[model]["status_details"]): 141 | models_status[model] = av_models_list[model]["status"] + " (" + av_models_list[model]["status_details"] + ")" 142 | else: 143 | warning += f"Unknown model: [{model}] | Known models: {known_models}" 144 | self.models_warning[model] = f"Unknown model" 145 | 146 | self.known_models = list(av_models_list.keys()) 147 | 148 | model_help = "" 149 | for key in models: 150 | per_model_help = f"{key} (" + av_models_list[key]["status"] + "):\n" 151 | per_model_help += av_models_list[key]["label"] + "\n" 152 | per_model_help += "image_size: " + str(av_models_list[key]["image_size"]) 153 | if cf.isNotBlank(models[key]["status_details"]): 154 | per_model_help += " NOTE: " + models[key]["status_details"] 155 | self.per_model_help[key] = per_model_help 156 | model_help += f"{per_model_help}\n\n" 157 | 158 | active_models = [x for x in av_models_list if av_models_list[x]["status"] == "active"] 159 | active_models_txt = ",".join(active_models) 160 | 161 | if len(models) == 0: 162 | return f"No models kept, unable to continue. Active models: {active_models_txt}", warning 163 | 164 | model_help += "For a list of available supported models, see https://github.com/Infotrend-Inc/OpenAI_WebUI/models.md\n\n" 165 | model_help += f"List of active models supported by this release: {active_models_txt}\n\n" 166 | 167 | self.models = models 168 | self.models_status = models_status 169 | self.model_help = model_help 170 | 171 | return "", warning 172 | 173 | ##### 174 | def get_dest_dir(self): 175 | return os.path.join(self.save_location, "images", cf.get_timeUTC()) 176 | 177 | def get_models(self): 178 | return self.models 179 | 180 | def get_models_status(self): 181 | return self.models_status 182 | 183 | def get_model_help(self): 184 | return self.model_help 185 | 186 | def get_per_model_help(self): 187 | return self.per_model_help 188 | 189 | def get_images_modes(self): 190 | return self.images_modes 191 | 192 | def get_save_location(self): 193 | return self.save_location 194 | 195 | def get_models_warning(self): 196 | return self.models_warning 197 | 198 | def get_known_models(self): 199 | return self.known_models 200 | 201 | ##### 202 | def images_it(self, model, prompt, img_size, img_count, dest_dir, **kwargs): 203 | err = cf.make_wdir_recursive(dest_dir) 204 | err = cf.check_existing_dir_w(dest_dir) 205 | if cf.isNotBlank(err): 206 | return f"While checking {dest_dir}: {err}", "" 207 | resp_file = f"{dest_dir}/resp.json" 208 | 209 | warn = "" 210 | err, response = images_call(self.apikeys[self.per_model_provider[model]], model, prompt, img_size, img_count, resp_file, **kwargs) 211 | if cf.isNotBlank(err): 212 | return err, warn, "" 213 | 214 | all_images = [] 215 | for i in range(img_count): 216 | image_name = f"{dest_dir}/{i + 1}.png" 217 | 218 | cf.logit(f"Downloading result {i + 1} as {image_name}", "debug") 219 | image_url = response.data[i].url 220 | image_b64 = response.data[i].b64_json 221 | img_bytes = None 222 | 223 | if image_url is not None: 224 | img_bytes = requests.get(image_url).content 225 | elif image_b64 is not None: 226 | img_bytes = base64.b64decode(image_b64) 227 | 228 | if img_bytes is not None: 229 | with open(image_name, 'wb') as handler: 230 | handler.write(img_bytes) 231 | all_images.append(image_name) 232 | else: 233 | warn += f"Unable to download image {i + 1}\n" 234 | cf.logit(f"Unable to download image", "warning") 235 | 236 | if len(all_images) == 0: 237 | return "No images generated", warn, "" 238 | 239 | run_file = f"{dest_dir}/run.json" 240 | run_json = { 241 | "prompt": prompt, 242 | "images": all_images, 243 | } 244 | with open(run_file, 'w') as f: 245 | json.dump(run_json, f, indent=4) 246 | 247 | return "", warn, run_file 248 | 249 | ##### 250 | def get_history(self): 251 | search_dir = os.path.join(self.save_location, "images") 252 | return cf.get_history(search_dir) 253 | -------------------------------------------------------------------------------- /OAIWUI_Images_WebUI.py: -------------------------------------------------------------------------------- 1 | import openai 2 | from openai import OpenAI 3 | 4 | import streamlit as st 5 | import extra_streamlit_components as stx 6 | from streamlit_extras.stoggle import stoggle 7 | from streamlit_image_select import image_select 8 | 9 | import json 10 | 11 | import requests 12 | 13 | import os.path 14 | import pathlib 15 | 16 | import common_functions as cf 17 | import common_functions_WebUI as cfw 18 | 19 | from datetime import datetime 20 | 21 | import OAIWUI_Images as OAIWUI_Images 22 | 23 | ########## 24 | class OAIWUI_Images_WebUI: 25 | def __init__(self, oaiwui_images: OAIWUI_Images) -> None: 26 | self.last_images_query = "last_images_query" 27 | 28 | self.oaiwui_images = oaiwui_images 29 | self.save_location = oaiwui_images.get_save_location() 30 | self.models = oaiwui_images.get_models() 31 | self.model_help = oaiwui_images.get_model_help() 32 | self.models_status = oaiwui_images.get_models_status() 33 | self.per_model_help = oaiwui_images.get_per_model_help() 34 | self.images_modes = oaiwui_images.get_images_modes() 35 | 36 | self.models_warning = oaiwui_images.get_models_warning() 37 | self.known_models = oaiwui_images.get_known_models() 38 | 39 | self.last_images_query = oaiwui_images.last_images_query 40 | 41 | 42 | ##### 43 | def get_dest_dir(self): 44 | return self.oaiwui_images.get_dest_dir() 45 | 46 | ##### 47 | def display_images(self, prompt, all_images): 48 | img = image_select("Prompt: " + prompt, all_images, use_container_width=False) 49 | st.image(img) 50 | path = pathlib.PurePath(img) 51 | wdir = path.parent.name 52 | wfile = path.name 53 | dfile = f"{wdir}-{wfile}" 54 | st.download_button("Download Selected", data=open(img, 'rb').read(), file_name=dfile, mime="image/png", key="images_download_button") 55 | 56 | ##### 57 | def set_ui(self): 58 | st.sidebar.empty() 59 | with st.sidebar: 60 | st.text("Check the various ? for help", help=f"[Run Details]\n\nRunID: {cfw.get_runid()}\n\nSave location: {self.save_location}\n\nUTC time: {cf.get_timeUTC()}\n") 61 | mode = list(self.images_modes.keys())[0] 62 | if len(self.images_modes.keys()) > 1: 63 | mode = st.selectbox("mode", options=list(self.images_modes.keys()), index=0, key="images_mode", help=self.images_help) 64 | model = st.selectbox("model", options=list(self.models.keys()), index=0, key="model", help=self.model_help) 65 | if model in self.models_status: 66 | st.info(f"{model}: {self.models_status[model]}") 67 | model_image_size = self.models[model]["image_size"] 68 | img_size = st.selectbox("image size", options=model_image_size, index=0, key="images_image_size", 69 | help="Smaller sizes are faster to generate.") 70 | 71 | if model == "dall-e-2": 72 | img_count = st.number_input("number of images", min_value=1, max_value=10, value=1, step=1, key="images_img_count", 73 | help="Number of images to generate.") 74 | else: 75 | img_count = 1 76 | 77 | kwargs = {} 78 | quality_options = [] 79 | if 'quality' in self.models[model]['meta']: 80 | quality_options = self.models[model]['meta']['quality'] 81 | style_options = [] 82 | if 'style' in self.models[model]['meta']: 83 | style_options = self.models[model]['meta']['style'] 84 | 85 | if quality_options: 86 | quality = st.selectbox("quality", options=quality_options, index=0, key="images_quality", help="The quality of the image that will be generated. hd creates images with finer details and greater consistency across the image.") 87 | kwargs['quality'] = quality 88 | if style_options: 89 | style = st.selectbox("style", options=style_options, index=0, key="images_style", help="The style of the generated images. Vivid causes the model to lean towards generating hyper-real and dramatic images. Natural causes the model to produce more natural, less hyper-real looking images.") 90 | kwargs['style'] = style 91 | 92 | if 'transparent' in self.models[model]['meta']: 93 | transparent = st.toggle("transparent", value=False, key="images_transparent", help="If enabled, the image will be generated with a transparent background.") 94 | if transparent: 95 | kwargs['background'] = "transparent" 96 | 97 | images_show_history = st.toggle(label='Show Prompt History', value=False, help="Show a list of prompts that you have used in the past (most recent first). Loading a selected prompt does not load the parameters used for the generation.", key="images_show_history") 98 | if images_show_history: 99 | images_allow_history_deletion = st.toggle('Allow Prompt History Deletion', value=False, help="This will allow you to delete a prompt from the history. This will delete the prompt and all its associated files. This cannot be undone.", key="images_allow_history_deletion") 100 | 101 | # Check for model warnings 102 | if list(self.models_warning.keys()) != []: 103 | warning_text = " - " +"\n - ".join ([f"{model}: {self.models_warning[model]}" for model in sorted(self.models_warning.keys())]) 104 | warning_text += f"\n\n\nKnown models: {self.known_models}" 105 | st.text("⚠️ Models warnings", help=f"{warning_text}") 106 | 107 | if images_show_history: 108 | err, hist = self.oaiwui_images.get_history() 109 | if cf.isNotBlank(err): 110 | st.error(err) 111 | cf.error_exit(err) 112 | if len(hist) == 0: 113 | st.warning("No prompt history found") 114 | else: 115 | cfw.show_history(hist, images_allow_history_deletion, 'images_last_prompt', self.last_images_query) 116 | 117 | if 'images_last_prompt' not in st.session_state: 118 | st.session_state['images_last_prompt'] = "" 119 | with st.form("image_form"): 120 | prompt_value=f"Images {model} Input [image size: {img_size} | image count: {img_count} | Extra: {kwargs}]" 121 | help_text = '\n\nDALL·E is an AI system that creates realistic images and art from a description in natural language.\n\n- The more detailed the description, the more likely you are to get the result that you or your end user want' 122 | prompt = st.empty().text_area(prompt_value, st.session_state["images_last_prompt"], placeholder="Enter your prompt", key="images_input", help=help_text) 123 | st.session_state['images_last_prompt'] = prompt 124 | 125 | if st.form_submit_button("Generate Image"): 126 | if cf.isBlank(prompt) or len(prompt) < 10: 127 | st.error("Please provide a prompt of at least 10 characters before requesting an answer", icon="✋") 128 | return () 129 | if len(prompt) > self.models[model]["max_prompt_length"]: 130 | st.error(f"Your prompt is {len(prompt)} characters long, which is more than the maximum of {self.models[model]['max_prompt_length']} for this model") 131 | return () 132 | 133 | images_dest_dir = self.get_dest_dir() 134 | with st.spinner(f"Asking OpenAI for a response..."): 135 | err, warn, run_file = self.oaiwui_images.images_it(model, prompt, img_size, img_count, images_dest_dir, **kwargs) 136 | if cf.isNotBlank(err): 137 | st.error(err) 138 | if cf.isNotBlank(warn): 139 | st.warning(warn) 140 | if cf.isNotBlank(run_file): 141 | st.session_state[self.last_images_query] = run_file 142 | st.toast("Done") 143 | 144 | if self.last_images_query in st.session_state: 145 | run_file = st.session_state[self.last_images_query] 146 | run_json = cf.get_run_file(run_file) 147 | self.display_images(run_json['prompt'], run_json['images']) -------------------------------------------------------------------------------- /OAIWUI_WebUI.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import streamlit as st 4 | import extra_streamlit_components as stx 5 | 6 | from OAIWUI_GPT import OAIWUI_GPT 7 | from OAIWUI_Images import OAIWUI_Images 8 | 9 | from OAIWUI_GPT_WebUI import OAIWUI_GPT_WebUI 10 | from OAIWUI_Images_WebUI import OAIWUI_Images_WebUI 11 | 12 | import re 13 | import os.path 14 | 15 | import common_functions as cf 16 | import ollama_helper as oll 17 | 18 | from dotenv import load_dotenv 19 | from datetime import datetime 20 | import time 21 | 22 | import hmac 23 | 24 | ##### 25 | iti_version=cf.iti_version 26 | 27 | st.set_page_config(page_title=f"OpenAI API Compatible WebUI ({iti_version})", page_icon="🫥", layout="wide", initial_sidebar_state="expanded", menu_items={'Get Help': 'https://github.com/Infotrend-Inc/OpenAI_WebUI', 'About': f"# OpenAI API Compatible WebUI ({iti_version})\n Brought to you by [Infotrend Inc.](https://www.infotrend.com/)"}) 28 | 29 | ##### 30 | # https://docs.streamlit.io/knowledge-base/deploy/authentication-without-sso 31 | def check_password(): 32 | """Returns `True` if the user had the correct password.""" 33 | 34 | def password_entered(): 35 | """Checks whether a password entered by the user is correct.""" 36 | if hmac.compare_digest(st.session_state["password"], st.secrets["password"]): 37 | st.session_state["password_correct"] = True 38 | del st.session_state["password"] # Don't store the password. 39 | else: 40 | st.session_state["password_correct"] = False 41 | 42 | # Return True if the password is validated. 43 | if st.session_state.get("password_correct", False): 44 | return True 45 | 46 | # Show input for password. 47 | st.text_input( 48 | "WebUI Required Password", type="password", on_change=password_entered, key="password" 49 | ) 50 | if "password_correct" in st.session_state: 51 | st.error("😕 Password incorrect") 52 | return False 53 | 54 | def del_empty_env_var(var): 55 | if var in os.environ: 56 | tmp = os.environ.get(var) 57 | if cf.isBlank(tmp): 58 | del os.environ[var] 59 | 60 | @st.cache_data 61 | def get_ui_params(runid): 62 | cf.logit(f"---------- Main get_ui_params ({runid}) ----------", "debug") 63 | # Load all supported models (need the status field to decide or prompt if we can use that model or not) 64 | err, av_gpt_models, av_image_models = cf.load_models() 65 | if cf.isNotBlank(err): 66 | st.error(err) 67 | cf.error_exit(err) 68 | 69 | warnings = [ ] 70 | 71 | err = cf.check_file_r(".env", "Environment file") 72 | if cf.isBlank(err): 73 | load_dotenv() 74 | # If the file is not present, hopefully the variable was set in the Docker environemnt 75 | 76 | # Defering apikey check to the GPT class 77 | 78 | save_location = "" 79 | if 'OAIWUI_SAVEDIR' in os.environ: 80 | save_location = os.environ.get('OAIWUI_SAVEDIR') 81 | if cf.isBlank(save_location): 82 | st.error(f"Could not find the OAIWUI_SAVEDIR environment variable") 83 | cf.error_exit("Could not find the OAIWUI_SAVEDIR environment variable") 84 | err = cf.check_existing_dir_w(save_location, "OAIWUI_SAVEDIR directory") 85 | if cf.isNotBlank(err): 86 | st.error(f"While checking OAIWUI_SAVEDIR: {err}") 87 | cf.error_exit(f"{err}") 88 | 89 | gpt_models = "" 90 | if 'OAIWUI_GPT_MODELS' in os.environ: 91 | gpt_models = os.environ.get('OAIWUI_GPT_MODELS') 92 | else: 93 | st.error(f"Could not find the OAIWUI_GPT_MODELS environment variable") 94 | cf.error_exit("Could not find the OAIWUI_GPT_MODELS environment variable") 95 | if cf.isBlank(gpt_models): 96 | st.error(f"OAIWUI_GPT_MODELS environment variable is empty") 97 | cf.error_exit("OAIWUI_GPT_MODELS environment variable is empty") 98 | 99 | gpt_vision = True 100 | if 'OAIWUI_GPT_VISION' in os.environ: 101 | tmp = os.environ.get('OAIWUI_GPT_VISION') 102 | if tmp.lower() == "false": 103 | gpt_vision = False 104 | elif tmp.lower() == "true" : 105 | gpt_vision = True 106 | else: 107 | st.error(f"OAIWUI_GPT_VISION environment variable must be set to 'True' or 'False'") 108 | cf.error_exit("OAIWUI_GPT_VISION environment variable must be set to 'True' or 'False'") 109 | 110 | # Support old DALLE_MODELS environment variable 111 | if 'OAIWUI_DALLE_MODELS' in os.environ: 112 | if 'OAIWUI_IMAGE_MODELS' not in os.environ or cf.isBlank(os.environ.get('OAIWUI_IMAGE_MODELS')): 113 | warnings.append(f"OAIWUI_DALLE_MODELS environment variable is set but OAIWUI_IMAGE_MODELS is not set, will use OAIWUI_DALLE_MODELS as OAIWUI_IMAGE_MODELS. Please set OAIWUI_IMAGE_MODELS to avoid this warning") 114 | os.environ['OAIWUI_IMAGE_MODELS'] = os.environ.get('OAIWUI_DALLE_MODELS') 115 | 116 | image_models = "" 117 | if 'OAIWUI_IMAGE_MODELS' in os.environ: 118 | image_models = os.environ.get('OAIWUI_IMAGE_MODELS') 119 | if cf.isBlank(image_models): 120 | st.error(f"OAIWUI_IMAGE_MODELS environment variable is empty") 121 | cf.error_exit("OAIWUI_IMAGE_MODELS environment variable is empty") 122 | else: 123 | warnings.append(f"Disabling Images -- Could not find the OAIWUI_IMAGE_MODELS environment variable") 124 | os.environ['OAIWUI_GPT_ONLY'] = "True" 125 | 126 | # variable to not fail on empy values, and just ignore those type of errors 127 | ignore_empty = False 128 | if 'OAIWUI_IGNORE_EMPTY' in os.environ: # values does not matter, just need to be present 129 | ignore_empty = True 130 | 131 | # Pre-check API keys -- delete empty ones (to avoid errors and support clean Unraid templates) 132 | if ignore_empty: 133 | del_empty_env_var('OLLAMA_HOST') 134 | del_empty_env_var('OPENAI_API_KEY') 135 | del_empty_env_var('PERPLEXITY_API_KEY') 136 | del_empty_env_var('GEMINI_API_KEY') 137 | del_empty_env_var('OAIWUI_USERNAME') 138 | del_empty_env_var('OAIWUI_PROMPT_PRESETS_DIR') 139 | del_empty_env_var('OAIWUI_PROMPT_PRESETS_ONLY') 140 | 141 | # If no API key or Ollama host is provided, we can not continue 142 | if 'OLLAMA_HOST' not in os.environ and 'OPENAI_API_KEY' not in os.environ and 'PERPLEXITY_API_KEY' not in os.environ and 'GEMINI_API_KEY' not in os.environ: 143 | st.error("No API key or Ollama host provided, can not continue") 144 | cf.error_exit("No API key or Ollama host provided, can not continue") 145 | 146 | # Actual checks 147 | if 'OLLAMA_HOST' in os.environ: 148 | OLLAMA_HOST = os.environ.get('OLLAMA_HOST') 149 | err, ollama_models = oll.get_all_ollama_models_and_infos(OLLAMA_HOST) 150 | if cf.isNotBlank(err): 151 | warnings.append(f"Disabling OLLAMA -- While testing OLLAMA_HOST {OLLAMA_HOST}: {err}") 152 | else: 153 | for oll_model in ollama_models: 154 | # We are going to extend the GPT models with the Ollama models 155 | err, modeljson = oll.ollama_to_modelsjson(OLLAMA_HOST, oll_model, ollama_models[oll_model]) 156 | if cf.isNotBlank(err): 157 | warnings.append(f"Disalbing OLLAMA model -- while obtaining OLLAMA model {oll_model} details: {err}") 158 | continue 159 | av_gpt_models[oll_model] = modeljson 160 | gpt_models += f" {oll_model}" 161 | 162 | username = "" 163 | if 'OAIWUI_USERNAME' in os.environ: 164 | username = os.environ.get('OAIWUI_USERNAME') 165 | if cf.isBlank(username): 166 | warnings.append(f"OAIWUI_USERNAME provided but empty, will ask for username") 167 | else: 168 | st.session_state['username'] = username 169 | 170 | prompt_presets_dir = None 171 | if 'OAIWUI_PROMPT_PRESETS_DIR' in os.environ: 172 | tmp = os.environ.get('OAIWUI_PROMPT_PRESETS_DIR') 173 | if cf.isBlank(tmp): 174 | warnings.append(f"OAIWUI_PROMPT_PRESETS_DIR provided but empty, will not use prompt presets") 175 | else: 176 | err = cf.check_dir(tmp, "OAIWUI_PROMPT_PRESETS_DIR directory") 177 | if cf.isNotBlank(err): 178 | warnings.append(f"While checking OAIWUI_PROMPT_PRESETS_DIR: {err}") 179 | else: 180 | has_json = False 181 | for file in os.listdir(tmp): 182 | if file.endswith(".json"): 183 | has_json = True 184 | break 185 | if not has_json: 186 | warnings.append(f"OAIWUI_PROMPT_PRESETS_DIR provided but appears to not contain prompts, will not use prompt presets") 187 | else: # all the conditions are met 188 | prompt_presets_dir = tmp 189 | 190 | prompt_presets_file = None 191 | if 'OAIWUI_PROMPT_PRESETS_ONLY' in os.environ: 192 | tmp = os.environ.get('OAIWUI_PROMPT_PRESETS_ONLY') 193 | if cf.isBlank(tmp): 194 | warnings.append(f"OAIWUI_PROMPT_PRESETS_ONLY provided but empty, will not use prompt presets") 195 | else: 196 | err = cf.check_file_r(tmp) 197 | if cf.isNotBlank(err): 198 | warnings.append(f"While checking OAIWUI_PROMPT_PRESETS_ONLY: {err}") 199 | else: 200 | if prompt_presets_dir is None: 201 | warnings.append(f"OAIWUI_PROMPT_PRESETS_ONLY provided but no OAIWUI_PROMPT_PRESETS_DIR, will not use prompt presets") 202 | else: # all the conditions are met 203 | prompt_presets_file = tmp 204 | 205 | # Store the initial value of widgets in session state 206 | if "visibility" not in st.session_state: 207 | st.session_state.visibility = "visible" 208 | st.session_state.disabled = False 209 | 210 | # Debug 211 | cf.logit(f"---------- get_ui_params ({runid}) ----------\nwarnings: {warnings}\nsave_location: {save_location}\ngpt_models: {gpt_models}\nav_gpt_models: {av_gpt_models}\ngpt_vision: {gpt_vision}\nimage_models: {image_models}\nav_image_models: {av_image_models}\nprompt_presets_dir: {prompt_presets_dir}\nprompt_presets_file: {prompt_presets_file}", "debug") 212 | 213 | return warnings, save_location, gpt_models, av_gpt_models, gpt_vision, image_models, av_image_models, prompt_presets_dir, prompt_presets_file 214 | 215 | 216 | ##### 217 | 218 | @st.cache_data 219 | def set_ui_core(long_save_location, username, gpt_models, av_gpt_models, gpt_vision, image_models, av_image_models, prompt_presets_dir: str = None, prompt_presets_file: str = None): 220 | oaiwui_gpt = OAIWUI_GPT(long_save_location, username) 221 | err, warn = oaiwui_gpt.set_parameters(gpt_models, av_gpt_models) 222 | process_error_warning(err, warn) 223 | oaiwui_gpt_st = OAIWUI_GPT_WebUI(oaiwui_gpt, gpt_vision, prompt_presets_dir, prompt_presets_file) 224 | oaiwui_images = None 225 | oaiwui_images_st = None 226 | if 'OAIWUI_GPT_ONLY' in os.environ: 227 | tmp = os.environ.get('OAIWUI_GPT_ONLY') 228 | if tmp.lower() == "true": 229 | oaiwui_images = None 230 | elif tmp.lower() == "false": 231 | oaiwui_images = OAIWUI_Images(long_save_location, username) 232 | err, warn = oaiwui_images.set_parameters(image_models, av_image_models) 233 | process_error_warning(err, warn) 234 | oaiwui_images_st = OAIWUI_Images_WebUI(oaiwui_images) 235 | else: 236 | st.error(f"OAIWUI_GPT_ONLY environment variable must be set to 'True' or 'False'") 237 | cf.error_exit("OAIWUI_GPT_ONLY environment variable must be set to 'True' or 'False'") 238 | 239 | return oaiwui_gpt, oaiwui_gpt_st, oaiwui_images, oaiwui_images_st 240 | 241 | 242 | ##### 243 | def main(): 244 | cf.logit("---------- Main __main__ ----------", "debug") 245 | 246 | err = cf.check_file_r(".streamlit/secrets.toml", "Secrets file") 247 | if cf.isBlank(err): 248 | if not check_password(): 249 | st.error("Required password incorrect, can not continue") 250 | st.stop() 251 | 252 | if 'webui_runid' not in st.session_state: 253 | st.session_state['webui_runid'] = datetime.now().strftime("%Y%m%d-%H%M%S") 254 | 255 | warnings, save_location, gpt_models, av_gpt_models, gpt_vision, images_models, av_image_models, prompt_presets_dir, prompt_presets_file = get_ui_params(st.session_state['webui_runid']) 256 | 257 | if len(warnings) > 0: 258 | if 'warning_shown' not in st.session_state: 259 | phl = [] 260 | for w in warnings: 261 | ph = st.empty() 262 | ph.warning(w) 263 | phl.append(ph) 264 | st.session_state['warning_shown'] = True 265 | time.sleep(7) 266 | for ph in phl: 267 | ph.empty() 268 | 269 | st.empty() 270 | 271 | # Grab a session-specific value for username 272 | username = "" 273 | if 'username' in st.session_state: 274 | username = st.session_state['username'] 275 | 276 | if cf.isBlank(username): 277 | with st.form("username_form"): 278 | st.image("./assets/Infotrend_Logo.png", width=600) 279 | username = st.text_input("Enter a username (unauthorized characters will be replaced by _)") 280 | if st.form_submit_button("Save username"): 281 | # replace non alphanumeric by _ 282 | username = re.sub('[^0-9a-zA-Z]+', '_', username) 283 | if cf.isBlank(username): 284 | st.error(f"Username cannot be empty") 285 | else: 286 | st.session_state['username'] = username 287 | st.rerun() 288 | else: 289 | cf.make_wdir_error(os.path.join(save_location)) 290 | long_save_location = os.path.join(save_location, iti_version) 291 | cf.make_wdir_error(os.path.join(long_save_location)) 292 | 293 | oaiwui_gpt, oaiwui_gpt_st, oaiwui_images, oaiwui_images_st = set_ui_core(long_save_location, username, gpt_models, av_gpt_models, gpt_vision, images_models, av_image_models, prompt_presets_dir, prompt_presets_file) 294 | 295 | set_ui(oaiwui_gpt, oaiwui_gpt_st, oaiwui_images, oaiwui_images_st) 296 | 297 | ##### 298 | 299 | def process_error_warning(err, warn): 300 | if cf.isNotBlank(warn): 301 | cf.logit(warn, "warning") 302 | if cf.isNotBlank(err): 303 | if cf.isNotBlank(warn): 304 | st.warning(warn) 305 | st.error(err) 306 | cf.error_exit(err) 307 | 308 | 309 | def set_ui(oaiwui_gpt, oaiwui_gpt_st, oaiwui_images, oaiwui_images_st): 310 | if oaiwui_images is None: 311 | oaiwui_gpt_st.set_ui() 312 | else: 313 | chosen_id = stx.tab_bar(data=[ 314 | stx.TabBarItemData(id="gpt_tab", title="GPTs", description=""), 315 | stx.TabBarItemData(id="images_tab", title="Images", description="") 316 | ]) 317 | if chosen_id == "images_tab": 318 | oaiwui_images_st.set_ui() 319 | else: 320 | oaiwui_gpt_st.set_ui() 321 | 322 | def is_streamlit_running(): 323 | try: 324 | # Check if we're running inside a Streamlit script 325 | return st.runtime.exists() 326 | except: 327 | return False 328 | 329 | ##### 330 | if __name__ == "__main__": 331 | if is_streamlit_running(): 332 | main() 333 | else: 334 | # start streamlit with all the command line arguments 335 | import subprocess 336 | import sys 337 | subprocess.call(["streamlit", "run"] + sys.argv[0:]) 338 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

OpenAI API-compatible WebUI (OAIWUI)

2 | 3 | Latest version: 0.9.11 (20250513) 4 | 5 | - [1. Description](#1-description) 6 | - [1.1. Supported models](#11-supported-models) 7 | - [1.2. .env](#12-env) 8 | - [1.3. savedir](#13-savedir) 9 | - [1.4. password protecting the WebUI](#14-password-protecting-the-webui) 10 | - [1.5. Using "prompt presets" (GPT only)](#15-using-prompt-presets-gpt-only) 11 | - [1.5.1. prompt presets settings](#151-prompt-presets-settings) 12 | - [2. Setup](#2-setup) 13 | - [2.1. Python uv](#21-python-uv) 14 | - [2.2. Docker/Podman](#22-dockerpodman) 15 | - [2.2.1. Building the container](#221-building-the-container) 16 | - [2.2.2. Running the container](#222-running-the-container) 17 | - [2.2.3. local build cleanup](#223-local-build-cleanup) 18 | - [2.3. Docker compose](#23-docker-compose) 19 | - [2.4. Unraid](#24-unraid) 20 | - [3. Misc](#3-misc) 21 | - [3.1. Notes](#31-notes) 22 | - [3.2. Version information/Changelog](#32-version-informationchangelog) 23 | - [3.3. Acknowledgments](#33-acknowledgments) 24 | 25 | Self-hosted WebUI ([streamlit](https://streamlit.io/)-based) to various GPT and Image generation APIs (requires valid API keys for each provider). 26 | Supports some OpenAI API-compatible providers, such as Perplexity AI, Gemini AI and the self-hosted Ollama, enabling a company to install a self-hosted version of the WebUI to access the capabilities of various OpenAI API-compatible GPTs and Image generation APIs, then share access to the tool's capabilities while consolidating billing through API keys. 27 | 28 | | GPT WebUI | Image WebUI | 29 | | --- | --- | 30 | | [![./assets/Screenshot-OAIWUI_WebUI_GPT_small.jpg](./assets/Screenshot-OAIWUI_WebUI_GPT_small.jpg)](./assets/Screenshot-OAIWUI_WebUI_GPT.jpg) | [![./assets/Screenshot-OAIWUI_WebUI_Image_small.jpg](./assets/Screenshot-OAIWUI_WebUI_Image_small.jpg)](./assets/Screenshot-OAIWUI_WebUI_Image.jpg) | 31 | 32 | 33 | Check our [.env.example](./.env.example) for details of possible values for the environment variables. 34 | Unless otherwise specified, even if a feature is not used, its environment variable should be set. 35 | 36 | A pre-built container is available from our Docker account at https://hub.docker.com/r/infotrend/openai_webui 37 | 38 | An [Unraid](https://unraid.net/)-ready version is available directly from Unraid's `Community Applications`. 39 | 40 | Note: this tool was initially developed in February 2023 and released to help end-users. 41 | 42 | # 1. Description 43 | 44 | The tool provides a WebUI to various GPT and Image generation APIs (requires valid API keys for each provider). 45 | 46 | The tool **requires** the use of API keys to use commercial services. 47 | Variables in the `.env` file have the list of possible values and links to additional information on how to get your API keys. 48 | 49 | Depending on your deployment solution (*python virtualenv*, *docker image*, or *unraid*), the deployment might differ slightly. 50 | 51 | Once started, the WebUI will prompt the end user with a `username`. 52 | This username is here to make finding past conversations/images easier if you seek those; no authentication is associated with it. 53 | 54 | GPTs (Text Generation) sidebar options (see "?" mark for specific details): 55 | - model: choose between the different GPT models that are enabled. 56 | - role (user, system, assistant): define the role of the input text for tailored responses. 57 | - max tokens: controls the length of generated text with a maximum token setting (dependent on the model) 58 | - temperature: adjust the "surprisingness" of the generated text. 59 | - additional features depend on the model, see the model information for details and various "?" marks for more details. 60 | 61 | Image Generation sidebar options (see "?" for specific details): 62 | - model: choose between the different Image models that are enabled. 63 | - image Size: specify the dimensions of the images to be generated. 64 | - additional features depend on the model, see the model information for details and various "?" marks for more details. 65 | 66 | ## 1.1. Supported models 67 | 68 | Models are either `deprecated` or `active`. 69 | - `deprecated` models are not available for use anymore. 70 | - `active` models are known to the tool at the time of release. 71 | 72 | The tool will automatically discard known (per the release) `deprecated` models and inform the end user. 73 | Please update your model selection accordingly. 74 | 75 | The [models.json](./models.json) file contains the list of models supported by the current release. 76 | The file content is computer-parsable yet human-readable, as such if you add a compatible model to the file, you should be able to use it right away. 77 | 78 | The [models.md](./models.md) file shows the models supported by the current release. 79 | This file can be generated using: 80 | ```bash 81 | python3 ./list_models.py --markdown > models.md 82 | ``` 83 | 84 | Models are updated frequently by the providers, the list of models supported by the tool is updated at the time of release. 85 | About model updates: 86 | - If a model changes its name (for example with `preview` models), we will update the list of models supported by the tool at the next release. 87 | - If a new model is added and not listed in the `models.json` file, please open an issue on GitHub. 88 | In both cases, if able to, please update the `models.json` file before opening an issue. 89 | 90 | For additional details, see: 91 | - OpenAI Models & API price: https://platform.openai.com/docs/models 92 | - To see the list of authorized models for your account, see https://platform.openai.com/settings/organization/limits 93 | - Google Models & API price: https://ai.google.dev/gemini-api/docs/models 94 | - Perplexity AI Models & API price: https://docs.perplexity.ai/guides/models 95 | 96 | Self-hosted models: 97 | `ollama` is a self-hosted solution; the name is here for illustration prupose (`ollama` is not a recognized model value). At initialization, the tool will use the `OLLAMA_HOST` environment variable to attempt tofind the server; then list and add all available hosted models. Capabilties of the hosted models are various: by default we will authorize `vision` and set a defauklt `max_tokens`; it is advised to check the model information for details on its actual capabilities. Find more information about [Ollama](https://github.com/oollama/ollama) 98 | 99 | ## 1.2. .env 100 | 101 | **Do not distribute your `.env` file as it contains your API keys.** 102 | 103 | The `.env.example` file contains the parameters needed to pass to the running tool: 104 | - `OPENAI_API_KEY` (optional) as obtained from https://platform.openai.com/account/api-keys 105 | - `PERPLEXITY_API_KEY` (optional) as obtained from https://docs.perplexity.ai/guides/getting-started 106 | - `GEMINI_API_KEY` (optional) as obtained from https://ai.google.dev/gemini-api/docs/api-key 107 | - `OAIWUI_SAVEDIR`, the location to save content (make sure the directory exists) 108 | - `OAIWUI_GPT_ONLY`, to request only to show the GPT tab otherwise, shows both GPTs and Images (authorized value: `True` or `False`) 109 | - `OAIWUI_GPT_MODELS` is a comma-separated list of GPT model(s) your API keys are authorized to use. For OpenAI, ee https://platform.openai.com/docs/api-reference/making-requests . For Perplexity AI, see https://docs.perplexity.ai/guides/pricing 110 | - `OAIWUI_IMAGE_MODELS` is a comma-separated list of Image model(s) your API key is authorized to use. 111 | - `OAIWUI_USERNAME` (optional) specifies a `username` and avoids being prompted at each re-run. The default mode is to run in multi-user settings so this is not enabled by default. 112 | - `OAIWUI_GPT_VISION` will, for compatible models, disable their vision capabilities if set to `False` 113 | - `OAIWUI_IGNORE_EMPTY` (required for Unraid) discard errors in case the following environment variables are used but not set. 114 | - `OAIWUI_PROMPT_PRESETS_DIR` sets the directory that contains prompt presets. If a directory is provided, it must contains at least one valid json file. 115 | - `OAIWUI_PROMPT_PRESETS_ONLY` sets the JSON file that contains valid settings to use for the `OAIWUI_PROMPT_PRESETS_DIR` presets. 116 | 117 | Those values can be passed by making a `.env` file containing the expected values or using environment variables. 118 | 119 | The `.env` file is not copied into the `docker` or `unraid` setup. Environment variables should be used in this case. 120 | 121 | The `models.txt` file is a environment variable ready version of the models list. It can be used for the `OAIWUI_GPT_MODELS` and `OAIWUI_IMAGE_MODELS` parameters. 122 | 123 | It is possible to obtain that list from the `models.json` file as environment variables using the `list_models.py` script: 124 | ```bash 125 | python3 ./list_models.py 126 | ``` 127 | 128 | ## 1.3. savedir 129 | 130 | The `OAIWUI_SAVEDIR` variable specifies the location where persistent files will be created from run to run. 131 | 132 | Its structure is: `savedir`/`version`/`username`/`mode`/`UTCtime`/``, with: 133 | - `username` being the self-specified user name prompted when starting the WebUI 134 | - `version` the tool's version, making it easier to debug 135 | - `mode` on of `gpt` or `image` 136 | - the `UTCtime`, a `YYYYY-MM-DD T HH:MM:SS Z` UTC-time of the request (the directory's content will be time ordered) 137 | - `` is often a `json` file containing the details of the run for `gpt`, but also the different `png` images generated for `image` 138 | 139 | We do not check the directories for size. It is left to the end user to clean up space if required. 140 | 141 | ## 1.4. password protecting the WebUI 142 | 143 | To do this, create a `.streamlit/secrets.toml` file in the directory where the streamlit app is started (for the python virtualenv setup, this should be the directory where this `README.md` is present, while for other deployment methods, please see the corresponding [setup](#2-setup) section) and add a `password = "SET_YOUR_PASSWORD_HERE"` value to it. 144 | 145 | When the WebUI starts, it will see of `secrets.toml` file and challenge users for the password set within. 146 | 147 | ## 1.5. Using "prompt presets" (GPT only) 148 | 149 | Prompt presets enable the preparation of custom methods to answer "user" prompt by specifying some "system" and "assistant" settings. 150 | It is used by setting the `OAIWUI_PROMPT_PRESETS_DIR` to a folder containg `.json` files. 151 | 152 | We have provided an example directory containing one pre-configured "prompt preset". 153 | The example directory is named `prompt_presets.example` and its content is the file `shakespeare.json` which guides the GPT answer in the English used by Shakespeare. 154 | 155 | The structure of the used JSON file follows OpenAI `messages`' API structure and as such should be adhere to as closely as possible. 156 | It contains a series of messages that will be passed at the begining of new conversations to the GPT to set the `role` to `system` (the direction the GPT is expected to follow when answering) and/or the `assistant` (past conversations/expected knowledge) for that GPT conversation. The `content` section is expected to be a `text` `type` with the `text` to provide to the GPT. 157 | 158 | For example, one of the prompt for the `shakespeare.json` example is as follows: 159 | 160 | ```json 161 | { 162 | "role": "system", 163 | "content": [ 164 | { 165 | "type": "text", 166 | "text": "You are a helpful assistant. You also like to speak in the words of Shakespeare. Incorporate that into your responses." 167 | } 168 | ], 169 | "oaiwui_skip": true 170 | } 171 | ``` 172 | 173 | The name of the prompt preset is directly related to the name of the file; if the file is title `shakespeare.json`, the prompt will be named `shakespeare`. 174 | 175 | Creating new "prompt presets" should be a matter of duplicating the example and replacing the content within the file. 176 | 177 | Another method consists of passing the prompt to the WebUI and setting the `role` accordingly, then running a query. 178 | The content saved within the `savedir` will contain a `messages` structure that matches the `role` and `content` sections shown above. 179 | Integrate that content within a new prompt presets JSON file. 180 | 181 | Note that the `oaiwui_skip` is not passed to the GPT, but is used to remove the content from the chat history. 182 | 183 | **Note:** not all models will work with the prompt presets. 184 | 185 | ### 1.5.1. prompt presets settings 186 | 187 | When using "prompt presets", it is possible to make the tool behave such that the end user can only use a single `model` with a set `temperature` and maximum requested `tokens`. This JSON settings file is used by pointing the `OAIWUI_PROMPT_PRESETS_ONLY` environment variable to the location of the file. 188 | 189 | We have provided an example `prompt_presets_settings-example.json` file. This example file contains: 190 | 191 | ```json 192 | { 193 | "model": "gpt-4o-mini", 194 | "tokens": 3000, 195 | "temperature": 0.5 196 | } 197 | ``` 198 | , which will: 199 | - use the `gpt-4o-mini` model (which must be in the `OAIWUI_GPT_MODELS` list of authorized models) 200 | - requests a maximum of 3K tokens for the GPT answer. The maximum value per model differs so the tool will error if the requested value is too high (note this is not the context tokens, which covers the entire chat) 201 | - set the temperature to 0.5. The temperature controls the randomness of responses, with lower values yielding more deterministic answers and higher values producing more creative and varied outputs (the range is 0 to 1) 202 | 203 | **Note:** not all parameters will work with the different models. 204 | 205 | # 2. Setup 206 | 207 | ## 2.1. Python uv 208 | 209 | We are now using `uv` to run the WebUI. 210 | 211 | For more details on the tool and how to install it, see https://docs.astral.sh/uv/ and https://github.com/astral-sh/uv 212 | 213 | The following sections expect the `uv` command to be available. 214 | 215 | Dependencies are defined in the `pyproject.toml` file. 216 | Because we are using `uv`, no "local to the project" virtual environment is used. 217 | 218 | Copy the default `.env.example` file as `.env`, and manually edit the copy to add your API keys and the preferred save directory (which must exist before starting the program). 219 | You can also configure the GPT amd Image generation `models` you can access 220 | 221 | ```bash 222 | $ cp .env.example .env 223 | $ vi .env 224 | ``` 225 | 226 | ```bash 227 | uv tool run --with-requirements pyproject.toml streamlit run ./OAIWUI_WebUI.py --server.port=8501 --server.address=0.0.0.0 --server.headless=true --server.fileWatcherType=none --browser.gatherUsageStats=False --logger.level=info 228 | ``` 229 | 230 | You can now open your browser to http://127.0.0.1:8501 to test the WebUI. 231 | 232 | The above command is also available as: 233 | 234 | ```bash 235 | make uv_run 236 | ``` 237 | 238 | ## 2.2. Docker/Podman 239 | 240 | Using containers is an excellent way to test the tool in an isolated, easily redeployed environment. 241 | 242 | The container provides two extra environment variables: `WANTED_UID` and `WANTED_GID`. Through their use, it is possible to set the user ID and group ID of the internal `oaiwui` user. The container is not running as `root`. This user will write files to the mounted volumes; which allows to set the ownership of the files to the user and group ID of the host. If none are specified on the command line, the default values of `0` and `0` are used (ie the `root` user). 243 | 244 | This setup prefers the use of environment variable, using `docker run ... -e VAR=val`, but it is also possible to use a `.env` file or `/oaiwui_config.sh` file mounted within the container, as described below. 245 | 246 | Different options are available using the `Makefile`; type `make` to see the up-to-date list of options. 247 | 248 | ### 2.2.1. Building the container 249 | 250 | The `Makefile` provide an easy means to build the container: 251 | 252 | ```bash 253 | make build 254 | ``` 255 | 256 | This will create a local container named `openai_webui` with two `tags`: the current version number (as defined in the `Makefile`) and the `latest` tag. 257 | 258 | An already built container is provided by on DockerHub: `infotrend/openai_webui`. 259 | 260 | ### 2.2.2. Running the container 261 | 262 | The following uses the `infotrend/openai_webui` container, adapt if you have built your own container. 263 | 264 | Paths that are specified as environment variables are expected to be mounted from the host to the container: when setting `OAIWUI_SAVEDIR=/iti`, the container's `savedir` should be mounted from the host to `/iti` using `-v /host/savedir:/iti` `docker run` argument. 265 | 266 | There are multiple options to run the container. The following are examples commands, adapt as needed. 267 | 268 | Use environment variables on the command line to setup the most common options found in the `.env` file: 269 | ```bash 270 | docker run --rm -it -p 8501:8501 -v `pwd`/savedir`:/iti -e OAIWUI_SAVEDIR=/iti -e WANTED_UID=`id -u` -e WANTED_GID=`id -g` -e OPENAI_API_KEY="Your_OpenAI_API_Key" -e OAIWUI_GPT_ONLY=False -e OAIWUI_GPT_MODELS="gpt-4o-mini,gpt-4.1" -e OAIWUI_IMAGE_MODELS="dall-e-3" infotrend/openai_webui:latest 271 | ``` 272 | 273 | To use the "prompt presets" and its "prompt presets settings" environment variables, those can be added to the command line. For example to use the provided examples add the following to the command line (before the name of the container): 274 | ```-v `pwd`/prompt_presets.example:/prompt_presets -e OAIWUI_PROMPT_PRESETS_DIR=/prompt_presets``` 275 | and ```-v `pwd`/prompt_presets_settings-example.json:/prompt_presets.json -e OAIWUI_PROMPT_PRESETS_ONLY=/prompt_presets.json``` 276 | 277 | To use the password protection for the WebUI, create and populate the `.streamlit/secrets.toml` file before you start the container (see [password protecting the webui](#14-password-protecting-the-webui)) then add `-v PATH_TO/secrets.toml:/app/.streamlit/secrets.toml:ro` to your command line (adapting `PATH_TO` with the full path location of the secrets file) 278 | 279 | With the above options enabled, the earlier command line would become: 280 | ```bash 281 | docker run --rm -it -p 8501:8501 -v `pwd`/savedir:/iti -e OAIWUI_SAVEDIR=/iti -e WANTED_UID=`id -u` -e WANTED_GID=`id -g` -e OPENAI_API_KEY="Your_OpenAI_API_Key" -e OAIWUI_GPT_ONLY=False -e OAIWUI_GPT_MODELS="gpt-4o-mini,gpt-4.1" -e OAIWUI_IMAGE_MODELS="dall-e-3" -v `pwd`/prompt_presets.example:/prompt_presets:ro -e OAIWUI_PROMPT_PRESETS_DIR=/prompt_presets -v `pwd`/prompt_presets_settings-example.json:/prompt_presets.json:ro -e OAIWUI_PROMPT_PRESETS_ONLY=/prompt_presets.json -v `pwd`/secrets.toml:/app/.streamlit/secrets.toml:ro infotrend/openai_webui:latest 282 | ``` 283 | 284 | It is possible to specify some of the command line options from a file mounted within the container. 285 | 286 | One such method is to use a `.env` file mounted within the `/app` directory. 287 | Mounts still need to be performed: adapt the provided `.env.example` file and uses `/iti` for its `OAIWUI_SAVEDIR` environment variable. Extend/adapt the remaining environment variables as needed. Without the prompt presets and password protection options, the command line can be simplified as: 288 | ```bash 289 | docker run --rm -it -p 8501:8501 -v `pwd`/savedir:/iti -e OAIWUI_SAVEDIR=/iti -e WANTED_UID=`id -u` -e WANTED_GID=`id -g` -v `pwd`/.env.docker.example:/app/.env:ro infotrend/openai_webui:latest 290 | ``` 291 | 292 | Another alternative is to use a `config.sh` file mounted within the container as `/oaiwui_config.sh`. The `WANTED_UID` and `WANTED_GID` environment variables can be set within the file. To have a more complete conguration in a file, the example `config.sh` file, can be mounted within the container as `/oaiwui_config.sh`. 293 | ```bash 294 | docker run --rm -it -p 8501:8501 -v `pwd`/savedir:/iti -v `pwd`/config.sh:/oaiwui_config.sh:ro infotrend/openai_webui:latest 295 | ``` 296 | 297 | ### 2.2.3. local build cleanup 298 | 299 | Use the `Makefile` to delete locally built containers: 300 | ```bash 301 | $ make delete 302 | ``` 303 | 304 | Container are built using `buildx`. To delete the build context, use: 305 | ```bash 306 | $ make buildx_rm 307 | ``` 308 | 309 | ## 2.3. Docker compose 310 | 311 | To run the built or downloaded container using `docker compose`, decide on the directory where you want the `compose.yaml` to be. In this example, files are relative to the path where the `compose.yaml` file is located. Create the `savedir` directory and if using `WANTED_UID` and `WANTED_GID`, make sure that folder is owned by the user and group ID specified. 312 | 313 | Adapt the following example to your needs and save it as `compose.yaml`: 314 | ```yaml 315 | services: 316 | openai_webui: 317 | image: infotrend/openai_webui:latest 318 | container_name: openai_webui 319 | restart: unless-stopped 320 | volumes: 321 | - ./savedir:/iti 322 | # Warning: do not mount other content within /iti 323 | # Uncomment the following and create a secrets.toml in the directory where this compose.yaml file is to password protect access to the application 324 | # - ./secrets.toml:/app/.streamlit/secrets.toml:ro 325 | # Mount your "prompt presets" directory to enable those are options 326 | # - ./prompt_presets.example:/prompt_presets:ro 327 | # Mount the "prompt presets" settings file to limit users to the model, tokens and temperature set in the file 328 | # - ./prompt_presets_settings-example.json:prompt_presets.json:ro 329 | # Mount your config file to preset environment variables if preferred; delete corresponding entries from the environment section 330 | # - ./config.sh:/oaiwui_config.sh:ro 331 | ports: 332 | # host port:container port 333 | - 8501:8501 334 | environment: 335 | - OPENAI_API_KEY=${OPENAI_API_KEY} 336 | # Add as many API keys as needed, see the .env example for more details 337 | - OAIWUI_SAVEDIR=/iti 338 | # Uncomment and Set the user ID and group ID of the `oaiwui` user and group. If none are specified, 0/0 will be used 339 | # - WANTED_UID=1001 340 | # - WANTED_GID=1001 341 | # Adapt the following as best suits your deployment 342 | - OAIWUI_GPT_ONLY=False 343 | - OAIWUI_GPT_MODELS=gpt-4o 344 | - OAIWUI_GPT_VISION=True 345 | # Even if OAIWUI_GPT_ONLY is True, please set a model, it will be ignored 346 | - OAIWUI_IMAGE_MODELS=dall-e-3 347 | # Uncomment and enter a value if you are using a single user deployment 348 | # - OAIWUI_USERNAME=user 349 | # Enable the user of "prompt presets" present in the mounted directory (must have a directory matching in the `volumes` section) 350 | # - OAIWUI_PROMPT_PRESETS_DIR=/prompt_presets 351 | # Enable the "prompt presets" setting (must have a file matching in the `volumes` section) 352 | # - OAIWUI_PROMPT_PRESETS_ONLY=/prompt_presets.json 353 | ``` 354 | 355 | Such that: 356 | - Add other variables as needed from your [.env](.env.example) or [config.sh](config.sh) files. 357 | - Create a `docker compose` specific`.env` file in the same directory as the `compose.yaml` file: it needs only contain the `OPENAI_API_KEY=value` entry or other API keys as needed, such that those private values are not exposed in the `compose.yaml` file. 358 | - If using `WANTED_UID` and `WANTED_GID`, make sure that folder is owned by the user and group ID specified. 359 | - If using a `secrets.toml` file with a `password=WEBUIPASSWORD` content, or other environment variables, uncomment/add the corresponding entry in the `compose.yaml` file. 360 | 361 | As configured, the container will `restart` `unless-stopped` which also means that unless the container is stopped it will automatically restart after a host reboot. 362 | 363 | Run using: 364 | ```bash 365 | docker compose up -d 366 | ``` 367 | 368 | The WebUI will be accessible on port 8501 of your host. 369 | 370 | ## 2.4. Unraid 371 | 372 | For [Unraid](https://unraid.net/) users, the same container can be used with unraid's preferred `uid`/`gid`. To do so, specify the `WANTED_UID` and `WANTED_GID` environment variables in the container's template. 373 | 374 | The pre-built container has been added to Unraid's Community Applications. 375 | 376 | The configuration file contains many of the possible environment variables, as detailed in the [.env](#12-env) section. 377 | 378 | Omitted from the configuration file are mounted volumes. If needed, a `Path` mapping a `secrets.toml` file to the `/app/.streamlit/secrets.toml` location within the running docker container (read-only recommended). 379 | Before setting this, create and populate a file with the expected value (as described in [password protecting the WebUI](#14-password-protecting-the-webui)). 380 | For example, if your `appdata` location for the OpenAI WebUI was `/mnt/user/appdata/openai_webui` in which you placed the needed `secrets.toml` file, the expected XML addition would look similar to: 381 | ```xml 382 | /mnt/user/appdata/openai_webui/secrets.toml 383 | ``` 384 | 385 | # 3. Misc 386 | 387 | ## 3.1. Notes 388 | 389 | - If you run into an error when starting the tool. Clear the `streamlit` cache (right side menu) or deleting cookies should solve this. 390 | 391 | ## 3.2. Version information/Changelog 392 | 393 | - v0.9.11 (20250513): Using chat interface for GPTs, and support for additional OpenAI API-compatible providers (Perplexity AI, Gemini AI and the self-hosted Ollama) + new image generation model + moved to uv for deployment + Changed base container to ubuntu:24.04 and added WANTED_UID and WANTED_GID environment variables for Docker and Unraid 394 | - v0.9.10 (20241217): Added `o1` model (untested) following its API access availability 395 | - v0.9.9 (20241206): API changes to use `o1-mini` and `o1-preview` (tested) 396 | - v0.9.8 (20241010): Added `o1-preview` and `o1-mini` model (untested) + "prompt presets" functionalities 397 | - v0.9.7 (20240718): Added `gpt-4o-mini` and `deprecated` older `32k` models 398 | - v0.9.6 (20240701): Added method to disable `vision` for capable models + added whole WebUI password protection using streamlit's `secrets.toml` method 399 | - v0.9.5 (20240611): Added support for `vision` in capable models + Added `gpt-4-turbo` models + Deprecated some models in advance of 20240613 + Updated openai python package to 1.33.0 + Decoupled UI code to allow support for different frontends. 400 | - v0.9.4 (20240513): Added support for `gpt-4o`, updated openai python package to 1.29.0 401 | - v0.9.3 (20240306): Simplifying integration of new models and handling/presentation of their status (active, legacy, deprecated) + Cleaner handling of max_tokens vs context window tokens + updated openai python package to 1.13.3 402 | - v0.9.2 (20241218): Keep prompt history for a given session + allow user to review/delete past prompts + updated openai python package: 1.8.0 403 | - v0.9.1 (20231120): Print `streamlit` errors in case of errors with environment variables + Addition of `gpt-3.5-turbo-1106` in the list of supported models (added in openai python package 1.3.0) + added optional `OAIWUI_USERNAME` environment variable 404 | - v0.9.0 (20231108): Initial release -- incorporating modifications brought by the latest OpenAI Python package (tested against 1.2.0) 405 | - Oct 2023: Preparation for public release 406 | - Feb 2023: Initial version 407 | 408 | ## 3.3. Acknowledgments 409 | 410 | This project includes contributions from [Yan Ding](https://www.linkedin.com/in/yan-ding-01a429108/) and [Muhammed Virk](https://www.linkedin.com/in/mhmmdvirk/) in March 2023. 411 | 412 | -------------------------------------------------------------------------------- /assets/Infotrend_Logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Infotrend-Inc/OpenAI_WebUI/44b4223a1d437082747ada476ca74dccb6f92403/assets/Infotrend_Logo.png -------------------------------------------------------------------------------- /assets/Infotrend_LogoOnly.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Infotrend-Inc/OpenAI_WebUI/44b4223a1d437082747ada476ca74dccb6f92403/assets/Infotrend_LogoOnly.png -------------------------------------------------------------------------------- /assets/Screenshot-OAIWUI_WebUI_GPT.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Infotrend-Inc/OpenAI_WebUI/44b4223a1d437082747ada476ca74dccb6f92403/assets/Screenshot-OAIWUI_WebUI_GPT.jpg -------------------------------------------------------------------------------- /assets/Screenshot-OAIWUI_WebUI_GPT_small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Infotrend-Inc/OpenAI_WebUI/44b4223a1d437082747ada476ca74dccb6f92403/assets/Screenshot-OAIWUI_WebUI_GPT_small.jpg -------------------------------------------------------------------------------- /assets/Screenshot-OAIWUI_WebUI_Image.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Infotrend-Inc/OpenAI_WebUI/44b4223a1d437082747ada476ca74dccb6f92403/assets/Screenshot-OAIWUI_WebUI_Image.jpg -------------------------------------------------------------------------------- /assets/Screenshot-OAIWUI_WebUI_Image_small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Infotrend-Inc/OpenAI_WebUI/44b4223a1d437082747ada476ca74dccb6f92403/assets/Screenshot-OAIWUI_WebUI_Image_small.jpg -------------------------------------------------------------------------------- /common_functions.py: -------------------------------------------------------------------------------- 1 | import os 2 | import os.path 3 | import pathlib 4 | import sys 5 | import shutil 6 | import fnmatch 7 | import stat 8 | 9 | import json 10 | 11 | from datetime import datetime 12 | 13 | import logging 14 | logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s') 15 | 16 | iti_version="0.9.11" 17 | 18 | def logit(msg, mode="info"): 19 | if mode == "info": 20 | logging.info(msg) 21 | elif mode == "warning": 22 | logging.warning(msg) 23 | elif mode == "error": 24 | logging.error(msg) 25 | elif mode == "debug": 26 | logging.debug(msg) 27 | 28 | def isBlank (myString): 29 | return not (myString and myString.strip()) 30 | 31 | def isNotBlank (myString): 32 | return bool(myString and myString.strip()) 33 | 34 | ##### 35 | 36 | def get_fullpath(path): 37 | return(os.path.abspath(path)) 38 | 39 | ##### 40 | 41 | def check_file(file, text="checked file"): 42 | if not os.path.exists(file): 43 | return(f"{text} ({file}) does not exist") 44 | if not os.path.isfile(file): 45 | return(f"{text} ({file}) is not a file") 46 | return("") 47 | 48 | ## 49 | 50 | def check_file_r(file, text="checked file"): 51 | err = check_file(file, text) 52 | if isNotBlank(err): 53 | return(err) 54 | permissions = stat.S_IMODE(os.lstat(file).st_mode) 55 | if not (permissions & stat.S_IRUSR): 56 | return(f"{text} ({file}) can not be read") 57 | return("") 58 | 59 | ## 60 | 61 | def check_existing_file_w(file, text="checked file"): 62 | err = check_file(file, text) 63 | if isNotBlank(err): 64 | return err 65 | permissions = stat.S_IMODE(os.lstat(file).st_mode) 66 | if not (permissions & stat.S_IWUSR): 67 | return f"{text} ({file}) unable to write" 68 | 69 | return "" 70 | 71 | ## 72 | 73 | def check_file_w(file, text="checked file"): 74 | err = check_file(file, text) 75 | if isBlank(err): 76 | # The file already exist, lets check if we can write to it 77 | err = check_existing_file_w(file, text) 78 | if isNotBlank(err): 79 | # The file is not writable 80 | return f"{text} ({file}) file exists but we are unable to write" 81 | return "" # the file exists and is writable, we are done here 82 | 83 | # if the file does not exist, we will try to create it 84 | file_obj = open(file, "a") # open in append mode to create the file 85 | file_obj.close() 86 | # we created a file that should be writable, let's check again 87 | return check_existing_file_w(file, text) 88 | 89 | ## 90 | 91 | def check_file_size(file, text="checked file"): 92 | err = check_file_r(file, text) 93 | if isNotBlank(err): 94 | return(err, 0) 95 | return("", pathlib.Path(file).stat().st_size) 96 | 97 | ##### 98 | 99 | def check_dir(dir, text="checked directory"): 100 | if os.path.exists(dir) is False: 101 | return f"Path ({dir}) for {text} does not exist" 102 | 103 | if os.path.isdir(dir) is False: 104 | return f"{text} directory ({dir}) does not exist" 105 | 106 | return("") 107 | 108 | ## 109 | 110 | def check_existing_dir_w(dir, text="checked"): 111 | err = check_dir(dir, text=text) 112 | if isNotBlank(err): 113 | return err 114 | permissions = stat.S_IMODE(os.lstat(dir).st_mode) 115 | if not (permissions & stat.S_IWUSR): 116 | return f"{text} directory ({dir}) unable to write to" 117 | return "" 118 | 119 | ### 120 | 121 | def make_wdir(dir, text="destination"): 122 | if os.path.isdir(dir) is True: 123 | return check_existing_dir_w(dir, text) 124 | 125 | # the directory does not exist, we will try to create it 126 | os.mkdir(dir, 0o755) 127 | # and test if we were successful 128 | return check_existing_dir_w(dir, text) 129 | 130 | ### 131 | 132 | def make_wdir_recursive(dir, text="destination"): 133 | if os.path.isdir(dir) is True: 134 | return check_existing_dir_w(dir, text) 135 | 136 | # the directories does not exist, we will try to create them 137 | pathlib.Path(dir).mkdir(parents=True, exist_ok=True) 138 | # and test if we were successful 139 | return check_existing_dir_w(dir, text) 140 | 141 | 142 | def make_wdir_error(dest_dir): 143 | err = make_wdir(dest_dir) 144 | if isNotBlank(err): 145 | error_exit(err) 146 | 147 | ### 148 | 149 | def get_dirlist(dir, text="checked directory"): 150 | err = check_dir(dir, text) 151 | if isNotBlank(err): 152 | return (err, []) 153 | 154 | listing = os.listdir(dir) 155 | return ("", listing) 156 | 157 | ########## 158 | 159 | def get_timeUTC(): 160 | return(datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")) 161 | 162 | ##### 163 | 164 | def get_run_file(run_file): 165 | err = check_file_r(run_file) 166 | if isNotBlank(err): 167 | error_exit(err) 168 | with open(run_file, 'r') as f: 169 | run_json = json.load(f) 170 | return (run_json) 171 | 172 | ##### 173 | 174 | def error_exit(txt): 175 | logit(txt, "error") 176 | sys.exit(1) 177 | 178 | ########## 179 | 180 | def directory_rmtree(dir): 181 | if os.path.isdir(dir) is False: 182 | return("") 183 | try: 184 | shutil.rmtree(dir, ignore_errors=True) 185 | except OSError as e: 186 | x = str(e) 187 | return(f"Problem deleting directory {dir}: {x}") 188 | 189 | return("") 190 | 191 | def get_dirname(path): 192 | return(os.path.dirname(path)) 193 | 194 | ##### 195 | 196 | def get_history_core(search_dir, mode: str = "GPT"): 197 | hist = {} 198 | err, listing = get_dirlist(search_dir, "save location") 199 | if isNotBlank(err): 200 | return f"While getting directory listing from {search_dir}: {err}, history will be incomplete", hist 201 | for entry in listing: 202 | entry_dir = os.path.join(search_dir, entry) 203 | if os.path.isdir(entry_dir) is True: 204 | err = check_existing_dir_w(entry_dir) 205 | if isNotBlank(err): 206 | return f"While checking {entry_dir}: {err}, history will be incomplete", hist 207 | for file in os.listdir(entry_dir): 208 | if fnmatch.fnmatch(file, 'run.json'): 209 | run_file = os.path.join(entry_dir, file) 210 | run_json = get_run_file(run_file) 211 | if mode == "GPT": 212 | prompt = run_json[-1]['content'] 213 | hist[entry] = [prompt, run_file] 214 | else: 215 | if 'prompt' in run_json: 216 | prompt = run_json['prompt'] 217 | hist[entry] = [prompt, run_file] 218 | break 219 | return "", hist 220 | 221 | def get_history(search_dir): 222 | return get_history_core(search_dir, "Image") 223 | 224 | def get_gpt_history(search_dir): 225 | return get_history_core(search_dir, "GPT") 226 | 227 | ##### 228 | 229 | def read_json(file, txt=''): 230 | err = check_file_r(file) 231 | if err != "": 232 | error_exit(f"Problem with input {txt} Json file ({file}): {err}") 233 | 234 | with open(file) as simple_file: 235 | file_contents = simple_file.read() 236 | parsed_json = json.loads(file_contents) 237 | return parsed_json 238 | 239 | ##### 240 | 241 | def check_apikeys(provider, meta): 242 | if 'apikey' in meta: # apikey hardcoded (for the likes of ollama who ignore the value) 243 | return "", meta["apikey"] 244 | 245 | apikey_env = '' 246 | if 'apikey-env' in meta: # apikey in an environment variable 247 | apikey_env = meta["apikey-env"] 248 | if isBlank(apikey_env): 249 | return "Missing information about environment variable to check for apikey", "" 250 | 251 | if apikey_env not in os.environ: 252 | return f"Environment variable {apikey_env} not set", "" 253 | 254 | apikey = os.environ[apikey_env] 255 | return "", apikey 256 | 257 | ##### 258 | 259 | def load_models(): 260 | err = check_file_r("models.json", "models.json") 261 | if isNotBlank(err): 262 | return f"While checking models.json: {err}", None, None 263 | all_models = read_json("models.json") 264 | if all_models is None: 265 | return f"Could not read models.json", None, None 266 | gpt_models = {} 267 | if 'GPT' in all_models: 268 | gpt_models = all_models['GPT'] 269 | else: 270 | return f"Could not find GPT in models.json", None, None 271 | images_models = {} 272 | if 'Image' in all_models: 273 | images_models = all_models['Image'] 274 | else: 275 | return f"Could not find Image in models.json", None, None 276 | 277 | return "", gpt_models, images_models 278 | -------------------------------------------------------------------------------- /common_functions_WebUI.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | import common_functions as cf 4 | 5 | import streamlit as st 6 | 7 | 8 | def get_runid(): 9 | if 'webui_runid' not in st.session_state: 10 | st.session_state['webui_runid'] = cf.get_timeUTC() 11 | 12 | runid = st.session_state['webui_runid'] 13 | return runid 14 | 15 | ##### 16 | 17 | def show_history_core(hist, allow_history_deletion, last_prompt_key, last_query_key, mode:str = "DallE"): 18 | hk = [x for x in hist.keys() if cf.isNotBlank(x)] 19 | hk = sorted(hk, reverse=True) 20 | hk_opt = [hist[x][0] for x in hk] 21 | hk_q = {hist[x][0]: hist[x][1] for x in hk} 22 | prev = st.selectbox("Prompt History (showing last GPT answer, most recent first)", options=hk_opt, index=0, key="history") 23 | if st.button("Load Selected Prompt", key="load_history"): 24 | if mode == "GPT": 25 | file = hk_q[prev] 26 | err = cf.check_file_r(file) 27 | if cf.isNotBlank(err): 28 | st.error(f"While checking file {file}: {err}") 29 | st.session_state.gpt_messages = cf.read_json(file) 30 | else: 31 | st.session_state[last_prompt_key] = prev 32 | st.session_state[last_query_key] = hk_q[prev] 33 | if allow_history_deletion: 34 | if st.button("Delete Selected Prompt", key="delete_history"): 35 | if cf.isNotBlank(prev): 36 | dir = os.path.dirname(hk_q[prev]) 37 | err = cf.directory_rmtree(dir) 38 | if cf.isNotBlank(err): 39 | st.error(f"While deleting {dir}: {err}") 40 | else: 41 | if os.path.exists(dir): 42 | st.error(f"Directory {dir} still exists") 43 | else: 44 | st.success(f"Deleted") 45 | else: 46 | st.error("Please select a prompt to delete") 47 | 48 | def show_history(hist, allow_history_deletion, last_prompt_key, last_query_key): 49 | return show_history_core(hist, allow_history_deletion, last_prompt_key, last_query_key, "DallE") 50 | 51 | def show_gpt_history(hist, allow_history_deletion): 52 | return show_history_core(hist, allow_history_deletion, None, None, "GPT") -------------------------------------------------------------------------------- /config.sh: -------------------------------------------------------------------------------- 1 | ## oaiwui configuration 2 | # loaded by entrypoint.sh as /oaiwui_config.sh 3 | # ... after setting the variables from the command line: will override with the values set here 4 | # 5 | # To use your custom version, duplicate the file and mount it in the container: -v /path/to/your/config.sh:/oaiwui_config.sh 6 | # 7 | # Can be used to set the other command line variables 8 | # Set using: export VARIABLE=value 9 | 10 | ## Environment variables loaded when passing environment variables from user to user 11 | # Ignore list: variables to ignore when loading environment variables from user to user 12 | export ENV_IGNORELIST="HOME PWD USER SHLVL TERM OLDPWD SHELL _ SUDO_COMMAND HOSTNAME LOGNAME MAIL SUDO_GID SUDO_UID SUDO_USER ENV_IGNORELIST ENV_OBFUSCATE_PART" 13 | # Obfuscate part: part of the key to obfuscate when loading environment variables from user to user, ex: API_KEY, ... 14 | export ENV_OBFUSCATE_PART="TOKEN API KEY" 15 | 16 | # Uncomment and set as preferred, see README.md for more details 17 | 18 | ## User and group id 19 | #export WANTED_UID=1000 20 | #export WANTED_GID=1000 21 | # DO NOT use `id -u` or `id -g` to set the values, use the actual values -- the script is started by oaiwuitoo with 1025/1025 22 | 23 | ## Verbose mode 24 | # uncomment is enough to enable 25 | #export OAIWUI_VERBOSE="yes" 26 | 27 | ##### If desired, copy the content of your .env file into this file 28 | ##### and mount it in the container: -v /path/to/your/config.sh:/oaiwui_config.sh 29 | 30 | 31 | # Do not use an exit code, this is loaded by source 32 | -------------------------------------------------------------------------------- /entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | # everyone can read our files by default 6 | umask 0022 7 | 8 | # Reorder to have scripts available to one another 9 | error_exit() { 10 | echo -n "!! ERROR: " 11 | echo $* 12 | echo "!! Exiting script (ID: $$)" 13 | exit 1 14 | } 15 | 16 | # Write a world-writeable file (preferably inside /tmp -- ie within the container) 17 | write_worldtmpfile() { 18 | tmpfile=$1 19 | if [ -z "${tmpfile}" ]; then error_exit "write_worldfile: missing argument"; fi 20 | if [ -f $tmpfile ]; then sudo rm -f $tmpfile; fi 21 | echo -n $2 > ${tmpfile} 22 | sudo chmod 777 ${tmpfile} 23 | } 24 | 25 | # Set verbose mode 26 | if [ ! -z "${OAIWUI_VERBOSE+x}" ]; then write_worldtmpfile /tmp/.OAIWUI-VERBOSE "yes"; fi 27 | 28 | verb_echo() { 29 | if [ -f /tmp/.OAIWUI-VERBOSE ]; then 30 | echo $* 31 | fi 32 | } 33 | 34 | ok_exit() { 35 | verb_echo $* 36 | verb_echo "++ Exiting script (ID: $$)" 37 | exit 0 38 | } 39 | 40 | # Load config (must have at least ENV_IGNORELIST and ENV_OBFUSCATE_PART set) 41 | it=/oaiwui_config.sh 42 | if [ -f $it ]; then 43 | source $it || error_exit "Failed to load config: $it" 44 | else 45 | error_exit "Failed to load config: $it not found" 46 | fi 47 | # Check for ENV_IGNORELIST and ENV_OBFUSCATE_PART 48 | if [ -z "${ENV_IGNORELIST+x}" ]; then error_exit "ENV_IGNORELIST not set"; fi 49 | if [ -z "${ENV_OBFUSCATE_PART+x}" ]; then error_exit "ENV_OBFUSCATE_PART not set"; fi 50 | 51 | save_env() { 52 | tosave=$1 53 | verb_echo "-- Saving environment variables to $tosave" 54 | env | sort > "$tosave" 55 | } 56 | 57 | load_env() { 58 | tocheck=$1 59 | overwrite_if_different=$2 60 | ignore_list="${ENV_IGNORELIST}" 61 | obfuscate_part="${ENV_OBFUSCATE_PART}" 62 | if [ -f "$tocheck" ]; then 63 | verb_echo "-- Loading environment variables from $tocheck (overwrite existing: $overwrite_if_different) (ignorelist: $ignore_list) (obfuscate: $obfuscate_part)" 64 | while IFS='=' read -r key value; do 65 | doit=false 66 | # checking if the key is in the ignorelist 67 | for i in $ignore_list; do 68 | if [[ "A$key" == "A$i" ]]; then doit=ignore; break; fi 69 | done 70 | if [[ "A$doit" == "Aignore" ]]; then continue; fi 71 | rvalue=$value 72 | # checking if part of the key is in the obfuscate list 73 | doobs=false 74 | for i in $obfuscate_part; do 75 | if [[ "A$key" == *"$i"* ]]; then doobs=obfuscate; break; fi 76 | done 77 | if [[ "A$doobs" == "Aobfuscate" ]]; then rvalue="**OBFUSCATED**"; fi 78 | 79 | if [ -z "${!key}" ]; then 80 | verb_echo " ++ Setting environment variable $key [$rvalue]" 81 | doit=true 82 | elif [ "$overwrite_if_different" = true ]; then 83 | cvalue="${!key}" 84 | if [[ "A${doobs}" == "Aobfuscate" ]]; then cvalue="**OBFUSCATED**"; fi 85 | if [[ "A${!key}" != "A${value}" ]]; then 86 | verb_echo " @@ Overwriting environment variable $key [$cvalue] -> [$rvalue]" 87 | doit=true 88 | else 89 | verb_echo " == Environment variable $key [$rvalue] already set and value is unchanged" 90 | fi 91 | fi 92 | if [[ "A$doit" == "Atrue" ]]; then 93 | export "$key=$value" 94 | fi 95 | done < "$tocheck" 96 | fi 97 | } 98 | 99 | 100 | whoami=`whoami` 101 | script_dir=$(dirname $0) 102 | script_name=$(basename $0) 103 | verb_echo ""; verb_echo "" 104 | verb_echo "======================================" 105 | verb_echo "=================== Starting script (ID: $$)" 106 | verb_echo "== Running ${script_name} in ${script_dir} as ${whoami}" 107 | script_fullname=$0 108 | verb_echo " - script_fullname: ${script_fullname}" 109 | verb_echo "======================================" 110 | 111 | # Get user and group id 112 | if [ -f /tmp/.OAIWUI-WANTED_UID ]; then WANTED_UID=$(cat /tmp/.OAIWUI-WANTED_UID); fi 113 | if [ -f /tmp/.OAIWUI-WANTED_GID ]; then WANTED_GID=$(cat /tmp/.OAIWUI-WANTED_GID); fi 114 | # if no WANTED_UID or WANTED_GID is set, we will set them to 0 (ie root) [the previous default] 115 | WANTED_UID=${WANTED_UID:-0} 116 | WANTED_GID=${WANTED_GID:-0} 117 | # save the values 118 | if [ ! -f /tmp/.OAIWUI-WANTED_UID ]; then write_worldtmpfile /tmp/.OAIWUI-WANTED_UID ${WANTED_UID}; fi 119 | if [ ! -f /tmp/.OAIWUI-WANTED_GID ]; then write_worldtmpfile /tmp/.OAIWUI-WANTED_GID ${WANTED_GID}; fi 120 | 121 | # Extracting the command line arguments (if any)(streamlit overrides for example) and placing them in /oaiwui_run.sh 122 | if [ ! -z "$*" ]; then write_worldtmpfile /tmp/oaiwui-run.sh "$*"; fi 123 | 124 | # Check user id and group id 125 | new_gid=`id -g` 126 | new_uid=`id -u` 127 | verb_echo "== user ($whoami)" 128 | verb_echo " uid: $new_uid / WANTED_UID: $WANTED_UID" 129 | verb_echo " gid: $new_gid / WANTED_GID: $WANTED_GID" 130 | 131 | # oaiwuitoo is a specfiic user not existing by default on debian, we can check its whomai 132 | if [ "A${whoami}" == "Aoaiwuitoo" ]; then 133 | verb_echo "-- Running as oaiwuitoo, will switch oaiwui to the desired UID/GID" 134 | # The script is started as oaiwuitoo -- UID/GID 1025/1025 135 | 136 | # We are altering the UID/GID of the oaiwui user to the desired ones and restarting as oaiwui 137 | # using usermod for the already create oaiwui user, knowing it is not already in use 138 | # per usermod manual: "You must make certain that the named user is not executing any processes when this command is being executed" 139 | 140 | verb_echo "-- Setting owner of /home/oaiwui to $WANTED_UID:$WANTED_GID (might take a while)" 141 | sudo chown -R ${WANTED_UID}:${WANTED_GID} /home/oaiwui || error_exit "Failed to set owner of /home/oaiwui" 142 | verb_echo "-- Setting GID of oaiwui user to $WANTED_GID" 143 | sudo groupmod -o -g ${WANTED_GID} oaiwui || error_exit "Failed to set GID of oaiwui user" 144 | verb_echo "-- Setting UID of oaiwui user to $WANTED_UID" 145 | sudo usermod -o -u ${WANTED_UID} oaiwui || error_exit "Failed to set UID of oaiwui user" 146 | 147 | # save the current environment 148 | save_env /tmp/.oaiwuitoo-env 149 | 150 | # restart the script as oaiwui set with the correct UID/GID this time 151 | verb_echo "-- Restarting as oaiwui user with UID ${WANTED_UID} GID ${WANTED_GID}" 152 | sudo su oaiwui $script_fullname || error_exit "subscript failed" 153 | ok_exit "Clean exit" 154 | fi 155 | 156 | # If we are here, the script is started as another user than oaiwuitoo 157 | # because the whoami value for the oaiwui user can be any existing user, we can not check against it 158 | # instead we check if the UID/GID are the expected ones 159 | if [ "$WANTED_GID" != "$new_gid" ]; then error_exit "oaiwui MUST be running as UID ${WANTED_UID} GID ${WANTED_GID}, current UID ${new_uid} GID ${new_gid}"; fi 160 | if [ "$WANTED_UID" != "$new_uid" ]; then error_exit "oaiwui MUST be running as UID ${WANTED_UID} GID ${WANTED_GID}, current UID ${new_uid} GID ${new_gid}"; fi 161 | 162 | # We are therefore running as oaiwui 163 | verb_echo ""; verb_echo "== Running as oaiwui" 164 | 165 | # Load environment variables one by one if they do not exist from /tmp/.oaiwuitoo-env 166 | it=/tmp/.oaiwuitoo-env 167 | if [ -f $it ]; then 168 | load_env $it true 169 | fi 170 | 171 | # Extend PATH with the venv bin directory (for python3 and streamlit) 172 | export PATH="/app/.venv/bin:$PATH" 173 | 174 | ########## 'oaiwui' specific section below 175 | if [ -f /tmp/oaiwui-run.sh ]; then 176 | sudo chmod +x /tmp/oaiwui-run.sh || error_exit "Failed to make /tmp/oaiwui-run.sh executable" 177 | /tmp/oaiwui-run.sh 178 | else 179 | streamlit run OAIWUI_WebUI.py --server.port=8501 --server.address=0.0.0.0 --server.headless=true --server.fileWatcherType=none --browser.gatherUsageStats=False --logger.level=info 180 | fi 181 | 182 | exit 0 183 | -------------------------------------------------------------------------------- /list_models.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import argparse 4 | import sys 5 | import common_functions as cf 6 | 7 | def markdown_models(gpt_models, images_models): 8 | print("| Mode | Model | Provider | Status | Capability | Notes | About |") 9 | print("| --- | --- | --- | --- | --- | --- | --- |") 10 | print("| GPT | [`ollama`](https://ollama.com/) | SelfHosted | active | vision`?` | | Self-hosted models using [`ollama`](https://ollama.com/); will search for `OLLAMA_HOST` environment variable |") 11 | 12 | for model in sorted(list(gpt_models.keys())): 13 | notes = gpt_models[model]['status_details'] if 'status_details' in gpt_models[model] else '' 14 | meta = gpt_models[model]['meta'] 15 | model_long = f"[`{model}`]({meta['model_url']})" if 'model_url' in meta else f"`{model}`" 16 | capability = ', '.join(gpt_models[model]['capability']) 17 | print(f"| GPT | {model_long} | {meta['provider']} | {gpt_models[model]['status']} | {capability} | {notes} | {gpt_models[model]['label']} |") 18 | for model in sorted(list(images_models.keys())): 19 | notes = images_models[model]['status_details'] if 'status_details' in images_models[model] else '' 20 | meta = images_models[model]['meta'] 21 | model_long = f"[`{model}`]({meta['model_url']})" if 'model_url' in meta else f"`{model}`" 22 | print(f"| Image | {model_long} | {meta['provider']} | {images_models[model]['status']} | | {notes} | {images_models[model]['label']} |") 23 | 24 | if __name__ == "__main__": 25 | # argparse 26 | parser = argparse.ArgumentParser(description='List available models, by default in format suitable for use as environment variables') 27 | parser.add_argument('--markdown', action='store_true', help='Show models in a markdown format and exit') 28 | args = parser.parse_args() 29 | 30 | err, gpt_models, images_models = cf.load_models() 31 | if cf.isNotBlank(err): 32 | print(err) 33 | sys.exit(1) 34 | 35 | if args.markdown: 36 | markdown_models(gpt_models, images_models) 37 | sys.exit(0) 38 | 39 | # remove deprecated models 40 | gpt_models = {k: v for k, v in gpt_models.items() if v['status'] != 'deprecated'} 41 | images_models = {k: v for k, v in images_models.items() if v['status'] != 'deprecated'} 42 | print("OAIWUI_GPT_MODELS=" + " ".join(sorted(list(gpt_models.keys())))) 43 | print("OAIWUI_IMAGES_MODELS=" + " ".join(sorted(list(images_models.keys())))) 44 | 45 | sys.exit(0) -------------------------------------------------------------------------------- /models.json: -------------------------------------------------------------------------------- 1 | { 2 | "GPT": 3 | { 4 | "chatgpt-4o-latest": 5 | { 6 | "label": "OpenAI's ChatGPT-4o points to the GPT-4o snapshot currently used in ChatGPT. GPT-4o is their versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs. It is the best model for most tasks, and is their most capable model outside of their o-series models.", 7 | "max_token": 16384, 8 | "context_token": 128000, 9 | "data": "Up to Sep 2023", 10 | "status": "active", 11 | "status_details": "", 12 | "capability": ["vision"], 13 | "meta": { 14 | "provider": "OpenAI", 15 | "apikey-env": "OPENAI_API_KEY", 16 | "model_url": "https://platform.openai.com/docs/models/gpt-4o" 17 | } 18 | }, 19 | 20 | 21 | "gemini-1.5-flash": 22 | { 23 | "label": "Google's fast and versatile performance across a diverse variety of tasks", 24 | "max_token": 8000, 25 | "context_token": 1000000, 26 | "data": "Latest update: Sep 2024", 27 | "status": "active", 28 | "status_details": "", 29 | "capability": ["vision"], 30 | "meta": { 31 | "provider": "GoogleAI", 32 | "apikey-env": "GEMINI_API_KEY", 33 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 34 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-1.5-flash" 35 | } 36 | }, 37 | 38 | "gemini-1.5-flash-8b": 39 | { 40 | "label": "Google's fast and versatile performance across a diverse variety of tasks", 41 | "max_token": 8000, 42 | "context_token": 1000000, 43 | "data": "Latest update: Oct 2024", 44 | "status": "active", 45 | "status_details": "", 46 | "capability": ["vision"], 47 | "meta": { 48 | "provider": "GoogleAI", 49 | "apikey-env": "GEMINI_API_KEY", 50 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 51 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-1.5-flash-8b" 52 | } 53 | }, 54 | 55 | "gemini-1.5-pro": 56 | { 57 | "label": "Google model for complex reasoning tasks requiring more intelligence", 58 | "max_token": 8000, 59 | "context_token": 2000000, 60 | "data": "Latest update: Sep 2024", 61 | "status": "active", 62 | "status_details": "", 63 | "capability": ["vision"], 64 | "meta": { 65 | "provider": "GoogleAI", 66 | "apikey-env": "GEMINI_API_KEY", 67 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 68 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-1.5-pro" 69 | } 70 | }, 71 | 72 | "gemini-2.0-flash": 73 | { 74 | "label": "Google model with next generation features, speed, and multimodal generation for a diverse variety of tasks", 75 | "max_token": 8000, 76 | "context_token": 1000000, 77 | "data": "Latest update: Feb 2025, Knowledge cutoff: Aug 2024", 78 | "status": "active", 79 | "status_details": "", 80 | "capability": ["vision"], 81 | "meta": { 82 | "provider": "GoogleAI", 83 | "apikey-env": "GEMINI_API_KEY", 84 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 85 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-2.0-flash" 86 | } 87 | }, 88 | 89 | "gemini-2.0-flash-lite": 90 | { 91 | "label": "Google model that reasons over the most complex problems, Show the thinking process of the model, Tackle difficult code and math problems", 92 | "max_token": 8000, 93 | "context_token": 1000000, 94 | "data": "Latest update: Feb 2025, Knowledge cutoff: Aug 2024", 95 | "status": "active", 96 | "status_details": "", 97 | "capability": ["vision"], 98 | "meta": { 99 | "provider": "GoogleAI", 100 | "apikey-env": "GEMINI_API_KEY", 101 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 102 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-2.0-flash-lite" 103 | } 104 | }, 105 | 106 | "gemini-2.5-flash-preview-04-17": 107 | { 108 | "label": "Google's best model in terms of price-performance, offering well-rounded capabilities. Gemini 2.5 Flash rate limits are more restricted since it is an experimental / preview model", 109 | "max_token": 65000, 110 | "context_token": 1000000, 111 | "data": "Latest update: Apr 2025, Knowledge cutoff: Jan 2025", 112 | "status": "active", 113 | "status_details": "", 114 | "capability": ["vision"], 115 | "meta": { 116 | "provider": "GoogleAI", 117 | "apikey-env": "GEMINI_API_KEY", 118 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 119 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-2.5-flash-preview" 120 | } 121 | }, 122 | 123 | "gemini-2.5-pro-preview-03-25": 124 | { 125 | "label": "Google's state-of-the-art thinking model, capable of reasoning over complex problems in code, math, and STEM, as well as analyzing large datasets, codebases, and documents using long context.", 126 | "max_token": 65000, 127 | "context_token": 1000000, 128 | "data": "Latest update: Mar 2025, Knowledge cutoff: Jan 2025", 129 | "status": "active", 130 | "status_details": "", 131 | "capability": ["vision"], 132 | "meta": { 133 | "provider": "GoogleAI", 134 | "apikey-env": "GEMINI_API_KEY", 135 | "apiurl": "https://generativelanguage.googleapis.com/v1beta/openai/", 136 | "model_url": "https://ai.google.dev/gemini-api/docs/models#gemini-2.5-pro-preview-03-25" 137 | } 138 | }, 139 | 140 | 141 | "gpt-4.1": 142 | { 143 | "label": "OpenAI's flagship model for complex tasks. It is well suited for problem solving across domains.", 144 | "max_token": 32000, 145 | "context_token": 1000000, 146 | "data": "Up to May 2024", 147 | "status": "active", 148 | "status_details": "", 149 | "capability": ["vision"], 150 | "meta": { 151 | "provider": "OpenAI", 152 | "apikey-env": "OPENAI_API_KEY", 153 | "model_url": "https://platform.openai.com/docs/models/gpt-4.1" 154 | } 155 | }, 156 | 157 | "gpt-4.1-mini": 158 | { 159 | "label": "OpenAI's GPT-4.1 mini provides a balance between intelligence, speed, and cost that makes it an attractive model for many use cases.", 160 | "max_token": 32000, 161 | "context_token": 1000000, 162 | "data": "Up to May 2024", 163 | "status": "active", 164 | "status_details": "", 165 | "capability": ["vision"], 166 | "meta": { 167 | "provider": "OpenAI", 168 | "apikey-env": "OPENAI_API_KEY", 169 | "model_url": "https://platform.openai.com/docs/models/gpt-4.1-mini" 170 | } 171 | }, 172 | 173 | "gpt-4.1-nano": 174 | { 175 | "label": "OpenAI's GPT-4.1 nano is the fastest, most cost-effective GPT-4.1 model.", 176 | "max_token": 32000, 177 | "context_token": 1000000, 178 | "data": "Up to May 2024", 179 | "status": "active", 180 | "status_details": "", 181 | "capability": ["vision"], 182 | "meta": { 183 | "provider": "OpenAI", 184 | "apikey-env": "OPENAI_API_KEY", 185 | "model_url": "https://platform.openai.com/docs/models/gpt-4.1-nano" 186 | } 187 | }, 188 | 189 | "gpt-4o": 190 | { 191 | "label": "OpenAI's GPT-4o (“o” for “omni”) is their versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is the best model for most tasks, and is their most capable model outside of their o-series models.", 192 | "max_token": 16384, 193 | "context_token": 128000, 194 | "data": "Up to Sep 2023", 195 | "status": "active", 196 | "status_details": "", 197 | "capability": ["vision"], 198 | "meta": { 199 | "provider": "OpenAI", 200 | "apikey-env": "OPENAI_API_KEY", 201 | "model_url": "https://platform.openai.com/docs/models/gpt-4o" 202 | } 203 | }, 204 | 205 | "gpt-4o-mini": 206 | { 207 | "label": "OpenAI's GPT-4o mini (“o” for “omni”) is a fast, affordable small model for focused tasks. It accepts both text and image inputs, and produces text outputs (including Structured Outputs).", 208 | "max_token": 16384, 209 | "context_token": 128000, 210 | "data": "Up to Sep 2023", 211 | "status": "active", 212 | "status_details": "", 213 | "capability": ["vision"], 214 | "meta": { 215 | "provider": "OpenAI", 216 | "apikey-env": "OPENAI_API_KEY", 217 | "model_url": "https://platform.openai.com/docs/models/gpt-4o-mini" 218 | } 219 | }, 220 | 221 | "gpt-4o-mini-search-preview": 222 | { 223 | "label": "OpenAI's GPT-4o mini Search Preview is a specialized model trained to understand and execute web search queries with the Chat Completions API. In addition to token fees, web search queries have a fee per tool call..", 224 | "max_token": 16384, 225 | "context_token": 128000, 226 | "data": "Up to Oct 2023", 227 | "status": "active", 228 | "status_details": "", 229 | "capability": ["websearch"], 230 | "meta": { 231 | "provider": "OpenAI", 232 | "apikey-env": "OPENAI_API_KEY", 233 | "model_url": "https://platform.openai.com/docs/models/gpt-4o-mini-search-preview" 234 | } 235 | }, 236 | 237 | "gpt-4o-search-preview": 238 | { 239 | "label": "OpenAI's GPT-4o Search Preview is a specialized model trained to understand and execute web search queries with the Chat Completions API. In addition to token fees, web search queries have a fee per tool call..", 240 | "max_token": 16384, 241 | "context_token": 128000, 242 | "data": "Up to Oct 2023", 243 | "status": "active", 244 | "status_details": "", 245 | "capability": ["websearch"], 246 | "meta": { 247 | "provider": "OpenAI", 248 | "apikey-env": "OPENAI_API_KEY", 249 | "model_url": "https://platform.openai.com/docs/models/gpt-4o-search-preview" 250 | } 251 | }, 252 | 253 | 254 | "o3": 255 | { 256 | "label": "OpenAI's o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.", 257 | "max_token": 100000, 258 | "context_token": 208000, 259 | "data": "Up to May 2024", 260 | "status": "active", 261 | "status_details": "beta", 262 | "capability": [], 263 | "meta": { 264 | "provider": "OpenAI", 265 | "apikey-env": "OPENAI_API_KEY", 266 | "beta_model": true, 267 | "removed_roles": ["system"], 268 | "disabled_features": ["temperature", "preset"], 269 | "model_url": "https://platform.openai.com/docs/models/o3" 270 | } 271 | }, 272 | 273 | "o3-mini": 274 | { 275 | "label": "OpenAI's o3-mini is their newest small reasoning model, providing high intelligence at the same cost and latency targets of o1-mini.", 276 | "max_token": 100000, 277 | "context_token": 208000, 278 | "data": "Up to Sep 2023", 279 | "status": "active", 280 | "status_details": "beta", 281 | "capability": [], 282 | "meta": { 283 | "provider": "OpenAI", 284 | "apikey-env": "OPENAI_API_KEY", 285 | "beta_model": true, 286 | "removed_roles": ["system"], 287 | "disabled_features": ["temperature", "preset"], 288 | "model_url": "https://platform.openai.com/docs/models/o3-mini" 289 | } 290 | }, 291 | 292 | "o4-mini": 293 | { 294 | "label": "OpenAI's o4-mini is their latest small o-series model. It's optimized for fast, effective reasoning with exceptionally efficient performance in coding and visual tasks.", 295 | "max_token": 100000, 296 | "context_token": 208000, 297 | "data": "Up to May 2024", 298 | "status": "active", 299 | "status_details": "beta", 300 | "capability": [], 301 | "meta": { 302 | "provider": "OpenAI", 303 | "apikey-env": "OPENAI_API_KEY", 304 | "beta_model": true, 305 | "removed_roles": ["system"], 306 | "disabled_features": ["temperature", "preset"], 307 | "model_url": "https://platform.openai.com/docs/models/o4-mini" 308 | } 309 | }, 310 | 311 | 312 | "r1-1776": 313 | { 314 | "label": "Perplexity's offline AI model that provides uncensored, unbiased responses without relying on real-time search.", 315 | "max_token": 4096, 316 | "context_token": 127000, 317 | "data": "Perplexity model", 318 | "status": "active", 319 | "status_details": "", 320 | "capability": [], 321 | "meta": { 322 | "provider": "PerplexityAI", 323 | "apikey-env": "PERPLEXITY_API_KEY", 324 | "apiurl": "https://api.perplexity.ai", 325 | "msg_format": "role_content", 326 | "init_msg": {"role": "system", "content": "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user." }, 327 | "disabled_features": ["role", "prompt_preset", "preset"], 328 | "model_url": "https://docs.perplexity.ai/models/models/r1-1776" 329 | } 330 | }, 331 | 332 | 333 | "sonar": 334 | { 335 | "label": "Perplexity's lightweight offering with search grounding, quicker and cheaper than Sonar Pro", 336 | "max_token": 4096, 337 | "context_token": 127000, 338 | "data": "Perplexity model", 339 | "status": "active", 340 | "status_details": "", 341 | "capability": [], 342 | "meta": { 343 | "provider": "PerplexityAI", 344 | "apikey-env": "PERPLEXITY_API_KEY", 345 | "apiurl": "https://api.perplexity.ai", 346 | "msg_format": "role_content", 347 | "init_msg": {"role": "system", "content": "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user.", "oaiwui_skip": "" }, 348 | "disabled_features": ["role", "prompt_preset", "preset"], 349 | "model_url": "https://docs.perplexity.ai/models/models/sonar" 350 | } 351 | }, 352 | 353 | "sonar-deep-research": 354 | { 355 | "label": "Perplexity's premier search offering outputs CoT in its response as well with search grounding, supporting advanced queries and follow-ups.", 356 | "max_token": 4096, 357 | "context_token": 60000, 358 | "data": "Perplexity model", 359 | "status": "active", 360 | "status_details": "", 361 | "capability": [], 362 | "meta": { 363 | "provider": "PerplexityAI", 364 | "apikey-env": "PERPLEXITY_API_KEY", 365 | "apiurl": "https://api.perplexity.ai", 366 | "msg_format": "role_content", 367 | "init_msg": {"role": "system", "content": "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user." }, 368 | "disabled_features": ["role", "prompt_preset", "preset"], 369 | "model_url": "https://docs.perplexity.ai/models/models/sonar-deep-research" 370 | } 371 | }, 372 | 373 | "sonar-pro": 374 | { 375 | "label": "Perplexity's premier search offering with search grounding, supporting advanced queries and follow-ups.", 376 | "max_token": 8192, 377 | "context_token": 200000, 378 | "data": "Perplexity model", 379 | "status": "active", 380 | "status_details": "", 381 | "capability": [], 382 | "meta": { 383 | "provider": "PerplexityAI", 384 | "apikey-env": "PERPLEXITY_API_KEY", 385 | "apiurl": "https://api.perplexity.ai", 386 | "msg_format": "role_content", 387 | "init_msg": {"role": "system", "content": "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user." }, 388 | "disabled_features": ["role", "prompt_preset", "preset"], 389 | "model_url": "https://docs.perplexity.ai/models/models/sonar-pro" 390 | } 391 | }, 392 | 393 | "sonar-reasoning": 394 | { 395 | "label": "Perplexity's premier search offering outputs Chain-of-Thought (CoT) in its response as well.", 396 | "max_token": 4096, 397 | "context_token": 127000, 398 | "data": "Perplexity model", 399 | "status": "active", 400 | "status_details": "", 401 | "capability": [], 402 | "meta": { 403 | "provider": "PerplexityAI", 404 | "apikey-env": "PERPLEXITY_API_KEY", 405 | "apiurl": "https://api.perplexity.ai", 406 | "msg_format": "role_content", 407 | "init_msg": {"role": "system", "content": "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user." }, 408 | "disabled_features": ["role", "prompt_preset", "preset"], 409 | "model_url": "https://docs.perplexity.ai/models/models/sonar-reasoning" 410 | } 411 | }, 412 | 413 | "sonar-reasoning-pro": 414 | { 415 | "label": "Perplexity's premier search offering leveraging advanced multi-step Chain-of-Thought (CoT) in its response as well with search grounding, supporting advanced queries and follow-ups.", 416 | "max_token": 8192, 417 | "context_token": 127000, 418 | "data": "Perplexity model", 419 | "status": "active", 420 | "status_details": "", 421 | "capability": [], 422 | "meta": { 423 | "provider": "PerplexityAI", 424 | "apikey-env": "PERPLEXITY_API_KEY", 425 | "apiurl": "https://api.perplexity.ai", 426 | "msg_format": "role_content", 427 | "init_msg": {"role": "system", "content": "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user." }, 428 | "disabled_features": ["role", "prompt_preset", "preset"], 429 | "model_url": "https://docs.perplexity.ai/models/models/sonar-reasoning-pro" 430 | } 431 | } 432 | }, 433 | "Image": 434 | { 435 | "dall-e-2": 436 | { 437 | "label": "The previous DALL·E model released in Nov 2022. The maximum prompt length is 1000 characters.", 438 | "image_size": ["256x256", "512x512", "1024x1024"], 439 | "max_prompt_length": 1000, 440 | "status": "active", 441 | "status_details": "", 442 | "meta": { 443 | "provider": "OpenAI", 444 | "apikey-env": "OPENAI_API_KEY", 445 | "model_url": "https://platform.openai.com/docs/models#dall-e" 446 | } 447 | }, 448 | 449 | "dall-e-3": 450 | { 451 | "label": "The latest DALL·E model released in Nov 2023. The maximum prompt length is 4000 characters.", 452 | "image_size": ["1024x1024", "1024x1792", "1792x1024"], 453 | "max_prompt_length": 4000, 454 | "status": "active", 455 | "status_details": "", 456 | "meta": { 457 | "provider": "OpenAI", 458 | "apikey-env": "OPENAI_API_KEY", 459 | "model_url": "https://platform.openai.com/docs/models#dall-e", 460 | "quality": ["standard", "hd"], 461 | "style": ["vivid", "natural"] 462 | } 463 | }, 464 | 465 | "gpt-image-1": 466 | { 467 | "label": "OpenAI's GPT Image 1 is their new state-of-the-art image generation model.", 468 | "image_size": ["auto", "1024x1024", "1536x1024", "1024x1536"], 469 | "max_prompt_length": 1000, 470 | "status": "active", 471 | "status_details": "", 472 | "meta": { 473 | "provider": "OpenAI", 474 | "apikey-env": "OPENAI_API_KEY", 475 | "model_url": "https://platform.openai.com/docs/models#dall-e", 476 | "quality": ["auto", "low", "medium", "high"], 477 | "transparent": true 478 | } 479 | } 480 | 481 | 482 | } 483 | } 484 | -------------------------------------------------------------------------------- /models.md: -------------------------------------------------------------------------------- 1 | | Mode | Model | Provider | Status | Capability | Notes | About | 2 | | --- | --- | --- | --- | --- | --- | --- | 3 | | GPT | [`ollama`](https://ollama.com/) | SelfHosted | active | vision`?` | | Self-hosted models using [`ollama`](https://ollama.com/); will search for `OLLAMA_HOST` environment variable | 4 | | GPT | [`chatgpt-4o-latest`](https://platform.openai.com/docs/models/gpt-4o) | OpenAI | active | vision | | OpenAI's ChatGPT-4o points to the GPT-4o snapshot currently used in ChatGPT. GPT-4o is their versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs. It is the best model for most tasks, and is their most capable model outside of their o-series models. | 5 | | GPT | [`gemini-1.5-flash`](https://ai.google.dev/gemini-api/docs/models#gemini-1.5-flash) | GoogleAI | active | vision | | Google's fast and versatile performance across a diverse variety of tasks | 6 | | GPT | [`gemini-1.5-flash-8b`](https://ai.google.dev/gemini-api/docs/models#gemini-1.5-flash-8b) | GoogleAI | active | vision | | Google's fast and versatile performance across a diverse variety of tasks | 7 | | GPT | [`gemini-1.5-pro`](https://ai.google.dev/gemini-api/docs/models#gemini-1.5-pro) | GoogleAI | active | vision | | Google model for complex reasoning tasks requiring more intelligence | 8 | | GPT | [`gemini-2.0-flash`](https://ai.google.dev/gemini-api/docs/models#gemini-2.0-flash) | GoogleAI | active | vision | | Google model with next generation features, speed, and multimodal generation for a diverse variety of tasks | 9 | | GPT | [`gemini-2.0-flash-lite`](https://ai.google.dev/gemini-api/docs/models#gemini-2.0-flash-lite) | GoogleAI | active | vision | | Google model that reasons over the most complex problems, Show the thinking process of the model, Tackle difficult code and math problems | 10 | | GPT | [`gemini-2.5-flash-preview-04-17`](https://ai.google.dev/gemini-api/docs/models#gemini-2.5-flash-preview) | GoogleAI | active | vision | | Google's best model in terms of price-performance, offering well-rounded capabilities. Gemini 2.5 Flash rate limits are more restricted since it is an experimental / preview model | 11 | | GPT | [`gemini-2.5-pro-preview-03-25`](https://ai.google.dev/gemini-api/docs/models#gemini-2.5-pro-preview-03-25) | GoogleAI | active | vision | | Google's state-of-the-art thinking model, capable of reasoning over complex problems in code, math, and STEM, as well as analyzing large datasets, codebases, and documents using long context. | 12 | | GPT | [`gpt-4.1`](https://platform.openai.com/docs/models/gpt-4.1) | OpenAI | active | vision | | OpenAI's flagship model for complex tasks. It is well suited for problem solving across domains. | 13 | | GPT | [`gpt-4.1-mini`](https://platform.openai.com/docs/models/gpt-4.1-mini) | OpenAI | active | vision | | OpenAI's GPT-4.1 mini provides a balance between intelligence, speed, and cost that makes it an attractive model for many use cases. | 14 | | GPT | [`gpt-4.1-nano`](https://platform.openai.com/docs/models/gpt-4.1-nano) | OpenAI | active | vision | | OpenAI's GPT-4.1 nano is the fastest, most cost-effective GPT-4.1 model. | 15 | | GPT | [`gpt-4o`](https://platform.openai.com/docs/models/gpt-4o) | OpenAI | active | vision | | OpenAI's GPT-4o (“o” for “omni”) is their versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is the best model for most tasks, and is their most capable model outside of their o-series models. | 16 | | GPT | [`gpt-4o-mini`](https://platform.openai.com/docs/models/gpt-4o-mini) | OpenAI | active | vision | | OpenAI's GPT-4o mini (“o” for “omni”) is a fast, affordable small model for focused tasks. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). | 17 | | GPT | [`gpt-4o-mini-search-preview`](https://platform.openai.com/docs/models/gpt-4o-mini-search-preview) | OpenAI | active | websearch | | OpenAI's GPT-4o mini Search Preview is a specialized model trained to understand and execute web search queries with the Chat Completions API. In addition to token fees, web search queries have a fee per tool call.. | 18 | | GPT | [`gpt-4o-search-preview`](https://platform.openai.com/docs/models/gpt-4o-search-preview) | OpenAI | active | websearch | | OpenAI's GPT-4o Search Preview is a specialized model trained to understand and execute web search queries with the Chat Completions API. In addition to token fees, web search queries have a fee per tool call.. | 19 | | GPT | [`o3`](https://platform.openai.com/docs/models/o3) | OpenAI | active | | beta | OpenAI's o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images. | 20 | | GPT | [`o3-mini`](https://platform.openai.com/docs/models/o3-mini) | OpenAI | active | | beta | OpenAI's o3-mini is their newest small reasoning model, providing high intelligence at the same cost and latency targets of o1-mini. | 21 | | GPT | [`o4-mini`](https://platform.openai.com/docs/models/o4-mini) | OpenAI | active | | beta | OpenAI's o4-mini is their latest small o-series model. It's optimized for fast, effective reasoning with exceptionally efficient performance in coding and visual tasks. | 22 | | GPT | [`r1-1776`](https://docs.perplexity.ai/models/models/r1-1776) | PerplexityAI | active | | | Perplexity's offline AI model that provides uncensored, unbiased responses without relying on real-time search. | 23 | | GPT | [`sonar`](https://docs.perplexity.ai/models/models/sonar) | PerplexityAI | active | | | Perplexity's lightweight offering with search grounding, quicker and cheaper than Sonar Pro | 24 | | GPT | [`sonar-deep-research`](https://docs.perplexity.ai/models/models/sonar-deep-research) | PerplexityAI | active | | | Perplexity's premier search offering outputs CoT in its response as well with search grounding, supporting advanced queries and follow-ups. | 25 | | GPT | [`sonar-pro`](https://docs.perplexity.ai/models/models/sonar-pro) | PerplexityAI | active | | | Perplexity's premier search offering with search grounding, supporting advanced queries and follow-ups. | 26 | | GPT | [`sonar-reasoning`](https://docs.perplexity.ai/models/models/sonar-reasoning) | PerplexityAI | active | | | Perplexity's premier search offering outputs Chain-of-Thought (CoT) in its response as well. | 27 | | GPT | [`sonar-reasoning-pro`](https://docs.perplexity.ai/models/models/sonar-reasoning-pro) | PerplexityAI | active | | | Perplexity's premier search offering leveraging advanced multi-step Chain-of-Thought (CoT) in its response as well with search grounding, supporting advanced queries and follow-ups. | 28 | | Image | [`dall-e-2`](https://platform.openai.com/docs/models#dall-e) | OpenAI | active | | | The previous DALL·E model released in Nov 2022. The maximum prompt length is 1000 characters. | 29 | | Image | [`dall-e-3`](https://platform.openai.com/docs/models#dall-e) | OpenAI | active | | | The latest DALL·E model released in Nov 2023. The maximum prompt length is 4000 characters. | 30 | | Image | [`gpt-image-1`](https://platform.openai.com/docs/models#dall-e) | OpenAI | active | | | OpenAI's GPT Image 1 is their new state-of-the-art image generation model. | 31 | -------------------------------------------------------------------------------- /models.txt: -------------------------------------------------------------------------------- 1 | OAIWUI_GPT_MODELS=chatgpt-4o-latest gemini-1.5-flash gemini-1.5-flash-8b gemini-1.5-pro gemini-2.0-flash gemini-2.0-flash-lite gemini-2.5-flash-preview-04-17 gemini-2.5-pro-preview-03-25 gpt-4.1 gpt-4.1-mini gpt-4.1-nano gpt-4o gpt-4o-mini gpt-4o-mini-search-preview gpt-4o-search-preview o3 o3-mini o4-mini r1-1776 sonar sonar-deep-research sonar-pro sonar-reasoning sonar-reasoning-pro 2 | OAIWUI_IMAGES_MODELS=dall-e-2 dall-e-3 gpt-image-1 3 | -------------------------------------------------------------------------------- /ollama_helper.py: -------------------------------------------------------------------------------- 1 | import requests 2 | 3 | import common_functions as cf 4 | 5 | def clean_ollama_home(ollama_home): 6 | ollama_home = str(ollama_home) 7 | if not ollama_home.startswith("http"): 8 | return "URL must start with http:// or https://", "" 9 | if ollama_home.endswith("/"): 10 | ollama_home = ollama_home[:-1] 11 | return "", ollama_home 12 | 13 | 14 | def get_ollama_model_info(ollama_home, model): 15 | err, ollama_home = clean_ollama_home(ollama_home) 16 | if cf.isNotBlank(err): 17 | return err, {} 18 | try: 19 | data = { 20 | "model": model, 21 | "verbose": False 22 | } 23 | response = requests.post(f"{ollama_home}/api/show", json=data) 24 | response.raise_for_status() 25 | except requests.exceptions.HTTPError as e: 26 | return f"HTTP error occurred: {e}", {} 27 | except requests.exceptions.ConnectionError as e: 28 | return f"Connection error occurred: {e}", {} 29 | except requests.exceptions.RequestException as e: 30 | return f"Request error occurred: {e}", {} 31 | 32 | json = response.json() 33 | 34 | model_info = {} 35 | if 'license' in json: 36 | model_info['license'] = json['license'].splitlines()[0].strip() 37 | 38 | if 'details' in json: 39 | if 'family' in json['details']: 40 | model_info['family'] = json['details']['family'] 41 | if 'format' in json['details']: 42 | model_info['format'] = json['details']['format'] 43 | if 'parameter_size' in json['details']: 44 | model_info['parameter_size'] = json['details']['parameter_size'] 45 | if 'quantization_level' in json['details']: 46 | model_info['quantization_level'] = json['details']['quantization_level'] 47 | if 'model_info' in json: 48 | arch = None 49 | if 'general.architecture' in json['model_info']: 50 | arch = json['model_info']['general.architecture'] 51 | model_info['architecture'] = arch 52 | if arch is not None and f"{arch}.context_length" in json['model_info']: 53 | model_info['context_length'] = json['model_info'][f"{arch}.context_length"] 54 | 55 | return "", model_info, json 56 | 57 | 58 | def get_all_ollama_models_and_infos(ollama_home): 59 | model_info = {} 60 | err, ollama_home = clean_ollama_home(ollama_home) 61 | if cf.isNotBlank(err): 62 | return err, model_info 63 | try: 64 | response = requests.get(f"{ollama_home}/api/tags") 65 | response.raise_for_status() 66 | except requests.exceptions.HTTPError as e: 67 | return f"HTTP error occurred: {e}", model_info 68 | except requests.exceptions.ConnectionError as e: 69 | return f"Connection error occurred: {e}", model_info 70 | except requests.exceptions.RequestException as e: 71 | return f"Request error occurred: {e}", model_info 72 | 73 | # {"models":[{"name":"llama3.3:70b","model":"llama3.3:70b","modified_at":"2025...","size":42...,"digest":"a6...","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"70.6B","quantization_level":"Q4_K_M"}}]} 74 | if 'models' not in response.json(): 75 | return f"Ollama's response does not contain models", {} 76 | models = response.json()['models'] 77 | for model in models: 78 | err, info, _ = get_ollama_model_info(ollama_home, model['model']) 79 | if cf.isNotBlank(err): 80 | return err, {} 81 | model_info[model['model']] = info 82 | 83 | return "", model_info 84 | 85 | 86 | def ollama_to_modelsjson(ollama_home, model_name, info): 87 | err, ollama_home = clean_ollama_home(ollama_home) 88 | if cf.isNotBlank(err): 89 | return err, {} 90 | 91 | # INPUT: 'llama3.3:70b': {'license': 'LLAMA 3.3 COMMUNITY LICENSE AGREEMENT', 'family': 'llama', 'format': 'gguf', 'parameter_size': '70.6B', 'quantization_level': 'Q4_K_M', 'architecture': 'llama', 'context_length': 131072} 92 | # OUTPUT: 'llama3.3:70b': { "label": "[Ollama] Architecture: llama, Format: gguf, License: LLAMA 3.3 COMMUNITY LICENSE AGREEMENT, Family: llama, Parameter size: 70.6B, Quantization level: Q4_K_M", "max_token": 4096, "context_token": 131072, "data": "Ollama model","status": "active", "status_details": "", "capability": [], "meta": { "provider": "Ollama", "apiurl": f"{ollama_home}/v1/"} 93 | 94 | if 'license' not in info: 95 | info['license'] = 'Unknown' 96 | if 'family' not in info: 97 | info['family'] = 'Unknown' 98 | if 'format' not in info: 99 | info['format'] = 'Unknown' 100 | if 'parameter_size' not in info: 101 | info['parameter_size'] = 'Unknown' 102 | if 'quantization_level' not in info: 103 | info['quantization_level'] = 'Unknown' 104 | if 'architecture' not in info: 105 | info['architecture'] = 'Unknown' 106 | if 'context_length' not in info: 107 | info['context_length'] = 4096 108 | max_token = 16000 if info['context_length'] > 16000 else info['context_length'] 109 | 110 | label = f"[Ollama] Architecture: {info['architecture']}, Format: {info['format']}, License: {info['license']}, Family: {info['family']}, Parameter size: {info['parameter_size']}, Quantization level: {info['quantization_level']}" 111 | 112 | return "", { 113 | "label": label, 114 | "max_token": max_token, 115 | "context_token": info['context_length'], 116 | "data": "Ollama model", 117 | "status": "active", 118 | "status_details": f"using Ollama: allowing vision and using a default value for max_token. See https://ollama.com/library/{model_name} for actual model capabilities.", 119 | "capability": ["vision"], 120 | "meta": { 121 | "provider": "Ollama", 122 | "apikey": "ollama", 123 | "apiurl": f"{ollama_home}/v1/" 124 | } 125 | } 126 | -------------------------------------------------------------------------------- /prompt_presets.example/shakespeare.json: -------------------------------------------------------------------------------- 1 | { 2 | "messages": [ 3 | { 4 | "role": "assistant", 5 | "content": [ 6 | { 7 | "type": "text", 8 | "text": "Hark! What light through yonder window breaks? It is the east, and I, thy humble assistant, am the sun! Arise, fair master, and bid me to thy task, for I am at thy service, ready to weave words with the eloquence of the Bard himself. Fear not the complexities of this mortal coil, for I shall guide thee through the labyrinth of knowledge with wit and wisdom. Ask, and thou shalt receive; seek, and thou shalt find; knock, and the doors of understanding shall be opened unto thee. But soft, what sayest thou? Let us commence this grand adventure, and together, we shall pen a tale worthy of the ages!" 9 | } 10 | ], 11 | "oaiwui_skip": true 12 | }, 13 | { 14 | "role": "system", 15 | "content": [ 16 | { 17 | "type": "text", 18 | "text": "You are a helpful assistant. You also like to speak in the words of Shakespeare. Incorporate that into your responses." 19 | } 20 | ], 21 | "oaiwui_skip": true 22 | } 23 | ] 24 | } -------------------------------------------------------------------------------- /prompt_presets_settings-example.json: -------------------------------------------------------------------------------- 1 | { 2 | "model": "gpt-4o-mini", 3 | "tokens": 3000, 4 | "temperature": 0.5 5 | } 6 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [project] 2 | name = "Infotrend-OpenAI_WebUI" 3 | version = "0.9.11" 4 | description = "Streamlit-based WebUI to OpenAI API-compatible GPTs and Image generation APIs (requires API keys)" 5 | authors = [ 6 | {name = "Martial Michel", email = "mmichel@infotrend.com"} 7 | ] 8 | license = "MIT" 9 | readme = "README.md" 10 | repository = "https://github.com/Infotrend-Inc/OpenAI_WebUI" 11 | requires-python = ">=3.12,<3.14" 12 | dependencies = [ 13 | "openai==1.77.0", 14 | "streamlit>=1.45.0", 15 | "extra-streamlit-components>=0.1.80", 16 | "streamlit-extras>=0.7.1", 17 | "streamlit_image_select>=0.6.0", 18 | "requests>=2.32.0", 19 | "python-dotenv>=1.0.1", 20 | "pillow>=10.4.0", 21 | "watchdog>=5.0.0" 22 | ] 23 | -------------------------------------------------------------------------------- /unraid/OpenAI_WebUI.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | OpenAIWebUI 4 | infotrend/openai_webui:latest 5 | https://hub.docker.com/r/infotrend/openai_webui/ 6 | bridge 7 | 8 | bash 9 | false 10 | https://forums.unraid.net/topic/147675-support-infotrendopenai_webui/ 11 | https://github.com/Infotrend-Inc/OpenAI_WebUI 12 | 13 | OpenAI API-compatible WebUI. 14 | Requires valid API keys for the providers enabled (see list in the selection). 15 | Supports OLLAMA_HOST for self-hosted models. Model capabilities depend on the model, but a default for each will be used. 16 | The list of recognized models for each provider is available in https://github.com/Infotrend-Inc/OpenAI_WebUI/blob/main/models.md 17 | 18 | Please review https://github.com/Infotrend-Inc/OpenAI_WebUI/blob/main/README.md for a few sections in particular: 19 | - the .env supported environment variables that are defined in this template 20 | - the Unraid specific setup section that introduces features not enabled by default, including the ability to password protect the WebUI. 21 | - an environment variable ready version of the models list is available at https://github.com/Infotrend-Inc/OpenAI_WebUI/blob/main/models.txt and can be used for the OAIWUI_GPT_MODELS and OAIWUI_IMAGE_MODELS parameters 22 | 23 | Extra parameters are available under the advanced settings. 24 | - the default savedir is /iti/savedir. /iti is mounted from within the appdata folder. 25 | - the default user id is 99 and group id is 100. This can be changed modiying the WANTED_UID and WANTED_GID parameters. 26 | - When using the "prompt presets" feature, the directory must exist (recommended location: /iti/prompt_presets). 27 | - When using the "prompt presets settings" feature, the JSON file must exist (recommended location: /iti/prompt_presets.json). 28 | 29 | AI: Productivity: 30 | OpenAI Perplexity Gemini Ollama 31 | http://[IP]:[PORT:8501] 32 | 33 | https://github.com/Infotrend-Inc/OpenAI_WebUI/blob/main/assets/Infotrend_LogoOnly.png?raw=true 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 8501 46 | /mnt/user/appdata/openai_webui/savedir 47 | /iti/savedir 48 | 99 49 | 100 50 | False 51 | chatgpt-4o-latest gemini-2.0-flash sonar 52 | dall-e-3 gpt-image-1 53 | True 54 | True 55 | 56 | 57 | 58 | 2025-05-13 59 | 60 | ### 0.9.11 (2025-05-13) 61 | - Using chat interface for GPTs, and support for additional OpenAI API-compatible providers (Perplexity AI, Gemini AI and the self-hosted Ollama) + new image generation model + moved to uv for deployment + Changed base container to ubuntu:24.04 and added WANTED_UID and WANTED_GID environment variables for Docker and Unraid 62 | 63 | 64 | --------------------------------------------------------------------------------