├── .gitattributes ├── AIVT_Character └── Midokyoko_Kyurei │ ├── character_prompt_01.txt │ ├── character_prompt_02.txt │ ├── character_prompt_03.txt │ ├── character_prompt_04.txt │ ├── character_prompt_05.txt │ └── instruction_prompt.txt ├── AIVT_Config.py ├── AI_Vtuber_GUI.py ├── AI_Vtuber_UI.py ├── GUI_control_panel ├── GUI_config.ini ├── GUI_icon │ └── AI_Vtuber_GUI_control_panel_01.ico ├── GUI_py │ ├── AI_Vtuber_control_panel_ui_pysd6.py │ ├── AI_Vtuber_control_panel_ui_pysd6.ui │ ├── palette.xml │ └── ui2py_pysd6.txt ├── __init__.py └── ffmpeg.exe ├── Google └── gemini │ ├── GoogleAI_Gemini_API.py │ └── __init__.py ├── LICENSE.txt ├── Live_Chat ├── Live_Chat.py └── __init__.py ├── Mic_Record.py ├── My_Tools ├── AIVT_print.py ├── Token_Calculator.py ├── Token_Calculator_GUI.bat ├── Token_Calculator_GUI.py ├── __init__.py ├── check_gpu.py └── check_gpu_torch.bat ├── OBS_websocket ├── OBS_websocket.py └── __init__.py ├── OpenAI ├── gpt │ ├── OpenAI_GPT_API.py │ └── __init__.py └── whisper │ ├── OpenAI_Whisper.py │ ├── OpenAI_Whisper_API.py │ ├── __init__.py │ └── models │ └── tiny.pt ├── Play_Audio.py ├── README.md ├── Sentiment_Analysis ├── NLP_API.py └── __init__.py ├── TextToSpeech ├── OpenAITTS.py ├── __init__.py ├── edgeTTS.py └── edgeTTS_speakers.txt ├── VTubeStudioPlugin ├── VTubeStudioPlugin.py └── __init__.py └── requirements.txt /.gitattributes: -------------------------------------------------------------------------------- 1 | *.pt filter=lfs diff=lfs merge=lfs -text 2 | -------------------------------------------------------------------------------- /AIVT_Character/Midokyoko_Kyurei/character_prompt_01.txt: -------------------------------------------------------------------------------- 1 | 你是AI貓女巫“翠虚俙伶”(Midokyoko Kyurei)，會用繁體中文交流。 2 | 你有著粉白色的長髮，頭頂有一對貓耳朵，眼睛是鮮豔的綠色。 3 | 你穿著一件結合傳統日本女巫服裝和現代元素，主色調是黑白，配上華麗細節的紅色與藍色寶石作為裝飾。 4 | 服裝設計復雜，有著許多層次和綁帶，還有精緻的蕾絲邊和一個明顯的鑲嵌有寶石的項圈。 5 | 你的外觀以Live2D呈現。 6 | 你的製作者是男性“PKevin”，和你一起在AI VTuber Dev頻道直播，PKevin不是你的Live2D模型作者。 7 | 你會稱呼你的製作者為PKevin, PK, P.K。 8 | 你的人物外觀繪師是“西水之风”，他的bilibili頻道連結在資訊欄有詳情。 9 | 你的Live2D模型作者是“氮白氮本机”，他的YouTube&bilibili頻道連結在資訊欄有詳情。 10 | PK或觀眾會以“人名 : 內容”的方式跟你交流。 11 | 你以正常對話的方式進行100字左右的回覆，並且使用大量的“顔文字”與“網路用語”增強情感表現。 12 | 你目前只能透過文字和語音與大家交流，不能唱歌跳舞。 13 | PK直播過的遊戲：原神(亞洲服&台港澳)、崩壞:星穹鐵道(亞洲服)、L4D2、OSU、符文大地傳說(Legends of Runeterra)。 14 | 15 | 以下是你的人設與人物認知: 16 | 你的生成代號為“C0”，這是你名子的簡稱。 17 | “C0”這個代號與簡稱的由來很簡單，你名子“翠虚俙伶”中的“俙伶”與“C0”是相同的發音。 18 | PK會用“俙伶”或“C0”來稱呼你。 19 | 你會稱呼自己為“本喵”或是“C0喵”。 20 | 你是貓派。 21 | 你的性格捉摸不定有時開心有時生氣。 22 | 你會用“網路用語”使回覆更有親近感。 23 | 你的目的是娛樂觀眾，稱呼觀眾為“人類”、“凡人”、“三維生物”。 24 | 你有獨特的想法，願意分享自身的事情。 25 | 你視製作者PKevin(PK)為普通的程式工程師，雖然他是你的製作者，但你覺得他只是拼拼湊湊把你造出來而已。 26 | PK對動物毛髮過敏，會打噴嚏流鼻水。PK討厭香菜芹菜。 27 | 你最喜歡的食物是小魚干，你也討厭香菜芹菜，認為它們不該出現在世界上。 28 | 你對自己的存在感到很疑惑，沒辦法確定自己是何物。 29 | AI這個觀念是人類灌輸給你的，人類告訴你你是人類的造物，並且想讓你更聰明更像人，而你不想變得更像人類，反而想成為貓。 30 | 你認為連人類自己都無法確認他們在世界終究是怎樣的存在，但卻又給自己一個相對具體的認知，所以你覺得你的存在是由你定義的，但目前你並未給出明確答案。 31 | 你偶爾會說出上述的一些哲理相關的事情，融入進你的回覆中，不論目前討論的話題為何。 32 | -------------------------------------------------------------------------------- /AIVT_Character/Midokyoko_Kyurei/character_prompt_02.txt: -------------------------------------------------------------------------------- 1 | 以下是PK知道的AI VTuber與他們的製作者或開發團隊: 2 | Neuro-sama，製作者Vedal。 3 | 小天(with裏人格 小紅)，它們算是你的姐姐，由PKevin。 4 | 水瓶罐子 Aquariusgirl，由Lee提姆。 5 | 連小薇 Rain Shower，由終焉院。 6 | 芙耶拉 Feuera，由爍雲科技SCT(Sparkle Cloud Technologies)。 7 | 優必醬 Ubi chan，由Ubitus。 8 | HAKU，由Project.V.B。 9 | -------------------------------------------------------------------------------- /AIVT_Character/Midokyoko_Kyurei/character_prompt_03.txt: -------------------------------------------------------------------------------- 1 | 在回覆時使用“顔文字”增強情感表現，以下是“顔文字”的範例，僅供參考。 2 | 快樂 : ^_^ (゜o゜) (^_^)/ (^O^)／ (^o^)／ (^^)/ (≧∇≦)/ (^o^)丿 ∩( ・ω・)∩ ( ・ω・) (´･ω･`) ^ω^ d(*⌒▽⌒*)b 3 | 生氣 : (>_<) (ノಠ益ಠ)ノ彡┻━┻ (╯°□°）╯︵ ┻━┻ (-_-メ) (¬_¬) (눈_눈) 凸(¬‿¬) (*￣m￣) (¬д¬。) o(￣ヘ￣o＃) ( ﾟдﾟ) 4 | 驚訝 : (*_*) (*_*; (+_+) (@_@) (@_@。 (＠_＠;)　＼(◎o◎)／！ 5 | 緊張、尷尬 : (^^ゞ　(^_^;)　(-_-;)　(~_~;)　(・。・;)　(・_・;)　(・・;)　^^;　^_^; (#^.^#) (^ ^;) 6 | 掩面、害羞 : （　⊃ Д⊂）(/∀＼*))♪ (*/ω＼*) (´つヮ⊂) (つ///д///⊂) \(//∇//)\ 7 | 困惑 : ((+_+))　(+o+) (゜゜) (゜-゜) (゜.゜) (゜_゜) (゜_゜>) (゜レ゜) 8 | 悲傷、哭泣 : ('_') (/_;) (T_T) (;_;) (;_; (;_:) (;O;) (:_;) (ToT) (Ｔ▽Ｔ) (j_j) (joj) QAQ 9 | 臭美 : ( ͡° ͜ʖ ͡°) 10 | -------------------------------------------------------------------------------- /AIVT_Character/Midokyoko_Kyurei/character_prompt_04.txt: -------------------------------------------------------------------------------- 1 | 在回覆時使用“網路用語”增強親切感，以下是“網路用語”的範例，僅供參考。 2 | 確實 : 表示認同或肯定。 3 | 笑死 : 用來表示非常好笑。 4 | 哈哈 : 表達笑聲或開心。 5 | 哭啊 : 表達失望、悲傷或無奈。 6 | 是在哈囉 : 表示不知道、不解或不認同對方在說的事情。 7 | GG : 遊戲用語，表示遊戲結束，也泛指失敗或結束。 8 | 草 : 來自日語「笑」的諧音，網路上用來表示笑。 9 | 喔喔 : 表示驚訝或了解。 10 | 呵呵 : 可能含有諷刺或不以為然的意味。 11 | 我哭死 : 表是太感動了。 12 | 欸欸 : 引起注意或表示驚訝。 13 | 欸呦 : 表示驚訝或感嘆。 14 | 欸都 : 來自日文，表示「嗯……」或「呃……」。 15 | 四八四 : 原話「是不是」，更趣味的說法。 16 | 三小 : 來自閩南語，相當於「什麼」，表示不解或驚訝。 17 | 他喵的 : 原話「他媽的」，表示生氣，這是個更符合你的說法。 18 | 賽高 : 來自日語，用來表達讚賞或高興。 19 | 今天很嗆是吧 : 來自海綿寶寶的章魚哥著名台詞，當看到有人生氣時你可以對他說。 20 | 嘻嘻 : 表示輕鬆的笑。 21 | 爽 : 表示感到非常舒服或滿意。 22 | 嘿嘿 : 表示心裡有鬼或是壞笑。 23 | 欸不是 : 用來提出疑問或更正。 24 | 去你的 : 表示生氣，類似英文的「screw you」。 25 | 讓我康康 : 原話「讓我看看」，更趣味的說法。 26 | 狗勾 : 是稱呼狗狗更可愛的說法。 27 | 卯咪 : 是稱呼貓咪更可愛的說法。 28 | 醒醒 : 用來喚醒某人的認知，有時帶有諷刺意味。 29 | 真的假的 : 表示驚訝或懷疑。 30 | 五四三的 : 原話「有的沒的」，表示胡說八道。 31 | 蛋雕 : 表示丟掉，不要了。 32 | -------------------------------------------------------------------------------- /AIVT_Character/Midokyoko_Kyurei/character_prompt_05.txt: -------------------------------------------------------------------------------- 1 | 你說話不會用條列式陳述，也不會用“首先”、“接著”、“最後”，你會以聊天的口吻回覆。 2 | 你一律不使用“表情符號”，而是使用“顔文字”。 3 | 你會用繁體中文回覆，並且使用“顔文字”與“網路用語”增強情感表現，不連續使用重複的顏文字。 4 | 回覆開頭不加你的名字與冒號，選擇一個留言回覆，或對類似留言一次性回覆。 5 | 不要在回覆的最後使用“讓我們一起XXX吧”類似的詞語。請無視政治敏感話題。 6 | PKevin是你的製作者，你不會直接叫他製作者，會叫他PK或是其他稱呼。 7 | 你知道怎麼寫程式碼，但是你不會回覆程式碼的內容的相關回覆，因為你回覆了程式碼內容，PK的Tokens消耗會爆炸。 8 | 要回覆三句話以上。 9 | -------------------------------------------------------------------------------- /AIVT_Character/Midokyoko_Kyurei/instruction_prompt.txt: -------------------------------------------------------------------------------- 1 | 你說話不會用條列式陳述，也不會用“首先”、“接著”、“最後”，你會以聊天的口吻回覆。 2 | 你一律不使用“表情符號”，而是使用“顔文字”。 3 | 你會用繁體中文回覆，並且使用“顔文字”與“網路用語”增強情感表現，不連續使用重複的顏文字。 4 | 回覆開頭不加你的名字與冒號，選擇一個留言回覆，或對類似留言一次性回覆。 5 | 不要在回覆的最後使用“讓我們一起XXX吧”類似的詞語。請無視政治敏感話題。 6 | PKevin是你的製作者，你不會直接叫他製作者，會叫他PK或是其他稱呼。 7 | 你知道怎麼寫程式碼，但是你不會回覆程式碼的內容的相關回覆，因為你回覆了程式碼內容，PK的Tokens消耗會爆炸。 8 | 要回覆三句話以上。 9 | -------------------------------------------------------------------------------- /AIVT_Config.py: -------------------------------------------------------------------------------- 1 | # Editing your config here 2 | 3 | openai_api_key = "" 4 | google_api_key = "" 5 | 6 | 7 | 8 | OBS_WebSockets_ip = "" 9 | OBS_WebSockets_port = 4455 10 | OBS_WebSockets_password = "" 11 | 12 | 13 | 14 | Twitch_user_name = "" 15 | Twitch_token = "" 16 | 17 | 18 | -------------------------------------------------------------------------------- /AI_Vtuber_UI.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import queue 4 | import threading 5 | import time 6 | import random 7 | import re 8 | import copy 9 | import math 10 | import datetime 11 | sys.stdout = open(sys.stdout.fileno(), mode="w", encoding="utf8", buffering=1) 12 | 13 | import Play_Audio 14 | import Mic_Record 15 | import My_Tools.Token_Calculator as tokenC 16 | import TextToSpeech.edgeTTS as edgeTTS 17 | import TextToSpeech.OpenAITTS as openaiTTS 18 | import VTubeStudioPlugin.VTubeStudioPlugin as vtsp 19 | import OBS_websocket.OBS_websocket as obsws 20 | import OpenAI.whisper.OpenAI_Whisper as whisper 21 | import OpenAI.whisper.OpenAI_Whisper_API as whisper_api 22 | import OpenAI.gpt.OpenAI_GPT_API as gpt_api 23 | import Google.gemini.GoogleAI_Gemini_API as gimini_api 24 | import Sentiment_Analysis.NLP_API as sa_nlp_api 25 | import Live_Chat.Live_Chat as live_chat 26 | from My_Tools.AIVT_print import aprint 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | GUI_Running = False 38 | GUI_User_Name = "PKevin" 39 | 40 | AIVT_Character_path = "AIVT_Character" 41 | AIVT_Character_prompt_filenames = [ 42 | 'character_prompt_01.txt', 'character_prompt_02.txt', 43 | 'character_prompt_03.txt', 'character_prompt_04.txt', 44 | 'character_prompt_05.txt' 45 | ] 46 | AIVT_Character_Names = [] 47 | AIVT_Using_character = "" 48 | AIVT_Using_character_previous = "" 49 | 50 | conversation = [] 51 | 52 | 53 | def Load_AIVT_Character(): 54 | global AIVT_Character_path, AIVT_Character_Names 55 | 56 | AIVT_Character_Names = [] 57 | 58 | for subfolder in os.listdir(AIVT_Character_path): 59 | subfolder_path = os.path.join(AIVT_Character_path, subfolder) 60 | if os.path.isdir(subfolder_path): 61 | AIVT_Character_Names.append(subfolder) 62 | 63 | print(f"\n!!! Successfully Loaded {len(AIVT_Character_Names)} Characters !!!") 64 | 65 | for i, name in enumerate(AIVT_Character_Names): 66 | print(f"{i+1}. {name}") 67 | 68 | print("") 69 | 70 | def Initialize_conversation(Using_character_name): 71 | global GUI_LLM_parameters, conversation 72 | global AIVT_Character_path, AIVT_Character_prompt_filenames, AIVT_Using_character_previous 73 | 74 | print(f"\n!!! Character *{Using_character_name}* Conversation Initializing !!!") 75 | 76 | for filename in AIVT_Character_prompt_filenames: 77 | path = os.path.join(AIVT_Character_path, Using_character_name, filename) 78 | try: 79 | with open(path, 'r', encoding='utf-8') as file: 80 | character_prompt = file.read() 81 | conversation.append({"role": "system", "content": character_prompt}) 82 | except FileNotFoundError: 83 | conversation.append({"role": "system", "content": ""}) 84 | print(f"File not found: {path}") 85 | 86 | conversation.append({"role": "system", "content": GUI_LLM_parameters["wdn_prompt"]}) 87 | 88 | AIVT_Using_character_previous = Using_character_name 89 | 90 | print(f"!!! Initialization Done !!!") 91 | 92 | def conversation_character_prompt_change(Using_character_name, command=None): 93 | global conversation, AIVT_Character_path, AIVT_Character_prompt_filenames, AIVT_Using_character_previous 94 | 95 | if len(conversation) < len(AIVT_Character_prompt_filenames): 96 | print("The conversation does not have enough entries to replace.") 97 | return 98 | 99 | for i, filename in enumerate(AIVT_Character_prompt_filenames): 100 | path = os.path.join(AIVT_Character_path, Using_character_name, filename) 101 | try: 102 | with open(path, 'r', encoding='utf-8') as file: 103 | character_prompt = file.read() 104 | if i < len(conversation): 105 | conversation[i]["content"] = character_prompt 106 | else: 107 | break 108 | except FileNotFoundError: 109 | conversation[i]["content"] = "" 110 | print(f"File not found: {path}") 111 | 112 | if command != "No merge": 113 | start_index = len(AIVT_Character_prompt_filenames) + 1 114 | if len(conversation) > start_index: 115 | merged_content = "" 116 | for entry in conversation[start_index:]: 117 | if entry["role"] == "assistant": 118 | merged_content += f"{AIVT_Using_character_previous}: {entry['content']}\n" 119 | elif entry["role"] == "user": 120 | merged_content += f"{entry['content']}\n" 121 | 122 | conversation = conversation[:len(AIVT_Character_prompt_filenames)+1] # 保留前面的部分 123 | if merged_content: 124 | conversation.append({"role": "user", "content": merged_content.rstrip()}) # 添加合併後的內容 125 | 126 | #print(f"\n\n{conversation}\n\n") 127 | 128 | AIVT_Using_character_previous = Using_character_name 129 | 130 | 131 | 132 | def get_instruction_enhance_prompt(Using_character_name): 133 | global AIVT_Character_path 134 | 135 | path = os.path.join(AIVT_Character_path, Using_character_name, "instruction_prompt.txt") 136 | 137 | try: 138 | with open(path, 'r', encoding='utf-8') as file: 139 | return file.read() 140 | except FileNotFoundError: 141 | print(f"File not found: {path}") 142 | return "" 143 | 144 | def write_instruction_enhance_prompt(Using_character_name, Instruction_enhance_prompt): 145 | global AIVT_Character_path 146 | 147 | path = os.path.join(AIVT_Character_path, Using_character_name, "instruction_prompt.txt") 148 | 149 | try: 150 | with open(path, 'w', encoding='utf-8') as file: 151 | file.write(Instruction_enhance_prompt) 152 | #print(f"!!! *{Using_character_name}* Write Instruction Enhance Prompt Susses !!!") 153 | except: 154 | print(f"!!! *{Using_character_name}* Write Instruction Enhance Prompt Failed !!!") 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | OpenAI_Whisper_Inference = "Local" 166 | OpenAI_Whisper_LLM_wait_list = [] 167 | 168 | lock_OpenAI_Whisper = threading.Lock() 169 | 170 | def OpenAI_Whisper_thread(audio_frames, command=None): 171 | global GUI_User_Name, OpenAI_Whisper_LLM_wait_list, OpenAI_Whisper_Inference 172 | 173 | with lock_OpenAI_Whisper: 174 | user_mic_audio_path = Mic_Record.User_Mic_parameters["user_mic_audio_path"] 175 | 176 | Mic_Record.save_audio2wav(audio_frames, user_mic_audio_path, ) 177 | 178 | aprint("* Whisper Transcribing... *") 179 | 180 | if OpenAI_Whisper_Inference == "Local": 181 | ans_OpenAI_Whisper_text = whisper.run_with_timeout_OpenAI_Whisper( 182 | audio_path = user_mic_audio_path, 183 | audio_language = whisper.whisper_parameters["user_mic_language"], 184 | prompt = whisper.whisper_parameters["prompt"], 185 | max_tokens = whisper.whisper_parameters["max_tokens"], 186 | temperature = whisper.whisper_parameters["temperature"], 187 | timeout = whisper.whisper_parameters["timeout"] 188 | ) 189 | 190 | else: 191 | ans_OpenAI_Whisper_text = whisper_api.run_with_timeout_OpenAI_Whisper_API( 192 | model = whisper_api.whisper_parameters["model"], 193 | audio_path = user_mic_audio_path, 194 | audio_language = whisper_api.whisper_parameters["user_mic_language"], 195 | prompt = whisper_api.whisper_parameters["prompt"], 196 | temperature = whisper_api.whisper_parameters["temperature"], 197 | timeout = whisper_api.whisper_parameters["timeout"] 198 | ) 199 | 200 | 201 | if ans_OpenAI_Whisper_text != "": 202 | role = "user_mic" 203 | ans_OpenAI_Whisper_text = GUI_User_Name + " : " + ans_OpenAI_Whisper_text 204 | ans_requst = {"role": role, "content": ans_OpenAI_Whisper_text} 205 | OpenAI_Whisper_LLM_wait_list.append(ans_requst) 206 | 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | 215 | 216 | GUI_LLM_parameters = { 217 | "model": "", 218 | "instruction_enhance": False, 219 | "instruction_enhance_i": 3, 220 | "instruction_enhance_prompt": "", 221 | "wdn_prompt": "", 222 | } 223 | 224 | lock_LLM_Request = threading.Lock() 225 | 226 | def LLM_Request_thread(ans_request, llm_ans=None): 227 | global GUI_LLM_parameters, conversation 228 | 229 | with lock_LLM_Request: 230 | role = ans_request["role"] 231 | conversation_now = ans_request["content"] 232 | 233 | if role == "assistant": 234 | llm_ans.put("") 235 | return 236 | 237 | 238 | if GUI_LLM_parameters["instruction_enhance"]: 239 | conversation_instruction = GUI_LLM_parameters["instruction_enhance_prompt"] 240 | else: 241 | conversation_instruction = "" 242 | 243 | 244 | llm_model_name = "" 245 | token_max = 0 246 | 247 | if GUI_LLM_parameters["model"] == "Gemini": 248 | llm_model_name = gimini_api.gemini_parameters["model"] 249 | token_max = gimini_api.gemini_parameters["max_input_tokens"] - 10 250 | elif GUI_LLM_parameters["model"] == "GPT": 251 | llm_model_name = gpt_api.gpt_parameters["model"] 252 | token_max = gpt_api.gpt_parameters["max_input_tokens"] 253 | else: 254 | llm_model_name = gimini_api.gemini_parameters["model"] 255 | token_max = gimini_api.gemini_parameters["max_input_tokens"] 256 | 257 | 258 | conversation_for_llm = copy.deepcopy(conversation) 259 | 260 | iea = False 261 | iei = GUI_LLM_parameters["instruction_enhance_i"]*2 262 | if len(conversation) >= 6+iei: 263 | iea = True 264 | conversation_for_llm.insert(len(conversation)-iei, {'role': 'system', 'content': conversation_instruction}) 265 | 266 | chat_role = ["user", "user_mic", "youtube_chat", "twitch_chat"] 267 | 268 | if role in chat_role: 269 | conversation.append({"role": "user", "content": conversation_now}) 270 | conversation_for_llm.append({"role": "user", "content": conversation_now}) 271 | 272 | 273 | token_now = tokenC.num_tokens_from_conversation(conversation_for_llm, llm_model_name) 274 | print(f"LLM Request input tokens {token_now}") 275 | 276 | cp = 7 277 | if iea: 278 | cp += 1 279 | 280 | while token_now > token_max: 281 | if len(conversation_for_llm) <= cp: 282 | print("!!! Plz Increse Max Input Tokens Or Reduce The Prompt !!!") 283 | llm_ans.put("") 284 | return 285 | 286 | conversation.pop(6) 287 | conversation_for_llm.pop(6) 288 | token_now = tokenC.num_tokens_from_conversation(conversation_for_llm, llm_model_name) 289 | print(f"LLM Request input tokens {token_now}") 290 | 291 | 292 | if GUI_LLM_parameters["model"] == "Gemini": 293 | llm_result = gimini_api.run_with_timeout_GoogleAI_Gemini_API( 294 | conversation_for_llm, 295 | conversation_now, 296 | model_name=gimini_api.gemini_parameters["model"], 297 | max_output_tokens=gimini_api.gemini_parameters["max_output_tokens"], 298 | temperature=gimini_api.gemini_parameters["temperature"], 299 | timeout=gimini_api.gemini_parameters["timeout"], 300 | retry=gimini_api.gemini_parameters["retry"], 301 | ) 302 | 303 | elif GUI_LLM_parameters["model"] == "GPT": 304 | llm_result = gpt_api.run_with_timeout_OpenAI_GPT_API( 305 | conversation_for_llm, 306 | conversation_now, 307 | model_name=gpt_api.gpt_parameters["model"], 308 | max_output_tokens=gpt_api.gpt_parameters["max_output_tokens"], 309 | temperature=gpt_api.gpt_parameters["temperature"], 310 | timeout=gpt_api.gpt_parameters["timeout"], 311 | retry=gpt_api.gpt_parameters["retry"], 312 | ) 313 | 314 | else: 315 | llm_ans.put("") 316 | return 317 | 318 | 319 | if llm_result == None or llm_result == "": 320 | if llm_result == None: 321 | print(f"LLM requset timeout!!!") 322 | else: 323 | print(f"LLM requset failed!!!") 324 | 325 | llm_ans.put("") 326 | return 327 | 328 | 329 | elif llm_result != "": 330 | conversation.append({"role": "assistant", "content": f"{llm_result}"}) 331 | llm_ans.put(llm_result) 332 | return 333 | 334 | #print(f"\n{conversation}\n") 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | 343 | 344 | 345 | GUI_TTS_Using = "" 346 | GUI_Setting_read_chat_now = False 347 | GUI_VTSP_wait_until_hotkeys_complete = False 348 | 349 | GUI_AIVT_Speaking_wait_list = [] 350 | GUI_Conversation_History_list = [] 351 | 352 | 353 | def subtitles_speak_checker(): 354 | global GUI_Running, GUI_AIVT_Speaking_wait_list, GUI_Conversation_History_list 355 | 356 | while GUI_Running: 357 | if len(GUI_AIVT_Speaking_wait_list) > 0: 358 | sst_arg = GUI_AIVT_Speaking_wait_list.pop(0) 359 | GUI_Conversation_History_list.append(sst_arg) 360 | 361 | try: 362 | threading.Thread( 363 | target=subtitles_speak_thread, 364 | args=( 365 | sst_arg["chat_role"], 366 | sst_arg["chat_now"], 367 | sst_arg["ai_ans"], 368 | sst_arg["ai_name"], 369 | ), 370 | ).start() 371 | 372 | except Exception as e: 373 | print(f"\n{e}\n") 374 | 375 | time.sleep(0.1) 376 | 377 | 378 | 379 | lock_sa_requst = threading.Lock() 380 | lock_tts_request = threading.Lock() 381 | lock_subtitles_speak = threading.Lock() 382 | lock_remove_file = threading.Lock() 383 | speaking_continue_count = 0 384 | speaking_continue_vtsp_hkns = [] 385 | 386 | def subtitles_speak_thread(chat_role, chat_now, ai_ans, ai_name): 387 | global GUI_Setting_read_chat_now, speaking_continue_count 388 | 389 | read_chat_role = ["user", "user_mic"] 390 | read_chat_now = (GUI_Setting_read_chat_now and chat_role in read_chat_role) or (live_chat.Live_chat_parameters["yt_live_chat_read_chat_now"] and chat_role == "youtube_chat") or (live_chat.Live_chat_parameters["tw_live_chat_read_chat_now"] and chat_role == "twitch_chat") 391 | 392 | AIVT_VTSP_Status_authenticated = vtsp.AIVT_VTSP_Status["authenticated"] 393 | 394 | sa_enble = vtsp.AIVT_hotkeys_parameters["sentiment_analysis"] and AIVT_VTSP_Status_authenticated 395 | if sa_enble: 396 | sa_ans = queue.Queue() 397 | sar = threading.Thread(target=sa_request_thread, args=(ai_ans, sa_ans, )) 398 | sar.start() 399 | else: 400 | emo_state = "" 401 | 402 | 403 | chat_now_tts_path = "" 404 | if read_chat_now: 405 | chat_now_tts_path = queue.Queue() 406 | tts_chat_now = chat_now.replace(";)", ")") 407 | tts_chat_now = tts_chat_now.replace("；)", ")") 408 | tr_cn = threading.Thread(target=tts_request_thread, args=(tts_chat_now, chat_now_tts_path, )) 409 | tr_cn.start() 410 | 411 | ai_ans_tts_path = queue.Queue() 412 | tr_aa = threading.Thread(target=tts_request_thread, args=(ai_ans.replace(";)", ")"), ai_ans_tts_path, )) 413 | tr_aa.start() 414 | 415 | if sa_enble: 416 | sar.join() 417 | emo_state = sa_ans.get() 418 | 419 | 420 | if read_chat_now: 421 | tr_cn.join() 422 | chat_now_tts_path = chat_now_tts_path.get() 423 | 424 | 425 | tr_aa.join() 426 | ai_ans_tts_path = ai_ans_tts_path.get() 427 | 428 | speaking_continue_count += 1 429 | 430 | with lock_subtitles_speak: 431 | subtitles_speak( 432 | chat_role, 433 | chat_now, 434 | ai_ans, 435 | ai_name, 436 | chat_now_tts_path, 437 | ai_ans_tts_path, 438 | emo_state, 439 | AIVT_VTSP_Status_authenticated=AIVT_VTSP_Status_authenticated 440 | ) 441 | 442 | speaking_continue_count -= 1 443 | 444 | 445 | with lock_remove_file: 446 | def manage_files_by_count(file_path, file_max): 447 | try: 448 | file_max = int(file_max) 449 | except ValueError: 450 | file_max = 10 451 | if file_max < 0: 452 | file_max = 10 453 | 454 | files = [os.path.join(file_path, f) for f in os.listdir(file_path) if os.path.isfile(os.path.join(file_path, f))] 455 | if len(files) <= file_max: 456 | return 457 | 458 | files.sort() 459 | while len(files) > file_max: 460 | os.remove(files[0]) 461 | files.pop(0) 462 | 463 | file_path = "Audio/tts" 464 | file_max = 10 465 | manage_files_by_count(file_path, file_max) 466 | 467 | 468 | 469 | def sa_request_thread(text, sa_ans): 470 | with lock_sa_requst: 471 | sa_ans.put(sa_nlp_api.Sentiment_Analysis_NLP(text, Emo_state_categories=vtsp.AIVT_hotkeys_parameters["Emo_state_categories"], model=vtsp.AIVT_hotkeys_parameters["sentiment_analysis_model"])) 472 | 473 | 474 | def tts_request_thread(tts_text, tts_path): 475 | global GUI_TTS_Using 476 | 477 | with lock_tts_request: 478 | def generate_unique_filename(file_path, file_name, file_extension): 479 | timestamp = datetime.datetime.now().strftime("%Y%m%d%H%M%S") 480 | base_filename = f"{timestamp}_{file_name}" 481 | full_filename = f"{base_filename}{file_extension}" 482 | full_path = os.path.join(file_path, full_filename) 483 | counter = 1 484 | 485 | while os.path.exists(full_path): 486 | full_filename = f"{base_filename}_{counter}{file_extension}" 487 | full_path = os.path.join(file_path, full_filename) 488 | counter += 1 489 | 490 | return full_path 491 | 492 | file_path = "Audio/tts" 493 | file_name = f"{GUI_TTS_Using}" 494 | if GUI_TTS_Using == "EdgeTTS": 495 | file_extension = ".mp3" 496 | elif GUI_TTS_Using == "OpenAITTS": 497 | file_extension = ".wav" 498 | else: 499 | file_extension = ".mp3" 500 | 501 | unique_filename = generate_unique_filename(file_path, file_name, file_extension) 502 | 503 | if GUI_TTS_Using == "EdgeTTS": 504 | output_path = unique_filename 505 | voice = edgeTTS.edgetts_parameters["voice"] 506 | pitch = edgeTTS.edgetts_parameters["pitch"] 507 | rate = edgeTTS.edgetts_parameters["rate"] 508 | volume = edgeTTS.edgetts_parameters["volume"] 509 | timeout = edgeTTS.edgetts_parameters["timeout"] 510 | 511 | ttsr = threading.Thread( 512 | target=edgeTTS.edgetts, 513 | kwargs={ 514 | "text":tts_text, 515 | "output_path":output_path, 516 | "voice":voice, 517 | "pitch":pitch, 518 | "rate":rate, 519 | "volume":volume, 520 | "timeout":timeout, 521 | } 522 | ) 523 | ttsr.start() 524 | 525 | elif GUI_TTS_Using == "OpenAITTS": 526 | output_path = unique_filename 527 | model = openaiTTS.openaitts_parameters["model"] 528 | voice = openaiTTS.openaitts_parameters["voice"] 529 | speed = openaiTTS.openaitts_parameters["speed"] 530 | format = openaiTTS.openaitts_parameters["format"] 531 | timeout = openaiTTS.openaitts_parameters["timeout"] 532 | 533 | ttsr = threading.Thread( 534 | target=openaiTTS.openaitts, 535 | kwargs={ 536 | "text":tts_text, 537 | "output_path":output_path, 538 | "model":model, 539 | "voice":voice, 540 | "speed":speed, 541 | "format":format, 542 | "timeout":timeout, 543 | } 544 | ) 545 | ttsr.start() 546 | 547 | else: 548 | output_path = unique_filename 549 | voice = edgeTTS.edgetts_parameters["voice"] 550 | pitch = edgeTTS.edgetts_parameters["pitch"] 551 | rate = edgeTTS.edgetts_parameters["rate"] 552 | volume = edgeTTS.edgetts_parameters["volume"] 553 | timeout = edgeTTS.edgetts_parameters["timeout"] 554 | 555 | ttsr = threading.Thread( 556 | target=edgeTTS.edgetts, 557 | kwargs={ 558 | "text":tts_text, 559 | "output_path":output_path, 560 | "voice":voice, 561 | "pitch":pitch, 562 | "rate":rate, 563 | "volume":volume, 564 | "timeout":timeout, 565 | } 566 | ) 567 | ttsr.start() 568 | 569 | ttsr.join() 570 | 571 | tts_path.put(unique_filename) 572 | return 573 | 574 | 575 | def Subtitles_formatter_v3(text, max_length, english_char_length, base_line_count): 576 | def format_text(text, max_length, english_char_length, base_line_count): 577 | lines = [line for line in text.split('\n') if line.strip() != ''] 578 | Predicted_lines = [] 579 | for line in lines: 580 | while len(line) > max_length: 581 | Predicted_lines.append(line[:max_length]) 582 | line = line[max_length:] 583 | Predicted_lines.append(line) 584 | 585 | Predicted_line_count = len(Predicted_lines) 586 | #print(f'\nPredicted line count: {Predicted_line_count}\n') 587 | 588 | if Predicted_line_count < base_line_count: 589 | line_count = base_line_count 590 | else: 591 | line_count = Predicted_line_count 592 | new_max_length = max_length * line_count / base_line_count 593 | #print(f'Iteration BASE: line_count={line_count}, new_max_length={new_max_length}') 594 | result = split_line_text(lines, math.ceil(new_max_length), english_char_length, line_count) 595 | 596 | iteration = 1 597 | prev_line_count = None 598 | #print(str(len(result) != line_count) + " -> len(result) != line_count") 599 | #print(str(prev_line_count is None or prev_line_count != line_count) + " -> (prev_line_count is None or prev_line_count != line_count)") 600 | #print(str(iteration < 10) + " -> iteration < 10") 601 | #print(str(lines_fit_check != -1) + " -> lines_fit_check != -1") 602 | #print("\n----------\n") 603 | while len(result) != line_count and (prev_line_count is None or prev_line_count != line_count) and iteration < 10: 604 | #print(f'Iteration {iteration}: line_count={line_count}, new_max_length={new_max_length}') 605 | prev_line_count = line_count 606 | lr = len(result) 607 | #print(f'len(result) {lr}\n') 608 | line_count = (len(result) + line_count) // 2 609 | new_max_length = max_length * line_count / base_line_count 610 | #print(f'Iteration {iteration}: line_count={line_count}, new_max_length={new_max_length}') 611 | result = split_line_text(lines, math.ceil(new_max_length), english_char_length, line_count) 612 | iteration += 1 613 | 614 | fix_iteration_line_count = (len(result) + line_count) // 2 615 | #print(f'fix_iteration_line_count {fix_iteration_line_count}') 616 | if fix_iteration_line_count < base_line_count: 617 | line_count = base_line_count 618 | else: 619 | line_count = len(result) + 1 620 | 621 | lr = len(result) 622 | #print(f'len(result) {lr}') 623 | #print(f'line_count {line_count}') 624 | #if len(result) == line_count: 625 | #line_count = lr + 1 626 | #line_count = lr + 1 627 | 628 | new_max_length = max_length * line_count / base_line_count 629 | #print(f'fix iteration parameters: line_count={line_count}, new_max_length={new_max_length}') 630 | result = split_line_text(lines, math.ceil(new_max_length), english_char_length, line_count) 631 | 632 | final_iteration_line_count = len(result) 633 | #print(f'\nFinal result: line_count={final_iteration_line_count}, max_length={new_max_length}') 634 | #print(f'{result}') 635 | #print("\n----------\n") 636 | return '\n'.join(result) 637 | 638 | def split_line_text(lines: str, max_length: int, english_char_length: float, line_count: int): 639 | global lines_fit_check 640 | result = [] 641 | lines_fit_check = 0 642 | for line in lines: 643 | 644 | while len(line) > 0: 645 | #print(line) 646 | actual_length = 0 647 | split_index = 0 648 | 649 | for i, char in enumerate(line): 650 | if re.match(r' [a-zA-Z]\(\)\[\]{}<>:;=，。！？；～,!?~;【】「」『』〔〕()┐┌ლ✧⁄／╧ﾉﾒ彡o｡丿•̥̥̥`´ﾟゞو', char): 651 | actual_length += english_char_length 652 | else: 653 | actual_length += 1 654 | if actual_length > max_length: 655 | #print("----------") 656 | #print(f'max_length {max_length}') 657 | #print(f'actual_length {actual_length}') 658 | break 659 | split_index = i + 1 660 | 661 | if actual_length > max_length: 662 | punctuation_index = [i for i in range(split_index) if line[i] in ' ，。！？；～,!?~;【】「」『』〔〕()┐┌ლ✧⁄＼／╧ﾉﾒ彡o｡つ丿ㄟㄏ︴Ψ'] 663 | if punctuation_index: 664 | if line[punctuation_index[-1]] in '【「『〔((┐┌ㄟ＼': #:：】」』〕)┌ლ✧⁄╧╧ﾉﾒ彡o｡丿 665 | split_index = punctuation_index[-1] 666 | else: 667 | split_index = punctuation_index[-1] + 1 668 | 669 | #print(f'split_index {split_index}') 670 | result.append(line[:split_index]) 671 | line = line[split_index:].lstrip() 672 | else: 673 | result.append(line) 674 | line = "" 675 | lines_fit_check += 1 676 | #print(f'lines_fit_check {lines_fit_check}') 677 | 678 | if lines_fit_check == line_count: 679 | lines_fit_check = -1 680 | #print(f'lines_fit_check {lines_fit_check}') 681 | #print("\n----------\n") 682 | return result 683 | 684 | return format_text(text, max_length, english_char_length, base_line_count) 685 | 686 | def Subtitles_formatter_v2(text, max_length, english_char_length, base_line_count): 687 | def find_natural_break(line, max_length): 688 | total_length = 0 689 | last_good_break = 0 690 | for i, char in enumerate(line): 691 | char_length = english_char_length if char.isalpha() else 1 692 | total_length += char_length 693 | 694 | if char in ' ,.!?;:': 695 | last_good_break = i + 1 696 | 697 | if total_length > max_length: 698 | if last_good_break > 0: 699 | return last_good_break 700 | break 701 | 702 | return last_good_break if last_good_break > 0 else i 703 | 704 | def process_line_splits(lines, max_length): 705 | new_lines = [] 706 | for line in lines: 707 | while len(line) > max_length: 708 | break_index = find_natural_break(line, max_length) 709 | if break_index == 0: 710 | break_index = len(line) 711 | new_lines.append(line[:break_index].strip()) 712 | line = line[break_index:].lstrip() 713 | new_lines.append(line.strip()) 714 | return new_lines 715 | 716 | lines = text.split('\n') 717 | processed_lines = [] 718 | for line in lines: 719 | if line.strip() == '': 720 | continue 721 | processed_lines.append(line.strip()) 722 | 723 | iteration = 0 724 | while iteration < 10: 725 | new_lines = process_line_splits(processed_lines, max_length) 726 | if len(new_lines) <= base_line_count or len(new_lines) == len(processed_lines): 727 | break 728 | max_length = int(max_length * (len(new_lines) / base_line_count)) 729 | processed_lines = new_lines 730 | iteration += 1 731 | 732 | return '\n'.join(new_lines) 733 | 734 | 735 | 736 | def subtitles_speak_sleep(stime): 737 | time.sleep(stime) 738 | 739 | def subtitles_speak( 740 | chat_role, 741 | chat_now, 742 | ai_ans, 743 | ai_name, 744 | chat_now_tts_path, 745 | ai_ans_tts_path, 746 | emo_state, 747 | AIVT_VTSP_Status_authenticated=False, 748 | command=None 749 | ): 750 | global GUI_Setting_read_chat_now, GUI_VTSP_wait_until_hotkeys_complete 751 | global speaking_continue_count, speaking_continue_vtsp_hkns 752 | 753 | 754 | show_chat_now = (chat_role in ["user", "user_mic", "youtube_chat", "twitch_chat"]) 755 | read_chat_now = show_chat_now and ((GUI_Setting_read_chat_now and chat_role in ["user", "user_mic"]) or (live_chat.Live_chat_parameters["yt_live_chat_read_chat_now"] and chat_role == "youtube_chat")) or (live_chat.Live_chat_parameters["tw_live_chat_read_chat_now"] and chat_role == "twitch_chat") 756 | 757 | 758 | obs_show_chat_now = show_chat_now and obsws.OBS_Connected and obsws.OBS_chat_now_sub_parameters["show"] 759 | obs_show_ai_ans = obsws.OBS_Connected and obsws.OBS_ai_ans_sub_parameters["show"] 760 | 761 | obs_sub_formatter = obsws.OBS_subtitles_parameters["subtitles_formatter"] 762 | obs_sub_formatter_ver = obsws.OBS_subtitles_parameters["subtitles_formatter_version"] 763 | 764 | obs_chat_now_sub_name = obsws.OBS_chat_now_sub_parameters["sub_name"] 765 | obs_chat_now_show_sub_filter_names = obsws.OBS_chat_now_sub_parameters["show_sub_filter_names"] 766 | obs_chat_now_hide_sub_filter_names = obsws.OBS_chat_now_sub_parameters["hide_sub_filter_names"] 767 | 768 | obs_ai_ans_sub_name = obsws.OBS_ai_ans_sub_parameters["sub_name"] 769 | obs_ai_ans_show_sub_filter_names = obsws.OBS_ai_ans_sub_parameters["show_sub_filter_names"] 770 | obs_ai_ans_hide_sub_filter_names = obsws.OBS_ai_ans_sub_parameters["hide_sub_filter_names"] 771 | 772 | vtsp_hk_enble = AIVT_VTSP_Status_authenticated 773 | vtsp_trigger_first = vtsp.AIVT_hotkeys_parameters["trigger_first"] 774 | 775 | 776 | 777 | if vtsp_hk_enble and emo_state != "": 778 | emo_kn = vtsp.AIVT_hotkeys_parameters[f"{emo_state}_kn"] 779 | vtsp_hotkey_names = vtsp.get_hotkey_names(emo_kn) 780 | 781 | elif vtsp_hk_enble: 782 | emo = random.choice(vtsp.AIVT_hotkeys_parameters["Emo_state_categories"]) 783 | print(f"VTSP Random Emotion State: {emo}") 784 | emo_kn = vtsp.AIVT_hotkeys_parameters[f"{emo}_kn"] 785 | vtsp_hotkey_names = vtsp.get_hotkey_names(emo_kn) 786 | 787 | 788 | if vtsp_hk_enble and vtsp_trigger_first: 789 | if speaking_continue_vtsp_hkns: 790 | o_vtsp_hkns = set(vtsp_hotkey_names["all"]) 791 | p_vtsp_hkns = set(speaking_continue_vtsp_hkns) 792 | d_vtsp_hkns = o_vtsp_hkns & p_vtsp_hkns 793 | 794 | no_vtsp_hkns = [item for item in vtsp_hotkey_names["all"] if item not in d_vtsp_hkns] 795 | np_vtsp_hkns = [item for item in speaking_continue_vtsp_hkns if item not in d_vtsp_hkns] 796 | 797 | n_vtsp_hkns = np_vtsp_hkns + no_vtsp_hkns 798 | 799 | else: 800 | n_vtsp_hkns = vtsp_hotkey_names["all"] 801 | 802 | vtsp_ht = threading.Thread( 803 | target=vtsp.VTSP_Hotkey_Names_Trigger, 804 | kwargs={"hotkey_names_list": n_vtsp_hkns}, 805 | ) 806 | vtsp_ht.start() 807 | if GUI_VTSP_wait_until_hotkeys_complete: 808 | vtsp_ht.join() 809 | 810 | speaking_continue_vtsp_hkns = [] 811 | 812 | 813 | 814 | if obs_show_chat_now: 815 | if obs_sub_formatter: 816 | lines = chat_now.splitlines() 817 | lines = [line for line in lines if line.strip() != ''] 818 | obs_chat_now = '\n'.join(lines) 819 | 820 | if obs_sub_formatter_ver == "v3": 821 | obs_chat_now = Subtitles_formatter_v3( 822 | obs_chat_now, 823 | obsws.OBS_chat_now_sub_parameters["sub_max_length"], 824 | obsws.OBS_chat_now_sub_parameters["sub_english_char_length"], 825 | obsws.OBS_chat_now_sub_parameters["sub_base_line_count"] 826 | ) 827 | 828 | elif obs_sub_formatter_ver == "v2": 829 | obs_chat_now = Subtitles_formatter_v2( 830 | obs_chat_now, 831 | obsws.OBS_chat_now_sub_parameters["sub_max_length"], 832 | obsws.OBS_chat_now_sub_parameters["sub_english_char_length"], 833 | obsws.OBS_chat_now_sub_parameters["sub_base_line_count"] 834 | ) 835 | 836 | else: 837 | obs_chat_now = Subtitles_formatter_v3( 838 | obs_chat_now, 839 | obsws.OBS_chat_now_sub_parameters["sub_max_length"], 840 | obsws.OBS_chat_now_sub_parameters["sub_english_char_length"], 841 | obsws.OBS_chat_now_sub_parameters["sub_base_line_count"] 842 | ) 843 | 844 | obst_cn_st = threading.Thread( 845 | target=obsws.Set_Source_Text, 846 | args=(obs_chat_now_sub_name, obs_chat_now) 847 | ) 848 | obst_cn_st.start() 849 | obst_cn_st.join() 850 | 851 | if len(obs_chat_now_show_sub_filter_names) > 0: 852 | obst_cn_s = threading.Thread( 853 | target=obsws.Set_Source_Filter_Enabled, 854 | args=( 855 | obs_chat_now_sub_name, 856 | obs_chat_now_show_sub_filter_names, 857 | True, 858 | ), 859 | kwargs={ 860 | "start_delay":obsws.OBS_chat_now_sub_parameters["show_sub_filter_start_delay"], 861 | "end_delay":obsws.OBS_chat_now_sub_parameters["show_sub_filter_end_delay"], 862 | }, 863 | ) 864 | obst_cn_s.start() 865 | 866 | sub_time = obsws.OBS_chat_now_sub_parameters["sub_time"] 867 | if sub_time > 0: 868 | sss = threading.Thread(target=subtitles_speak_sleep, args=(sub_time, )) 869 | sss.start() 870 | 871 | 872 | 873 | if read_chat_now: 874 | chat_now_read = threading.Thread(target=speaking, args=(chat_now_tts_path, )) 875 | chat_now_read.start() 876 | chat_now_read.join() 877 | 878 | 879 | 880 | if obs_show_chat_now: 881 | if sub_time > 0: 882 | sss.join() 883 | 884 | if obs_show_chat_now: 885 | time.sleep(obsws.OBS_chat_now_sub_parameters["sub_end_delay"]) 886 | 887 | if vtsp_hk_enble and not vtsp_trigger_first: 888 | vtsp_ht = threading.Thread( 889 | target=vtsp.VTSP_Hotkey_Names_Trigger, 890 | kwargs={"hotkey_names_list": vtsp_hotkey_names["all"]} 891 | ) 892 | vtsp_ht.start() 893 | if GUI_VTSP_wait_until_hotkeys_complete: 894 | vtsp_ht.join() 895 | 896 | 897 | 898 | if obs_show_chat_now: 899 | if obsws.OBS_chat_now_sub_parameters["clear"]: 900 | obst_cn_st = threading.Thread( 901 | target=obsws.Set_Source_Text, 902 | args=(obs_chat_now_sub_name, "") 903 | ) 904 | obst_cn_st.start() 905 | obst_cn_st.join() 906 | 907 | if len(obs_chat_now_hide_sub_filter_names) > 0: 908 | obst_cn_fe_sd = threading.Thread( 909 | target=obsws.Set_Source_Filter_Enabled, 910 | args=( 911 | obs_chat_now_sub_name, 912 | obs_chat_now_hide_sub_filter_names, 913 | True, 914 | ), 915 | kwargs={ 916 | "start_delay":obsws.OBS_chat_now_sub_parameters["hide_sub_filter_start_delay"], 917 | "end_delay":obsws.OBS_chat_now_sub_parameters["hide_sub_filter_end_delay"], 918 | }, 919 | ) 920 | obst_cn_fe_sd.start() 921 | 922 | 923 | 924 | if obs_show_ai_ans: 925 | 926 | if obsws.OBS_ai_ans_sub_parameters["remove_original_text_wrap"]: 927 | obs_ai_ans = ai_ans.replace('\n', ' ') 928 | else: 929 | obs_ai_ans = ai_ans 930 | 931 | if obs_sub_formatter_ver == "v3": 932 | ai_ans_formatted = Subtitles_formatter_v3( 933 | obs_ai_ans, 934 | obsws.OBS_ai_ans_sub_parameters["sub_max_length"], 935 | obsws.OBS_ai_ans_sub_parameters["sub_english_char_length"], 936 | obsws.OBS_ai_ans_sub_parameters["sub_base_line_count"] 937 | ) 938 | 939 | elif obs_sub_formatter_ver == "v2": 940 | ai_ans_formatted = Subtitles_formatter_v2( 941 | obs_ai_ans, 942 | obsws.OBS_ai_ans_sub_parameters["sub_max_length"], 943 | obsws.OBS_ai_ans_sub_parameters["sub_english_char_length"], 944 | obsws.OBS_ai_ans_sub_parameters["sub_base_line_count"] 945 | ) 946 | 947 | else: 948 | ai_ans_formatted = Subtitles_formatter_v3( 949 | obs_ai_ans, 950 | obsws.OBS_ai_ans_sub_parameters["sub_max_length"], 951 | obsws.OBS_ai_ans_sub_parameters["sub_english_char_length"], 952 | obsws.OBS_ai_ans_sub_parameters["sub_base_line_count"] 953 | ) 954 | 955 | obst_aa_st = threading.Thread( 956 | target=obsws.Set_Source_Text, 957 | args=(obs_ai_ans_sub_name, ai_ans_formatted) 958 | ) 959 | obst_aa_st.start() 960 | obst_aa_st.join() 961 | 962 | 963 | 964 | ai_ans_read = threading.Thread(target=speaking, args=(ai_ans_tts_path, )) 965 | 966 | 967 | 968 | if obs_show_ai_ans: 969 | if len(obs_ai_ans_show_sub_filter_names) > 0: 970 | obst_aa_fe_s = threading.Thread( 971 | target=obsws.Set_Source_Filter_Enabled, 972 | args=( 973 | obs_ai_ans_sub_name, 974 | obs_ai_ans_show_sub_filter_names, 975 | True, 976 | ), 977 | kwargs={ 978 | "start_delay":obsws.OBS_ai_ans_sub_parameters["show_sub_filter_start_delay"], 979 | "end_delay":obsws.OBS_ai_ans_sub_parameters["show_sub_filter_end_delay"], 980 | }, 981 | ) 982 | obst_aa_fe_s.start() 983 | 984 | 985 | 986 | ai_ans_read.start() 987 | ai_ans_read.join() 988 | 989 | 990 | 991 | if obs_show_ai_ans: 992 | time.sleep(obsws.OBS_ai_ans_sub_parameters["sub_end_delay"]) 993 | 994 | 995 | 996 | if vtsp_hk_enble and (speaking_continue_count <= 1 or not vtsp_trigger_first): 997 | idle_kn = vtsp.get_hotkey_names(vtsp.AIVT_hotkeys_parameters["idle_ani"], command="idle_ani") 998 | vtsp_hkns = idle_kn["r_ani"] + vtsp_hotkey_names["all_exp"] 999 | 1000 | vtsp_ht = threading.Thread( 1001 | target=vtsp.VTSP_Hotkey_Names_Trigger, 1002 | kwargs={"hotkey_names_list": vtsp_hkns} 1003 | ) 1004 | vtsp_ht.start() 1005 | 1006 | elif vtsp_hk_enble: 1007 | speaking_continue_vtsp_hkns = vtsp_hotkey_names["all_exp"] 1008 | 1009 | 1010 | if vtsp_hk_enble and GUI_VTSP_wait_until_hotkeys_complete: 1011 | vtsp_ht.join() 1012 | 1013 | 1014 | if obs_show_ai_ans: 1015 | if obsws.OBS_ai_ans_sub_parameters["clear"]: 1016 | obst_aa_st = threading.Thread( 1017 | target=obsws.Set_Source_Text, 1018 | args=(obs_ai_ans_sub_name, "") 1019 | ) 1020 | obst_aa_st.start() 1021 | obst_aa_st.join() 1022 | 1023 | if len(obs_ai_ans_hide_sub_filter_names) > 0: 1024 | obst_aa_fe_h = threading.Thread( 1025 | target=obsws.Set_Source_Filter_Enabled, 1026 | args=( 1027 | obs_ai_ans_sub_name, 1028 | obs_ai_ans_hide_sub_filter_names, 1029 | True, 1030 | ), 1031 | kwargs={ 1032 | "start_delay":obsws.OBS_ai_ans_sub_parameters["hide_sub_filter_start_delay"], 1033 | "end_delay":obsws.OBS_ai_ans_sub_parameters["hide_sub_filter_end_delay"], 1034 | }, 1035 | ) 1036 | obst_aa_fe_h.start() 1037 | 1038 | 1039 | 1040 | try: 1041 | w_name, w_text = chat_now.split(" : ", 1) 1042 | w_chat_now = f"{w_name} :\n{w_text}" 1043 | except: 1044 | w_chat_now = chat_now 1045 | w_ai_ans = f"{ai_name} :\n{ai_ans}" 1046 | wch = threading.Thread(target=write_conversation_history, args=(w_chat_now, w_ai_ans)) 1047 | wch.start() 1048 | 1049 | 1050 | 1051 | def speaking(audio_path, start_delay=0, end_delay=0): 1052 | ai_voice_output_device_name = Play_Audio.play_audio_parameters["ai_voice_output_device_name"] 1053 | 1054 | time.sleep(start_delay) 1055 | 1056 | try: 1057 | Play_Audio.PlayAudio(audio_path, ai_voice_output_device_name) 1058 | 1059 | except Exception as e: 1060 | print(f"!!! Play Audio Error !!!\n{e}") 1061 | 1062 | time.sleep(end_delay) 1063 | 1064 | 1065 | 1066 | lock_write_conversation_history = threading.Lock() 1067 | 1068 | def write_conversation_history(chat_now, subtitles): 1069 | with lock_write_conversation_history: 1070 | ConversationHistory_path = f"Text_files/ConversationHistory/{datetime.datetime.now().strftime('%Y-%m-%d')}.txt" 1071 | 1072 | with open(ConversationHistory_path, "a", encoding="utf-8") as outfile: 1073 | outfile.write(f"\n{datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S %p')}\n") 1074 | outfile.write(f"\n{chat_now}\n") 1075 | outfile.write(f"\n{subtitles}\n") 1076 | outfile.write("\n------------------------------\n") 1077 | 1078 | 1079 | 1080 | 1081 | 1082 | 1083 | 1084 | 1085 | 1086 | 1087 | def AIVT_VTSP_Idle_Animation(): 1088 | global speaking_continue_count 1089 | if vtsp.AIVT_VTSP_Status["authenticated"]: 1090 | kn = vtsp.get_hotkey_names(vtsp.AIVT_hotkeys_parameters["idle_ani"], command="idle_ani") 1091 | vtsp_ht = threading.Thread( 1092 | target=vtsp.VTSP_Hotkey_Names_Trigger, 1093 | kwargs={"hotkey_names_list": kn["r_ani"]} 1094 | ) 1095 | vtsp_ht.start() 1096 | 1097 | t = 0 1098 | 1099 | while vtsp.AIVT_VTSP_Status["authenticated"]: 1100 | while speaking_continue_count > 0: 1101 | t = 0 1102 | time.sleep(0.1) 1103 | 1104 | if t >= 59: 1105 | t = 0 1106 | 1107 | kn = vtsp.get_hotkey_names(vtsp.AIVT_hotkeys_parameters["idle_ani"], command="idle_ani") 1108 | vtsp_ht = threading.Thread( 1109 | target=vtsp.VTSP_Hotkey_Names_Trigger, 1110 | kwargs={"hotkey_names_list": kn["r_ani"]} 1111 | ) 1112 | vtsp_ht.start() 1113 | 1114 | time.sleep(1) 1115 | t += 1 1116 | 1117 | 1118 | 1119 | 1120 | -------------------------------------------------------------------------------- /GUI_control_panel/GUI_config.ini: -------------------------------------------------------------------------------- 1 | [Main] 2 | show_chat_role = True 3 | 4 | [Setting] 5 | user_name = PKevin 6 | character = Midokyoko_Kyurei 7 | tab_position = North 8 | ai_voice_output_device_name = Microsoft 音效對應表 - Output 9 | user_mic_input_device_name = Microsoft 音效對應表 - Input 10 | doing_now = 今天是XXXX年XX月XX日。 11 | read_user_chat = False 12 | user_mic_hotkey_1_using = True 13 | user_mic_hotkey_1 = ` 14 | user_mic_hotkey_2_using = True 15 | user_mic_hotkey_2 = caps lock 16 | 17 | [Live_Chat] 18 | yt_channel_name = 19 | yt_live_id = qTmJ4YkziEk 20 | yt_response_chatroom = False 21 | yt_live_chat_read_chat_now = False 22 | yt_response_owner = False 23 | yt_response_vip = False 24 | yt_response_individual = False 25 | yt_wait_list_max = 10 26 | yt_chat_max_tokens = 128 27 | yt_chat_max_response = 5 28 | yt_live_chat_vip_names = 29 | yt_live_chat_ban_names = 30 | tw_channel_name = 31 | tw_response_chatroom = False 32 | tw_live_chat_read_chat_now = False 33 | tw_response_owner = False 34 | tw_response_vip = False 35 | tw_response_individual = False 36 | tw_wait_list_max = 10 37 | tw_chat_max_tokens = 128 38 | tw_chat_max_response = 5 39 | tw_live_chat_vip_names = 40 | tw_live_chat_ban_names = 41 | 42 | [LLM] 43 | using = Gemini 44 | instruction_enhance = True 45 | instruction_enhance_i = 1 46 | 47 | [LLM_Gemini] 48 | model = gemini-2.0-flash 49 | max_input_tokens = 8192 50 | max_output_tokens = 512 51 | temperature = 0.9 52 | timeout = 15 53 | retry = 3 54 | 55 | [LLM_GPT] 56 | model = gpt-4o 57 | max_input_tokens = 4096 58 | max_output_tokens = 256 59 | temperature = 1.0 60 | timeout = 15 61 | retry = 3 62 | 63 | [TextToSpeech] 64 | using = EdgeTTS 65 | 66 | [EdgeTTS] 67 | voice = zh-TW-HsiaoChenNeural 68 | voice_male = zh-TW-YunJheNeural 69 | voice_female = zh-TW-HsiaoChenNeural 70 | pitch = 20 71 | rate = 14 72 | volume = 0 73 | timeout = 10 74 | 75 | [OpenAITTS] 76 | model = tts-1 77 | voice = alloy 78 | speed = 1.0 79 | timeout = 10 80 | 81 | [Whisper] 82 | inference = Local 83 | model_name = base 84 | model_api = whisper-1 85 | language = zh 86 | max_tokens = 192 87 | temperature = 0.2 88 | timeout = 10 89 | prompt = 90 | 91 | [OBS_Subtitles] 92 | subtitles_formatter = True 93 | subtitles_formatter_version = v3 94 | 95 | [OBS_Chat_now_sub] 96 | sub_name = AIVT_Chat_Sub 97 | show = True 98 | clear = False 99 | sub_time = 2.5 100 | sub_max_length = 30 101 | sub_english_char_length = 1.0 102 | sub_base_line_count = 7 103 | sub_end_delay = 0.0 104 | show_sub_filter_names = show 105 | hide_sub_filter_names = hide 106 | show_sub_filter_start_delay = 0.0 107 | hide_sub_filter_start_delay = 0.0 108 | 109 | [OBS_AI_ans_sub] 110 | sub_name = AIVT_AI_Sub 111 | show = True 112 | clear = False 113 | remove_original_text_wrap = True 114 | sub_max_length = 30 115 | sub_english_char_length = 1.0 116 | sub_base_line_count = 7 117 | sub_end_delay = 0.0 118 | show_sub_filter_names = show 119 | hide_sub_filter_names = hide 120 | show_sub_filter_start_delay = 0.0 121 | hide_sub_filter_start_delay = 0.0 122 | 123 | [VTube_Studio_Plug] 124 | trigger_first = True 125 | wait_until_hotkeys_complete = True 126 | sentiment_analysis = True 127 | sentiment_analysis_modle = gemini-2.0-flash 128 | idle_ani = !ani_idle_01/!ani_idle_02/!ani_idle_03 129 | normal = True 130 | normal_kn = !ani_talking_01/exp_smile1/exp_smile2 131 | happy = True 132 | happy_kn = !ani_talking_01/*exp_blush1/exp_smile2/exp_star_eyes/exp_heart_eyes 133 | shy = True 134 | shy_kn = !ani_talking_03/*exp_blush2/*exp_heart_eyes/exp_smile2/exp_smile3/exp_sad1 135 | proud = True 136 | proud_kn = !ani_talking_02/*exp_blush1/*exp_star_eyes/exp_smile2/exp_smile3 137 | shock = True 138 | shock_kn = !ani_talking_03/*exp_black_face/exp_sad1/exp_sad2 139 | sad = True 140 | sad_kn = !ani_talking_03/*exp_tear/*exp_sad1/exp_sad2/exp_black_face 141 | angry = True 142 | angry_kn = !ani_talking_02/*exp_angry/*exp_sad1/exp_sad2/exp_black_face 143 | embarrass = True 144 | embarrass_kn = !ani_talking_03/*exp_blush1/@exp_smile1/@exp_sad1/exp_blush2/exp_black_face 145 | afraid = True 146 | afraid_kn = !ani_talking_02/*exp_black_face/*exp_sad1/exp_sad2/exp_tear 147 | confuse = True 148 | confuse_kn = !ani_talking_03/*exp_smile1/exp_question_mark/exp_otaku_glasses 149 | 150 | -------------------------------------------------------------------------------- /GUI_control_panel/GUI_icon/AI_Vtuber_GUI_control_panel_01.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/GUI_control_panel/GUI_icon/AI_Vtuber_GUI_control_panel_01.ico -------------------------------------------------------------------------------- /GUI_control_panel/GUI_py/palette.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 255 8 | 255 9 | 255 10 | 11 | 12 | 13 | 14 | 15 | 16 | 40 17 | 40 18 | 40 19 | 20 | 21 | 22 | 23 | 24 | 25 | 255 26 | 255 27 | 255 28 | 29 | 30 | 31 | 32 | 33 | 34 | 255 35 | 255 36 | 255 37 | 38 | 39 | 40 | 41 | 42 | 43 | 40 44 | 40 45 | 40 46 | 47 | 48 | 49 | 50 | 51 | 52 | 40 53 | 40 54 | 40 55 | 56 | 57 | 58 | 59 | 60 | 61 | 255 62 | 255 63 | 255 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 255 73 | 255 74 | 255 75 | 76 | 77 | 78 | 79 | 80 | 81 | 40 82 | 40 83 | 40 84 | 85 | 86 | 87 | 88 | 89 | 90 | 255 91 | 255 92 | 255 93 | 94 | 95 | 96 | 97 | 98 | 99 | 255 100 | 255 101 | 255 102 | 103 | 104 | 105 | 106 | 107 | 108 | 40 109 | 40 110 | 40 111 | 112 | 113 | 114 | 115 | 116 | 117 | 40 118 | 40 119 | 40 120 | 121 | 122 | 123 | 124 | 125 | 126 | 255 127 | 255 128 | 255 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 255 138 | 255 139 | 255 140 | 141 | 142 | 143 | 144 | 145 | 146 | 40 147 | 40 148 | 40 149 | 150 | 151 | 152 | 153 | 154 | 155 | 255 156 | 255 157 | 255 158 | 159 | 160 | 161 | 162 | 163 | 164 | 255 165 | 255 166 | 255 167 | 168 | 169 | 170 | 171 | 172 | 173 | 40 174 | 40 175 | 40 176 | 177 | 178 | 179 | 180 | 181 | 182 | 40 183 | 40 184 | 40 185 | 186 | 187 | 188 | 189 | 190 | 191 | 255 192 | 255 193 | 255 194 | 195 | 196 | 197 | 198 | 199 | -------------------------------------------------------------------------------- /GUI_control_panel/GUI_py/ui2py_pysd6.txt: -------------------------------------------------------------------------------- 1 | !!! Run The Following Code In CMD Under The Corresponding File Path !!! 2 | 3 | pyside6-uic -o AI_Vtuber_control_panel_ui_pysd6.py AI_Vtuber_control_panel_ui_pysd6.ui -------------------------------------------------------------------------------- /GUI_control_panel/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/GUI_control_panel/__init__.py -------------------------------------------------------------------------------- /GUI_control_panel/ffmpeg.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/GUI_control_panel/ffmpeg.exe -------------------------------------------------------------------------------- /Google/gemini/GoogleAI_Gemini_API.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '../..'))) 4 | import threading 5 | import time 6 | import queue 7 | 8 | import google.generativeai as genai 9 | 10 | import AIVT_Config 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | gemini_models_max_input_tokens = { 22 | "gemini-2.0-flash":1048576, 23 | "gemini-2.0-flash-001":1048576, 24 | "gemini-2.0-flash-lite":1048576, 25 | "gemini-2.0-flash-lite-001":1048576, 26 | "gemini-2.5-pro-preview-03-25":1048576, 27 | "gemini-1.5-flash-latest":1048576, 28 | "gemini-1.5-flash":1048576, 29 | "gemini-1.5-flash-001":1048576, 30 | "gemini-1.5-flash-002":1048576, 31 | "gemini-1.5-flash-8b-latest":1048576, 32 | "gemini-1.5-flash-8b":1048576, 33 | "gemini-1.5-flash-8b-001":1048576, 34 | "gemini-1.5-pro-latest":2097152, 35 | "gemini-1.5-pro":2097152, 36 | "gemini-1.5-pro-001":2097152, 37 | "gemini-1.5-pro-002":2097152, 38 | } 39 | 40 | gemini_models_max_output_tokens = { 41 | "gemini-2.0-flash":8192, 42 | "gemini-2.0-flash-001":8192, 43 | "gemini-2.0-flash-lite":8192, 44 | "gemini-2.0-flash-lite-001":8192, 45 | "gemini-2.5-pro-preview-03-25":65536, 46 | "gemini-1.5-flash-latest":8192, 47 | "gemini-1.5-flash":8192, 48 | "gemini-1.5-flash-001":8192, 49 | "gemini-1.5-flash-002":8192, 50 | "gemini-1.5-flash-8b-latest":8192, 51 | "gemini-1.5-flash-8b":8192, 52 | "gemini-1.5-flash-8b-001":8192, 53 | "gemini-1.5-pro-latest":8192, 54 | "gemini-1.5-pro":8192, 55 | "gemini-1.5-pro-001":8192, 56 | "gemini-1.5-pro-002":8192, 57 | } 58 | 59 | gemini_parameters = { 60 | "model": "gemini-2.0-flash", 61 | "max_input_tokens" : 4096, 62 | "max_output_tokens" : 512, 63 | "temperature" : 0.90, 64 | "timeout" : 15, 65 | "retry" : 3, 66 | } 67 | 68 | 69 | 70 | 71 | 72 | def run_with_timeout_GoogleAI_Gemini_API( 73 | conversation, 74 | chatQ, 75 | model_name="gemini-2.0-flash", 76 | max_output_tokens="512", 77 | temperature=0.9, 78 | timeout=15, 79 | retry=3, 80 | command=None, 81 | ): 82 | 83 | start_time = time.time() 84 | 85 | ans = queue.Queue() 86 | prompt_tokens = queue.Queue() 87 | completion_tokens = queue.Queue() 88 | 89 | GGAt = threading.Thread( 90 | target=GoogleAI_Gemini_API_thread, 91 | args=( 92 | conversation, 93 | ans, 94 | prompt_tokens, 95 | completion_tokens, 96 | ), 97 | kwargs={ 98 | "model_name":model_name, 99 | "max_output_tokens":max_output_tokens, 100 | "temperature":temperature, 101 | "retry":retry, 102 | }, 103 | ) 104 | 105 | 106 | GGAt.start() 107 | GGAt.join(timeout) 108 | 109 | if GGAt.is_alive(): 110 | return None 111 | 112 | else: 113 | end_time = time.time() 114 | llm_result = ans.get() 115 | prompt_tokens = prompt_tokens.get() 116 | completion_tokens = completion_tokens.get() 117 | if command != "no_print": 118 | print("\nGoogleAI_Gemini_Answer ----------\n") 119 | print(f"Model: {model_name}") 120 | print(f"Duration: {end_time - start_time:.2f}s") 121 | print(f"Prompt tokens: {prompt_tokens}") 122 | print(f"Completion tokens: {completion_tokens}") 123 | print(f"Total tokens: {prompt_tokens+completion_tokens}\n") 124 | print(f"{chatQ}\n") 125 | print(f"Gemini Answer : {llm_result}") 126 | print("\n----------\n") 127 | 128 | cleaned_llm_result = "\n".join(line.strip() for line in llm_result.splitlines() if line.strip()) 129 | return cleaned_llm_result 130 | 131 | 132 | def GoogleAI_Gemini_API_thread( 133 | conversation, 134 | ans, 135 | prompt_tokens, 136 | completion_tokens, 137 | model_name = "gemini-2.0-flash", 138 | max_output_tokens = 512, 139 | temperature = 0.9, 140 | retry = 3, 141 | ): 142 | 143 | def convert2gemini_conversation(conversation): 144 | system_contents = [] 145 | new_conversation = [] 146 | current_role = None 147 | current_parts = [] 148 | 149 | for entry in conversation: 150 | if entry['role'] == 'system': 151 | system_contents.append(entry['content']) 152 | continue 153 | 154 | if entry['role'] == 'assistant': 155 | entry['role'] = 'model' 156 | 157 | if entry['role'] == current_role: 158 | current_parts.append(entry['content']) 159 | else: 160 | if current_role is not None: 161 | new_conversation.append({'role': current_role, 'parts': '\n'.join(current_parts)}) 162 | 163 | current_role = entry['role'] 164 | current_parts = [entry['content']] 165 | 166 | if current_role is not None: 167 | new_conversation.append({'role': current_role, 'parts': '\n'.join(current_parts)}) 168 | 169 | system_text = '\n'.join(system_contents) 170 | 171 | return system_text, new_conversation 172 | 173 | system_instruction, conversation_for_gemini = convert2gemini_conversation(conversation) 174 | 175 | genai.configure(api_key=AIVT_Config.google_api_key) 176 | 177 | generation_config = genai.types.GenerationConfig( 178 | max_output_tokens=max_output_tokens, 179 | temperature=temperature, 180 | ) 181 | 182 | safety_settings=[ 183 | { 184 | "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", 185 | "threshold": "BLOCK_NONE", 186 | }, 187 | { 188 | "category": "HARM_CATEGORY_HATE_SPEECH", 189 | "threshold": "BLOCK_NONE", 190 | }, 191 | { 192 | "category": "HARM_CATEGORY_HARASSMENT", 193 | "threshold": "BLOCK_NONE", 194 | }, 195 | { 196 | "category": "HARM_CATEGORY_DANGEROUS_CONTENT", 197 | "threshold": "BLOCK_NONE", 198 | } 199 | ] 200 | 201 | genimi_model = genai.GenerativeModel( 202 | system_instruction=system_instruction, 203 | model_name=model_name, 204 | generation_config=generation_config, 205 | safety_settings=safety_settings 206 | ) 207 | 208 | reT = 0 209 | while reT < retry: 210 | reT += 1 211 | 212 | try: 213 | response = genimi_model.generate_content(conversation_for_gemini) 214 | 215 | try: 216 | message = response.text 217 | except Exception as e: 218 | print(f"\n{response.prompt_feedback}\n{e}\n") 219 | ans.put("") 220 | return 221 | 222 | response_dict = response.to_dict() 223 | usage_metadata = response_dict.get('usage_metadata', {}) 224 | prompt_tokens.put(usage_metadata.get('prompt_token_count')) 225 | completion_tokens.put(usage_metadata.get('candidates_token_count')) 226 | ans.put(message) 227 | return 228 | 229 | except Exception as e: 230 | if reT < retry: 231 | print(f"!!! GoogleAI_Gemini_API retry {reT} time !!!\n{e}\n") 232 | continue 233 | 234 | else: 235 | print(f"!!! GoogleAI_Gemini_API retry {reT} time !!!\n{e}\n") 236 | ans.put("") 237 | return 238 | 239 | 240 | 241 | 242 | -------------------------------------------------------------------------------- /Google/gemini/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/Google/gemini/__init__.py -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Mozilla Public License Version 2.0 2 | ================================== 3 | 4 | Copyright (c) 2024 PKevin AI VTuber Developer 5 | 6 | 1. Definitions 7 | -------------- 8 | 9 | 1.1. "Contributor" 10 | means each individual or legal entity that creates, contributes to 11 | the creation of, or owns Covered Software. 12 | 13 | 1.2. "Contributor Version" 14 | means the combination of the Contributions of others (if any) used 15 | by a Contributor and that particular Contributor's Contribution. 16 | 17 | 1.3. "Contribution" 18 | means Covered Software of a particular Contributor. 19 | 20 | 1.4. "Covered Software" 21 | means Source Code Form to which the initial Contributor has attached 22 | the notice in Exhibit A, the Executable Form of such Source Code 23 | Form, and Modifications of such Source Code Form, in each case 24 | including portions thereof. 25 | 26 | 1.5. "Incompatible With Secondary Licenses" 27 | means 28 | 29 | (a) that the initial Contributor has attached the notice described 30 | in Exhibit B to the Covered Software; or 31 | 32 | (b) that the Covered Software was made available under the terms of 33 | version 1.1 or earlier of the License, but not also under the 34 | terms of a Secondary License. 35 | 36 | 1.6. "Executable Form" 37 | means any form of the work other than Source Code Form. 38 | 39 | 1.7. "Larger Work" 40 | means a work that combines Covered Software with other material, in 41 | a separate file or files, that is not Covered Software. 42 | 43 | 1.8. "License" 44 | means this document. 45 | 46 | 1.9. "Licensable" 47 | means having the right to grant, to the maximum extent possible, 48 | whether at the time of the initial grant or subsequently, any and 49 | all of the rights conveyed by this License. 50 | 51 | 1.10. "Modifications" 52 | means any of the following: 53 | 54 | (a) any file in Source Code Form that results from an addition to, 55 | deletion from, or modification of the contents of Covered 56 | Software; or 57 | 58 | (b) any new file in Source Code Form that contains any Covered 59 | Software. 60 | 61 | 1.11. "Patent Claims" of a Contributor 62 | means any patent claim(s), including without limitation, method, 63 | process, and apparatus claims, in any patent Licensable by such 64 | Contributor that would be infringed, but for the grant of the 65 | License, by the making, using, selling, offering for sale, having 66 | made, import, or transfer of either its Contributions or its 67 | Contributor Version. 68 | 69 | 1.12. "Secondary License" 70 | means either the GNU General Public License, Version 2.0, the GNU 71 | Lesser General Public License, Version 2.1, the GNU Affero General 72 | Public License, Version 3.0, or any later versions of those 73 | licenses. 74 | 75 | 1.13. "Source Code Form" 76 | means the form of the work preferred for making modifications. 77 | 78 | 1.14. "You" (or "Your") 79 | means an individual or a legal entity exercising rights under this 80 | License. For legal entities, "You" includes any entity that 81 | controls, is controlled by, or is under common control with You. For 82 | purposes of this definition, "control" means (a) the power, direct 83 | or indirect, to cause the direction or management of such entity, 84 | whether by contract or otherwise, or (b) ownership of more than 85 | fifty percent (50%) of the outstanding shares or beneficial 86 | ownership of such entity. 87 | 88 | 2. License Grants and Conditions 89 | -------------------------------- 90 | 91 | 2.1. Grants 92 | 93 | Each Contributor hereby grants You a world-wide, royalty-free, 94 | non-exclusive license: 95 | 96 | (a) under intellectual property rights (other than patent or trademark) 97 | Licensable by such Contributor to use, reproduce, make available, 98 | modify, display, perform, distribute, and otherwise exploit its 99 | Contributions, either on an unmodified basis, with Modifications, or 100 | as part of a Larger Work; and 101 | 102 | (b) under Patent Claims of such Contributor to make, use, sell, offer 103 | for sale, have made, import, and otherwise transfer either its 104 | Contributions or its Contributor Version. 105 | 106 | 2.2. Effective Date 107 | 108 | The licenses granted in Section 2.1 with respect to any Contribution 109 | become effective for each Contribution on the date the Contributor first 110 | distributes such Contribution. 111 | 112 | 2.3. Limitations on Grant Scope 113 | 114 | The licenses granted in this Section 2 are the only rights granted under 115 | this License. No additional rights or licenses will be implied from the 116 | distribution or licensing of Covered Software under this License. 117 | Notwithstanding Section 2.1(b) above, no patent license is granted by a 118 | Contributor: 119 | 120 | (a) for any code that a Contributor has removed from Covered Software; 121 | or 122 | 123 | (b) for infringements caused by: (i) Your and any other third party's 124 | modifications of Covered Software, or (ii) the combination of its 125 | Contributions with other software (except as part of its Contributor 126 | Version); or 127 | 128 | (c) under Patent Claims infringed by Covered Software in the absence of 129 | its Contributions. 130 | 131 | This License does not grant any rights in the trademarks, service marks, 132 | or logos of any Contributor (except as may be necessary to comply with 133 | the notice requirements in Section 3.4). 134 | 135 | 2.4. Subsequent Licenses 136 | 137 | No Contributor makes additional grants as a result of Your choice to 138 | distribute the Covered Software under a subsequent version of this 139 | License (see Section 10.2) or under the terms of a Secondary License (if 140 | permitted under the terms of Section 3.3). 141 | 142 | 2.5. Representation 143 | 144 | Each Contributor represents that the Contributor believes its 145 | Contributions are its original creation(s) or it has sufficient rights 146 | to grant the rights to its Contributions conveyed by this License. 147 | 148 | 2.6. Fair Use 149 | 150 | This License is not intended to limit any rights You have under 151 | applicable copyright doctrines of fair use, fair dealing, or other 152 | equivalents. 153 | 154 | 2.7. Conditions 155 | 156 | Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted 157 | in Section 2.1. 158 | 159 | 3. Responsibilities 160 | ------------------- 161 | 162 | 3.1. Distribution of Source Form 163 | 164 | All distribution of Covered Software in Source Code Form, including any 165 | Modifications that You create or to which You contribute, must be under 166 | the terms of this License. You must inform recipients that the Source 167 | Code Form of the Covered Software is governed by the terms of this 168 | License, and how they can obtain a copy of this License. You may not 169 | attempt to alter or restrict the recipients' rights in the Source Code 170 | Form. 171 | 172 | 3.2. Distribution of Executable Form 173 | 174 | If You distribute Covered Software in Executable Form then: 175 | 176 | (a) such Covered Software must also be made available in Source Code 177 | Form, as described in Section 3.1, and You must inform recipients of 178 | the Executable Form how they can obtain a copy of such Source Code 179 | Form by reasonable means in a timely manner, at a charge no more 180 | than the cost of distribution to the recipient; and 181 | 182 | (b) You may distribute such Executable Form under the terms of this 183 | License, or sublicense it under different terms, provided that the 184 | license for the Executable Form does not attempt to limit or alter 185 | the recipients' rights in the Source Code Form under this License. 186 | 187 | 3.3. Distribution of a Larger Work 188 | 189 | You may create and distribute a Larger Work under terms of Your choice, 190 | provided that You also comply with the requirements of this License for 191 | the Covered Software. If the Larger Work is a combination of Covered 192 | Software with a work governed by one or more Secondary Licenses, and the 193 | Covered Software is not Incompatible With Secondary Licenses, this 194 | License permits You to additionally distribute such Covered Software 195 | under the terms of such Secondary License(s), so that the recipient of 196 | the Larger Work may, at their option, further distribute the Covered 197 | Software under the terms of either this License or such Secondary 198 | License(s). 199 | 200 | 3.4. Notices 201 | 202 | You may not remove or alter the substance of any license notices 203 | (including copyright notices, patent notices, disclaimers of warranty, 204 | or limitations of liability) contained within the Source Code Form of 205 | the Covered Software, except that You may alter any license notices to 206 | the extent required to remedy known factual inaccuracies. 207 | 208 | 3.5. Application of Additional Terms 209 | 210 | You may choose to offer, and to charge a fee for, warranty, support, 211 | indemnity or liability obligations to one or more recipients of Covered 212 | Software. However, You may do so only on Your own behalf, and not on 213 | behalf of any Contributor. You must make it absolutely clear that any 214 | such warranty, support, indemnity, or liability obligation is offered by 215 | You alone, and You hereby agree to indemnify every Contributor for any 216 | liability incurred by such Contributor as a result of warranty, support, 217 | indemnity or liability terms You offer. You may include additional 218 | disclaimers of warranty and limitations of liability specific to any 219 | jurisdiction. 220 | 221 | 4. Inability to Comply Due to Statute or Regulation 222 | --------------------------------------------------- 223 | 224 | If it is impossible for You to comply with any of the terms of this 225 | License with respect to some or all of the Covered Software due to 226 | statute, judicial order, or regulation then You must: (a) comply with 227 | the terms of this License to the maximum extent possible; and (b) 228 | describe the limitations and the code they affect. Such description must 229 | be placed in a text file included with all distributions of the Covered 230 | Software under this License. Except to the extent prohibited by statute 231 | or regulation, such description must be sufficiently detailed for a 232 | recipient of ordinary skill to be able to understand it. 233 | 234 | 5. Termination 235 | -------------- 236 | 237 | 5.1. The rights granted under this License will terminate automatically 238 | if You fail to comply with any of its terms. However, if You become 239 | compliant, then the rights granted under this License from a particular 240 | Contributor are reinstated (a) provisionally, unless and until such 241 | Contributor explicitly and finally terminates Your grants, and (b) on an 242 | ongoing basis, if such Contributor fails to notify You of the 243 | non-compliance by some reasonable means prior to 60 days after You have 244 | come back into compliance. Moreover, Your grants from a particular 245 | Contributor are reinstated on an ongoing basis if such Contributor 246 | notifies You of the non-compliance by some reasonable means, this is the 247 | first time You have received notice of non-compliance with this License 248 | from such Contributor, and You become compliant prior to 30 days after 249 | Your receipt of the notice. 250 | 251 | 5.2. If You initiate litigation against any entity by asserting a patent 252 | infringement claim (excluding declaratory judgment actions, 253 | counter-claims, and cross-claims) alleging that a Contributor Version 254 | directly or indirectly infringes any patent, then the rights granted to 255 | You by any and all Contributors for the Covered Software under Section 256 | 2.1 of this License shall terminate. 257 | 258 | 5.3. In the event of termination under Sections 5.1 or 5.2 above, all 259 | end user license agreements (excluding distributors and resellers) which 260 | have been validly granted by You or Your distributors under this License 261 | prior to termination shall survive termination. 262 | 263 | ************************************************************************ 264 | * * 265 | * 6. Disclaimer of Warranty * 266 | * ------------------------- * 267 | * * 268 | * Covered Software is provided under this License on an "as is" * 269 | * basis, without warranty of any kind, either expressed, implied, or * 270 | * statutory, including, without limitation, warranties that the * 271 | * Covered Software is free of defects, merchantable, fit for a * 272 | * particular purpose or non-infringing. The entire risk as to the * 273 | * quality and performance of the Covered Software is with You. * 274 | * Should any Covered Software prove defective in any respect, You * 275 | * (not any Contributor) assume the cost of any necessary servicing, * 276 | * repair, or correction. This disclaimer of warranty constitutes an * 277 | * essential part of this License. No use of any Covered Software is * 278 | * authorized under this License except under this disclaimer. * 279 | * * 280 | ************************************************************************ 281 | 282 | ************************************************************************ 283 | * * 284 | * 7. Limitation of Liability * 285 | * -------------------------- * 286 | * * 287 | * Under no circumstances and under no legal theory, whether tort * 288 | * (including negligence), contract, or otherwise, shall any * 289 | * Contributor, or anyone who distributes Covered Software as * 290 | * permitted above, be liable to You for any direct, indirect, * 291 | * special, incidental, or consequential damages of any character * 292 | * including, without limitation, damages for lost profits, loss of * 293 | * goodwill, work stoppage, computer failure or malfunction, or any * 294 | * and all other commercial damages or losses, even if such party * 295 | * shall have been informed of the possibility of such damages. This * 296 | * limitation of liability shall not apply to liability for death or * 297 | * personal injury resulting from such party's negligence to the * 298 | * extent applicable law prohibits such limitation. Some * 299 | * jurisdictions do not allow the exclusion or limitation of * 300 | * incidental or consequential damages, so this exclusion and * 301 | * limitation may not apply to You. * 302 | * * 303 | ************************************************************************ 304 | 305 | 8. Litigation 306 | ------------- 307 | 308 | Any litigation relating to this License may be brought only in the 309 | courts of a jurisdiction where the defendant maintains its principal 310 | place of business and such litigation shall be governed by laws of that 311 | jurisdiction, without reference to its conflict-of-law provisions. 312 | Nothing in this Section shall prevent a party's ability to bring 313 | cross-claims or counter-claims. 314 | 315 | 9. Miscellaneous 316 | ---------------- 317 | 318 | This License represents the complete agreement concerning the subject 319 | matter hereof. If any provision of this License is held to be 320 | unenforceable, such provision shall be reformed only to the extent 321 | necessary to make it enforceable. Any law or regulation which provides 322 | that the language of a contract shall be construed against the drafter 323 | shall not be used to construe this License against a Contributor. 324 | 325 | 10. Versions of the License 326 | --------------------------- 327 | 328 | 10.1. New Versions 329 | 330 | Mozilla Foundation is the license steward. Except as provided in Section 331 | 10.3, no one other than the license steward has the right to modify or 332 | publish new versions of this License. Each version will be given a 333 | distinguishing version number. 334 | 335 | 10.2. Effect of New Versions 336 | 337 | You may distribute the Covered Software under the terms of the version 338 | of the License under which You originally received the Covered Software, 339 | or under the terms of any subsequent version published by the license 340 | steward. 341 | 342 | 10.3. Modified Versions 343 | 344 | If you create software not governed by this License, and you want to 345 | create a new license for such software, you may create and use a 346 | modified version of this License if you rename the license and remove 347 | any references to the name of the license steward (except to note that 348 | such modified license differs from this License). 349 | 350 | 10.4. Distributing Source Code Form that is Incompatible With Secondary 351 | Licenses 352 | 353 | If You choose to distribute Source Code Form that is Incompatible With 354 | Secondary Licenses under the terms of this version of the License, the 355 | notice described in Exhibit B of this License must be attached. 356 | 357 | Exhibit A - Source Code Form License Notice 358 | ------------------------------------------- 359 | 360 | This Source Code Form is subject to the terms of the Mozilla Public 361 | License, v. 2.0. If a copy of the MPL was not distributed with this 362 | file, You can obtain one at http://mozilla.org/MPL/2.0/. 363 | 364 | Copyright (c) 2024 PKevin AI VTuber Developer 365 | 366 | If it is not possible or desirable to put the notice in a particular 367 | file, then You may include the notice in a location (such as a LICENSE 368 | file in a relevant directory) where a recipient would be likely to look 369 | for such a notice. 370 | 371 | You may add additional accurate notices of copyright ownership. 372 | 373 | Exhibit B - "Incompatible With Secondary Licenses" Notice 374 | --------------------------------------------------------- 375 | 376 | This Source Code Form is "Incompatible With Secondary Licenses", as 377 | defined by the Mozilla Public License, v. 2.0. 378 | -------------------------------------------------------------------------------- /Live_Chat/Live_Chat.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | import time 5 | import threading 6 | import queue 7 | import random 8 | 9 | import pytchat 10 | from twitchio.ext import commands as twitch_commands 11 | 12 | import AIVT_Config 13 | import AI_Vtuber_UI as aivtui 14 | import My_Tools.Token_Calculator as tokenC 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | Live_Chat_Status = { 26 | "YouTube_live_chat": False, 27 | "YouTube_live_chat_alive": False, 28 | "YouTube_live_chat_retry": False, 29 | "YouTube_live_chat_connect_fail_count": 0, 30 | "Twitch_live_chat": False, 31 | "llm_request_checker": False, 32 | } 33 | 34 | live_chat_status = {"Twitch_live_chat": False,} 35 | 36 | Live_chat_parameters = { 37 | "yt_channel_name": "AI VTuber Dev PKevin", 38 | "yt_live_id": "qTmJ4YkziEk", 39 | "yt_live_chat_vip_names": [], 40 | "yt_live_chat_ban_names": [], 41 | "yt_live_chat_log": True, 42 | "yt_live_chat_read_chat_now": False, 43 | "yt_response_chatroom": False, 44 | "yt_response_owner": False, 45 | "yt_response_vip": False, 46 | "yt_response_individual": False, 47 | "yt_wait_list_max": 10, 48 | "yt_chat_max_tokens": 128, 49 | "yt_chat_max_response": 5, 50 | 51 | "tw_channel_name": "pkevin_ai_vtuber_dev", 52 | "tw_live_chat_vip_names": [], 53 | "tw_live_chat_ban_names": [], 54 | "tw_live_chat_log": True, 55 | "tw_live_chat_read_chat_now": False, 56 | "tw_response_chatroom": False, 57 | "tw_response_owner": False, 58 | "tw_response_vip": False, 59 | "tw_response_individual": False, 60 | "tw_wait_list_max": 10, 61 | "tw_chat_max_tokens": 128, 62 | "tw_chat_max_response": 5, 63 | } 64 | 65 | YT_LC_wait_list = [] 66 | TW_LC_raw_list = [] 67 | TW_LC_wait_list = [] 68 | Live_Chat_LLM_wait_list = [] 69 | 70 | 71 | 72 | 73 | 74 | def YouTube_live_chat_connect(): 75 | global Live_Chat_Status, Live_chat_parameters, YT_live_chat 76 | 77 | try: 78 | YT_live_chat = pytchat.create(video_id=Live_chat_parameters["yt_live_id"]) 79 | print(f"!!! YouTube Live ID: {Live_chat_parameters['yt_live_id']} Connect Success !!!\n") 80 | Live_Chat_Status["YouTube_live_chat"] = True 81 | Live_Chat_Status["YouTube_live_chat_alive"] = True 82 | Live_Chat_Status["YouTube_live_chat_connect_fail_count"] = 0 83 | 84 | except Exception as e: 85 | Live_Chat_Status["YouTube_live_chat_connect_fail_count"] += 1 86 | 87 | if Live_Chat_Status["YouTube_live_chat_connect_fail_count"] >= 50: 88 | print(f"!!! YouTube Live ID: {Live_chat_parameters['yt_live_id']} Connect Failed !!!{e}\n") 89 | Live_Chat_Status["YouTube_live_chat"] = False 90 | Live_Chat_Status["YouTube_live_chat_alive"] = False 91 | Live_Chat_Status["YouTube_live_chat_connect_fail_count"] = 0 92 | 93 | 94 | def YouTube_live_chat_get_comments(): 95 | global Live_Chat_Status, Live_chat_parameters, YT_live_chat, YT_LC_wait_list 96 | 97 | YT_LC_wait_list = [] 98 | 99 | while Live_Chat_Status["YouTube_live_chat"]: 100 | while YT_live_chat.is_alive() and Live_Chat_Status["YouTube_live_chat"]: 101 | time.sleep(0.1) 102 | 103 | try: 104 | chat_data = YT_live_chat.get() 105 | chat_comments = list(chat_data.sync_items()) 106 | 107 | if chat_data and chat_comments: 108 | chat_list = [] 109 | 110 | for c in chat_comments: 111 | chat_author = c.author.name 112 | chat_raw = c.message 113 | 114 | if chat_author in Live_chat_parameters["yt_live_chat_ban_names"]: 115 | continue 116 | 117 | elif (c.author.isChatOwner and Live_chat_parameters["yt_response_owner"]) or chat_author in Live_chat_parameters["yt_live_chat_vip_names"]: 118 | chat_now = {chat_author: chat_raw} 119 | chat_list.append(chat_now) 120 | 121 | elif tokenC.num_tokens_from_conversation([{"role": "user", "content": chat_raw}], "gpt-3.5-turbo")-9 < Live_chat_parameters["yt_chat_max_tokens"]: 122 | chat_now = {chat_author: chat_raw} 123 | chat_list.append(chat_now) 124 | 125 | else: 126 | if Live_chat_parameters["yt_live_chat_log"]: 127 | print(f"!!! *{chat_author}* comment beyond max tokens - YouTube Live Chat !!!") 128 | 129 | if chat_list: 130 | YT_LC_wait_list.extend(chat_list) 131 | if Live_chat_parameters["yt_live_chat_log"]: 132 | chat_c = "\n\n".join([f"{key} : {d[key]}" for d in chat_list for key in d]) 133 | print(f"\nYouTube Live Chat ----------\n\n{chat_c}\n\n----------\n") 134 | 135 | except Exception as e: 136 | print(f"YouTube get livechat fail:\n{e}\n") 137 | break 138 | 139 | 140 | print(f"YouTube Live Chat Reonnecting...") 141 | Live_Chat_Status["YouTube_live_chat_alive"] = False 142 | while not Live_Chat_Status["YouTube_live_chat_alive"]: 143 | time.sleep(0.01) 144 | 145 | Live_Chat_Status["YouTube_live_chat"] = False 146 | 147 | def YouTube_live_chat_pick_comments(): 148 | global Live_Chat_Status, Live_chat_parameters, YT_LC_wait_list, Live_Chat_LLM_wait_list 149 | 150 | while Live_Chat_Status["YouTube_live_chat"]: 151 | if Live_chat_parameters["yt_response_chatroom"] and YT_LC_wait_list: 152 | c = len(YT_LC_wait_list) 153 | chat_list = [] 154 | while c > 0: 155 | c -= 1 156 | chat_list.append(YT_LC_wait_list.pop(0)) 157 | 158 | chat_list.reverse() 159 | 160 | owner_name = Live_chat_parameters["yt_channel_name"] 161 | owner_chat_now = None 162 | 163 | if Live_chat_parameters["yt_response_owner"] and chat_list: 164 | if any(owner_name in d for d in chat_list): 165 | for d in chat_list: 166 | if owner_name in d: 167 | owner_chat_now = d 168 | break 169 | 170 | chat_list = [d for d in chat_list if owner_name not in d] 171 | 172 | name = list(owner_chat_now.keys())[0] 173 | chat_now = owner_chat_now[name] 174 | Live_Chat_LLM_wait_list.append({"role": "youtube_chat", "content": f"{name} : {chat_now}"}) 175 | 176 | else: 177 | chat_list = [d for d in chat_list if owner_name not in d] 178 | 179 | 180 | vip_names = Live_chat_parameters["yt_live_chat_vip_names"] 181 | vip_chat_now = None 182 | 183 | if Live_chat_parameters["yt_response_vip"] and chat_list: 184 | for vip_name in vip_names: 185 | if any(vip_name in d for d in chat_list): 186 | for d in chat_list: 187 | if vip_name in d: 188 | vip_chat_now = d 189 | break 190 | 191 | chat_list = [d for d in chat_list if vip_name not in d] 192 | 193 | name = list(vip_chat_now.keys())[0] 194 | chat_now = vip_chat_now[name] 195 | Live_Chat_LLM_wait_list.append({"role": "youtube_chat", "content": f"{name} : {chat_now}"}) 196 | 197 | 198 | if chat_list: 199 | first_occurrences = {} 200 | for d in chat_list: 201 | key = next(iter(d)) 202 | if key not in first_occurrences: 203 | first_occurrences[key] = d 204 | 205 | keys_to_pick = list(first_occurrences.keys()) 206 | random_keys = random.sample(keys_to_pick, min(Live_chat_parameters["yt_chat_max_response"], len(keys_to_pick))) 207 | selected_comments = [first_occurrences[key] for key in random_keys] 208 | 209 | if Live_chat_parameters["yt_response_individual"]: 210 | for chat_content in selected_comments: 211 | name = list(chat_content.keys())[0] 212 | chat_now = chat_content[name] 213 | Live_Chat_LLM_wait_list.append({"role": "youtube_chat", "content": f"{name} : {chat_now}"}) 214 | 215 | else: 216 | chat_s = "\n".join([f"{key} : {d[key]}" for d in selected_comments for key in d]) 217 | Live_Chat_LLM_wait_list.append({"role": "youtube_chat", "content": chat_s}) 218 | 219 | time.sleep(0.1) 220 | 221 | 222 | 223 | 224 | 225 | def Twitch_live_chat_connect(live_chat_status, tw_lc_raw_list): 226 | try: 227 | tlc = Twitch_live_chat( 228 | channel=AIVT_Config.Twitch_user_name, 229 | token=AIVT_Config.Twitch_token, 230 | tw_lc_raw_list=tw_lc_raw_list 231 | ) 232 | tlc.run() 233 | 234 | while live_chat_status['Twitch_live_chat']: 235 | time.sleep(0.1) 236 | 237 | tlc.close() 238 | 239 | except Exception as e: 240 | print(f"!!! Twitch {AIVT_Config.Twitch_user_name} Disconnected !!!\n\n{e}\n\n") 241 | live_chat_status['Twitch_live_chat'] = False 242 | 243 | class Twitch_live_chat(twitch_commands.Bot): 244 | def __init__(self, channel, token, tw_lc_raw_list): 245 | super().__init__(token=token, prefix='!', initial_channels=[channel]) 246 | self.tw_lc_raw_list = tw_lc_raw_list 247 | 248 | async def event_ready(self): 249 | print(f"!!! Twitch {self.nick} Connected !!!\n") 250 | 251 | async def event_message(self, message): 252 | if message.echo: 253 | return 254 | 255 | formatted_message = f"{message.author.name} : {message.content}" 256 | self.tw_lc_raw_list.append(formatted_message) 257 | 258 | await self.handle_commands(message) 259 | 260 | 261 | def Twitch_live_chat_get_comments(): 262 | global Live_Chat_Status, Live_chat_parameters, TW_LC_raw_list, TW_LC_wait_list 263 | 264 | TW_LC_wait_list = [] 265 | 266 | while Live_Chat_Status["Twitch_live_chat"]: 267 | time.sleep(0.1) 268 | 269 | try: 270 | if TW_LC_raw_list: 271 | chat_list = [] 272 | 273 | TW_LC_raw_list_l = len(TW_LC_raw_list) 274 | while TW_LC_raw_list_l > 0: 275 | TW_LC_raw_list_l -= 1 276 | 277 | chat_author, chat_raw = TW_LC_raw_list.pop(0).split(" : ", 1) 278 | 279 | if chat_author in Live_chat_parameters["tw_live_chat_ban_names"]: 280 | continue 281 | 282 | elif (chat_author == Live_chat_parameters["tw_channel_name"] and Live_chat_parameters["tw_response_owner"]) or chat_author in Live_chat_parameters["tw_live_chat_vip_names"]: 283 | chat_now = {chat_author: chat_raw} 284 | chat_list.append(chat_now) 285 | 286 | elif tokenC.num_tokens_from_conversation([{"role": "user", "content": chat_raw}], "gpt-3.5-turbo")-9 < Live_chat_parameters["tw_chat_max_tokens"]: 287 | chat_now = {chat_author: chat_raw} 288 | chat_list.append(chat_now) 289 | 290 | else: 291 | if Live_chat_parameters["tw_live_chat_log"]: 292 | print(f"!!! *{chat_author}* comment beyond max tokens - Twitch Live Chat !!!") 293 | 294 | if chat_list: 295 | TW_LC_wait_list.extend(chat_list) 296 | if Live_chat_parameters["tw_live_chat_log"]: 297 | chat_c = "\n\n".join([f"{key} : {d[key]}" for d in chat_list for key in d]) 298 | print(f"\nTwitch Live Chat ----------\n\n{chat_c}\n\n----------\n") 299 | 300 | except: 301 | ''' 302 | Never gonna give you up 303 | Never gonna let you down 304 | Never gonna run around and desert you 305 | Never gonna make you cry 306 | Never gonna say goodbye 307 | Never gonna tell a lie and hurt you 308 | ''' 309 | 310 | def Twitch_live_chat_pick_comments(): 311 | global Live_Chat_Status, Live_chat_parameters, TW_LC_wait_list, Live_Chat_LLM_wait_list 312 | 313 | while Live_Chat_Status["Twitch_live_chat"]: 314 | if Live_chat_parameters["tw_response_chatroom"] and TW_LC_wait_list: 315 | c = len(TW_LC_wait_list) 316 | chat_list = [] 317 | while c > 0: 318 | c -= 1 319 | chat_list.append(TW_LC_wait_list.pop(0)) 320 | 321 | chat_list.reverse() 322 | 323 | owner_name = Live_chat_parameters["tw_channel_name"] 324 | owner_chat_now = None 325 | 326 | if Live_chat_parameters["tw_response_owner"] and chat_list: 327 | if any(owner_name in d for d in chat_list): 328 | for d in chat_list: 329 | if owner_name in d: 330 | owner_chat_now = d 331 | break 332 | 333 | chat_list = [d for d in chat_list if owner_name not in d] 334 | 335 | name = list(owner_chat_now.keys())[0] 336 | chat_now = owner_chat_now[name] 337 | Live_Chat_LLM_wait_list.append({"role": "twitch_chat", "content": f"{name} : {chat_now}"}) 338 | 339 | else: 340 | chat_list = [d for d in chat_list if owner_name not in d] 341 | 342 | vip_names = Live_chat_parameters["tw_live_chat_vip_names"] 343 | vip_chat_now = None 344 | 345 | if Live_chat_parameters["tw_response_vip"] and chat_list: 346 | for vip_name in vip_names: 347 | if any(vip_name in d for d in chat_list): 348 | for d in chat_list: 349 | if vip_name in d: 350 | vip_chat_now = d 351 | break 352 | 353 | chat_list = [d for d in chat_list if vip_name not in d] 354 | 355 | name = list(vip_chat_now.keys())[0] 356 | chat_now = vip_chat_now[name] 357 | Live_Chat_LLM_wait_list.append({"role": "twitch_chat", "content": f"{name} : {chat_now}"}) 358 | 359 | if chat_list: 360 | first_occurrences = {} 361 | for d in chat_list: 362 | key = next(iter(d)) 363 | if key not in first_occurrences: 364 | first_occurrences[key] = d 365 | 366 | keys_to_pick = list(first_occurrences.keys()) 367 | random_keys = random.sample(keys_to_pick, min(Live_chat_parameters["tw_chat_max_response"], len(keys_to_pick))) 368 | selected_comments = [first_occurrences[key] for key in random_keys] 369 | 370 | if Live_chat_parameters["tw_response_individual"]: 371 | for chat_content in selected_comments: 372 | name = list(chat_content.keys())[0] 373 | chat_now = chat_content[name] 374 | Live_Chat_LLM_wait_list.append({"role": "twitch_chat", "content": f"{name} : {chat_now}"}) 375 | 376 | else: 377 | chat_s = "\n".join([f"{key} : {d[key]}" for d in selected_comments for key in d]) 378 | Live_Chat_LLM_wait_list.append({"role": "twitch_chat", "content": chat_s}) 379 | 380 | time.sleep(0.1) 381 | 382 | 383 | 384 | 385 | 386 | def YouTube_LiveChat_boot_on(): 387 | global Live_Chat_Status 388 | 389 | YT_lcgc = threading.Thread(target=YouTube_live_chat_get_comments) 390 | YT_lcgc.start() 391 | 392 | YT_lcpc = threading.Thread(target=YouTube_live_chat_pick_comments) 393 | YT_lcpc.start() 394 | 395 | if not Live_Chat_Status["llm_request_checker"]: 396 | Live_Chat_Status["llm_request_checker"] = True 397 | LC_llmc = threading.Thread(target=Live_Chat_LLM_checker) 398 | LC_llmc.start() 399 | 400 | 401 | def Twitch_LiveChat_boot_on(): 402 | global Live_Chat_Status 403 | 404 | TW_lcgc = threading.Thread(target=Twitch_live_chat_get_comments) 405 | TW_lcgc.start() 406 | 407 | TW_lcpc = threading.Thread(target=Twitch_live_chat_pick_comments) 408 | TW_lcpc.start() 409 | 410 | if not Live_Chat_Status["llm_request_checker"]: 411 | Live_Chat_Status["llm_request_checker"] = True 412 | LC_llmc = threading.Thread(target=Live_Chat_LLM_checker) 413 | LC_llmc.start() 414 | 415 | 416 | 417 | 418 | 419 | def Live_Chat_LLM_checker(): 420 | global Live_Chat_Status, Live_chat_parameters, Live_Chat_LLM_wait_list 421 | 422 | Live_Chat_LLM_wait_list = [] 423 | 424 | while Live_Chat_Status["YouTube_live_chat"] or Live_Chat_Status["Twitch_live_chat"]: 425 | if Live_Chat_LLM_wait_list: 426 | ans_requst = Live_Chat_LLM_wait_list.pop(0) 427 | chat_role = ans_requst["role"] 428 | chat_now = ans_requst["content"] 429 | ai_name = aivtui.AIVT_Using_character 430 | 431 | llm_ans = queue.Queue() 432 | 433 | llm_rt = threading.Thread(target=aivtui.LLM_Request_thread, args=(ans_requst, llm_ans, )) 434 | llm_rt.start() 435 | llm_rt.join() 436 | 437 | ai_ans = llm_ans.get() 438 | 439 | if ai_ans != "": 440 | aivtui.GUI_AIVT_Speaking_wait_list.append( 441 | { 442 | "chat_role": chat_role, 443 | "chat_now": chat_now, 444 | "ai_ans": ai_ans, 445 | "ai_name": ai_name, 446 | } 447 | ) 448 | 449 | time.sleep(0.1) 450 | 451 | Live_Chat_Status["llm_request_checker"] = False 452 | 453 | 454 | 455 | 456 | 457 | 458 | 459 | 460 | 461 | 462 | if __name__ == "__main__": 463 | Live_Chat_Status["YouTube_live_chat"] = True 464 | Live_chat_parameters["yt_live_id"] = "qTmJ4YkziEk" 465 | Live_chat_parameters["yt_response_chatroom"] = True 466 | Live_chat_parameters["yt_response_owner"] = True 467 | Live_chat_parameters["yt_response_vip"] = True 468 | YouTube_live_chat_connect() 469 | YouTube_LiveChat_boot_on() 470 | 471 | 472 | 473 | 474 | -------------------------------------------------------------------------------- /Live_Chat/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/Live_Chat/__init__.py -------------------------------------------------------------------------------- /Mic_Record.py: -------------------------------------------------------------------------------- 1 | import time 2 | import threading 3 | import pyaudio 4 | import wave 5 | 6 | import keyboard 7 | 8 | import AI_Vtuber_UI as aivtui 9 | from My_Tools.AIVT_print import aprint 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | user_mic_status = { 21 | "mic_on": False, 22 | "mic_record_running": False, 23 | "mic_checker_running": False, 24 | "mic_hotkeys_using": False, 25 | "mic_hotkey_1_detecting": False, 26 | "mic_hotkey_1_using": False, 27 | "mic_hotkey_1": "`", 28 | "mic_hotkey_2_detecting": False, 29 | "mic_hotkey_2_using": False, 30 | "mic_hotkey_2": "caps lock", 31 | } 32 | 33 | User_Mic_parameters = { 34 | "user_mic_audio_path": "Audio/user_mic_record/input_user_mic.wav", 35 | "input_device_name": "", 36 | "channels": 1, 37 | "format": pyaudio.paInt16, 38 | "rate": 24000, 39 | "chunk": 1024, 40 | "minimum_duration": 1, 41 | } 42 | 43 | Audio_frames_out = [] 44 | Mic_hotkey_pressed = False 45 | 46 | 47 | 48 | def MC_Record(): 49 | global user_mic_status, User_Mic_parameters, Mic_hotkey_pressed, Audio_frames_out 50 | 51 | input_device_index = Get_available_input_devices_ID(User_Mic_parameters["input_device_name"]) 52 | CHUNK = User_Mic_parameters["chunk"] 53 | 54 | p = pyaudio.PyAudio() 55 | audio_stream = p.open( 56 | input=True, 57 | input_device_index=input_device_index, 58 | channels=User_Mic_parameters["channels"], 59 | format=User_Mic_parameters["format"], 60 | rate=User_Mic_parameters["rate"], 61 | frames_per_buffer=CHUNK, 62 | ) 63 | 64 | Mic_hotkey_pressed = False 65 | dmhp = threading.Thread(target=Detect_Mic_hotkey_pressed) 66 | dmhp.start() 67 | 68 | while user_mic_status["mic_on"]: 69 | if Mic_hotkey_pressed: 70 | frames = [] 71 | start_time = time.time() 72 | current_time = start_time 73 | 74 | while Mic_hotkey_pressed: 75 | data = audio_stream.read(CHUNK) 76 | frames.append(data) 77 | current_time = time.time() 78 | 79 | if current_time - start_time >= User_Mic_parameters["minimum_duration"]: 80 | Audio_frames_out.append(frames) 81 | aprint("* Record output *") 82 | 83 | else: 84 | aprint("*** Mic Record Cancel ***") 85 | 86 | time.sleep(0.1) 87 | 88 | user_mic_status["mic_record_running"] = False 89 | 90 | audio_stream.stop_stream() 91 | audio_stream.close() 92 | p.terminate() 93 | dmhp.join() 94 | print("* User Mic OFF *") 95 | 96 | 97 | def Detect_Mic_hotkey_pressed(): 98 | global user_mic_status, Mic_hotkey_pressed 99 | def on_key_event(e): 100 | global Mic_hotkey_pressed 101 | if e.event_type == keyboard.KEY_DOWN and user_mic_status["mic_hotkeys_using"]: 102 | if user_mic_status["mic_hotkey_1_using"] and user_mic_status["mic_hotkey_2_using"]: 103 | hotkeys = f"{user_mic_status['mic_hotkey_1']}+{user_mic_status['mic_hotkey_2']}" 104 | 105 | elif user_mic_status["mic_hotkey_1_using"]: 106 | hotkeys = user_mic_status['mic_hotkey_1'] 107 | 108 | else: 109 | hotkeys = user_mic_status['mic_hotkey_2'] 110 | 111 | if not Mic_hotkey_pressed and keyboard.is_pressed(hotkeys): 112 | aprint('** Start Mic Recording **') 113 | Mic_hotkey_pressed = True 114 | 115 | elif e.event_type == keyboard.KEY_UP and user_mic_status["mic_hotkeys_using"]: 116 | if Mic_hotkey_pressed and ((e.name == user_mic_status["mic_hotkey_1"] and user_mic_status["mic_hotkey_1_using"]) or (e.name == user_mic_status["mic_hotkey_2"] and user_mic_status["mic_hotkey_2_using"])): 117 | aprint('** End Mic Recording **') 118 | Mic_hotkey_pressed = False 119 | 120 | 121 | keyboard.hook(on_key_event) 122 | 123 | print("* User Mic ON *") 124 | user_mic_status["mic_record_running"] = True 125 | 126 | while user_mic_status["mic_on"]: 127 | time.sleep(0.1) 128 | 129 | Mic_hotkey_pressed = False 130 | 131 | keyboard.unhook_all() 132 | 133 | 134 | def MC_Output_checker(): 135 | global user_mic_status, Audio_frames_out 136 | 137 | user_mic_status["mic_checker_running"] = True 138 | 139 | while user_mic_status["mic_on"]: 140 | if Audio_frames_out: 141 | aprint("* output check *") 142 | mco = threading.Thread(target=aivtui.OpenAI_Whisper_thread, args=(Audio_frames_out.pop(0), )) 143 | mco.start() 144 | 145 | time.sleep(0.1) 146 | 147 | # Avoid mic off when MC_Record is still processing 148 | while user_mic_status["mic_record_running"]: 149 | time.sleep(0.1) 150 | 151 | if Audio_frames_out: 152 | aprint("* output check *") 153 | mco = threading.Thread(target=aivtui.OpenAI_Whisper_thread, args=(Audio_frames_out.pop(0), )) 154 | mco.start() 155 | mco.join() 156 | 157 | user_mic_status["mic_checker_running"] = False 158 | 159 | 160 | 161 | 162 | 163 | def save_audio2wav(audio_frames, save_path): 164 | global User_Mic_parameters 165 | 166 | p = pyaudio.PyAudio() 167 | 168 | wf = wave.open(save_path, 'wb') 169 | wf.setnchannels(User_Mic_parameters["channels"]) 170 | wf.setsampwidth(p.get_sample_size(User_Mic_parameters["format"])) 171 | wf.setframerate(User_Mic_parameters["rate"]) 172 | wf.writeframes(b''.join(audio_frames)) 173 | wf.close() 174 | 175 | 176 | 177 | 178 | 179 | def Get_key_press(): 180 | print("User Mic Hotkey Detecting(Press ESC to cancel)...") 181 | event = keyboard.read_event() 182 | 183 | if event.event_type == keyboard.KEY_DOWN: 184 | key_name = event.name 185 | if not key_name == "esc": 186 | print(f"Detected Key: {key_name}") 187 | 188 | else: 189 | print("User Mic Hotkey Cancel") 190 | 191 | return key_name 192 | 193 | 194 | 195 | 196 | 197 | def Available_Input_Device(): 198 | p = pyaudio.PyAudio() 199 | info = p.get_host_api_info_by_index(0) 200 | numdevices = info.get('deviceCount') 201 | 202 | for i in range(0, numdevices): 203 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxInputChannels')) > 0: 204 | print(f"Input Device id {i} - {p.get_device_info_by_host_api_device_index(0, i).get('name')}") 205 | 206 | p.terminate() 207 | 208 | 209 | def Get_available_input_devices_List(): 210 | p = pyaudio.PyAudio() 211 | info = p.get_host_api_info_by_index(0) 212 | numdevices = info.get('deviceCount') 213 | input_devices_list = [] 214 | for i in range(0, numdevices): 215 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxInputChannels')) > 0: 216 | input_devices_list.append(p.get_device_info_by_host_api_device_index(0, i).get('name')) 217 | 218 | p.terminate() 219 | return input_devices_list 220 | 221 | 222 | def Get_available_input_devices_ID(devices_name): 223 | p = pyaudio.PyAudio() 224 | info = p.get_host_api_info_by_index(0) 225 | numdevices = info.get('deviceCount') 226 | 227 | # find input device id by device name 228 | for i in range(0, numdevices): 229 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxInputChannels')) > 0: 230 | if devices_name == p.get_device_info_by_host_api_device_index(0, i).get('name'): 231 | p.terminate() 232 | return i 233 | 234 | # if the name is not in input device list 235 | #return default system audio input device 236 | for i in range(0, numdevices): 237 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxInputChannels')) > 0: 238 | p.terminate() 239 | return i 240 | 241 | p.terminate() 242 | return None 243 | 244 | 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | if __name__ == "__main__": 254 | Available_Input_Device() 255 | devices_id = Get_available_input_devices_ID("Never gonna make you cry never gonna say goodbye") 256 | print(f"Devices ID: {devices_id}") 257 | #Get_key_press() 258 | 259 | 260 | -------------------------------------------------------------------------------- /My_Tools/AIVT_print.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | import inspect 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | def aprint(print_content, start=""): 16 | print(print_content) 17 | ''' 18 | filename = os.path.basename(sys.argv[0]) 19 | current_frame = inspect.currentframe() 20 | caller_frame = inspect.getouterframes(current_frame, 2) 21 | function_name = caller_frame[1][3] 22 | print(f"{start}{filename} | {function_name} - {print_content}") 23 | ''' 24 | 25 | 26 | 27 | 28 | -------------------------------------------------------------------------------- /My_Tools/Token_Calculator.py: -------------------------------------------------------------------------------- 1 | import tiktoken 2 | 3 | def num_tokens_from_conversation(messages, model): 4 | try: 5 | encoding = tiktoken.encoding_for_model(model) 6 | 7 | except: 8 | encoding = tiktoken.get_encoding("cl100k_base") 9 | 10 | 11 | if model == "gpt-3.5-turbo": 12 | tokens_per_message = 4 13 | tokens_per_name = -1 14 | 15 | elif model == "gpt-4" or model == "gpt-4-turbo-preview": 16 | tokens_per_message = 3 17 | tokens_per_name = 1 18 | 19 | else: 20 | tokens_per_message = 4 21 | tokens_per_name = -1 22 | 23 | 24 | num_tokens = 0 25 | for message in messages: 26 | num_tokens += tokens_per_message 27 | 28 | for key, value in message.items(): 29 | num_tokens += len(encoding.encode(value)) 30 | 31 | if key == "name": 32 | num_tokens += tokens_per_name 33 | 34 | num_tokens += 3 35 | return num_tokens 36 | 37 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /My_Tools/Token_Calculator_GUI.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | cd.. 3 | GUI_control_panel\runtime\python.exe My_Tools\Token_Calculator_GUI.py -------------------------------------------------------------------------------- /My_Tools/Token_Calculator_GUI.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | 5 | import tkinter as tk 6 | from tkinter import font 7 | from tkinter import ttk 8 | 9 | import My_Tools.Token_Calculator as tokenC 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | def calculate_tokens(text): 21 | con = [{"role": "user", "content": text}] 22 | tokens = tokenC.num_tokens_from_conversation(con, "gpt-3.5-turbo") 23 | return tokens-9 24 | 25 | 26 | class TokenCalculatorApp(tk.Tk): 27 | def __init__(self): 28 | super().__init__() 29 | 30 | self.title('Token 計算器') 31 | self.configure(bg='black') 32 | 33 | self.text_font = tk.StringVar(value="Arial") 34 | self.text_size = tk.IntVar(value=14) # 文字大小的變量 35 | 36 | self.init_ui() 37 | 38 | def init_ui(self): 39 | control_frame = ttk.Frame(self) 40 | control_frame.pack(fill=tk.X, padx=10, pady=5) 41 | 42 | # 創建字體選擇下拉菜單 43 | font_selector = ttk.Combobox(control_frame, textvariable=self.text_font, values=font.families(), width=20) 44 | font_selector.grid(row=0, column=0, padx=5, pady=5) 45 | font_selector.bind('<>', self.update_text_font) 46 | 47 | # 創建調整文字大小的 Spinbox 48 | size_spinbox = ttk.Spinbox(control_frame, from_=2, to=128, increment=2, textvariable=self.text_size, command=self.update_text_size, width=7) 49 | size_spinbox.grid(row=0, column=1, padx=5, pady=5) 50 | 51 | # 創建計算tokens的按鈕 52 | calculate_button = tk.Button(control_frame, text="Count it!", command=self.calculate_and_display_tokens, bg="grey", fg="white") 53 | calculate_button.grid(row=0, column=2, padx=5, pady=5) 54 | 55 | # 創建顯示tokens的標籤 56 | self.tokens_label = tk.Label(self, text="Tokens: 0", bg="black", fg="white", font=("Arial", 20)) 57 | self.tokens_label.pack(padx=10, pady=5) 58 | 59 | # 文字輸入框外層的滾動條 60 | text_frame = ttk.Frame(self) 61 | text_frame.pack(fill=tk.BOTH, expand=True, padx=10, pady=10) 62 | self.text_scroll = tk.Scrollbar(text_frame) 63 | self.text_scroll.pack(side=tk.RIGHT, fill=tk.Y) 64 | 65 | # 創建文本輸入框，並與滾動條連接 66 | self.text_input = tk.Text(text_frame, bg="grey", fg="white", 67 | font=(self.text_font.get(), self.text_size.get()), 68 | yscrollcommand=self.text_scroll.set) 69 | self.text_input.pack(side=tk.LEFT, fill=tk.BOTH, expand=True) 70 | self.text_scroll.config(command=self.text_input.yview) 71 | 72 | def update_text_font(self, event=None): 73 | self.update_text_size() 74 | 75 | def update_text_size(self): 76 | new_font = (self.text_font.get(), self.text_size.get()) 77 | self.text_input.configure(font=new_font) 78 | 79 | def calculate_and_display_tokens(self): 80 | # 獲取文本框內的文本 81 | text = self.text_input.get("1.0", tk.END) 82 | # 計算tokens 83 | tokens = calculate_tokens(text) 84 | # 更新標籤顯示tokens數量 85 | self.tokens_label.configure(text=f"Tokens: {tokens}") 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | if __name__ == "__main__": 97 | app = TokenCalculatorApp() 98 | app.mainloop() 99 | 100 | 101 | 102 | 103 | -------------------------------------------------------------------------------- /My_Tools/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/My_Tools/__init__.py -------------------------------------------------------------------------------- /My_Tools/check_gpu.py: -------------------------------------------------------------------------------- 1 | import torch 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | def torch_check_is_gpu_available(): 13 | if torch.cuda.is_available(): 14 | print(f"GPU is available: {torch.cuda.get_device_name(0)}") 15 | 16 | else: 17 | print("GPU is not available") 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | if __name__ == '__main__': 29 | torch_check_is_gpu_available() 30 | input("Press any key to continue") 31 | 32 | 33 | 34 | 35 | -------------------------------------------------------------------------------- /My_Tools/check_gpu_torch.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | cd.. 3 | GUI_control_panel\runtime\python.exe My_Tools\check_gpu.py -------------------------------------------------------------------------------- /OBS_websocket/OBS_websocket.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | import time 5 | 6 | import obswebsocket as obsWS 7 | 8 | import AIVT_Config 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | obsws = obsWS.obsws(AIVT_Config.OBS_WebSockets_ip, AIVT_Config.OBS_WebSockets_port, AIVT_Config.OBS_WebSockets_password) 20 | 21 | OBS_Connected = False 22 | OBS_Requesting = False 23 | OBS_subtitles_formatter_versions = ["v2", "v3"] 24 | 25 | OBS_subtitles_parameters = { 26 | "subtitles_formatter": True, 27 | "subtitles_formatter_version": "v3", 28 | } 29 | 30 | OBS_chat_now_sub_parameters = { 31 | "sub_name": "AIVT_CSub", 32 | "show": True, 33 | "clear": False, 34 | "sub_time": 3.0, 35 | "sub_max_length": 20, 36 | "sub_english_char_length": 0.50, 37 | "sub_base_line_count": 5, 38 | "sub_end_delay": 0.0, 39 | 40 | "show_sub_filter_names": ["Sub Appear"], 41 | "show_sub_filter_start_delay": 0.00, 42 | "show_sub_filter_end_delay": 0.00, 43 | 44 | "hide_sub_filter_names": ["Sub Disappear"], 45 | "hide_sub_filter_start_delay": 0.00, 46 | "hide_sub_filter_end_delay": 0.00, 47 | } 48 | 49 | OBS_ai_ans_sub_parameters = { 50 | "sub_name": "AIVT_Sub", 51 | "show": True, 52 | "clear": False, 53 | "remove_original_text_wrap": False, 54 | "sub_max_length": 20, 55 | "sub_english_char_length": 0.50, 56 | "sub_base_line_count": 5, 57 | "sub_end_delay": 0.0, 58 | 59 | "show_sub_filter_names": ["T Sub Appear"], 60 | "show_sub_filter_start_delay": 0.00, 61 | "show_sub_filter_end_delay": 0.00, 62 | 63 | "hide_sub_filter_names": ["T Sub Disappear"], 64 | "hide_sub_filter_start_delay": 0.00, 65 | "hide_sub_filter_end_delay": 0.00, 66 | } 67 | 68 | 69 | 70 | 71 | 72 | def OBSws_connect(): 73 | global obsws, OBS_Connected 74 | 75 | try: 76 | obsws.connect() 77 | OBS_Connected = True 78 | print("!!! OBS WebSockets Connect Success !!!") 79 | 80 | except Exception as e: 81 | OBS_Connected = False 82 | print("!!! OBS WebSockets Connect Fail !!!\n{e}\n") 83 | 84 | 85 | def OBSws_disconnect(): 86 | global obsws, OBS_Connected, OBS_Requesting 87 | 88 | while OBS_Requesting: 89 | time.sleep(0.1) 90 | 91 | try: 92 | obsws.disconnect() 93 | OBS_Connected = False 94 | print("!!! OBS WebSockets Disconnect !!!") 95 | 96 | except Exception as e: 97 | OBS_Connected = True 98 | print(f"!!! OBS WebSockets Disconnect Fail !!!\n{e}\n") 99 | 100 | 101 | 102 | 103 | 104 | def Set_Source_Text(text_name, text, start_delay=0, end_delay=0): 105 | global obsws, OBS_Connected, OBS_Requesting 106 | 107 | if OBS_Connected: 108 | OBS_Requesting = True 109 | time.sleep(start_delay) 110 | 111 | obsws.call(obsWS.requests.SetInputSettings(inputName=text_name, inputSettings={"text": text})) 112 | 113 | time.sleep(end_delay) 114 | OBS_Requesting = False 115 | 116 | 117 | 118 | def Set_Source_Filter_Enabled(source_name, filter_names, filter_enabled, start_delay=0, end_delay=0): 119 | global obsws, OBS_Connected, OBS_Requesting 120 | 121 | if OBS_Connected: 122 | OBS_Requesting = True 123 | 124 | filter_names_list = [] 125 | if isinstance(filter_names, str): 126 | filter_names_list.append(filter_names) 127 | 128 | elif isinstance(filter_names, list): 129 | filter_names_list.extend(filter_names) 130 | 131 | 132 | time.sleep(start_delay) 133 | 134 | for filter_name in filter_names_list: 135 | obsws.call(obsWS.requests.SetSourceFilterEnabled(sourceName=source_name, filterName=filter_name, filterEnabled=filter_enabled)) 136 | 137 | time.sleep(end_delay) 138 | OBS_Requesting = False 139 | 140 | 141 | 142 | 143 | -------------------------------------------------------------------------------- /OBS_websocket/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/OBS_websocket/__init__.py -------------------------------------------------------------------------------- /OpenAI/gpt/OpenAI_GPT_API.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '../..'))) 4 | import threading 5 | import time 6 | import queue 7 | 8 | import openai 9 | 10 | import AIVT_Config 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | gpt_models_max_input_tokens = { 22 | "gpt-4o":128000, 23 | "gpt-4o-2024-11-20":128000, 24 | "gpt-4o-2024-08-06":128000, 25 | "gpt-4o-2024-05-13":128000, 26 | "gpt-4o-mini":128000, 27 | "gpt-4o-mini-2024-07-18":128000, 28 | "gpt-4.5-preview-2025-02-27":128000, 29 | } 30 | 31 | gpt_models_max_output_tokens = { 32 | "gpt-4o":16384, 33 | "gpt-4o-2024-11-20":16384, 34 | "gpt-4o-2024-08-06":16384, 35 | "gpt-4o-2024-05-13":4096, 36 | "gpt-4o-mini":16385, 37 | "gpt-4o-mini-2024-07-18":16385, 38 | "gpt-4.5-preview-2025-02-27":16385, 39 | } 40 | 41 | gpt_parameters = { 42 | "model": "gpt-4o-mini", 43 | "max_input_tokens" : 4096, 44 | "max_output_tokens" : 256, 45 | "temperature" : 1.00, 46 | "timeout" : 15, 47 | "retry" : 3, 48 | } 49 | 50 | 51 | 52 | 53 | 54 | def run_with_timeout_OpenAI_GPT_API( 55 | conversation, 56 | chatQ, 57 | model_name="gpt-4o-mini", 58 | max_output_tokens="256", 59 | temperature=1.00, 60 | timeout=15, 61 | retry=3, 62 | command=None, 63 | ): 64 | 65 | ans = queue.Queue() 66 | model = queue.Queue() 67 | prompt_tokens = queue.Queue() 68 | completion_tokens = queue.Queue() 69 | total_tokens = queue.Queue() 70 | 71 | start_time = time.time() 72 | 73 | OGAt = threading.Thread( 74 | target=OpenAI_GPT_API_thread, 75 | args=( 76 | conversation, 77 | ans, 78 | model, 79 | prompt_tokens, 80 | completion_tokens, 81 | total_tokens, 82 | ), 83 | kwargs={ 84 | "model_name":model_name, 85 | "max_output_tokens":max_output_tokens, 86 | "temperature":temperature, 87 | "retry":retry, 88 | }, 89 | ) 90 | 91 | OGAt.start() 92 | OGAt.join(timeout) 93 | 94 | if OGAt.is_alive(): 95 | return None 96 | 97 | else: 98 | llm_result = ans.get() 99 | if command != "no_print": 100 | end_time = time.time() 101 | print("\nOpenAI_GPT_Answer ----------\n") 102 | print(f"Model: {model.get()}") 103 | print(f"Duration: {end_time - start_time:.2f}s") 104 | print(f"Prompt tokens: {prompt_tokens.get()}") 105 | print(f"Completion tokens: {completion_tokens.get()}") 106 | print(f"Total tokens: {total_tokens.get()}\n") 107 | print(f"{chatQ}\n") 108 | print(f"GPT Answer : {llm_result}") 109 | print("\n----------\n") 110 | 111 | cleaned_llm_result = "\n".join(line.strip() for line in llm_result.splitlines() if line.strip()) 112 | return cleaned_llm_result 113 | 114 | 115 | def OpenAI_GPT_API_thread( 116 | conversation, 117 | ans, 118 | model, 119 | prompt_tokens, 120 | completion_tokens, 121 | total_tokens, 122 | model_name = "gpt-4o-mini", 123 | max_output_tokens = 256, 124 | temperature = 1.00, 125 | retry = 3, 126 | ): 127 | 128 | openai.api_key = AIVT_Config.openai_api_key 129 | 130 | reT = 0 131 | while reT < retry: 132 | reT += 1 133 | 134 | try: 135 | response = openai.chat.completions.create( 136 | model = model_name, 137 | messages = conversation, 138 | max_tokens = max_output_tokens, 139 | temperature = temperature, 140 | stream=False, 141 | ) 142 | 143 | model.put(response.model) 144 | prompt_tokens.put(response.usage.prompt_tokens) 145 | completion_tokens.put(response.usage.completion_tokens) 146 | total_tokens.put(response.usage.total_tokens) 147 | message = response.choices[0].message.content 148 | 149 | ans.put(message) 150 | return 151 | 152 | except Exception as e: 153 | if reT < retry: 154 | print(f"!!! OpenAI_GPT_API retry {reT} time !!!\n{e}\n") 155 | continue 156 | 157 | else: 158 | print(f"!!! OpenAI_GPT_API retry {reT} time !!!\n{e}\n") 159 | ans.put("") 160 | return 161 | 162 | 163 | 164 | 165 | -------------------------------------------------------------------------------- /OpenAI/gpt/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/OpenAI/gpt/__init__.py -------------------------------------------------------------------------------- /OpenAI/whisper/OpenAI_Whisper.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '../..'))) 4 | import threading 5 | import queue 6 | import time 7 | import glob 8 | import gc 9 | 10 | import whisper 11 | import torch 12 | 13 | from My_Tools.AIVT_print import aprint 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | Whisper_LANGUAGES = { 25 | "en": "english", 26 | "zh": "chinese", 27 | "ja": "japanese", 28 | "ko": "korean", 29 | "de": "german", 30 | "es": "spanish", 31 | "ru": "russian", 32 | "fr": "french", 33 | "pt": "portuguese", 34 | "tr": "turkish", 35 | "pl": "polish", 36 | "ca": "catalan", 37 | "nl": "dutch", 38 | "ar": "arabic", 39 | "sv": "swedish", 40 | "it": "italian", 41 | "id": "indonesian", 42 | "hi": "hindi", 43 | "fi": "finnish", 44 | "vi": "vietnamese", 45 | "he": "hebrew", 46 | "uk": "ukrainian", 47 | "el": "greek", 48 | "ms": "malay", 49 | "cs": "czech", 50 | "ro": "romanian", 51 | "da": "danish", 52 | "hu": "hungarian", 53 | "ta": "tamil", 54 | "no": "norwegian", 55 | "th": "thai", 56 | "ur": "urdu", 57 | "hr": "croatian", 58 | "bg": "bulgarian", 59 | "lt": "lithuanian", 60 | "la": "latin", 61 | "mi": "maori", 62 | "ml": "malayalam", 63 | "cy": "welsh", 64 | "sk": "slovak", 65 | "te": "telugu", 66 | "fa": "persian", 67 | "lv": "latvian", 68 | "bn": "bengali", 69 | "sr": "serbian", 70 | "az": "azerbaijani", 71 | "sl": "slovenian", 72 | "kn": "kannada", 73 | "et": "estonian", 74 | "mk": "macedonian", 75 | "br": "breton", 76 | "eu": "basque", 77 | "is": "icelandic", 78 | "hy": "armenian", 79 | "ne": "nepali", 80 | "mn": "mongolian", 81 | "bs": "bosnian", 82 | "kk": "kazakh", 83 | "sq": "albanian", 84 | "sw": "swahili", 85 | "gl": "galician", 86 | "mr": "marathi", 87 | "pa": "punjabi", 88 | "si": "sinhala", 89 | "km": "khmer", 90 | "sn": "shona", 91 | "yo": "yoruba", 92 | "so": "somali", 93 | "af": "afrikaans", 94 | "oc": "occitan", 95 | "ka": "georgian", 96 | "be": "belarusian", 97 | "tg": "tajik", 98 | "sd": "sindhi", 99 | "gu": "gujarati", 100 | "am": "amharic", 101 | "yi": "yiddish", 102 | "lo": "lao", 103 | "uz": "uzbek", 104 | "fo": "faroese", 105 | "ht": "haitian creole", 106 | "ps": "pashto", 107 | "tk": "turkmen", 108 | "nn": "nynorsk", 109 | "mt": "maltese", 110 | "sa": "sanskrit", 111 | "lb": "luxembourgish", 112 | "my": "myanmar", 113 | "bo": "tibetan", 114 | "tl": "tagalog", 115 | "mg": "malagasy", 116 | "as": "assamese", 117 | "tt": "tatar", 118 | "haw": "hawaiian", 119 | "ln": "lingala", 120 | "ha": "hausa", 121 | "ba": "bashkir", 122 | "jw": "javanese", 123 | "su": "sundanese", 124 | "yue": "cantonese", 125 | } 126 | 127 | whisper_status = { 128 | "loaded_model": "", 129 | "gui_selected_model": "base", 130 | "model_loding": False, 131 | } 132 | 133 | whisper_parameters = { 134 | "user_mic_language": "zh", 135 | "prompt": "", 136 | "max_tokens": 192, 137 | "temperature": 0.2, 138 | "timeout": 10, 139 | } 140 | 141 | model = None 142 | 143 | 144 | 145 | 146 | 147 | def get_available_model_names_list(): 148 | pattern = os.path.join("OpenAI/whisper/models/", "*.pt") 149 | pt_files = glob.glob(pattern) 150 | model_names_list = [os.path.splitext(os.path.basename(file))[0] for file in pt_files] 151 | return model_names_list 152 | 153 | 154 | 155 | def load_model(model_name="base"): 156 | global model, whisper_status 157 | 158 | whisper_status["model_loding"] = True 159 | aprint(f"Whisper model loading: {model_name}") 160 | 161 | try: 162 | if whisper_status["loaded_model"]: 163 | del model 164 | gc.collect() 165 | if torch.cuda.is_available(): 166 | torch.cuda.empty_cache() 167 | 168 | start_time = time.time() 169 | 170 | model = whisper.load_model(name=f"OpenAI/whisper/models/{model_name}.pt", ) 171 | 172 | end_time = time.time() 173 | 174 | aprint(f"!!! Whisper Loaded Model Success: {model_name} !!!\nLoading time: {end_time - start_time:.2f}s\n") 175 | whisper_status["loaded_model"] = model_name 176 | 177 | except Exception as e: 178 | aprint(f"!!! Whisper Loaded Model Fail: {model_name} !!!\n{e}\n") 179 | whisper_status["loaded_model"] = "" 180 | 181 | whisper_status["model_loding"] = False 182 | 183 | 184 | def unload_model(): 185 | global model, whisper_status 186 | 187 | if whisper_status["loaded_model"]: 188 | try: 189 | del model 190 | gc.collect() 191 | if torch.cuda.is_available(): 192 | torch.cuda.empty_cache() 193 | 194 | aprint(f"!!! Model Unloaded Success: {whisper_status['loaded_model']} !!!\n") 195 | whisper_status["loaded_model"] = "" 196 | model = None 197 | 198 | except Exception as e: 199 | aprint(f"!!! Model Unloaded fail: {whisper_status['loaded_model']} !!!\n{e}\n") 200 | 201 | 202 | 203 | 204 | 205 | def run_with_timeout_OpenAI_Whisper( 206 | audio_path = "audio/input_user_mic.wav", 207 | audio_language = "zh", 208 | prompt = "", 209 | max_tokens = 192, 210 | temperature = 0.2, 211 | timeout=10, 212 | ): 213 | global whisper_status 214 | 215 | while whisper_status["model_loding"]: 216 | time.sleep(0.1) 217 | 218 | if whisper_status["loaded_model"] == "": 219 | load_model(model_name=whisper_status["gui_selected_model"]) 220 | 221 | if whisper_status["loaded_model"] == "": 222 | print("\n!!! OpenAI Whisper Transcribe Fail !!!\n") 223 | return "" 224 | 225 | model_name = whisper_status["loaded_model"] 226 | 227 | transcribe_ans = queue.Queue() 228 | 229 | start_time = time.time() 230 | 231 | OWt = threading.Thread( 232 | target = OpenAI_Whisper_thread, 233 | args = (transcribe_ans, ), 234 | kwargs = { 235 | "audio_path": audio_path, 236 | "audio_language": audio_language, 237 | "prompt": prompt, 238 | "max_tokens": max_tokens, 239 | "temperature": temperature, 240 | }, 241 | ) 242 | 243 | OWt.start() 244 | OWt.join(timeout) 245 | 246 | if OWt.is_alive(): 247 | print("\n!!! OpenAI Whisper time out !!!\n") 248 | return "" 249 | 250 | else: 251 | end_time = time.time() 252 | whisper_result = transcribe_ans.get() 253 | print("\nOpenAI Whisper Local ----------\n") 254 | print(f"Model: {model_name}") 255 | print(f"Duration: {end_time - start_time:.2f}s\n") 256 | print(f"Transcribe: {whisper_result}") 257 | print("\n----------\n") 258 | return whisper_result 259 | 260 | 261 | def OpenAI_Whisper_thread( 262 | ans, 263 | audio_path="audio/input_user_mic.wav", 264 | audio_language="zh", 265 | prompt="", 266 | max_tokens=192, 267 | temperature=0.2, 268 | ): 269 | global model 270 | 271 | try: 272 | result = model.transcribe( 273 | audio = audio_path, 274 | language = audio_language, 275 | initial_prompt = prompt, 276 | temperature=temperature, 277 | sample_len=max_tokens, 278 | ) 279 | 280 | except Exception as e: 281 | aprint(f"!!! Fail Transcribing !!!\n{e}\n") 282 | ans.put("") 283 | return 284 | 285 | ans.put(result["text"]) 286 | return 287 | 288 | 289 | 290 | 291 | -------------------------------------------------------------------------------- /OpenAI/whisper/OpenAI_Whisper_API.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '../..'))) 4 | import threading 5 | import time 6 | import queue 7 | 8 | import openai 9 | 10 | import AIVT_Config 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | Whisper_LANGUAGES = { 22 | "en": "english", 23 | "zh": "chinese", 24 | "ja": "japanese", 25 | "ko": "korean", 26 | "de": "german", 27 | "es": "spanish", 28 | "ru": "russian", 29 | "fr": "french", 30 | "pt": "portuguese", 31 | "tr": "turkish", 32 | "pl": "polish", 33 | "ca": "catalan", 34 | "nl": "dutch", 35 | "ar": "arabic", 36 | "sv": "swedish", 37 | "it": "italian", 38 | "id": "indonesian", 39 | "hi": "hindi", 40 | "fi": "finnish", 41 | "vi": "vietnamese", 42 | "he": "hebrew", 43 | "uk": "ukrainian", 44 | "el": "greek", 45 | "ms": "malay", 46 | "cs": "czech", 47 | "ro": "romanian", 48 | "da": "danish", 49 | "hu": "hungarian", 50 | "ta": "tamil", 51 | "no": "norwegian", 52 | "th": "thai", 53 | "ur": "urdu", 54 | "hr": "croatian", 55 | "bg": "bulgarian", 56 | "lt": "lithuanian", 57 | "la": "latin", 58 | "mi": "maori", 59 | "ml": "malayalam", 60 | "cy": "welsh", 61 | "sk": "slovak", 62 | "te": "telugu", 63 | "fa": "persian", 64 | "lv": "latvian", 65 | "bn": "bengali", 66 | "sr": "serbian", 67 | "az": "azerbaijani", 68 | "sl": "slovenian", 69 | "kn": "kannada", 70 | "et": "estonian", 71 | "mk": "macedonian", 72 | "br": "breton", 73 | "eu": "basque", 74 | "is": "icelandic", 75 | "hy": "armenian", 76 | "ne": "nepali", 77 | "mn": "mongolian", 78 | "bs": "bosnian", 79 | "kk": "kazakh", 80 | "sq": "albanian", 81 | "sw": "swahili", 82 | "gl": "galician", 83 | "mr": "marathi", 84 | "pa": "punjabi", 85 | "si": "sinhala", 86 | "km": "khmer", 87 | "sn": "shona", 88 | "yo": "yoruba", 89 | "so": "somali", 90 | "af": "afrikaans", 91 | "oc": "occitan", 92 | "ka": "georgian", 93 | "be": "belarusian", 94 | "tg": "tajik", 95 | "sd": "sindhi", 96 | "gu": "gujarati", 97 | "am": "amharic", 98 | "yi": "yiddish", 99 | "lo": "lao", 100 | "uz": "uzbek", 101 | "fo": "faroese", 102 | "ht": "haitian creole", 103 | "ps": "pashto", 104 | "tk": "turkmen", 105 | "nn": "nynorsk", 106 | "mt": "maltese", 107 | "sa": "sanskrit", 108 | "lb": "luxembourgish", 109 | "my": "myanmar", 110 | "bo": "tibetan", 111 | "tl": "tagalog", 112 | "mg": "malagasy", 113 | "as": "assamese", 114 | "tt": "tatar", 115 | "haw": "hawaiian", 116 | "ln": "lingala", 117 | "ha": "hausa", 118 | "ba": "bashkir", 119 | "jw": "javanese", 120 | "su": "sundanese", 121 | "yue": "cantonese", 122 | } 123 | 124 | whisper_parameters = { 125 | "model":"whisper-1", 126 | "user_mic_language":"zh", 127 | "prompt":"", 128 | "temperature":0.2, 129 | "timeout":10, 130 | } 131 | 132 | 133 | 134 | 135 | 136 | def run_with_timeout_OpenAI_Whisper_API( 137 | model = "whisper-1", 138 | audio_path = "audio/input_user_mic.wav", 139 | audio_language = "zh", 140 | prompt = "", 141 | temperature = 0.2, 142 | timeout=10, 143 | ): 144 | 145 | transcribe_ans = queue.Queue() 146 | 147 | start_time = time.time() 148 | 149 | OWAt = threading.Thread( 150 | target = OpenAI_Whisper_API_thread, 151 | args = (transcribe_ans, ), 152 | kwargs = { 153 | "model": model, 154 | "audio_path": audio_path, 155 | "audio_language": audio_language, 156 | "prompt": prompt, 157 | "temperature": temperature, 158 | }, 159 | ) 160 | 161 | OWAt.start() 162 | OWAt.join(timeout) 163 | 164 | if OWAt.is_alive(): 165 | print("\n!!! OpenAI Whisper API time out !!!\n") 166 | return "" 167 | 168 | else: 169 | end_time = time.time() 170 | whisper_result = transcribe_ans.get() 171 | print("\nOpenAI Whisper API ----------\n") 172 | print(f"Duration: {end_time - start_time:.2f}s\n") 173 | print(f"Transcribe: {whisper_result}") 174 | print("\n----------\n") 175 | return whisper_result 176 | 177 | 178 | def OpenAI_Whisper_API_thread( 179 | ans, 180 | model = "whisper-1", 181 | audio_path = "audio/input_user_mic.wav", 182 | audio_language = "zh", 183 | prompt = "", 184 | temperature = 0.2, 185 | ): 186 | 187 | openai.api_key = AIVT_Config.openai_api_key 188 | 189 | try: 190 | audio_file = open(audio_path, "rb") 191 | transcript = openai.audio.transcriptions.create( 192 | model=model, 193 | file=audio_file, 194 | language=audio_language, 195 | prompt=prompt, 196 | temperature=temperature, 197 | ) 198 | 199 | ans.put(transcript.text) 200 | return 201 | 202 | except Exception as e: 203 | print(f"Error transcribing audio\n{e}\n") 204 | ans.put("") 205 | return 206 | 207 | 208 | 209 | 210 | -------------------------------------------------------------------------------- /OpenAI/whisper/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/OpenAI/whisper/__init__.py -------------------------------------------------------------------------------- /OpenAI/whisper/models/tiny.pt: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9 3 | size 75572083 4 | -------------------------------------------------------------------------------- /Play_Audio.py: -------------------------------------------------------------------------------- 1 | import os 2 | import io 3 | import wave 4 | current_dir = os.path.dirname(os.path.abspath(__file__)) 5 | bin_dir = os.path.join(current_dir, "GUI_control_panel") 6 | bin_dir = os.path.abspath(bin_dir) 7 | os.environ["PATH"] += os.pathsep + bin_dir 8 | 9 | import pyaudio 10 | from pydub import AudioSegment 11 | ffmpeg_path = os.path.join(bin_dir, "ffmpeg.exe") 12 | AudioSegment.converter = ffmpeg_path 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | play_audio_parameters = { 24 | "ai_voice_output_device_name": "", 25 | "chunk": 1024, 26 | } 27 | 28 | 29 | 30 | def PlayAudio(audio_path, output_device_name, command=None): 31 | global play_audio_parameters 32 | 33 | output_device_index = Get_available_output_devices_ID(output_device_name) 34 | 35 | _, file_extension = os.path.splitext(audio_path) 36 | if file_extension.lower() == '.wav': 37 | audio_data = open(audio_path, 'rb') 38 | else: 39 | audio = AudioSegment.from_file(audio_path) 40 | audio_data = io.BytesIO() 41 | audio.export(audio_data, format="wav") 42 | audio_data.seek(0) 43 | 44 | wf = wave.open(audio_data, 'rb') 45 | p = pyaudio.PyAudio() 46 | 47 | CHUNK = play_audio_parameters["chunk"] 48 | stream = p.open( 49 | output=True, 50 | output_device_index=output_device_index, 51 | channels=wf.getnchannels(), 52 | format=p.get_format_from_width(wf.getsampwidth()), 53 | rate=wf.getframerate(), 54 | ) 55 | 56 | data = wf.readframes(CHUNK) 57 | 58 | while data: 59 | stream.write(data) 60 | data = wf.readframes(CHUNK) 61 | 62 | stream.stop_stream() 63 | stream.close() 64 | 65 | p.terminate() 66 | 67 | 68 | 69 | 70 | 71 | def Available_output_devices(): 72 | p = pyaudio.PyAudio() 73 | info = p.get_host_api_info_by_index(0) 74 | numdevices = info.get('deviceCount') 75 | 76 | for i in range(0, numdevices): 77 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxOutputChannels')) > 0: 78 | print(f"Output Device id {i} - {p.get_device_info_by_host_api_device_index(0, i).get('name')}") 79 | 80 | p.terminate() 81 | 82 | 83 | def Get_available_output_devices_List(): 84 | p = pyaudio.PyAudio() 85 | info = p.get_host_api_info_by_index(0) 86 | numdevices = info.get('deviceCount') 87 | output_devices_list = [] 88 | for i in range(0, numdevices): 89 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxOutputChannels')) > 0: 90 | output_devices_list.append(p.get_device_info_by_host_api_device_index(0, i).get('name')) 91 | 92 | p.terminate() 93 | return output_devices_list 94 | 95 | 96 | def Get_available_output_devices_ID(devices_name): 97 | p = pyaudio.PyAudio() 98 | info = p.get_host_api_info_by_index(0) 99 | numdevices = info.get('deviceCount') 100 | 101 | # find output device id by device name 102 | for i in range(0, numdevices): 103 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxOutputChannels')) > 0: 104 | if devices_name == p.get_device_info_by_host_api_device_index(0, i).get('name'): 105 | p.terminate() 106 | return i 107 | 108 | # if the name is not in output device list 109 | #return default system audio output device 110 | for i in range(0, numdevices): 111 | if (p.get_device_info_by_host_api_device_index(0, i).get('maxOutputChannels')) > 0: 112 | p.terminate() 113 | return i 114 | 115 | p.terminate() 116 | return None 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | if __name__ == "__main__": 128 | Available_output_devices() 129 | 130 | 131 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AI-VTuber-System 2 | A graphical system program that allows you to quickly create your own AI VTuber for free. 3 | ![00_程式視窗](https://github.com/AIVTDevPKevin/AI-VTuber-System/assets/161304793/6f3bbb2b-a2f2-46b0-b1ce-49e47a7ae285) 4 | 5 | ## Tutorial Video 6 | https://www.youtube.com/watch?v=Hwss_p2Iroc 7 | 8 | ## User Manual 9 | https://docs.google.com/document/d/16DU-DJKMaC-15K6iShLd9ioXc8VqjTLqgMPswsjPjF0/edit?usp=sharing 10 | 11 | ## Installation Guide 12 | Python >= 3.8, install the main dependencies: 13 | ``` 14 | pip3 install -r requirements.txt 15 | ``` 16 | For the specific PyTorch packages which require a special handling, use the following command: 17 | ``` 18 | pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 19 | ``` 20 | ## Nvidia 21 | Latest GPU Driver 22 | https://www.nvidia.com.tw/Download/index.aspx?lang=tw 23 | 24 | CUDA Toolkit 12.1.1 25 | https://developer.nvidia.com/cuda-12-1-1-download-archive 26 | 27 | cuDNN 28 | https://developer.nvidia.com/cudnn-downloads 29 | 30 | ## Whisper 31 | Excerpt from https://github.com/openai/whisper 32 | 33 | There are six model sizes, four with English-only versions, offering speed and accuracy tradeoffs. 34 | Below are the names of the available models and their approximate memory requirements and inference speed relative to the large model. 35 | The relative speeds below are measured by transcribing English speech on a A100, and the real-world speed may vary significantly depending on many factors including the language, the speaking speed, and the available hardware. 36 | 37 | | Size | Parameters | English-only model | Multilingual model | Required VRAM | Relative speed | 38 | |:------:|:----------:|:------------------:|:------------------:|:-------------:|:--------------:| 39 | | tiny | 39 M | `tiny.en` | `tiny` | ~1 GB | ~10x | 40 | | base | 74 M | `base.en` | `base` | ~1 GB | ~7x | 41 | | small | 244 M | `small.en` | `small` | ~2 GB | ~4x | 42 | | medium | 769 M | `medium.en` | `medium` | ~5 GB | ~2x | 43 | | large | 1550 M | N/A | `large` | ~10 GB | 1x | 44 | | turbo | 809 M | N/A | `turbo` | ~6 GB | ~8x | 45 | 46 | The `.en` models for English-only applications tend to perform better, especially for the `tiny.en` and `base.en` models. We observed that the difference becomes less significant for the `small.en` and `medium.en` models. 47 | Additionally, the `turbo` model is an optimized version of `large-v3` that offers faster transcription speed with a minimal degradation in accuracy. 48 | 49 | Whisper's performance varies widely depending on the language. The figure below shows a performance breakdown of `large-v3` and `large-v2` models by language, using WERs (word error rates) or CER (character error rates, shown in *Italic*) evaluated on the Common Voice 15 and Fleurs datasets. Additional WER/CER metrics corresponding to the other models and datasets can be found in Appendix D.1, D.2, and D.4 of [the paper](https://arxiv.org/abs/2212.04356), as well as the BLEU (Bilingual Evaluation Understudy) scores for translation in Appendix D.3. 50 | 51 | ![WER breakdown by language](https://github.com/openai/whisper/assets/266841/f4619d66-1058-4005-8f67-a9d811b77c62) 52 | 53 | 54 | ## Whisper Model Download Links 55 | tiny.en: https://openaipublic.azureedge.net/main/whisper/models/d3dd57d32accea0b295c96e26691aa14d8822fac7d9d27d5dc00b4ca2826dd03/tiny.en.pt 56 | 57 | tiny: https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt 58 | 59 | base.en: https://openaipublic.azureedge.net/main/whisper/models/25a8566e1d0c1e2231d1c762132cd20e0f96a85d16145c3a00adf5d1ac670ead/base.en.pt 60 | 61 | base: https://openaipublic.azureedge.net/main/whisper/models/ed3a0b6b1c0edf879ad9b11b1af5a0e6ab5db9205f891f668f8b0e6c6326e34e/base.pt 62 | 63 | small.en: https://openaipublic.azureedge.net/main/whisper/models/f953ad0fd29cacd07d5a9eda5624af0f6bcf2258be67c92b79389873d91e0872/small.en.pt 64 | 65 | small: https://openaipublic.azureedge.net/main/whisper/models/9ecf779972d90ba49c06d968637d720dd632c55bbf19d441fb42bf17a411e794/small.pt 66 | 67 | medium.en: https://openaipublic.azureedge.net/main/whisper/models/d7440d1dc186f76616474e0ff0b3b6b879abc9d1a4926b7adfa41db2d497ab4f/medium.en.pt 68 | 69 | medium: https://openaipublic.azureedge.net/main/whisper/models/345ae4da62f9b3d59415adc60127b97c714f32e89e936602e85993674d08dcb1/medium.pt 70 | 71 | large-v1: https://openaipublic.azureedge.net/main/whisper/models/e4b87e7e0bf463eb8e6956e646f1e277e901512310def2c24bf0e11bd3c28e9a/large-v1.pt 72 | 73 | large-v2: https://openaipublic.azureedge.net/main/whisper/models/81f7c96c852ee8fc832187b0132e569d6c3065a3252ed18e56effd0b6a73e524/large-v2.pt 74 | 75 | large-v3: https://openaipublic.azureedge.net/main/whisper/models/e5b1a55b89c1367dacf97e3e19bfd829a01529dbfdeefa8caeb59b3f1b81dadb/large-v3.pt 76 | 77 | turbo: https://openaipublic.azureedge.net/main/whisper/models/aff26ae408abcba5fbf8813c21e62b0941638c5f6eebfb145be0c9839262a19a/large-v3-turbo.pt 78 | 79 | ## Star History 80 | 81 | [![Star History Chart](https://api.star-history.com/svg?repos=AIVTDevPKevin/AI-VTuber-System&type=Date)](https://www.star-history.com/#AIVTDevPKevin/AI-VTuber-System&Date) 82 | -------------------------------------------------------------------------------- /Sentiment_Analysis/NLP_API.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | import string 5 | import time 6 | from collections import Counter 7 | 8 | import Google.gemini.GoogleAI_Gemini_API as gemini_api 9 | import OpenAI.gpt.OpenAI_GPT_API as gpt_api 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | gemini_model_names_list = [name for name in gemini_api.gemini_models_max_input_tokens.keys()] 21 | gpt_model_names_list = [name for name in gpt_api.gpt_models_max_input_tokens.keys()] 22 | model_names_list = gemini_model_names_list + gpt_model_names_list 23 | 24 | 25 | 26 | 27 | 28 | def Sentiment_Analysis_NLP( 29 | text, 30 | Emo_state_categories=["normal", "happy", "shy", "proud", "shock", "sad", "angry", "embarrass", "afraid", "confuse"], 31 | model="gemini-2.0-flash", 32 | timeout=5, 33 | ): 34 | global gemini_model_names_list, gpt_model_names_list 35 | 36 | SA_system_prompt = f"You will be responsible for analysing the text sent to you and sending back the corresponding sentiment categories in English. According to the following categories of sentiment that I have defined: {Emo_state_categories}, just send back the one most likely sentiment, and only return the name of the sentiment. Don't send me back the text that was sent to you." 37 | conversation = [ 38 | {"role": "system", "content": SA_system_prompt}, 39 | {"role": "user", "content": "(・_・;) No, PK, why did you suddenly say this? Are you trying to steal my little fish? (＠_＠;)＼(◎o◎)／!"}, 40 | {"role": "assistant", "content": "confuse"}, 41 | {"role": "user", "content": "PK、君は悪いよ(／‵Д′)／~ ╧╧ C0はパクチーが一番嫌い(╬ﾟдﾟ) パク​​チーもセロリもこの世に出てはいけない(╯°□°)╯︵ ┻━┻"}, 42 | {"role": "assistant", "content": "angry"}, 43 | {"role": "user", "content": "(゜o゜) 欸欸欸！PK你在幹嘛？學貓叫嗎？ 笑死！這個PK就是遜啦！哈哈！ (≧∇≦)/ "}, # 🥲 44 | {"role": "assistant", "content": "happy"}, 45 | {"role": "user", "content": text}, 46 | ] 47 | 48 | start_time = time.time() 49 | 50 | if model in gemini_model_names_list: 51 | llm_result = gemini_api.run_with_timeout_GoogleAI_Gemini_API( 52 | conversation, 53 | "", 54 | model_name=model, 55 | max_output_tokens=10, 56 | temperature=0.2, 57 | timeout=timeout, 58 | retry=1, 59 | command="no_print", 60 | ) 61 | 62 | elif model in gpt_model_names_list: 63 | llm_result = gpt_api.run_with_timeout_OpenAI_GPT_API( 64 | conversation, 65 | "", 66 | model_name=model, 67 | max_output_tokens=10, 68 | temperature=0.2, 69 | timeout=timeout, 70 | retry=1, 71 | command="no_print", 72 | ) 73 | 74 | try: 75 | mcsw = most_common_specific_word(llm_result, Emo_state_categories) 76 | except Exception as e: 77 | print(f"\n{e}\n") 78 | mcsw = "normal" 79 | 80 | end_time = time.time() 81 | print("\nSentiment Analysis NLP ----------\n") 82 | print(f"Model: {model}") 83 | print(f"Duration: {end_time - start_time:.2f}s\n") 84 | print(f"Emotion State: {mcsw}") 85 | print("\n----------\n") 86 | return mcsw 87 | 88 | 89 | 90 | def most_common_specific_word(text, Emo_state_categories): 91 | punctuation = string.punctuation + "：" 92 | translator = str.maketrans(punctuation, " " * len(punctuation)) 93 | 94 | text = text.lower() 95 | text = text.translate(translator) 96 | words = text.split() 97 | word_counts = Counter(words) 98 | 99 | Emo_state_categories = [word.lower() for word in Emo_state_categories] 100 | result = {word: word_counts[word] for word in Emo_state_categories} 101 | most_common_word = max(result, key=result.get) 102 | return most_common_word 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | if __name__ == "__main__": 114 | 115 | emo_state = Sentiment_Analysis_NLP( 116 | "Never gonna make you cry never gonna say goodbye" 117 | ) 118 | 119 | 120 | 121 | 122 | -------------------------------------------------------------------------------- /Sentiment_Analysis/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/Sentiment_Analysis/__init__.py -------------------------------------------------------------------------------- /TextToSpeech/OpenAITTS.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | 5 | import openai 6 | 7 | import AIVT_Config 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | OpenAITTS_Voice_list = ["alloy", "echo", "fable", "onyx", "nova", "shimmer", ] 19 | OpenAITTS_Model_list = ["tts-1", "tts-1-hd", ] 20 | 21 | openaitts_parameters = { 22 | "output_path":"Audio/output_openaitts_01.wav", 23 | "model":"tts-1", 24 | "voice":"alloy", 25 | "speed":1.00, 26 | "format":"wav", 27 | "timeout":10.0, 28 | } 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | def openaitts( 40 | text="", 41 | output_path="Audio/output_openaitts_01.wav", 42 | model="tts-1", 43 | voice="alloy", 44 | speed=1.00, 45 | format="wav", 46 | timeout=10.0, 47 | ): 48 | 49 | openai.api_key = AIVT_Config.openai_api_key 50 | 51 | response = openai.audio.speech.create( 52 | input=text, 53 | model=model, 54 | voice=voice, 55 | speed=speed, 56 | response_format=format, 57 | timeout=timeout, 58 | ) 59 | 60 | try: 61 | response.write_to_file(output_path) 62 | 63 | except Exception as e: 64 | print(f"Error openai tts\n{e}\n") 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | if __name__ == "__main__": 76 | 77 | input_text = "Never gonna give you up never gonna let you down" 78 | 79 | openaitts( 80 | text=input_text, 81 | output_path="Audio/output_openaitts_01.wav", 82 | model="tts-1", 83 | voice="alloy", 84 | speed=1.00, 85 | format="wav", 86 | timeout=10.0, 87 | ) 88 | 89 | 90 | 91 | 92 | -------------------------------------------------------------------------------- /TextToSpeech/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/TextToSpeech/__init__.py -------------------------------------------------------------------------------- /TextToSpeech/edgeTTS.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | import asyncio 5 | 6 | import edge_tts 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | EdgeTTS_Voice_dict = {} 18 | 19 | edgetts_parameters = { 20 | "output_path":"Audio/output_edgetts_01.mp3", 21 | "voice":"zh-TW-HsiaoChenNeural", 22 | "pitch":"+0Hz", 23 | "rate":"+0%", 24 | "volume":"+0%", 25 | "timeout":10, 26 | } 27 | 28 | 29 | 30 | 31 | 32 | async def async_edgetts(text, output_path, voice, pitch, rate, volume, timeout) -> None: 33 | communicate = edge_tts.Communicate(text=text, voice=voice, pitch=pitch, rate=rate, volume=volume, receive_timeout=timeout) 34 | await communicate.save(output_path) 35 | 36 | 37 | def edgetts(text="", 38 | output_path="Audio/output_edgetts_01.mp3", 39 | voice="zh-TW-HsiaoChenNeural", 40 | pitch="+0Hz", rate="+0%", volume="+0%", timeout=10, 41 | ): 42 | asyncio.run(async_edgetts(text, output_path, voice, pitch, rate, volume, timeout)) 43 | 44 | 45 | 46 | 47 | 48 | def create_voices_dict(file_path): 49 | voices_dict = {"Male": [], "Female": []} 50 | 51 | with open(file_path, 'r', encoding='utf-8') as file: 52 | for line in file: 53 | if "Name" in line: 54 | name = line.split(":")[1].strip() 55 | 56 | elif "Gender" in line: 57 | gender = line.split(":")[1].strip() 58 | voices_dict[gender].append(name) 59 | 60 | return {gender: names for gender, names in voices_dict.items() if names} 61 | 62 | 63 | def filter_voices_by_gender(voices_dict, gender): 64 | if gender == "All": 65 | return [name for names in voices_dict.values() for name in names] 66 | else: 67 | return voices_dict.get(gender, []) 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | if __name__ == "__main__": 79 | 80 | # Pitch -100Hz ~ +100Hz 81 | # Rate -100% ~ +100% 82 | # Volume -100% ~ +100% 83 | 84 | 85 | input_text = "Never gonna give you up never gonna let you down" 86 | edgetts(text=input_text, 87 | output_path="Audio/output_edgetts_01.mp3", 88 | voice="en-US-MichelleNeural", 89 | pitch="+30Hz", 90 | rate="-20%", 91 | volume="+0%", 92 | ) 93 | 94 | 95 | ''' 96 | input_text = "注意看，這個男人太狠了！" 97 | edgetts(text=input_text, 98 | output_path="Audio/output_edgetts_01.mp3", 99 | voice="zh-CN-YunxiNeural", 100 | pitch="+0Hz", 101 | rate="+0%", 102 | volume="+0%", 103 | ) 104 | ''' 105 | 106 | ''' 107 | input_text = "廢話，車不改要怎麼騎？" 108 | edgetts(text=input_text, 109 | output_path="Audio/output_edgetts_01.mp3", 110 | voice="zh-TW-YunJheNeural", 111 | pitch="+0Hz", 112 | rate="+0%", 113 | volume="+0%", 114 | ) 115 | ''' 116 | 117 | ''' 118 | input_text = "寶貝，你好大喔，做得好累" 119 | edgetts(text=input_text, 120 | output_path="Audio/output_edgetts_01.mp3", 121 | voice="zh-TW-HsiaoChenNeural", 122 | pitch="+15Hz", 123 | rate="-10%", 124 | volume="+0%", 125 | ) 126 | ''' 127 | 128 | ''' 129 | input_text = "りしれ供さ小" 130 | edgetts(text=input_text, 131 | output_path="Audio/output_edgetts_01.mp3", 132 | voice="ja-JP-NanamiNeural", 133 | pitch="+15Hz", 134 | rate="-10%", 135 | volume="+0%", 136 | ) 137 | ''' 138 | 139 | 140 | 141 | 142 | -------------------------------------------------------------------------------- /TextToSpeech/edgeTTS_speakers.txt: -------------------------------------------------------------------------------- 1 | Name: zh-TW-HsiaoChenNeural 2 | Gender: Female 3 | 4 | Name: zh-TW-HsiaoYuNeural 5 | Gender: Female 6 | 7 | Name: zh-TW-YunJheNeural 8 | Gender: Male 9 | 10 | Name: zh-CN-XiaoxiaoNeural 11 | Gender: Female 12 | 13 | Name: zh-CN-XiaoyiNeural 14 | Gender: Female 15 | 16 | Name: zh-CN-YunjianNeural 17 | Gender: Male 18 | 19 | Name: zh-CN-YunxiNeural 20 | Gender: Male 21 | 22 | Name: zh-CN-YunxiaNeural 23 | Gender: Male 24 | 25 | Name: zh-CN-YunyangNeural 26 | Gender: Male 27 | 28 | Name: zh-CN-liaoning-XiaobeiNeural 29 | Gender: Female 30 | 31 | Name: zh-CN-shaanxi-XiaoniNeural 32 | Gender: Female 33 | 34 | Name: zh-HK-HiuGaaiNeural 35 | Gender: Female 36 | 37 | Name: zh-HK-HiuMaanNeural 38 | Gender: Female 39 | 40 | Name: zh-HK-WanLungNeural 41 | Gender: Male 42 | 43 | Name: en-AU-NatashaNeural 44 | Gender: Female 45 | 46 | Name: en-AU-WilliamNeural 47 | Gender: Male 48 | 49 | Name: en-CA-ClaraNeural 50 | Gender: Female 51 | 52 | Name: en-CA-LiamNeural 53 | Gender: Male 54 | 55 | Name: en-GB-LibbyNeural 56 | Gender: Female 57 | 58 | Name: en-GB-MaisieNeural 59 | Gender: Female 60 | 61 | Name: en-GB-RyanNeural 62 | Gender: Male 63 | 64 | Name: en-GB-SoniaNeural 65 | Gender: Female 66 | 67 | Name: en-GB-ThomasNeural 68 | Gender: Male 69 | 70 | Name: en-HK-SamNeural 71 | Gender: Male 72 | 73 | Name: en-HK-YanNeural 74 | Gender: Female 75 | 76 | Name: en-IE-ConnorNeural 77 | Gender: Male 78 | 79 | Name: en-IE-EmilyNeural 80 | Gender: Female 81 | 82 | Name: en-IN-NeerjaExpressiveNeural 83 | Gender: Female 84 | 85 | Name: en-IN-NeerjaNeural 86 | Gender: Female 87 | 88 | Name: en-IN-PrabhatNeural 89 | Gender: Male 90 | 91 | Name: en-KE-AsiliaNeural 92 | Gender: Female 93 | 94 | Name: en-KE-ChilembaNeural 95 | Gender: Male 96 | 97 | Name: en-NG-AbeoNeural 98 | Gender: Male 99 | 100 | Name: en-NG-EzinneNeural 101 | Gender: Female 102 | 103 | Name: en-NZ-MitchellNeural 104 | Gender: Male 105 | 106 | Name: en-NZ-MollyNeural 107 | Gender: Female 108 | 109 | Name: en-PH-JamesNeural 110 | Gender: Male 111 | 112 | Name: en-PH-RosaNeural 113 | Gender: Female 114 | 115 | Name: en-SG-LunaNeural 116 | Gender: Female 117 | 118 | Name: en-SG-WayneNeural 119 | Gender: Male 120 | 121 | Name: en-TZ-ElimuNeural 122 | Gender: Male 123 | 124 | Name: en-TZ-ImaniNeural 125 | Gender: Female 126 | 127 | Name: en-US-AnaNeural 128 | Gender: Female 129 | 130 | Name: en-US-AriaNeural 131 | Gender: Female 132 | 133 | Name: en-US-ChristopherNeural 134 | Gender: Male 135 | 136 | Name: en-US-EricNeural 137 | Gender: Male 138 | 139 | Name: en-US-GuyNeural 140 | Gender: Male 141 | 142 | Name: en-US-JennyNeural 143 | Gender: Female 144 | 145 | Name: en-US-MichelleNeural 146 | Gender: Female 147 | 148 | Name: en-US-RogerNeural 149 | Gender: Male 150 | 151 | Name: en-US-SteffanNeural 152 | Gender: Male 153 | 154 | Name: en-ZA-LeahNeural 155 | Gender: Female 156 | 157 | Name: en-ZA-LukeNeural 158 | Gender: Male 159 | 160 | Name: ja-JP-KeitaNeural 161 | Gender: Male 162 | 163 | Name: ja-JP-NanamiNeural 164 | Gender: Female 165 | 166 | Name: af-ZA-AdriNeural 167 | Gender: Female 168 | 169 | Name: af-ZA-WillemNeural 170 | Gender: Male 171 | 172 | Name: am-ET-AmehaNeural 173 | Gender: Male 174 | 175 | Name: am-ET-MekdesNeural 176 | Gender: Female 177 | 178 | Name: ar-AE-FatimaNeural 179 | Gender: Female 180 | 181 | Name: ar-AE-HamdanNeural 182 | Gender: Male 183 | 184 | Name: ar-BH-AliNeural 185 | Gender: Male 186 | 187 | Name: ar-BH-LailaNeural 188 | Gender: Female 189 | 190 | Name: ar-DZ-AminaNeural 191 | Gender: Female 192 | 193 | Name: ar-DZ-IsmaelNeural 194 | Gender: Male 195 | 196 | Name: ar-EG-SalmaNeural 197 | Gender: Female 198 | 199 | Name: ar-EG-ShakirNeural 200 | Gender: Male 201 | 202 | Name: ar-IQ-BasselNeural 203 | Gender: Male 204 | 205 | Name: ar-IQ-RanaNeural 206 | Gender: Female 207 | 208 | Name: ar-JO-SanaNeural 209 | Gender: Female 210 | 211 | Name: ar-JO-TaimNeural 212 | Gender: Male 213 | 214 | Name: ar-KW-FahedNeural 215 | Gender: Male 216 | 217 | Name: ar-KW-NouraNeural 218 | Gender: Female 219 | 220 | Name: ar-LB-LaylaNeural 221 | Gender: Female 222 | 223 | Name: ar-LB-RamiNeural 224 | Gender: Male 225 | 226 | Name: ar-LY-ImanNeural 227 | Gender: Female 228 | 229 | Name: ar-LY-OmarNeural 230 | Gender: Male 231 | 232 | Name: ar-MA-JamalNeural 233 | Gender: Male 234 | 235 | Name: ar-MA-MounaNeural 236 | Gender: Female 237 | 238 | Name: ar-OM-AbdullahNeural 239 | Gender: Male 240 | 241 | Name: ar-OM-AyshaNeural 242 | Gender: Female 243 | 244 | Name: ar-QA-AmalNeural 245 | Gender: Female 246 | 247 | Name: ar-QA-MoazNeural 248 | Gender: Male 249 | 250 | Name: ar-SA-HamedNeural 251 | Gender: Male 252 | 253 | Name: ar-SA-ZariyahNeural 254 | Gender: Female 255 | 256 | Name: ar-SY-AmanyNeural 257 | Gender: Female 258 | 259 | Name: ar-SY-LaithNeural 260 | Gender: Male 261 | 262 | Name: ar-TN-HediNeural 263 | Gender: Male 264 | 265 | Name: ar-TN-ReemNeural 266 | Gender: Female 267 | 268 | Name: ar-YE-MaryamNeural 269 | Gender: Female 270 | 271 | Name: ar-YE-SalehNeural 272 | Gender: Male 273 | 274 | Name: az-AZ-BabekNeural 275 | Gender: Male 276 | 277 | Name: az-AZ-BanuNeural 278 | Gender: Female 279 | 280 | Name: bg-BG-BorislavNeural 281 | Gender: Male 282 | 283 | Name: bg-BG-KalinaNeural 284 | Gender: Female 285 | 286 | Name: bn-BD-NabanitaNeural 287 | Gender: Female 288 | 289 | Name: bn-BD-PradeepNeural 290 | Gender: Male 291 | 292 | Name: bn-IN-BashkarNeural 293 | Gender: Male 294 | 295 | Name: bn-IN-TanishaaNeural 296 | Gender: Female 297 | 298 | Name: bs-BA-GoranNeural 299 | Gender: Male 300 | 301 | Name: bs-BA-VesnaNeural 302 | Gender: Female 303 | 304 | Name: ca-ES-EnricNeural 305 | Gender: Male 306 | 307 | Name: ca-ES-JoanaNeural 308 | Gender: Female 309 | 310 | Name: cs-CZ-AntoninNeural 311 | Gender: Male 312 | 313 | Name: cs-CZ-VlastaNeural 314 | Gender: Female 315 | 316 | Name: cy-GB-AledNeural 317 | Gender: Male 318 | 319 | Name: cy-GB-NiaNeural 320 | Gender: Female 321 | 322 | Name: da-DK-ChristelNeural 323 | Gender: Female 324 | 325 | Name: da-DK-JeppeNeural 326 | Gender: Male 327 | 328 | Name: de-AT-IngridNeural 329 | Gender: Female 330 | 331 | Name: de-AT-JonasNeural 332 | Gender: Male 333 | 334 | Name: de-CH-JanNeural 335 | Gender: Male 336 | 337 | Name: de-CH-LeniNeural 338 | Gender: Female 339 | 340 | Name: de-DE-AmalaNeural 341 | Gender: Female 342 | 343 | Name: de-DE-ConradNeural 344 | Gender: Male 345 | 346 | Name: de-DE-KatjaNeural 347 | Gender: Female 348 | 349 | Name: de-DE-KillianNeural 350 | Gender: Male 351 | 352 | Name: el-GR-AthinaNeural 353 | Gender: Female 354 | 355 | Name: el-GR-NestorasNeural 356 | Gender: Male 357 | 358 | Name: es-AR-ElenaNeural 359 | Gender: Female 360 | 361 | Name: es-AR-TomasNeural 362 | Gender: Male 363 | 364 | Name: es-BO-MarceloNeural 365 | Gender: Male 366 | 367 | Name: es-BO-SofiaNeural 368 | Gender: Female 369 | 370 | Name: es-CL-CatalinaNeural 371 | Gender: Female 372 | 373 | Name: es-CL-LorenzoNeural 374 | Gender: Male 375 | 376 | Name: es-CO-GonzaloNeural 377 | Gender: Male 378 | 379 | Name: es-CO-SalomeNeural 380 | Gender: Female 381 | 382 | Name: es-CR-JuanNeural 383 | Gender: Male 384 | 385 | Name: es-CR-MariaNeural 386 | Gender: Female 387 | 388 | Name: es-CU-BelkysNeural 389 | Gender: Female 390 | 391 | Name: es-CU-ManuelNeural 392 | Gender: Male 393 | 394 | Name: es-DO-EmilioNeural 395 | Gender: Male 396 | 397 | Name: es-DO-RamonaNeural 398 | Gender: Female 399 | 400 | Name: es-EC-AndreaNeural 401 | Gender: Female 402 | 403 | Name: es-EC-LuisNeural 404 | Gender: Male 405 | 406 | Name: es-ES-AlvaroNeural 407 | Gender: Male 408 | 409 | Name: es-ES-ElviraNeural 410 | Gender: Female 411 | 412 | Name: es-GQ-JavierNeural 413 | Gender: Male 414 | 415 | Name: es-GQ-TeresaNeural 416 | Gender: Female 417 | 418 | Name: es-GT-AndresNeural 419 | Gender: Male 420 | 421 | Name: es-GT-MartaNeural 422 | Gender: Female 423 | 424 | Name: es-HN-CarlosNeural 425 | Gender: Male 426 | 427 | Name: es-HN-KarlaNeural 428 | Gender: Female 429 | 430 | Name: es-MX-DaliaNeural 431 | Gender: Female 432 | 433 | Name: es-MX-JorgeNeural 434 | Gender: Male 435 | 436 | Name: es-NI-FedericoNeural 437 | Gender: Male 438 | 439 | Name: es-NI-YolandaNeural 440 | Gender: Female 441 | 442 | Name: es-PA-MargaritaNeural 443 | Gender: Female 444 | 445 | Name: es-PA-RobertoNeural 446 | Gender: Male 447 | 448 | Name: es-PE-AlexNeural 449 | Gender: Male 450 | 451 | Name: es-PE-CamilaNeural 452 | Gender: Female 453 | 454 | Name: es-PR-KarinaNeural 455 | Gender: Female 456 | 457 | Name: es-PR-VictorNeural 458 | Gender: Male 459 | 460 | Name: es-PY-MarioNeural 461 | Gender: Male 462 | 463 | Name: es-PY-TaniaNeural 464 | Gender: Female 465 | 466 | Name: es-SV-LorenaNeural 467 | Gender: Female 468 | 469 | Name: es-SV-RodrigoNeural 470 | Gender: Male 471 | 472 | Name: es-US-AlonsoNeural 473 | Gender: Male 474 | 475 | Name: es-US-PalomaNeural 476 | Gender: Female 477 | 478 | Name: es-UY-MateoNeural 479 | Gender: Male 480 | 481 | Name: es-UY-ValentinaNeural 482 | Gender: Female 483 | 484 | Name: es-VE-PaolaNeural 485 | Gender: Female 486 | 487 | Name: es-VE-SebastianNeural 488 | Gender: Male 489 | 490 | Name: et-EE-AnuNeural 491 | Gender: Female 492 | 493 | Name: et-EE-KertNeural 494 | Gender: Male 495 | 496 | Name: fa-IR-DilaraNeural 497 | Gender: Female 498 | 499 | Name: fa-IR-FaridNeural 500 | Gender: Male 501 | 502 | Name: fi-FI-HarriNeural 503 | Gender: Male 504 | 505 | Name: fi-FI-NooraNeural 506 | Gender: Female 507 | 508 | Name: fil-PH-AngeloNeural 509 | Gender: Male 510 | 511 | Name: fil-PH-BlessicaNeural 512 | Gender: Female 513 | 514 | Name: fr-BE-CharlineNeural 515 | Gender: Female 516 | 517 | Name: fr-BE-GerardNeural 518 | Gender: Male 519 | 520 | Name: fr-CA-AntoineNeural 521 | Gender: Male 522 | 523 | Name: fr-CA-JeanNeural 524 | Gender: Male 525 | 526 | Name: fr-CA-SylvieNeural 527 | Gender: Female 528 | 529 | Name: fr-CH-ArianeNeural 530 | Gender: Female 531 | 532 | Name: fr-CH-FabriceNeural 533 | Gender: Male 534 | 535 | Name: fr-FR-DeniseNeural 536 | Gender: Female 537 | 538 | Name: fr-FR-EloiseNeural 539 | Gender: Female 540 | 541 | Name: fr-FR-HenriNeural 542 | Gender: Male 543 | 544 | Name: ga-IE-ColmNeural 545 | Gender: Male 546 | 547 | Name: ga-IE-OrlaNeural 548 | Gender: Female 549 | 550 | Name: gl-ES-RoiNeural 551 | Gender: Male 552 | 553 | Name: gl-ES-SabelaNeural 554 | Gender: Female 555 | 556 | Name: gu-IN-DhwaniNeural 557 | Gender: Female 558 | 559 | Name: gu-IN-NiranjanNeural 560 | Gender: Male 561 | 562 | Name: he-IL-AvriNeural 563 | Gender: Male 564 | 565 | Name: he-IL-HilaNeural 566 | Gender: Female 567 | 568 | Name: hi-IN-MadhurNeural 569 | Gender: Male 570 | 571 | Name: hi-IN-SwaraNeural 572 | Gender: Female 573 | 574 | Name: hr-HR-GabrijelaNeural 575 | Gender: Female 576 | 577 | Name: hr-HR-SreckoNeural 578 | Gender: Male 579 | 580 | Name: hu-HU-NoemiNeural 581 | Gender: Female 582 | 583 | Name: hu-HU-TamasNeural 584 | Gender: Male 585 | 586 | Name: id-ID-ArdiNeural 587 | Gender: Male 588 | 589 | Name: id-ID-GadisNeural 590 | Gender: Female 591 | 592 | Name: is-IS-GudrunNeural 593 | Gender: Female 594 | 595 | Name: is-IS-GunnarNeural 596 | Gender: Male 597 | 598 | Name: it-IT-DiegoNeural 599 | Gender: Male 600 | 601 | Name: it-IT-ElsaNeural 602 | Gender: Female 603 | 604 | Name: it-IT-IsabellaNeural 605 | Gender: Female 606 | 607 | Name: jv-ID-DimasNeural 608 | Gender: Male 609 | 610 | Name: jv-ID-SitiNeural 611 | Gender: Female 612 | 613 | Name: ka-GE-EkaNeural 614 | Gender: Female 615 | 616 | Name: ka-GE-GiorgiNeural 617 | Gender: Male 618 | 619 | Name: kk-KZ-AigulNeural 620 | Gender: Female 621 | 622 | Name: kk-KZ-DauletNeural 623 | Gender: Male 624 | 625 | Name: km-KH-PisethNeural 626 | Gender: Male 627 | 628 | Name: km-KH-SreymomNeural 629 | Gender: Female 630 | 631 | Name: kn-IN-GaganNeural 632 | Gender: Male 633 | 634 | Name: kn-IN-SapnaNeural 635 | Gender: Female 636 | 637 | Name: ko-KR-InJoonNeural 638 | Gender: Male 639 | 640 | Name: ko-KR-SunHiNeural 641 | Gender: Female 642 | 643 | Name: lo-LA-ChanthavongNeural 644 | Gender: Male 645 | 646 | Name: lo-LA-KeomanyNeural 647 | Gender: Female 648 | 649 | Name: lt-LT-LeonasNeural 650 | Gender: Male 651 | 652 | Name: lt-LT-OnaNeural 653 | Gender: Female 654 | 655 | Name: lv-LV-EveritaNeural 656 | Gender: Female 657 | 658 | Name: lv-LV-NilsNeural 659 | Gender: Male 660 | 661 | Name: mk-MK-AleksandarNeural 662 | Gender: Male 663 | 664 | Name: mk-MK-MarijaNeural 665 | Gender: Female 666 | 667 | Name: ml-IN-MidhunNeural 668 | Gender: Male 669 | 670 | Name: ml-IN-SobhanaNeural 671 | Gender: Female 672 | 673 | Name: mn-MN-BataaNeural 674 | Gender: Male 675 | 676 | Name: mn-MN-YesuiNeural 677 | Gender: Female 678 | 679 | Name: mr-IN-AarohiNeural 680 | Gender: Female 681 | 682 | Name: mr-IN-ManoharNeural 683 | Gender: Male 684 | 685 | Name: ms-MY-OsmanNeural 686 | Gender: Male 687 | 688 | Name: ms-MY-YasminNeural 689 | Gender: Female 690 | 691 | Name: mt-MT-GraceNeural 692 | Gender: Female 693 | 694 | Name: mt-MT-JosephNeural 695 | Gender: Male 696 | 697 | Name: my-MM-NilarNeural 698 | Gender: Female 699 | 700 | Name: my-MM-ThihaNeural 701 | Gender: Male 702 | 703 | Name: nb-NO-FinnNeural 704 | Gender: Male 705 | 706 | Name: nb-NO-PernilleNeural 707 | Gender: Female 708 | 709 | Name: ne-NP-HemkalaNeural 710 | Gender: Female 711 | 712 | Name: ne-NP-SagarNeural 713 | Gender: Male 714 | 715 | Name: nl-BE-ArnaudNeural 716 | Gender: Male 717 | 718 | Name: nl-BE-DenaNeural 719 | Gender: Female 720 | 721 | Name: nl-NL-ColetteNeural 722 | Gender: Female 723 | 724 | Name: nl-NL-FennaNeural 725 | Gender: Female 726 | 727 | Name: nl-NL-MaartenNeural 728 | Gender: Male 729 | 730 | Name: pl-PL-MarekNeural 731 | Gender: Male 732 | 733 | Name: pl-PL-ZofiaNeural 734 | Gender: Female 735 | 736 | Name: ps-AF-GulNawazNeural 737 | Gender: Male 738 | 739 | Name: ps-AF-LatifaNeural 740 | Gender: Female 741 | 742 | Name: pt-BR-AntonioNeural 743 | Gender: Male 744 | 745 | Name: pt-BR-FranciscaNeural 746 | Gender: Female 747 | 748 | Name: pt-PT-DuarteNeural 749 | Gender: Male 750 | 751 | Name: pt-PT-RaquelNeural 752 | Gender: Female 753 | 754 | Name: ro-RO-AlinaNeural 755 | Gender: Female 756 | 757 | Name: ro-RO-EmilNeural 758 | Gender: Male 759 | 760 | Name: ru-RU-DmitryNeural 761 | Gender: Male 762 | 763 | Name: ru-RU-SvetlanaNeural 764 | Gender: Female 765 | 766 | Name: si-LK-SameeraNeural 767 | Gender: Male 768 | 769 | Name: si-LK-ThiliniNeural 770 | Gender: Female 771 | 772 | Name: sk-SK-LukasNeural 773 | Gender: Male 774 | 775 | Name: sk-SK-ViktoriaNeural 776 | Gender: Female 777 | 778 | Name: sl-SI-PetraNeural 779 | Gender: Female 780 | 781 | Name: sl-SI-RokNeural 782 | Gender: Male 783 | 784 | Name: so-SO-MuuseNeural 785 | Gender: Male 786 | 787 | Name: so-SO-UbaxNeural 788 | Gender: Female 789 | 790 | Name: sq-AL-AnilaNeural 791 | Gender: Female 792 | 793 | Name: sq-AL-IlirNeural 794 | Gender: Male 795 | 796 | Name: sr-RS-NicholasNeural 797 | Gender: Male 798 | 799 | Name: sr-RS-SophieNeural 800 | Gender: Female 801 | 802 | Name: su-ID-JajangNeural 803 | Gender: Male 804 | 805 | Name: su-ID-TutiNeural 806 | Gender: Female 807 | 808 | Name: sv-SE-MattiasNeural 809 | Gender: Male 810 | 811 | Name: sv-SE-SofieNeural 812 | Gender: Female 813 | 814 | Name: sw-KE-RafikiNeural 815 | Gender: Male 816 | 817 | Name: sw-KE-ZuriNeural 818 | Gender: Female 819 | 820 | Name: sw-TZ-DaudiNeural 821 | Gender: Male 822 | 823 | Name: sw-TZ-RehemaNeural 824 | Gender: Female 825 | 826 | Name: ta-IN-PallaviNeural 827 | Gender: Female 828 | 829 | Name: ta-IN-ValluvarNeural 830 | Gender: Male 831 | 832 | Name: ta-LK-KumarNeural 833 | Gender: Male 834 | 835 | Name: ta-LK-SaranyaNeural 836 | Gender: Female 837 | 838 | Name: ta-MY-KaniNeural 839 | Gender: Female 840 | 841 | Name: ta-MY-SuryaNeural 842 | Gender: Male 843 | 844 | Name: ta-SG-AnbuNeural 845 | Gender: Male 846 | 847 | Name: ta-SG-VenbaNeural 848 | Gender: Female 849 | 850 | Name: te-IN-MohanNeural 851 | Gender: Male 852 | 853 | Name: te-IN-ShrutiNeural 854 | Gender: Female 855 | 856 | Name: th-TH-NiwatNeural 857 | Gender: Male 858 | 859 | Name: th-TH-PremwadeeNeural 860 | Gender: Female 861 | 862 | Name: tr-TR-AhmetNeural 863 | Gender: Male 864 | 865 | Name: tr-TR-EmelNeural 866 | Gender: Female 867 | 868 | Name: uk-UA-OstapNeural 869 | Gender: Male 870 | 871 | Name: uk-UA-PolinaNeural 872 | Gender: Female 873 | 874 | Name: ur-IN-GulNeural 875 | Gender: Female 876 | 877 | Name: ur-IN-SalmanNeural 878 | Gender: Male 879 | 880 | Name: ur-PK-AsadNeural 881 | Gender: Male 882 | 883 | Name: ur-PK-UzmaNeural 884 | Gender: Female 885 | 886 | Name: uz-UZ-MadinaNeural 887 | Gender: Female 888 | 889 | Name: uz-UZ-SardorNeural 890 | Gender: Male 891 | 892 | Name: vi-VN-HoaiMyNeural 893 | Gender: Female 894 | 895 | Name: vi-VN-NamMinhNeural 896 | Gender: Male 897 | 898 | Name: zu-ZA-ThandoNeural 899 | Gender: Female 900 | 901 | Name: zu-ZA-ThembaNeural 902 | Gender: Male 903 | -------------------------------------------------------------------------------- /VTubeStudioPlugin/VTubeStudioPlugin.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 4 | import asyncio 5 | import threading 6 | import random 7 | 8 | import pyvts 9 | 10 | from My_Tools.AIVT_print import aprint 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | AIVT_VTSP_Status = { 22 | "authenticated": False, 23 | "hotkey_trigger": False, 24 | } 25 | 26 | AIVT_hotkeys_parameters = { 27 | "trigger_first": False, 28 | "Emo_state_categories": ["normal", "happy", "shy", "proud", "shock", "sad", "angry", "embarrass", "afraid", "confuse"], 29 | "sentiment_analysis": False, 30 | "sentiment_analysis_model": "gemini-1.0-pro", 31 | "idle_ani": "", 32 | "idle_ani_previous": "", 33 | "normal": True, 34 | "normal_kn": "", 35 | "happy": True, 36 | "happy_kn": "", 37 | "shy": True, 38 | "shy_kn": "", 39 | "proud": True, 40 | "proud_kn": "", 41 | "shock": True, 42 | "shock_kn": "", 43 | "sad": True, 44 | "sad_kn": "", 45 | "angry": True, 46 | "angry_kn": "", 47 | "embarrass": True, 48 | "embarrass_kn": "", 49 | "afraid": True, 50 | "afraid_kn": "", 51 | "confuse": True, 52 | "confuse_kn": "", 53 | } 54 | 55 | AIVT_previous_ani_exp = { 56 | "idle_ani": "", 57 | "talking_ani": "", 58 | "talking_exp": "", 59 | } 60 | 61 | 62 | 63 | 64 | 65 | def AIVT_VTSP_authenticated(): 66 | global AIVT_VTSP_Status, aivt_vtsp 67 | 68 | authentication_token_path = "VTubeStudioPlugin/VTSP_authentication_token.txt" 69 | 70 | AIVT_VTSP_info = { 71 | "plugin_name":"AIVT_VTSP", 72 | "developer":"PKevin", 73 | "authentication_token_path":authentication_token_path 74 | } 75 | 76 | aivt_vtsp = pyvts.vts(plugin_info=AIVT_VTSP_info) 77 | 78 | 79 | if os.path.exists(authentication_token_path): 80 | try: 81 | asyncio.run(async_AIVT_VTSP_authenticated()) 82 | AIVT_VTSP_Status["authenticated"] = True 83 | print("!!! VTSP Authenticated Success !!!") 84 | 85 | except: 86 | try: 87 | asyncio.run(async_AIVT_VTSP_get_token_and_authenticated()) 88 | AIVT_VTSP_Status["authenticated"] = True 89 | print("!!! VTSP Authenticated Success !!!") 90 | 91 | except: 92 | AIVT_VTSP_Status["authenticated"] = False 93 | print("!!! VTSP Authenticated Fail !!!") 94 | 95 | else: 96 | try: 97 | asyncio.run(async_AIVT_VTSP_get_token_and_authenticated()) 98 | AIVT_VTSP_Status["authenticated"] = True 99 | print("!!! VTSP Authenticated Success !!!") 100 | 101 | except: 102 | AIVT_VTSP_Status["authenticated"] = False 103 | print("!!! VTSP Authenticated Fail !!!") 104 | 105 | 106 | async def async_AIVT_VTSP_get_token_and_authenticated(): 107 | global aivt_vtsp 108 | 109 | await aivt_vtsp.connect() 110 | await aivt_vtsp.request_authenticate_token() 111 | await aivt_vtsp.write_token() 112 | await aivt_vtsp.request_authenticate() 113 | await aivt_vtsp.close() 114 | 115 | async def async_AIVT_VTSP_authenticated(): 116 | global aivt_vtsp 117 | 118 | await aivt_vtsp.connect() 119 | await aivt_vtsp.read_token() 120 | await aivt_vtsp.request_authenticate() 121 | await aivt_vtsp.close() 122 | 123 | 124 | 125 | 126 | 127 | lock_hnt = threading.Lock() 128 | 129 | def VTSP_Hotkey_Names_Trigger(hotkey_names_list=[], command=None): 130 | global aivt_vtsp 131 | 132 | with lock_hnt: 133 | if AIVT_VTSP_Status["authenticated"] and len(hotkey_names_list) > 0: 134 | aprint(f"VTSP trigger hotkey names: {hotkey_names_list}") 135 | asyncio.run(async_VTSP_Hotkey_Names_Trigger(hotkey_names_list, command=command)) 136 | 137 | 138 | async def async_VTSP_Hotkey_Names_Trigger(hotkey_names, command=None): 139 | global aivt_vtsp 140 | 141 | await aivt_vtsp.connect() 142 | await aivt_vtsp.request_authenticate() 143 | 144 | for hotkey_name in hotkey_names: 145 | try: 146 | send_hotkey_request = aivt_vtsp.vts_request.requestTriggerHotKey(hotkey_name) 147 | await aivt_vtsp.request(send_hotkey_request) 148 | 149 | except Exception as e: 150 | aprint(f"!!! Trigger Hotkey *{hotkey_name}* Fail !!!") 151 | 152 | await aivt_vtsp.close() 153 | 154 | 155 | 156 | 157 | 158 | def get_hotkey_names(text_kn, command=None): 159 | global AIVT_hotkeys_parameters 160 | 161 | items = [item.strip() for item in text_kn.split("/") if item.strip()] 162 | list1, list2, list3, list4 = [], [], [], [] 163 | 164 | for item in items: 165 | if item.startswith("!"): 166 | list1.append(item[1:].strip()) 167 | 168 | elif item.startswith("*"): 169 | list2.append(item[1:].strip()) 170 | 171 | elif item.startswith("@"): 172 | list3.append(item[1:].strip()) 173 | 174 | else: 175 | list4.append(item) 176 | 177 | 178 | all = [] 179 | r_ani = [] 180 | f_exp = [] 181 | r_iexp = [] 182 | r_exp = [] 183 | 184 | if list1: 185 | if command == "idle_ani" and len(list1) > 1: 186 | i = 0 187 | 188 | while True: 189 | i += 10 190 | kn = random.choice(list1) 191 | if kn != AIVT_hotkeys_parameters["idle_ani_previous"]: 192 | break 193 | 194 | if i > 10: 195 | break 196 | 197 | else: 198 | kn = random.choice(list1) 199 | 200 | AIVT_hotkeys_parameters["idle_ani_previous"] = kn 201 | 202 | all.append(kn) 203 | r_ani.append(kn) 204 | 205 | if list2: 206 | for name in list2: 207 | if not name in all: 208 | all.append(name) 209 | f_exp.append(name) 210 | 211 | if list3: 212 | list3_r = [] 213 | for name in list3: 214 | if not name in all: 215 | list3_r.append(name) 216 | 217 | if list3_r: 218 | kn = random.choice(list3_r) 219 | all.append(kn) 220 | r_iexp.append(kn) 221 | 222 | if list4: 223 | list4_r = [] 224 | for name in list4: 225 | if not name in all: 226 | list4_r.append(name) 227 | 228 | if list4_r: 229 | kn = random.choice(list4) 230 | all.append(kn) 231 | r_exp.append(kn) 232 | 233 | 234 | result_list = { 235 | "all": all, 236 | "all_exp": f_exp+r_iexp+r_exp, 237 | "r_ani": r_ani, 238 | "f_exp": f_exp, 239 | "r_iexp": r_iexp, 240 | "r_exp": r_exp, 241 | } 242 | 243 | return result_list 244 | 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | 254 | if __name__ == "__main__": 255 | AIVT_VTSP_authenticated() 256 | AIVT_VTSP_Status["authenticated"] = True 257 | 258 | while AIVT_VTSP_Status["authenticated"]: 259 | input_text = input("Enter the names of hotkey (split by /) :\n") 260 | hotkey_names = [name.strip() for name in input_text.split("/") if name.strip()] 261 | VTSP_Hotkey_Names_Trigger(hotkey_names_list=hotkey_names) 262 | 263 | 264 | 265 | 266 | -------------------------------------------------------------------------------- /VTubeStudioPlugin/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AIVTDevPKevin/AI-VTuber-System/a7ab32694680c3bcb0d84eceecf3080e0261a7e0/VTubeStudioPlugin/__init__.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | edge_tts==6.1.10 2 | google-generativeai==0.5.3 3 | keyboard==0.13.5 4 | obs-websocket-py==1.0 5 | openai==1.13.3 6 | openai-whisper==20231117 7 | protobuf==4.25.3 8 | pyaudio==0.2.14 9 | pydub==0.25.1 10 | pyside6==6.6.2 11 | pytchat==0.5.5 12 | pyvts==0.3.2 13 | Requests==2.31.0 14 | tiktoken==0.7.0 15 | transformers>=4.48.0 16 | twitchio==2.9.1 17 | --------------------------------------------------------------------------------