├── generate_image.py ├── .gitignore ├── pyautogui_images ├── chat.png ├── add_photo.png ├── ig_photo.png ├── my_mj_button.png ├── next_button.png ├── next_button2.png ├── share_button.png ├── upload_photo.png ├── caption_space.png └── create_ig_post.png ├── requirements.txt ├── input └── ai_austin_A_realistic_image_of_a_futuristic_AI_robot_in_New_Yor_000893ee-f2f9-4a8e-9109-3210aba26c14.png ├── LICENSE ├── README.md ├── ig_poster.py ├── image_generator.py └── function_call.py /generate_image.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | test.py 2 | main.py 3 | finances.py 4 | ig_img.jpg 5 | config/ 6 | __pycache__/ -------------------------------------------------------------------------------- /pyautogui_images/chat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/chat.png -------------------------------------------------------------------------------- /pyautogui_images/add_photo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/add_photo.png -------------------------------------------------------------------------------- /pyautogui_images/ig_photo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/ig_photo.png -------------------------------------------------------------------------------- /pyautogui_images/my_mj_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/my_mj_button.png -------------------------------------------------------------------------------- /pyautogui_images/next_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/next_button.png -------------------------------------------------------------------------------- /pyautogui_images/next_button2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/next_button2.png -------------------------------------------------------------------------------- /pyautogui_images/share_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/share_button.png -------------------------------------------------------------------------------- /pyautogui_images/upload_photo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/upload_photo.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | openai>=0.27.8 2 | discord==2.2.3 3 | Pillow>=10.2.0 4 | pyautogui==0.9.54 5 | osascript==2020.12.3 6 | -------------------------------------------------------------------------------- /pyautogui_images/caption_space.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/caption_space.png -------------------------------------------------------------------------------- /pyautogui_images/create_ig_post.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/pyautogui_images/create_ig_post.png -------------------------------------------------------------------------------- /input/ai_austin_A_realistic_image_of_a_futuristic_AI_robot_in_New_Yor_000893ee-f2f9-4a8e-9109-3210aba26c14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-Austin/GPT_Auto_IG/HEAD/input/ai_austin_A_realistic_image_of_a_futuristic_AI_robot_in_New_Yor_000893ee-f2f9-4a8e-9109-3210aba26c14.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Austin Dobbins 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # This is a research project to test the new GPT function calling 2 | Watch the YouTube demo: https://www.youtube.com/watch?v=7QdeJ1Q8ppk 3 | 4 | In order to use this code you will need a mid to advanced level understanding of Python. This code is by no means perfect and should serve more as an example of how to use function calling and not the optimal way to automate midjourney or instagram with Python. The function_call.py code will function as is and serve as a good example of how to communicate with the OpenAI API to automate a task that requires two function calls from one prompt input. But in order for the image_generator.py and ig_poster.py files to work will require quite a few setup steps. 5 | This program will run like a chatgpt terminal interface for questions and answers or run the available functions if relevant to your prompt input. 6 | 7 | ## Set API keys as system variable 8 | ``` 9 | OPEN_AI_KEY 10 | DISCORD_BOT_TOKEN 11 | ``` 12 | 13 | ## To get image_generator.py to work: 14 | * Create your own Discord channel and add the Midjourney bot. 15 | * Create a Discord bot and give it permissions to access that channel. 16 | * Replace the images in the 'pyautogui_images' directory with screenshots of the correct places to click for automating sending the /imagine prompt to midjourney in the Discord GUI. 17 | 18 | ## To get ig_poster.py to work: 19 | * Login to your instagram account in Safari on mac 20 | * If not on mac login with your desired browser and change the subprocess call to open that application name. 21 | * Again replace the images in the 'pyautogui_images' directory of screenshots of the correct buttons on your screen. 22 | 23 | -------------------------------------------------------------------------------- /ig_poster.py: -------------------------------------------------------------------------------- 1 | import subprocess 2 | import osascript 3 | import pyautogui 4 | import time 5 | 6 | def auto_post(caption): 7 | # 1. OPEN SAFARI 8 | subprocess.call(['open', '-a', 'Safari']) 9 | # 2. NAVIGATE TO INSTAGRAM.COM 10 | url = 'https://www.instagram.com' 11 | osascript.osascript('tell application "Safari" to open location "{0}"'.format(url)) 12 | # 3. CLICK CREATE POST 13 | time.sleep(4) 14 | create_button = pyautogui.locateOnScreen('pyautogui_images/create_ig_post.png') 15 | pyautogui.click(create_button) 16 | time.sleep(2) 17 | # 4. UPLOAD IMAGE 18 | add_photo = pyautogui.locateOnScreen('pyautogui_images/add_photo.png') 19 | pyautogui.click(add_photo) 20 | time.sleep(2) 21 | ig_photo = pyautogui.locateOnScreen('pyautogui_images/ig_photo.png') 22 | pyautogui.click(ig_photo) 23 | time.sleep(1) 24 | upload_photo = pyautogui.locateOnScreen('pyautogui_images/upload_photo.png') 25 | pyautogui.click(upload_photo) 26 | time.sleep(5) 27 | next_button = pyautogui.locateOnScreen('pyautogui_images/next_button.png') 28 | pyautogui.click(next_button) 29 | pyautogui.move(0, -50) 30 | time.sleep(3) 31 | next_button2 = pyautogui.locateOnScreen('pyautogui_images/next_button2.png') 32 | pyautogui.click(next_button2) 33 | time.sleep(3) 34 | # 5. PASTE CAPTION 35 | caption_space = pyautogui.locateOnScreen('pyautogui_images/caption_space.png') 36 | pyautogui.click(caption_space) 37 | time.sleep(1.5) 38 | pyautogui.write(caption) 39 | time.sleep(2) 40 | # 6. POST 41 | share_button = pyautogui.locateOnScreen('pyautogui_images/share_button.png') 42 | pyautogui.click(share_button) 43 | # 7. Mimimize Safari 44 | time.sleep(3) 45 | pyautogui.hotkey('command', 'm') 46 | 47 | if __name__ =='__main__': 48 | auto_post('AI is lit AF.') 49 | -------------------------------------------------------------------------------- /image_generator.py: -------------------------------------------------------------------------------- 1 | import discord 2 | from discord.ext import commands 3 | import aiohttp # <-- Added 4 | from PIL import Image 5 | import os 6 | import asyncio 7 | import time 8 | import subprocess 9 | import pyautogui 10 | 11 | discord_token = os.environ.get("DISCORD_BOT_TOKEN") 12 | client = commands.Bot(command_prefix="*", intents=discord.Intents.all()) 13 | directory = os.getcwd() 14 | 15 | def split_image(image_file): 16 | with Image.open(image_file) as im: 17 | width, height = im.size 18 | mid_x = width // 2 19 | mid_y = height // 2 20 | top_left = im.crop((0, 0, mid_x, mid_y)) 21 | top_right = im.crop((mid_x, 0, width, mid_y)) 22 | bottom_left = im.crop((0, mid_y, mid_x, height)) 23 | bottom_right = im.crop((mid_x, mid_y, width, height)) 24 | 25 | return top_left, top_right, bottom_left, bottom_right 26 | 27 | async def download_image(url, filename): 28 | async with aiohttp.ClientSession() as session: 29 | async with session.get(url) as resp: 30 | if resp.status == 200: 31 | data = await resp.read() 32 | 33 | input_folder = "input" 34 | output_folder = "output" 35 | 36 | if not os.path.exists(output_folder): 37 | os.makedirs(output_folder) 38 | if not os.path.exists(input_folder): 39 | os.makedirs(input_folder) 40 | 41 | with open(f"{directory}/{input_folder}/{filename}", "wb") as f: 42 | f.write(data) 43 | 44 | print(f"Image downloaded: {filename}") 45 | input_file = os.path.join(input_folder, filename) 46 | 47 | if "UPSCALED_" not in filename: 48 | quadrant = '1' 49 | else: 50 | file_prefix = "UPSCALED_" 51 | 52 | top_left, top_right, bottom_left, bottom_right = split_image(input_file) 53 | 54 | if quadrant == "1": 55 | top_left.save(os.path.join(output_folder, "ig_img.jpg")) 56 | elif quadrant == "2": 57 | top_right.save(os.path.join(output_folder, "ig_img.jpg")) 58 | elif quadrant == "3": 59 | bottom_left.save(os.path.join(output_folder, "ig_img.jpg")) 60 | elif quadrant == "4": 61 | bottom_right.save(os.path.join(output_folder, "ig_img.jpg")) 62 | os.remove(f"{directory}/{input_folder}/{filename}") 63 | pyautogui.hotkey('command', 'm') 64 | await client.close() 65 | 66 | def run_bot(text_input_prompt): 67 | @client.event 68 | async def on_ready(): 69 | print("Discord bot connected.") 70 | asyncio.create_task(send_prompt(text_input_prompt)) 71 | 72 | async def send_prompt(text_input_prompt): 73 | print('Running Discord GUI automation to send /imagine prompt to your Midjourney channel.') 74 | subprocess.call(['open', '-a', 'Discord']) 75 | time.sleep(2) 76 | my_midjourney_server_button = pyautogui.locateOnScreen('pyautogui_images/my_mj_button.png') 77 | my_midjourney_server_button_point = pyautogui.center(my_midjourney_server_button) 78 | pyautogui.click(my_midjourney_server_button_point) 79 | time.sleep(2) 80 | chat_input = pyautogui.locateOnScreen('pyautogui_images/chat.png') 81 | pyautogui.click(chat_input) 82 | time.sleep(.5) 83 | pyautogui.write('/imagine') 84 | time.sleep(1) 85 | pyautogui.press('enter') 86 | time.sleep(.5) 87 | pyautogui.write(text_input_prompt) 88 | pyautogui.press('enter') 89 | 90 | @client.event 91 | async def on_message(message): 92 | for attachment in message.attachments: 93 | await download_image(attachment.url, attachment.filename) 94 | await client.process_commands(message) 95 | client.run(discord_token) 96 | 97 | if __name__ == "__main__": 98 | sample_prompt = 'futuristic AI robot in New York living among human cyborgs in year 2500.' 99 | run_bot(sample_prompt) 100 | -------------------------------------------------------------------------------- /function_call.py: -------------------------------------------------------------------------------- 1 | import openai 2 | import json 3 | import os 4 | import image_generator 5 | import ig_poster 6 | 7 | openai.api_key = os.environ.get('OPEN_AI_KEY') 8 | input_tokens = 0 9 | output_tokens = 0 10 | messages = [{'role': 'system', 'content': 'You are a highly intelligent AI capable of running predefined functions to complete user tasks. You have been given the ability to create AI generated art with the "generate_image" function and even post to instagram if the user asks using the "post_to_ig" function. The "generate_image" function MUST RECEIVE INPUT AS "text_input_prompt". The "post_to_ig" function MUST RECEIVE INPUT AS "instagram_caption". You will commonly get prompts from the user that require multiple functions to be called. Think of the logical order the functions must be ran, and run them in order. You will get a second response from the system once the first function finishes, then decide if there are more functions and run them at that point.'}] 11 | functions = [ 12 | { 13 | 'name': 'generate_image', 14 | 'description': 'Uses Midjourney AI image generation to create an image based off the users text input prompt. Extract the details from the users prompt that describes the image they want to pass as input to this function. No response will be returned from this function, it will open an image after generating the image for the user.', 15 | 'parameters': { 16 | 'type': 'object', 17 | 'properties': { 18 | 'text_input_prompt': { 19 | 'type': 'string', 20 | 'description': 'The description of the photo the user would like generated.', 21 | }, 22 | }, 23 | 'required': ['text_input_prompt'], 24 | } 25 | }, 26 | { 27 | 'name': 'post_to_ig', 28 | 'description': 'Allows you to post to the users instagram. After using the generate_image function in a conversation you can post that image to instagram by passing this function the caption.', 29 | 'parameters':{ 30 | 'type': 'object', 31 | 'properties': { 32 | 'instagram_caption': { 33 | 'type': 'string', 34 | 'description': 'The text caption for the instagram post.' 35 | }, 36 | }, 37 | 'required': ['instagram_caption'] 38 | }, 39 | } 40 | ] 41 | 42 | def generate_image(text_input_prompt): 43 | image_generator.run_bot(text_input_prompt=text_input_prompt) 44 | return 'Image successfully generated.' 45 | 46 | def post_to_ig(instagram_caption): 47 | ig_poster.auto_post(instagram_caption) 48 | return 'Instagram post is live!' 49 | 50 | def handle_function_call(response): 51 | available_functions = { 52 | 'generate_image': generate_image, 53 | 'post_to_ig': post_to_ig, 54 | } 55 | 56 | function_name = response['choices'][0]['message']['function_call']['name'] 57 | function_to_call = available_functions[function_name] 58 | function_args = json.loads(response['choices'][0]['message']['function_call']['arguments']) 59 | 60 | if function_name == 'generate_image': 61 | function_response = function_to_call(function_args['text_input_prompt']) 62 | elif function_name == 'post_to_ig': 63 | function_response = function_to_call(function_args['instagram_caption']) 64 | 65 | print(function_name + ' function call has completed.') 66 | messages.append({'role': 'function', 'name': function_name, 'content': function_response}) 67 | 68 | def run_conversation(): 69 | global messages 70 | global input_tokens 71 | global output_tokens 72 | while True: 73 | # Makes sure we are only sending 4 total converation messages and system prompt as messages in our new API calls. 74 | # This will avoid spending unnecessarily for context prompt tokens and prevent input prompt size from exceeding max tokens. 75 | if len(messages) > 5: 76 | messages = [messages[0]] + messages[-4:] 77 | 78 | user_prompt = input('User: ') 79 | messages.append({'role': 'user', 'content': user_prompt}) 80 | response = openai.ChatCompletion.create( 81 | model='gpt-3.5-turbo-0613', 82 | messages=messages, 83 | functions=functions, 84 | function_call='auto', 85 | ) 86 | input_tokens += response['usage']['prompt_tokens'] 87 | output_tokens += response['usage']['completion_tokens'] 88 | if response['choices'][0]['finish_reason'] == 'function_call': 89 | handle_function_call(response) 90 | # Check if a second function needs to be called, if not GPT will create a message response. 91 | second_response = openai.ChatCompletion.create( 92 | model='gpt-3.5-turbo-0613', 93 | messages=messages, 94 | functions=functions, 95 | function_call='auto' 96 | ) 97 | 98 | input_tokens += second_response['usage']['prompt_tokens'] 99 | output_tokens += second_response['usage']['completion_tokens'] 100 | 101 | if second_response['choices'][0]['finish_reason'] == 'function_call': 102 | handle_function_call(second_response) 103 | else: 104 | print('ChatGPT: ' + response['choices'][0]['message']['content']) 105 | messages.append({'role': 'assistant', 'content': response['choices'][0]['message']['content']}) 106 | 107 | else: 108 | print('ChatGPT: ' + response['choices'][0]['message']['content']) 109 | messages.append({'role': 'assistant', 'content': response['choices'][0]['message']['content']}) 110 | 111 | print('Input tokens: ' + str(input_tokens)) 112 | print('Output tokens: ' + str(output_tokens)) 113 | input_price = input_tokens / 1000 * 0.0015 114 | output_price = output_tokens / 1000 * 0.002 115 | print('Task Price in USD: $' + str("{:.5f}".format(input_price + output_price))) 116 | 117 | run_conversation() 118 | --------------------------------------------------------------------------------