├── README.md
└── main.py


/README.md:
--------------------------------------------------------------------------------
 1 | # LLaMA Assistant for Mac
 2 | 
 3 | A simple assistant for Mac that uses `llama-cpp-python` to assist you on your predefined needs. This uses pretty much 90% of the code from [here](https://github.com/patrickloeber/ai-typing-assistant/blob/main/main.py) but replaces the ollama stuff with [llama-cpp-python](https://github.com/abetlen/llama-cpp-python).
 4 | 
 5 | ## Why?
 6 | 
 7 | Because I wanted a more pythonic solution and wanted to build something w/ llama-cpp-python.
 8 | 
 9 | ## Setup
10 | 
11 | Create a virtual environment and follow the steps below.
12 | 
13 | 1. Install llama-cpp-python
14 | 
15 | `CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python`
16 | 
17 | Note: If you are on CUDA/ CPU, you can remove the `CMAKE_ARGS="-DGGML_METAL=on"` part and  replace with appropriate cmake args [here](https://llama-cpp-python.readthedocs.io/en/latest/#supported-backends).
18 | 
19 | 2. Install pynput and pyperclip
20 | 
21 | `pip install pynput pyperclip`
22 | 
23 | 3. Run main.py
24 | 
25 | Note: These steps are for macos. If you are on windows or linux, you will need to install llama-cpp-python differently.
26 | 
27 | ## Acknowledgements:
28 | 
29 | - [patrickloeber/ai-typing-assistant](https://github.com/patrickloeber/ai-typing-assistant)
30 | - [abetlen/llama-cpp-python](https://github.com/abetlen/llama-cpp-python)


--------------------------------------------------------------------------------
/main.py:
--------------------------------------------------------------------------------
  1 | # copied from https://github.com/patrickloeber/ai-typing-assistant/blob/main/main.py
  2 | # main changes are to strip out the ollama stuff and replace it with llama-cpp-python to make it truly pythonic
  3 | 
  4 | import time
  5 | from string import Template
  6 | 
  7 | from llama_cpp import Llama
  8 | from pynput import keyboard
  9 | from pynput.keyboard import Key, Controller
 10 | import pyperclip
 11 | 
 12 | llm = Llama.from_pretrained(
 13 |     repo_id="Qwen/Qwen2-0.5B-Instruct-GGUF",
 14 |     filename="*q8_0.gguf",
 15 |     verbose=False
 16 | )
 17 | 
 18 | controller = Controller()
 19 | 
 20 | FIX_PROMPT_TEMPLATE = Template(
 21 |     """Fix all typos and casing and punctuation in this text, but preserve all new line characters:
 22 | 
 23 | $text
 24 | 
 25 | Return only the corrected text, don't include a preamble.
 26 | """
 27 | )
 28 | 
 29 | TRANSLATE_PROMPT_TEMPLATE = Template(
 30 |     """Translate this text to French, but preserve all new line characters:
 31 | 
 32 | $text
 33 | 
 34 | Return only the translated text, don't include a preamble.
 35 | """
 36 | )
 37 | 
 38 | def fix_text(text):
 39 |     prompt = FIX_PROMPT_TEMPLATE.substitute(text=text)
 40 |     output = llm.create_chat_completion(
 41 |         messages = [
 42 |             {
 43 |                 "role": "user",
 44 |                 "content": prompt
 45 |             }
 46 |         ]
 47 |     )
 48 |     print(output)
 49 |     return output["choices"][0]["message"]["content"].strip()
 50 | 
 51 | def translate_text(text):
 52 |     prompt = TRANSLATE_PROMPT_TEMPLATE.substitute(text=text)
 53 |     output = llm.create_chat_completion(
 54 |         messages = [
 55 |             {
 56 |                 "role": "user",
 57 |                 "content": prompt
 58 |             }
 59 |         ]
 60 |     )
 61 |     print(output)
 62 |     return output["choices"][0]["message"]["content"].strip()
 63 | 
 64 | 
 65 | def fix_current_line(usecase="fix"):
 66 |     # macOS short cut to select current line: Cmd+Shift+Left
 67 |     controller.press(Key.cmd)
 68 |     controller.press(Key.shift)
 69 |     controller.press(Key.left)
 70 | 
 71 |     controller.release(Key.cmd)
 72 |     controller.release(Key.shift)
 73 |     controller.release(Key.left)
 74 | 
 75 |     if usecase == "fix":
 76 |         fix_selection(usecase="fix")
 77 |     elif usecase == "translate":
 78 |         fix_selection(usecase="translate")
 79 | 
 80 | 
 81 | def fix_selection(usecase="fix"):
 82 |     # 1. Copy selection to clipboard
 83 |     with controller.pressed(Key.cmd):
 84 |         controller.tap("c")
 85 | 
 86 |     # 2. Get the clipboard string
 87 |     time.sleep(0.1)
 88 |     text = pyperclip.paste()
 89 | 
 90 |     # 3. Fix string
 91 |     if not text:
 92 |         return
 93 |     
 94 |     if usecase == "fix":
 95 |         fixed_text = fix_text(text)
 96 |     elif usecase == "translate":
 97 |         fixed_text = translate_text(text)
 98 |     if not fixed_text:
 99 |         return
100 | 
101 |     # 4. Paste the fixed string to the clipboard
102 |     pyperclip.copy(fixed_text)
103 |     time.sleep(0.1)
104 | 
105 |     # 5. Paste the clipboard and replace the selected text
106 |     with controller.pressed(Key.cmd):
107 |         controller.tap("v")
108 | 
109 | def on_f9():
110 |     fix_selection(usecase="fix")
111 | 
112 | def on_f10():
113 |     fix_selection(usecase="translate")
114 | 
115 | 
116 | with keyboard.GlobalHotKeys({"<101>": on_f9, "<109>": on_f10}) as h:
117 |     h.join()


--------------------------------------------------------------------------------