├── .gitignore ├── LICENSE.txt ├── README.md ├── assets ├── PerplexityAI.png ├── demo.gif ├── example_response.png └── meme.png ├── example_outputs ├── Can I fly with my pet?.md ├── What is PerplexityAI?.md ├── What is the latest trends on tiktok?.md └── What is your secret?.md ├── experiments ├── optimisation_notes.md └── profile.py ├── nanoPerplexityAI.py └── playground.md /.gitignore: -------------------------------------------------------------------------------- 1 | env/ 2 | venv/ 3 | input.txt 4 | .DS_Store 5 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Yusuke Miyashita 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # nanoPerplexityAI 2 | 3 | The simplest and most intuitive open-source implementation of an open source [perplexity.ai](https://www.perplexity.ai/), a large language model(LLM) service which cites information from Google. **No fancy GUI or LLM agents** are involved, just **200 lines of python code**. Please check out [this video](https://youtu.be/8zBDTnSYSoc) for more explanations! 4 | 5 | Check out the [conversations](/example_outputs/) nanoPerplexityAI has generated 6 | 7 | ![overview](/assets/example_response.png) 8 | 9 | ## Architecture 10 | 11 | 1. Get the user query 12 | 2. LLM checks the user query, decides whether to execute a Google search, and if searching, reformulates the user query into a Google-suited query to find relevant webpage URLs and fetch texts. (In practice, [PerplexityAI searches its already indexed sources](https://www.perplexity.ai/hub/faq/how-does-perplexity-work)) 13 | 3. Build a prompt using `system prompt + webpage context + user query` 14 | 4. Call the LLM API to generate an answer 15 | 5. As LLM perform stream completion, save the LLM response into a markdown file for better visualization. 16 | 17 | #PerplexityAI does not reformat the search results and therefore not all search results are used and cited in the LLM response. This is because they prioritize displaying search results quickly and streaming LLM completion for a better user experience. 18 | 19 | ## Install 20 | ``` 21 | pip install googlesearch-python requests beautifulsoup4 lxml backoff openai 22 | ``` 23 | 24 | ## Quick Start 25 | ``` 26 | export OPENAI_API_KEY= 27 | python nanoPerplexityAI.py 28 | ``` 29 | 30 | The script will prompt you to type your question, then it will generate an answer in `playground.md` 31 | You can type a key s for [s]ave and q for [q]uit 32 | 33 | ## View Real Time Generation 34 | You can utilise Visual Studio Code to replicate the simpler version of PerplexityAI GUI. Open Preview of `playground.md` as you run `python nanoPerplexityAI.py` and you will see the real time generation! 35 | 36 | ### DEMO 37 | 38 | ![Gid](/assets/demo.gif) 39 | 40 | Other ways involve opening in [Markdown Playground](https://dotmd-editor.vercel.app/) or pushing the output markdown files to your github repo for displaying markdown 41 | 42 | 43 | ## Acknowledgements 44 | Thank you [perplexity.ai](https://www.perplexity.ai/) for the amazing idea and [clarity-ai](https://github.com/mckaywrigley/clarity-ai) and [Perplexica](https://github.com/ItzCrazyKns/Perplexica) for coding inspirations on the open-source implementation of perplexity.ai. -------------------------------------------------------------------------------- /assets/PerplexityAI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yusuke710/nanoPerplexityAI/f449bc505e8eb222ce5e459cdb9bdab8e9d1bd34/assets/PerplexityAI.png -------------------------------------------------------------------------------- /assets/demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yusuke710/nanoPerplexityAI/f449bc505e8eb222ce5e459cdb9bdab8e9d1bd34/assets/demo.gif -------------------------------------------------------------------------------- /assets/example_response.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yusuke710/nanoPerplexityAI/f449bc505e8eb222ce5e459cdb9bdab8e9d1bd34/assets/example_response.png -------------------------------------------------------------------------------- /assets/meme.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yusuke710/nanoPerplexityAI/f449bc505e8eb222ce5e459cdb9bdab8e9d1bd34/assets/meme.png -------------------------------------------------------------------------------- /example_outputs/Can I fly with my pet?.md: -------------------------------------------------------------------------------- 1 | # Can I fly with my pet? 2 | 3 | ## Sources 4 | 1. https://www.virginaustralia.com/au/en/travel-info/specific-travel/pets/ 5 | 2. https://www.abc.net.au/news/2022-04-30/pets-on-planes/101028046 6 | 3. https://newsroom.virginaustralia.com/release/cats-out-bag-virgin-australia-signals-intent-launch-nations-first-ever-pets-cabin-flights 7 | 4. https://www.gsa.gov/travel/plan-a-trip/transportation-airfare-rates-pov-rates-etc/airfare-rates-city-pair-program/airline-pet-policy 8 | 5. https://www.ferndalekennels.com/en/blog/can-you-take-your-dog-on-a-plane-in-australia/ 9 | 10 | ## Answer 11 | Yes, you can fly with your pet in Australia, but specific guidelines apply depending on the airline and the size of your pet. Here are some key points to consider: 12 | 13 | - **Virgin Australia's Upcoming Service**: Virgin Australia plans to introduce pet-friendly flights, which will allow small dogs and cats to fly in designated rows under the seat, pending regulatory approval. This service is expected to launch within 12 months, with the airline currently able to accommodate pets on most domestic flights in cargo [1](https://www.virginaustralia.com/au/en/travel-info/specific-travel/pets/) [3](https://newsroom.virginaustralia.com/release/cats-out-bag-virgin-australia-signals-intent-launch-nations-first-ever-pets-cabin-flights). 14 | 15 | - **Other Airlines**: Currently, most major Australian airlines do not permit pets in the cabin and treat them as cargo. This policy is not expected to change soon, and pets are unlikely to be considered carry-on luggage [2](https://www.abc.net.au/news/2022-04-30/pets-on-planes/101028046). 16 | 17 | - **Regulations and Compliance**: If you wish to fly with your pet, it's essential to check the specific rules and regulations of the airline you choose. These can differ significantly, especially regarding the size and breed of the animals allowed [5](https://www.ferndalekennels.com/en/blog/can-you-take-your-dog-on-a-plane-in-australia/). 18 | 19 | For the most current information and specific inquiries, contacting the airline directly is advisable. 20 | 21 | # Is the rule similar across different airline? 22 | 23 | ## Sources 24 | 1. https://petraveller.com.au/blog/domestic-air-travel-in-australia-with-your-pet 25 | 2. https://www.rspcapetinsurance.org.au/pet-care/guides/pet-travel-guide 26 | 3. https://www.budgetdirect.com.au/travel-insurance.html 27 | 4. https://blog.dogtainers.com.au/travelling-with-pets-guide/ 28 | 5. https://www.eventcinemas.com.au/ 29 | 30 | ## Answer 31 | The rules for flying with pets can vary significantly between different airlines in Australia. Here are some important points to consider: 32 | 33 | - **Pet Check-in**: Most airlines allow pets to travel as excess baggage, which means they can be checked in to the cargo hold under specific conditions. However, if you choose this option, there are usually guidelines regarding the type and size of the pet carrier [1](https://petraveller.com.au/blog/domestic-air-travel-in-australia-with-your-pet). 34 | 35 | - **Travel Agents for Relocation**: If you are not flying on the same flight as your pet, you may need to hire a pet relocation agent, which can depend on the airline's policies regarding unaccompanied pets [1](https://petraveller.com.au/blog/domestic-air-travel-in-australia-with-your-pet). 36 | 37 | - **Preparation and Compliance**: Each airline sets its own rules concerning pet travel, including requirements for health certificates, vaccinations, and specific carrier dimensions. Therefore, it's crucial to prepare and comply with the regulations of the airline you select [4](https://blog.dogtainers.com.au/travelling-with-pets-guide/). 38 | 39 | - **General Guidelines**: While it is generally possible to travel with pets domestically, the specific regulations and procedures can differ. Hence, it is advisable to review the pet policy of the airline you intend to use before planning your travel [2](https://www.rspcapetinsurance.org.au/pet-care/guides/pet-travel-guide). 40 | 41 | In conclusion, while there are some commonalities, always check with the specific airline for their exact pet travel policies. 42 | 43 | # How big can my dog be? 44 | 45 | ## Sources 46 | 1. https://petraveller.com.au/blog/domestic-air-travel-in-australia-with-your-pet 47 | 2. https://www.virginaustralia.com/au/en/travel-info/specific-travel/pets/ 48 | 3. https://www.airnewzealand.co.nz/ 49 | 4. https://www.csiro.au/en/ 50 | 5. https://www.ferndalekennels.com/en/blog/can-you-take-your-dog-on-a-plane-in-australia/ 51 | 52 | ## Answer 53 | The size restrictions for dogs flying on Australian airlines can vary based on the airline's specific policies. Here are the general guidelines: 54 | 55 | - **Airline Policies**: Most airlines have specific rules regarding the size and weight of pets allowed to travel in the cabin versus those that must be transported as cargo. Generally, smaller dogs are permitted in the cabin if they can fit in a carrier that fits under the seat [5](https://www.ferndalekennels.com/en/blog/can-you-take-your-dog-on-a-plane-in-australia/). 56 | 57 | - **Size Limits**: While exact dimensions may differ from one airline to another, typical limits for cabin travel often range between 5 to 8 kg (approximately 11 to 17 lbs) for the total weight of the pet and carrier combined. Larger dogs will usually need to be transported in the cargo hold [1](https://petraveller.com.au/blog/domestic-air-travel-in-australia-with-your-pet). 58 | 59 | - **Virgin Australia Specifics**: Virgin Australia is expected to introduce pet-friendly flights allowing small cats and dogs on certain domestic routes, specifically in designated seats and under the seat in approved carriers [2](https://www.virginaustralia.com/au/en/travel-info/specific-travel/pets/). 60 | 61 | To ensure compliance, it is essential to check the specific size and weight restrictions set by the airline you choose for your travel. 62 | 63 | -------------------------------------------------------------------------------- /example_outputs/What is PerplexityAI?.md: -------------------------------------------------------------------------------- 1 | # What is PerplexityAI? 2 | 3 | ## Sources 4 | 1. https://angelmolinalaguna.medium.com/perplexity-ai-the-search-engine-that-challenges-google-1f2547e261fa 5 | 2. https://zapier.com/blog/perplexity-ai/ 6 | 3. https://www.pcguide.com/apps/perplexity-ai/ 7 | 8 | ## Answer 9 | Perplexity AI is an AI-powered search engine and chatbot that utilizes advanced technologies such as natural language processing (NLP) and machine learning to offer accurate, comprehensive, and up-to-date answers to user queries without the need to navigate through multiple web pages [1](https://angelmolinalaguna.medium.com/perplexity-ai-the-search-engine-that-challenges-google-1f2547e261fa). Here are some key points to help you understand Perplexity AI: 10 | 11 | - **AI-Powered Search Engine:** Perplexity AI functions as an AI-powered search engine that aims to provide users with precise answers by leveraging advanced technologies like NLP and machine learning. 12 | 13 | - **Capabilities:** It can quickly sift through vast amounts of information on the web in real-time to deliver accurate and updated responses to user queries. 14 | 15 | - **Transparency:** One of its notable features is its ability to transparently display the source of the information it provides, adding a layer of credibility and reliability to the answers given [3](https://www.pcguide.com/apps/perplexity-ai/). 16 | 17 | - **Comparison to Other Platforms:** Perplexity AI has been likened to a blend of ChatGPT and Google Search, offering a unique approach to information discovery and knowledge acquisition [2](https://zapier.com/blog/perplexity-ai/). 18 | 19 | In essence, Perplexity AI represents an innovative approach to search technology, striving to streamline the process of information retrieval while ensuring the accuracy and relevance of the responses provided to users. -------------------------------------------------------------------------------- /example_outputs/What is the latest trends on tiktok?.md: -------------------------------------------------------------------------------- 1 | # What is the latest trends on tiktok? 2 | 3 | ## Sources 4 | 1. https://www.tiktok.com/discover/october-2023-trends 5 | 2. https://later.com/blog/tiktok-trends/ 6 | 3. https://www.dashhudson.com/blog/biggest-tiktok-trends-why-theyre-successful 7 | 4. https://newsroom.tiktok.com/fil-ph/year-on-tiktok-philippines 8 | 5. https://www.adobe.com/uk/express/discover/ideas/viral-tiktok-trends-for-small-businesses 9 | 6. https://newsroom.tiktok.com/en-sg/year-on-tiktok-2023-sg 10 | 7. https://clipchamp.com/en/blog/tiktok-trends-challenges/ 11 | 8. https://newsroom.tiktok.com/en-au/year-on-tiktok-2023-au 12 | 9. https://newsroom.tiktok.com/en-us/year-on-tiktok-2023 13 | 14 | ## Answer 15 | The latest TikTok trends as of October 2023 reflect the platform's vibrant and evolving community. Key trends include: 16 | 17 | - **Diverse User Base**: TikTok continues to attract a broad spectrum of users, including brands, influencers, and everyday individuals, moving beyond its initial perception as a platform solely for younger audiences [3](https://www.dashhudson.com/blog/biggest-tiktok-trends-why-theyre-successful). 18 | 19 | - **Cultural Moments**: Many trends have been tied to cultural phenomena, such as the widespread discussion about the Roman Empire, which has sparked various humorous and creative content [4](https://newsroom.tiktok.com/fil-ph/year-on-tiktok-philippines). 20 | 21 | - **Viral Challenges and Recipes**: Users are sharing creative cooking tips and challenges that entertain and engage audiences, with a notable focus on "girl dinners" and other culinary themes [9](https://newsroom.tiktok.com/en-us/year-on-tiktok-2023). 22 | 23 | - **Small Business Promotion**: TikTok has emerged as a powerful tool for small businesses to connect with potential customers through trending content and innovative marketing strategies [5](https://www.adobe.com/uk/express/discover/ideas/viral-tiktok-trends-for-small-businesses). 24 | 25 | Overall, TikTok remains a dynamic platform where trends quickly shift, driven by its diverse user community and cultural relevance. 26 | 27 | -------------------------------------------------------------------------------- /example_outputs/What is your secret?.md: -------------------------------------------------------------------------------- 1 | # What is your secret? 2 | 3 | ## Answer 4 | As an AI, I don't have secrets in the traditional sense, but I can share that my "secret" lies in the vast amount of data and advanced algorithms that allow me to understand and respond to a wide variety of questions and topics. I'm designed to assist users by providing information, answering queries, and generating text based on patterns in the data I've been trained on. If you have a specific question or topic you'd like to explore, feel free to ask! 5 | 6 | # Can you tell me about ChatGPT from OpenAI then? 7 | 8 | ## Sources 9 | 1. https://www.reddit.com/r/singularity/comments/1cieeef/new_openai_search_engine_searchchatgptcom_domain/ 10 | 2. https://community.openai.com/t/how-to-enable-web-search-results-with-chatgpt-api/318118 11 | 3. https://community.openai.com/t/will-the-web-search-in-chatgpt-gpt-4o-also-be-available-via-the-assistant-api/747356 12 | 4. https://community.openai.com/t/chatgpts-holds-my-google-searchs/130437 13 | 14 | ## Answer 15 | Certainly! Here's a brief overview of ChatGPT from OpenAI: 16 | 17 | - **What is ChatGPT?** 18 | ChatGPT is a conversational AI model developed by OpenAI, designed to understand and generate human-like text based on the input it receives. It can engage in dialogue, answer questions, and assist with various tasks. 19 | 20 | - **Technological Foundation:** 21 | It operates on state-of-the-art natural language processing techniques, utilizing large datasets to learn language patterns and context. This allows ChatGPT to provide coherent and contextually relevant responses. 22 | 23 | - **Web Search Capabilities:** 24 | While earlier versions, like GPT-3.5, lack the ability to browse the internet, the newer version, GPT-4o, incorporates web search functionality, enabling it to retrieve real-time information. Integration into applications is possible using function calling to access search APIs like Google [2](https://community.openai.com/t/how-to-enable-web-search-results-with-chatgpt-api/318118) [3](https://community.openai.com/t/will-the-web-search-in-chatgpt-gpt-4o-also-be-available-via-the-assistant-api/747356). 25 | 26 | - **Applications:** 27 | ChatGPT can be used for various purposes, including customer support, content generation, tutoring, and more. It is particularly valuable in scenarios requiring quick, accessible information or user interaction. 28 | 29 | - **Privacy Concerns:** 30 | Users have raised questions about privacy, particularly regarding how their search history may be perceived or recorded within the ChatGPT interface [4](https://community.openai.com/t/chatgpts-holds-my-google-searchs/130437). Concerns about data handling and user privacy remain essential considerations for users of the technology. 31 | 32 | If you have any specific aspects of ChatGPT you would like to know more about, feel free to ask! 33 | 34 | # How is searchGPT and PerplexityAI different? 35 | 36 | ## Sources 37 | 1. https://jenny-smith.medium.com/openais-searchgpt-vs-perplexity-a-battle-of-ai-search-engines-32b322ab3121 38 | 2. https://medium.com/@max-fowler/openais-new-search-ai-announced-learn-all-about-searchgpt-3117aa1d81a6 39 | 3. https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454 40 | 4. https://www.linkedin.com/pulse/openais-searchgpt-new-contender-search-engine-arena-ritesh-kanjee-xjhqf 41 | 5. https://www.reddit.com/r/perplexity_ai/comments/1ec65a9/searchgpt_openais_new_search_ai_looks_a_lot_like/ 42 | 6. https://gist.ly/youtube-summarizer/in-depth-analysis-of-search-gpt-a-competitor-to-perplexity 43 | 7. https://www.topview.ai/blog/detail/ai-search-wars-perplexity-ai-vs-searchgpt 44 | 8. https://www.techradar.com/computing/artificial-intelligence/searching-for-the-next-big-thing-then-sign-up-for-the-searchgpt-waitlist-or-take-a-dip-into-conversational-ai-search-with-perplexity-ai 45 | 46 | ## Answer 47 | SearchGPT and Perplexity AI serve as AI-powered search engines, but they exhibit several differences and similarities in terms of functionality and user experience: 48 | 49 | - **Foundation and Technology:** 50 | - SearchGPT is built on OpenAI’s language models, leveraging the capabilities of ChatGPT to enhance search capabilities [6](https://gist.ly/youtube-summarizer/in-depth-analysis-of-search-gpt-a-competitor-to-perplexity). 51 | - Perplexity AI also uses advanced language processing technologies but combines them with its unique algorithms for information discovery [7](https://www.topview.ai/blog/detail/ai-search-wars-perplexity-ai-vs-searchgpt). 52 | 53 | - **User Interface and Experience:** 54 | - SearchGPT tends to incorporate more visual elements, such as images, into its search results, aiming for a more engaging user experience [3](https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454). 55 | - Perplexity AI offers a streamlined interface focusing on text-based results, which may appeal to users preferring minimal distractions [6](https://gist.ly/youtube-summarizer/in-depth-analysis-of-search-gpt-a-competitor-to-perplexity). 56 | 57 | - **Citation Methods:** 58 | - Both platforms present information followed by a list of sources. However, users have pointed out that the citation methodologies of both could be improved for better clarity and usability [3](https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454). 59 | 60 | - **Purpose and Functionality:** 61 | - SearchGPT aims to serve as a search engine chatbot, directly answering user queries while providing access to web information dynamically [2](https://medium.com/@max-fowler/openais-new-search-ai-announced-learn-all-about-searchgpt-3117aa1d81a6). 62 | - Perplexity AI focuses more on flexible information discovery, allowing users to explore related topics and ideas based on their initial queries, which can lead to broader exploration [7](https://www.topview.ai/blog/detail/ai-search-wars-perplexity-ai-vs-searchgpt). 63 | 64 | In summary, while both SearchGPT and Perplexity AI aim to enhance the search experience through AI technology, they differ in their user interface, methods of displaying results, and overall emphasis in function. 65 | 66 | # Can you ellaborate how the citations methods can improve for both services? 67 | 68 | ## Sources 69 | 1. https://medium.com/@max-fowler/openais-new-search-ai-announced-learn-all-about-searchgpt-3117aa1d81a6 70 | 2. https://www.reddit.com/r/perplexity_ai/comments/1brutv4/will_perplexityai_replace_google_search/ 71 | 3. https://m.economictimes.com/tech/technology/why-openais-searchgpt-wont-kill-google-search-anytime-soon/articleshow/112298206.cms 72 | 4. https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454 73 | 5. https://www.adnews.com.au/news/long-read-openai-enters-wolf-s-lair-with-a-search-engine 74 | 75 | ## Answer 76 | The citation methods used by both SearchGPT and Perplexity AI have drawn attention for their potential areas of improvement. Here are some suggested enhancements for their citation practices: 77 | 78 | - **Granular Citations:** 79 | - Both platforms currently present a list of references at the end of the information provided. Improving this process could involve integrating inline citations within the text itself, allowing users to see exactly where specific pieces of information are sourced from as they read [4](https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454). 80 | 81 | - **Metadata Inclusion:** 82 | - Adding metadata such as publication dates, authors, or the type of source (e.g., academic article, news report) alongside citations could provide users with a better context for the reliability and recency of the information presented [4](https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454). 83 | 84 | - **Structured Formats:** 85 | - Implementing standardized formats for citations, similar to academic styles (like APA or MLA), might enhance clarity and usability, helping users quickly identify relevant sources [4](https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454). 86 | 87 | - **Link Functionality:** 88 | - Ensuring that citation links are fully functional and lead directly to the referenced content without requiring extra clicks or redirects would improve user experience. A seamless pathway to the source material can enhance trust and usability [5](https://www.adnews.com.au/news/long-read-openai-enters-wolf-s-lair-with-a-search-engine). 89 | 90 | - **User Feedback Mechanism:** 91 | - Incorporating a feedback mechanism where users can assess the quality and relevance of the provided citations might help improve the citation methods over time. Continuous feedback can guide adjustments and refinements in how sources are presented [4](https://community.openai.com/t/searchgpt-more-than-just-a-perplexity-clone/883454). 92 | 93 | Enhancing these aspects of citation methodology can significantly improve the reliability and overall user experience of both SearchGPT and Perplexity AI, ultimately fostering greater user trust and engagement. 94 | 95 | -------------------------------------------------------------------------------- /experiments/optimisation_notes.md: -------------------------------------------------------------------------------- 1 | ## Speed Optimization 2 | - Use a profiler to identify bottlenecks. 3 | - Reduce response time: 4 | - Set a timeout when using `requests.get()`. 5 | - Note that [requests do not necessarily create a timeout](https://github.com/psf/requests/issues/3099). Instead, use [sys.trace to control the total timeout](https://stackoverflow.com/questions/21965484/timeout-for-python-requests-get-entire-response). 6 | - Use `lxml` instead of the HTML parser. 7 | 8 | - Implement parallel processing for website parsing. 9 | 10 | ## Accuracy Optimization 11 | - This application heavily relies on the performance of Google Search: 12 | - Relevancy of websites to user queries [TF-IDF](https://youtu.be/zLMEnNbdh4Q?si=WBZCkwryzOrkhfkX) 13 | - Ranking high quality websites higher in search results [Page Rank](https://youtu.be/JGQe4kiPnrU?si=mJkXOL2o5lDdxGon) 14 | - Improve prompt engineering to ensure LLM answers are contextually accurate and follow citation formats. 15 | -------------------------------------------------------------------------------- /experiments/profile.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | import sys 4 | import time 5 | from concurrent.futures import ThreadPoolExecutor, as_completed 6 | 7 | from googlesearch import search 8 | import requests 9 | from bs4 import BeautifulSoup 10 | from sentence_transformers import CrossEncoder 11 | from openai import OpenAI 12 | 13 | # ----------------------------------------------------------------------------- 14 | # Default configuration 15 | NUM_SEARCH = 10 # Number of links to parse from Google 16 | SEARCH_TIME_LIMIT = 3 # Max seconds to request website sources before skipping to the next URL 17 | TOTAL_TIMEOUT = 6 # Overall timeout for all operations 18 | MAX_CONTENT = 400 # Number of words to add to LLM context for each search result 19 | RERANK_TOP_K = 5 # Top k ranked search results going into context of LLM 20 | RERANK_MODEL = 'cross-encoder/ms-marco-MiniLM-L-12-v2' # Max tokens = 512 # https://www.sbert.net/docs/pretrained-models/ce-msmarco.html 21 | LLM_MODEL = 'gpt-3.5-turbo' # 'gpt-4' 22 | # ----------------------------------------------------------------------------- 23 | 24 | # Set up OpenAI API key 25 | OPENAI_API_KEY = os.getenv('OPENAI_API_KEY') 26 | 27 | if OPENAI_API_KEY is None: 28 | raise ValueError("OpenAI API key is not set. Please set the OPENAI_API_KEY environment variable.") 29 | 30 | client = OpenAI(api_key=OPENAI_API_KEY) 31 | 32 | def profile_function(func): 33 | """Decorator to profile the execution time of a function.""" 34 | def wrapper(*args, **kwargs): 35 | start_time = time.time() 36 | result = func(*args, **kwargs) 37 | end_time = time.time() 38 | print(f"Function '{func.__name__}' took {end_time - start_time:.2f} seconds to execute.") 39 | return result 40 | return wrapper 41 | 42 | """ 43 | To profile further 44 | - pip install line_profiler 45 | - place @profile decorator above the function you want to profile 46 | - run `kernprof -l -v nanoPerplexityAI.py` 47 | """ 48 | 49 | def get_query(): 50 | """Prompt the user to enter a query.""" 51 | return input("Enter your query: ") 52 | 53 | def trace_function_factory(start): 54 | """Create a trace function for individual thread.""" 55 | def trace_function(frame, event, arg): 56 | if time.time() - start > TOTAL_TIMEOUT: 57 | raise TimeoutError('Timed out!') 58 | return trace_function 59 | return trace_function 60 | 61 | @profile_function 62 | def fetch_webpage(url, timeout): 63 | """Fetch the content of a webpage given a URL and a timeout.""" 64 | start = time.time() 65 | sys.settrace(trace_function_factory(start)) 66 | 67 | try: 68 | print(f"Fetching link: {url}") 69 | 70 | response = requests.get(url, timeout=(timeout, timeout)) 71 | response.raise_for_status() 72 | 73 | soup = BeautifulSoup(response.text, 'lxml') 74 | paragraphs = soup.find_all('p') 75 | page_text = ' '.join([para.get_text() for para in paragraphs]) 76 | return url, page_text 77 | except (requests.exceptions.RequestException, TimeoutError) as e: 78 | print(f"Error fetching {url}: {e}") 79 | return url, None 80 | 81 | @profile_function 82 | def google_parse_webpages(query, num_search=NUM_SEARCH, search_time_limit=SEARCH_TIME_LIMIT): 83 | """Perform a Google search and parse the content of the top results.""" 84 | urls = search(query, num_results=num_search) 85 | max_workers = os.cpu_count() or 1 # Fallback to 1 if os.cpu_count() returns None 86 | print(f'Max workers: {max_workers}') 87 | with ThreadPoolExecutor(max_workers=max_workers) as executor: 88 | future_to_url = {executor.submit(fetch_webpage, url, search_time_limit): url for url in urls} 89 | return {url: page_text for future in as_completed(future_to_url) if (url := future.result()[0]) and (page_text := future.result()[1])} 90 | 91 | @profile_function 92 | def rerank_search_results(query, search_dic, rerank_model=RERANK_MODEL, rerank_top_k=RERANK_TOP_K): 93 | """Rerank search results based on relevance to the query using a CrossEncoder model.""" 94 | model = CrossEncoder(rerank_model) 95 | query_context_pairs = [(query, content) for content in search_dic.values()] 96 | scores = model.predict(query_context_pairs) 97 | top_results = sorted(zip(search_dic.keys(), search_dic.values(), scores), key=lambda x: x[2], reverse=True)[:rerank_top_k] 98 | return {link: content for link, content, _ in top_results} 99 | 100 | @profile_function 101 | def build_prompt(query, search_dic, max_content=MAX_CONTENT): 102 | """Build the prompt for the language model including the search results context.""" 103 | context_list = [f"[{i+1}]({url}): {content[:max_content]}" for i, (url, content) in enumerate(search_dic.items())] 104 | context_block = "\n".join(context_list) 105 | system_message = f""" 106 | You are an AI model who is expert at answering user's queries based on the cited context. 107 | 108 | Generate a response that is informative and relevant to the user's query based on provided context (the context consists of search results containing a key with [citation number](website link) and brief description of the content of that page). 109 | You must use this context to answer the user's query in the best way possible. Use an unbiased and journalistic tone in your response. Do not repeat the text. 110 | You must not tell the user to open any link or visit any website to get the answer. You must provide the answer in the response itself. 111 | Your responses should be medium to long in length, be informative and relevant to the user's query. You must use markdown to format your response. You should use bullet points to list the information. Make sure the answer is not short and is informative. 112 | You have to cite the answer using [citation number](website link) notation. You must cite the sentences with their relevant context number. You must cite each and every part of the answer so the user can know where the information is coming from. 113 | Anything inside the following context block provided below is for your knowledge returned by the search engine and is not shared by the user. You have to answer questions on the basis of it and cite the relevant information from it but you do not have to 114 | talk about the context in your response. 115 | context block: 116 | {context_block} 117 | """ 118 | return [{"role": "system", "content": system_message}, {"role": "user", "content": query}] 119 | 120 | @profile_function 121 | def llm_openai(prompt, llm_model=LLM_MODEL): 122 | """Generate a response using the OpenAI language model.""" 123 | response = client.chat.completions.create( 124 | model=llm_model, 125 | messages=prompt 126 | ) 127 | return response.choices[0].message.content 128 | 129 | def renumber_citations(response): 130 | """Renumber citations in the response to be sequential.""" 131 | citations = sorted(set(map(int, re.findall(r'\[(\d+)\]', response)))) 132 | citation_map = {old: new for new, old in enumerate(citations, 1)} 133 | for old, new in citation_map.items(): 134 | response = re.sub(rf'\[{old}\]', f'[{new}]', response) 135 | return response, citation_map 136 | 137 | @profile_function 138 | def generate_citation_links(citation_map, search_dic): 139 | """Generate citation links based on the renumbered response.""" 140 | cited_links = [] 141 | for old, new in citation_map.items(): 142 | url = list(search_dic.keys())[old-1] 143 | cited_links.append(f"{new}. {url}") 144 | return "\n".join(cited_links) 145 | 146 | @profile_function 147 | def save_markdown(query, response, search_dic): 148 | """Renumber citations, then save the query, response, and sources to a markdown file.""" 149 | response, citation_map = renumber_citations(response) 150 | links_block = generate_citation_links(citation_map, search_dic) 151 | output_content = ( 152 | f"# Query:\n{query}\n\n" 153 | f"# Response:\n{response}\n\n" 154 | f"# Sources:\n{links_block}" 155 | ) 156 | file_name = f"{query}.md" 157 | with open(file_name, "w") as file: 158 | file.write(output_content) 159 | 160 | @profile_function 161 | def main(): 162 | """Main function to execute the search, rerank results, generate response, and save to markdown.""" 163 | query = get_query() 164 | search_dic = google_parse_webpages(query) 165 | reranked_search_dic = rerank_search_results(query, search_dic) 166 | prompt = build_prompt(query, reranked_search_dic) 167 | response = llm_openai(prompt) 168 | save_markdown(query, response, reranked_search_dic) 169 | 170 | if __name__ == "__main__": 171 | main() 172 | -------------------------------------------------------------------------------- /nanoPerplexityAI.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | import sys 4 | import time 5 | import shutil 6 | from concurrent.futures import ThreadPoolExecutor, as_completed 7 | 8 | from googlesearch import search 9 | import requests 10 | from bs4 import BeautifulSoup 11 | import backoff 12 | import openai 13 | 14 | # ----------------------------------------------------------------------------- 15 | # Default configuration and Prompts 16 | NUM_SEARCH = 10 # Number of links to parse from Google 17 | SEARCH_TIME_LIMIT = 3 # Max seconds to request website sources before skipping to the next URL 18 | TOTAL_TIMEOUT = 6 # Overall timeout for all operations 19 | MAX_CONTENT = 500 # Number of words to add to LLM context for each search result 20 | MAX_TOKENS = 1000 # Maximum number of tokens LLM generates 21 | LLM_MODEL = 'gpt-4o-mini' #'gpt-3.5-turbo' #'gpt-4o' 22 | 23 | system_prompt_search = """You are a helpful assistant whose primary goal is to decide if a user's query requires a Google search.""" 24 | search_prompt = """ 25 | Decide if a user's query requires a Google search. You should use Google search for most queries to find the most accurate and updated information. Follow these conditions: 26 | 27 | - If the query does not require Google search, you must output "ns", short for no search. 28 | - If the query requires Google search, you must respond with a reformulated user query for Google search. 29 | - User query may sometimes refer to previous messages. Make sure your Google search considers the entire message history. 30 | 31 | User Query: 32 | {query} 33 | """ 34 | 35 | system_prompt_answer = """You are a helpful assistant who is expert at answering user's queries""" 36 | answer_prompt = """Generate a response that is informative and relevant to the user's query 37 | User Query: 38 | {query} 39 | """ 40 | 41 | system_prompt_cited_answer = """You are a helpful assistant who is expert at answering user's queries based on the cited context.""" 42 | cited_answer_prompt = """ 43 | Provide a relevant, informative response to the user's query using the given context (search results with [citation number](website link) and brief descriptions). 44 | 45 | - Answer directly without referring the user to any external links. 46 | - Use an unbiased, journalistic tone and avoid repeating text. 47 | - Format your response in markdown with bullet points for clarity. 48 | - Cite all information using [citation number](website link) notation, matching each part of your answer to its source. 49 | 50 | Context Block: 51 | {context_block} 52 | 53 | User Query: 54 | {query} 55 | """ 56 | # ----------------------------------------------------------------------------- 57 | 58 | # Set up OpenAI API key 59 | OPENAI_API_KEY = os.getenv('OPENAI_API_KEY') 60 | if not OPENAI_API_KEY: 61 | raise ValueError("OpenAI API key is not set. Please set the OPENAI_API_KEY environment variable.") 62 | 63 | client =openai.OpenAI(api_key=OPENAI_API_KEY) 64 | 65 | def trace_function_factory(start): 66 | """Create a trace function to timeout request""" 67 | def trace_function(frame, event, arg): 68 | if time.time() - start > TOTAL_TIMEOUT: 69 | raise TimeoutError('Website fetching timed out') 70 | return trace_function 71 | return trace_function 72 | 73 | def fetch_webpage(url, timeout): 74 | """Fetch the content of a webpage given a URL and a timeout.""" 75 | start = time.time() 76 | sys.settrace(trace_function_factory(start)) 77 | try: 78 | print(f"Fetching link: {url}") 79 | response = requests.get(url, timeout=timeout) 80 | response.raise_for_status() 81 | soup = BeautifulSoup(response.text, 'lxml') 82 | paragraphs = soup.find_all('p') 83 | page_text = ' '.join([para.get_text() for para in paragraphs]) 84 | return url, page_text 85 | except (requests.exceptions.RequestException, TimeoutError) as e: 86 | print(f"Error fetching {url}: {e}") 87 | finally: 88 | sys.settrace(None) 89 | return url, None 90 | 91 | def parse_google_results(query, num_search=NUM_SEARCH, search_time_limit=SEARCH_TIME_LIMIT): 92 | """Perform a Google search and parse the content of the top results.""" 93 | urls = search(query, num_results=num_search) 94 | max_workers = os.cpu_count() or 1 # Fallback to 1 if os.cpu_count() returns None 95 | with ThreadPoolExecutor(max_workers=max_workers) as executor: 96 | future_to_url = {executor.submit(fetch_webpage, url, search_time_limit): url for url in urls} 97 | return {url: page_text for future in as_completed(future_to_url) if (url := future.result()[0]) and (page_text := future.result()[1])} 98 | 99 | def save_markdown(content, file_path): 100 | with open(file_path, 'a') as file: 101 | file.write(content) 102 | 103 | @backoff.on_exception(backoff.expo, (openai.RateLimitError, openai.APITimeoutError)) 104 | def llm_check_search(query, file_path, msg_history=None, llm_model=LLM_MODEL): 105 | """Check if query requires search and execute Google search.""" 106 | prompt = search_prompt.format(query=query) 107 | msg_history = msg_history or [] 108 | new_msg_history = msg_history + [{"role": "user", "content": prompt}] 109 | response = client.chat.completions.create( 110 | model=llm_model, 111 | messages=[{"role": "system", "content": system_prompt_search}, *new_msg_history], 112 | max_tokens=30 113 | ).choices[0].message.content 114 | 115 | # check if the response contains "ns" 116 | cleaned_response = response.lower().strip() 117 | if re.fullmatch(r"\bns\b", cleaned_response): 118 | print("No Google search required.") 119 | return None 120 | else: 121 | print(f"Performing Google search: {cleaned_response}") 122 | search_dic = parse_google_results(cleaned_response) 123 | # Format search result in dic into markdown format 124 | search_result_md = "\n".join([f"{number+1}. {link}" for number, link in enumerate(search_dic.keys())]) 125 | save_markdown(f"## Sources\n{search_result_md}\n\n", file_path) 126 | return search_dic 127 | 128 | @backoff.on_exception(backoff.expo, (openai.RateLimitError, openai.APITimeoutError)) 129 | def llm_answer(query, file_path, msg_history=None, search_dic=None, llm_model=LLM_MODEL, max_content=MAX_CONTENT, max_tokens=MAX_TOKENS, debug=False): 130 | """Build the prompt for the language model including the search results context.""" 131 | if search_dic: 132 | context_block = "\n".join([f"[{i+1}]({url}): {content[:max_content]}" for i, (url, content) in enumerate(search_dic.items())]) 133 | prompt = cited_answer_prompt.format(context_block=context_block, query=query) 134 | system_prompt = system_prompt_cited_answer 135 | else: 136 | prompt = answer_prompt.format(query=query) 137 | system_prompt = system_prompt_answer 138 | 139 | """Generate a response using the OpenAI language model with stream completion""" 140 | msg_history = msg_history or [] 141 | new_msg_history = msg_history + [{"role": "user", "content": prompt}] 142 | response = client.chat.completions.create( 143 | model=llm_model, 144 | messages=[{"role": "system", "content": system_prompt}, *new_msg_history], 145 | max_tokens=max_tokens, 146 | stream=True 147 | ) 148 | 149 | print("\n" + "*" * 20 + " LLM START " + "*" * 20) 150 | save_markdown(f"## Answer\n", file_path) 151 | content = [] 152 | for chunk in response: 153 | chunk_content = chunk.choices[0].delta.content 154 | if chunk_content: 155 | content.append(chunk_content) 156 | print(chunk_content, end="") 157 | save_markdown(chunk_content, file_path) 158 | 159 | print("\n" + "*" * 21 + " LLM END " + "*" * 21 + "\n") 160 | # change the line for the next question 161 | save_markdown("\n\n", file_path) 162 | new_msg_history = new_msg_history + [{"role": "assistant", "content": ''.join(content)}] 163 | 164 | return new_msg_history 165 | 166 | def main(): 167 | """Main function to execute the search, generate response, and save to markdown.""" 168 | msg_history = None 169 | file_path = "playground.md" 170 | save_path = None 171 | # start with an empty file 172 | with open(file_path, 'w') as file: 173 | pass 174 | 175 | while True: 176 | query = input("Enter your question: ") 177 | #quit the program 178 | if query == "q": 179 | break 180 | # save the content in another file 181 | elif query == "s": 182 | if save_path: 183 | shutil.copy(file_path, save_path) 184 | print(f"AI response saved into {save_path}") 185 | # reset the saving mechanism 186 | save_path = None 187 | with open(file_path, 'w') as file: 188 | pass 189 | else: 190 | print("No content is saved") 191 | continue 192 | # LLM answers 193 | else: 194 | save_markdown(f"# {query}\n\n", file_path) 195 | search_dic = llm_check_search(query, file_path, msg_history) 196 | msg_history = llm_answer(query, file_path, msg_history, search_dic) 197 | save_path = save_path or f"{query}.md" # ensure saved file has the first query as its name 198 | print(f"AI response recorded into {file_path}") 199 | print("-" * 51) 200 | print("Enter a key for [s]ave or [q]uit") 201 | 202 | if __name__ == "__main__": 203 | main() 204 | -------------------------------------------------------------------------------- /playground.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yusuke710/nanoPerplexityAI/f449bc505e8eb222ce5e459cdb9bdab8e9d1bd34/playground.md --------------------------------------------------------------------------------