├── assets
├── CHAT.png
├── encoders_medical.png
└── vector_database.png
├── requirements.txt
├── Readme.md
├── vector_database.py
└── E2E.ipynb
/assets/CHAT.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/somiljain7/Med_bot_receptionist/main/assets/CHAT.png
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | qdrant_client
2 | numpy<2
3 | sentence_transformers
4 | transformers
5 | spacy
6 |
--------------------------------------------------------------------------------
/assets/encoders_medical.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/somiljain7/Med_bot_receptionist/main/assets/encoders_medical.png
--------------------------------------------------------------------------------
/assets/vector_database.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/somiljain7/Med_bot_receptionist/main/assets/vector_database.png
--------------------------------------------------------------------------------
/Readme.md:
--------------------------------------------------------------------------------
1 | ## Receptionist Bot assisting a Doctor
2 | 
3 |
4 | ### INSTRUCTIONS FOR INFERENCING
5 | ```
6 | - pip install -r requirements.txt
7 | - docker run -p 6333:6333 qdrant/qdrant
8 | ```
9 | 
10 |
11 | ```
12 | - Generate vector database for semantic search
13 | - python vector_database.py
14 |
15 | ```
16 |
17 | ### Encoders available
18 | 
19 |
20 | ```
21 | - Run all cells in jupyter notebook 'E2E.ipynb'
22 | ```
23 |
24 | ### OUTPUT SAMPLE ####
25 | ```
26 | Hello! My name is RIYA and i work as a receptionist for Dr. Adrin. How can I help you today?
27 |
28 | RIYA : Please Send a message or Report an emergency
29 |
30 | RIYA : You selected 'Report an emergency'.
31 | User: I am not able to breathe properly dealing with chest pain
32 |
33 | Processing the emergency report...
34 | RIYA : Please hold just a sec
35 | User: Indore MP India
36 | RIYA : Location 'Indore MP India' received. Dr. Adrin will be coming to their location immediately. ETA:00:23:44
37 |
38 | RIYA : Processing complete. Here is the analysis from our system:
39 | The analysis indicates a possible condition based on the symptoms: ({'name': 'Unconscious', 'remedy': 'If someone is unconscious or unresponsive, follow the ABC principle: Airway, Breathing, and Circulation. \n 1. Airway: Open their airway if they are not breathing. \n 2. Breathing: If the airway is clear but the person is still not breathing, provide rescue breathing.\n 3. Circulation: Perform chest compressions to maintain blood circulation while doing rescue breathing. \n If the person is unresponsive, check their pulse. Begin CPR by pushing against the chest and blowing air into their mouth in a constant rhythm. If their heart has stopped, continue chest compressions.'}, 'score:', 0.4441438)
40 | User: it says 23 min , it will too late for his arrival
41 | RIYA : I understand that you are worried that Dr. Adrin will arrive too late, meanwhile we would suggest that you follow following remedy:The analysis indicates a possible condition based on the symptoms: ({'name': 'Unconscious', 'remedy': 'If someone is unconscious or unresponsive, follow the ABC principle: Airway, Breathing, and Circulation. \n 1. Airway: Open their airway if they are not breathing. \n 2. Breathing: If the airway is clear but the person is still not breathing, provide rescue breathing.\n 3. Circulation: Perform chest compressions to maintain blood circulation while doing rescue breathing. \n If the person is unresponsive, check their pulse. Begin CPR by pushing against the chest and blowing air into their mouth in a constant rhythm. If their heart has stopped, continue chest compressions.'}, 'score:', 0.4441438)
42 |
43 |
44 | RIYA : Thank you for using the service. Stay safe!
45 | ```
46 |
--------------------------------------------------------------------------------
/vector_database.py:
--------------------------------------------------------------------------------
1 | from qdrant_client import models,QdrantClient
2 | import numpy
3 | from sentence_transformers import SentenceTransformer, util
4 | from transformers import AutoTokenizer, AutoModelForTokenClassification
5 | from transformers import pipeline
6 |
7 |
8 | documents = [
9 | {"name": "Unconscious",
10 | "remedy": """If someone is unconscious or unresponsive, follow the ABC principle: Airway, Breathing, and Circulation.
11 | 1. Airway: Open their airway if they are not breathing.
12 | 2. Breathing: If the airway is clear but the person is still not breathing, provide rescue breathing.
13 | 3. Circulation: Perform chest compressions to maintain blood circulation while doing rescue breathing.
14 | If the person is unresponsive, check their pulse. Begin CPR by pushing against the chest and blowing air into their mouth in a constant rhythm. If their heart has stopped, continue chest compressions."""},
15 |
16 | {"name": "Cardiac Arrest",
17 | "remedy": "Perform CPR (Cardiopulmonary Resuscitation) and use an AED (Automated External Defibrillator) if available. Call emergency services immediately."},
18 |
19 | {"name": "Severe Bleeding",
20 | "remedy": "Apply direct pressure to the wound using a clean cloth or bandage. Elevate the injured area if possible. Seek immediate medical attention."},
21 |
22 | {"name": "Choking",
23 | "remedy": "Perform the Heimlich maneuver to dislodge the object blocking the airway."},
24 |
25 | {"name": "Stroke",
26 | "remedy": "Use the FAST acronym to recognize symptoms: Face drooping, Arm weakness, Speech difficulty, and Time to call emergency services. Seek immediate medical help."},
27 |
28 | {"name": "Severe Allergic Reaction (Anaphylaxis)",
29 | "remedy": "Administer an epinephrine auto-injector (EpiPen) if available and call emergency services."},
30 |
31 | {"name": "Broken Bones",
32 | "remedy": "Immobilize the affected area, apply a cold pack to reduce swelling, and seek medical attention."},
33 |
34 | {"name": "Ingestion of Poison",
35 | "remedy": "Call poison control or emergency services immediately. Do not induce vomiting unless instructed by a professional."},
36 |
37 | {"name": "Carbon Monoxide Poisoning",
38 | "remedy": "Move the person to fresh air immediately, call emergency services, and administer oxygen if trained to do so."}
39 | ]
40 |
41 | def vector_database_init():
42 | # initializing Qdrant client at port 6333
43 | qdrant = QdrantClient(
44 | host='localhost',
45 | port=6333
46 | )
47 | return qdrant
48 |
49 | def vector_database_gen(qdrant,encoder,documents):
50 | # Generating a Vector database
51 | #args:
52 | #qdrant: object of QdrantClient
53 | #encoder: encoder for semantic similarity
54 | #documents: a list of json objects stored as records
55 |
56 | qdrant.recreate_collection(
57 | collection_name='Clinical',
58 | vectors_config=models.VectorParams(
59 | size=encoder.get_sentence_embedding_dimension(),
60 | distance=models.Distance.COSINE
61 | )
62 | )
63 | records=[]
64 |
65 | for idx, doc in enumerate(documents):
66 | record = models.Record(
67 | id=idx,
68 | vector=encoder.encode(doc['remedy']).tolist(),
69 | payload=doc
70 | )
71 | records.append(record)
72 |
73 | qdrant.upload_records(
74 | collection_name='Clinical',
75 | records=records
76 | )
77 |
78 |
79 | def vector_search(qdrant,encoder,x):
80 | # Search within a Vector database
81 | #args:
82 | #qdrant: object of QdrantClient
83 | #encoder: encoder for semantic similarity
84 | #x: sentence for calculating semantic similarity with records
85 | hits = qdrant.search(
86 | collection_name='Clinical',
87 | query_vector=encoder.encode(x).tolist(),
88 | limit=1
89 | )
90 | for hit in hits:
91 | return (hit.payload)#,'score:',hit.score)
92 |
93 |
94 | def process(symptoms):
95 | # Generating the Output of vector search
96 | print("\nProcessing the emergency report...")
97 | time.sleep(15) # Simulate processing time
98 | encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
99 | qdrant=vector_database_init()
100 | llm_output = f"The analysis indicates a possible condition based on the symptoms: {vector_search(qdrant,encoder,symptoms)}"
101 | return llm_output
102 |
103 |
104 | #qdrant=vector_database_init()
105 | #encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
106 | #vector_database_gen(qdrant,encoder,documents)
--------------------------------------------------------------------------------
/E2E.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 1,
6 | "metadata": {},
7 | "outputs": [
8 | {
9 | "name": "stderr",
10 | "output_type": "stream",
11 | "text": [
12 | "e:\\conda\\envs\\GEN_AI_T1\\Lib\\site-packages\\sentence_transformers\\cross_encoder\\CrossEncoder.py:11: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
13 | " from tqdm.autonotebook import tqdm, trange\n"
14 | ]
15 | }
16 | ],
17 | "source": [
18 | "from qdrant_client import models,QdrantClient\n",
19 | "import numpy\n",
20 | "#pip install 'numpy<2'\n",
21 | "import time\n",
22 | "import threading\n",
23 | "from sentence_transformers import SentenceTransformer, util\n",
24 | "from transformers import AutoTokenizer, AutoModelForTokenClassification\n",
25 | "from transformers import pipeline\n",
26 | "import spacy\n",
27 | "import random\n",
28 | "import datetime\n"
29 | ]
30 | },
31 | {
32 | "cell_type": "code",
33 | "execution_count": 2,
34 | "metadata": {},
35 | "outputs": [],
36 | "source": [
37 | "def model_init():\n",
38 | " #tokenizer = AutoTokenizer.from_pretrained(\"dslim/bert-base-NER\") #for a different NER \n",
39 | " #model = AutoModelForTokenClassification.from_pretrained(\"dslim/bert-base-NER\") #for a different NER\n",
40 | " ref_model = SentenceTransformer('all-MiniLM-L6-v2')\n",
41 | " nlp = spacy.load('en_core_web_sm')\n",
42 | " #nlp = pipeline(\"ner\", model=model, tokenizer=tokenizer) #for a different NER\n",
43 | " return ref_model,nlp\n",
44 | "\n",
45 | "def location_ner(nlp,res):\n",
46 | " doc = nlp(res)\n",
47 | " # Extract entities recognized as locations (GPE: Geopolitical Entities, LOC: Non-GPE locations)\n",
48 | " locations = [ent.text for ent in doc.ents if ent.label_ in ['GPE', 'LOC']]\n",
49 | " if locations:\n",
50 | " return ', '.join(locations)\n",
51 | " else:\n",
52 | " return 0\n",
53 | " #\"I didn't understand your response. Could you please provide the area that you are located in right now?\"\n",
54 | " \n",
55 | "def is_semantically_related(ref_model,sentence, reference_sentences, threshold=0.7):\n",
56 | " sentence_embedding = ref_model.encode(sentence, convert_to_tensor=True)\n",
57 | " for ref_sentence in reference_sentences:\n",
58 | " ref_embedding = ref_model.encode(ref_sentence, convert_to_tensor=True)\n",
59 | " similarity = util.pytorch_cos_sim(sentence_embedding, ref_embedding)\n",
60 | " if similarity.item() > threshold:\n",
61 | " return True, similarity.item()\n",
62 | "\n",
63 | " return False, 0\n",
64 | "\n",
65 | "def relevance_check(ref_model,user_sentence): \n",
66 | " reference_sentences_set1 = [\"too late\", \"arrive too late\",\"i am panicing\", \"won't make it in time\", \"delayed\", \"take too long\", \"not soon enough\", \"we can't wait\", \"need help immediately\", \"time-sensitive\", \"urgent\" ,\"The doctor will not arrive in time.\", \"I am worried the doctor will be too late.\"]\n",
67 | " reference_sentences_set2 = [\"No Thank you\",'Thanks for all the help','Thanks']\n",
68 | " is_related_1, similarity_score_1 = is_semantically_related(ref_model,user_sentence, reference_sentences=reference_sentences_set1)\n",
69 | " is_related_2, similarity_score_2 = is_semantically_related(ref_model,user_sentence, reference_sentences=reference_sentences_set2)\n",
70 | " if is_related_1 or is_related_2:\n",
71 | " if similarity_score_1 > similarity_score_2:\n",
72 | " return 1\n",
73 | " else:\n",
74 | " return 2\n",
75 | " else:\n",
76 | " return 0\n",
77 | " \n",
78 | "def vector_search(qdrant,encoder,x,collect_name):\n",
79 | " # Search within a Vector database \n",
80 | " #args:\n",
81 | " #qdrant: object of QdrantClient\n",
82 | " #encoder: encoder for semantic similarity\n",
83 | " #x: sentence for calculating semantic similarity with records \n",
84 | " hits = qdrant.search(\n",
85 | " collection_name=collect_name,\n",
86 | " query_vector=encoder.encode(x).tolist(),\n",
87 | " limit=1\n",
88 | " )\n",
89 | " for hit in hits:\n",
90 | " return (hit.payload,'score:',hit.score)\n",
91 | "\n",
92 | "\n",
93 | "def process_response(res):\n",
94 | " # Generating the Output of vector search\n",
95 | " encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')\n",
96 | " qdrant=vector_database_init()\n",
97 | " llm_output = f\"{vector_search(qdrant,encoder,res,\"Responses\")[0][\"name\"]}\"\n",
98 | " return llm_output\n",
99 | "\n",
100 | "def vector_database_init():\n",
101 | " # initializing Qdrant client at port 6333\n",
102 | " qdrant = QdrantClient(\n",
103 | " host='localhost',\n",
104 | " port=6333\n",
105 | " )\n",
106 | " return qdrant\n",
107 | "\n",
108 | "\n",
109 | "def process(symptoms):\n",
110 | " # Generating the Output of vector search\n",
111 | " print(\"\\nProcessing the emergency report...\")\n",
112 | " time.sleep(15) # Simulate processing time\n",
113 | " encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')\n",
114 | " qdrant=vector_database_init()\n",
115 | " llm_output = f\"The analysis indicates a possible condition based on the symptoms: {vector_search(qdrant,encoder,symptoms,\"Clinical\")}\"\n",
116 | " return llm_output\n",
117 | "\n",
118 | "\n",
119 | "def generate_random_eta():\n",
120 | " random_seconds = random.randint(60, 7200)\n",
121 | " eta = datetime.timedelta(seconds=random_seconds)\n",
122 | " total_seconds = int(eta.total_seconds())\n",
123 | " hours = total_seconds // 3600\n",
124 | " minutes = (total_seconds % 3600) // 60\n",
125 | " seconds = total_seconds % 60\n",
126 | " formatted_eta = f\"{hours:02}:{minutes:02}:{seconds:02}\"\n",
127 | " return formatted_eta"
128 | ]
129 | },
130 | {
131 | "cell_type": "code",
132 | "execution_count": 3,
133 | "metadata": {},
134 | "outputs": [
135 | {
136 | "name": "stderr",
137 | "output_type": "stream",
138 | "text": [
139 | "e:\\conda\\envs\\GEN_AI_T1\\Lib\\site-packages\\transformers\\tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884\n",
140 | " warnings.warn(\n"
141 | ]
142 | },
143 | {
144 | "name": "stdout",
145 | "output_type": "stream",
146 | "text": [
147 | "Hello! My name is RIYA and i work as a receptionist for Dr. Adrin. How can I help you today?\n",
148 | "\n",
149 | "RIYA : Please Send a message or Report an emergency\n",
150 | "\n",
151 | "RIYA : You selected 'Report an emergency'.\n",
152 | "User: I am not able to breathe properly dealing with chest pain\n",
153 | "\n",
154 | "Processing the emergency report...\n",
155 | "RIYA : Please hold just a sec\n",
156 | "User: Indore MP India\n",
157 | "RIYA : Location 'Indore MP India' received. Dr. Adrin will be coming to their location immediately. ETA:00:23:44\n",
158 | "\n",
159 | "RIYA : Processing complete. Here is the analysis from our system:\n",
160 | "The analysis indicates a possible condition based on the symptoms: ({'name': 'Unconscious', 'remedy': 'If someone is unconscious or unresponsive, follow the ABC principle: Airway, Breathing, and Circulation. \\n 1. Airway: Open their airway if they are not breathing. \\n 2. Breathing: If the airway is clear but the person is still not breathing, provide rescue breathing.\\n 3. Circulation: Perform chest compressions to maintain blood circulation while doing rescue breathing. \\n If the person is unresponsive, check their pulse. Begin CPR by pushing against the chest and blowing air into their mouth in a constant rhythm. If their heart has stopped, continue chest compressions.'}, 'score:', 0.4441438)\n",
161 | "User: it says 23 min , it will too late for his arrival\n",
162 | "RIYA : I understand that you are worried that Dr. Adrin will arrive too late, meanwhile we would suggest that you follow following remedy:The analysis indicates a possible condition based on the symptoms: ({'name': 'Unconscious', 'remedy': 'If someone is unconscious or unresponsive, follow the ABC principle: Airway, Breathing, and Circulation. \\n 1. Airway: Open their airway if they are not breathing. \\n 2. Breathing: If the airway is clear but the person is still not breathing, provide rescue breathing.\\n 3. Circulation: Perform chest compressions to maintain blood circulation while doing rescue breathing. \\n If the person is unresponsive, check their pulse. Begin CPR by pushing against the chest and blowing air into their mouth in a constant rhythm. If their heart has stopped, continue chest compressions.'}, 'score:', 0.4441438) \n",
163 | "\n",
164 | "\n",
165 | "RIYA : Thank you for using the service. Stay safe!\n"
166 | ]
167 | }
168 | ],
169 | "source": [
170 | "\n",
171 | "ref_model,nlp=model_init()\n",
172 | "\n",
173 | "print(\"Hello! My name is RIYA and i work as a receptionist for Dr. Adrin. How can I help you today?\")\n",
174 | "while True:\n",
175 | " print(\"\\nRIYA : Please Send a message or Report an emergency\")\n",
176 | " user_choice = input(\"RIYA : Please Send a message or Report an emergency\")\n",
177 | " choice=process_response(user_choice)\n",
178 | " if choice==\"Message\":\n",
179 | " message = input(\"\\nPlease enter the message you want to send: \")\n",
180 | " print( \"RIYA :Thanks for the message, we will forward it to Dr. Adrin\")\n",
181 | " break\n",
182 | " elif choice == \"Emergency\":\n",
183 | " print(\"\\nRIYA : You selected 'Report an emergency'.\")\n",
184 | " symptoms = input(\"Please describe the symptoms: \")\n",
185 | " print(f\"User: {symptoms}\")\n",
186 | " llm_output = []\n",
187 | " def process_and_store():\n",
188 | " result = process(symptoms)\n",
189 | " llm_output.append(result) # Store the semantic similarity search output in a list\n",
190 | " processing_thread = threading.Thread(target=process_and_store)\n",
191 | " processing_thread.start()\n",
192 | "\n",
193 | " print(\"RIYA : Please hold just a sec\") \n",
194 | " # Simultaneously ask for location\n",
195 | " location = input(\"RIYA : I am checking what you should do immediately, meanwhile, can you tell me which area are you located right now?\")\n",
196 | " ETA=generate_random_eta()\n",
197 | " print(f\"User: {location}\")\n",
198 | " if location_ner(nlp,location)!=0:\n",
199 | " print(f\"RIYA : Location '{location}' received. Dr. Adrin will be coming to their location immediately. ETA:{ETA}\")\n",
200 | " else: \n",
201 | " location=input(\"I don't understand that and could you please repeat the area you are located right now (in proper format)? \")\n",
202 | " print(f\"User: {location}\")\n",
203 | " if(location_ner(nlp,location)==0):\n",
204 | " print(\"RIYA : I don’t understand that and unable to fetch your location ,could you please try again later .\")\n",
205 | " exit()\n",
206 | " else:\n",
207 | " print(f\"RIYA : Location '{location}' received. Dr. Adrin will be coming to their location immediately. ETA:{ETA}\")\n",
208 | " processing_thread.join()\n",
209 | " \n",
210 | " # Print the semantic similarity search output after location input\n",
211 | " print(\"\\nRIYA : Processing complete. Here is the analysis from our system:\")\n",
212 | " print(llm_output[0]) # Access the first (and only) element in the list\n",
213 | " response = input(\"Do you have anything else to ask ?\") \n",
214 | " print(f\"User: {response}\")\n",
215 | " res_rev=process_response(response)\n",
216 | " #res_rev=relevance_check(ref_model,response)\n",
217 | " if res_rev==\"Late\":\n",
218 | " print(f\"RIYA : I understand that you are worried that Dr. Adrin will arrive too late, meanwhile we would suggest that you follow following remedy:{llm_output[0]} \\n\" )\n",
219 | "\n",
220 | " else:\n",
221 | " print(\"RIYA : I don't understand that but Don’t worry, please follow the steps, Dr. Adrin will be with you shortly\")\n",
222 | " break\n",
223 | " else:\n",
224 | " print(\"\\nRIYA : Invalid Response. Please mention wether you like to send a message or to report an emergency.\")\n",
225 | " \n",
226 | "print(\"\\nRIYA : Thank you for using the service. Stay safe!\")\n",
227 | "\n"
228 | ]
229 | },
230 | {
231 | "cell_type": "code",
232 | "execution_count": null,
233 | "metadata": {},
234 | "outputs": [],
235 | "source": [
236 | "\n"
237 | ]
238 | }
239 | ],
240 | "metadata": {
241 | "kernelspec": {
242 | "display_name": "GEN_AI_T1",
243 | "language": "python",
244 | "name": "python3"
245 | },
246 | "language_info": {
247 | "codemirror_mode": {
248 | "name": "ipython",
249 | "version": 3
250 | },
251 | "file_extension": ".py",
252 | "mimetype": "text/x-python",
253 | "name": "python",
254 | "nbconvert_exporter": "python",
255 | "pygments_lexer": "ipython3",
256 | "version": "3.12.5"
257 | }
258 | },
259 | "nbformat": 4,
260 | "nbformat_minor": 2
261 | }
262 |
--------------------------------------------------------------------------------