├── LICENSE
├── README.md
├── app.py
├── requirements.txt
└── templates
└── index.html
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2025 KARRI VAMSI KRISHNA
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
50 |
51 |
52 |
53 |
55 |
56 |
57 |
58 |
60 |
61 |
62 |
63 | 📋 Table of Contents
67 |
68 | | | Section | Description |
69 | |----|----------------------------------------|--------------------------------------|
70 | | 📺 | [Demo Video](#-demo-video) | See KV in action |
71 | | 🌟 | [Key Features](#-key-features) | What makes KV special |
72 | | 🖼️ | [Screenshots](#-screenshots) | Visual previews |
73 | | 🚀 | [Why Choose KV](#-why-choose-kv) | Benefits & advantages |
74 | | ⚙️ | [Installation](#%EF%B8%8F-installation) | Get up and running |
75 | | 🎮 | [Usage Guide](#-usage-guide) | How to use KV effectively |
76 | | 🤝 | [Contribution](#-contribution) | Join our community |
77 | | 📄 | [License](#-license) | MIT License |
78 | | 📞 | [Contact & Support](#-contact--support) | Get in touch |
79 |
80 |
81 |
82 |
83 |
84 |
85 |
86 |
89 | KV is a revolutionary open-source platform that combines the power of Google Gemini AI with advanced web scraping for unparalleled research and job search capabilities. KV goes beyond conventional tools to provide deep insights, analyze resumes, identify skill gaps, and help you find your ideal career path.
90 |
91 |
92 |
93 |
94 |
96 |
97 | ## 📺 Demo Video
98 |
99 |
108 |
109 |
110 |
111 |
113 |
114 | ## 🌟 Key Features
115 |
116 |
117 |
118 | | 🔥 Feature | 💫 Description |
119 | |:--:|:--|
120 | |
| **Advanced AI Integration**
Leverages Google Gemini's powerful AI capabilities for deep analysis |
121 | |
| **Intelligent Web Scraping**
Gathers comprehensive data from across the internet |
122 | |
| **Multi-Search Engine Support**
Access Google, Bing, DuckDuckGo, and LinkedIn simultaneously |
123 | |
| **Iterative Research**
Self-refining search strategies for more precise results |
124 | |
| **Resume Analysis**
AI-powered evaluation of your resume with improvement suggestions |
125 | |
| **Job Match Algorithm**
Finds ideal job opportunities based on your profile |
126 | |
| **Skill Gap Detection**
Identifies missing skills needed for your target positions |
127 | |
| **Professional Reports**
Generates beautiful PDF reports with data visualizations |
128 | |
| **UI Customization**
Dark & light mode with responsive design for all devices |
129 |
130 |
134 |
136 |
137 | ## 🖼️ Screenshots
138 |
139 |
140 |
141 |
142 |
📱 Modern Chat Interface
143 |
144 |
145 |
146 |
147 |
148 |
149 |
🔍 Deep Research in Action
150 |
151 |
152 |
153 |
154 |
155 |
156 |
⚙️ Customization Options
157 |
158 |
159 |
163 |
165 |
166 | ## 🚀 Why Choose KV
167 |
168 |
197 |
198 |
199 |
200 |
202 |
203 | ## ⚙️ Installation
204 |
205 |
206 |
207 |
⚡ Quick Setup
208 |
209 |
210 | ```bash
211 | # Clone repository
212 | git clone https://github.com/kvcops/Deep-Research-using-Gemini-api.git
213 |
214 | # Change directory
215 | cd Deep-Research-using-Gemini-api
216 |
217 | # Create virtual environment
218 | python -m venv venv
219 |
220 | # Activate virtual environment
221 | # On Windows:
222 | venv\Scripts\activate
223 | # On macOS/Linux:
224 | source venv/bin/activate
225 |
226 | # Install dependencies
227 | pip install -r requirements.txt
228 |
229 | # Create .env file with your API key
230 | echo "GEMINI_API_KEY=YOUR_ACTUAL_GEMINI_API_KEY" > .env
231 |
232 | # Launch KV
233 | uvicorn app:app --host 127.0.0.1 --port 8000 --reload
234 | ```
235 |
236 |
242 |
243 |
244 |
245 |
247 |
248 | ## 🎮 Usage Guide
249 |
250 |
251 |
🚀 Get Started in 3 Easy Steps
252 |
253 |
254 |
255 |
256 |
257 |
259 |
260 | Start a new chat session or research query
261 | Navigate to the homepage and select your desired research mode
262 |
263 |
264 |
265 |
266 |
268 |
269 | Upload your resume for job search or enter research topic
270 | Provide relevant documents and specify your preferences
271 |
272 |
273 |
274 |
275 |
277 |
278 | Review results, download reports, and implement recommendations
279 | KV provides actionable insights and suggestions for improvement
280 |
281 |
282 |
283 |
284 |
285 |
286 |
288 |
289 | ## 🤝 Contribution
290 |
291 |
292 |
293 |
296 |
We welcome all contributions! Here's how to get involved:
297 |
298 |
299 |
323 |
324 |
332 |
333 |
334 |
335 |
337 |
338 | ## 📄 License
339 |
340 |
341 |
342 | This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
343 |
344 |
345 |
346 |
363 |
364 |
✨ Thank you for using KV! ✨
365 |
If you found it helpful, please consider giving it a star! ⭐
366 |
367 |
368 |
370 |
371 |
372 |
373 |
374 |
375 |
\1 ', text) # Bold
1091 | text = re.sub(r'\*(.*?)\*', r'\1 ', text) # Italics
1092 | text = re.sub(r'`(.*?)`', r'\1 ', text) # Inline code
1093 | text = re.sub(r'\[(.*?)\]\((.*?)\)', r'\1 ', text) # Links
1094 | # Escape HTML entities to prevent issues in Paragraph
1095 | text = text.replace('&', '&').replace('<', '<').replace('>', '>')
1096 | return text.strip()
1097 |
1098 |
1099 | def process_table(table_text):
1100 | rows = [row.strip() for row in table_text.split('\n') if row.strip()]
1101 | if len(rows) < 2:
1102 | return None # Not a valid table
1103 |
1104 | header = [clean_text(cell) for cell in rows[0].strip('|').split('|')]
1105 | data_rows = []
1106 | for row in rows[2:]: # Skip header and separator lines
1107 | cells = [clean_text(cell) for cell in row.strip('|').split('|')]
1108 | data_rows.append(cells)
1109 |
1110 | # Convert to ReportLab Paragraph objects
1111 | table_data = [[Paragraph(cell, custom_styles['TableCell']) for cell in header]]
1112 | for row in data_rows:
1113 | table_data.append([Paragraph(cell, custom_styles['TableCell']) for cell in row])
1114 |
1115 | table_style = TableStyle([
1116 | ('BACKGROUND', (0, 0), (-1, 0), HexColor("#f5f5f5")), # Header background
1117 | ('TEXTCOLOR', (0, 0), (-1, 0), HexColor("#1a237e")), # Header text color
1118 | ('ALIGN', (0, 0), (-1, -1), 'CENTER'), # Center-align all cells
1119 | ('FONTNAME', (0, 0), (-1, 0), 'Helvetica-Bold'), # Header font
1120 | ('FONTSIZE', (0, 0), (-1, 0), 10),
1121 | ('BOTTOMPADDING', (0, 0), (-1, 0), 8), # Header padding
1122 | ('TOPPADDING', (0, 0), (-1, 0), 8),
1123 | ('GRID', (0, 0), (-1, -1), 1, HexColor("#e0e0e0")), # Grid
1124 | ('ROWBACKGROUNDS', (0, 1), (-1, -1), [HexColor("#ffffff"), HexColor("#f8f9fa")]), # Alternating row colors
1125 | ('ALIGN', (0, 0), (-1, -1), 'LEFT'), # Left-align cell content
1126 | ('FONTNAME', (0, 1), (-1, -1), 'Helvetica'), # Body font
1127 | ('FONTSIZE', (0, 1), (-1, -1), 9), # Body font size
1128 | ('TOPPADDING', (0, 1), (-1, -1), 6), # padding
1129 | ('BOTTOMPADDING', (0, 1), (-1, -1), 6),
1130 | ('VALIGN', (0, 0), (-1, -1), 'MIDDLE'), # Vertically center cell content
1131 | ])
1132 |
1133 | col_widths = [doc.width/len(header) for _ in header] # Equal column widths
1134 | table = Table(table_data, colWidths=col_widths)
1135 | table.setStyle(table_style)
1136 | return table
1137 |
1138 | def footer(canvas, doc):
1139 | canvas.saveState()
1140 | footer_text = f"Generated by Kv - AI Companion & Deep Research Tool • {formatted_date}"
1141 | footer = Paragraph(footer_text, custom_styles['Footer'])
1142 | w, h = footer.wrap(doc.width, doc.bottomMargin)
1143 | footer.drawOn(canvas, doc.leftMargin, h) # Draw footer
1144 | canvas.restoreState()
1145 |
1146 | story = []
1147 | story.append(Paragraph(report_title, custom_styles['Title'])) # Title
1148 | story.append(Paragraph(formatted_date, custom_styles['Footer'])) # Date
1149 | story.append(Spacer(1, 0.2*inch)) # Initial space
1150 |
1151 | current_table = []
1152 | in_table = False
1153 | lines = content.split('\n')
1154 | i = 0
1155 | current_paragraph = [] # Accumulate lines for a paragraph
1156 |
1157 | while i < len(lines):
1158 | line = lines[i].strip()
1159 |
1160 | if not line:
1161 | # Empty line: End current paragraph (if any), add it to story.
1162 | if current_paragraph:
1163 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1164 | current_paragraph = []
1165 | # Handle any pending table
1166 | if in_table and current_table:
1167 | table = process_table('\n'.join(current_table))
1168 | if table:
1169 | story.append(table)
1170 | story.append(Spacer(1, 0.1*inch))
1171 | current_table = []
1172 | in_table = False
1173 | story.append(Spacer(1, 0.05*inch)) # Consistent spacing
1174 | i += 1
1175 | continue
1176 |
1177 | if '|' in line and (line.count('|') > 1 or (i + 1 < len(lines) and '|' in lines[i + 1])):
1178 | # Likely a table row. Start/continue accumulating table lines.
1179 | in_table = True
1180 | current_table.append(line)
1181 | # End any current paragraph before starting a table.
1182 | if current_paragraph:
1183 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1184 | current_paragraph = []
1185 | elif in_table:
1186 | # End of table. Process accumulated table lines.
1187 | if current_table:
1188 | table = process_table('\n'.join(current_table))
1189 | if table:
1190 | story.append(table)
1191 | story.append(Spacer(1, 0.1*inch))
1192 | current_table = []
1193 | in_table = False
1194 | continue #Crucial to continue here, and not add to current_paragraph below.
1195 |
1196 | elif line.startswith('# '):
1197 | # End current paragraph (if any) before starting a heading.
1198 | if current_paragraph:
1199 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1200 | current_paragraph = []
1201 | story.append(Paragraph(clean_text(line[2:]), custom_styles['Heading1'])) # H1
1202 | elif line.startswith('## '):
1203 | if current_paragraph:
1204 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1205 | current_paragraph = []
1206 | story.append(Paragraph(clean_text(line[3:]), custom_styles['Heading2'])) # H2
1207 | elif line.startswith('### '):
1208 | if current_paragraph:
1209 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1210 | current_paragraph = []
1211 | story.append(Paragraph(clean_text(line[4:]), custom_styles['Heading3'])) # H3
1212 | elif line.startswith('* ') or line.startswith('- '):
1213 | # End current paragraph before a list item.
1214 | if current_paragraph:
1215 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1216 | current_paragraph = []
1217 | story.append(Paragraph(f"• {clean_text(line[2:])}", custom_styles['Bullet'])) # Bullet points
1218 |
1219 | else:
1220 | # Regular text line: add to the current paragraph.
1221 | current_paragraph.append(line)
1222 |
1223 | i += 1
1224 | # Add any remaining paragraph content (important!).
1225 | if current_paragraph:
1226 | story.append(Paragraph(clean_text(" ".join(current_paragraph)), custom_styles['Paragraph']))
1227 |
1228 | if current_table:
1229 | table = process_table('\n'.join(current_table))
1230 | if table:
1231 | story.append(table)
1232 | story.append(Spacer(1, 0.1*inch))
1233 |
1234 | if references:
1235 | story.append(PageBreak()) # References on a new page
1236 | story.append(Paragraph("References", custom_styles['Heading1']))
1237 | story.append(Spacer(1, 0.1*inch))
1238 | for i, ref in enumerate(references, 1):
1239 | story.append(Paragraph(f"[{i}] {ref}", custom_styles['Reference']))
1240 |
1241 | doc.build(story, onLaterPages=footer, onFirstPage=footer) # Apply to all pages.
1242 | buffer.seek(0)
1243 | return buffer
1244 |
1245 | # --- Product Scraping ---
1246 | def scrape_product_details(url):
1247 | """Scrapes product details from a given URL."""
1248 | try:
1249 | response = requests.get(url, headers={'User-Agent': get_random_user_agent()}, timeout=config.REQUEST_TIMEOUT)
1250 | response.raise_for_status()
1251 |
1252 | soup = BeautifulSoup(response.text, 'html.parser')
1253 | product_data = {}
1254 | # Title
1255 | for tag in ['h1', 'h2', 'span', 'div']:
1256 | for class_name in ['product-title', 'title', 'productName', 'product-name']:
1257 | if title_element := soup.find(tag, class_=class_name):
1258 | product_data['title'] = title_element.get_text(strip=True)
1259 | break
1260 | if 'title' in product_data:
1261 | break
1262 |
1263 | # Price
1264 | for tag in ['span', 'div', 'p']:
1265 | for class_name in ['price', 'product-price', 'sales-price', 'regular-price']:
1266 | if price_element := soup.find(tag, class_=class_name):
1267 | product_data['price'] = price_element.get_text(strip=True)
1268 | break
1269 | if 'price' in product_data:
1270 | break
1271 |
1272 | # Description
1273 | if (description_element := soup.find('div', {'itemprop': 'description'})):
1274 | product_data['description'] = description_element.get_text(strip=True)
1275 | else:
1276 | for class_name in ['description', 'product-description', 'product-details', 'details']:
1277 | if desc_element := soup.find(['div', 'p'], class_=class_name):
1278 | product_data['description'] = desc_element.get_text(separator='\n', strip=True)
1279 | break
1280 |
1281 | # Image URL
1282 | if (image_element := soup.find('img', {'itemprop': 'image'})):
1283 | product_data['image_url'] = urljoin(url, image_element['src'])
1284 | else:
1285 | for tag in ['img', 'div']:
1286 | for class_name in ['product-image', 'image', 'main-image', 'productImage']:
1287 | if (image_element := soup.find(tag, class_=class_name)) and image_element.get('src'):
1288 | product_data['image_url'] = urljoin(url, image_element['src'])
1289 | break
1290 | if 'image_url' in product_data:
1291 | break
1292 |
1293 | # Rating
1294 | if (rating_element := soup.find(['span', 'div'], class_=['rating', 'star-rating', 'product-rating'])):
1295 | product_data['rating'] = rating_element.get_text(strip=True)
1296 |
1297 | return product_data
1298 |
1299 | except requests.exceptions.RequestException as e:
1300 | logging.error(f"Error scraping product details from {url}: {e}")
1301 | return None # Return None on error
1302 | except Exception as e:
1303 | logging.error(f"Unexpected error scraping {url}: {e}")
1304 | return None # Return None on unexpected error
1305 |
1306 | @app.post("/api/scrape_product")
1307 | async def scrape_product_endpoint(request: Request):
1308 | try:
1309 | data = await request.json()
1310 | product_query = data.get('query', '')
1311 | if not product_query:
1312 | raise HTTPException(status_code=400, detail="No product query provided")
1313 |
1314 | search_results = []
1315 | with concurrent.futures.ThreadPoolExecutor(max_workers=config.MAX_WORKERS) as executor:
1316 | futures = [executor.submit(scrape_search_engine, product_query, engine)
1317 | for engine in config.SEARCH_ENGINES]
1318 | for future in concurrent.futures.as_completed(futures):
1319 | try:
1320 | search_results.extend(future.result())
1321 | except Exception as e:
1322 | logging.error(f"Error in search engine scrape: {e}")
1323 |
1324 | unique_urls = list(set(search_results)) # Remove duplicate URLs
1325 |
1326 | all_product_data = []
1327 | with concurrent.futures.ThreadPoolExecutor(max_workers=config.MAX_WORKERS) as executor:
1328 | futures = {executor.submit(scrape_product_details, url): url for url in unique_urls}
1329 | for future in concurrent.futures.as_completed(futures):
1330 | try:
1331 | product_data = future.result()
1332 | if product_data: # Only add if data was successfully scraped
1333 | all_product_data.append(product_data)
1334 | except Exception as e:
1335 | url = futures[future]
1336 | logging.error(f"Error processing {url}: {e}")
1337 |
1338 | if all_product_data:
1339 | # Create a prompt to summarize the product information
1340 | prompt = "Summarize the following product information:\n\n"
1341 | for product in all_product_data:
1342 | prompt += f"- Title: {product.get('title', 'N/A')}\n"
1343 | prompt += f" Price: {product.get('price', 'N/A')}\n"
1344 | prompt += f" Description: {product.get('description', 'N/A')}\n"
1345 | prompt += "\n" # Add a separator between products
1346 |
1347 | prompt += "\nProvide a concise summary, including key features and price range."
1348 |
1349 | summary = generate_gemini_response(prompt) # default model
1350 |
1351 | return JSONResponse({"summary": summary, "products": all_product_data})
1352 | else:
1353 | raise HTTPException(status_code=404, detail="No product information found")
1354 |
1355 | except HTTPException as e:
1356 | raise e # Re-raise HTTP exceptions
1357 | except Exception as e:
1358 | logging.error(f"Error in product scraping endpoint: {e}")
1359 | raise HTTPException(status_code=500, detail=str(e))
1360 |
1361 | # --- Job Scraping ---
1362 | def extract_text_from_resume(resume_data: bytes) -> str:
1363 | """Extracts text from a resume (PDF, DOCX, or plain text)."""
1364 | try:
1365 | if resume_data.startswith(b"%PDF"):
1366 | # PDF file
1367 | resume_text = pdf_extract_text(io.BytesIO(resume_data))
1368 | elif resume_data.startswith(b"PK\x03\x04"): # Common DOCX header
1369 | # DOCX file
1370 | resume_text = docx2txt.process(io.BytesIO(resume_data))
1371 | else:
1372 | # Assume plain text
1373 | try:
1374 | resume_text = resume_data.decode('utf-8')
1375 | except UnicodeDecodeError:
1376 | resume_text = resume_data.decode('latin-1', errors='replace') # Fallback encoding
1377 | return resume_text
1378 | except Exception as e:
1379 | logging.error(f"Error extracting resume text: {e}")
1380 | return ""
1381 |
1382 | # --- Helper functions for job scraping ---
1383 |
1384 | def linkedin_params(job_title: str, job_location: str, start: int = 0, experience_level: Optional[str] = None) -> Dict:
1385 | """Generates parameters for a LinkedIn job search URL."""
1386 | params = {
1387 | 'keywords': job_title,
1388 | 'location': job_location,
1389 | 'f_TPR': 'r86400', # Past 24 hours (consider making configurable)
1390 | 'sortBy': 'R', # Sort by relevance
1391 | 'start': start # pagination
1392 | }
1393 | if experience_level:
1394 | # LinkedIn-specific experience level filters (add others as needed)
1395 | if experience_level.lower() == "fresher":
1396 | params['f_E'] = '1' # Internship
1397 | elif experience_level.lower() == "entry-level":
1398 | params['f_E'] = '2' # Entry-Level
1399 | elif experience_level.lower() == "mid-level":
1400 | params['f_E'] = '3' # Associate
1401 | elif experience_level.lower() == "senior":
1402 | params['f_E'] = '4' # Senior
1403 | elif experience_level.lower() == "executive":
1404 | params['f_E'] = '5' # Director , '6' - executive
1405 | # params['f_E'] = ['4', '5'] # Combine Senior/Executive for LinkedIn
1406 |
1407 | return params
1408 |
1409 | def indeed_params(job_title: str, job_location: str, start: int = 0, experience_level: Optional[str] = None) -> Dict:
1410 | """Generates parameters for an Indeed job search URL."""
1411 | params = {
1412 | 'q': job_title,
1413 | 'l': job_location,
1414 | 'sort': 'relevance', # Sort by relevance
1415 | 'fromage': '1', # Past 24 hours
1416 | 'limit': 50, # Fetch more results per page
1417 | 'start': start # Pagination
1418 | }
1419 | if experience_level:
1420 | params['q'] = f"{experience_level} {params['q']}" # Add to the main query
1421 |
1422 | return params
1423 | def parse_linkedin_job_card(job_card: BeautifulSoup) -> Dict:
1424 | """Parses a single LinkedIn job card and extracts relevant information."""
1425 | try:
1426 | job_url_element = job_card.find('a', class_='base-card__full-link')
1427 | job_url = job_url_element['href'] if job_url_element else None
1428 |
1429 | title_element = job_card.find('h3', class_='base-search-card__title')
1430 | title = title_element.get_text(strip=True) if title_element else "N/A"
1431 |
1432 | company_element = job_card.find('h4', class_='base-search-card__subtitle')
1433 | company = company_element.get_text(strip=True) if company_element else "N/A"
1434 |
1435 | location_element = job_card.find('span', class_='job-search-card__location')
1436 | location = location_element.get_text(strip=True) if location_element else "N/A"
1437 |
1438 | return {
1439 | 'url': job_url,
1440 | 'title': title,
1441 | 'company': company,
1442 | 'location': location,
1443 | 'relevance': 0.0,
1444 | 'missing_skills': [],
1445 | 'justification': "Relevance not assessed.",
1446 | 'experience': 'N/A' # default value
1447 | }
1448 | except Exception as e:
1449 | logging.error(f"Error parsing LinkedIn job card: {e}")
1450 | return { # Return defaults on error
1451 | 'url': None,
1452 | 'title': "N/A",
1453 | 'company': "N/A",
1454 | 'location': "N/A",
1455 | 'relevance': 0.0,
1456 | 'missing_skills': [],
1457 | 'justification': f"Error parsing job card: {type(e).__name__}",
1458 | 'experience': 'N/A'
1459 | }
1460 |
1461 | def parse_indeed_job_card(job_card: BeautifulSoup) -> Dict:
1462 | """Parses a single Indeed job card and extracts relevant information."""
1463 | try:
1464 | title_element = job_card.find(['h2', 'a'], class_=lambda x: x and ('title' in x or 'jobtitle' in x))
1465 | title = title_element.get_text(strip=True) if title_element else "N/A"
1466 |
1467 | company_element = job_card.find(['span', 'a'], class_='companyName')
1468 | company = company_element.get_text(strip=True) if company_element else "N/A"
1469 |
1470 | location_element = job_card.find('div', class_='companyLocation')
1471 | location = location_element.get_text(strip=True) if location_element else "N/A"
1472 |
1473 | job_url = None
1474 | link_element = job_card.find('a', href=True)
1475 | if link_element and 'pagead' not in link_element['href']:
1476 | job_url = urljoin("https://www.indeed.com/jobs", link_element['href'])
1477 | if not job_url:
1478 | data_jk = job_card.get('data-jk')
1479 | if data_jk:
1480 | job_url = f"https://www.indeed.com/viewjob?jk={data_jk}"
1481 |
1482 | return {
1483 | 'url': job_url,
1484 | 'title': title,
1485 | 'company': company,
1486 | 'location': location,
1487 | 'relevance': 0.0,
1488 | 'missing_skills': [],
1489 | 'justification': "Relevance not assessed.",
1490 | 'experience':'N/A' # Default
1491 | }
1492 | except Exception as e:
1493 | logging.error(f"Error parsing Indeed job card: {e}")
1494 | return { # Return defaults on error
1495 | 'url': None,
1496 | 'title': "N/A",
1497 | 'company': "N/A",
1498 | 'location': "N/A",
1499 | 'relevance': 0.0,
1500 | 'missing_skills': [],
1501 | 'justification': f"Error parsing job card: {type(e).__name__}",
1502 | 'experience':'N/A'
1503 | }
1504 |
1505 | @retry(
1506 | wait=wait_exponential(multiplier=config.INDEED_BASE_DELAY, max=config.INDEED_MAX_DELAY),
1507 | stop=stop_after_attempt(config.INDEED_RETRIES),
1508 | retry=retry_if_exception_type(requests.exceptions.RequestException),
1509 | before_sleep=lambda retry_state: logging.warning(
1510 | f"Indeed request failed (attempt {retry_state.attempt_number}). Retrying in {retry_state.next_action.sleep} seconds..."
1511 | )
1512 | )
1513 | def scrape_job_site(job_title: str, job_location: str, resume_text: Optional[str],
1514 | base_url: str, params_func: callable, parse_func: callable, site_name:str,
1515 | experience_level: Optional[str] = None) -> List[Dict]:
1516 | """
1517 | Generic function to scrape job listings from a given site.
1518 |
1519 | Args:
1520 | job_title: The job title to search for.
1521 | job_location: The location to search for jobs.
1522 | resume_text: Optional resume text for relevance assessment.
1523 | base_url: The base URL of the job site.
1524 | params_func: A function that generates the URL parameters for the site.
1525 | parse_func: A function that parses a job card from the site's HTML.
1526 | site_name: The name of the job site (e.g., "LinkedIn", "Indeed").
1527 | experience_level: Optional experience level string
1528 |
1529 | Returns:
1530 | A list of dictionaries, where each dictionary represents a job listing.
1531 | """
1532 |
1533 | search_results = []
1534 | start = 0 # pagination
1535 | MAX_PAGES = 10 # Limit pages to prevent infinite loops. Adjust as needed.
1536 |
1537 | while True:
1538 | params = params_func(job_title, job_location, start, experience_level) # Pass experience
1539 | try:
1540 | headers = {'User-Agent': get_random_user_agent()} # Rotate User-Agent
1541 | response = requests.get(base_url, params=params, headers=headers, timeout=config.REQUEST_TIMEOUT)
1542 | response.raise_for_status() # Raises HTTPError for bad (4xx, 5xx) responses
1543 |
1544 | if "captcha" in response.text.lower():
1545 | logging.warning(f"{site_name} CAPTCHA detected. Stopping.")
1546 | break # Exit pagination
1547 |
1548 | soup = BeautifulSoup(response.text, 'html.parser')
1549 |
1550 | # Use a general way to find job cards (more robust to site changes)
1551 | job_cards = soup.find_all('div', class_=lambda x: x and x.startswith('job_'))
1552 | if not job_cards:
1553 | job_cards = soup.find_all('div', class_='base-card') # For linkedin try another method
1554 |
1555 |
1556 | if not job_cards:
1557 | if start == 0:
1558 | logging.warning(f"No {site_name} jobs found for: {params}")
1559 | else:
1560 | logging.info(f"No more {site_name} jobs found (page {start//50 + 1}).")
1561 | break # No more jobs, stop pagination
1562 |
1563 | for job_card in job_cards:
1564 | try:
1565 | job_data = parse_func(job_card) # Parse Individual job card.
1566 |
1567 | if not job_data['url']: # Skip if no URL
1568 | continue
1569 |
1570 | if resume_text:
1571 | try:
1572 | # Fetch the full job description
1573 | job_response = requests.get(job_data['url'], headers={'User-Agent': get_random_user_agent()},
1574 | timeout=config.REQUEST_TIMEOUT)
1575 | job_response.raise_for_status()
1576 | job_soup = BeautifulSoup(job_response.text, 'html.parser')
1577 | description_element = job_soup.find('div', id='jobDescriptionText') # Indeed
1578 | if not description_element: # for linkedin
1579 | description_element = job_soup.find('div', class_='description__text')
1580 | job_description = description_element.get_text(separator='\n', strip=True) if description_element else ""
1581 |
1582 | # --- Experience Level Extraction (from description) ---
1583 | experience_match = re.search(r'(\d+\+?)\s*(?:-|to)?\s*(\d*)\s*years?', job_description, re.IGNORECASE)
1584 | if experience_match:
1585 | if experience_match.group(2): # If range
1586 | job_data['experience'] = f"{experience_match.group(1)}-{experience_match.group(2)} years"
1587 | else: # Just single number
1588 | job_data['experience'] = f"{experience_match.group(1)} years"
1589 | else: # Check for keywords
1590 | exp_keywords = {
1591 | 'fresher': ['fresher', 'graduate', 'entry level', '0 years', 'no experience'],
1592 | 'entry-level': ['0-2 years', '1-3 years', 'entry level', 'junior'],
1593 | 'mid-level' : ['3-5 years','2-5 years','mid level','intermediate'],
1594 | 'senior' : ['5+ years','5-10 years', 'senior','expert', 'lead'],
1595 | 'executive': ['10+ years', 'executive', 'director', 'vp', 'c-level']
1596 | }
1597 | for level, keywords in exp_keywords.items():
1598 | for keyword in keywords:
1599 | if keyword.lower() in job_description.lower():
1600 | job_data['experience'] = level
1601 | break # Stop checking once a level is found
1602 | if job_data['experience'] != 'N/A':
1603 | break # Stop checking other levels
1604 |
1605 |
1606 |
1607 |
1608 | job_description = job_description[:2000] # Truncate!
1609 | resume_text_trunc = resume_text[:2000] # Truncate!
1610 |
1611 | relevance_prompt = (
1612 | f"Assess the relevance of the following job to the resume. "
1613 | f"Provide a JSON object with ONLY the following keys:\n"
1614 | f"'relevance': float (between 0.0 and 1.0, where 1.0 is perfectly relevant),\n"
1615 | f"'missing_skills': list of strings (skills in the job description but not in the resume, or an empty list if none),\n"
1616 | f"'justification': string (REQUIRED. Explain the relevance score, including factors like experience level mismatch, skill gaps, or industry differences.).\n\n"
1617 | f"Job Description:\n{job_description}\n\nResume:\n{resume_text_trunc}"
1618 | )
1619 | relevance_assessment = generate_gemini_response(relevance_prompt, response_format="json", model_name=config.JOB_RELEVANCE_MODEL)
1620 | if isinstance(relevance_assessment, dict):
1621 | job_data['relevance'] = relevance_assessment.get('relevance', 0.0) # Provide default
1622 | job_data['missing_skills'] = relevance_assessment.get('missing_skills', [])
1623 | job_data['justification'] = relevance_assessment.get('justification', "Relevance assessed.")
1624 | # Basic validation (optional, but good practice):
1625 | if not isinstance(job_data['relevance'], (int, float)):
1626 | logging.warning(f"Invalid relevance value: {job_data['relevance']}")
1627 | job_data['relevance'] = 0.0
1628 | if not isinstance(job_data['missing_skills'], list):
1629 | logging.warning(f"Invalid missing_skills: {job_data['missing_skills']}")
1630 | job_data['missing_skills'] = []
1631 | if not isinstance(job_data['justification'], str):
1632 | logging.warning(f"Invalid justification: {job_data['justification']}")
1633 | job_data['justification'] = "Error: Could not assess relevance properly."
1634 | elif isinstance(relevance_assessment, dict) and "error" in relevance_assessment:
1635 | # Handle the "Invalid JSON" case specifically
1636 | logging.warning(f"Invalid JSON from Gemini: {relevance_assessment['raw_text']}")
1637 | job_data['relevance'] = 0.0 # Set default values
1638 | job_data['missing_skills'] = []
1639 | job_data['justification'] = "Error: Could not assess relevance due to invalid JSON response."
1640 |
1641 | else: # Unexpected return from Gemini
1642 | logging.warning(f"Unexpected response from relevance assessment: {relevance_assessment}")
1643 | job_data['relevance'] = 0.0
1644 | job_data['missing_skills'] = []
1645 | job_data['justification'] = "Error: Could not assess relevance (unexpected response)."
1646 |
1647 | except requests.exceptions.RequestException as e:
1648 | logging.warning(f"Failed to fetch job description from {job_data['url']}: {e}")
1649 | job_data['justification'] = f"Error: Could not fetch job description ({type(e).__name__})."
1650 | except Exception as e:
1651 | logging.exception(f"Error during relevance assessment for {job_data['url']}: {e}")
1652 | job_data['relevance'] = 0.0
1653 | job_data['missing_skills'] = []
1654 | job_data['justification'] = "Error: Could not assess relevance (unexpected error)."
1655 |
1656 | search_results.append(job_data) # Append even with errors
1657 |
1658 | except Exception as e:
1659 | logging.warning(f"Error processing a {site_name} job card: {e}")
1660 | continue # skip to the next job card
1661 |
1662 | start += 50 # pagination
1663 | if start//50 +1 > MAX_PAGES:
1664 | logging.info(f"Reached max pages ({MAX_PAGES}) for {site_name}.")
1665 | break
1666 |
1667 | except requests.exceptions.HTTPError as e:
1668 | logging.error(f"{site_name} HTTP Error: {e}")
1669 | break # unrecoverable errors
1670 | except requests.exceptions.RequestException as e:
1671 | logging.error(f"{site_name} Request Exception: {e}")
1672 | break # network errors
1673 |
1674 | return search_results
1675 |
1676 |
1677 | @app.post("/api/scrape_jobs")
1678 | async def scrape_jobs_endpoint(job_title: Optional[str] = Form(""), job_location: str = Form(...), resume: UploadFile = File(None),
1679 | job_experience: Optional[str] = Form(None)): # New parameter
1680 | try:
1681 | # Removed the 'not job_title' check to allow it to be optional
1682 |
1683 | resume_text = None
1684 | if resume:
1685 | resume_content = await resume.read()
1686 | resume_text = extract_text_from_resume(resume_content)
1687 | if not resume_text:
1688 | raise HTTPException(status_code=400, detail="Could not extract text from resume.")
1689 |
1690 | all_job_results = []
1691 |
1692 | with concurrent.futures.ThreadPoolExecutor(max_workers=config.MAX_WORKERS) as executor:
1693 | # Submit both scraping tasks *concurrently*
1694 | linkedin_future = executor.submit(scrape_job_site, job_title, job_location, resume_text,
1695 | "https://www.linkedin.com/jobs/search", linkedin_params, parse_linkedin_job_card, "LinkedIn", job_experience) # Pass experience
1696 | indeed_future = executor.submit(scrape_job_site, job_title, job_location, resume_text,
1697 | "https://www.indeed.com/jobs", indeed_params, parse_indeed_job_card, "Indeed", job_experience) # Pass experience
1698 |
1699 | # Get results, handling exceptions gracefully. Don't stop if one fails.
1700 | try:
1701 | all_job_results.extend(linkedin_future.result())
1702 | except Exception as e:
1703 | logging.error(f"Error scraping LinkedIn: {e}") # Log, but don't stop
1704 | try:
1705 | all_job_results.extend(indeed_future.result())
1706 | except Exception as e:
1707 | logging.error(f"Error scraping Indeed: {e}") # Log, but don't stop
1708 |
1709 | # --- Filtering and Sorting ---
1710 | # Filter by experience first
1711 | if job_experience:
1712 | filtered_jobs = [job for job in all_job_results if job.get('experience', '').lower() == job_experience.lower()]
1713 | else:
1714 | filtered_jobs = all_job_results
1715 |
1716 |
1717 | # Then, sort by experience level, then by relevance WITHIN each experience level
1718 | experience_order = ['fresher', 'entry-level', 'mid-level', 'senior', 'executive', 'N/A']
1719 | def sort_key(job):
1720 | # Get experience level (default to 'N/A' if missing, put last)
1721 | exp = job.get('experience', 'N/A').lower()
1722 | if exp not in experience_order:
1723 | exp = 'N/A' # Normalize to 'N/A'
1724 |
1725 | return (experience_order.index(exp), -job.get('relevance', 0.0))
1726 |
1727 | filtered_jobs.sort(key=sort_key)
1728 |
1729 |
1730 |
1731 | if filtered_jobs:
1732 | return JSONResponse({'jobs': filtered_jobs, 'jobs_found': len(all_job_results)})
1733 | else:
1734 | # More specific message if no jobs *after* filtering
1735 | return JSONResponse({'jobs': [], 'jobs_found': len(all_job_results)}, status_code=200) # Return 200 OK even if no jobs are found after filtering
1736 |
1737 |
1738 |
1739 | except HTTPException as e:
1740 | raise e # Re-raise HTTP exceptions for FastAPI to handle
1741 | except Exception as e:
1742 | logging.error(f"Error in jobs scraping endpoint: {e}")
1743 | raise HTTPException(status_code=500, detail=str(e))
1744 |
1745 | # Image Analysis Tool
1746 | @app.post("/api/analyze_image")
1747 | async def analyze_image_endpoint(request: Request):
1748 | try:
1749 | data = await request.json()
1750 | image_data = data.get('image')
1751 | if not image_data:
1752 | return JSONResponse({"error": "No image provided"}, status_code=400)
1753 |
1754 | image_part = process_base64_image(image_data)
1755 | if not image_part:
1756 | return JSONResponse({"error": "Failed to process image"}, status_code=400)
1757 |
1758 | model = genai.GenerativeModel('gemini-2.0-flash')
1759 | # image = Image.open(io.BytesIO(image_part['data'])) # No longer needed
1760 | response = model.generate_content(["Describe this image in detail", image_part]) # Simple description prompt ,pass image part directly
1761 | response.resolve() # Ensure generation is complete
1762 |
1763 | return JSONResponse({"description": response.text})
1764 |
1765 | except Exception as e:
1766 | logging.exception("Error in image analysis")
1767 | return JSONResponse({"error": "Image analysis failed."}, status_code=500)
1768 |
1769 |
1770 | # Sentiment Analysis Tool
1771 | @app.post("/api/analyze_sentiment")
1772 | async def analyze_sentiment_endpoint(request: Request):
1773 | try:
1774 | data = await request.json()
1775 | text = data.get('text')
1776 | if not text:
1777 | return JSONResponse({"error": "No text provided"}, status_code=400)
1778 |
1779 | prompt = f"Analyze the sentiment of the following text and classify it as 'Positive', 'Negative', or 'Neutral'. Provide a brief justification:\n\n{text}"
1780 | sentiment_result = generate_gemini_response(prompt)
1781 |
1782 | return JSONResponse({"sentiment": sentiment_result})
1783 | except Exception as e:
1784 | logging.exception("Error in sentiment analysis")
1785 | return JSONResponse({"error": "Sentiment analysis failed."}, status_code=500)
1786 |
1787 | # Website Summarization Tool
1788 | @app.post("/api/summarize_website")
1789 | async def summarize_website_endpoint(request: Request):
1790 | try:
1791 | data = await request.json()
1792 | url = data.get('url')
1793 | if not url:
1794 | return JSONResponse({"error": "No URL provided"}, status_code=400)
1795 |
1796 | content_snippets, _, _ = fetch_page_content(url, snippet_length=config.SNIPPET_LENGTH)
1797 | if not content_snippets:
1798 | return JSONResponse({"error": "Could not fetch website content"}, status_code=400)
1799 | combined_content = "\n\n".join(content_snippets)
1800 | prompt = f"Summarize the following webpage content concisely:\n\n{combined_content}"
1801 | summary = generate_gemini_response(prompt)
1802 | return JSONResponse({"summary": summary})
1803 | except Exception as e:
1804 | logging.exception("Error in website summarization")
1805 | return JSONResponse({"error": "Website summarization failed."}, status_code=500)
1806 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | fastapi==0.110.0
2 | uvicorn==0.29.0
3 | python-dotenv==1.0.1
4 | Pillow==10.2.0
5 | requests==2.31.0
6 | beautifulsoup4==4.12.3
7 | soupsieve==2.5
8 | google-generativeai==0.4.1
9 | reportlab==4.1.0
10 | python-multipart==0.0.9
11 | tenacity==8.2.3
12 | urllib3==2.2.1
13 | brotli==1.1.0
14 | pdfminer.six==20231228
15 | docx2txt==0.8
16 | chardet==5.2.0
17 | Jinja2==3.1.3
18 | aiofiles==23.2.1
19 |
--------------------------------------------------------------------------------
/templates/index.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | KV AI Assistant | Next-Gen Research Tool
8 |
1359 |
1360 |
1385 |
1386 |
1387 |
1390 |
1391 |
1420 |
1421 |
1422 |
1423 |
1424 |
1425 |
1434 |
1435 |
1436 |
1437 |
1455 |
1683 |
1687 |
1688 |
×
1689 |
Settings
1690 |
1691 | Custom Instruction:
1692 |
1693 |
1694 |
1695 | AI Model:
1696 |
1697 | gemini-2.0-flash
1698 | gemini-2.0-flash-thinking-exp-01-21
1699 |
1700 |
1701 |
Save Settings
1702 |
1703 |
1707 |
1708 |
1709 |
Generating Report... 🚀🧠💡
1710 |
1711 |
1715 |
1716 |
1717 |
1718 |
1719 |
1720 |
2861 |
2862 |
2863 |
--------------------------------------------------------------------------------
7 | AI-Powered Deep Research & Job Search Companion
8 | 
17 | 
20 | 
23 | 
26 | 
30 | 
33 | 
36 | 
43 | 
46 | 
54 | 
95 | 
102 | 
112 | 
| **Advanced AI Integration** 
| **Intelligent Web Scraping** 
| **Multi-Search Engine Support** 
| **Iterative Research** 
| **Resume Analysis** 
| **Job Match Algorithm** 
| **Skill Gap Detection** 
| **Professional Reports** 
| **UI Customization** 
135 | 
164 | 
201 | 
241 | 
246 | 
258 | 
267 | 
276 | 
287 | 
327 | 
330 | 
336 | 
353 | 
356 | 
369 |