└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # 55 Fundamental SQL in Machine Learning Interview Questions in 2025. 2 | 3 |
4 |

5 | 6 | machine-learning-and-data-science 7 | 8 |

9 | 10 | #### You can also find all 55 answers here 👉 [Devinterview.io - SQL](https://devinterview.io/questions/machine-learning-and-data-science/sql-ml-interview-questions) 11 | 12 |
13 | 14 | ## 1. What are the different types of _JOIN_ operations in SQL? 15 | 16 | **INNER JOIN**, **LEFT JOIN**, **RIGHT JOIN**, and **FULL JOIN** are different SQL join types, each with its distinct characteristics. 17 | 18 | ### Join Types at a Glance 19 | 20 | - **INNER JOIN**: Returns matching records from both tables. 21 | - **LEFT JOIN**: Retrieves all records from the left table and matching ones from the right. 22 | - **RIGHT JOIN**: Gets all records from the right table and matching ones from the left. 23 | - **FULL JOIN**: Includes all records when there is a match in either of the tables. 24 | 25 | ### Visual Representation 26 | 27 | ![SQL Joins](https://firebasestorage.googleapis.com/v0/b/dev-stack-app.appspot.com/o/sql%2Fsql-joins-min.png?alt=media&token=143a48c2-6ebe-4b27-9012-adf8a7ba8948) 28 | 29 | ### Code Example: SQL Joins 30 | 31 | Here is the SQL code: 32 | 33 | ```sql 34 | -- CREATE TABLES 35 | CREATE TABLE employees ( 36 | id INT PRIMARY KEY, 37 | name VARCHAR(100), 38 | department_id INT 39 | ); 40 | 41 | CREATE TABLE departments ( 42 | id INT PRIMARY KEY, 43 | name VARCHAR(100) 44 | ); 45 | 46 | -- INSERT SOME DATA 47 | INSERT INTO employees (id, name, department_id) VALUES 48 | (1, 'John', 1), 49 | (2, 'Alex', 2), 50 | (3, 'Lisa', 1), 51 | (4, 'Mia', 1); 52 | 53 | INSERT INTO departments (id, name) VALUES 54 | (1, 'HR'), 55 | (2, 'Finance'), 56 | (3, 'IT'); 57 | 58 | -- INNER JOIN 59 | SELECT employees.name, departments.name as department 60 | FROM employees 61 | INNER JOIN departments ON employees.department_id = departments.id; 62 | 63 | -- LEFT JOIN 64 | SELECT employees.name, departments.name as department 65 | FROM employees 66 | LEFT JOIN departments ON employees.department_id = departments.id; 67 | 68 | -- RIGHT JOIN 69 | SELECT employees.name, departments.name as department 70 | FROM employees 71 | RIGHT JOIN departments ON employees.department_id = departments.id; 72 | 73 | -- FULL JOIN 74 | SELECT employees.name, departments.name as department 75 | FROM employees 76 | FULL JOIN departments ON employees.department_id = departments.id; 77 | ``` 78 |
79 | 80 | ## 2. Explain the difference between _WHERE_ and _HAVING_ clauses. 81 | 82 | The **WHERE** and **HAVING** clauses are both used in SQL, but they serve distinct purposes, 83 | 84 | ### Key Distinctions 85 | 86 | - **WHERE**: Filters records based on conditions for individual rows. 87 | - **HAVING**: Filters the results of aggregate functions, such as `COUNT`, `SUM`, `AVG`, and others, for groups of rows defined by the GROUP BY clause. 88 | 89 | ### Code Example: Basic Usage 90 | 91 | Here is the SQL code: 92 | 93 | ```sql 94 | -- WHERE: Simple data filtering 95 | SELECT product_type, AVG(product_price) AS avg_price 96 | FROM products 97 | WHERE product_price > 100 98 | GROUP BY product_type; 99 | 100 | -- HAVING: Filtered results post-aggregation 101 | SELECT order_id, SUM(order_total) AS total_amount 102 | FROM orders 103 | GROUP BY order_id 104 | HAVING SUM(order_total) > 1000; 105 | ``` 106 |
107 | 108 | ## 3. How would you write a SQL query to select _distinct values_ from a column? 109 | 110 | When you have duplicates in a column, you can use the `DISTINCT` clause to **retrieved unique values**. 111 | 112 | For instance: 113 | 114 | ```sql 115 | SELECT UNIQUE_COLUMN 116 | FROM YOUR_TABLE 117 | ORDER BY UNIQUE_COLUMN; 118 | ``` 119 | 120 | In this query, replace `UNIQUE_COLUMN` with the column name from which you want distinct values, and `YOUR_TABLE` with your specific table name. 121 | 122 | ### Practical Example: Using `DISTINCT` 123 | 124 | Let's say you have a table `students_info` with a `grade` column indicating the grade level of students. You want to find all unique grade levels. 125 | 126 | Here's the corresponding SQL query: 127 | 128 | ```sql 129 | SELECT DISTINCT grade 130 | FROM students_info 131 | ORDER BY grade; 132 | ``` 133 | 134 | Executing this query would return a list of unique grade levels in ascending order. 135 | 136 | ### When to Use `DISTINCT` 137 | 138 | - **Unique Records**: When you only want to see and count unique values within a specific column or set of columns. 139 | 140 | ```sql 141 | SELECT COUNT(DISTINCT column_name) FROM table_name; 142 | ``` 143 | 144 | - **Criteria Comparison**: Using `IN` and `NOT IN` can involve multiple selections; `DISTINCT` ensures the return of unique results. 145 | 146 | - **Insight into Overlapping Data**: Useful for data analysis tasks where you want to identify shared information between rows. 147 | 148 | - **Subset Selection**: When you are working with large tables and want to zero in on unique records within a specific range, such as for pagination. 149 |
150 | 151 | ## 4. What does _GROUP BY_ do in a SQL query? 152 | 153 | **GROUP BY** is a powerful clause in Structured Query Language (SQL) that allows for data summarization and grouping. 154 | 155 | ### Key Functions 156 | 157 | - **Aggregation**: Performs tasks like sum, count, average, among others within subsets (groups). 158 | - **Grouping**: Identifies data subsets based on predetermined commonalities. 159 | - **Filtering**: Enables filtering both pre- and post-aggregation. 160 | 161 | ### When to Use GROUP BY 162 | 163 | - **Summarizing Data**: For instance, calculating a 'Total Sales' from individual transactions. 164 | - **Categorization**: Such as counting the number of 'Customers' or 'Products' within specific groups (like regions or categories). 165 | - **Data Integrity Checks**: To identify potential duplicates or check for data consistency. 166 | - **Combining with Aggregate Functions**: Pairing with functions such as `COUNT`, `SUM`, `AVG`, `MAX`, and `MIN` for more sophisticated calculations. 167 | 168 | ### The Mechanism Behind GROUP BY 169 | 170 | - **Division into Groups**: The system sorts the result set by the specified columns in the `GROUP BY` clause and groups rows that have the same group column values. This step creates a distinct group for each unique combination of 'group by' columns. 171 | - **Aggregation within Groups**: The system then applies the aggregation function (or functions) to each group independently, summarizing the data within each group. 172 | - **Result Generation**: After the groups are processed, the final result set is produced. 173 | 174 | ### Code Example: GROUP BY in Action 175 | 176 | Here is the SQL code: 177 | 178 | ```sql 179 | SELECT SUM(Revenue), Region 180 | FROM Sales 181 | GROUP BY Region; 182 | ``` 183 | 184 | In this example, the `Sales` table is grouped by `Region`, and the sum of `Revenue` is calculated for each group. 185 | 186 | ### Potential Challenges with GROUP BY 187 | 188 | - **Single-Column Limitation**: Without employing **additional techniques**, such as using subqueries or rollup or cube extensions, data can be grouped on only one column. 189 | - **Data Types Consideration**: When grouping by certain data types, such as dates or floating points, results may not be as expected due to inherent characteristics of those types. 190 | 191 | ### Advanced Techniques with GROUP BY 192 | 193 | - **Rollup and Cube**: Extensions providing multi-level summaries. 194 | - ROLLUP: Computes higher-level subtotals, moving from right to left in the grouping columns. 195 | - CUBE: Computes all possible subtotals. 196 | 197 | - **Grouping Sets**: Defines multiple groups in one query, e.g., grouping by year, month, and day in a date column. 198 |
199 | 200 | ## 5. How can you _aggregate data_ in SQL (e.g., _COUNT_, _AVG_, _SUM_, _MAX_, _MIN_)? 201 | 202 | Aggregating data in SQL is **essential** for making sense of large data sets. Common aggregate functions include `COUNT`, `SUM`, `AVG` (mean), `MIN`, and `MAX`. 203 | 204 | ### Syntax 205 | 206 | Here is an example of the SQL code: 207 | 208 | ```sql 209 | SELECT AGG_FUNCTION(column_name) 210 | FROM table_name 211 | GROUP BY column_name; 212 | ``` 213 | 214 | - `AGG_FUNCTION`: Replace with any of the aggregate operations. 215 | - `column_name`: The specific column to which the function will be applied. 216 | 217 | If you don't use a `GROUP BY` clause, the query will apply the aggregate function to the **entire result set**. 218 | 219 | ### Examples 220 | 221 | #### Without `GROUP BY` 222 | 223 | ```sql 224 | SELECT COUNT(id) AS num_orders 225 | FROM orders; 226 | ``` 227 | 228 | #### With `GROUP BY` 229 | 230 | ```sql 231 | SELECT customer_id, COUNT(id) AS num_orders 232 | FROM orders 233 | GROUP BY customer_id; 234 | ``` 235 | 236 | In this example, the `COUNT` aggregates the number of orders for each unique customer ID. 237 | 238 | ### Considerations 239 | 240 | - **Null Values**: Most aggregates ignore nulls, but you can use `COUNT(*)` to include them. 241 | - **Multiple Functions**: It's possible to include multiple aggregate functions in one query. 242 | - **Data Type Compatibility**: Ensure that the chosen aggregate function is compatible with the data type of the selected column. For instance, you can't calculate the mean of a text field. 243 | 244 | ### Code Example: Aggregating Data in SQL 245 | 246 | Here is the SQL code: 247 | 248 | ```sql 249 | CREATE TABLE orders (id INT, customer_id INT, total_amount DECIMAL(10, 2)); 250 | 251 | INSERT INTO orders (id, customer_id, total_amount) 252 | VALUES 253 | (1, 101, 25.00), 254 | (2, 102, 35.50), 255 | (3, 101, 42.25), 256 | (4, 103, 20.75), 257 | (5, 102, 60.00); 258 | 259 | -- Total Number of Orders 260 | SELECT COUNT(id) AS num_orders 261 | FROM orders; 262 | 263 | -- Number of Orders per Customer 264 | SELECT customer_id, COUNT(id) AS num_orders 265 | FROM orders 266 | GROUP BY customer_id; 267 | 268 | -- Total Sales 269 | SELECT SUM(total_amount) AS total_sales 270 | FROM orders; 271 | 272 | -- Average Order Value 273 | SELECT AVG(total_amount) AS avg_order_value 274 | FROM orders; 275 | 276 | -- Highest Ordered Value 277 | SELECT MAX(total_amount) AS max_order_value 278 | FROM orders; 279 | 280 | -- Lowest Ordered Value 281 | SELECT MIN(total_amount) AS min_order_value 282 | FROM orders; 283 | ``` 284 |
285 | 286 | ## 6. Describe a _subquery_ and its typical use case. 287 | 288 | A **subquery** consists of a complete SQL statement nested within another query. It's often used for complex filtering, calculations, and data retrieval. 289 | 290 | Subqueries are broadly classified into two types: 291 | 292 | - **Correlated**: They depend on the outer query's results. Each time the outer query iterates, the subquery is re-evaluated with the updated outer result. It can be less efficient as it often involves repeated subquery evaluation. 293 | - **Uncorrelated**: These are self-contained and don't rely on the outer query. They are typically executed only once and their result is used throughout the outer query. 294 | 295 | ### Common Use Cases 296 | 297 | - **Filtering with Aggregates**: Subqueries can be used in combination with aggregate functions to filter group-level results based on specific criteria. For instance, you can retrieve departments with an average salary above a certain threshold. 298 | 299 | - **Multi-Criteria Filtering**: Subqueries are often handy when traditional `WHERE`, `IN`, or `EXISTS` clauses can't accommodate complex, multi-criteria filters. 300 | 301 | - **Data Integrity Checks**: Subqueries can help identify inconsistent data by comparing values to related tables. 302 | 303 | - **Hierarchical Data Queries**: With the advent of Common Table Expressions (CTEs) and recursive queries in modern SQL standards, a direct use of subqueries for hierarchical data searches is now uncommon - CTEs are the preferred means of such queries. 304 | 305 | - **Data Retention**: Subqueries can be used to identify specific records to be deleted or retained based on certain conditions. 306 | 307 | ### Common Use Cases 308 | 309 | #### Multi-Criteria Filtering 310 | - **Task**: Return all customers from a specific city who have placed orders within the last month. 311 | - **Code**: 312 | ```sql 313 | SELECT * FROM Customers 314 | WHERE City = 'London' 315 | AND CustomerID IN (SELECT CustomerID FROM Orders WHERE OrderDate > DATEADD(month, -1, GETDATE())) 316 | ``` 317 | 318 | #### Data Integrity Checks 319 | - **Task**: Retrieve customers with inconsistent states in the Customer and Order tables. 320 | - **Code**: 321 | ```sql 322 | SELECT * FROM Customer 323 | WHERE State NOT IN (SELECT DISTINCT State FROM Orders) 324 | ``` 325 | 326 | #### Data Retention 327 | - **Task**: Archive orders older than three years. 328 | - **Code**: 329 | ```sql 330 | DELETE FROM Orders 331 | WHERE OrderID IN (SELECT OrderID FROM Orders WHERE OrderDate < DATEADD(year, -3, GETDATE())) 332 | ``` 333 |
334 | 335 | ## 7. Can you explain the use of _indexes_ in databases and how they relate to Machine Learning? 336 | 337 | Database **indexes** enable systems to retrieve data more efficiently by offering a faster look-up mechanism. This optimization technique is directly pertinent to **Machine Learning**. 338 | 339 | ### Indexes in Databases 340 | 341 | Databases traditionally use **B-Tree** indexes, but are equipped with several index types, catering to varying data environments and query patterns. 342 | 343 | - **B-Tree (Balanced Tree)**: Offers balanced search capabilities, ensuring nodes are at least half-full. 344 | - **Hash**: Employed for point queries, dedicating fixed-size chunks to keys. 345 | - **Bitmap**: Particularly suitable for low-cardinality data where keys are better represented as bits. 346 | - **Text Search**: Facilitates efficient text matching. 347 | 348 | ### Key Concepts of B-Trees 349 | 350 | - **Node Structure**: Contains keys and pointers. Leaf nodes harbor actual data, enabling direct access. 351 | - **Data Positioning**: Organizes data in a sorted, multi-level structure to expedite lookups. 352 | - **Range Queries**: Suited for both singular and **range-based** queries. 353 | 354 | ### Machine Learning Query Scenarios 355 | 356 | - **Similarity Look-Up**: A dataset with user preferences can be indexed to expedite locating individuals with matching profiles, advantageous in applications such as recommendation systems. 357 | - **Range-Based Searches**: For datasets containing time-specific information, like a sales record, B-Trees excel in furnishing time-ordered data within designated intervals. 358 | 359 | ### Code Example: Implementing B-Trees for Range Queries 360 | 361 | Here is the Python code: 362 | 363 | ```python 364 | class Node: 365 | def __init__(self, keys=[], children=[]): 366 | self.keys = keys 367 | self.children = children 368 | 369 | # Perform range query on tree rooted at 'node' 370 | def range_query(node, start, end): 371 | # Base case: Leaf node 372 | if not node.children: 373 | return [key for key in node.keys if start <= key <= end] 374 | # Locate appropriate child node 375 | index = next((i for i, key in enumerate(node.keys) if key >= start), len(node.children) - 1) 376 | return range_query(node.children[index], start, end) 377 | ``` 378 |
379 | 380 | ## 8. How would you _optimize_ a SQL query that seems to be running slowly? 381 | 382 | When a SQL query is sluggish, various optimization techniques can be employed to enhance its speed. Let's focus on the **logical and physical** design aspects of the **database structure and the query itself**. 383 | 384 | ### Key Optimization Techniques 385 | 386 | #### 1. Query Optimization 387 | 388 | - **Simplify Complex Queries**: Break the query into smaller parts for better readability and performance. Use common table expressions or derived tables to modularize SQL logic. Alternatively, you can use temporary tables. 389 | - **Limit Result Set**: Use `TOP`, `LIMIT`, or `ROWNUM`/`ROWID` to restrict the number of records returned. 390 | - **Reduce JOIN Complexity**: Replace multiple JOINs with fewer, multi-table JOINs and **explicit JOIN** notation. 391 | 392 | 393 | #### 2. Indexing 394 | 395 | - **Proper Indexing**: Select suitable columns for indexing to speed up data retrieval. Use composite indexes for frequent combinations of columns in WHERE or JOIN conditions. 396 | - **Avoid Over-Indexing**: Numerous indexes can slow down write operations and data modifications. Strike a balance. 397 | 398 | #### 3. Schema and Data Design 399 | 400 | - **Normalization**: Ensure the database is in an optimal normal form, which can reduce redundancy, maintain data integrity, and minimize disk space. 401 | - **Data Types**: Use appropriate data types for columns to conserve space and support efficient data operations. 402 | 403 | #### 4. Under The Hood: The Query Plan 404 | 405 | - **Analyze Query Execution Plan**: Look at the query execution plan, generated by the SQL query optimizer, to identify bottlenecks and improve them. Many RDBMS provide commands and tools to access the query execution plan. 406 | 407 | #### 5. More Ideas from SQL Performance Tuning 408 | 409 | - **Test Under Load**: Simulate the production environment and monitor query response times to identify performance issues. 410 | - **Limit Data Reallocations in tempdb**: Data reallocation operations such as INSERT INTO.. SELECT FROM can be resource-intensive on tempdb. 411 | - **Partition Data**: Split large tables into smaller, more manageable segments to speed up query performance. 412 | 413 | #### Tools and Techniques for Query Analysis 414 | 415 | - **Profiling Tools**: Use graphical query builders and visual execution plan tools provided by much RDBMS to examine data flow and performance. 416 | - **Query Plan Viewer**: Databases such as SQL Server have a graphical representation of query execution plans. 417 | - **Index Analysis**: Some databases, like MySQL and SQL Server, provide tools to check the efficiency of indexes and suggest index changes through Index Tuning Wizard and Optimizer Index Advisor. 418 | 419 | ### Practical Steps for Query Optimization 420 | 421 | 1. **Determine the Performance Problem**: Understand what specific aspect of the query is underperforming. 422 | 2. **Profile Your Query**: Use **EXPLAIN** (or its equivalent on other databases) to see the query plan and identify potential bottlenecks. 423 | 3. **Analyze Query Execution Time**: Use database tools to analyze real execution time and get insights into I/O, CPU, and memory usage. 424 | 4. **Identify the Bottleneck**: Focus on the slowest part of the query or most resource-intensive part, for example, I/O or CPU. 425 | 5. **Tune That Portion**: Make changes to the query or the table structure or consider using versioned views or indexed views. Take time to understand the reason it is being slow and focus your efforts on correcting that. 426 |
427 | 428 | ## 9. How do you handle _missing values_ in a SQL dataset? 429 | 430 | Handling **missing values** is crucial for accurate analysis in SQL. Let's look at the various techniques for managing them. 431 | 432 | ### Removing Records 433 | 434 | One of the simplest ways to deal with missing values is to discard rows with NULLs. 435 | 436 | #### Examples 437 | 438 | Here's a SQL query that deletes rows containing NULL in the column `age`: 439 | 440 | ```sql 441 | DELETE FROM students 442 | WHERE age IS NULL; 443 | ``` 444 | 445 | ### Direct Replacement 446 | 447 | Replace missing values with specific defaults using `COALESCE` or `CASE` statements. 448 | 449 | #### Examples 450 | 451 | If `grade` can have NULL values and you want to treat them as "ungraded": 452 | 453 | ```sql 454 | SELECT student_name, COALESCE(grade, 'Ungraded') AS actual_grade 455 | FROM student_grades; 456 | ``` 457 | 458 | An example using `CASE`: 459 | 460 | ```sql 461 | SELECT book_title, 462 | CASE WHEN publication_year IS NULL THEN 'Unknown' 463 | ELSE publication_year 464 | END AS year 465 | FROM books; 466 | ``` 467 | 468 | ### Using Aggregates 469 | 470 | Apply SQL aggregate functions to compute statistics without explicitly removing NULLs. For example, `COUNT` ignores NULLs on a column. 471 | 472 | ```sql 473 | SELECT department, COUNT(*) AS total_students 474 | FROM students 475 | GROUP BY department; 476 | ``` 477 | 478 | ### Flexible Joins 479 | 480 | Depending on your specific situation, you might want to include or exclude missing values when joining tables. 481 | 482 | #### Examples 483 | 484 | Using `LEFT JOIN`: 485 | 486 | ```sql 487 | SELECT s.student_id, s.name, e.enrollment_date 488 | FROM students s 489 | LEFT JOIN enrollments e ON s.student_id = e.student_id; 490 | ``` 491 | 492 | Using `INNER JOIN`: 493 | 494 | ```sql 495 | SELECT s.student_id, s.name, e.enrollment_date 496 | FROM students s 497 | INNER JOIN enrollments e ON s.student_id = e.student_id 498 | WHERE e.enrollment_date IS NOT NULL; 499 | ``` 500 | 501 | ### Handle Missing Date Fields 502 | 503 | If **Date** fields are missing, the appropriate strategy would depend on the context. 504 | 505 | 1. **Replace with Defaults**: For missing dates, you can use a default, such as the current date, or another specific date. 506 | 507 | 2. **Remove or Flag**: Another option, based on context, is to either delete the record with the missing date or flag it for later review. 508 | 509 | #### Examples 510 | 511 | For replacing with the current date: 512 | 513 | ```sql 514 | SELECT action_id, COALESCE(action_date, CURRENT_DATE) AS actual_date 515 | FROM actions; 516 | ``` 517 | 518 | 3. **Impute from Adjacent Data**: In time series data, it's often useful to fill in missing dates with the nearest available data point to maintain a continuous date sequence. This can be done using window functions. 519 | 520 | #### Examples 521 | 522 | Using `LAG()` to fill missing dates with the previous non-missing date: 523 | 524 | ```sql 525 | SELECT action_id, 526 | COALESCE(action_date, LAG(action_date) OVER (ORDER BY action_id)) AS imputed_date 527 | FROM actions; 528 | ``` 529 | 530 | ### Advanced Techniques 531 | 532 | 1. **Using Temp Tables**: You can create a temporary table, excluding rows with NULLs, and then work with this cleaner dataset. 533 | 534 | Example: 535 | 536 | ```sql 537 | CREATE TEMPORARY TABLE clean_students AS 538 | SELECT * 539 | FROM students 540 | WHERE age IS NOT NULL; 541 | 542 | -- Perform further tasks using "clean_students" table 543 | ``` 544 | 545 | 2. **Machine Learning Methods**: Advanced SQL engines supporting ML functionalities might offer methods like imputation based on models. 546 | 547 | 3. **Dynamic Imputation**: For scenarios involving complex rules or sequences, you might consider using stored procedures to dynamically impute missing values. 548 |
549 | 550 | ## 10. Write a SQL query that _joins_ two tables and retrieves only the rows with matching keys. 551 | 552 | ### Problem Statement 553 | 554 | The task is to perform a **SQL join** operation between two tables and retrieve the rows where the keys match. 555 | 556 | ### Solution 557 | 558 | To accomplish this task, use the following SQL query. 559 | 560 | #### MySQL 561 | 562 | ```sql 563 | SELECT * 564 | FROM table1 565 | INNER JOIN table2 ON table1.key = table2.key; 566 | ``` 567 | 568 | #### PostgreSQL 569 | 570 | ```sql 571 | SELECT * 572 | FROM table1 573 | INNER JOIN table2 USING (key); 574 | ``` 575 | 576 | #### Oracle 577 | 578 | ```sql 579 | SELECT * 580 | FROM table1 581 | JOIN table2 ON table1.key = table2.key; 582 | ``` 583 | 584 | #### SQL Server 585 | 586 | ```sql 587 | SELECT * 588 | FROM table1 589 | JOIN table2 ON table1.key = table2.key; 590 | ``` 591 | 592 | ### Key Points 593 | 594 | - **`INNER JOIN`**: Retrieves the matching rows from both tables based on the specified condition. 595 | - **`ON`, `USING`**: Specifies the column(s) used for joining. 596 | - **`SELECT`**: You can specify individual columns instead of `*` based on requirement. 597 | - **Table Aliases**: When dealing with long table names, aliases (e.g., `t1`, `t2`) provide a more concise syntax. 598 |
599 | 600 | ## 11. How would you _merge_ multiple result sets in SQL without duplicates? 601 | 602 | When you need to **combine** the result sets of multiple SELECT queries without **duplicates**, use the **UNION** set operator. If you want to include duplicates, you can use **UNION ALL**. 603 | 604 | Here is a visual representation of how these set operations work: 605 | 606 | ![Union vs Union All](https://firebasestorage.googleapis.com/v0/b/dev-stack-app.appspot.com/o/sql%2Funion-and-union-all-in-sql%20(1).jpg?alt=media&token=661a27b1-acab-49f0-8456-274315349d98) 607 | 608 | ### Code Example: Using UNION 609 | 610 | Here is some SQL code: 611 | 612 | ```sql 613 | SELECT employee_id 614 | FROM full_time_employees 615 | 616 | UNION 617 | 618 | SELECT intern_id 619 | FROM interns; 620 | ``` 621 | 622 | This code retrieves a combined list of unique employee IDs from both `full_time_employees` and `interns` tables. 623 |
624 | 625 | ## 12. Create a SQL query to _pivot_ a table transforming rows into columns. 626 | 627 | ### Problem Statement 628 | 629 | "Pivoting" a table in SQL is the process of **reorganizing** and **transforming** row data into columnar data, commonly used for reporting or data analysis. 630 | 631 | ### Solution 632 | 633 | Two methods for pivoting data in SQL are: 634 | 635 | 1. **Static Pivot**: When the distinct values of the pivoted column are known in advance. 636 | 2. **Dynamic Pivot**: When the distinct values are not known in advance and need to be determined at runtime. 637 | 638 | #### Key Considerations 639 | 640 | - **Pivot Column Values**: Aware vs. Unaware of distinct values. 641 | - **Performance Impact**: Dynamic pivoting often involves complex operations at runtime. 642 | - **SQL Compatibility**: Dynamic pivoting can be limited in certain SQL dialects. 643 | 644 | Here is an example table named `salesdata`: 645 | 646 | | Date | Product | Quantity | Amount | 647 | |-----------|-----------|----------|--------| 648 | | 1/1/2020 | Apples | 10 | 50 | 649 | | 1/1/2020 | Oranges | 8 | 40 | 650 | | 2/1/2020 | Apples | 12 | 60 | 651 | | 2/1/2020 | Oranges | 15 | 75 | 652 | 653 | #### Static Pivot 654 | 655 | The `PIVOT` keyword is used in SQL Server, and `crosstab()` is used in PostgreSQL. 656 | 657 | #### Implementing Static Pivot 658 | 659 | **PostgreSQL**: 660 | 661 | ```sql 662 | SELECT * 663 | FROM crosstab( 664 | 'SELECT date, amount, product FROM salesdata ORDER BY 1,3', 665 | 'SELECT DISTINCT product FROM salesdata ORDER BY 1' 666 | ) AS ct ("Date" date, "Apples" int, "Oranges" int); 667 | ``` 668 | 669 | **SQL Server**: 670 | 671 | ```sql 672 | SELECT * 673 | FROM (SELECT Date, Product, Amount 674 | FROM salesdata) AS SourceTable 675 | PIVOT (SUM(Amount) FOR Product IN ([Apples], [Oranges])) AS PivotTable; 676 | ``` 677 | 678 | #### Dynamic Pivot 679 | 680 | For **SQL Server**, a stored procedure is necessary, as it dynamically constructs the query based on the distinct values. 681 | 682 | #### Implementing Dynamic Pivot 683 | 684 | **SQL Server**: 685 | 686 | - Create a stored procedure: 687 | 688 | ```sql 689 | CREATE PROCEDURE dynamicPivot 690 | AS 691 | BEGIN 692 | DECLARE @cols AS NVARCHAR(MAX), @query AS NVARCHAR(MAX); 693 | SELECT @cols = STUFF((SELECT DISTINCT ',' + QUOTENAME(Product) FROM salesdata FOR XML PATH('')), 1, 1, ''); 694 | SET @query = 'SELECT Date, ' + @cols + ' FROM (SELECT Date, Product, Amount FROM salesdata) AS SourceTable PIVOT (SUM(Amount) FOR Product IN (' + @cols + ' )) AS PivotTable;'; 695 | EXEC sp_executesql @query; 696 | END; 697 | ``` 698 | 699 | - Execute the stored procedure: 700 | 701 | ```sql 702 | EXEC dynamicPivot; 703 | ``` 704 |
705 | 706 | ## 13. Explain the importance of _data normalization_ in SQL and how it affects Machine Learning models. 707 | 708 | **Data normalization** is a crucial foundational step in preparing datasets for efficient storage and improved analysis. It is related to the **First Normal Form (1NF)** in relational databases and is essential for maintaining data integrity. 709 | 710 | ### Why is Data Normalization Important? 711 | 712 | - **Data Consistency**: It avoids redundancy and the potential for update anomalies. With normalized data, updates are made in a single place, ensuring consistency throughout the database. 713 | - **Data Integrity**: Foreign key constraints can be applied effectively only when data is normalized. 714 | - **Query Performance**: Normalized tables are often smaller, leading to better performance. 715 | 716 | ### Implications for Machine Learning 717 | 718 | - **Feature Engineering**: Normalized data ensures that feature scaling is consistent, which is often a prerequisite for machine learning algorithms like $k$-means clustering and algorithms that require gradient descent. If features are not normalized, certain features might have undue importance during model training. 719 | - **Ease of Integration**: Normalized data is easier to incorporate into machine learning pipelines. Many machine learning libraries assume, and, in some cases, require normalized data. 720 | - **Reduction of Overfitting**: Normalized data can help with overfitting issues in certain algorithms. If different features span different ranges, the model may give undue importance to the one with the larger scale. 721 | - **Enhanced Model Interpretability**: Normalized data can give more intuitive interpretations of coefficients, especially in linear models. 722 | 723 | ### Code Example: Normalizing Data in SQL 724 | 725 | Here is the SQL code: 726 | 727 | ```sql 728 | -- Create tables in First Normal Form (1NF) 729 | CREATE TABLE Driver ( 730 | DriverID int PRIMARY KEY, 731 | Name varchar(255), 732 | Age int 733 | ); 734 | 735 | CREATE TABLE Car ( 736 | CarID int PRIMARY KEY, 737 | Model varchar(255), 738 | Make varchar(255), 739 | Year int, 740 | DriverID int, 741 | FOREIGN KEY (DriverID) REFERENCES Driver(DriverID) 742 | ); 743 | 744 | -- Normalization to 3NF 745 | CREATE TABLE Driver ( 746 | DriverID int PRIMARY KEY, 747 | Name varchar(255), 748 | Age int 749 | ); 750 | 751 | CREATE TABLE Car ( 752 | CarID int PRIMARY KEY, 753 | Model varchar(255), 754 | Make varchar(255), 755 | Year int, 756 | DriverID int, 757 | FOREIGN KEY (DriverID) REFERENCES Driver(DriverID) 758 | ); 759 | ``` 760 |
761 | 762 | ## 14. How can you extract _time-based features_ from a SQL _datetime_ field for use in a Machine Learning model? 763 | 764 | Extracting **time-based features** from a SQL `datetime` field is essential for time series analysis. These features can be used to predict future events, study patterns, and make data-driven decisions. 765 | 766 | ### Time-Based Features: 767 | 768 | 1. **Year**: Extract the year using the SQL function `EXTRACT`. 769 | 2. **Month**: Use `EXTRACT` to retrieve the month. 770 | 3. **Day**: Similar to month and year, employ `EXTRACT` for the day. 771 | 4. **Day of Week**: Utilize `EXTRACT` with the `DOW` or `DAYOFWEEK` options. 772 | 5. **Weekend**: A binary feature indicating whether the day falls on a weekend. 773 | 774 | #### Example: SQL Queries for Time-Based Features 775 | 776 | Assuming a `sales` table with a `transaction_date` column, here are the SQL queries: 777 | 778 | ```sql 779 | -- Year 780 | SELECT EXTRACT(YEAR FROM transaction_date) AS transaction_year FROM sales; 781 | 782 | -- Month 783 | SELECT EXTRACT(MONTH FROM transaction_date) AS transaction_month FROM sales; 784 | 785 | -- Day 786 | SELECT EXTRACT(DAY FROM transaction_date) AS transaction_day FROM sales; 787 | 788 | -- Day of Week 789 | SELECT EXTRACT(DOW FROM transaction_date) AS transaction_dayofweek FROM sales; 790 | 791 | -- Weekend 792 | SELECT CASE WHEN EXTRACT(DOW FROM transaction_date) IN (0, 6) THEN 1 ELSE 0 END AS is_weekend FROM sales; 793 | ``` 794 | 795 | ### Time Period Features: 796 | 797 | 1. **Time of Day**: Use `EXTRACT` with `HOUR` to split the day into different segments. 798 | 2. **Time of Day (Cyclical)**: Normalize the time into a 24-hour cycle using trigonometric functions like `SIN` and `COS`, which can better capture patterns. 799 | 800 | #### Example: Creating a Cyclical Time Feature 801 | 802 | ```sql 803 | WITH time AS ( 804 | SELECT 805 | EXTRACT(HOUR FROM transaction_date) AS hour, 806 | EXTRACT(MINUTE FROM transaction_date) AS minute 807 | FROM sales 808 | ) 809 | SELECT 810 | SIN((hour + minute / 60) * 2 * PI() / 24) AS time_of_day_sin, 811 | COS((hour + minute / 60) * 2 * PI() / 24) AS time_of_day_cos 812 | FROM time; 813 | ``` 814 | 815 | ### Additional Features: 816 | 817 | 1. **Time Since Last Event**: Use a subquery to calculate the time difference between the current event and the previous one. 818 | 2. **Time Until Next Event**: Employ a similar subquery to determine the time remaining until the subsequent event. 819 | 820 | #### Example: Calculating Time Since the Previous Event 821 | 822 | ```sql 823 | WITH ranked_sales AS ( 824 | SELECT 825 | transaction_date, 826 | ROW_NUMBER() OVER (ORDER BY transaction_date) AS row_num 827 | FROM sales 828 | ) 829 | SELECT 830 | transaction_date - LAG(transaction_date) OVER (ORDER BY transaction_date) AS time_since_prev_event 831 | FROM ranked_sales; 832 | ``` 833 | 834 | These time-based and time period features can enhance the predictive power of your machine learning models. 835 |
836 | 837 | ## 15. What are SQL _Window Functions_ and how can they be used for Machine Learning _feature engineering_? 838 | 839 | **Window functions** in SQL allow for computations across **specific data windows** rather than the entire dataset. This makes them highly useful for ML feature engineering, providing advanced capabilities for data aggregation and ordering. 840 | 841 | ### Benefits for Machine Learning 842 | 843 | Window functions are optimized for efficient handling of large datasets. Their scope can be fine-tuned using **PARTITION BY** and ordering operators like **ORDER BY**, making them perfect for time series calculations, customer cohorts, and data denoising. 844 | 845 | 1. **Calculation of Lag/Lead Values** 846 | 847 | Which are useful in constructing **time-serial features** like deltas and moving averages. 848 | 849 | 2. **Data Ranking** 850 | 851 | This assists in creating features like **quantiles**, which are common in distributions. \[1\.0, -2.0, 1.0, 1.0, 0.5, -1.5, 0.5, ...], for example. 852 | 853 | 3. **Data Accumulation and Running Sums** 854 | 855 | This is often used in **time series** feature engineering, for example, a rolling sum over the past 7 days or to calculate an **exponential moving average**. 856 | 857 | 4. **Identification of Data Groups** 858 | 859 | This helps in creating features that are sensitive to **group-level** distinctiveness (e.g., buying habits of certain customers). 860 | 861 | 5. **Advanced Data Imputation** 862 | 863 | While missing data is a common challenge in datasets, approaches like **forward-filling" or "back-filling** can help in this regard. 864 | 865 | 6. **Smoother Kernel Calculation** 866 | 867 | Functions like **ROW_NUMBER** along with **OVER (ORDER BY...)** operator can compute rolling averages on a **smaller window**, leading to a less noisy distribution, which can be specially beneficial if your goal is to accurately predict a trend amidst other fluctuations. 868 | 869 | 7. **Efficient Sampling** 870 | 871 | This is useful in balancing datasets for classifications. By partitioning datasets and then using **INTEGER RATIO** or **FRACTIONAL RATIO**, you can ensure the partitioned datasets are uniformly sampled. 872 | 873 | 874 | ### PASAD Unit Example 875 | 876 | Consider the following query that utilizes a window function, **ROW_NUMBER** along with **PARTITION BY**, to assign section numbers to a set of records in a table ordered by a certain criterion. 877 | 878 | ```sql 879 | SELECT 880 | id, 881 | seq, -- Sequence within the section 882 | section_no, 883 | attribute 884 | FROM 885 | ( 886 | SELECT 887 | id, 888 | attribute, 889 | ROW_NUMBER() OVER(PARTITION BY attribute ORDER BY seq) as seq, 890 | (ROW_NUMBER() OVER(ORDER BY attribute, seq))::float / 891 | (COUNT(*) OVER (PARTITION BY attribute)) AS order_ratio, 892 | FROM table1 893 | ) 894 | ``` 895 |
896 | 897 | 898 | 899 | #### Explore all 55 answers here 👉 [Devinterview.io - SQL](https://devinterview.io/questions/machine-learning-and-data-science/sql-ml-interview-questions) 900 | 901 |
902 | 903 | 904 | machine-learning-and-data-science 905 | 906 |

907 | 908 | --------------------------------------------------------------------------------