├── Demos ├── README.md ├── IF Statement -PQ - DEMO.pdf └── IF Statement - PQ - DEMO.pbix ├── Tips-and-Tricks.md ├── Snippets └── Snippets.md └── README.md /Demos/README.md: -------------------------------------------------------------------------------- 1 | # Demos 2 | -------------------------------------------------------------------------------- /Demos/IF Statement -PQ - DEMO.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NajiElKotob/Awesome-PowerQuery/HEAD/Demos/IF Statement -PQ - DEMO.pdf -------------------------------------------------------------------------------- /Demos/IF Statement - PQ - DEMO.pbix: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NajiElKotob/Awesome-PowerQuery/HEAD/Demos/IF Statement - PQ - DEMO.pbix -------------------------------------------------------------------------------- /Tips-and-Tricks.md: -------------------------------------------------------------------------------- 1 | # Power Query Tips and Tricks 2 | 3 | ## M 4 | * =#shared 5 | 6 | ## UI 7 | * Quick Access Toolbar (QAT) 8 | 9 | ## Editor 10 | * Sort Columns using Choose Columns 11 | 12 | ## Import 13 | * Select Folder (All sheets, Excel) or Right-click -> Transform Data (Power BI) 14 | 15 | ## Shortcuts 16 | * ALT + F12 (Opens Power query editor, Excel) 17 | * [Copy/Paste queries](https://learn.microsoft.com/en-us/power-query/keyboard-shortcuts): CTRL+C/CTRL+V 18 | * CTRL + G (Go to Column) 19 | * CTRL+SHIFT+ +/- (Zoom in/Zoom out) 20 | -------------------------------------------------------------------------------- /Snippets/Snippets.md: -------------------------------------------------------------------------------- 1 | # PQ Snippets 2 | 3 | ### Quick Snippets 4 | * [Text.From](https://docs.microsoft.com/en-us/powerquery-m/text-from)([Year]) & "W" & [Text.PadStart](https://docs.microsoft.com/en-us/powerquery-m/text-padstart)(Text.From([Week of Year]),2,"0") 5 | * Underscore _ references to the current row (record) of the table. e.g., #"Removed Columns" = Table.RemoveColumns(#"Changed Type", List.Select(Table.ColumnNames(#"Changed Type"), each Text.EndsWith(_,"Price"))) // remove all columns that end with "Price" 6 | * Add Leading Zeros; Text.End( "000" & Text.From ([MonthNo] ), 4 ), Text.PadStart( Text.From( [MonthNo] ), 4, "0" ) 7 | * Get named range value: Excel.CurrentWorkbook(){[Name=NamedRange]}[Content]{0}[Column1] 8 | * Last Refreshed Date: DateTime.LocalNow() 9 | 10 | ## Custom Functions 11 | 12 | ----- 13 | ### ConvertRuntimeToMinutes 14 | * To use the function in Power Query, simply add a custom column and apply the function to your runtime column, like this: ConvertRuntimeToMinutes([Runtime]), where ConvertRuntimeToMinutes is the name of the function and [Runtime] is your column name. Examples "2h 22m" => 142, "1h 45m" => 105, "3h" => 180, "45m" => 45. 15 | ``` 16 | (source as text) as number => 17 | let 18 | isEmpty = source = null or source = "", 19 | result = if isEmpty then 0 else 20 | let 21 | hasHours = Text.Contains(source, "h"), 22 | hasMinutes = Text.Contains(source, "m"), 23 | hours = if hasHours then Number.FromText(Text.BeforeDelimiter(source, "h")) else 0, 24 | minutes = if hasMinutes then 25 | if hasHours then Number.FromText(Text.BeforeDelimiter(Text.AfterDelimiter(source, "h"), "m")) 26 | else Number.FromText(Text.BeforeDelimiter(source, "m")) 27 | else 0, 28 | totalMinutes = hours * 60 + minutes 29 | in 30 | totalMinutes 31 | in 32 | result 33 | 34 | ``` 35 | 36 | ----- 37 | 38 | ### TypeChecker 39 | * This custom function in Power Query, named TypeChecker, determines the data type of a given column, identifying it as "Text", "Number", "Date", or "Other", and also handles null values by returning "Null". 40 | 41 | ``` 42 | let 43 | // Define the custom function with a parameter 44 | TypeChecker = (inputColumn as any) as text => 45 | let 46 | // Check if the input is null 47 | result = if inputColumn = null then "Null" 48 | 49 | // If not null, determine the data type 50 | else 51 | let 52 | dataType = Value.Type(inputColumn), 53 | typeResult = if dataType = type text then "Text" 54 | else if dataType = type number then "Number" 55 | else if dataType = type datetime or dataType = type date then "Date" 56 | else "Other" 57 | in 58 | typeResult 59 | in 60 | result 61 | in 62 | TypeChecker 63 | 64 | 65 | ``` 66 | 67 | #### FilteredQuery 68 | ``` 69 | = Table.SelectRows(Source, each ([DateTypeCheck] <> "Date" 70 | or [TextTypeCheck] <> "Text" 71 | or [NumberGreaterThanCheck] = true 72 | )) 73 | ``` 74 | 75 | #### Trim Special Edges 76 | ``` 77 | // Removes leading and trailing characters from the specified list. 78 | 79 | = Text.Trim("__- Hello World -,", {",", "-", " ", "_"}) 80 | ``` 81 | 82 | 83 | #### Remove Last Character 84 | 85 | ``` 86 | = 87 | Table.TransformColumns( 88 | #"Previous Step", 89 | { 90 | {"Number of Ratings - Copy", 91 | each Number.FromText(Text.Start(_, Text.Length(_) - 1)), 92 | type number} 93 | } 94 | ) 95 | ``` 96 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Awesome PowerQuery 2 | {Awesome Works in Progress} 3 | 4 | ` 5 | Power Query is a data transformation and data preparation engine. Where the 'M'agic happens! 6 | ` 7 | 8 | ## YouTube :tv: 9 | * Kevin Stratvert 10 | - [How to use Microsoft Power Query ~16min](https://www.youtube.com/watch?v=0aeZX1l4JT4) 11 | * [Curbal - Power Query](https://www.youtube.com/watch?v=dbTvOk1IyNU&list=PLDz00l_jz6zxF_OSmQhWBCVmQOaROoxWj) 12 | * [Pragmatic Works](https://www.youtube.com/user/PragmaticWorks/search?query=Power+Query) 13 | * [Power Query Jumpstarter](https://www.youtube.com/watch?v=7Vn6uOxcAc0&list=PLHYaVuyjhcqyYD7qss7lsFVBLf8B_zZrx) - Brian Grant 14 | * Goodly 15 | - [Power Query Tricks n Tips](https://www.youtube.com/playlist?list=PLr7RyN24TvNaQPHeViV3koJrZR_U7UhD4) 16 | * Microsoft Power BI 17 | - [Power BI Dev Camp March Intro to M Programming](https://www.youtube.com/watch?v=BsgOU9eeCBg) 18 | 19 | 20 | ----- 21 | ## Learning 22 | 23 | ### Power Query Docs 24 | * [Power Query - Microsoft Learn](https://learn.microsoft.com/en-us/training/browse/?products=power-query&source=learn) 25 | * [Power Query for Excel Help](https://support.microsoft.com/en-us/office/power-query-for-excel-help-2b433a85-ddfb-420b-9cda-fe0e60b82a94) ⭐ 26 | 27 | ### The Power Query user interface 28 | * [The Power Query editor user experience](https://learn.microsoft.com/en-us/power-query/power-query-ui#the-power-query-editor-user-experience) 29 | * [Using the Applied Steps list](https://learn.microsoft.com/en-us/power-query/applied-steps) 30 | - Rename (F2); Add Description => // in the M Code; Parametrization 31 | * [Using the Queries pane](https://learn.microsoft.com/en-us/power-query/queries-pane) 32 | - Comments; Folders; Order 33 | 34 | ### Data types 35 | * [Data types in Power Query](https://learn.microsoft.com/en-us/power-query/data-types) 36 | * [4 Ways to Fix Date Errors in Power Query + Locale & Regional Settings 📺 ~10min](https://www.excelcampus.com/powerquery/power-query-date-errors-settings/) 37 | * [ISO 8601 - Date and time format](https://www.iso.org/iso-8601-date-and-time-format.html) - YYYY-MM-DD e.g., 2022-09-27 18:00:00.000 38 | 39 | ### Transform Columns 40 | * [Rename columns](https://learn.microsoft.com/en-us/power-query/rename-column) 41 | * [Fill values in a column](https://learn.microsoft.com/en-us/power-query/fill-values-column) - Fill up, Fill down, Replace values, Filter 42 | - Ghost columns e.g., Empty columns in excel 43 | * [Replace values and errors](https://learn.microsoft.com/en-us/power-query/replace-values) 44 | 45 | ### Add Columns 46 | * Column from examples 47 | - [Add a column from examples](https://learn.microsoft.com/en-us/power-query/column-from-example) 48 | - [Blown AWAY by Column From Examples in Power Query 📺](https://www.youtube.com/watch?v=BSmmNgO_EOU) 49 | * Index Column 50 | - [Add an index column](https://learn.microsoft.com/en-us/power-query/add-index-column) 51 | - [Create Index Column By Group in Power Query 📺 ~10min](https://www.youtube.com/watch?v=_PBX3RPXxHw) - BI Gorilla 52 | 53 | ### Transform Tables 54 | * [Unpivot columns](https://learn.microsoft.com/en-us/power-query/unpivot-column) 55 | * [Working with duplicate values](https://learn.microsoft.com/en-us/power-query/working-with-duplicates) 56 | * [Filter a table by row position](https://learn.microsoft.com/en-us/power-query/filter-row-position) - Keep/Remove top/bottom rows 57 | 58 | ### Combine Data 59 | * [Merge](https://learn.microsoft.com/en-us/power-query/merge-queries-overview) 60 | * [Append queries](https://docs.microsoft.com/en-us/power-query/append-queries) - The append operation creates a single table by adding the contents of one or more tables to another. 61 | 62 | ### Splitting 63 | * Delimeters with more than one character e.g., ;; 64 | 65 | ### Consolidation 66 | * [Grouping or summarizing rows](https://learn.microsoft.com/en-us/power-query/group-by) - Grouping; Aggregating 67 | 68 | 69 | 70 | 71 | ### Dates and Times 72 | * Deal with differnt date regions 73 | 74 | ### JSON 75 | * [How to Parse Custom JSON Data using Excel](https://theexcelclub.com/how-to-parse-custom-json-data-using-excel/) 76 | 77 | ### Remove Duplicates 78 | * [Keep The Most Recent Entry](https://www.excelguru.ca/blog/2016/05/25/keep-the-most-recent-entry/) - Ken Puls (May 2016) 79 | * [Remove Duplicate Doesn’t Work in Power Query for Power BI](https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution) 80 | 81 | ### Splitting 82 | * [Split by line breaks in Power Query](https://www.excelguru.ca/blog/2015/10/16/split-by-line-breaks/) 83 | * Ref. Line feed: #(lf), Carriage return: #(cr), Tab: #(tab) 84 | * [When is a space not a space? When Power Query Clean Fails 📺](https://www.youtube.com/watch?v=61qnHOcXvLs); [When Is a Space Not a Space?](https://help.analyticsedge.com/article/when-is-a-space-not-a-space/) 85 | 86 | ### Images 87 | * [Embedding Images in Power BI using Base64](http://sqljason.com/2018/01/embedding-images-in-power-bi-using-base64.html) - sqljason.com 88 | 89 | ### Web 90 | * [Handling 404–Not Found Errors With Web.Contents() In Power Query And Power BI](https://blog.crossjoin.co.uk/2016/08/09/handling-404-not-found-errors-with-web-contents-in-power-query-and-power-bi/) 91 | 92 | ### Errors and Diagnostics 93 | * [4 Ways to Fix Date Errors in Power Query + Locale & Regional Settings](https://www.excelcampus.com/powerquery/power-query-date-errors-settings/) - Jon Acampora (April 2020) 94 | * [Awesome Way To Log in Power Query M Using Diagnostics.Trace](https://blog.learningtree.com/awesome-way-log-power-query-m-using-diagnostics-trace/) - Dan Buskirk (June 2016) 95 | * [Viewing Error Messages For All Rows In Power Query](https://blog.crossjoin.co.uk/2014/12/22/viewing-error-messages-for-all-rows-in-power-query/) - Chris Webb (December 2014) 96 | 97 | ### M Language 98 | * [Power Query M formula language](https://docs.microsoft.com/en-us/powerquery-m/) 99 | * [Quick tour of the Power Query M formula language](https://docs.microsoft.com/en-us/powerquery-m/quick-tour-of-the-power-query-m-formula-language) 100 | * [IF Statements](https://www.myonlinetraininghub.com/power-query-if-statements) - myonlinetraininghub.com 101 | * [Dynamic M query parameters ](https://docs.microsoft.com/en-us/power-bi/connect-data/desktop-dynamic-m-query-parameters) 102 | 103 | ### Best Practices 104 | * [Best practices when working with Power Query](https://docs.microsoft.com/en-us/power-query/best-practices) 105 | 106 | 107 | ## Microsoft Learn 108 | * [Clean, transform, and load data in Power BI](https://learn.microsoft.com/en-us/training/modules/clean-data-power-bi/?source=najielkotob) 109 | 110 | ----- 111 | 112 | ## Data Modeling 113 | * [Understand star schema and the importance for Power BI](https://learn.microsoft.com/en-us/power-bi/guidance/star-schema) 114 | * [From Flat File To Data Model 📺 ~11min](https://www.youtube.com/watch?v=vjBprojOCzU) - Guy in a Cube 115 | 116 | ----- 117 | 118 | ## Articles 119 | * [Value from previous row, using List.Range](https://exceltown.com/en/tutorials/power-bi/powerbi-com-and-power-bi-desktop/power-bi-data-sources/power-query-get-value-from-previous-row/) - exceltown.com 120 | * [Power Query Skills Apply to Excel, Power BI and SSAS Tabular](https://sqlserverbi.blog/2017/12/04/power-query-skills-apply-to-excel-power-bi-and-ssas-tabular/) 121 | * [Comparing ‘null’ values in Power Query](http://excel-inside.pro/blog/2018/05/17/comparing-null-values-in-power-query/) 122 | * [Power Query tips for every Power BI Developer](https://towardsdatascience.com/power-query-tips-for-every-power-bi-developer-da9ebd3dcd93) - Nikola Ilic 123 | * [Using M to dynamically change column names in PowerQuery](https://exceed.hr/blog/using-m-to-dynamically-change-column-names-in-powerquery/) - Krešimir Ledinski (October 2020) 124 | 125 | ----- 126 | 127 | ## Blogs 128 | * [Chris Webb's BI Blog](https://blog.crossjoin.co.uk/) - Chris Webb 129 | * [Power Query](https://www.powerquery.io/) - powerquery.io 130 | 131 | ----- 132 | 133 | ## Tools and Add-ins 134 | * [Power Query Formatter](https://powerqueryformatter.com/) - Beautify your Power Query (M-Language) Code with Power Query Formatter 135 | * [Power Query SDK](https://marketplace.visualstudio.com/items?itemName=Dakahn.PowerQuerySDK) - A Power Query language service for Visual Studio 136 | * [Power Query Source (SSIS)](https://docs.microsoft.com/en-us/sql/integration-services/data-flow/power-query-source) 137 | 138 | ----- 139 | 140 | ## Power Query Online 141 | * [Visual Data Prep](https://powerbi.microsoft.com/en-us/blog/announcing-visual-data-prep-general-availability-diagram-view-in-power-query-online/) 142 | 143 | 144 | 145 | 146 | --------------------------------------------------------------------------------