├── LICENSES ├── CODE_OF_CONDUCT.md ├── LICENSE ├── LICENSE-CODE ├── README.md └── SECURITY.md ├── README.md ├── SECURITY.md ├── graphics └── microsoftlogo.png ├── slides ├── SQL Server 2019 Modern Data Platform.pptx ├── SQL Server Ground to Cloud.pptx ├── SQL Server on the Microsoft Azure Platform.pptx └── g2c.pdf └── sqlgroundtocloud ├── 00-prerequisites.md ├── 01-IntroductionAndWorkshopMethodology.md ├── 02-ModernizingYourDataEstateWithSQLServer2019.md ├── 03-WorkingWithBigDataAndDataScienceBigDataClustersForSQLServer2019.md ├── 04-SQLServerOnTheMicrosoftAzurePlatform.md ├── 05-MigratingToAzureSQL.md └── 06-WhatToUseWhen.md /LICENSES/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Microsoft Open Source Code of Conduct 2 | 3 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 4 | 5 | Resources: 6 | 7 | - [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/) 8 | - [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) 9 | - Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns 10 | -------------------------------------------------------------------------------- /LICENSES/LICENSE: -------------------------------------------------------------------------------- 1 | Attribution 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution 4.0 International Public License 58 | 59 | By exercising the Licensed Rights (defined below), You accept and agree 60 | to be bound by the terms and conditions of this Creative Commons 61 | Attribution 4.0 International Public License ("Public License"). To the 62 | extent this Public License may be interpreted as a contract, You are 63 | granted the Licensed Rights in consideration of Your acceptance of 64 | these terms and conditions, and the Licensor grants You such rights in 65 | consideration of benefits the Licensor receives from making the 66 | Licensed Material available under these terms and conditions. 67 | 68 | 69 | Section 1 -- Definitions. 70 | 71 | a. Adapted Material means material subject to Copyright and Similar 72 | Rights that is derived from or based upon the Licensed Material 73 | and in which the Licensed Material is translated, altered, 74 | arranged, transformed, or otherwise modified in a manner requiring 75 | permission under the Copyright and Similar Rights held by the 76 | Licensor. For purposes of this Public License, where the Licensed 77 | Material is a musical work, performance, or sound recording, 78 | Adapted Material is always produced where the Licensed Material is 79 | synched in timed relation with a moving image. 80 | 81 | b. Adapter's License means the license You apply to Your Copyright 82 | and Similar Rights in Your contributions to Adapted Material in 83 | accordance with the terms and conditions of this Public License. 84 | 85 | c. Copyright and Similar Rights means copyright and/or similar rights 86 | closely related to copyright including, without limitation, 87 | performance, broadcast, sound recording, and Sui Generis Database 88 | Rights, without regard to how the rights are labeled or 89 | categorized. For purposes of this Public License, the rights 90 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 91 | Rights. 92 | 93 | d. Effective Technological Measures means those measures that, in the 94 | absence of proper authority, may not be circumvented under laws 95 | fulfilling obligations under Article 11 of the WIPO Copyright 96 | Treaty adopted on December 20, 1996, and/or similar international 97 | agreements. 98 | 99 | e. Exceptions and Limitations means fair use, fair dealing, and/or 100 | any other exception or limitation to Copyright and Similar Rights 101 | that applies to Your use of the Licensed Material. 102 | 103 | f. Licensed Material means the artistic or literary work, database, 104 | or other material to which the Licensor applied this Public 105 | License. 106 | 107 | g. Licensed Rights means the rights granted to You subject to the 108 | terms and conditions of this Public License, which are limited to 109 | all Copyright and Similar Rights that apply to Your use of the 110 | Licensed Material and that the Licensor has authority to license. 111 | 112 | h. Licensor means the individual(s) or entity(ies) granting rights 113 | under this Public License. 114 | 115 | i. Share means to provide material to the public by any means or 116 | process that requires permission under the Licensed Rights, such 117 | as reproduction, public display, public performance, distribution, 118 | dissemination, communication, or importation, and to make material 119 | available to the public including in ways that members of the 120 | public may access the material from a place and at a time 121 | individually chosen by them. 122 | 123 | j. Sui Generis Database Rights means rights other than copyright 124 | resulting from Directive 96/9/EC of the European Parliament and of 125 | the Council of 11 March 1996 on the legal protection of databases, 126 | as amended and/or succeeded, as well as other essentially 127 | equivalent rights anywhere in the world. 128 | 129 | k. You means the individual or entity exercising the Licensed Rights 130 | under this Public License. Your has a corresponding meaning. 131 | 132 | 133 | Section 2 -- Scope. 134 | 135 | a. License grant. 136 | 137 | 1. Subject to the terms and conditions of this Public License, 138 | the Licensor hereby grants You a worldwide, royalty-free, 139 | non-sublicensable, non-exclusive, irrevocable license to 140 | exercise the Licensed Rights in the Licensed Material to: 141 | 142 | a. reproduce and Share the Licensed Material, in whole or 143 | in part; and 144 | 145 | b. produce, reproduce, and Share Adapted Material. 146 | 147 | 2. Exceptions and Limitations. For the avoidance of doubt, where 148 | Exceptions and Limitations apply to Your use, this Public 149 | License does not apply, and You do not need to comply with 150 | its terms and conditions. 151 | 152 | 3. Term. The term of this Public License is specified in Section 153 | 6(a). 154 | 155 | 4. Media and formats; technical modifications allowed. The 156 | Licensor authorizes You to exercise the Licensed Rights in 157 | all media and formats whether now known or hereafter created, 158 | and to make technical modifications necessary to do so. The 159 | Licensor waives and/or agrees not to assert any right or 160 | authority to forbid You from making technical modifications 161 | necessary to exercise the Licensed Rights, including 162 | technical modifications necessary to circumvent Effective 163 | Technological Measures. For purposes of this Public License, 164 | simply making modifications authorized by this Section 2(a) 165 | (4) never produces Adapted Material. 166 | 167 | 5. Downstream recipients. 168 | 169 | a. Offer from the Licensor -- Licensed Material. Every 170 | recipient of the Licensed Material automatically 171 | receives an offer from the Licensor to exercise the 172 | Licensed Rights under the terms and conditions of this 173 | Public License. 174 | 175 | b. No downstream restrictions. You may not offer or impose 176 | any additional or different terms or conditions on, or 177 | apply any Effective Technological Measures to, the 178 | Licensed Material if doing so restricts exercise of the 179 | Licensed Rights by any recipient of the Licensed 180 | Material. 181 | 182 | 6. No endorsement. Nothing in this Public License constitutes or 183 | may be construed as permission to assert or imply that You 184 | are, or that Your use of the Licensed Material is, connected 185 | with, or sponsored, endorsed, or granted official status by, 186 | the Licensor or others designated to receive attribution as 187 | provided in Section 3(a)(1)(A)(i). 188 | 189 | b. Other rights. 190 | 191 | 1. Moral rights, such as the right of integrity, are not 192 | licensed under this Public License, nor are publicity, 193 | privacy, and/or other similar personality rights; however, to 194 | the extent possible, the Licensor waives and/or agrees not to 195 | assert any such rights held by the Licensor to the limited 196 | extent necessary to allow You to exercise the Licensed 197 | Rights, but not otherwise. 198 | 199 | 2. Patent and trademark rights are not licensed under this 200 | Public License. 201 | 202 | 3. To the extent possible, the Licensor waives any right to 203 | collect royalties from You for the exercise of the Licensed 204 | Rights, whether directly or through a collecting society 205 | under any voluntary or waivable statutory or compulsory 206 | licensing scheme. In all other cases the Licensor expressly 207 | reserves any right to collect such royalties. 208 | 209 | 210 | Section 3 -- License Conditions. 211 | 212 | Your exercise of the Licensed Rights is expressly made subject to the 213 | following conditions. 214 | 215 | a. Attribution. 216 | 217 | 1. If You Share the Licensed Material (including in modified 218 | form), You must: 219 | 220 | a. retain the following if it is supplied by the Licensor 221 | with the Licensed Material: 222 | 223 | i. identification of the creator(s) of the Licensed 224 | Material and any others designated to receive 225 | attribution, in any reasonable manner requested by 226 | the Licensor (including by pseudonym if 227 | designated); 228 | 229 | ii. a copyright notice; 230 | 231 | iii. a notice that refers to this Public License; 232 | 233 | iv. a notice that refers to the disclaimer of 234 | warranties; 235 | 236 | v. a URI or hyperlink to the Licensed Material to the 237 | extent reasonably practicable; 238 | 239 | b. indicate if You modified the Licensed Material and 240 | retain an indication of any previous modifications; and 241 | 242 | c. indicate the Licensed Material is licensed under this 243 | Public License, and include the text of, or the URI or 244 | hyperlink to, this Public License. 245 | 246 | 2. You may satisfy the conditions in Section 3(a)(1) in any 247 | reasonable manner based on the medium, means, and context in 248 | which You Share the Licensed Material. For example, it may be 249 | reasonable to satisfy the conditions by providing a URI or 250 | hyperlink to a resource that includes the required 251 | information. 252 | 253 | 3. If requested by the Licensor, You must remove any of the 254 | information required by Section 3(a)(1)(A) to the extent 255 | reasonably practicable. 256 | 257 | 4. If You Share Adapted Material You produce, the Adapter's 258 | License You apply must not prevent recipients of the Adapted 259 | Material from complying with this Public License. 260 | 261 | 262 | Section 4 -- Sui Generis Database Rights. 263 | 264 | Where the Licensed Rights include Sui Generis Database Rights that 265 | apply to Your use of the Licensed Material: 266 | 267 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 268 | to extract, reuse, reproduce, and Share all or a substantial 269 | portion of the contents of the database; 270 | 271 | b. if You include all or a substantial portion of the database 272 | contents in a database in which You have Sui Generis Database 273 | Rights, then the database in which You have Sui Generis Database 274 | Rights (but not its individual contents) is Adapted Material; and 275 | 276 | c. You must comply with the conditions in Section 3(a) if You Share 277 | all or a substantial portion of the contents of the database. 278 | 279 | For the avoidance of doubt, this Section 4 supplements and does not 280 | replace Your obligations under this Public License where the Licensed 281 | Rights include other Copyright and Similar Rights. 282 | 283 | 284 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 285 | 286 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 287 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 288 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 289 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 290 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 291 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 292 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 293 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 294 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 295 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 296 | 297 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 298 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 299 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 300 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 301 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 302 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 303 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 304 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 305 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 306 | 307 | c. The disclaimer of warranties and limitation of liability provided 308 | above shall be interpreted in a manner that, to the extent 309 | possible, most closely approximates an absolute disclaimer and 310 | waiver of all liability. 311 | 312 | 313 | Section 6 -- Term and Termination. 314 | 315 | a. This Public License applies for the term of the Copyright and 316 | Similar Rights licensed here. However, if You fail to comply with 317 | this Public License, then Your rights under this Public License 318 | terminate automatically. 319 | 320 | b. Where Your right to use the Licensed Material has terminated under 321 | Section 6(a), it reinstates: 322 | 323 | 1. automatically as of the date the violation is cured, provided 324 | it is cured within 30 days of Your discovery of the 325 | violation; or 326 | 327 | 2. upon express reinstatement by the Licensor. 328 | 329 | For the avoidance of doubt, this Section 6(b) does not affect any 330 | right the Licensor may have to seek remedies for Your violations 331 | of this Public License. 332 | 333 | c. For the avoidance of doubt, the Licensor may also offer the 334 | Licensed Material under separate terms or conditions or stop 335 | distributing the Licensed Material at any time; however, doing so 336 | will not terminate this Public License. 337 | 338 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 339 | License. 340 | 341 | 342 | Section 7 -- Other Terms and Conditions. 343 | 344 | a. The Licensor shall not be bound by any additional or different 345 | terms or conditions communicated by You unless expressly agreed. 346 | 347 | b. Any arrangements, understandings, or agreements regarding the 348 | Licensed Material not stated herein are separate from and 349 | independent of the terms and conditions of this Public License. 350 | 351 | 352 | Section 8 -- Interpretation. 353 | 354 | a. For the avoidance of doubt, this Public License does not, and 355 | shall not be interpreted to, reduce, limit, restrict, or impose 356 | conditions on any use of the Licensed Material that could lawfully 357 | be made without permission under this Public License. 358 | 359 | b. To the extent possible, if any provision of this Public License is 360 | deemed unenforceable, it shall be automatically reformed to the 361 | minimum extent necessary to make it enforceable. If the provision 362 | cannot be reformed, it shall be severed from this Public License 363 | without affecting the enforceability of the remaining terms and 364 | conditions. 365 | 366 | c. No term or condition of this Public License will be waived and no 367 | failure to comply consented to unless expressly agreed to by the 368 | Licensor. 369 | 370 | d. Nothing in this Public License constitutes or may be interpreted 371 | as a limitation upon, or waiver of, any privileges and immunities 372 | that apply to the Licensor or You, including from the legal 373 | processes of any jurisdiction or authority. 374 | 375 | 376 | ======================================================================= 377 | 378 | Creative Commons is not a party to its public 379 | licenses. Notwithstanding, Creative Commons may elect to apply one of 380 | its public licenses to material it publishes and in those instances 381 | will be considered the “Licensor.” The text of the Creative Commons 382 | public licenses is dedicated to the public domain under the CC0 Public 383 | Domain Dedication. Except for the limited purpose of indicating that 384 | material is shared under a Creative Commons public license or as 385 | otherwise permitted by the Creative Commons policies published at 386 | creativecommons.org/policies, Creative Commons does not authorize the 387 | use of the trademark "Creative Commons" or any other trademark or logo 388 | of Creative Commons without its prior written consent including, 389 | without limitation, in connection with any unauthorized modifications 390 | to any of its public licenses or any other arrangements, 391 | understandings, or agreements concerning use of licensed material. For 392 | the avoidance of doubt, this paragraph does not form part of the 393 | public licenses. 394 | 395 | Creative Commons may be contacted at creativecommons.org. -------------------------------------------------------------------------------- /LICENSES/LICENSE-CODE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) Microsoft Corporation. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE 22 | -------------------------------------------------------------------------------- /LICENSES/README.md: -------------------------------------------------------------------------------- 1 | 2 | # Contributing 3 | 4 | This project welcomes contributions and suggestions. Most contributions require you to agree to a 5 | Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us 6 | the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. 7 | 8 | When you submit a pull request, a CLA bot will automatically determine whether you need to provide 9 | a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions 10 | provided by the bot. You will only need to do this once across all repos using our CLA. 11 | 12 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 13 | For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or 14 | contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. 15 | 16 | # Legal Notices 17 | 18 | Microsoft and any contributors grant you a license to the Microsoft documentation and other content 19 | in this repository under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/legalcode), 20 | see the [LICENSE](LICENSE) file, and grant you a license to any code in the repository under the [MIT License](https://opensource.org/licenses/MIT), see the 21 | [LICENSE-CODE](LICENSE-CODE) file. 22 | 23 | Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation 24 | may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. 25 | The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. 26 | Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653. 27 | 28 | Privacy information can be found at https://privacy.microsoft.com/en-us/ 29 | 30 | Microsoft and any contributors reserve all other rights, whether under their respective copyrights, patents, 31 | or trademarks, whether by implication, estoppel or otherwise. 32 | -------------------------------------------------------------------------------- /LICENSES/SECURITY.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Security 4 | 5 | Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/). 6 | 7 | If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)) of a security vulnerability, please report it to us as described below. 8 | 9 | ## Reporting Security Issues 10 | 11 | **Please do not report security vulnerabilities through public GitHub issues.** 12 | 13 | Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://msrc.microsoft.com/create-report). 14 | 15 | If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc). 16 | 17 | You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). 18 | 19 | Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: 20 | 21 | * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) 22 | * Full paths of source file(s) related to the manifestation of the issue 23 | * The location of the affected source code (tag/branch/commit or direct URL) 24 | * Any special configuration required to reproduce the issue 25 | * Step-by-step instructions to reproduce the issue 26 | * Proof-of-concept or exploit code (if possible) 27 | * Impact of the issue, including how an attacker might exploit the issue 28 | 29 | This information will help us triage your report more quickly. 30 | 31 | If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://microsoft.com/msrc/bounty) page for more details about our active programs. 32 | 33 | ## Preferred Languages 34 | 35 | We prefer all communications to be in English. 36 | 37 | ## Policy 38 | 39 | Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd). 40 | 41 | 42 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![](graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft Workshop from the SQL Server team 6 | 7 |

8 | 9 |

About this Workshop

10 | 11 | Welcome to this Microsoft solutions workshop on SQL Server in on-premises, in-cloud and hybrid solutions. In this one-day workshop, you'll learn how SQL Server 2019 and Azure SQL help you solve real-world challenges. 12 | 13 | The Modules in this workshop lead you through conceptual and hands-on topics ranging from the newest technical features in SQL Server to its implementation in all the platforms it runs on. You'll learn not only specific technologies, but how to assemble them into a complete solution based on customer needs and requests. 14 | 15 | You'll start by learning about the latest improvements in SQL Server 2019, work with the Big Data Clusters configuration, and then learn about the ways you can leverage SQL in Microsoft Azure (and how to get there) - all with a focus on how to extrapolate what you have learned to create other solutions for your organization. You'll end the day with a "What to Use When" module explaining how to create your own solutions. 16 | 17 | This Workshop contains lecture and hands on lab work, and is particularly useful for Solution Architects, Data Architects, Application Architects, Technical Sellers, and Application Developers. A laptop, Microsoft Azure account, and experience with SQL is considered a prerequisite. 18 | 19 | This README.MD file explains how the workshop is laid out, what you will learn, and the technologies you will use in this solution. 20 | 21 | (You can view all of the [source files for this workshop on this GitHub site, along with other workshops as well. Open this link in a new tab to find out more.](https://github.com/microsoft/sqlworkshops-sqlg2c)) 22 | 23 |

24 | 25 | 26 |

Learning Objectives

27 | 28 | In this day-long, hands-on Workshop you’ll learn how to: 29 | 30 |
31 | 32 | - Articulate the key differentiators between SQL Server on-prem, in Azure VM, in Azure SQL, and hybrid configurations 33 | - Explain the different service tiers within Azure SQL, and what to choose when 34 | - Understand how Azure SQL is secured and address security concerns 35 | - Explain hybrid configurations for Azure SQL and SQL Server in common examples 36 | - Experiment with various features of SQL Server and Azure SQL 37 | - Understand additional Azure platform features for data pipelines and security 38 | - Understand what services are available to migrate and modernize your entire SQL Server stack 39 | - Make informed decisions about how your business or customers should modernize their data estate 40 | - Use Big Data solutions 41 | 42 |

Role Applications

43 | 44 | The concepts and skills taught in this workshop form the starting points for: 45 | 46 | - Solution Architects and Developers, to understand how to put together an end to end solution. 47 | - Data Professionals and DevOps teams, to implement and operate SQL Server systems on premises and in the cloud. 48 | - Data Scientists, to understand the environment used to analyze and solve specific predictive problems. 49 | 50 |

51 |

Business Applications of this Workshop

52 | 53 | Businesses require near real-time insights from ever-larger sets of data from a variety of sources. Many have not explored the improvements made in the latest versions of SQL Server, and some are only now exploring the cloud as a computing platform. As time has progressed, a more dramatic upgrade process may be required. 54 | 55 | In addition to traditional Online Transaction Processing (OLTP) and Online Analytic Processing (OLAP) workloads, some industry examples of data processing from multiple sources of data at scale are: 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 |
IndustryExample
RetailDemand Prediction
Market-Basket Analysis
FinanceFraud detection
customer segmentation
HealthcareFiscal control analytics
Disease Prevention prediction and classification
Clinical Trials optimization
Public SectorRevenue prediction
Education effectiveness analysis
ManufacturingPredictive Maintenance
Anomaly Detection
AgricultureFood Safety analysis
Crop forecasting
65 | 66 |

67 | 68 |

Technologies and Topics covered in this Workshop

69 | 70 | The information covered in this workshop includes the following technologies and topics - although you are not limited to these, they form the basis of the workshop. At the end of the workshop you will learn how to extrapolate these components into other solutions. You will cover these at an overview level, with references to much deeper training provided. 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 |
Technology/Concept Description
SQL Server 2019 improvements (on-prem and in-cloud)Covers challenges and solutions in performance, security, availability, data virtualization, and Linux and containers in SQL Server 2019 spanning on-premise, containers, Kubernetes, and cloud platforms.
Big Data Clusters for SQL Server (on-prem and in-cloud)Explains the architecture for Big Data Clusters for SQL Server in on-premises installations, containers, Kubernetes, and on the Microsoft Azure platform and in hybrid configurations
SQL Server Virtual Machines in Microsoft Azure Covers the tools, processes and procedures for SQL Server Virtual Machines on the Microsoft Azure platform
Azure SQL Covers the tools, processes and procedures for Azure SQL (Managed Instance, Single Database, and Elastic Pool)
Migrating SQL Server installations to Microsoft Azure Explains the tools and processes to migrate on-premises SQL Server installations to the Microsoft Azure platform
"What to Use When" Teaches a complete set of tools and processes used to determine the best architecture for a given customer scenario on the Microsoft Azure platform
83 | 84 |

85 | 86 |

Before Taking this Workshop

87 | 88 | You'll need a local system or Virtual Machine that you are able to install software on. The workshop demonstrations use Microsoft Windows as an operating system and all examples use Windows for the workshop. Optionally, you can use a Microsoft Azure Virtual Machine (VM) to install the software on and work with the solution. 89 | 90 | You must have a Microsoft Azure account with the ability to create assets, specifically the Azure Kubernetes Service (AKS). 91 | 92 | This workshop expects that you understand data structures and working with SQL Server and computer networks. This workshop does not expect you to have any prior data science knowledge, but a basic knowledge of statistics and data science is helpful in the Data Science sections. Knowledge of SQL Server, Azure Data and AI services, Python, and Jupyter Notebooks is helpful for the Big Data Clusters feature. AI techniques are implemented in Python packages. Solution templates are implemented using Azure services, development tools, and SDKs. You should have experience working with the Microsoft Azure Platform. 93 | 94 | If you are new to these, here are a few references you can complete prior to class: 95 | 96 | - [Microsoft SQL Server](https://docs.microsoft.com/en-us/sql/relational-databases/database-engine-tutorials?view=sql-server-ver15) 97 | - [Microsoft Azure](https://docs.microsoft.com/en-us/learn/paths/azure-fundamentals/) 98 | 99 | 100 |

Setup

101 | 102 | A full prerequisites document is located here. These instructions should be completed before the workshop starts, since you will not have time to cover these in class. Remember to "Stop" any Virtual Machines from the Azure Portal when not taking the class so that you do incur charges (shutting down the machine in the VM itself is not sufficient). 103 | 104 |

105 | 106 |

Intended Audience

107 | 108 |

The following roles will find this workshop useful. Others may also attend, as described in the Secondary Audience section.

109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 |
Primary Audience:Solution Architects and Data Professionals tasked with implementing modern Data Systems, Big Data, Machine Learning and AI solutions
Secondary Audience: Security Architects, Developers, and Data Scientists
Level: 300
Type:In-Person or Self-Paced
Length: 8-9 hours
119 | 120 |

121 | 122 |

Workshop Modules and In-Class Agenda

123 | 124 | This is a modular workshop, and in each section, you'll learn concepts, technologies and processes to help you complete the solution. The times shown below are for an instructor-led course, you may also take the modules in a self-paced fashion. 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 145 | 146 | 147 | 148 | 155 | 156 | 165 | 166 | 167 | 168 |
ModuleTimeTopics
01 - Introduction and Workshop Methodology 9:00AM-9:15AM Workshop introduction, logistics, setup check
02 - Modernizing Your Data Estate with SQL Server 2019 9:15AM-11:15AM This module covers challenges and solutions using the latest version of SQL Server including:
135 |
    136 |
  • Overall SQL Server 2019
    • 137 |
  • Intelligent Performance
    • 138 |
  • What's New in Security
    • 139 |
  • Mission Critical Availability
    • 140 |
  • Data Virtualization
    • 141 |
  • SQL Server Linux and Containers
    • 142 |
  • What Else, Migration and Next Steps
    • 143 |
144 |
03 - Working with Big Data and Data Science (Big Data Clusters for SQL Server 2019) 11:30AM-12:30PM Abstraction levels, frameworks, architectures and components within SQL Server big data clusters
04 - SQL Server on the Microsoft Azure Platform 1:30PM-2:30PM Covers the multiple ways to use SQL Server technologies on the Microsoft Azure Platform, along with the fundamentals of SQL in Azure with additional deeper resources provided. Topics covered include: 149 |
    150 |
  • Azure SQL: SQL VMs, Single instance, Single database, Elastic pools
    • 151 |
  • Fundamentals of SQL in Azure
    • 152 |
  • Migration process
    • 153 |
154 |
05 - Migrating SQL Server to Azure 2:45PM-3:45PM Covers the migration workflow and tools for assessing, planning, and migrating SQL workloads to Azure that meets the business requirements. Some of the tools and topics (not exhaustive) covered are: 158 |
    159 |
  • Azure Migrate
    • 160 |
  • Data Migration Assistant
    • 161 |
  • Azure Database Migration Service
    • 162 |
  • Post-migration operations
    • 163 |
164 |
06 - What to use When 3:45PM-5:00PM Covers the decision process and provides tools for deciding on the proper technologies on-premises and in-cloud for a solution based on requirements and constraints.
169 | 170 |

171 | 172 |

Related Workshops

173 | 174 | - [Other SQL Workshops by Microsoft](https://aka.ms/sqlworkshops) 175 | 176 |

177 | 178 |

Next Steps

179 | 180 | Next, Continue to Prerequisites 181 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Security 4 | 5 | Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/). 6 | 7 | If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below. 8 | 9 | ## Reporting Security Issues 10 | 11 | **Please do not report security vulnerabilities through public GitHub issues.** 12 | 13 | Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report). 14 | 15 | If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey). 16 | 17 | You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc). 18 | 19 | Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: 20 | 21 | * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) 22 | * Full paths of source file(s) related to the manifestation of the issue 23 | * The location of the affected source code (tag/branch/commit or direct URL) 24 | * Any special configuration required to reproduce the issue 25 | * Step-by-step instructions to reproduce the issue 26 | * Proof-of-concept or exploit code (if possible) 27 | * Impact of the issue, including how an attacker might exploit the issue 28 | 29 | This information will help us triage your report more quickly. 30 | 31 | If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs. 32 | 33 | ## Preferred Languages 34 | 35 | We prefer all communications to be in English. 36 | 37 | ## Policy 38 | 39 | Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd). 40 | 41 | 42 | -------------------------------------------------------------------------------- /graphics/microsoftlogo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/sqlworkshops-sqlg2c/ba18ed184b915c3d8841cb17e39f73c60c9a1265/graphics/microsoftlogo.png -------------------------------------------------------------------------------- /slides/SQL Server 2019 Modern Data Platform.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/sqlworkshops-sqlg2c/ba18ed184b915c3d8841cb17e39f73c60c9a1265/slides/SQL Server 2019 Modern Data Platform.pptx -------------------------------------------------------------------------------- /slides/SQL Server Ground to Cloud.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/sqlworkshops-sqlg2c/ba18ed184b915c3d8841cb17e39f73c60c9a1265/slides/SQL Server Ground to Cloud.pptx -------------------------------------------------------------------------------- /slides/SQL Server on the Microsoft Azure Platform.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/sqlworkshops-sqlg2c/ba18ed184b915c3d8841cb17e39f73c60c9a1265/slides/SQL Server on the Microsoft Azure Platform.pptx -------------------------------------------------------------------------------- /slides/g2c.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/sqlworkshops-sqlg2c/ba18ed184b915c3d8841cb17e39f73c60c9a1265/slides/g2c.pdf -------------------------------------------------------------------------------- /sqlgroundtocloud/00-prerequisites.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft Workshop from the SQL Server team 6 | 7 |

8 | 9 |

00 Prerequisites

10 | 11 | The *Workshop: SQL Ground-to-Cloud* workshop is taught using the following components, which you will install and configure in the sections that follow. 12 | 13 | > NOTE: If you take this Workshop in person, you may be provided with a complete environment, or have different instructions. The invitation where you registered for the Workshop will be followed up with details for that environment, otherwise assume that you need to complete all of the following steps to complete the Workshop. The Labs in this Workshop are optional, all screenshots and Notebooks are provided for you to follow along if you cannot meet these requirements. 14 | 15 | For this workshop, you will use Microsoft Windows as the base workstation, although Apple and Linux operating systems can be used in production. You can download a Windows 10 Workstation Image for VirtualBox, Hyper-V, VMWare, or Parallels for free here to use in the course. You must be able to install software on the system you will use for the course, so a Virtual Machine is often a good choice. This Workshop was built and designed for a server or VM to run SQL Server with at least 8Gb RAM and 4 CPUs. 16 | 17 | Note that you will need to be able to run the Docker program on your workstation, so [Nested Virtualization must be enabled on the system](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/nested-virtualization), or your physical workstation must be able to run Hyper-V images. 18 | 19 | Other requirements include: 20 | 21 | - **Microsoft Azure**: This Workshop uses the Microsoft Azure platform to host various components, and optionally you can deploy a system there to act as a workstation. You can use a free Azure account, an MSDN Account, your own account, or potentially one provided for you, as long as you can create about $100.00 (U.S.) worth of assets. 22 | 23 | > Note that all following activities must be completed prior to class - there will not be time to perform these operations during the workshop. 24 | 25 |

Activity 1: Set up a Microsoft Azure Account

26 | 27 | You have multiple options for setting up Microsoft Azure account to complete this workshop. You can [use a free account](https://azure.microsoft.com/en-us/free/), a Microsoft Developer Network (MSDN) account, a personal or corporate account, or in some cases a pass may be provided by the instructor. (Note: for most classes, the MSDN account is best) 28 | 29 | **Unless you are explicitly told you will be provided an account by the instructor in the invitation to this workshop, you must have your Microsoft Azure account set up before you arrive at class.** 30 | 31 |

Option 1 - Free Account

32 | 33 | The free account gives you twelve months of time, and a limited amount of resources. Set this up prior to coming to class, and ensure you can access it from the system you will bring to the class. 34 | 35 | - [Open this resource, and click the "Start Free" button you see there](https://azure.microsoft.com/en-us/free/) 36 | 37 | **NOTE: You can only use the Free subscription once, and it expires in 12 months. Set up your account per the instructions below, but ensure that you "Stop" any VM's in the Portal to ensure that you do not exceed the cost limits on this account. You will turn it off and on in the classroom per the instructor's directions.** 38 | 39 |

Option 2 - Microsoft Developer Network Account (MSDN) Account

40 | 41 | The best way to take this workshop is to use your [Microsoft Developer Network (MSDN) benefits if you have a subscription](https://marketplace.visualstudio.com/subscriptions). 42 | 43 | - [Open this resource and click the "Activate your monthly Azure credit" button](https://azure.microsoft.com/en-us/pricing/member-offers/credit-for-visual-studio-subscribers/) 44 | 45 |

Option 3 - Use Your Own Account

46 | 47 | You can also use your own account or one provided to you by your organization, but you must be able to create a resource group and create, start, and manage a Virtual Machine and Azure SQL components. 48 | 49 |

Option 4 - Use an account provided by your instructor

50 | 51 | Your workshop invitation may have instructed you that they will provide a Microsoft Azure account for you to use. If so, you will receive instructions that it will be provided. 52 | 53 | **Unless you received explicit instructions in your workshop invitations, you must create either a free, MSDN or Personal account. You must have an account prior to the workshop.** 54 | 55 |

Activity 2: Set up your environment

56 | 57 | In order to complete this workshop you need to install the following software: 58 | 59 | 1. Install [SQL Server 2019 CTP 3.2 or later](https://www.microsoft.com/en-us/sql-server/sql-server-2019#Install) with all features, *PolyBase Query Service for External Data* enabled, and ensure you have 20GB of drive space available. 60 | 61 | 2. Install SQL Server Management Studio (SSMS) 18.2 or higher from https://docs.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms. Several of the modules require features built only into SSMS. 62 | 63 | 3. Install Azure Data Studio June 2019 or higher from https://docs.microsoft.com/en-us/sql/azure-data-studio/download. T-SQL notebooks are used extensively in this course. 64 | 65 | 4. [Install Docker Desktop for Windows](https://docs.docker.com/docker-for-windows/install/). 66 | 67 | 5. Download the WideWorldImporters sample backup from https://github.com/Microsoft/sql-server-samples/releases/download/wide-world-importers-v1.0/WideWorldImporters-Full.bak 68 | 69 | 70 |

Activity 3: Clone the Workshop

71 | 72 | To complete this Workshop you will work from local files. You can either use the `git` program or a ZIP file, which you will save on your workstation. 73 | 74 |

Getting the Workshop files

75 | 76 | - You can [download the repository holding these files using this link](https://github.com/microsoft/sqlworkshops/archive/master.zip). 77 | 78 | - If you have git installed, The workshop git repository is: https://github.com/microsoft/sqlworkshops.git 79 | `git clone --config core.autocrlf=false https://github.com/microsoft/sqlworkshops.git` 80 | 81 |

Activity 4: Set up your environment for Modules 4 and 5

82 | 83 | > **Important Note!** If you are being provided an account by your instructor (see Option 4 in Activity 1), you do not need to complete this activity. 84 | 85 | Before arriving at the workshop, if you intend to complete `04-SQLServerOnTheMicrosoftAzurePlatform` and/or `05-MigratingToAzureSQL`, you must complete the prerequisites outlined [here](https://github.com/microsoft/sqlworkshops-azuresqllabs/blob/master/lab-files/Module4and5Prerequisites.md). Note that all screenshots are provided, so you can "audit" this part of the Workshop if you like. 86 | 87 |

Next Steps

88 | 89 | Next, Continue to 01 - Introduction and Workshop Methodology. 90 | -------------------------------------------------------------------------------- /sqlgroundtocloud/01-IntroductionAndWorkshopMethodology.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft workshop from the SQL Server team 6 | 7 |

8 | 9 |

01 - Introduction and Workshop Methodology

10 | 11 | In this workshop you'll cover using the latest SQL Server features, from on-premises to the cloud, for various solutions. You'll work through a series of modules that will lead you from working with the latest improvements in SQL Server, working with large sets of data (in on-premises, cloud, and hybrid configurations), to working with the SQL options in the Microsoft Azure cloud platform. You'll close out the workshop with a set of scenarios to discover "what to use when", with a complete set of resources to help you in your real-world implementations. Along the way you'll have Lab exercises walking you through a complete assessment and migration of an on-premises database to Azure SQL. 12 | 13 | In each module you'll get more references, which you should follow up on to learn more. Also watch for links within the text - click on each one to explore that topic. 14 | 15 |

16 | 17 |

Workshop Scenario

18 | 19 | The Workshop follows the process of evaluating a solution from "Ground To Cloud" - using various phases in a defined process: 20 | 21 | - 01 **Discovery**: The original statement of the problem from the customer 22 | - 02 **Envisioning**: A "blue-sky" description of what success in the project would look like. Often phrased as *"I can..."* statements 23 | - 03 **Architecture Design Session** (ADS): An initial layout of the technology options and choices for a preliminary solution 24 | - 04 **Proof-Of-Concept** (POC): After the optimal solution technologies and processes are selected, a POC is set up with a small representative example of what a solution might look like, as much as possible. If available, a currently-running solution in a parallel example can be used 25 | - 05 **Implementation**: Implementing a phased-in rollout of the completed solution based on findings from the previous phases 26 | - 06 **Handoff**: A post-mortem on the project with a discussion of future enhancements 27 | 28 | In these modules you'll work through each of these phases for a fictional company called [Wide World Importers](https://docs.microsoft.com/en-us/sql/samples/wide-world-importers-what-is?view=sql-server-2017) (WWI) - a wholesale novelty goods importer and distributor operating from the San Francisco bay area in the United States. WWI will need to integrate their operations with other companies, such as [The Contoso company](https://docs.microsoft.com/en-us/microsoft-365/enterprise/contoso-overview), a multi-national business with headquarters in Paris, France, and [Adventure Works Cycles](https://docs.microsoft.com/en-us/previous-versions/sql/sql-server-2008/ms124825(v=sql.100) ) - a large, multinational company that manufactures and sells metal and composite bicycles to North American, European and Asian commercial markets. Another company, [Tailspin Toys](http://tailspintoys.azurewebsites.net/), is the developer of several popular toys and online video games. Founded in 2010, and acquired shortly after by Wide World Importers, the company has experienced exponential growth since releasing the first installment of their most popular game franchise to include online multiplayer gameplay. They have since built upon this success by adding online capabilities to the majority of their game portfolio. 29 | 30 | Each of these companies have different challenges, and you'll get hands-on experience in the Labs with each of them. 31 | 32 | At the end of the Workshop, you'll be presented with additional needs from each of these companies, and get tools, processes and other assets that you can use to design your own unique solution. By the end of this Workshop, you'll have everything you need to repeat this process in real-world production scenarios. 33 | 34 |

35 | 36 |

Workshop Methodolgy

37 | 38 | This is a modular workshop, designed to be used in a linear fashion - you'll start at Module 1 and work your way through to the last one. You can, however, use these modules independently in many cases, and the Workshop exercises point off to Labs, which can often be taken independently as well. 39 | 40 | If you are working through the Workshop with an instructor, you'll get additional instructions on the Workshop and you may be provided with a special environment for the class. You'll be notified if that's the case when you sign up for the Workshop. Otherwise, if you are following along on your own, Make sure you check out the prerequisite page before you start. You'll need all of the items loaded there before you can proceed with the Workshop Labs. If you plan to just read through the Workshop, the Labs are optional. 41 | 42 |

43 | 44 |

Workshop Modules and Agenda

45 | 46 | This is a modular workshop, and in each section, you'll learn concepts, technologies and processes to help you complete the solution. The times shown below are for an instructor-led course, you may also take the modules in a self-paced fashion. 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 67 | 68 | 69 | 70 | 77 | 78 | 87 | 88 | 89 | 90 |
ModuleTimeTopics
01 - Introduction and Workshop Methodology 9:00AM-9:15AM Workshop introduction, logistics, setup check
02 - Modernizing Your Data Estate with SQL Server 2019 9:15AM-11:15AM This module covers challenges and solutions using the latest version of SQL Server including:
57 |
    58 |
  • Overall SQL Server 2019
    • 59 |
  • Intelligent Performance
    • 60 |
  • What's New in Security
    • 61 |
  • Mission Critical Availability
    • 62 |
  • Data Virtualization
    • 63 |
  • SQL Server Linux and Containers
    • 64 |
  • What Else, Migration and Next Steps
    • 65 |
66 |
03 - Working with Big Data and Data Science (Big Data Clusters for SQL Server 2019) 11:30AM-12:30PM Abstraction levels, frameworks, architectures and components within SQL Server big data clusters
04 - SQL Server on the Microsoft Azure Platform 1:30PM-2:30PM Covers the multiple ways to use SQL Server technologies on the Microsoft Azure Platform, along with the fundamentals of SQL in Azure with additional deeper resources provided. Topics covered include: 71 |
    72 |
  • Azure SQL: SQL VMs, Single instance, Single database, Elastic pools
    • 73 |
  • Fundamentals of SQL in Azure
    • 74 |
  • Migration process
    • 75 |
76 |
05 - Migrating SQL Server to Azure 2:45PM-3:45PM Covers the migration workflow and tools for assessing, planning, and migrating SQL workloads to Azure that meets the business requirements. Some of the tools and topics (not exhaustive) covered are: 80 |
    81 |
  • Azure Migrate
    • 82 |
  • Data Migration Assistant
    • 83 |
  • Azure Database Migration Service
    • 84 |
  • Post-migration operations
    • 85 |
86 |
06 - What to use When 3:45PM-5:00PM Covers the decision process and provides tools for deciding on the proper technologies on-premises and in-cloud for a solution based on requirements and constraints.
91 | 92 |

93 | 94 |

Next Steps

95 | 96 | Next, Continue to 02 - Modernizing Your Data Estate with SQL Server 2019. 97 | 98 |
99 | 100 |
101 | -------------------------------------------------------------------------------- /sqlgroundtocloud/02-ModernizingYourDataEstateWithSQLServer2019.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft workshop from the SQL Server team 6 | 7 |

8 | 9 |

02 - Modernizing Your Data Estate with SQL Server 2019

10 | 11 | SQL Server 2019 is a modern data platform designed to tackle the challenges of today's data professional. SQL Server 2019 is not just an upgrade option from previous releases of SQL Server. 12 | 13 | The WideWorldImporters company has been evaluating upgrading several SQL Server 2008 instances along with some instances of SQL Server 2012, 2014, and even 2016. WideWorldImporters wants to know whether SQL Server 2019 will solve some of the challenges they face including: 14 | 15 | - Improving query performance without having to make application changes 16 | - Classifying key data columns and being able to audit access to these columns to meet the needs of GDPR compliance. 17 | - Providing better database availability when applications use long-running transactions. 18 | - Allowing the WideWorldImporters team to access data from sources like Oracle, Azure SQL Database, and Azure CosmosDB without having to develop expensive ETL jobs. 19 | - The database team at WideWorldImporters has evaluated SQL Server on Linux but SQL Server 2017 didn't include Replication, a feature they need for their application. They would like to evaluate how SQL Server Replication on Linux works. They also want to understand more about container technology and how it can be used with SQL Server 20 | - WideWorldImporters also would like to know other capabilities exist in SQL Server 2019 that might help them before more efficient and extend the capabilities of T-SQL. 21 | - WideWorldImporters also would like to learn more about how they can plan and execute a migration to SQL Server 2019 and reduce their risk for upgrades. 22 | - WideWorldImporters is also evaluating Azure SQL so wants to know what features in SQL Server 2019 also work in Azure. 23 | 24 | In this workshop module, you will see the capabilities in SQL Server 2019 that can provide solutions for these challenges. 25 | 26 | > **NOTE**: Make sure you check out the prerequisite page before you start. You'll need all of the items loaded there before you can proceed with the workshop. Instructor led workshops may have provided all the resources in the prerequisites. 27 | 28 |

29 | 30 |

Activity: SQL Server 2019 Lab

31 | 32 | In this activity, you will complete a series of modules from the **SQL Server 2019 Lab**. 33 | 34 | Instructor led workshops will use modules from the SQL Server 2019 lab along with the PowerPoint slides SQL Server 2019 Modern Data Platform. Instructor led labs may not go through all modules in the SQL Server 2019 lab. 35 | 36 |

Activity Steps

37 | 38 | To complete this module, go to [https://github.com/microsoft/sqlworkshops/blob/master/sql2019lab/README.md](https://github.com/microsoft/sqlworkshops-sql2019lab). 39 | 40 | When you are done with the SQL Server 2019 lab you have completed Module 02 of the SQL Server Ground to Cloud Workshop! Proceed to the next module using the link in Next Steps below. 41 | 42 |

43 | 44 |

For Further Study

45 | 46 | 52 | 53 |

54 | 55 |

Next Steps

56 | 57 | Next, Continue to 03 - Working with Big Data Clusters on SQL Server 2019. 58 | -------------------------------------------------------------------------------- /sqlgroundtocloud/03-WorkingWithBigDataAndDataScienceBigDataClustersForSQLServer2019.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft workshop from the SQL Server team 6 | 7 |

8 | 9 |

03 - Working with Big Data and Data Science - Big Data Clusters for SQL Server 2019

10 | 11 | In this Module of the Workshop you'll cover using SQL Server on-premises and in-cloud configurations, as well as hybrid applications as a solution for data processing. In each section you'll get more references, which you should follow up on to learn more. Also watch for links within the text - click on each one to explore that topic. The end of this module contains several helpful references you can use in this course and in production. 12 | 13 | (Make sure you check out the prerequisite page before you start. You'll need all of the items loaded there before you can proceed with the workshop.) 14 | 15 | In this module you'll cover working with Data Science workloads with a focus on larger sets of data. Starting in SQL Server 2019, big data clusters allows for large-scale, near real-time processing of data over the HDFS file system and other data sources. It also leverages the Apache Spark framework which is integrated into one environment for management, monitoring, and security of your environment. This means that organizations can implement everything from queries to analysis to Machine Learning and Artificial Intelligence within SQL Server, over large-scale, heterogeneous data. SQL Server big data clusters can be implemented fully on-premises, in the cloud using a Kubernetes service such as Azure's AKS, and in a hybrid fashion. This allows for full, partial, and mixed security and control as desired. 16 | 17 | > Note: This is a complex topic, so you can find a complete workshop on this topic here. 19 | 20 |

21 | 22 |

3.1 Data Science and Big Data Processing

23 | 24 | Businesses require near real-time insights from ever-larger sets of data from a variety of sources. Large-scale data ingestion requires scale-out storage and processing in ways that allow fast response times. In addition to simply querying this data, organizations want full analysis and even predictive capabilities over their data. Machine Learning, Artificial Intelligence, and Deep Learning techniques all require large sets of data for training their models to be effective. Two technologies have emerged as the primary methods for processing large sets of data, using a scale-out paradigm - Hadoop and Spark. 25 | 26 | Hadoop uses a set of computing nodes that position the workload over distributed data nodes: 27 | 28 |

29 | 30 | Spark is a technology that uses various libraries to make processing over distributed storage more efficient and faster: 31 | 32 |
33 |

34 |
35 | 36 | Both of these technologies assume many nodes (computers), and since you only need the computation elements (not all the drivers and other components of a full computer or Virtual Machine), Container technologies work well for this solution. 37 | 38 |
39 | 40 |
41 | 42 | To control containers, a technology called Kubernetes is used for deployment, management and storage of a grouping of containers, called a Cluster. 43 | 44 |
45 |

46 |

3.2 SQL Server 2019 Big Data Clusters Architecture

47 | 48 | Using the technologies described above, SQL Server uses a Kubernetes Cluster to deploy multiple components for SQL Server processing, distributed queries using PolyBase, Spark, and Storage on HDFS for a complete environment to work with large sets of data, and includes Machine Learning, Artificial Intelligence and Deep Learning capabilities. 49 | 50 |
51 | 52 |
53 | 54 | A SQL Server Big Data Cluster (BDC) can be deployed to multiple environments, in-cloud and on-premises: 55 | 56 | - In a Cloud Service (Such as the Azure Kubernetes Service or AKS) 57 | - On premises (using KubeADM) 58 | 59 | These architectures are not mutually exclusive - you can install some components on-premises, and others as a service. Your connections can interconnect across these environments. 60 | 61 |

3.3 SQL Server 2019 Big Data Clusters Programming

62 | 63 | In addition to traditional OLTP workloads, the SQL Server Big Data Cluster has three other uses that work with data at scale. 64 | 65 |

Data Virtualization with SQL Server Big Data Clusters

66 | 67 | Using the Data Virtualization capability of PolyBase in SQL Server Big Data Clusters you create External Tables for data outside the Cluster. These External Table definitions are stored in the database on the SQL Server Master Instance within the cluster. When queried by the user, the queries are engaged from the SQL Server Master Instance through the Compute Pool in the SQL Server BDC, which holds Kubernetes Nodes containing the Pods running SQL Server Instances. These Instances send the query to the PolyBase Connector at the target data system, which processes the query based on the type of target system. The results are processed and returned through the PolyBase Connector to the Compute Pool and then on to the Master Instance, and then on to the user. 68 | 69 |
70 | 71 |
72 | 73 | This process allows not only a query to disparate systems, but also those remote systems can hold extremely large sets of data. Normally you are querying a subset of that data, so the results are all that are sent back over the network. These results can be joined with internal tables for a single view, and all from within the same Transact-SQL statements. 74 | 75 |

Data Marts with SQL Server Big Data Clusters

76 | 77 | Ad-hoc queries over larges sets of normalized data are very useful for many scenarios. There are times when you would like to bring the data into storage, so that you can create denormalized representations of datasets, aggregated data, and other purpose-specific data tasks. 78 | 79 | Using the Data Virtualization capability (PolyBase), the IT team creates External Tables using PolyBase statements. These External Table definitions are stored in the database on the SQL Server Master Instance within the cluster. When queried by the user, the queries are engaged from the SQL Server Master Instance through the Compute Pool in the SQL Server BDC, which holds Kubernetes Nodes containing the Pods running SQL Server Instances. These Instances send the query to the PolyBase Connector at the target data system, which processes the query based on the type of target system. The results are processed and returned through the PolyBase Connector to the Compute Pool and then on to the Master Instance, and the PolyBase statements can specify the target of the Data Pool. The SQL Server Instances in the Data Pool store the data in a distributed fashion across multiple databases, called Shards. 80 | 81 |
82 | 83 |
84 | 85 |

Data Science and Apache Spark with SQL Server Big Data Clusters

86 | 87 | There are various uses for a large cluster of data processing systems for Machine Learning and AI applications. One primary use-case is the creation of the Features and Labels used in various ML and AI algorithms. Large sets of data stored in the Data Pool allow Python, R and Spark calls and libraries are available for this task in BDC. This process is often called Data Engineering. 88 | 89 | The SQL Server Master Instance in the BDC installs with Machine Learning Services, which allow creation, training, evaluation and persisting of Machine Learning Models. Data from all parts of the BDC are available, and Data Science oriented languages and libraries in R, Python and Java are enabled. In this scenario, the Data Scientist creates the R or Python code, and the Transact-SQL Developer wraps that code in a Stored Procedure. This code can be used to train, evaluate and create Machine Learning Models. The Models can be stored in the Master Instance for scoring, or sent on to the App Pool where the Machine Learning Server is running, waiting to accept REST-based calls from applications. 90 | 91 | The Data Scientist has another option to create and train ML and AI models. The Spark platform within the Storage Pool is accessible through the Knox gateway, using Livy to send Spark Jobs. This gives access to the full Spark platform, using Jupyter Notebooks (included in Azure Data Studio) or any other standard tools that can access Spark through REST calls. 92 | 93 |
94 | 95 |
96 | 97 |

Activity: Big Data Clusters for SQL Server Lab Review

98 | 99 | In this lab, you will review a series of Jupyter Notebooks you can open in Azure Data Studio to work with a Big Data Cluster. It demonstrates loading data, querying SQL Server databases, accessing HDFS using PolyBase, loading data into the Data Pool, working with Spark, and implementing a Data Science Machine Learning Model in Python. 100 | 101 | Note: This lab assumes a completely installed and function SQL Server Big Data Cluster if you wish to run the Notebooks, otherwise review the completed Notebooks at the reference. 102 | 103 |

Steps

104 | 105 | Open this reference and review the notebooks you find there. 107 | 108 |

109 | 110 |

For Further Study

111 | 120 | 121 | 122 |

Next Steps

123 | 124 | Next, Continue to 04 - SQL Server on the Microsoft Azure Platform. -------------------------------------------------------------------------------- /sqlgroundtocloud/04-SQLServerOnTheMicrosoftAzurePlatform.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft workshop from the SQL Server team 6 | 7 |

8 | 9 |

04 - SQL Server on the Microsoft Azure Platform

10 | 11 | In the previous modules, you learned about SQL Server 2019, big data, and more. In this module, you'll learn more about Azure SQL, including the benefits, the options, and how to get there. Near the end of the module, you'll learn how to assess your on-premises estate, through an example with the Tailspin Toys Gaming division of Wide World Importers, with tools like [Azure Migrate](https://docs.microsoft.com/en-us/azure/migrate/migrate-services-overview) and [Data Migration Assistant](https://docs.microsoft.com/en-us/sql/dma/dma-overview?view=sql-server-2017). The module is broken up into the following sections: 12 | 13 | [4.1](#4.1): Introduction to Azure SQL 14 | [4.2](#4.2): Azure SQL Platform Benefits 15 | [4.3](#4.3): Migrating to Azure SQL 16 | [4.4](#4.4): Database Discovery and Assessment 17 | 18 | (Make sure you check out the Prerequisites page before you start. You'll need all of the items loaded there before you can proceed with the workshop.) 19 | 20 | SELF-PACED USERS ONLY: If you are using this module self-paced, carefully read through Module 4 of this workshop and the references provided before completing the lab in Module 4.4. You can also check out the [slides that will be presented in a live workshop](https://github.com/microsoft/sqlworkshops/blob/master/SQLGroundToCloud/slides/SQL%20Server%20on%20the%20Microsoft%20Azure%20Platform.pptx). Then, continue to Module 5. 21 | 22 | > **Note**: 23 | > While all the deployment options will be referenced, the focus of this module is on **Azure SQL Managed Instance**, with some details that are also related to **Azure SQL Database (single database)**. 24 | 25 |

26 | 27 |

4.1 Introduction to Azure SQL

28 | 29 | In this section, you'll learn about the different Azure SQL deployment options and service tiers available for Tailspin Toys Gaming, a division of Wide World Importers that focuses on online gameplay. You'll start with understanding Tailspin Toys Gaming's scenario deeper, so you can determine which options are best suited to their business needs. 30 | 31 | ### Scenario Review: Tailspin Toys Gaming 32 | 33 | Tailspin Toys is the developer of several popular online video games. Founded in 2010, and acquired shortly after by Wide World Importers, the company has experienced exponential growth since releasing the first installment of their most popular game franchise to include online multiplayer gameplay. They have since built upon this success by adding online capabilities to the majority of their game portfolio. 34 | 35 | Adding online gameplay has greatly increased popularity of their games, but the rapid increase in demand for their services has made supporting the current setup problematic. To facilitate online gameplay, they host gaming services on-premises using rented hardware. For each game, their gaming services setup consists of three virtual machines running the gaming software and five game databases hosted on a single SQL Server 2008 R2 instance. In addition to the dedicated gaming VMs and databases, they also host authentication and gateway VMs and databases, which are shared by all their games. At its foundation, Tailspin Toys is a game development company, made up primarily of software developers. The few dedicated database and infrastructure resources they do have are struggling to keep up with their ever-increasing workload. 36 | 37 | Tailspin Toys is hoping that migrating their services from on-premises to the cloud can help to alleviate some of their infrastructure management issues, while simultaneously helping them to refocus their efforts on delivering business value by releasing new and improved games. They are looking for a proof-of-concept (PoC) for migrating their gamer information web application and database into the cloud. They maintain their gamer information database, `TailspinToys`, on an on-premises SQL Server 2008 R2 database. This system is used by gamers to update their profiles, view leader boards, purchase game add-ons and more. Since this system helps to drive revenue, it is considered a business-critical application, and needs to be highly-available. They are aware that SQL Server 2008 R2 has reached end of support, and are looking at options for migrating this database into Azure. They have read about some of the advanced security and performance tuning options that are available only in Azure and would prefer to a migrate the database into a platform-as-a-service (PaaS) offering, if possible. Tailspin Toys is using the Service Broker feature of SQL Server for messaging within the `TailspinToys` database. This functionality is being used for several critical processes, and they cannot afford to lose this capability when migrating their operations database to the cloud. They have also stated that, at this time, they do not have the resources to rearchitect the solution to use an alternative message broker. 38 | 39 | > If you'd like to conduct a deep whiteboard design session for this scenario, it is available [here](https://github.com/microsoft/MCW-Migrating-SQL-databases-to-Azure/tree/master/Whiteboard%20design%20session). 40 | 41 | ### Azure SQL deployment options 42 | 43 | Tailspin Toys has requested your help in determining where and how they should land their data in Azure. This is a decision that all businesses moving to the cloud have to make, and the result will depend on their unique business requirements. Microsoft has recently introduced Azure SQL, which brings all the SQL Server products in Azure under one suite. If you're already familiar with [Azure SQL Database](https://azure.microsoft.com/en-us/services/sql-database/), this slight shift means that Azure SQL also includes [SQL Server on Virtual Machines](https://azure.microsoft.com/en-us/services/virtual-machines/sql-server/) (also referred to as Azure SQL VMs), which Microsoft is continuously investing in and enhancing the benefits associated with it. 44 | 45 | 46 | There are several different options within Azure SQL, and one visual is shown below. 47 | ![Azure SQL Overview](https://github.com/microsoft/sqlworkshops-azuresqllabs/blob/master/graphics/azuresql-overview.png?raw=true) 48 | 49 | > Recently, we announced [Azure SQL Database Instance Pools in preview](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-instance-pools), which provide a convenient and cost-efficient way to migrate smaller SQL instances to the cloud at scale. 50 | 51 | 52 | Every business is going to have unique requirements that make some options better for them than others. There won't necessarily be one 'correct' answer, but there are tradeoffs to consider. Let's look at some examples of Azure SQL customers and what they decided: 53 | 54 | * [**Allscripts**](https://customers.microsoft.com/en-us/story/allscripts-partner-professional-services-azure): Allscripts is a leading healthcare software manufacturer, serving physician practices, hospitals, health plans, and Big Pharma. To transform its applications frequently and host them securely and reliably, Allscripts wanted to move to Azure quickly. In just three weeks, the company lifted and shifted dozens of acquired applications running on 1,000 virtual machines to Azure with [Azure Site Recovery](https://azure.microsoft.com/en-us/services/site-recovery/). After the migration to **Azure SQL VMs**, Allscripts began to evaluate and test **Azure SQL Managed Instance**, and started to move some workloads there. 55 | * [**Komatsu**](https://customers.microsoft.com/en-us/story/komatsu-australia-manufacturing-azure): Komatsu is a manufacturing company that produces and sells heavy equipment for construction. They had multiple mainframe applications for different types of data, which they wanted to consolidate to get a holistic view. Additionally, they wanted a way reduce overhead. Because Komatsu uses a large surface area of SQL Server features, they chose to move to **Azure SQL Managed Instance**. They were able to move about 1.5 terabytes of data smoothly, and [start enjoying benefits like automatic patching and version updates, automated backups, high availability, and reduced management overhead](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-technical-overview). After migrating, they reported ~49% cost reduction and ~25-30% performance gains. 56 | * [**AccuWeather**](https://customers.microsoft.com/en-us/story/accuweather-partner-professional-services-azure): AccuWeather has been analyzing and predicting the weather for more than 55 years. They wanted access to the rich and rapidly advanced platform of Azure that includes big data, machine learning, and AI capabilities. They want to focus on building new models and applications, not managing databases. They selected **Azure SQL Database [**(single database)**](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-single-index)** to use with other services, like [Azure Data Factory](https://docs.microsoft.com/en-us/azure/data-factory/) and [Azure Machine Learning Services](https://docs.microsoft.com/en-us/azure/machine-learning/service/), to quickly and easily deploy new internal applications to make sales and customer predictions. 57 | * [**Paychex**](https://customers.microsoft.com/en-us/story/paychex-azure-sql-database-us): Paychex is a human capital management firm that serves more than 650,000 businesses across the US and Europe. They needed a way to separately manage the time and pay management for each of their businesses, and cut costs. They opted for **Azure SQL Database** [**(elastic pools)**](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-elastic-pool), which allowed them to simplify the management and enable resource sharing between separate databases to lower costs. 58 | 59 | If you want to dive deeper into the deployment options and how to choose, check out the following resources: 60 | * [Blog announcement for Azure SQL](https://techcommunity.microsoft.com/t5/Azure-SQL-Database/Unified-Azure-SQL-experience/ba-p/815368) which explains and walks through Azure SQL and some of the resulting views and experiences available in the Azure portal. 61 | * [Microsoft Customer Stories](https://customers.microsoft.com/en-us/home?sq=&ff=&p=0) for many more stories similar to the ones above. You can use this to explore various use cases, industries, and solutions. 62 | * [Choose the right deployment option in Azure SQL](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-paas-vs-sql-server-iaas) is a page in the documentation regularly updated to help provide insight into making the decisions between the Azure SQL options. 63 | * [Choosing your database migration path to Azure](https://azure.microsoft.com/mediahandler/files/resourcefiles/choosing-your-database-migration-path-to-azure/Choosing_your_database_migration_path_to_Azure.pdf) is a white paper that talks about tools for discovering, assessing, planning and migrating SQL databases to Azure. This workshop will refer to it several times, and it's a highly recommended read. Chapter 5 deeply discusses choosing the right deployment option. 64 | * [Feature comparison between SQL database, SQL managed instance, and SQL Server](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-features) 65 | * There are also tools covered later in this module that can help in recommending a deployment option. 66 | 67 | 68 | ### Azure SQL service tiers and other options 69 | 70 | Once you have an understanding of the deployment options, it's time to consider some of the details within the service tiers that are available. 71 | 72 | #### SQL Server on Virtual Machines options 73 | 74 | Azure SQL VMs don't have specific service tiers, but there are decisions to be made around which images to choose, how to configure storage and what sizes, etc. This isn't the focus of this workshop, but if you're considering Azure SQL VMs, you'll want to review the [guidance on images to choose from](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-server-iaas-overview), the [quick checklist](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-performance) to obtain optimal performance of Azure SQL VMs, and the guidance for [storage configuration](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-server-storage-configuration). 75 | 76 | Recently, [Resource Provider](http://www.aka.ms/sqlvm_rp_documentation) was [announced](http://www.aka.ms/sqlvm_rp), which brings the functionality of Azure Marketplace images to SQL Server instances self-installed on Azure VMs. 77 | 78 | > Note: If you're specifically looking at SQL Server on RHEL Azure VMs, there's a full operations guide available [here](https://azure.microsoft.com/en-us/resources/sql-server-on-rhel-azure-vms-operations-guide/ 79 | ). 80 | 81 | > Note: In an earlier module of this workshop, you learned about some of the problems SQL Server 2019 is solving. The same applies in an Azure SQL VM (if you choose 2019 as the target). 82 | 83 | #### Azure SQL Database options 84 | 85 | For Azure SQL Database, which is the focus of this module, there are several options and tiers available, and the choices will depend on the scenario. 86 | 87 | There are a few main decisions to be made, which will be explored next. 88 | 89 | *Decision 1: Choose the purchasing model* 90 | You have two options, [virtual core (vCore)-based](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tiers-vcore) (recommended) or [Database transaction unit (DTU)-based](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tiers-dtu 91 | ). 92 | 93 | The vCore-based model is recommended because it allows you to independently choose compute and storage resources, while the DTU-based model is a bundled measure of compute, storage and I/O resources, which means you have less control over paying only for what you need. This model also allows you to use [Azure Hybrid Benefit for SQL Server](https://azure.microsoft.com/pricing/hybrid-benefit/) to gain cost savings. In the [vCore model](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tiers-vcore), you pay for: 94 | 95 | * Compute resources (the service tier + the number of vCores and the amount of memory + the generation of hardware). 96 | * The type and amount of data and log storage. 97 | * Backup storage ([read-access, geo-redundant storage (RA-GRS)](https://docs.microsoft.com/en-us/azure/storage/common/storage-designing-ha-apps-with-ragrs)). 98 | 99 | For the purposes of this workshop, we'll focus on the vCore purchasing model (recommended), but you can [compare vCores and DTUs here](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-purchase-models 100 | ). 101 | 102 | *Decision 2: Choose service tier for performance and availability* 103 | There are three tiers available in the vCore model for Azure SQL Database: 104 | * **[General purpose](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tier-general-purpose)**: Most business workloads. Offers budget-oriented, balanced, and scalable compute and storage options. 105 | * **[Business critical](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tier-business-critical)**: Business applications with low-latency response requirements. Offers highest resilience to failures by using several isolated replicas. This is the only tier that can leverage [in-memory OLTP](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-in-memory) to improve performance. 106 | * **[Hyperscale](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tier-hyperscale)**: Most business workloads with highly scalable storage and read-scale requirements. *Currently only available for single databases, not managed instances*. 107 | 108 | A member of the Product Group recently released a [blog](https://azure.microsoft.com/en-gb/blog/understanding-and-leveraging-azure-sql-database-sla/) and [video](https://www.youtube.com/watch?v=l7FUNJd5TSE) explaining the SLA (service level agreements that set an expectation for uptime and performance). This resource will help you make an informed decision about which tier to move to. 109 | 110 | For a deeper explanation between the three tiers (including scenarios), you can also refer to the [service-tier characteristics](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tiers-vcore#service-tier-characteristics) in the documentation. 111 | 112 | *Decision 3: Provisioned or serverless compute?* 113 | If you choose **General Purpose within Single databases** and the **vCore-based model**, you have an additional decision to make regarding the compute that you pay for: 114 | * **Provisioned compute** is meant for more regular usage patterns with higher average compute utilization over time, or multiple databases using elastic pools. 115 | * **Serverless compute** is meant for intermittent, unpredictable usage with lower average compute utilization over time. Serverless has auto-pause and resume capabilities (with a time delay you set), meaning when your database is paused, you only pay for storage. 116 | 117 | For a deeper explanation between the two compute options (including scenarios), you can refer to the detailed [comparison in the documentation](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-serverless#comparison-with-provisioned-compute-tier). 118 | 119 | > If you're looking for compute cost saving opportunities, you can prepay for compute resources with [Azure SQL Database reserved capacity](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-reserved-capacity). 120 | 121 | *Decision 4: Choose hardware generation* 122 | The vCore model lets you choose the generation of hardware: 123 | * **Gen4**: Up to 24 logical CPUs based on Intel E5-2673 v3 (Haswell) 2.4-GHz processors, vCore = 1 PP (physical core), 7 GB per core, attached SSD 124 | * **Gen5**: Up to 80 logical CPUs based on Intel E5-2673 v4 (Broadwell) 2.3-GHz processors, vCore = 1 LP (hyper-thread), 5.1 GB per core, fast eNVM SSD 125 | 126 | Basically, Gen4 hardware offers substantially more memory per vCore. However, Gen5 hardware allows you to scale up compute resources much higher. 127 | 128 | > Note: If you choose General Purpose within Single databases and want to use the serverless compute tier, Gen5 hardware is the only option. 129 | 130 | *Summary* 131 | As you've hopefully noticed, while there are a lot of options, Azure is able to provide flexibility so you get exactly what you need, nothing less. A summary of the service tier options with some additional considerations is included below, but be sure to check out [pricing information](https://azure.microsoft.com/en-us/pricing/details/sql-database/managed/) for the latest details. 132 | 133 | ![Azure SQL tiers](https://github.com/microsoft/sqlworkshops-azuresqllabs/blob/master/graphics/service-tiers-summary.png?raw=true) 134 | 135 | 136 | > Note: Data Migration Assistant (covered later in this module) runs scans that can help you choose some of the options as well as the SKU. [Learn more here](https://docs.microsoft.com/en-us/sql/dma/dma-sku-recommend-sql-db?view=sql-server-2017). 137 | 138 | Tailspin Toys Gaming is leaning towards Azure SQL Managed Instance, because it supports movement to Azure without changing the database features that are used (in this case, Service Broker). They think business critical is the right tier for them, as they need to maximize performance and availability. By using the vCore-based model, they're able to leverage their existing investments with Azure Hybrid Benefits (1 Enterprise license core for SQL Server = 1 Business Critical vCore). Additionally, they want to leverage the three replicas provided free with business critical, where one can be used for [read scale-out](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-read-scale-out), that is needed as the game experiences heavy read workloads. 139 | 140 | 141 | 142 |

143 | 144 |

4.2 Azure SQL Platform Benefits

145 | 146 | Before Tailspin Toys Gaming commits to moving to Azure SQL Managed Instance, they want to understand some of the benefits that come with running their SQL workloads in Azure. 147 | 148 | 149 | In this section, we will highlight some of the key benefits of choosing Azure for your SQL workloads. If you want to go deeper, additional reference information is available. 150 | 151 | 152 | 153 | Azure SQL Database (including single databases, elastic pools, and managed instances) is a fully managed Database Engine that automates most of the database management functions such as upgrading, patching, backups, and monitoring. Some of the built-in capabilities include: 154 | * [Business continuity](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-business-continuity) enables your business to continue operating in the face of disruption, particularly to its computing infrastructure. 155 | * [High availability](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-high-availability) of Azure SQL Database guarantees your databases are up and running 99.99% of the time, no need to worry about maintenance/downtimes. 156 | * [Automated backups](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-automated-backups) are created and use Azure read-access geo-redundant storage (RA-GRS) to provide geo-redundancy. 157 | * [Long term backup retention](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-long-term-retention) enables you to store specific full databases for up to 10 years. 158 | * [Geo-replication](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-active-geo-replication) by creating readable replicas of your database in the same or different data center (region). 159 | * [Scale](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-scale-resources) by easily adding more resources (CPU, memory, storage) without long provisioning. 160 | * Network Security 161 | * [Azure SQL Database (single database and elastic pool)](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-security-overview#network-security) provides firewalls to prevent network access to the database server until access is explicitly granted based on IP address or Azure Virtual Network traffic origin. 162 | * [Azure SQL Managed Instance](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-connectivity-architecture) has an extra layer of security in providing native virtual network implementation and connectivity to your on-premises environment using [Azure ExpressRoute](https://docs.microsoft.com/en-us/azure/expressroute/) or [VPN Gateway](https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways). 163 | * [Advanced security](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-security-index) detects threats and vulnerabilities in your databases and enables you to secure your data. 164 | * [Automatic tuning](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-automatic-tuning) analyzes your workload and provides you the recommendations that can optimize performance of your applications by adding indexes, removing unused indexes, and automatically fixing the query plan issues. 165 | * [Built-in monitoring](https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-sql) capabilities enable you to get the insights into performance of your databases and workload, and troubleshoot the performance issues. 166 | * [Built-in intelligence](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-intelligent-insights) automatically identifies the potential issues in your workload and provides you the recommendations that can [help you to fix the problems](https://azure.microsoft.com/en-us/blog/ai-helped-troubleshoot-an-intermittent-sql-database-performance-issue-in-one-day/). 167 | 168 | > Note: Many benefits typically thought of as Platform as a Service (PaaS) are surfacing in Infrastructure as a Service (IaaS). You can learn more about automated updates, automated backups, high availability, and performance provided in Azure SQL VMs [here](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-server-iaas-overview). 169 | 170 | In addition to the resources linked to above, several [slide decks](https://aka.ms/azuresqlslides) are available (with notes and animations) that you can review to learn more. You can also check out the [Core Cloud Services - Azure architecture and service guarantees](https://docs.microsoft.com/en-us/learn/modules/explore-azure-infrastructure/) module from Microsoft Learn. 171 |
172 | 173 |

174 | 175 |

4.3 Migrating to Azure SQL

176 | 177 | Tailspin Toys has spent some time with you learning more about the benefits of Azure SQL. Based on their research, they've decided to migrate the Tailspin Toys database to Azure as a PoC for migrating larger workloads. 178 | 179 | The general process for [migrating assets to Azure](https://docs.microsoft.com/en-us/azure/architecture/cloud-adoption/getting-started/migrate) (whether it's apps, infrastructure, or data) is as follows: 180 | * **Assess**: Evaluate an existing asset and establish a plan for migration of the asset. 181 | * **Migrate**: Replicate the functionality of an asset in the cloud. 182 | * **Optimize**: Balance the performance, cost, access, and operational capacity of a cloud asset. 183 | * **Secure and manage**: Ensure a cloud asset is ready for ongoing operations. 184 | 185 | 186 | In guiding Tailspin Toys through the *data* migration journey, it's helpful to leverage the following process that is specific to data: 187 | ![Data migration process](https://github.com/microsoft/sqlworkshops-azuresqllabs/blob/master/graphics/data-migration-process.png?raw=true) 188 | 189 | Below you'll find a detailed survey of some of the resources and tips available for going through each phase of the migration journey to Azure (you can find a [full list of tools based on phase and target here](https://docs.microsoft.com/en-us/azure/dms/dms-tools-matrix)). The number of resources shows the dedication Microsoft and partners have to helping you move successfully to the cloud. 190 | 191 | In an effort to simplify, we recommend using the [Data migration guide](https://datamigration.microsoft.com/) which allows you to pick your source and target, and then guides you step-by-step through your specific migration journey. 192 | 193 | Additionally, you can check out the [Azure migration center](https://azure.microsoft.com/en-us/migration/) which includes detailed information about the migration journey, and the [Azure Migrate Program (AMP)](https://azure.microsoft.com/en-us/migration/migration-program/) where you can get support, migration partners, webinars, and more. 194 | 195 | With your help, Tailspin Toys Gaming decided to apply for the Azure Migrate Program, to get support and guidance from Microsoft and partners. Then, they leveraged the integration of Azure Migrate, Data Migration Assistant, and Azure Database Migration Service to migrate their databases. 196 | 197 | Once the Tailspin Toys Gaming division has migrated to Azure, the overall corporation may choose to scale the migration to Azure. In that case, [refer to guidance here](https://docs.microsoft.com/en-us/azure/architecture/cloud-adoption/migrate/azure-best-practices/contoso-migration-scale 198 | ). 199 | 200 | 201 | Below, you'll find a reference chart that explains each step in the three-phase process following the diagram above. Each phase contains steps with descriptions and helpful resources. *This chart is not meant to be consumed in its entirety here, but feel free to follow what is interesting or relevant to your needs, or refer back to it later*. 202 | 203 | 204 | | STAGE | PHASE | DESCRIPTION | RESOURCES | 205 | |----------------|-----------------------------------|----------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 206 | | Pre-migration | Discover | Inventory database assets, and application stack discovery. |
  • [Azure Migrate](https://docs.microsoft.com/azure/migrate/migrate-overview) for managing the whole migration process
    • | 207 | | | Assess | Assess workloads and fix recommendations. |
  • [MAP Toolkit for inventory, assessment, and reporting](https://www.microsoft.com/en-us/download/details.aspx?id=7826)
  • [Data workload assessment model and tool](https://github.com/Microsoft/DataMigrationTeam/tree/master/Data%20Workload%20Assessment%20Model%20and%20Tool)
  • [Database Experimentation Assistant (DEA)](https://www.microsoft.com/en-us/download/details.aspx?id=54090) for upgrades
  • [DMA SKU Recommender](https://docs.microsoft.com/en-us/sql/dma/dma-sku-recommend-sql-db?view=sql-server-2017) for Azure SQL
    • | 208 | | | Convert | Convert the source schema to work in the target environment. This is only relevant for heterogeneous (non-SQL Server) migrations. |
  • [SQL Server Migration Assistant](https://docs.microsoft.com/en-us/sql/ssma/sql-server-migration-assistant?view=sql-server-2017)
    • | 209 | | Migration | Migrate schema, data, and objects | Migrate the source schema, and then migrate the source data to the target. |
  • [Migrate with Azure Database Migration Services (DMS)](https://docs.microsoft.com/en-us/azure/dms/) (pull model)
  • [Migrate to Azure with Data Migration Assistant (DMA)](https://docs.microsoft.com/en-us/sql/dma/dma-migrateonpremsqltosqldb?view=sql-server-2017) (push model)
  • [Online migrations](https://docs.microsoft.com/en-us/azure/dms/tutorial-sql-server-managed-instance-online) with DMS
  • [Offline migrations](https://docs.microsoft.com/en-us/azure/dms/tutorial-sql-server-to-managed-instance) with DMS
  • [Utility](https://github.com/Microsoft/DataMigrationTeam/tree/master/IP%20and%20Scripts/MoveLogins) to move on-premises SQL Server logins to Azure SQL MI
  • [Bulk database creation](https://github.com/Microsoft/DataMigrationTeam/tree/master/Bulk%20Database%20Creation%20with%20PowerShell) with PowerShell
  • [Bulk schema deployment](https://github.com/Microsoft/DataMigrationTeam/tree/master/Bulk%20Schema%20Deployment%20with%20MSSQL-Scripter%20&%20PowerShell) with MSSQL-Scripter and PowerShell
  • [Migrate SSIS](https://docs.microsoft.com/azure/dms/how-to-migrate-ssis-packages) packages to Azure
  • SSRS can be deployed using SQL Server on an Azure VM or rewritten in Power BI
  • Migrate [SSAS to Azure Analysis Services](https://azure.microsoft.com/en-us/resources/videos/azure-analysis-services-moving-models/)
    • | 210 | | | Data Sync | Sync your target schema and data with the source. This is only relevant for minimal-downtime migrations. |
  • [Azure Data Sync](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-sync-data) for syncing data bi-directionally on-prem and in Azure
  • [Transactional replication](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-transactional-replication) for migrations by continuously publishing the changes
    • | 211 | | | Cutover | Cut over from the source to the target environment. This is only relevant for minimal-downtime migrations. |
  • [Cutover with DMS](https://docs.microsoft.com/en-us/azure/dms/tutorial-sql-server-managed-instance-online#performing-migration-cutover)
    • | 212 | | Post-migration | Remediate applications | Iteratively make any necessary changes to your applications. |
  • [Application migration patterns and examples](https://docs.microsoft.com/en-us/azure/architecture/cloud-adoption/migrate/azure-best-practices/contoso-migration-overview)
    • | 213 | | | Perform tests | Iteratively run functional and performance tests. |
  • [Data Quality Solution available from the partner QuerySurge](http://www.querysurge.com/company/partners/microsoft)
    • | 214 | | | Optimize | Based on the tests you performed, address any performance issues, and then retest to confirm the performance improvements. |
  • [Post-migration Validation and Optimization Guide](https://docs.microsoft.com/en-us/sql/relational-databases/post-migration-validation-and-optimization-guide)
    • | 215 | 216 | 217 | 218 | 219 |
    220 | 221 | 222 |

    223 | 224 |

    4.4 Database discovery and assessment

    225 | 226 | 227 | 228 | In this series of hands-on labs (throughout Modules 4 and 5), you will implement a proof-of-concept (PoC) for migrating an on-premises SQL Server 2008 R2 database into Azure SQL Database Managed Instance (SQL MI). You will perform assessments to reveal any feature parity and compatibility issues between the on-premises SQL Server 2008 R2 database and the managed database offerings in Azure. You will then migrate the customer's on-premises gamer information, web application and database into Azure, with minimal to no down-time. Finally, you will enable some of the advanced SQL features available in SQL MI to improve security and performance in the customer's application. 229 | 230 | At the end of these hands-on labs, you will be better able to implement a cloud migration solution for business-critical applications and databases. 231 | 232 | > **Note**: These labs were modified from an existing day-long, hands-on-labs workshop to fit into this workshop. If you'd like to access the extended version of these labs refer to [MCW: Migrating SQL databases to Azure](https://github.com/microsoft/MCW-Migrating-SQL-databases-to-Azure). 233 | 234 | 235 | 236 | ## Solution architecture 237 | 238 | Below is a diagram of the solution architecture that Tailspin Toys decided to go with, and what you will build in this lab. Please study this carefully, so you understand the whole of the solution as you are working on the various components. 239 | 240 | ![This solution diagram includes a virtual network containing SQL MI in a isolated subnet, along with a JumpBox VM and Database Migration Service in a management subnet. The MI Subnet displays both the primary managed instance, along with a read-only replica, which is accessed by reports from the web app. The web app connects to SQL MI via a subnet gateway and point-to-site VPN. The web app is published to App Services using Visual Studio 2019. An online data migration is conducted from the on-premises SQL Server to SQL MI using the Azure Database Migration Service, which reads backup files from an SMB network share.](https://github.com/microsoft/sqlworkshops-azuresqllabs/blob/master/graphics/preferred-solution-architecture.png?raw=true "Preferred Solution diagram") 241 | 242 | Throughout the solution, you will use [Azure Migrate](https://docs.microsoft.com/en-us/azure/migrate/migrate-services-overview) as the central hub to track the discovery, assessment, and migration of Tailspin Toys. The solution begins with using the [Microsoft Data Migration Assistant](https://docs.microsoft.com/en-us/sql/dma/dma-overview?view=sql-server-2017) to perform assessments of feature parity and compatibility of the on-premises SQL Server 2008 R2 database against both Azure SQL Database (Azure SQL DB) and Azure SQL Database Managed Instance (SQL MI), with the goal of migrating the `TailspinToys` database into an Azure PaaS offering with minimal or no changes. After completing the assessments and reviewing the findings, the SQL Server 2008 R2 database is migrated into SQL MI using the Azure Database Migration Service's online data migration option. This allows the database to be migrated with little to no downtime, by using a backup and transaction logs stored in an SMB network share. 243 | 244 | They'll also leverage their existing licenses to get [Azure Hybrid Benefits](https://azure.microsoft.com/en-us/pricing/hybrid-benefit/), and they'll prepay for [reserved capacity](https://azure.microsoft.com/en-us/blog/announcing-general-availability-of-azure-sql-database-reserved-capacity/ 245 | ). This will help them save money from the start. 246 | 247 | The web app is deployed to an Azure App Service Web App using Visual Studio 2019. Once the database has been migrated and cutover, the `TailspinToysWeb` application is configured to talk to the SQL MI VNet through a virtual network gateway using [point-to-site VPN](https://docs.microsoft.com/en-us/azure/vpn-gateway/point-to-site-about), and its connection strings are updated to point to the new SQL MI database. 248 | 249 | >**Note**: Due to time constraints, the lab will deal with switching the app running on a Jumpbox VM locally from leveraging data in SQL Server 2008 to SQL MI (but not the deployment to Azure or integrating the App Service with the Virtual Network). In the [extended version of these labs](https://github.com/microsoft/MCW-Migrating-SQL-databases-to-Azure/blob/master/Hands-on%20lab/HOL%20step-by-step%20-%20Migrating%20SQL%20databases%20to%20Azure.md#exercise-3-update-the-web-application-to-use-the-new-sql-mi-database), or if you have time at the end of the lab, you can do that. 250 | 251 | Once in SQL MI, several features of Azure SQL Database are examined. [Advanced Data Security (ADS)](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-advanced-data-security?view=sql-server-2017) is enabled and [Data Discovery and Classification](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-data-discovery-and-classification?view=sql-server-2017) is used to better understand the data and potential compliance issues with data in the database. The ADS [Vulnerability Assessment](https://docs.microsoft.com/en-us/azure/sql-database/sql-vulnerability-assessment?view=sql-server-2017) is used to identify potential security vulnerabilities and issues in the database, and those finding are used to mitigate one finding by enabling [Transparent Data Encryption](https://docs.microsoft.com/en-us/azure/sql-database/transparent-data-encryption-azure-sql?view=sql-server-2017) in the database. [Dynamic Data Masking (DDM)](https://docs.microsoft.com/en-us/sql/relational-databases/security/dynamic-data-masking?view=sql-server-2017) is used to prevent sensitive data from appearing when querying the database. Finally, [Read Scale-out](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-read-scale-out) is used to point reports on the Tailspin Toys web app to a read-only secondary, allowing reporting, particularly for the Leaderboard statistics page, to occur without impacting the performance of the primary database. 252 | 253 | > **Note:** 254 | > If you are attending this lab as part of a day-long workshop, the activities below should be skipped (they will be demoed in class). 255 | 256 | 257 |

    Activities

    258 | 259 | > **Note:** 260 | > If you are attending this lab as part of a day-long workshop, the activities below should be skipped (they will be demoed in class), but they are available should you want to review them again. 261 | 262 | In these activities, you'll set up Azure Migrate, and explore some of the new integrations between Microsoft's Data Migration Assistant (DMA) and Azure Database Migration Services (DMS). Before you begin the assessments, you need to restore a copy of the `TailspinToys` database in your SQL Server 2008 R2 instance. Finally, you will use the Microsoft Data Migration Assistant (DMA) to perform an assessment of the `TailspinToys` database against Azure SQL Database (Single database and Managed Instance). The assessment will provide a report about any feature parity and compatibility issues between the on-premises database and the various services. 263 | 264 | To complete the activities for this module, proceed directly to the Activities in [**Lab: Database Discovery and Assessment for Migrating to Azure**](https://github.com/microsoft/sqlworkshops-azuresqllabs/blob/master/Lab-DatabaseDiscoveryAndAssessmentForMigratingToAzure.md) for the hands-on exercises for Tailspin Toys Gaming. When are you done with these activities, come back here and continue to Module 5. 265 | 266 | 267 | 268 | 269 | 270 | 271 |

    272 | 273 |

    For Further Study

    274 | 286 | 287 |

    Next Steps

    288 | 289 | Next, Continue to 05 - Migrating to Azure SQL. -------------------------------------------------------------------------------- /sqlgroundtocloud/05-MigratingToAzureSQL.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft workshop from the SQL Server team 6 | 7 |

    8 | 9 |

    05 - Migrate to Azure SQL

    10 | 11 | In the previous module, you learned about Azure SQL, the benefits, the options, and how to get there. You reviewed how to assess your on-premises estate, and in this module you'll actually migrate to [Azure SQL Database Managed Instance](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance). 12 | 13 | 14 | (Make sure you check out the Prerequisites page before you start. You'll need all of the items loaded there before you can proceed with the workshop.) 15 | 16 | In this module, you will use the [Azure Database Migration Service](https://azure.microsoft.com/services/database-migration/) (DMS) to migrate the `TailspinToys` database from the on-premises SQL 2008 R2 database to SQL MI. At the end of the module, you'll also explore some of the security and performance features available. The activities in this module include: 17 | 18 | [5.1](#5.1): Migrate the database to SQL Managed instance 19 | [5.2](#5.2): Improve database security with Advanced Data Security (*Bonus*) 20 | [5.3](#5.3): Use online secondary for read-only queries (*Bonus*) 21 | [5.4](#5.4): After the Migration 22 | 23 | > **Note**: 24 | > This module is mainly guided labs. Proceed as instructed in each section. You may not have the time or desire to complete 5.2 and 5.3. They are optional, supplementary sections you can review at the end or at a later time. 25 | 26 | 27 |

    28 | 29 |

    5.1 Migrate the database to SQL Managed instance

    30 | 31 | In this section, you will use the [Azure Database Migration Service](https://azure.microsoft.com/services/database-migration/) (DMS) to migrate the `TailspinToys` database from the on-premises SQL 2008 R2 database to SQL MI. Tailspin Toys mentioned the importance of their gamer information web application in driving revenue, so for this migration you will target the [Business Critical service tier](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tier-business-critical). 32 | 33 | > The Business Critical service tier is designed for business applications with the highest performance and high-availability (HA) requirements. To learn more, read the [Managed Instance service tiers documentation](https://docs.microsoft.com/azure/sql-database/sql-database-managed-instance#managed-instance-service-tiers). 34 | 35 | This is the longest exercise in the module, so take your time (estimate about 30-45 minutes). 36 | 37 | First, you will create a new SMB network share on the SqlServer2008 VM. This will be the folder used by DMS for retrieving backups of the `TailspinToys` database during the database migration process. Next, you will use the SQL Server Configuration Manager to update the service account used by the SQL Server (MSSQLSERVER) to the `sqlmiuser` account. This is done to ensure the SQL Server service has the appropriate permissions to write backups to the shared folder. 38 | 39 | You'll then use the Azure Cloud shell to retrieve the information necessary to connect to your SQL MI and SqlServer2008 VM from DMS. You will also use the Azure Cloud Shell to create an Azure Active Directory (Azure AD) application and service principal (SP) that will provide DMS access to Azure SQL MI. You will grant the SP permissions to the hands-on-lab-SUFFIX resource group. 40 | 41 | Now that your environment is prepared, you can create a new online data migration project in DMS and migrate the `TailspinToys` database. Since you perform the migration as an "online data migration," the migration wizard will continue to monitor the SMB network share for newly added log files. This allows for any updates that happen on the source database to be captured until you cut over to the SQL MI database. You'll then add a record to one of the database tables, backup the logs, and complete the migration of the `TailspinToys` database by cutting over to the SQL MI database. You will then be able to connect to the SQL MI database using SSMS, and quickly verify the migration. 42 | 43 | With the `TailspinToys` database now running on SQL MI in Azure, the next step is to make the required modifications to the TailspinToys gamer information web application. 44 | 45 | 46 | 47 | 48 |

    Activities

    49 | 50 | Complete activities 1-9 by following the links below. Once you're in Activity 1, you can continue to the next activity through Activity 9 (the links below all point to different parts of the same overall lab). When are you done with activities 1-9, come back here for the next section (5.2). 51 | 52 | > **Note**: 53 | > If you were provided an environment to do this workshop, you should review Activities 4 and 5, but a service principal has already been created (no action required). 54 | 55 |       Activity 1: Create an SMB network share on the SQLServer2008VM 56 | 57 |       Activity 2: Change MSSQLSERVER service to run under sqlmiuser account 58 | 59 |       Activity 3: Create a backup of TailspinToys database 60 | 61 |       Activity 4: Retrieve SQL MI, SQL Server 2008 VM, and service principal connection information **(review only - no action needed)** 62 | 63 |       Activity 5: Create a service principal **(review only - no action needed)** 64 | 65 |       Activity 6: Create and run an online data migration project 66 | 67 |       Activity 7: Perform migration cutover 68 | 69 |       Activity 8: Verify database and transaction log migration 70 | 71 |       Activity 9: Update the application 72 | 73 | 74 | 75 |

    76 | 77 |

    5.2 Improve database security with Advanced Data Security (Bonus)

    78 | 79 | [Advanced Data Security](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-advanced-data-security) is a unified package for advanced SQL security capabilities. It includes functionality for discovering and classifying sensitive data, surfacing and mitigating potential database vulnerabilities, and detecting anomalous activities that could indicate a threat to your database. It provides a single go-to location for enabling and managing these capabilities. 80 | 81 | In this exercise, you'll enable Advanced Data Security, configure Data Discovery and Classification, and review the Vulnerability Assessment. At the end, you'll also receive a pointer to a Dynamic Data Masking lab extension. 82 | 83 | 84 |

    Activities

    85 | 86 | Complete activities 1-3 by following the links below. Once you're in Activity 1, you can continue to the next activity through Activity 3 (the links below all point to different parts of the same overall lab). When are you done with activities 1-3, come back here for the next section (5.3). 87 | 88 | 89 |       Activity 1: Enable Advanced Data Security 90 | 91 |       Activity 2: Configure SQL Data Discovery and Classification 92 | 93 |       Activity 3: Review Advanced Data Security Vulnerability Assessment 94 | 95 |

    96 |

    5.3 Use online secondary for read-only queries (Bonus)

    97 | 98 | In this exercise, you will look at how you can use the automatically created online secondary for reporting, without feeling the impacts of a heavy transactional load on the primary database. Each database in the SQL MI Business Critical tier is automatically provisioned with several AlwaysON replicas to support the availability SLA. Using [**Read Scale-Out**](https://docs.microsoft.com/azure/sql-database/sql-database-read-scale-out) allows you to load balance Azure SQL Database read-only workloads using the capacity of one read-only replica. 99 | 100 |

    Activities

    101 | 102 | Complete activities 1-3 by following the links below. Once you're in Activity 1, you can continue to the next activity through Activity 3 (the links below all point to different parts of the same overall lab). When are you done with activities 1-3, come back here for the next section (5.4). 103 | 104 |       Activity 1: View Leaderboard report in Tailspin Toys web application 105 | 106 |       Activity 2: Update read only connection string 107 | 108 |       Activity 3: Reload leaderboard report in the Tailspin Toys web application 109 | 110 |

    111 | 112 |

    5.4 After the Migration

    113 | 114 | In this module, you used the [Azure Database Migration Service](https://azure.microsoft.com/services/database-migration/) (DMS) to migrate the `TailspinToys` database from the on-premises SQL 2008 R2 database to SQL MI. You then updated the web application to use the SQL MI created, and enabled [advanced security features](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-advanced-data-security). Finally, you set up your application to leverage the [online secondary replica](https://docs.microsoft.com/en-us/azure/sql-database/sql-database-read-scale-out) to handle heavy read workloads. 115 | 116 | Now that Tailspin Toys has completed a migration for their gaming database. They'll want to leverage the [post-migration validation and optimization guide](https://docs.microsoft.com/en-us/sql/relational-databases/post-migration-validation-and-optimization-guide?view=sql-server-2017) to ensure data completeness and uncover and resolve performance issues. 117 | 118 | If and when Tailspin Toys chooses to scale their migration to other instances and databases, they can leverage the same process you've seen in Modules 4 and 5, but should also refer to the guidance Microsoft provides on [scaling a migration to Azure](https://docs.microsoft.com/en-us/azure/architecture/cloud-adoption/migrate/azure-best-practices/contoso-migration-scale). 119 | 120 |

    121 | 122 |

    For Further Study

    123 | 150 | 151 |

    Next Steps

    152 | 153 | Next, Continue to 06 - What to Use When. -------------------------------------------------------------------------------- /sqlgroundtocloud/06-WhatToUseWhen.md: -------------------------------------------------------------------------------- 1 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/microsoftlogo.png?raw=true) 2 | 3 | # Workshop: SQL Ground-to-Cloud 4 | 5 | #### A Microsoft workshop from the SQL Server team 6 | 7 |

    8 | 9 |

    06 - What to Use When: Designing a Solution

    10 | 11 | In this workshop you have covered using SQL Server both on-premises and in-cloud configurations, as well as hybrid applications as a solution for data processing. The end of this Module contains several helpful references you can use in these exercises and in production. 12 | 13 | This module can be used stand-alone, and does not require any prerequisite other than a laptop and some sort of design software (such as Microsoft Visio) - although you may also just use a whiteboard or paper for your design. 14 | 15 | There are many elements in a single solution, and in this module you'll learn how to take the business scenario and determine the best resources and processes to use to satisfy the requirements while considering the constraints within the scenario. 16 | 17 | In production, there are normally 6 phases to create a solution. These can be done in-person, or through recorded documents: 18 | 19 | - 01 **Discovery**: The original statement of the problem from the customer 20 | - 02 **Envisioning**: A "blue-sky" description of what success in the project would look like. Often phrased as *"I can..."* statements 21 | - 03 **Architecture Design Session**: An initial layout of the technology options and choices for a preliminary solution 22 | - 04 **Proof-Of-Concept** (POC): After the optimal solution technologies and processes are selected, a POC is set up with a small representative example of what a solution might look like, as much as possible. If available, a currently-running solution in a prallel example can be used 23 | - 05 **Implementation**: Implementing a phased-in rollout of the completed solution based on findings from the previous phases 24 | - 06 **Handoff**: A post-mortem on the project with a discussion of future enhancements 25 | 26 | Throughout this module, you can use various templates, icons, stencils and other assets to assist you with each phase and also use these with your exercises. These assets can also be used in your production workloads: https://github.com/microsoft/sqlworkshops/tree/master/ProjectResources 27 | 28 | For this module, you'll focus on the Discovery and the Architecture Design Session phases only. If you wish to develop your solution further after the course, you can use the assets above to complete all phases. 29 | 30 |

    31 | 32 |

    6.1 Understand the Problem Space

    33 | 34 | The first step in any project is to fully understand the problem the company needs to solve, and any requirements and constraints they have on those goals. This is often in the form of a "Problem Statement", which is a formal set of paragraphs clearly defining the circumstances, present condition, and desired outcomes for a solution. At this point you want to avoid exploring how to solve the problem, and focus on what you want to solve. 35 | 36 | Begin with as complete an examination of the company and organization as you can. Gather information from as many sources as possible, and simplify the descriptions to have specific measurements and depictions of the environment. 37 | 38 | From there, lay out the problem, and then review that with all stakeholders. 39 | 40 | After everyone agrees on the problem statement, pull out as many requirements (*goals*) for the project as you can find, and then lay in any constraints the solution has. At this point, it's acceptable to have unrealistic constraints - later you can pull those back after showing a cost/benefit ratio on each requirement and constraint. 41 | 42 |

    Activity: Review Business Scenarios

    43 | 44 | In this activity you will review three business scenarios, and pick one to focus on for the rest of this module. The company descriptions, project goals, and constraints have already been laid out for you. 45 | 46 | After you make your choice, copy the problem statement into your working documents (see the [Resources](https://github.com/microsoft/sqlworkshops/tree/master/ProjectResources) for examples) and make any changes or additions you want to make to the scenario. Feel free to adapt it to have more information where you want clarity - you can make assumptions about any part of the scenario. Are there sub-goals that have been left out? Any other constraints you can think of? 47 | 48 |

    49 | 50 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/AdventureWorksLogo.png?raw=true") 51 | 52 | **AdventureWorks** 53 | 54 | [Adventure Works Cycles](https://docs.microsoft.com/en-us/previous-versions/sql/sql-server-2008/ms124825(v=sql.100) ) is a large, multinational manufacturing company. The company manufactures and sells metal and composite bicycles to North American, European and Asian commercial markets. While its base operation is located in Bothell, Washington with 290 employees, several regional sales teams are located throughout their market base. 55 | 56 | Starting in the year 2000, Adventure Works Cycles bought a small manufacturing plant, Importadores Neptuno, located in Mexico. Importadores Neptuno manufactures several critical subcomponents for the Adventure Works Cycles product line. These subcomponents are shipped to the Bothell location for final product assembly. In 2001, Importadores Neptuno, became the sole manufacturer and distributor of the touring bicycle product group. 57 | 58 | Coming off a successful fiscal year, Adventure Works Cycles is looking to broaden its market share by targeting their sales to their best customers, extending their product availability through an external Web site, and reducing their cost of sales through lower production costs. They are also looking to modernize their data estate. 59 | 60 | Project Goals 61 | 62 | - Modernize to a newer SQL version 63 | - Move to Cloud wherever possible 64 | - Cloud Integration 65 | - Increase Performance 66 | - Publish Product Catalog to the Web 67 | - Enable Business-To-Business (B2B) systems 68 | 69 | Project Constraints 70 | 71 | - Some systems must stay on-premises 72 | - In some cases, no code change is possible 73 | - The B2B system should be a "Pull" from partners 74 | 75 |

    76 | 77 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/contosologo.png?raw=true) 78 | 79 | **Contoso** 80 | 81 | [The Contoso company](https://docs.microsoft.com/en-us/microsoft-365/enterprise/contoso-overview) is a multi-national business with headquarters in Paris, France. It is a conglomerate manufacturing, sales, and support organization with over 100,000 products. They are embarking on a multi-year process of migrating from company-owned datacenters to a cloud provider. They have narrowed the list of potential vendors to three, including Microsoft. They have [high security](https://docs.microsoft.com/en-us/microsoft-365/enterprise/contoso-info-protect) and [interoperability with mobile device](https://docs.microsoft.com/en-us/microsoft-365/enterprise/contoso-mdm) concerns. There is also an Open-Source (OSS) investigation at the company. 82 | 83 | 84 | Project Goals 85 | 86 | - Move everything to Cloud 87 | - Multi-Cloud strategy desired - standards-based 88 | - All client apps should be available worldwide 89 | - Server-side should be API's by default 90 | - Interest in parity for platforms (OSS Support) 91 | 92 | Project Constraints 93 | 94 | - High Security and Auditing capabilities required 95 | - International Compliance required 96 | - Access Tracking required 97 | - Must be user-friendly on mobile devices 98 | 99 | 100 | 101 |

    102 | 103 | ![](https://github.com/microsoft/sqlworkshops/blob/master/graphics/WideWorldImportersLogo.png?raw=true) 104 | 105 | **Wide World Importers** 106 | 107 | [Wide World Importers](https://docs.microsoft.com/en-us/sql/samples/wide-world-importers-what-is?view=sql-server-2017) (WWI) is a wholesale novelty goods importer and distributor operating from the San Francisco bay area in the United States. 108 | 109 | As a wholesaler, WWI's customers are mostly companies who resell to individuals. WWI sells to retail customers across the United States including specialty stores, supermarkets, computing stores, tourist attraction shops, and some individuals. WWI also sells to other wholesalers via a network of agents who promote the products on WWI's behalf. While all of WWI's customers are currently based in the United States, the company is intending to push for expansion into other countries. 110 | 111 | WWI buys goods from suppliers including novelty and toy manufacturers, and other novelty wholesalers. They stock the goods in their WWI warehouse and reorder from suppliers as needed to fulfil customer orders. They also purchase large volumes of packaging materials, and sell these in smaller quantities as a convenience for the customers. 112 | 113 | Recently WWI started to sell a variety of edible novelties such as "chili chocolates". The company previously did not have to handle chilled items. Now, to meet food handling requirements, they must monitor the temperature in their chiller room and any of their trucks that have chiller sections. 114 | 115 | Project Goals 116 | 117 | - Enable Big Data processing 118 | - Enable Machine Learning and Artificial Intelligence prediction capabilities 119 | - Cloud platform desired, but may need to consider on-premises options 120 | 121 | Project Constraints 122 | 123 | - Short timeframe 124 | - Unknown Data Sources 125 | - Use only API calls for the predictions 126 | 127 |

    128 | 129 |

    6.2 Understand the Technologies and Processes for Solving the Problem

    130 | 131 | With a firm understanding of what the customer needs, you can now consider the technologies and processes at your disposal for the solution. Each technology will have benefits and considerations, so at this point you just want to list out all of your options. Everything is on the table at this phase, and ensure that you check the list you create with other professionals to ensure you have considered everything that could solve the problem. 132 | 133 |

    Activity: List the Technologies and Processes for the Problem Space

    134 | 135 | In this activity you will list out all of the options you have for your problem space. 136 | 137 |

    Steps

    138 | 139 | - Open your Architecture Design Session (ADS) document and detail all technologies you have studied in this workshop, and list them out. (Order is not important during this step) 140 | - Next, write the problem element next to each technology that it could solve 141 | - Document any processes you should follow when using each technology 142 | 143 |

    144 | 145 |

    6.3 The Decision Matrix

    146 | 147 | Following the process, you now know the problems you want to solve, the desired outcomes for the solution, and several tools and technique options that you can use to achieve your goals. In most situations, there are several ways to solve a given problem. Sometimes the "best" solution is too costly, inconvenient or unworkable due to the requirements or constraints the customer puts on the solution. 148 | 149 | Because most solutions are fairly complex, and there are mutiple technology and process choices, considerations and requirements, a *Decision Matrix* that lists these elements is useful. It contains columns for the technology and process options you have, and the requirements and constraints as rows. Each column gets a score you assign from a low number (does not meet this requirement) to a higher one (does meet the requirement). These numbers are summed at the end of each row, per requirement. The highest number is usually the best technology for that aspect of the solution. 150 | 151 | As an example, assume you have an application that is written using T-SQL statements, and you want to store data that has high security requirements and is available online: 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 |
    Requirement/ConstraintSQL Server in Azure VMAzure SQL DBPostgres as a Service
    Low Cost233
    Easy to Manage233
    Highly Securable332
    Fully Supports T-SQL330
    Score: 10128
    162 | 163 | In this simple example, Azure SQL DB is a high candidate for your solution. (In production, there would be far more requirements and constraints, and you may need to use a 1-5 scale rather than 1-3) 164 | 165 |

    Activity: Create a Decision Matrix

    166 | 167 | In this activity you will use the scenario you selected from above and create a Decision Matrix using a spreadsheet. 168 | 169 |

    Steps

    170 | 171 | - [Open this reference and read through it](https://www.smartsheet.com/decision-matrix-templates). 172 | - Create a spreadsheet for your Decision Matrix, or download one of the samples from the reference. 173 | - Fill out the Decision Matrix based on the problem requirements and constraints using the technologies and processes you developed in the previous steps 174 | 175 |

    176 | 177 |

    6.4 Explain the Solution

    178 | 179 | The architecture design session above is most often conducted by the technical staff with representation from the business. After the design session is complete, the findings should be condensed into an instrument (slides, notes or other graphics tool) that allow the team to explain the proposal along with a set of options. At the end of the presentation, your team should also include a description of the project, timelines, and responsibilities. 180 | 181 |

    Activity: Create a Solution Presentation

    182 | 183 | In this activity your team will create a solution briefing with options and timelines. 184 | 185 |

    Steps

    186 | 187 | - Open this reference: https://github.com/microsoft/sqlworkshops/tree/master/ProjectResources 188 | - You may use the PowerPoint template provided (6 - Handoff.pptx) or create your own briefing tool, using anything you like. 189 | - Make sure you include at least one alternative option and explain why you chose your original one. 190 | - If time permits, review the schedule documents in the Resources area and assign reasonable timelines to your soluition. 191 | 192 |

    193 | 194 |

    References

    195 | 196 | 207 | 208 | 225 | 226 | 227 | Congratulations! You have completed this workshop on "SQL Ground-to-Cloud". You now have the tools, assets, and processes you need to extrapolate this information into other applications. --------------------------------------------------------------------------------