├── README.md └── zh_tw.md /README.md: -------------------------------------------------------------------------------- 1 | Trace Dapper.NET Source Code 2 | ==== 3 | 4 | English : [Link](https://github.com/shps951023/Trace-Dapper.NET-Source-Code) 5 | Traditional Chinese : [Link](https://github.com/shps951023/Trace-Dapper.NET-Source-Code/blob/master/zh_tw.md) 6 | Simplified Chinese : [Link](https://www.cnblogs.com/ITWeiHan/p/11614704.html) 7 | 8 | 9 | ---- 10 | 11 | ## 1. Introduction 12 | 13 | After years of promotion by Industry Veterans and StackOverflow, “Dapper with Entity Framework” is a powerful combination that deal the needs of `“safe, convenient, efficient, maintainable” `. 14 | 15 | But the current network articles, although there are many articles on Dapper but stay on how to use, no one systematic explanation of the source code logic. So with this article “Trace Dapper Source Code” want to take you into the Dapper code, to understand the details of the design, efficient principles, and learn up practical application in the work. 16 | 17 | ## 2. Installation Environment 18 | 19 | 1. Clone the latest version from [Dapper's Github](https://github.com/StackExchange/Dapper) 20 | 2. Create .Net Core Console project 21 | ![](https://i.imgur.com/i97jmVQ.png) 22 | 3. Install the [NuGet SqlClient](https://www.nuget.org/packages/System.Data.SqlClient) and add the Dapper Project Reference. 23 | ![](https://i.imgur.com/inIOYfx.png) 24 | 4. Running console with breakpoint it allows runtime to view the logic. 25 | ![](https://i.imgur.com/iSgyvGb.png) 26 | 27 | My Personal Environment 28 | 29 | - MSSQLLOCALDB 30 | - Visaul Studio 2019 31 | - LINQPad 5 32 | - Dapper version: V2.0.30 33 | - ILSpy 34 | - Windows 10 pro 35 | 36 | ## 3. Dynamic Query 37 | 38 | With Dapper dynamic Query, you can save time in modifying class attributes in the early stages of development because the table structure is still `in the adjustment stage`, or it isn’t worth the extra effort to declare class lightweight requirements. 39 | 40 | When the table is stable, use the POCO generator to quickly generate the Class and convert it to strong type maintenance, e.g [PocoClassGenerator](https://github.com/shps951023/PocoClassGenerator).. 41 | 42 | ### Why can Dapper be so convenient and support dynamic? 43 | 44 | Two key points can be found by tracing the source code of the Query method 45 | 46 | 1. The entity class is actually `DapperRow` that transformed implicitly to dynamic. 47 | ![](https://i.imgur.com/ca1QuS8.png) 48 | 2. DapperRow inherits `IDynamicMetaObjectProviderand` & implements corresponding methods. 49 | ![](https://i.imgur.com/aBXnex4.png) 50 | 51 | For this logic, I will make a simplified version of Dapper dynamic Query to let readers understand the conversion logic: 52 | 53 | 1. Create a `dynamic` type variable, the entity type is `ExpandoObject`. 54 | 2. Because there’s an inheritance relationship that can be transformed to `IDictionary` 55 | 3. Use DataReader to get the field name using GetName, get the value from the `field index`, and add both to the Dictionary as a key and value 56 | 4. Because expandobject has the implementation IDynamicMetaObjectProvider interface that can be converted to dynamic 57 | 58 | ```C# 59 | public static class DemoExtension 60 | { 61 | public static IEnumerable Query(this IDbConnection cnn, string sql) 62 | { 63 | using (var command = cnn.CreateCommand()) 64 | { 65 | command.CommandText = sql; 66 | using (var reader = command.ExecuteReader()) 67 | { 68 | while (reader.Read()) 69 | { 70 | yield return reader.CastToDynamic(); 71 | } 72 | } 73 | } 74 | } 75 | 76 | private static dynamic CastToDynamic(this IDataReader reader) 77 | { 78 | dynamic e = new ExpandoObject(); 79 | var d = e as IDictionary; 80 | for (int i = 0; i < reader.FieldCount; i++) 81 | d.Add(reader.GetName(i),reader[i]); 82 | return e; 83 | } 84 | } 85 | ``` 86 | 87 | Now that we have the concept of the simple expandobject Dynamic Query example, go to the deep level to see how Dapper handles the details and why dapper customize the DynamicMetaObjectProvider. 88 | 89 | First, learn the Dynamic Query process logic: 90 | code: 91 | 92 | ```C# 93 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 94 | { 95 | var result = cn.Query("select N'Wei' Name,26 Age").First(); 96 | Console.WriteLine(result.Name); 97 | } 98 | ``` 99 | 100 | The value of the process would be: 101 | Create Dynamic FUNC > stored in the cache > use `result.Name` > transfer to call `((DapperRow)result)["Name"]` > from `DapperTable.Values Array` with `index value corresponding to the field "Name" in the Values array` to get value. 102 | 103 | Then look at the source code of the GetDapperRowDeserializer method, which controls the logic of how dynamic runs, and is dynamically created as Func for upper-level API calls and cache reuse. 104 | 105 | ![](https://i.imgur.com/H5zJAlV.png) 106 | 107 | This section of Func logic: 108 | 109 | 1. Although DapperTable is a local variable in the method, it is referenced by the generated Func, so it will not be GC and always stored in the memory and reused. 110 | ![](https://i.imgur.com/dhz0bND.png) 111 | 2. Because it is dynamic, there is no need to consider the type Mapping, here directly use `GetValue(index)` to get value from database. 112 | 113 | ```C# 114 | var values = new object[select columns count]; 115 | for (int i = 0; i < values.Length; i++) 116 | { 117 | object val = r.GetValue(i); 118 | values[i] = val is DBNull ? null : val; 119 | } 120 | ``` 121 | 122 | 3. Save the data in DapperRow 123 | 124 | ```C# 125 | public DapperRow(DapperTable table, object[] values) 126 | { 127 | this.table = table ?? throw new ArgumentNullException(nameof(table)); 128 | this.values = values ?? throw new ArgumentNullException(nameof(values)); 129 | } 130 | ``` 131 | 132 | 4. DapperRow inherits IDynamicMetaObjectProvider and implements the GetMetaObject method. The implementation logic is to return the DapperRowMetaObject object. 133 | 134 | ```C# 135 | private sealed partial class DapperRow : System.Dynamic.IDynamicMetaObjectProvider 136 | { 137 | DynamicMetaObject GetMetaObject(Expression parameter) 138 | { 139 | return new DapperRowMetaObject(parameter, System.Dynamic.BindingRestrictions.Empty, this); 140 | } 141 | } 142 | ``` 143 | 144 | 5. DapperRowMetaObject main function is to define behavior, by override `BindSetMember、BindGetMember` method, Dapper defines Get, Set of behavior were used `IDictionary - GetItem` , `DapperRow - SetValue` 145 | ![](https://i.imgur.com/LRuCkeB.png) 146 | 6. Finally, Dapper uses the DataReader `column order` , first using the column name to get Index, then using Index and Values. 147 | ![](https://i.imgur.com/W2SzdOF.png) 148 | 149 | ### Why inherit IDictionary? 150 | 151 | There is a question to think about: In DapperRowMetaObject, you can define the Get and Set behaviors by yourself, so instead of using the Dictionary-GetItem method, instead of using other methods, does it mean that `you don't need to inherit IDictionary`? 152 | 153 | One of the reasons for Dapper to do this is related to the open principle. DapperTable and DapperRow are all low-level implementation class. Based on the open and closed principle, they `should not be opened to users`, so they are set as `private`. 154 | 155 | ```C# 156 | private class DapperTable{/*...*/} 157 | private class DapperRow :IDictionary, IReadOnlyDictionary,System.Dynamic.IDynamicMetaObjectProvider{/*...*/} 158 | ``` 159 | 160 | What if the user wants to know the field name? 161 | 162 | Because DapperRow implements IDictionary, it can be `upcasting` to `IDictionary`, and use it to get field data by ` public interface`. 163 | 164 | ```C# 165 | public interface IDictionary : ICollection>, IEnumerable>, IEnumerable{/*..*/} 166 | ``` 167 | 168 | For example, I’ve created a tool called [HtmlTableHelper](https://github.com/shps951023/HtmlTableHelper) to use this feature to automatically convert Dapper Dynamic Query to Table Html, such as the following code and picture 169 | 170 | ```C# 171 | using (var cn = "Your Connection") 172 | { 173 | var sourceData = cn.Query(@"select 'ITWeiHan' Name,25 Age,'M' Gender"); 174 | var tablehtml = sourceData.ToHtmlTable(); //Result :
NameAgeGender
ITWeiHan25M
175 | } 176 | ``` 177 | 178 | ![](https://i.imgur.com/D8es51O.png) 179 | 180 | ## 4. Strongly Typed Mapping Part1: ADO.NET vs. Dapper 181 | 182 | Next is the key function of Dapper, `Strongly Typed Mapping`. Because of the difficulty, it will be divided into multiple parts for explanation. 183 | 184 | In the first part, compare ADO.NET DataReader GetItem By Index with Dapper Strongly Typed Query, check the difference between the IL and understand the main logic of Dapper Query Mapping. 185 | 186 | With the logic, how to implement it, I use three techniques in order: `Reflection、Expression、Emit` implement three versions of the Query method from scratch to let readers understand gradually. 187 | 188 | **ADO.NET vs. Dapper** 189 | 190 | First use the following code to trace the Dapper Query logic 191 | 192 | ```C# 193 | class Program 194 | { 195 | static void Main(string[] args) 196 | { 197 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 198 | { 199 | var result = cn.Query("select N'Wei' Name , 25 Age").First(); 200 | Console.WriteLine(result.Name); 201 | Console.WriteLine(result.Age); 202 | } 203 | } 204 | } 205 | 206 | public class User 207 | { 208 | public string Name { get; set; } 209 | public int Age { get; set; } 210 | } 211 | ``` 212 | 213 | Here we need to focus on the `Dapper.SqlMapper.GenerateDeserializerFromMap` method, it is responsible for the logic of Mapping, you can see that a large number of Emit IL technology is used inside. 214 | 215 | ![20191004012713.png](https://camo.githubusercontent.com/84c032f520434c337ee31c415bc9c27b931b88f9e0d705af5b977a8f15fe1791/68747470733a2f2f692e6c6f6c692e6e65742f323031392f31302f30342f4443564854614f474268634c3539732e706e67) 216 | 217 | 218 | 219 | To understand this IL logic, my way: `"You should not go directly to the details, but check the complete IL first"` As for how to view it, you need to prepare the [il\-visualizer](https://github.com/drewnoakes/il-visualizer) open source tool first, which can view the IL generated by DynamicMethod at Runtime. 220 | 221 | It supports vs 2015 and 2017 by default. If you use vs2019 like me 222 | 223 | 1. Need to manually extract the `%USERPROFILE%\Documents\Visual Studio 2019` path below 224 | 2. `.netstandard2.0` The project needs to be created `netstandard2.0` and unzipped to this folder 225 | ![image](https://user-images.githubusercontent.com/12729184/101119088-88b22c00-3625-11eb-9904-2d0033113522.png) 226 | 227 | Finally reopen visaul studio and run debug, enter the GetTypeDeserializerImpl method, click the magnifying glass> IL visualizer> view the `Runtime` generated IL code for the DynamicMethod 228 | 229 | ![image](https://user-images.githubusercontent.com/12729184/101119149-aaabae80-3625-11eb-8d99-06002b720da1.png) 230 | 231 | The following IL can be obtained 232 | 233 | ```C# 234 | IL_0000 : LDC . i4 .0 235 | IL_0001 : stloc .0 236 | IL_0002 : newobj Void . ctor () / Demo . Customer 237 | IL_0007 : stloc .1 238 | IL_0008 : ldloc .1 239 | IL_0009 : after 240 | IL_000a : LDC . i4 .0 241 | IL_000b : stloc .0 242 | IL_000c : ldarg .0 243 | IL_000d : LDC . i4 .0 244 | IL_000e: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 245 | IL_0013: dup 246 | IL_0014: stloc.2 247 | IL_0015: dup 248 | IL_0016: isinst System.DBNull 249 | IL_001b: brtrue.s IL_0029 250 | IL_001d: unbox.any System.String 251 | IL_0022: callvirt Void set_Name(System.String)/Demo.User 252 | IL_0027: br.s IL_002b 253 | IL_0029: pop 254 | IL_002a: pop 255 | IL_002b: dup 256 | IL_002c: ldc.i4.1 257 | IL_002d: stloc.0 258 | IL_002e: ldarg. 0 259 | IL_002f: ldc.i4.1 260 | IL_0030: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 261 | IL_0035: dup 262 | IL_0036: stloc.2 263 | IL_0037: dup 264 | IL_0038: isinst System.DBNull 265 | IL_003d: brtrue.s IL_004b 266 | IL_003f: unbox.any System.Int32 267 | IL_0044: callvirt Void set_Age(Int32)/Demo.User 268 | IL_0049: br.s IL_004d 269 | IL_004b: pop 270 | IL_004c: pop 271 | IL_004d: stloc.1 272 | IL_004e: leave IL_0060 273 | IL_0053: ldloc.0 274 | IL_0054: ldarg. 0 275 | IL_0055: ldloc.2 276 | IL_0056: call Void ThrowDataException(System.Exception, Int32, System.Data.IDataReader, System.Object)/Dapper.SqlMapper 277 | IL_005b: leave IL_0060 278 | IL_0060: ldloc.1 279 | IL_0061: ret 280 | ``` 281 | 282 | To understand this IL, you need to understand how it `ADO.NET DataReader fast way to read data` will be used `GetItem By Index` , such as the following code 283 | 284 | ```C# 285 | public static class DemoExtension 286 | { 287 | private static User CastToUser(this IDataReader reader) 288 | { 289 | var user = new User(); 290 | var value = reader[0]; 291 | if(!(value is System.DBNull)) 292 | user.Name = (string)value; 293 | var value = reader[1]; 294 | if(!(value is System.DBNull)) 295 | user.Age = (int)value; 296 | return user; 297 | } 298 | 299 | public static IEnumerable Query(this IDbConnection cnn, string sql) 300 | { 301 | if (cnn.State == ConnectionState.Closed) cnn.Open(); 302 | using (var command = cnn.CreateCommand()) 303 | { 304 | command.CommandText = sql; 305 | using (var reader = command.ExecuteReader()) 306 | while (reader.Read()) 307 | yield return reader.CastToUser(); 308 | } 309 | } 310 | } 311 | ``` 312 | 313 | Then look at the IL code generated by this Demo-CastToUser method 314 | 315 | ```C# 316 | DemoExtension.CastToUser: 317 | IL_0000: nop 318 | IL_0001: newobj User..ctor 319 | IL_0006: stloc.0 // user 320 | IL_0007: ldarg.0 321 | IL_0008: ldc.i4.0 322 | IL_0009: callvirt System.Data.IDataRecord.get_Item 323 | IL_000E: stloc.1 // value 324 | IL_000F: ldloc.1 // value 325 | IL_0010: isinst System.DBNull 326 | IL_0015: ldnull 327 | IL_0016: cgt.un 328 | IL_0018: ldc.i4.0 329 | IL_0019: ceq 330 | IL_001B: stloc.2 331 | IL_001C: ldloc.2 332 | IL_001D: brfalse.s IL_002C 333 | IL_001F: ldloc.0 // user 334 | IL_0020: ldloc.1 // value 335 | IL_0021: castclass System.String 336 | IL_0026: callvirt User.set_Name 337 | IL_002B: nop 338 | IL_002C: ldarg.0 339 | IL_002D: ldc.i4.1 340 | IL_002E: callvirt System.Data.IDataRecord.get_Item 341 | IL_0033: stloc.1 // value 342 | IL_0034: ldloc.1 // value 343 | IL_0035: isinst System.DBNull 344 | IL_003A: ldnull 345 | IL_003B: cgt.un 346 | IL_003D: ldc.i4.0 347 | IL_003E: ceq 348 | IL_0040: stloc.3 349 | IL_0041: ldloc.3 350 | IL_0042: brfalse.s IL_0051 351 | IL_0044: ldloc.0 // user 352 | IL_0045: ldloc.1 // value 353 | IL_0046: unbox.any System.Int32 354 | IL_004B: callvirt User.set_Age 355 | IL_0050: nop 356 | IL_0051: ldloc.0 // user 357 | IL_0052: stloc.s 04 358 | IL_0054: br.s IL_0056 359 | IL_0056: ldloc.s 04 360 | IL_0058: ret 361 | ``` 362 | 363 | It can be compared with the IL generated by Dapper shows that it is `roughly the same` (the differences will be explained later), which means that the logic and efficiency of the two operations will be similar, which is `Dapper efficiency is close to the native ado.net` the reasons why. 364 | 365 | ## 5. Strongly Typed Mapping Part2: Reflection version 366 | 367 | In the previous ado.net Mapping example, we found a serious problem with `there is no way to share multiple classes of methods, and each new class requires a code rewrite`. To solve this problem, write a common method that does different logical processing for different classes during the Runtime. 368 | 369 | There are three main implementation methods: Reflection, Expression, and Emit. Here, I will first introduce the simplest method: "Reflection". I will use reflection to simulate Query to write code from scratch to give readers a preliminary understanding of dynamic processing concepts. (If experienced readers can skip this article) 370 | 371 | Logic: 372 | 373 | 1. Use generics to pass dynamic class 374 | 2. Use `Generic constraints new()` to create objects dynamically 375 | 3. DataReader need to use `attribute string name is used as Key` , you can use Reflection to get the attribute name of the dynamic type and get the database data through the `DataReader this[string parameter]` 376 | 4. Use PropertyInfo.SetValue to dynamically assign database data to objects 377 | 378 | Finally got the following code: 379 | 380 | ```C# 381 | public static class DemoExtension 382 | { 383 | public static IEnumerable Query(this IDbConnection cnn, string sql) where T : new() 384 | { 385 | using (var command = cnn.CreateCommand()) 386 | { 387 | command.CommandText = sql; 388 | using (var reader = command.ExecuteReader()) 389 | while (reader.Read()) 390 | yield return reader.CastToType(); 391 | } 392 | } 393 | 394 | // 1. Use generics to pass dynamic class 395 | private static T CastToType < T >( this IDataReader reader ) where T : new () 396 | { 397 | // 2. Use `Generic constraints new()` to create objects dynamically 398 | var instance = new T (); 399 | 400 | // 3.DataReader need to use `attribute string name is used as Key` , you can use Reflection to get the attribute name of the dynamic type and get the database data through the `DataReader this[string parameter]` 401 | var type = typeof ( T ); 402 | var props = type . GetProperties (); 403 | foreach ( var p in props ) 404 | { 405 | var val = reader[p.Name]; 406 | 407 | // 4. Use PropertyInfo.SetValue to dynamically assign database data to objects 408 | if ( ! ( Val is System . DBNull )) 409 | p . SetValue ( instance , val ); 410 | } 411 | 412 | return instance; 413 | } 414 | } 415 | ``` 416 | 417 | The advantage of the Reflection version is that the code is `simple`, but it has the following problems 418 | 419 | 1. The attribute query should not be repeated, and it should be ignored if it is not used. Example: If the class has N properties, SQL means to query 3 fields, and the ORM PropertyInfo foreach N times is not 3 times each time. And Dapper specially optimized this logic in Emit IL: `「Check how much you use, not waste」`. 420 | ![image](https://user-images.githubusercontent.com/12729184/101120154-dcbe1000-3627-11eb-8e1b-79a6e3777dbc.png) 421 | 422 | 2. Efficiency issues: 423 | 424 | - The reflection efficiency will be slower. the solution will be introduced later: `「Key Cache + Dynamic Create Method」` exchange space for time. 425 | 426 | - Using the string Key value will call more `GetOrdinal` methods, you can check the official MSDN explanation `its efficiency is worse than Index value` . 427 | 428 | ![image](https://user-images.githubusercontent.com/12729184/101120383-75549000-3628-11eb-9c8b-be3c3e654c46.png) 429 | 430 | ## 6. Strongly Typed Mapping Part3: The important concept of dynamic create method "code from result" optimizes efficiency 431 | 432 | Then use Expression to solve the Reflection version problem, mainly using Expression features: `「Methods can be dynamically created during Runtime」` to solve the problem. 433 | 434 | Before this, we need to have an important concept: `「Reverse the most concise code from the result」` optimizing efficiency. For example: In the past, a classic topic of "printing regular triangle stars'' when learning a program to make a regular triangle of length 3, the common practice would be loop + recursion the way 435 | 436 | ```C# 437 | void Main () 438 | { 439 | Print(3,0); 440 | } 441 | 442 | static void Print(int length, int spaceLength) 443 | { 444 | if (length < 0) 445 | return; 446 | else 447 | Print(length - 1, spaceLength + 1); 448 | for (int i = 0; i < spaceLength; i++) 449 | Console.Write(" "); 450 | for (int i = 0; i < length; i++) 451 | Console.Write("* "); 452 | Console.WriteLine(""); 453 | } 454 | ``` 455 | 456 | But in fact, this topic can be changed to the following code when the length is already known 457 | 458 | ```C# 459 | Console.WriteLine(" * "); 460 | Console.WriteLine(" * * "); 461 | Console.WriteLine("* * * "); 462 | ``` 463 | 464 | This concept is very important, because the code is reversed from the result, so the logic is straightforward and `efficient` , and Dapper uses this concept to dynamically build methods. 465 | 466 | Example: 467 | 468 | - The Name property of User Class corresponds to Reader Index 0, the type is String, and the default value is null 469 | 470 | - The Age attribute of User Class corresponds to Reader Index 1, the type is int, and the default value is 0 471 | 472 | ```C# 473 | void Main () 474 | { 475 | using (var cn = Connection) 476 | { 477 | var result = cn.Query("select N'Wei' Name,26 Age").First(); 478 | } 479 | } 480 | 481 | class User 482 | { 483 | public string Name { get; set; } 484 | public int Age { get; set; } 485 | } 486 | ``` 487 | 488 | If the system can help generate the following logical methods, then the efficiency will be the best 489 | 490 | ```C# 491 | User dynamic method ( IDataReader reader ) 492 | { 493 | var user = new User(); 494 | var value = reader[0]; 495 | if( !(value is System.DBNull) ) 496 | user.Name = (string)value; 497 | value = reader[1]; 498 | if( !(value is System.DBNull) ) 499 | user.Age = (int)value; 500 | return user; 501 | } 502 | ``` 503 | 504 | In addition, the above example can be seen for Dapper `SQL Select corresponds to the Class attribute order is very important` , so the algorithm of Dapper in the cache will be explained later, which is specifically optimized for this. 505 | 506 | ## 7. Strongly Typed Mapping Part4: Expression version 507 | 508 | With the previous logic, we use Expression to implement dynamic creation methods, and then we can think `Why use Expression implementation first instead of Emit?` 509 | 510 | In addition to the ability to dynamically build methods, compared to Emit, it has the following advantages: 511 | 512 | - `Readadable`, You can use familiar keywords, such as the Variable corresponds to Expression.Variable, and the creation of object New corresponds to Expression.New 513 | ![image](https://user-images.githubusercontent.com/12729184/101126581-c5d2ea00-3636-11eb-9b4e-bdab6ff2a4d7.png) 514 | 515 | - `Easy Runtime Debug`, You can see the logic code corresponding to Expression in Debug mode 516 | ![image](https://user-images.githubusercontent.com/12729184/101126670-ef8c1100-3636-11eb-82dd-fac4f9d205c7.png) 517 | ![image](https://user-images.githubusercontent.com/12729184/101126691-f7e44c00-3636-11eb-93b2-6dc1ff20f548.png) 518 | 519 | So it is especially suitable for introducing dynamic method establishment, but Expression cannot do some detailed operations compared to Emit, which will be explained by Emit later. 520 | 521 | ### Rewrite Expression version 522 | 523 | Logic: 524 | 525 | 1. Get all field names of sql select 526 | 2. Obtain the attribute data of the mapping type > encapsulate the index, sql field and class attribute data in a variable for later use 527 | 3. Dynamic create method: Read the data we want from the database Reader in order, the code logic: 528 | 529 | ```C# 530 | User dynamic method ( IDataReader reader ) 531 | { 532 | var user = new User(); 533 | var value = reader[0]; 534 | if( !(value is System.DBNull) ) 535 | user.Name = (string)value; 536 | value = reader[1]; 537 | if( !(value is System.DBNull) ) 538 | user.Age = (int)value; 539 | return user; 540 | } 541 | ``` 542 | 543 | Finally, the following Exprssion version code 544 | 545 | ```C# 546 | public static class DemoExtension 547 | { 548 | public static IEnumerable Query(this IDbConnection cnn, string sql) where T : new() 549 | { 550 | using (var command = cnn.CreateCommand()) 551 | { 552 | command.CommandText = sql; 553 | using (var reader = command.ExecuteReader()) 554 | { 555 | var func = CreateMappingFunction(reader, typeof(T)); 556 | while (reader.Read()) 557 | { 558 | var result = func(reader as DbDataReader); 559 | yield return result is T ? (T)result : default(T); 560 | } 561 | 562 | } 563 | } 564 | } 565 | 566 | private static Func CreateMappingFunction(IDataReader reader, Type type) 567 | { 568 | // 1. Get all field names in sql select 569 | var names = Enumerable . Range ( 0 , reader . FieldCount ). Select ( index => reader . GetName ( index )). ToArray (); 570 | 571 | // 2. Get the attribute data of the mapping type > encapsulate the index, sql fields, and class attribute data in a variable for later use 572 | var props = type . GetProperties (). ToList (); 573 | var members = names . Select (( columnName , index ) => 574 | { 575 | var property = props.Find(p => string.Equals(p.Name, columnName, StringComparison.Ordinal)) 576 | ?? props.Find(p => string.Equals(p.Name, columnName, StringComparison.OrdinalIgnoreCase)); 577 | return new 578 | { 579 | index, 580 | columnName, 581 | property 582 | }; 583 | }); 584 | 585 | // 3. Dynamic creation method: read the data we want from the database Reader in order 586 | /*Method logic: 587 | User dynamic method (IDataReader reader) 588 | { 589 | var user = new User(); 590 | var value = reader[0]; 591 | if( !(value is System.DBNull)) 592 | user.Name = (string)value; 593 | value = reader[1]; 594 | if( !(value is System.DBNull)) 595 | user.Age = (int)value; 596 | return user; 597 | } 598 | */ 599 | var exBodys = new List < Expression >(); 600 | 601 | { 602 | // method(IDataReader reader) 603 | var exParam = Expression.Parameter(typeof(DbDataReader), "reader"); 604 | 605 | // Mapping class object = new Mapping class(); 606 | var exVar = Expression . Variable ( type , " mappingObj " ); 607 | var exNew = Expression . New ( type ); 608 | { 609 | exBodys.Add(Expression.Assign(exVar, exNew)); 610 | } 611 | 612 | // var value = defalut(object); 613 | var exValueVar = Expression.Variable(typeof(object), "value"); 614 | { 615 | exBodys.Add(Expression.Assign(exValueVar, Expression.Constant(null))); 616 | } 617 | 618 | 619 | var getItemMethod = typeof(DbDataReader).GetMethods().Where(w => w.Name == "get_Item") 620 | .First(w => w.GetParameters().First().ParameterType == typeof(int)); 621 | foreach (var m in members) 622 | { 623 | //reader[0] 624 | var exCall = Expression.Call( 625 | exParam, getItemMethod, 626 | Expression.Constant(m.index) 627 | ); 628 | 629 | // value = reader[0]; 630 | exBodys.Add(Expression.Assign(exValueVar, exCall)); 631 | 632 | //user.Name = (string)value; 633 | var exProp = Expression.Property(exVar, m.property.Name); 634 | var exConvert = Expression.Convert(exValueVar, m.property.PropertyType); //(string)value 635 | var exPropAssign = Expression.Assign(exProp, exConvert); 636 | 637 | //if ( !(value is System.DBNull)) 638 | // (string)value 639 | var exIfThenElse = Expression.IfThen( 640 | Expression.Not(Expression.TypeIs(exValueVar, typeof(System.DBNull))) 641 | , exPropAssign 642 | ); 643 | 644 | exBodys.Add(exIfThenElse); 645 | } 646 | 647 | 648 | // return user; 649 | exBodys.Add(exVar); 650 | 651 | // Compiler Expression 652 | var lambda = Expression.Lambda>( 653 | Expression.Block( 654 | new[] { exVar, exValueVar }, 655 | exBodys 656 | ), exParam 657 | ); 658 | 659 | return lambda.Compile(); 660 | } 661 | } 662 | } 663 | ``` 664 | 665 | ![image](https://user-images.githubusercontent.com/12729184/101126967-822cb000-3637-11eb-8f1f-4b2194951181.png) 666 | 667 | Finally, check Expression.Lambda > DebugView (note that it is a non-public attribute) : 668 | 669 | ```C# 670 | .Lambda #Lambda1(System.Data.Common.DbDataReader $reader) { 671 | .Block( 672 | UserQuery+User $mappingObj, 673 | System.Object $value) { 674 | $mappingObj = .New UserQuery+User(); 675 | $value = null; 676 | $value = .Call $reader.get_Item(0); 677 | .If ( 678 | !($value .Is System.DBNull) 679 | ) { 680 | $mappingObj.Name = (System.String)$value 681 | } .Else { 682 | .Default(System.Void) 683 | }; 684 | $value = .Call $reader.get_Item(1); 685 | .If ( 686 | !($value .Is System.DBNull) 687 | ) { 688 | $mappingObj.Age = (System.Int32)$value 689 | } .Else { 690 | .Default(System.Void) 691 | }; 692 | $mappingObj 693 | } 694 | } 695 | ``` 696 | 697 | ![image](https://user-images.githubusercontent.com/12729184/101127037-a2f50580-3637-11eb-863e-3fa558e19998.png) 698 | 699 | ## 8. Strongly Typed Mapping Part5: Emit IL convert to C# code 700 | 701 | With the concept of the previous Expression version, we can then enter the core technology of Dapper: Emit. 702 | 703 | First of all, there must be a concept, MSIL (CIL) is intended for JIT compiler, so the readability will be poor and difficult to debug, but more detailed logical operations can be done compared to Expression. 704 | 705 | In the actual environment development and use Emit, usually `c# code > Decompilation to IL > use Emit to build dynamic methods` , for example: 706 | 707 | 1. First create a simple printing example: 708 | 709 | ```C# 710 | void SyaHello() 711 | { 712 | Console.WriteLine("Hello World"); 713 | } 714 | ``` 715 | 716 | 2. Decompile and view IL 717 | 718 | ```C# 719 | SyaHello: 720 | IL_0000: nop 721 | IL_0001: ldstr "Hello World" 722 | IL_0006: call System.Console.WriteLine 723 | IL_000B: nop 724 | IL_000C: ret 725 | ``` 726 | 727 | 3. Use DynamicMethod + Emit to create a dynamic method 728 | 729 | ```C# 730 | void Main() 731 | { 732 | // 1. create void method() 733 | DynamicMethod methodbuilder = new DynamicMethod("Deserialize" + Guid.NewGuid().ToString(),typeof(void),null); 734 | 735 | // 2. Create the content of the method body by Emit 736 | var il = methodbuilder.GetILGenerator(); 737 | il.Emit(OpCodes.Ldstr, "Hello World"); 738 | Type[] types = new Type[1] 739 | { 740 | typeof(string) 741 | }; 742 | MethodInfo method = typeof(Console).GetMethod("WriteLine", types); 743 | il.Emit(OpCodes.Call,method); 744 | il.Emit(OpCodes.Ret); 745 | 746 | // 3. Convert the specified type of Func or Action 747 | var action = (Action)methodbuilder.CreateDelegate(typeof(Action)); 748 | 749 | action(); 750 | } 751 | ``` 752 | 753 | But this is not the process for a project that has been written. Developers may not kindly tell you the logic of the original design. 754 | 755 | ### How to check like Dapper, only Emit IL doesn’t have C# Source Code Project 756 | 757 | My solution is: `「Since only Runtime can know IL, save IL as a static file and decompile and view」` 758 | 759 | You can use the `MethodBuild + Save` method here `Save IL as static exe file > Decompile view` , but you need to pay special attention 760 | 761 | 1. Please correspond to the parameters and return type, otherwise it will compile error. 762 | 2. netstandard does not support this method, Dapper needs to be used `region if yourversion` to distinguish, otherwise it cannot be used, such as picture![image](https://user-images.githubusercontent.com/12729184/101128488-c1a8cb80-363a-11eb-8f35-38fafa1ccf8d.png) 763 | 764 | code show as below : 765 | 766 | ```C# 767 | //Use MethodBuilder to view Emit IL that others have written 768 | //1. Create MethodBuilder 769 | AppDomain ad = AppDomain.CurrentDomain; 770 | AssemblyName am = new AssemblyName(); 771 | am.Name = "TestAsm"; 772 | AssemblyBuilder ab = ad.DefineDynamicAssembly(am, AssemblyBuilderAccess.Save); 773 | ModuleBuilder mb = ab.DefineDynamicModule("Testmod", "TestAsm.exe"); 774 | TypeBuilder tb = mb.DefineType("TestType", TypeAttributes.Public); 775 | MethodBuilder dm = tb.DefineMethod("TestMeThod", MethodAttributes.Public | 776 | MethodAttributes.Static, type, new[] { typeof(IDataReader) }); 777 | ab.SetEntryPoint(dm); 778 | 779 | // 2. the IL code 780 | //.. 781 | 782 | // 3. Generate static files 783 | tb.CreateType(); 784 | ab.Save("TestAsm.exe"); 785 | ``` 786 | 787 | Then use this method to decompile Dapper Query Mapping IL in the GetTypeDeserializerImpl method, and you can get the C# code: 788 | 789 | ```C# 790 | public static User TestMeThod(IDataReader P_0) 791 | { 792 | int index = 0; 793 | User user = new User(); 794 | object value = default(object); 795 | try 796 | { 797 | User user2 = user; 798 | index = 0; 799 | object obj = value = P_0[0]; 800 | if (!(obj is DBNull)) 801 | { 802 | user2.Name = (string)obj; 803 | } 804 | index = 1; 805 | object obj2 = value = P_0[1]; 806 | if (!(obj2 is DBNull)) 807 | { 808 | user2.Age = (int)obj2; 809 | } 810 | user = user2; 811 | return user; 812 | } 813 | catch (Exception ex) 814 | { 815 | SqlMapper.ThrowDataException(ex, index, P_0, value); 816 | return user; 817 | } 818 | } 819 | ``` 820 | 821 | ![image](https://user-images.githubusercontent.com/12729184/101128692-25cb8f80-363b-11eb-8673-8a6a0ea8ba3d.png) 822 | 823 | 824 | 825 | After having the C# code, it will be much faster to understand the Emit logic. 826 | 827 | ## 9. Strongly Typed Mapping Principle Part6: Emit Version 828 | 829 | The following code is the Emit version, I wrote the corresponding IL part of C# code 830 | 831 | ```C# 832 | public static class DemoExtension 833 | { 834 | public static IEnumerable Query(this IDbConnection cnn, string sql) where T : new() 835 | { 836 | using (var command = cnn.CreateCommand()) 837 | { 838 | command.CommandText = sql; 839 | using (var reader = command.ExecuteReader()) 840 | { 841 | var func = GetTypeDeserializerImpl(typeof(T), reader); 842 | 843 | while (reader.Read()) 844 | { 845 | var result = func(reader as DbDataReader); 846 | yield return result is T ? (T)result : default(T); 847 | } 848 | } 849 | 850 | } 851 | } 852 | 853 | private static Func GetTypeDeserializerImpl(Type type, IDataReader reader, int startBound = 0, int length = -1, bool returnNullIfFirstMissing = false) 854 | { 855 | var returnType = type.IsValueType ? typeof(object) : type; 856 | 857 | var dm = new DynamicMethod("Deserialize" + Guid.NewGuid().ToString(), returnType, new[] { typeof(IDataReader) }, type, true); 858 | var il = dm.GetILGenerator(); 859 | 860 | //C# : User user = new User(); 861 | //IL : 862 | //IL_0001: newobj 863 | //IL_0006: stloc.0 864 | var constructor = returnType.GetConstructors(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic)[0]; 865 | il.Emit(OpCodes.Newobj, constructor); 866 | var returnValueLocal = il.DeclareLocal(type); 867 | il.Emit(OpCodes.Stloc, returnValueLocal); //User user = new User(); 868 | 869 | // C# : 870 | //object value = default(object); 871 | // IL : 872 | //IL_0007: ldnull 873 | //IL_0008: stloc.1 // value 874 | var valueLoacl = il.DeclareLocal(typeof(object)); 875 | il.Emit(OpCodes.Ldnull); 876 | il.Emit(OpCodes.Stloc, valueLoacl); 877 | 878 | 879 | int index = startBound; 880 | var getItem = typeof(IDataRecord).GetProperties(BindingFlags.Instance | BindingFlags.Public) 881 | .Where(p => p.GetIndexParameters().Length > 0 && p.GetIndexParameters()[0].ParameterType == typeof(int)) 882 | .Select(p => p.GetGetMethod()).First(); 883 | 884 | foreach (var p in type.GetProperties()) 885 | { 886 | //C# : value = P_0[0]; 887 | //IL: 888 | //IL_0009: ldarg.0 889 | //IL_000A: ldc.i4.0 890 | //IL_000B: callvirt System.Data.IDataRecord.get_Item 891 | //IL_0010: stloc.1 // value 892 | il.Emit(OpCodes.Ldarg_0); 893 | EmitInt32(il, index); 894 | il.Emit(OpCodes.Callvirt, getItem); 895 | il.Emit(OpCodes.Stloc, valueLoacl); 896 | 897 | 898 | //C#: if (!(value is DBNull)) user.Name = (string)value; 899 | //IL: 900 | // IL_0011: ldloc.1 // value 901 | // IL_0012: isinst System.DBNull 902 | // IL_0017: ldnull 903 | // IL_0018: cgt.un 904 | // IL_001A: ldc.i4.0 905 | // IL_001B: ceq 906 | // IL_001D: stloc.2 907 | // IL_001E: ldloc.2 908 | // IL_001F: brfalse.s IL_002E 909 | // IL_0021: ldloc.0 // user 910 | // IL_0022: ldloc.1 // value 911 | // IL_0023: castclass System.String 912 | // IL_0028: callvirt UserQuery+User.set_Name 913 | il.Emit(OpCodes.Ldloc, valueLoacl); 914 | il.Emit(OpCodes.Isinst, typeof(System.DBNull)); 915 | il.Emit(OpCodes.Ldnull); 916 | 917 | var tmpLoacl = il.DeclareLocal(typeof(int)); 918 | il.Emit(OpCodes.Cgt_Un); 919 | il.Emit(OpCodes.Ldc_I4_0); 920 | il.Emit(OpCodes.Ceq); 921 | 922 | il.Emit(OpCodes.Stloc,tmpLoacl); 923 | il.Emit(OpCodes.Ldloc,tmpLoacl); 924 | 925 | 926 | var labelFalse = il.DefineLabel(); 927 | il.Emit(OpCodes.Brfalse_S,labelFalse); 928 | il.Emit(OpCodes.Ldloc, returnValueLocal); 929 | il.Emit(OpCodes.Ldloc, valueLoacl); 930 | if (p.PropertyType.IsValueType) 931 | il.Emit(OpCodes.Unbox_Any, p.PropertyType); 932 | else 933 | il.Emit(OpCodes.Castclass, p.PropertyType); 934 | il.Emit(OpCodes.Callvirt, p.SetMethod); 935 | 936 | il.MarkLabel(labelFalse); 937 | 938 | index++; 939 | } 940 | 941 | // IL_0053: ldloc.0 // user 942 | // IL_0054: stloc.s 04 943 | // IL_0056: br.s IL_0058 944 | // IL_0058: ldloc.s 04 945 | // IL_005A: ret 946 | il.Emit(OpCodes.Ldloc, returnValueLocal); 947 | il.Emit(OpCodes.Ret); 948 | 949 | var funcType = System.Linq.Expressions.Expression.GetFuncType(typeof(IDataReader), returnType); 950 | return (Func)dm.CreateDelegate(funcType); 951 | } 952 | 953 | private static void EmitInt32(ILGenerator il, int value) 954 | { 955 | switch (value) 956 | { 957 | case -1: il.Emit(OpCodes.Ldc_I4_M1); break; 958 | case 0: il.Emit(OpCodes.Ldc_I4_0); break; 959 | case 1: il.Emit(OpCodes.Ldc_I4_1); break; 960 | case 2: il.Emit(OpCodes.Ldc_I4_2); break; 961 | case 3: il.Emit(OpCodes.Ldc_I4_3); break; 962 | case 4: il.Emit(OpCodes.Ldc_I4_4); break; 963 | case 5: il.Emit(OpCodes.Ldc_I4_5); break; 964 | case 6: il.Emit(OpCodes.Ldc_I4_6); break; 965 | case 7: il.Emit(OpCodes.Ldc_I4_7); break; 966 | case 8: il.Emit(OpCodes.Ldc_I4_8); break; 967 | default: 968 | if (value >= -128 && value <= 127) 969 | { 970 | il.Emit(OpCodes.Ldc_I4_S, (sbyte)value); 971 | } 972 | else 973 | { 974 | il.Emit(OpCodes.Ldc_I4, value); 975 | } 976 | break; 977 | } 978 | } 979 | } 980 | ``` 981 | 982 | There are many detailed of Emit here. First pick out the important concepts to explain. 983 | 984 | **Emit Label** 985 | 986 | In Emit if/else, you need to use Label positioning, tell the compiler which position to jump to when the condition is true/false, for example: `boolean to integer`, assuming that you want to simply convert Boolean to Int, C# code can use `If it is True Return 1 otherwise return 0` logic to write: 987 | 988 | ```C# 989 | public static int BoolToInt(bool input) => input ? 1 : 0; 990 | ``` 991 | 992 | When converting to Emit, the following logic is required: 993 | 994 | 1. Consider the label dynamic positioning problem 995 | 2. The label must be established first to let Brtrue\_S know which label position to set when the conditions are true `(Note:at this time the label position has not been determined yet)` 996 | 3. Continue to build IL from top to bottom in order 997 | 4. Wait until `match condition` you want to run the block `previous line` , use it `MarkLabel to position Label` . 998 | 999 | The final c # Emit Code: 1000 | 1001 | ```C# 1002 | public class Program 1003 | { 1004 | public static void Main(string[] args) 1005 | { 1006 | var func = CreateFunc(); 1007 | Console.WriteLine(func(true)); //1 1008 | Console.WriteLine(func(false)); //0 1009 | } 1010 | 1011 | static Func CreateFunc() 1012 | { 1013 | var dm = new DynamicMethod("Test" + Guid.NewGuid().ToString(), typeof(int), new[] { typeof(bool) }); 1014 | 1015 | var il = dm.GetILGenerator(); 1016 | var labelTrue = il.DefineLabel(); 1017 | 1018 | il.Emit(OpCodes.Ldarg_0); 1019 | il.Emit(OpCodes.Brtrue_S, labelTrue); 1020 | il.Emit(OpCodes.Ldc_I4_0); 1021 | il.Emit(OpCodes.Ret); 1022 | il.MarkLabel(labelTrue); 1023 | il.Emit(OpCodes.Ldc_I4_1); 1024 | il.Emit(OpCodes.Ret); 1025 | 1026 | var funcType = System.Linq.Expressions.Expression.GetFuncType(typeof(bool), typeof(int)); 1027 | return (Func)dm.CreateDelegate(funcType); 1028 | } 1029 | } 1030 | ``` 1031 | 1032 | Here you can find the Emit version, which has the advantage of: 1033 | 1034 | 1. Can do more detailed operations 1035 | 1036 | 2. Because the detail granularity is small, the efficiency that can be optimized is better 1037 | 1038 | Disadvantages: 1039 | 1040 | 1. Difficult to debug 1041 | 2. Poor readability 1042 | 3. The amount of code becomes larger and the complexity increases 1043 | 1044 | Then look at the suggestions of the author of Dapper. Now there is no need to use Emit in general projects. Using Expression + Func/Action can solve most of the needs of dynamic methods, especially when Expression supports Block and other methods. Link [c#\-What's faster: expression trees or manually emitting IL](https://stackoverflow.com/questions/16530539/whats-faster-expression-trees-or-manually-emitting-il) 1045 | 1046 | ![image](https://user-images.githubusercontent.com/12729184/101129368-67a90580-363c-11eb-84c4-019c290fc8d0.png) 1047 | 1048 | Having said that, there are some powerful open source projects that use Emit to manage details `If you want to understand them, you need the basic Emit IL concept` . 1049 | 1050 | ## 10. One of the keys to Dapper's fast efficiency: Cache principle 1051 | 1052 | Why can Dapper be so fast? 1053 | 1054 | I introduced the dynamic use of Emit IL to establish the ADO.NET Mapping method, but this function cannot make Dapper the king of lightweight ORM efficiency. 1055 | 1056 | Because the dynamic create method is `Cost and time consuming` action, simply using it will slow down the speed. But when it cooperates with Cache, it is different. By storing the established method in Cache, you can use the `『Space for time』` concept to speed up the efficiency of the query . 1057 | 1058 | Then trace the Dapper source code. This time, we need to pay special attention to Identity and GetCacheInfo under the QueryImpl method. 1059 | 1060 | ![image](https://user-images.githubusercontent.com/12729184/101129638-ef8f0f80-363c-11eb-8463-73a9ee081b1b.png) 1061 | 1062 | **Identity、GetCacheInfo** 1063 | 1064 | Identity mainly encapsulates the comparison Key attribute of each cache: 1065 | 1066 | * sql: distinguish different SQL strings 1067 | * type: distinguish Mapping type 1068 | * commandType: Responsible for distinguishing different databases 1069 | * gridIndex: Mainly used in QueryMultiple, explained later. 1070 | * connectionString: Mainly distinguish the same database manufacturer but different DB situation 1071 | * parametersType: Mainly distinguish parameter types 1072 | * typeCount: Mainly used in Multi Query multi\-mapping, it needs to be used with the override GetType method, which will be explained later 1073 | 1074 | Then match the cache type used by Dapper in the GetCacheInfo method. When `ConcurrentDictionary` using the `TryGetValue` method, it will first compare the HashCode and then compare the Equals features, such as the image source code. 1075 | 1076 | ![image](https://user-images.githubusercontent.com/12729184/101129762-2ebd6080-363d-11eb-898c-8fa84b7205fa.png) 1077 | 1078 | Using the Key type Identity to `override Equals` implement the cache comparison algorithm, you can see the following Dapper implementation logic. As long as one attribute is different, a new dynamic method and cache will be created. 1079 | 1080 | ```C# 1081 | public bool Equals(Identity other) 1082 | { 1083 | if (ReferenceEquals(this, other)) return true; 1084 | if (ReferenceEquals(other, null)) return false; 1085 | 1086 | int typeCount; 1087 | return gridIndex == other.gridIndex 1088 | && type == other.type 1089 | && sql == other.sql 1090 | && commandType == other.commandType 1091 | && connectionStringComparer.Equals(connectionString, other.connectionString) 1092 | && parametersType == other.parametersType 1093 | && (typeCount = TypeCount) == other.TypeCount 1094 | && (typeCount == 0 || TypesEqual(this, other, typeCount)); 1095 | } 1096 | ``` 1097 | 1098 | With this concept, the previous Emit version is modified into a simple Cache Demo : 1099 | 1100 | ```C# 1101 | public class Identity 1102 | { 1103 | public string sql { get; set; } 1104 | public CommandType? commandType { get; set; } 1105 | public string connectionString { get; set; } 1106 | public Type type { get; set; } 1107 | public Type parametersType { get; set; } 1108 | public Identity(string sql, CommandType? commandType, string connectionString, Type type, Type parametersType) 1109 | { 1110 | this.sql = sql; 1111 | this.commandType = commandType; 1112 | this.connectionString = connectionString; 1113 | this.type = type; 1114 | this.parametersType = parametersType; 1115 | unchecked 1116 | { 1117 | hashCode = 17; // we *know* we are using this in a dictionary, so pre-compute this 1118 | hashCode = (hashCode * 23) + commandType.GetHashCode(); 1119 | hashCode = (hashCode * 23) + (sql?.GetHashCode() ?? 0); 1120 | hashCode = (hashCode * 23) + (type?.GetHashCode() ?? 0); 1121 | hashCode = (hashCode * 23) + (connectionString == null ? 0 : StringComparer.Ordinal.GetHashCode(connectionString)); 1122 | hashCode = (hashCode * 23) + (parametersType?.GetHashCode() ?? 0); 1123 | } 1124 | } 1125 | 1126 | public readonly int hashCode; 1127 | public override int GetHashCode() => hashCode; 1128 | 1129 | public override bool Equals(object obj) => Equals(obj as Identity); 1130 | public bool Equals(Identity other) 1131 | { 1132 | if (ReferenceEquals(this, other)) return true; 1133 | if (ReferenceEquals(other, null)) return false; 1134 | 1135 | return type == other.type 1136 | && sql == other.sql 1137 | && commandType == other.commandType 1138 | && StringComparer.Ordinal.Equals(connectionString, other.connectionString) 1139 | && parametersType == other.parametersType; 1140 | } 1141 | } 1142 | 1143 | public static class DemoExtension 1144 | { 1145 | private static readonly Dictionary> readers = new Dictionary>(); 1146 | 1147 | public static IEnumerable Query(this IDbConnection cnn, string sql, object param = null) where T : new() 1148 | { 1149 | using (var command = cnn.CreateCommand()) 1150 | { 1151 | command.CommandText = sql; 1152 | using (var reader = command.ExecuteReader()) 1153 | { 1154 | var identity = new Identity(command.CommandText, command.CommandType, cnn.ConnectionString, typeof(T), param?.GetType()); 1155 | 1156 | // 2. If the cache has data, use it, and if there is no data, create a method dynamically and save it in the cache 1157 | if (!readers.TryGetValue(identity, out Func func)) 1158 | { 1159 | //The dynamic creation method 1160 | func = GetTypeDeserializerImpl(typeof(T), reader); 1161 | readers[identity] = func; 1162 | Console.WriteLine(" No cache, create a dynamic method and put it in the cache "); 1163 | } 1164 | else 1165 | { 1166 | Console.WriteLine(" Use cache "); 1167 | } 1168 | 1169 | 1170 | // 3. Call the generated method by reader, read the data and return 1171 | while (reader.Read()) 1172 | { 1173 | var result = func(reader as DbDataReader); 1174 | yield return result is T ? (T)result : default(T); 1175 | } 1176 | } 1177 | 1178 | } 1179 | } 1180 | 1181 | private static Func GetTypeDeserializerImpl(Type type, IDataReader reader, int startBound = 0, int length = -1, bool returnNullIfFirstMissing = false) 1182 | { 1183 | // .. 1184 | } 1185 | } 1186 | ``` 1187 | 1188 | ![image](https://user-images.githubusercontent.com/12729184/101130013-a5f2f480-363d-11eb-8e1d-1b9851c6e5b2.png) 1189 | 1190 | ## 11. Wrong SQL string concating will cause slow efficiency and memory leaks 1191 | 1192 | Here's an important concept used by Dapper. Its `SQL string` is one of the important key values to cache. If different SQL strings are used, Dapper will create new dynamic methods and caches for this, so even if you use StringBuilder improperly `can also cause slow query & memory leaks` . 1193 | 1194 | ![image](https://user-images.githubusercontent.com/12729184/101131153-c8860d00-363f-11eb-9553-f50baafbb2a7.png) 1195 | 1196 | 1197 | 1198 | Why the SQL string is used as one of keys, instead of simply using the Handle of the Mapping type, one of the reasons is `order of query column ` . As mentioned earlier, Dapper uses the `「result convert to code」` method to create a dynamic method, which means that the order and data must be `fixed` , avoid using the same set of dynamic methods with different SQL Select column order, there will be a `A column value to b column` wrong value problem. 1199 | 1200 | The most direct solution is to establish a different dynamic method for each different SQL string and save it in a different cache. 1201 | 1202 | For example, the following code is just a simple query action, but the number of Dapper Caches has reached 999999, such as image display 1203 | 1204 | ```C# 1205 | using (var cn = new SqlConnection(@"connectionString")) 1206 | { 1207 | for ( int i = 0; i < 999999 ; i ++ ) 1208 | { 1209 | var guid = Guid.NewGuid(); 1210 | for (int i2 = 0; i2 < 2; i2++) 1211 | { 1212 | var result = cn.Query($"select '{guid}' ").First(); 1213 | } 1214 | } 1215 | } 1216 | ``` 1217 | 1218 | ![image](https://user-images.githubusercontent.com/12729184/101131682-aa6cdc80-3640-11eb-9b71-697496d29b3b.png) 1219 | 1220 | To avoid this problem, you only need to maintain a principle `Reuse SQL string` , and the simplest way is `parametrization` , for example: Change the above code to the following code, the number of caches is reduced to `1` , to achieve the purpose of reuse: 1221 | 1222 | ```C# 1223 | using (var cn = new SqlConnection(@"connectionString")) 1224 | { 1225 | for ( int i = 0; i < 999999 ; i ++ ) 1226 | { 1227 | var guid = Guid.NewGuid(); 1228 | for (int i2 = 0; i2 < 2; i2++) 1229 | { 1230 | var result = cn.Query($"select @guid ",new { guid}).First(); 1231 | } 1232 | } 1233 | } 1234 | ``` 1235 | 1236 | ![image](https://user-images.githubusercontent.com/12729184/101131840-e9029700-3640-11eb-8f95-a4d317dc3842.png) 1237 | 1238 | ## 12. Dapper SQL correct string concating method: Literal Replacement 1239 | 1240 | If there is a need to splice SQL strings, for example: Sometimes it is more efficient to use string concating than not to use parameterization, especially if there are only a few `fixed values` . 1241 | 1242 | At this time, Dapper can use the `Literal Replacements` function, how to use it: `{=Attribute_Name}` replace the value string to be concated, and save the value in the Parameter, for example: 1243 | 1244 | ```C# 1245 | void Main() 1246 | { 1247 | using (var cn = Connection) 1248 | { 1249 | var result = cn.Query("select N'Wei' Name,26 Age,{=VipLevel} VipLevel", new User{ VipLevel = 1}).First(); 1250 | } 1251 | } 1252 | ``` 1253 | 1254 | ## 13. Why Literal Replacement can avoid caching problems? 1255 | 1256 | First, trace the GetLiteralTokens method under the source code GetCacheInfo, you can find that before cache Dapper will get the data `SQL string` that match `{=Attribute_Name}` role . 1257 | 1258 | ```C# 1259 | private static readonly Regex literalTokens = new Regex(@"(? GetLiteralTokens(string sql) 1261 | { 1262 | if (string.IsNullOrEmpty(sql)) return LiteralToken.None; 1263 | if (!literalTokens.IsMatch(sql)) return LiteralToken.None; 1264 | 1265 | var matches = literalTokens.Matches(sql); 1266 | var found = new HashSet(StringComparer.Ordinal); 1267 | List list = new List(matches.Count); 1268 | foreach (Match match in matches) 1269 | { 1270 | string token = match.Value; 1271 | if (found.Add(match.Value)) 1272 | { 1273 | list.Add(new LiteralToken(token, match.Groups[1].Value)); 1274 | } 1275 | } 1276 | return list.Count == 0 ? LiteralToken.None : list; 1277 | } 1278 | ``` 1279 | 1280 | Then generate Parameter parameterized dynamic method in the CreateParamInfoGenerator method. The method IL of this section is as below: 1281 | 1282 | ```C# 1283 | IL_0000: ldarg.1 1284 | IL_0001: castclass <>f__AnonymousType1`1[System.Int32] 1285 | IL_0006: stloc.0 1286 | IL_0007: ldarg.0 1287 | IL_0008: callvirt System.Data.IDataParameterCollection get_Parameters()/System.Data.IDbCommand 1288 | IL_000d: pop 1289 | IL_000e: ldarg.0 1290 | IL_000f: ldarg.0 1291 | IL_0010: callvirt System.String get_CommandText()/System.Data.IDbCommand 1292 | IL_0015: ldstr "{=VipLevel}" 1293 | IL_001a: ldloc.0 1294 | IL_001b: callvirt Int32 get_VipLevel()/<>f__AnonymousType1`1[System.Int32] 1295 | IL_0020: stloc.1 1296 | IL_0021: ldloca.s V_1 1297 | 1298 | IL_0023: call System.Globalization.CultureInfo get_InvariantCulture()/System.Globalization.CultureInfo 1299 | IL_0028: call System.String ToString(System.IFormatProvider)/System.Int32 1300 | IL_002d: callvirt System.String Replace(System.String, System.String)/System.String 1301 | IL_0032: callvirt Void set_CommandText(System.String)/System.Data.IDbCommand 1302 | IL_0037: ret 1303 | ``` 1304 | 1305 | Then generate the Mapping dynamic method. To understand this logic, I will make a simulation example here: 1306 | 1307 | ```C# 1308 | public static class DbExtension 1309 | { 1310 | public static IEnumerable Query(this DbConnection cnn, string sql, User parameter) 1311 | { 1312 | using (var command = cnn.CreateCommand()) 1313 | { 1314 | command.CommandText = sql; 1315 | CommandLiteralReplace(command, parameter); 1316 | using (var reader = command.ExecuteReader()) 1317 | while (reader.Read()) 1318 | yield return Mapping(reader); 1319 | } 1320 | } 1321 | 1322 | private static void CommandLiteralReplace(IDbCommand cmd, User parameter) 1323 | { 1324 | cmd.CommandText = cmd.CommandText.Replace("{=VipLevel}", parameter.VipLevel.ToString(System.Globalization.CultureInfo.InvariantCulture)); 1325 | } 1326 | 1327 | private static User Mapping ( IDataReader reader ) 1328 | { 1329 | var user = new User(); 1330 | var value = default(object); 1331 | value = reader[0]; 1332 | if(!(value is System.DBNull)) 1333 | user.Name = (string)value; 1334 | value = reader[1]; 1335 | if (!(value is System.DBNull)) 1336 | user.Age = (int)value; 1337 | value = reader[2]; 1338 | if (!(value is System.DBNull)) 1339 | user.VipLevel = (int)value; 1340 | return user; 1341 | } 1342 | } 1343 | ``` 1344 | 1345 | After reading the above example, you can find that the underlying principle of Dapper Literal Replacements is `string replace` that it also belongs to the string concating way. Why can the cache problem be avoided? 1346 | 1347 | This is because the replacement timing is in the SetParameter dynamic method, so the Cache `SQL Key is unchanged` can reuse the same SQL string and cache. 1348 | 1349 | Also because it is a string replace method, `only support basic value type` if you use the String type, the system will inform you `The type String is not supported for SQL literals.`to avoid SQL Injection problems. 1350 | 1351 | ## How to use Query Multi Mapping 1352 | 1353 | Then explain the `Dapper Multi Mapping`(multi-mapping) implementation and the underlying logic. After all, there can not always one-to-one relation in work. 1354 | 1355 | How to use: 1356 | 1357 | * You need to write your own Mapping logic and use it: `Query(SQL,Parameter,Mapping Func)` 1358 | * Need to specify the generic parameter type, the rule is `Query` (supports up to six sets of generic parameters) 1359 | * Specify the name of the cutting field `ID` , it is used by default , if it is different, it needs to be specified . 1360 | * The order is `from left to right` 1361 | 1362 | For example: There is an order (Order) and a member (User) form, the relationship is a one-to-many relationship, a member can have multiple orders, the following is the C# Demo code: 1363 | 1364 | ```C# 1365 | void Main() 1366 | { 1367 | using (var ts = new TransactionScope()) 1368 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 1369 | { 1370 | cn.Execute(@" 1371 | CREATE TABLE [User]([ID] int, [Name] nvarchar(10)); 1372 | INSERT INTO [User]([ID], [Name])VALUES(1, N'Jack'),(2, N'Lee'); 1373 | 1374 | CREATE TABLE [Order]([ID] int, [OrderNo] varchar(13), [UserID] int); 1375 | INSERT INTO [Order]([ID], [OrderNo], [UserID])VALUES(1, 'SO20190900001', 1),(2, 'SO20190900002', 1),(3, 'SO20190900003', 2),(4, 'SO20190900004', 2); 1376 | "); 1377 | 1378 | var result = cn.Query(@" 1379 | select * from [order] T1 1380 | left join [User] T2 on T1.UserId = T2.ID 1381 | ", (order, user) => 1382 | { 1383 | order.User = user; 1384 | return order; 1385 | } 1386 | ); 1387 | 1388 | ts.Dispose(); 1389 | } 1390 | } 1391 | 1392 | public class Order 1393 | { 1394 | public int ID { get; set; } 1395 | public string OrderNo { get; set; } 1396 | public User User { get; set; } 1397 | } 1398 | 1399 | public class User 1400 | { 1401 | public int ID { get; set; } 1402 | public string Name { get; set; } 1403 | } 1404 | ``` 1405 | 1406 | ![image](https://user-images.githubusercontent.com/12729184/101133422-ab533d80-3643-11eb-8df6-38d2424d50e4.png) 1407 | 1408 | ## 14. Support dynamic Multi Mapping 1409 | 1410 | In the initial stage, the table structure is often changed or the one-time function and does not want to declare the class. Dapper Multi Mapping also supports the dynamic method. 1411 | 1412 | ```C# 1413 | void Main () 1414 | { 1415 | using (var ts = new TransactionScope()) 1416 | using (var connection = Connection) 1417 | { 1418 | const string createSql = @" 1419 | create table Users (Id int, Name nvarchar(20)) 1420 | create table Posts (Id int, OwnerId int, Content nvarchar(20)) 1421 | 1422 | insert Users values(1, N'Jack') 1423 | insert Users values(2, N'Lee') 1424 | 1425 | insert Posts values(101, 1, N'Jack's first day diary') 1426 | insert Posts values(102, 1, N'Jack's second day diary') 1427 | insert Posts values(103, 2, N'Lee's first day diary ') 1428 | " ; 1429 | connection . Execute ( createSql ); 1430 | 1431 | const string sql = 1432 | @"select * from Posts p 1433 | left join Users u on u.Id = p.OwnerId 1434 | Order by p.Id 1435 | "; 1436 | 1437 | var data = connection.Query(sql, (post, user) => { post.Owner = user; return post; }).ToList(); 1438 | } 1439 | } 1440 | ``` 1441 | 1442 | ## 15. SplitOn distinguish type Mapping group 1443 | 1444 | Split Default is used to cut the primary key, so default cut string is `ID`, if the table structure PK name is ID can omit parameters, for example 1445 | 1446 | ```C# 1447 | var result = cn.Query(@" 1448 | select * from [order] T1 1449 | left join [User] T2 on T1.UserId = T2.ID 1450 | ", (order, user) => { 1451 | order.User = user; 1452 | return order; 1453 | } 1454 | ); 1455 | ``` 1456 | 1457 | If the primary key name is another name, ` specify the splitOn string name` and it corresponds to multiple names, it can be used `,`as a segmentation. For example, add a product table as Join: 1458 | 1459 | ```C# 1460 | var result = cn.Query(@" 1461 | select * from [order] T1 1462 | left join [User] T2 on T1.UserId = T2.ID 1463 | left join [Item] T3 on T1.ItemId = T3.ID 1464 | " 1465 | 1466 | ,map : (order, user,item) => { 1467 | order.User = user; 1468 | order.Item = item; 1469 | return order; 1470 | } 1471 | ,splitOn : "Id,Id" 1472 | ); 1473 | ``` 1474 | 1475 | ## 16. Query Multi Mapping underlying principle 1476 | 1477 | First, a simple Demo. 1478 | 1479 | 1. Create a Mapping FUNC collection corresponding to the number of generic class parameters 1480 | 2. The Mapping FUNC setup logic is the same as Query Emit IL 1481 | 3. Call the user's Custom Mapping Func, where the parameters are derived from the previously dynamically generated Mapping Func 1482 | 1483 | ```C# 1484 | public static class MutipleMappingDemo 1485 | { 1486 | public static IEnumerable Query(this IDbConnection connection, string sql, Func map) 1487 | where T1 : Order, new() 1488 | where T2 : User, new() 1489 | { 1490 | // 1. Create a Mapping FUNC collection corresponding to the number of generic class parameters 1491 | var deserializers = new List>(); 1492 | { 1493 | // 2. The Mapping FUNC setup logic is the same as Query Emit IL 1494 | deserializers.Add((reader) => 1495 | { 1496 | var newObj = new T1(); 1497 | var value = default(object); 1498 | value = reader[0]; 1499 | newObj.ID = value is DBNull ? 0 : (int)value; 1500 | value = reader[1]; 1501 | newObj.OrderNo = value is DBNull ? null : (string)value; 1502 | return newObj; 1503 | }); 1504 | 1505 | deserializers.Add((reader) => 1506 | { 1507 | var newObj = new T2(); 1508 | var value = default(object); 1509 | value = reader[2]; 1510 | newObj.ID = value is DBNull ? 0 : (int)value; 1511 | value = reader[4]; 1512 | newObj.Name = value is DBNull ? null : (string)value; 1513 | return newObj; 1514 | }); 1515 | } 1516 | 1517 | 1518 | using (var command = connection.CreateCommand()) 1519 | { 1520 | command.CommandText = sql; 1521 | using (var reader = command.ExecuteReader()) 1522 | { 1523 | while (reader.Read()) 1524 | { 1525 | // 3. Call the user's Custom Mapping Func, where the parameters are derived from the previously dynamically generated Mapping Func 1526 | yield return map(deserializers[0](reader) as T1, deserializers[1](reader) as T2); 1527 | } 1528 | } 1529 | } 1530 | } 1531 | } 1532 | ``` 1533 | 1534 | ### Support multiple groups of type + strongly typed return values 1535 | 1536 | Dapper using multiple generic parameter methods for ` strongly typed multi-class Mapping` has disadvantage that it can not be dynamically adjusted and needs to be fixed. 1537 | 1538 | For example, you can see that the image GenerateMapper method fix the strong transition logic in terms of the number of generic arguments, which is why Multiple Query has a maximum number of groups and can only support up to six. 1539 | 1540 | ![image](https://user-images.githubusercontent.com/12729184/101134421-4f89b400-3645-11eb-905e-b8fc58aeb55a.png) 1541 | 1542 | ### Multi-Class generic caching algorithm 1543 | 1544 | - Dapper use `Generic Class`to save multiple types of data by strong-type 1545 | [![20191001175139.png](https://camo.githubusercontent.com/720ebdacbccc81455dc58e0bcd06e3c843c24ac4169d03a4b1e9ec97661eae8c/68747470733a2f2f692e6c6f6c692e6e65742f323031392f31302f30312f356553666b6f615169495058767a342e706e67)](https://camo.githubusercontent.com/720ebdacbccc81455dc58e0bcd06e3c843c24ac4169d03a4b1e9ec97661eae8c/68747470733a2f2f692e6c6f6c692e6e65742f323031392f31302f30312f356553666b6f615169495058767a342e706e67) 1546 | - And cooperate with inheritance to share most of the identity verification logic 1547 | - Provide available `override` GetType method to customize generic comparison logic to avoid non-multiple query `Cache conflict`. 1548 | 1549 | ![image](https://user-images.githubusercontent.com/12729184/101134891-fff7b800-3645-11eb-814d-e73cc5671260.png) 1550 | 1551 | ![image](https://user-images.githubusercontent.com/12729184/101134914-084ff300-3646-11eb-9d13-f1b8721d9a07.png) 1552 | 1553 | ### Select order of Dapper Query Multi Mapping is important 1554 | 1555 | Because of SplitOn group logic depend on `Select Order`, it is possible that `attribute value wrong` when sequence is wrong . 1556 | 1557 | Example: If the SQL in the above example is changed to the following, the ID of User will become the ID of Order; the ID of Order will become the ID of User. 1558 | 1559 | ```sql 1560 | select T2.[ID],T1.[OrderNo],T1.[UserID],T1.[ID],T2.[Name] from [order] T1 1561 | left join [User] T2 on T1.UserId = T2.ID 1562 | ``` 1563 | 1564 | The reason can be traced to Dapper's cutting algorithm 1565 | 1566 | 1. First, the field group by `reverse order`, the GetNextSplit method can be seen `DataReader Index` from `large to small`. 1567 | ![image](https://user-images.githubusercontent.com/12729184/101135549-eb67ef80-3646-11eb-8230-b70ae986b343.png) 1568 | 1569 | 2. Then process the Mapping Emit IL Func of the type in `reverse order` 1570 | 1571 | 3. Finally, it is reversed to `positive order`, which is convenient for the use of Call Func corresponding to generics later. 1572 | 1573 | ![image](https://user-images.githubusercontent.com/12729184/101135698-1f431500-3647-11eb-8b24-9753f9b46065.png) 1574 | 1575 | ![image](https://user-images.githubusercontent.com/12729184/101135716-25d18c80-3647-11eb-9ed0-4c6ce1ddea75.png) 1576 | 1577 | ![image](https://user-images.githubusercontent.com/12729184/101135734-2c600400-3647-11eb-955f-aced6db3b221.png) 1578 | 1579 | ## 17. QueryMultiple underlying logic 1580 | 1581 | Example: 1582 | 1583 | ```C# 1584 | using (var cn = Connection) 1585 | { 1586 | using (var gridReader = cn.QueryMultiple("select 1; select 2;")) 1587 | { 1588 | Console.WriteLine(gridReader.Read()); //result : 1 1589 | Console.WriteLine(gridReader.Read()); //result : 2 1590 | } 1591 | } 1592 | ``` 1593 | 1594 | Advantages of using QueryMultiple: 1595 | 1596 | - Mainly reduce the number of Reqeust 1597 | - Multiple queries can `share the same set of parameter` 1598 | 1599 | The underlying implementation logic of QueryMultiple: 1600 | 1601 | - The underlying technology is ADO.NET-DataReader-MultipleResult 1602 | - QueryMultiple gets DataReader and encapsulates it into GridReader 1603 | - The Mapping dynamic method is only created when the Read method is called, and the Emit IL action is the same as the Query method. 1604 | - Then call ADO.NET `DataReader NextResult` to get the next set of query result 1605 | - `DataReader will be released` if there is no next set of query results 1606 | 1607 | ### Cache algorithm 1608 | 1609 | The caching algorithm adds more gridIndex judgments, mainly for each result mapping action as a cache. 1610 | 1611 | ![image](https://user-images.githubusercontent.com/12729184/101137558-fb350300-3649-11eb-9ddf-d8b7e32b6132.png) 1612 | 1613 | ### No delayed query feature 1614 | 1615 | Note that the Read method uses `buffer = true` the returned result is directly stored in the `ToList memory`, so there is no delayed query feature. 1616 | 1617 | ![image](https://user-images.githubusercontent.com/12729184/101137683-2c153800-364a-11eb-8b91-670ae17e49d6.png) 1618 | 1619 | ![image](https://user-images.githubusercontent.com/12729184/101137695-30d9ec00-364a-11eb-91ae-4b176c30dc74.png) 1620 | 1621 | ### Remember to manage the release of DataReader 1622 | 1623 | When Dapper calls the QueryMultiple method, the DataReader is encapsulated in the GridReader object, and the DataReader will be recycled only after `the last Read action`. 1624 | 1625 | ![image](https://user-images.githubusercontent.com/12729184/101137793-5666f580-364a-11eb-8a4f-033f921ebf37.png) 1626 | 1627 | Therefore, if you open a GridReader > Read before finishing reading, an error will show: `a DataReader related to this Command has been opened, and it must be closed first.` 1628 | 1629 | To avoid the above situation, you can change to the `using` block, and the DataReader will be automatically released after running the block code. 1630 | 1631 | ## 18. TypeHandler custom Mapping logic & its underlying logic 1632 | 1633 | When you want to customize some attribute Mapping logic, you can use `TypeHandler` in Dapper 1634 | 1635 | 1. Create a class to inherit SqlMapper.TypeHandler 1636 | 2. Assign the class to be customized to the generic, e.g: `JsonTypeHandler: SqlMapper.TypeHandler` 1637 | 3. Override `Parse` method to custom `Query` logic, and override `SetValue` method to custom `Create,Delete,Updte` logic 1638 | 4. If there are multiple class Parse and SetValue share the same logic, you can change the implementation class to a generic method. The custom class can be specified in `AddTypeHandler`, which can avoid creating a lot of class, eg: `JsonTypeHandler: SqlMapper.TypeHandler< T> where T: class` 1639 | 1640 | Example : when User level is changed, the change action will be automatically recorded in the Log field. 1641 | 1642 | ```C# 1643 | public class JsonTypeHandler : SqlMapper.TypeHandler 1644 | where T : class 1645 | { 1646 | public override T Parse(object value) 1647 | { 1648 | return JsonConvert.DeserializeObject((string)value); 1649 | } 1650 | 1651 | public override void SetValue(IDbDataParameter parameter, T value) 1652 | { 1653 | parameter.Value = JsonConvert.SerializeObject(value); 1654 | } 1655 | } 1656 | 1657 | public void Main() 1658 | { 1659 | SqlMapper.AddTypeHandler(new JsonTypeHandler>()); 1660 | 1661 | using (var ts = new TransactionScope()) 1662 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 1663 | { 1664 | 1665 | cn.Execute("create table [User] (Name nvarchar(200),Age int,Level int,Logs nvarchar(max))"); 1666 | 1667 | var user = new User() 1668 | { 1669 | Name = "Wei", 1670 | Age = 26, 1671 | Level = 1, 1672 | Logs = new List() { 1673 | new Log(){Time=DateTime.Now,Remark="CreateUser"} 1674 | } 1675 | }; 1676 | 1677 | // add 1678 | { 1679 | cn.Execute("insert into [User] (Name,Age,Level,Logs) values (@Name,@Age,@Level,@Logs);", user); 1680 | 1681 | var result = cn.Query("select * from [User]"); 1682 | Console.WriteLine(result); 1683 | } 1684 | 1685 | // Level up 1686 | { 1687 | user.Level = 9; 1688 | user.Logs.Add(new Log() {Remark="UpdateLevel"}); 1689 | cn.Execute("update [User] set Level = @Level,Logs = @Logs where Name = @Name", user); 1690 | var result = cn.Query("select * from [User]"); 1691 | Console.WriteLine(result); 1692 | } 1693 | 1694 | ts.Dispose(); 1695 | 1696 | } 1697 | } 1698 | 1699 | public class User 1700 | { 1701 | public string Name { get; set; } 1702 | public int Age { get; set; } 1703 | public int Level { get; set; } 1704 | public List Logs { get; set; } 1705 | 1706 | } 1707 | public class Log 1708 | { 1709 | public DateTime Time { get; set; } = DateTime.Now; 1710 | public string Remark { get; set; } 1711 | } 1712 | ``` 1713 | 1714 | ![image](https://user-images.githubusercontent.com/12729184/101142670-e27c1b80-3650-11eb-992d-4d7b96069460.png) 1715 | 1716 | Then trace the TypeHandler source code logic, which needs to be traced in two parts: SetValue, Parse 1717 | 1718 | ### The underlying logic of SetValue 1719 | 1720 | 1. AddTypeHandlerImpl method to manage the addition of cache 1721 | 1722 | 2. When creating a dynamic AddParameter method in the CreateParamInfoGenerator method Emit, if there is data in the TypeHandler cache of the Mapping type, Emit adds an action to call the SetValue method. 1723 | 1724 | ```C# 1725 | if (handler != null) 1726 | { 1727 | il.Emit(OpCodes.Call, typeof(TypeHandlerCache<>).MakeGenericType(prop.PropertyType).GetMethod(nameof(TypeHandlerCache.SetValue))); // stack is now [parameters] [[parameters]] [parameter] 1728 | } 1729 | ``` 1730 | 1731 | 3. LookupDbType will be used when calling the AddParameters method at Runtime to determine whether there is a custom TypeHandler 1732 | 1733 | ![image](https://user-images.githubusercontent.com/12729184/101142913-338c0f80-3651-11eb-8766-b577f1d9fcc1.png) 1734 | 1735 | ![image](https://user-images.githubusercontent.com/12729184/101142928-37b82d00-3651-11eb-9c08-e40cdcc42f60.png) 1736 | 1737 | 4. Then pass the created Parameter to the custom TypeHandler.SetValue method 1738 | 1739 | ![image](https://user-images.githubusercontent.com/12729184/101142987-4868a300-3651-11eb-99f4-b981dec8b97d.png) 1740 | 1741 | Finally, the C# code converted from IL 1742 | 1743 | ```C# 1744 | public static void TestMeThod(IDbCommand P_0, object P_1) 1745 | { 1746 | User user = (User)P_1; 1747 | IDataParameterCollection parameters = P_0.Parameters; 1748 | //... 1749 | IDbDataParameter dbDataParameter3 = P_0.CreateParameter(); 1750 | dbDataParameter3.ParameterName = "Logs"; 1751 | dbDataParameter3.Direction = ParameterDirection.Input; 1752 | SqlMapper.TypeHandlerCache>.SetValue(dbDataParameter3, ((object)user.Logs) ?? ((object)DBNull.Value)); 1753 | parameters.Add(dbDataParameter3); 1754 | //... 1755 | } 1756 | ``` 1757 | 1758 | It can be found that the generated Emit IL will get our implemented TypeHandler from TypeHandlerCache, and then `call the implemented SetValue method` to run the set logic, and TypeHandlerCache uses `generic type` to save different handlers in `Singleton mode` according to different generics. This has the following advantages : 1759 | 1760 | 1. Same handler can be obtained to avoid repeated creation of objects 1761 | 2. Because it is a generic type, reflection actions can be avoided when the handler is taken, and `efficiency can be improved` 1762 | 1763 | ![image](https://user-images.githubusercontent.com/12729184/101143328-b1501b00-3651-11eb-92aa-15cfde21e7c3.png) 1764 | ![image](https://user-images.githubusercontent.com/12729184/101143332-b319de80-3651-11eb-8291-90b6206bd82b.png) 1765 | ![image](https://user-images.githubusercontent.com/12729184/101143337-b4e3a200-3651-11eb-8d8a-c61251388351.png) 1766 | 1767 | ### Parse corresponds to the underlying principle 1768 | 1769 | The main logic is when the GenerateDeserializerFromMap method Emit establishes the dynamic Mapping method, if it is judged that the TypeHandler cache has data, the Parse method replaces the original Set attribute action. 1770 | 1771 | ![image](https://user-images.githubusercontent.com/12729184/101143441-d93f7e80-3651-11eb-8e00-070628abfe90.png) 1772 | 1773 | View the IL code generated by the dynamic Mapping method: 1774 | 1775 | ```C# 1776 | IL_0000: ldc.i4.0 1777 | IL_0001: stloc.0 1778 | IL_0002: newobj Void .ctor()/Demo.User 1779 | IL_0007: stloc.1 1780 | IL_0008: ldloc.1 1781 | IL_0009: dup 1782 | IL_000a: ldc.i4.0 1783 | IL_000b: stloc.0 1784 | IL_000c: ldarg.0 1785 | IL_000d: ldc.i4.0 1786 | IL_000e: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1787 | IL_0013: dup 1788 | IL_0014: stloc.2 1789 | IL_0015: dup 1790 | IL_0016: isinst System.DBNull 1791 | IL_001b: brtrue.s IL_0029 1792 | IL_001d: unbox.any System.String 1793 | IL_0022: callvirt Void set_Name(System.String)/Demo.User 1794 | IL_0027: br.s IL_002b 1795 | IL_0029: pop 1796 | IL_002a: pop 1797 | IL_002b: dup 1798 | IL_002c: ldc.i4.1 1799 | IL_002d: stloc.0 1800 | IL_002e: ldarg.0 1801 | IL_002f: ldc.i4.1 1802 | IL_0030: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1803 | IL_0035: dup 1804 | IL_0036: stloc.2 1805 | IL_0037: dup 1806 | IL_0038: isinst System.DBNull 1807 | IL_003d: brtrue.s IL_004b 1808 | IL_003f: unbox.any System.Int32 1809 | IL_0044: callvirt Void set_Age(Int32)/Demo.User 1810 | IL_0049: br.s IL_004d 1811 | IL_004b: pop 1812 | IL_004c: pop 1813 | IL_004d: dup 1814 | IL_004e: ldc.i4.2 1815 | IL_004f: stloc.0 1816 | IL_0050: ldarg.0 1817 | IL_0051: ldc.i4.2 1818 | IL_0052: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1819 | IL_0057: dup 1820 | IL_0058: stloc.2 1821 | IL_0059: dup 1822 | IL_005a: isinst System.DBNull 1823 | IL_005f: brtrue.s IL_006d 1824 | IL_0061: unbox.any System.Int32 1825 | IL_0066: callvirt Void set_Level(Int32)/Demo.User 1826 | IL_006b: br.s IL_006f 1827 | IL_006d: pop 1828 | IL_006e: pop 1829 | IL_006f: dup 1830 | IL_0070: ldc.i4.3 1831 | IL_0071: stloc.0 1832 | IL_0072: ldarg.0 1833 | IL_0073: ldc.i4.3 1834 | IL_0074: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1835 | IL_0079: dup 1836 | IL_007a: stloc.2 1837 | IL_007b: dup 1838 | IL_007c: isinst System.DBNull 1839 | IL_0081: brtrue.s IL_008f 1840 | IL_0083: call System.Collections.Generic.List`1[Demo.Log] Parse(System.Object)/Dapper.SqlMapper+TypeHandlerCache`1[System.Collections.Generic.List`1[Demo.Log]] 1841 | IL_0088: callvirt Void set_Logs(System.Collections.Generic.List`1[Demo.Log])/Demo.User 1842 | IL_008d: br.s IL_0091 1843 | IL_008f: pop 1844 | IL_0090: pop 1845 | IL_0091: stloc.1 1846 | IL_0092: leave IL_00a4 1847 | IL_0097: ldloc.0 1848 | IL_0098: ldarg.0 1849 | IL_0099: ldloc.2 1850 | IL_009a: call Void ThrowDataException(System.Exception, Int32, System.Data.IDataReader, System.Object)/Dapper.SqlMapper 1851 | IL_009f: leave IL_00a4 1852 | IL_00a4: ldloc.1 1853 | IL_00a5: ret 1854 | ``` 1855 | 1856 | Convert it into C# code to verify: 1857 | 1858 | ```C# 1859 | public static User TestMeThod(IDataReader P_0) 1860 | { 1861 | int index = 0; 1862 | User user = new User(); 1863 | object value = default(object); 1864 | try 1865 | { 1866 | User user2 = user; 1867 | index = 0; 1868 | object obj = value = P_0[0]; 1869 | //.. 1870 | index = 3; 1871 | object obj4 = value = P_0[3]; 1872 | if (!(obj4 is DBNull)) 1873 | { 1874 | user2.Logs = SqlMapper.TypeHandlerCache>.Parse(obj4); 1875 | } 1876 | user = user2; 1877 | return user; 1878 | } 1879 | catch (Exception ex) 1880 | { 1881 | SqlMapper.ThrowDataException(ex, index, P_0, value); 1882 | return user; 1883 | } 1884 | } 1885 | ``` 1886 | 1887 | ## 19. Detailed processing of CommandBehavior 1888 | 1889 | This article will take readers to understand how Dapper uses CommandBehavior to optimize query efficiency, and how to choose the correct Behavior at a specific time. 1890 | 1891 | I have compiled the Behavior table corresponding to each method here: 1892 | 1893 | | method | Behavior | 1894 | | -------------------- | ------------------------------------------------------------ | 1895 | | Query | CommandBehavior.SequentialAccess & CommandBehavior.SingleResult | 1896 | | QueryFirst | CommandBehavior.SequentialAccess & CommandBehavior.SingleResult & CommandBehavior.SingleRow | 1897 | | QueryFirstOrDefault | CommandBehavior.SequentialAccess & CommandBehavior.SingleResult & CommandBehavior.SingleRow | 1898 | | QuerySingle | CommandBehavior.SingleResult & CommandBehavior.SequentialAccess | 1899 | | QuerySingleOrDefault | CommandBehavior.SingleResult & CommandBehavior.SequentialAccess | 1900 | | QueryMultiple | CommandBehavior.SequentialAccess | 1901 | 1902 | --- 1903 | 1904 | ### SequentialAccess, SingleResult optimization logic 1905 | 1906 | First, you can see that each method uses `CommandBehavior.SequentialAccess`. The main function of this tag is to make the `DataReader read rows and columns sequentially without buffering`. `After reading a column, it will be deleted from memory`. It has the following advantages: 1907 | 1908 | 1. Resources can be read in order to `avoid binary large resources from being read into memory at one time`, especially Blob or Clob will cooperate with GetBytes or GetChars methods to limit the buffer size, Microsoft officials also have special attention: 1909 | 1910 | ![image](https://user-images.githubusercontent.com/12729184/101143904-8619fb80-3652-11eb-9354-cfa40b7f5879.png) 1911 | 1912 | 2. Actual environment testing show it can `speed up query efficiency`. But it is `not` the default behavior of DataReader, the system `default is CommandBehavior.Default` 1913 | 1914 | ![image](https://user-images.githubusercontent.com/12729184/101144007-ae095f00-3652-11eb-887d-8ef58bb0d2c6.png) 1915 | 1916 | CommandBehavior.Default has the below behaviors: 1917 | 1918 | 1. Can return `multiple` result sets 1919 | 2. Read row data to memory at once 1920 | 1921 | These two features are much different from the production environment. After all, most of the time,` only a set of result sets are needed with limited memory`, so in addition to SequentialAccess, Dapper also uses `CommandBehavior.SingleResult` in most methods, so that only one set of results is required. To avoid wasting resources. 1922 | 1923 | There is also a detailed processing of this paragraph. Looking at the source code, you can find that in addition to marking SingleResult, Dapper also specially adds code at the end `while(reader.NextResult()){}` instead of directly Return (such as the picture) 1924 | 1925 | ![image](https://user-images.githubusercontent.com/12729184/101144300-0e989c00-3653-11eb-8b1a-af7b0ab0797f.png) 1926 | 1927 | Earlier, I specifically posted an Issue ([Link #1210](https://github.com/StackExchange/Dapper/issues/1210)) to ask author, here is the answer: `mainly to avoid ignoring errors, such as when the DataReader is closed early`. 1928 | 1929 | ### QueryFirst with SingleRow 1930 | 1931 | Sometimes we will encounter a situation where `select top 1` knows that only one row of data will be read. At this time, `QueryFirst` can be used. It uses `CommandBehavior.SingleRow` to avoid wasting resources and only read one row of data. 1932 | 1933 | In addition, it can be found that in addition to `while (reader.NextResult()){}`, Dapper also has `while (reader.Read()) {}`, which is also to avoid ignoring errors. This is something that some companies’ self-make ORMs will ignore. 1934 | 1935 | ![image](https://user-images.githubusercontent.com/12729184/101144554-72bb6000-3653-11eb-9034-2fdfec1687ec.png) 1936 | 1937 | ### Differences with QuerySingle 1938 | 1939 | The difference between the two is that QuerySingle does not use CommandBehavior.SingleRow. As for why it is not used, it is because multiple rows of data are needed to `determine whether the conditions are not met and an Exception is thrown to inform the user`. 1940 | 1941 | There is a particularly fun trick to learn in Dapper. The error handling directly uses the Exception corresponding to LINQ. For example: more than one line of data is wrong, use `new int[2].Single()`, so you don't need to maintain the Exception class separately, and you can also have more `i18N` language message. 1942 | 1943 | ![image](https://user-images.githubusercontent.com/12729184/101144744-c168fa00-3653-11eb-90b9-798071e69cac.png) 1944 | 1945 | ![image](https://user-images.githubusercontent.com/12729184/101144760-c62dae00-3653-11eb-8b73-bded7da67e9f.png) 1946 | 1947 | ## 20. The underlying logic of Parameter parameterization 1948 | 1949 | One key function of Dapper: "Parameterization" 1950 | 1951 | Main logic: GetCacheInfo checks whether there is a dynamic method in the cache > If there is no cache, use the CreateParamInfoGenerator method Emit IL to create the AddParameter dynamic method > Save it in the cache after creation 1952 | 1953 | Next, we will focus on the underlying logic and "exquisite detail processing" in the CreateParamInfoGenerator method, using the result reverse code method, `ignoring the "unused fields"` and not generating the corresponding IL code to avoid resource waste. This is also the reason why the previous caching algorithm has to check different SQL strings. 1954 | 1955 | The following are the key parts of the source code I picked: 1956 | 1957 | ```C# 1958 | internal static Action CreateParamInfoGenerator(Identity identity, bool checkForDuplicates, bool removeUnused, IList literals) 1959 | { 1960 | //... 1961 | if (filterParams) 1962 | { 1963 | props = FilterParameters(props, identity.sql); 1964 | } 1965 | 1966 | var callOpCode = isStruct ? OpCodes.Call : OpCodes.Callvirt; 1967 | foreach (var prop in props) 1968 | { 1969 | //Emit IL action 1970 | } 1971 | //... 1972 | } 1973 | 1974 | 1975 | private static IEnumerable FilterParameters(IEnumerable parameters, string sql) 1976 | { 1977 | var list = new List(16); 1978 | foreach (var p in parameters) 1979 | { 1980 | if (Regex.IsMatch(sql, @"[?@:]" + p.Name + @"([^\p{L}\p{N}_]+|$)", RegexOptions.IgnoreCase | RegexOptions.Multiline | RegexOptions.CultureInvariant)) 1981 | list.Add(p); 1982 | } 1983 | return list; 1984 | } 1985 | ``` 1986 | 1987 | Then check IL to verify, the query code is as follows 1988 | 1989 | ```C# 1990 | var result = connection.Query("select @Name name ", new { Name = "Wei", Age = 26}).First(); 1991 | ``` 1992 | 1993 | The IL code of the CreateParamInfoGenerator AddParameter dynamic method is as below: 1994 | 1995 | ```C# 1996 | IL_0000: ldarg.1 1997 | IL_0001: castclass <>f__AnonymousType1`2[System.String,System.Int32] 1998 | IL_0006: stloc.0 1999 | IL_0007: ldarg.0 2000 | IL_0008: callvirt System.Data.IDataParameterCollection get_Parameters()/System.Data.IDbCommand 2001 | IL_000d: dup 2002 | IL_000e: ldarg.0 2003 | IL_000f: callvirt System.Data.IDbDataParameter CreateParameter()/System.Data.IDbCommand 2004 | IL_0014: dup 2005 | IL_0015: ldstr "Name" 2006 | IL_001a: callvirt Void set_ParameterName(System.String)/System.Data.IDataParameter 2007 | IL_001f: dup 2008 | IL_0020: ldc.i4.s 16 2009 | IL_0022: callvirt Void set_DbType(System.Data.DbType)/System.Data.IDataParameter 2010 | IL_0027: dup 2011 | IL_0028: ldc.i4.1 2012 | IL_0029: callvirt Void set_Direction(System.Data.ParameterDirection)/System.Data.IDataParameter 2013 | IL_002e: dup 2014 | IL_002f: ldloc.0 2015 | IL_0030: callvirt System.String get_Name()/<>f__AnonymousType1`2[System.String,System.Int32] 2016 | IL_0035: dup 2017 | IL_0036: brtrue.s IL_0042 2018 | IL_0038: pop 2019 | IL_0039: ldsfld System.DBNull Value/System.DBNull 2020 | IL_003e: ldc.i4.0 2021 | IL_003f: stloc.1 2022 | IL_0040: br.s IL_005a 2023 | IL_0042: dup 2024 | IL_0043: callvirt Int32 get_Length()/System.String 2025 | IL_0048: ldc.i4 4000 2026 | IL_004d: cgt 2027 | IL_004f: brtrue.s IL_0058 2028 | IL_0051: ldc.i4 4000 2029 | IL_0056: br.s IL_0059 2030 | IL_0058: ldc.i4.m1 2031 | IL_0059: stloc.1 2032 | IL_005a: callvirt Void set_Value(System.Object)/System.Data.IDataParameter 2033 | IL_005f: ldloc.1 2034 | IL_0060: brfalse.s IL_0069 2035 | IL_0062: dup 2036 | IL_0063: ldloc.1 2037 | IL_0064: callvirt Void set_Size(Int32)/System.Data.IDbDataParameter 2038 | IL_0069: callvirt Int32 Add(System.Object)/System.Collections.IList 2039 | IL_006e: pop 2040 | IL_006f: pop 2041 | IL_0070: ret 2042 | ``` 2043 | 2044 | IL converted to C# code: 2045 | 2046 | ```C# 2047 | public class TestType 2048 | { 2049 | public static void TestMeThod(IDataReader P_0, object P_1) 2050 | { 2051 | var anon = (<>f__AnonymousType1)P_1; 2052 | IDataParameterCollection parameters = ((IDbCommand)P_0).Parameters; 2053 | IDbDataParameter dbDataParameter = ((IDbCommand)P_0).CreateParameter(); 2054 | dbDataParameter.ParameterName = "Name"; 2055 | dbDataParameter.DbType = DbType.String; 2056 | dbDataParameter.Direction = ParameterDirection.Input; 2057 | object obj = anon.Name; 2058 | int num; 2059 | if (obj == null) 2060 | { 2061 | obj = DBNull.Value; 2062 | num = 0; 2063 | } 2064 | else 2065 | { 2066 | num = ((((string)obj).Length > 4000) ? (-1) : 4000); 2067 | } 2068 | dbDataParameter.Value = obj; 2069 | if (num != 0) 2070 | { 2071 | dbDataParameter.Size = num; 2072 | } 2073 | parameters.Add(dbDataParameter); 2074 | } 2075 | } 2076 | ``` 2077 | 2078 | It can be found that although the Age parameter is passed, the SQL string is not used, and Dapper will not generate the SetParameter action IL for this field. This detail processing really needs to give Dapper a thumbs up! 2079 | 2080 | ## 21. The underlying logic of IN multi-set parameterization 2081 | 2082 | Why ADO.NET does not support IN parameterization, but Dapper does? 2083 | 2084 | 1. Check whether the attribute of the parameter is a subclass of IEnumerable 2085 | 2086 | 2. If yes, use the parameter name + regular format to find the parameter string in SQL `(regular format: ([?@:]Parameter name)(?!\w)(\s+(?i)unknown(?- i))?)` 2087 | 2088 | 3. Replace the found string with `()` + multiple `attribute names + serial number` 2089 | 2090 | 4. CreateParameter> SetValue in order of serial number 2091 | 2092 | Key Code part 2093 | 2094 | ![image](https://user-images.githubusercontent.com/12729184/101145551-ea3dbf00-3654-11eb-9590-2ba3418b004a.png) 2095 | 2096 | The following uses sys.objects to check SQL Server tables and views as an example of tracking: 2097 | 2098 | ```C# 2099 | var result = cn.Query(@"select * from sys.objects where type_desc In @type_descs", new { type_descs = new[] { "USER_TABLE", "VIEW" } }); 2100 | ``` 2101 | 2102 | Dapper will change the SQL string to the following sql to execute 2103 | 2104 | ```C# 2105 | select * from sys.objects where type_desc In (@type_descs1,@type_descs2) 2106 | -- @type_descs1 = nvarchar(4000) - 'USER_TABLE' 2107 | -- @type_descs2 = nvarchar(4000) - 'VIEW' 2108 | ``` 2109 | 2110 | Looking at Emit IL, you can find that it is very different from the previous parameterized IL, which is very clean. 2111 | 2112 | ```C# 2113 | IL_0000: ldarg.1 2114 | IL_0001: castclass <>f__AnonymousType0`1[System.String[]] 2115 | IL_0006: stloc.0 2116 | IL_0007: ldarg.0 2117 | IL_0008: callvirt System.Data.IDataParameterCollection get_Parameters()/System.Data.IDbCommand 2118 | IL_000d: ldarg.0 2119 | IL_000e: ldstr "type_descs" 2120 | IL_0013: ldloc.0 2121 | IL_0014: callvirt System.String[] get_type_descs()/<>f__AnonymousType0`1[System.String[]] 2122 | IL_0019: call Void PackListParameters(System.Data.IDbCommand, System.String, System.Object)/Dapper.SqlMapper 2123 | IL_001e: pop 2124 | IL_001f: ret 2125 | ``` 2126 | 2127 | Turning to C# code, you will be surprised to find: `This code does not need to use Emit IL at all. It is simply unnecessary.` 2128 | 2129 | ```C# 2130 | public static void TestMeThod(IDbCommand P_0, object P_1) 2131 | { 2132 | var anon = (<>f__AnonymousType0)P_1; 2133 | IDataParameterCollection parameter = P_0.Parameters; 2134 | SqlMapper.PackListParameters(P_0, "type_descs", anon.type_descs); 2135 | } 2136 | ``` 2137 | 2138 | That's right, it is unnecessary, even `IDataParameterCollection parameter = P_0.Parameters`; this code will not be used at all. 2139 | 2140 | There is a reason for Dapper, because it can be used with `non-collective parameters`, such as the previous example and the data logic to find the name of the order. 2141 | 2142 | ```C# 2143 | var result = cn.Query(@"select * from sys.objects where type_desc In @type_descs and name like @name" 2144 | , new { type_descs = new[] { "USER_TABLE", "VIEW" }, @name = "order%" }); 2145 | ``` 2146 | 2147 | The corresponding generated IL conversion C# code will be the following code, which can be used together: 2148 | 2149 | ```C# 2150 | public static void TestMeThod(IDbCommand P_0, object P_1) 2151 | { 2152 | <>f__AnonymousType0 val = P_1; 2153 | IDataParameterCollection parameters = P_0.Parameters; 2154 | SqlMapper.PackListParameters(P_0, "type_descs", val.get_type_descs()); 2155 | IDbDataParameter dbDataParameter = P_0.CreateParameter(); 2156 | dbDataParameter.ParameterName = "name"; 2157 | dbDataParameter.DbType = DbType.String; 2158 | dbDataParameter.Direction = ParameterDirection.Input; 2159 | object obj = val.get_name(); 2160 | int num; 2161 | if (obj == null) 2162 | { 2163 | obj = DBNull.Value; 2164 | num = 0; 2165 | } 2166 | else 2167 | { 2168 | num = ((((string)obj).Length > 4000) ? (-1) : 4000); 2169 | } 2170 | dbDataParameter.Value = obj; 2171 | if (num != 0) 2172 | { 2173 | dbDataParameter.Size = num; 2174 | } 2175 | parameters.Add(dbDataParameter); 2176 | } 2177 | ``` 2178 | 2179 | In addition, why does Emit IL directly call the tool method `PackListParameters` on Dapper? Because the number of IN parameters is not fixed, the method `cannot be dynamically generated from the fixed result`. 2180 | 2181 | The main logic contained in this method: 2182 | 2183 | 1. Determine the type of the set parameter (if it is a string, the default size is 4000) 2184 | 2. Regular role of SQL parameters are replaced with serial number parameter strings 2185 | 3. Creation of DbCommand Paramter 2186 | 2187 | ![image](https://user-images.githubusercontent.com/12729184/101146110-97183c00-3655-11eb-90b8-0db8dec91d77.png) 2188 | 2189 | The replacement logic of the SQL parameter string is also written here, such as the picture 2190 | 2191 | ![image](https://user-images.githubusercontent.com/12729184/101146159-a303fe00-3655-11eb-92c5-99968b387881.png) 2192 | 2193 | ## 22. DynamicParameter underlying logic and custom implementation 2194 | 2195 | For example: 2196 | 2197 | ```C# 2198 | using (var cn = Connection) 2199 | { 2200 | var paramter = new { Name = "John", Age = 25 }; 2201 | var result = cn.Query("select @Name Name,@Age Age", paramter).First(); 2202 | } 2203 | ``` 2204 | 2205 | We already know that String type Dapper will automatically convert to database `Nvarchar` and a parameter with a `length of 4000`. The SQL actually executed by the database is as below: 2206 | 2207 | ```sql 2208 | exec sp_executesql N'select @Name Name,@Age Age',N'@Name nvarchar(4000),@Age int',@Name=N'John',@Age=25 2209 | ``` 2210 | 2211 | This is an intimate design that is convenient for rapid development, but if you encounter a situation where the field is of `varchar` type, it may cause the `index to fail` due to the `implicit transformation`, resulting in low query efficiency. 2212 | 2213 | At this time, the solution can use Dapper DynamicParamter to specify the database type and size to achieve the purpose of optimizing performance 2214 | 2215 | ```C# 2216 | using (var cn = Connection) 2217 | { 2218 | var paramters = new DynamicParameters(); 2219 | paramters.Add("Name","John",DbType.AnsiString,size:4); 2220 | paramters.Add("Age",25,DbType.Int32); 2221 | var result = cn.Query("select @Name Name,@Age Age", paramters).First(); 2222 | } 2223 | ``` 2224 | 2225 | Then go to the source to see how to implement it. First, pay attention to the GetCacheInfo method. You can see that DynamicParameters create a dynamic method. The code is very simple, just call the AddParameters method. 2226 | 2227 | ```C# 2228 | Action reader; 2229 | if (exampleParameters is IDynamicParameters) 2230 | { 2231 | reader = (cmd, obj) => ((IDynamicParameters)obj).AddParameters(cmd, identity); 2232 | } 2233 | ``` 2234 | 2235 | The reason why the code can be so simple is that Dapper uses an `"interface-dependent"` design here to increase the flexibility of the program and allow users to customize the implementation logic they want. This point will be explained below. First, let's look at the implementation logic of the `AddParameters` method in Dapper's default implementation class `DynamicParameters`. 2236 | 2237 | ```C# 2238 | public class DynamicParameters : SqlMapper.IDynamicParameters, SqlMapper.IParameterLookup, SqlMapper.IParameterCallbacks 2239 | { 2240 | protected void AddParameters(IDbCommand command, SqlMapper.Identity identity) 2241 | { 2242 | var literals = SqlMapper.GetLiteralTokens(identity.sql); 2243 | 2244 | foreach (var param in parameters.Values) 2245 | { 2246 | if (param.CameFromTemplate) continue; 2247 | 2248 | var dbType = param.DbType; 2249 | var val = param.Value; 2250 | string name = Clean(param.Name); 2251 | var isCustomQueryParameter = val is SqlMapper.ICustomQueryParameter; 2252 | 2253 | SqlMapper.ITypeHandler handler = null; 2254 | if (dbType == null && val != null && !isCustomQueryParameter) 2255 | { 2256 | #pragma warning disable 618 2257 | dbType = SqlMapper.LookupDbType(val.GetType(), name, true, out handler); 2258 | #pragma warning disable 618 2259 | } 2260 | if (isCustomQueryParameter) 2261 | { 2262 | ((SqlMapper.ICustomQueryParameter)val).AddParameter(command, name); 2263 | } 2264 | else if (dbType == EnumerableMultiParameter) 2265 | { 2266 | #pragma warning disable 612, 618 2267 | SqlMapper.PackListParameters(command, name, val); 2268 | #pragma warning restore 612, 618 2269 | } 2270 | else 2271 | { 2272 | bool add = !command.Parameters.Contains(name); 2273 | IDbDataParameter p; 2274 | if (add) 2275 | { 2276 | p = command.CreateParameter(); 2277 | p.ParameterName = name; 2278 | } 2279 | else 2280 | { 2281 | p = (IDbDataParameter)command.Parameters[name]; 2282 | } 2283 | 2284 | p.Direction = param.ParameterDirection; 2285 | if (handler == null) 2286 | { 2287 | #pragma warning disable 0618 2288 | p.Value = SqlMapper.SanitizeParameterValue(val); 2289 | #pragma warning restore 0618 2290 | if (dbType != null && p.DbType != dbType) 2291 | { 2292 | p.DbType = dbType.Value; 2293 | } 2294 | var s = val as string; 2295 | if (s?.Length <= DbString.DefaultLength) 2296 | { 2297 | p.Size = DbString.DefaultLength; 2298 | } 2299 | if (param.Size != null) p.Size = param.Size.Value; 2300 | if (param.Precision != null) p.Precision = param.Precision.Value; 2301 | if (param.Scale != null) p.Scale = param.Scale.Value; 2302 | } 2303 | else 2304 | { 2305 | if (dbType != null) p.DbType = dbType.Value; 2306 | if (param.Size != null) p.Size = param.Size.Value; 2307 | if (param.Precision != null) p.Precision = param.Precision.Value; 2308 | if (param.Scale != null) p.Scale = param.Scale.Value; 2309 | handler.SetValue(p, val ?? DBNull.Value); 2310 | } 2311 | 2312 | if (add) 2313 | { 2314 | command.Parameters.Add(p); 2315 | } 2316 | param.AttachedParam = p; 2317 | } 2318 | } 2319 | 2320 | // note: most non-priveleged implementations would use: this.ReplaceLiterals(command); 2321 | if (literals.Count != 0) SqlMapper.ReplaceLiterals(this, command, literals); 2322 | } 2323 | } 2324 | ``` 2325 | 2326 | It can be found that Dapper has made many conditions and actions in AddParameters for convenience and compatibility with other functions, such as Literal Replacement and EnumerableMultiParameter functions, so the amount of code will be more than the previous version of ADO.NET, so `the efficiency will be slower`. 2327 | 2328 | If you have demanding requirements for efficiency, you can implement the logic yourself, because this section of Dapper is specially designed to `"depend on the interface"`, and you only need to `implement the IDynamicParameters interface`. 2329 | 2330 | The following is a demo I made, you can use ADO.NET SqlParameter to establish parameters to cooperate with Dapper 2331 | 2332 | ```C# 2333 | public class CustomPraameters : SqlMapper.IDynamicParameters 2334 | { 2335 | private SqlParameter[] parameters; 2336 | public void Add(params SqlParameter[] mParameters) 2337 | { 2338 | parameters = mParameters; 2339 | } 2340 | 2341 | void SqlMapper.IDynamicParameters.AddParameters(IDbCommand command, SqlMapper.Identity identity) 2342 | { 2343 | if (parameters != null && parameters.Length > 0) 2344 | foreach (var p in parameters) 2345 | command.Parameters.Add(p); 2346 | } 2347 | } 2348 | ``` 2349 | 2350 | ![image](https://user-images.githubusercontent.com/12729184/101188682-fe9fad00-3690-11eb-9553-37ca92e7b10e.png) 2351 | 2352 | 2353 | 2354 | ## 23. The underlying logic of single and multiple Execute 2355 | 2356 | After the Query, Mapping, and Parameters are explained, we will then explain that use the Execute method in adding, deleting, and modifying by Dapper. Execute Dapper is divided into `single execute and `multiple execute`. 2357 | 2358 | ### Single Execute 2359 | 2360 | In terms of a single execution, the logic of Dapper is the encapsulation of ADO.NET's ExecuteNonQuery. The purpose of encapsulation is to be used with `Dapper's Parameter and caching functions`. The code logic is concise and clear. There is no more explanation here, such as the picture 2361 | 2362 | ![image](https://user-images.githubusercontent.com/12729184/101189043-7bcb2200-3691-11eb-954d-7b0087ad03a3.png) 2363 | 2364 | ### "Multiple" Execute 2365 | 2366 | This is a characteristic feature of Dapper, which simplifies the operations between the collection operations Execute and simplifies the code. Only: `connection.Execute("sql",collection parameters);`. 2367 | 2368 | Why it is so convenient, the following is the underlying logic: 2369 | 2370 | 1. Confirm whether it is a collection parameter 2371 | 2372 | ![image](https://user-images.githubusercontent.com/12729184/101189227-b8971900-3691-11eb-9f47-34a374eb4694.png) 2373 | 2374 | 2. Create a `common DbCommand` to provide foreach iterative call to avoid repeated create and waste of resources 2375 | 2376 | ![image](https://user-images.githubusercontent.com/12729184/101189341-debcb900-3691-11eb-8c70-efb0eab94782.png) 2377 | 2378 | 3. If it is a set of parameters, create an Emit IL dynamic method and put it in the cache for use 2379 | 2380 | ![image](https://user-images.githubusercontent.com/12729184/101189402-f8f69700-3691-11eb-949a-c6bf22ac6846.png) 2381 | 2382 | 4. The dynamic method logic is `CreateParameter> Assign Parameter> Use Parameters.Add to add a new parameter`. The following is the C# code converted by Emit IL: 2383 | 2384 | ```C# 2385 | public static void ParamReader(IDbCommand P_0, object P_1) 2386 | { 2387 | var anon = (<>f__AnonymousType0)P_1; 2388 | IDataParameterCollection parameters = P_0.Parameters; 2389 | IDbDataParameter dbDataParameter = P_0.CreateParameter(); 2390 | dbDataParameter.ParameterName = "V"; 2391 | dbDataParameter.DbType = DbType.Int32; 2392 | dbDataParameter.Direction = ParameterDirection.Input; 2393 | dbDataParameter.Value = anon.V; 2394 | parameters.Add(dbDataParameter); 2395 | } 2396 | ``` 2397 | 2398 | 5. `foreach` the set of parameters> except for the first time, clear the DbCommand parameters in each iteration> re-call the `same` dynamic method to add parameters> reqeust SQL query 2399 | 2400 | The implementation method is simple and clear, and the details consider sharing resources to avoid waste `(eg sharing the same DbCommand, Func)`, but in the case of a large number of execution pursuit efficiency requirements, you need to pay special attention to this method `every time you run to send a request to the database`, the efficiency will be the network transmission is slow, so this function is called `"multiple execution" instead of "batch execution"`. 2401 | 2402 | For example, simple Execute inserts ten pieces of data, and you can see that the system has received 10 Reqeust when viewing SQL Profiler: 2403 | 2404 | ```C# 2405 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=Northwind;")) 2406 | { 2407 | cn.Open(); 2408 | using (var tx = cn.BeginTransaction()) 2409 | { 2410 | cn.Execute("create table #T (V int);", transaction: tx); 2411 | cn.Execute("insert into #T (V) values (@V)", Enumerable.Range(1, 10).Select(val => new { V = val }).ToArray() , transaction:tx); 2412 | 2413 | var result = cn.Query("select * from #T", transaction: tx); 2414 | Console.WriteLine(result); 2415 | } 2416 | } 2417 | ``` 2418 | 2419 | ![image](https://user-images.githubusercontent.com/12729184/101189851-8afe9f80-3692-11eb-86dd-41064b259594.png) 2420 | 2421 | ## 24. ExecuteScalar 2422 | 2423 | ExecuteScalar is an often forgotten function because `it can only read the first set of results, the first row, and the first data`. However, it can still come in handy under specific needs, like `"Check Existence"`. 2424 | 2425 | First, how does Entity Framwork efficiently check whether data exists? 2426 | 2427 | If the reader with EF experience will answer to use `Any instead of Count()> 1`. Using the Count system will help convert SQL to: 2428 | 2429 | ```C# 2430 | SELECT COUNT(*) AS [value] FROM [Table] AS [t0] 2431 | ``` 2432 | 2433 | SQL Count is a summary function that will `iterate the qualified data rows to determine whether the data in each row is null` and return the number of rows. 2434 | 2435 | The Any syntax conversion SQL uses `EXISTS`. It only cares `whether there is data or not`. It means that there is `no need to check each column`, so the efficiency is fast. 2436 | 2437 | ```C# 2438 | SELECT 2439 | (CASE 2440 | WHEN EXISTS( 2441 | SELECT NULL AS [EMPTY] 2442 | FROM [Table] AS [t0] 2443 | ) THEN 1 2444 | ELSE 0 2445 | END) AS [value] 2446 | ``` 2447 | 2448 | ### How does Dapper achieve the same effect? 2449 | 2450 | SQL Server can use the SQL format `select top 1 1 from [table] where conditions` with the ExecuteScalar method, and then make an extension method, as follow: 2451 | 2452 | ```C# 2453 | public static class DemoExtension 2454 | { 2455 | public static bool Any(this IDbConnection cn,string sql,object paramter = null) 2456 | { 2457 | return cn.ExecuteScalar(sql,paramter); 2458 | } 2459 | } 2460 | ``` 2461 | 2462 | ![image](https://user-images.githubusercontent.com/12729184/101190490-65be6100-3693-11eb-94c2-c41ad718feec.png) 2463 | 2464 | The reason for this simple use is that Dapper ExecuteScalar will call ExecuteScalarImpl and its underlying Parse logic 2465 | 2466 | ```C# 2467 | private static T ExecuteScalarImpl(IDbConnection cnn, ref CommandDefinition command) 2468 | { 2469 | //.. 2470 | object result; 2471 | //.. 2472 | result = cmd.ExecuteScalar(); 2473 | //.. 2474 | return Parse(result); 2475 | } 2476 | 2477 | private static T Parse(object value) 2478 | { 2479 | if (value == null || value is DBNull) return default(T); 2480 | if (value is T) return (T)value; 2481 | var type = typeof(T); 2482 | //.. 2483 | return (T)Convert.ChangeType(value, type, CultureInfo.InvariantCulture); 2484 | } 2485 | ``` 2486 | 2487 | Use Convert.ChangeType to convert to bool: `"0=false, non-0=true"` logic, so that the system can simply convert to bool value. 2488 | 2489 | Note: Don't replace by QueryFirstOrDefault, because it requires additional Null check in SQL, otherwise "NullReferenceException" will show. 2490 | 2491 | The reason is that the two Parse implementations are different, and the QueryFirstOrDefault check the result to be null. 2492 | 2493 | ![20191003043941.png](https://camo.githubusercontent.com/80646646c310e49436934f8e71066c5a7d8727df3643b3374de01ab09fcd5329/68747470733a2f2f692e6c6f6c692e6e65742f323031392f31302f30332f546a4870647778444d5775566141352e706e67) 2494 | 2495 | The Parce implementation of ExecuteScalar has more check to `use the default value when it is empty` 2496 | 2497 | ![image](https://user-images.githubusercontent.com/12729184/101190897-fc8b1d80-3693-11eb-84a7-30c9d8dffef7.png) 2498 | 2499 | ## 25. Summary 2500 | 2501 | The Dapper series end here, and the important underlying logics are almost finished. This series took the me 25 consecutive days. In addition to helping readers, the biggest gain is that I understand the underlying logic of Dapper better during this period and learn Dapper details and processing. 2502 | 2503 | In addition, I would like to mention that Marc Gravell, one of the authors of Dapper, is really very enthusiastic. During the writing of the article, there are a few conceptual questions. If you ask an issue, he will reply enthusiastically and in detail. And he also found that he has high requirements for the quality of the code.For example: I asked a question on SO, and he left a message below: `"He is actually not satisfied with the current Dapper IL architecture, and even feels it rough, and wants to use protobuf-net technology to rewrite"`(Respect!) 2504 | 2505 | ![image](https://user-images.githubusercontent.com/12729184/101191369-8e932600-3694-11eb-9498-be0888f05efa.png) 2506 | 2507 | Finally, I would like to say: the original intention of writing this is to hope that this series can help readers 2508 | 2509 | 1. Understand the underlying logic, know why, avoid writing monsters that eat efficiency, and take full advantage of Dapper to develop projects 2510 | 2. You can easily face Dapper interviews, and answer deeper concepts than ordinary engineers. 2511 | 3. From the simplest Reflection to the commonly used Expression to the most detailed Emit builds the Mapping method from scratch, taking readers to gradually understand the underlying strong type Mapping logic of Dapper 2512 | 4. Understand the important concept of dynamic creation method `"Code Converted From Result"`. 2513 | 5. With basic IL capabilities, you can use IL to reverse C# code to understand the underlying Emit logic of other projects 2514 | 6. Understand that Dapper `cannot use error strings to concat SQL` because of the algorithm logic of the cache 2515 | 2516 | Thanks :) 2517 | -------------------------------------------------------------------------------- /zh_tw.md: -------------------------------------------------------------------------------- 1 | 繁體中文版本連接 : [深入Dapper.NET源碼](zh_tw.md) 2 | 简体中文版本连接 : [深入Dapper.NET源码 (文长) - 暐翰 - 博客园](https://www.cnblogs.com/ITWeiHan/p/11614704.html) 3 | 4 | --- 5 | 6 | # 深入Dapper.NET源碼 7 | 8 | ## 目錄 9 | 1. [前言、目錄、安裝環境 ](#page1) 10 | 2. [Dynamic Query 原理 Part1 ](#page2) 11 | 3. [Dynamic Query 原理 Part2 ](#page3) 12 | 4. [ Strongly Typed Mapping 原理 Part1 : ADO.NET對比Dapper ](#page4) 13 | 5. [ Strongly Typed Mapping 原理 Part2 : Reflection版本 ](#page5) 14 | 6. [Strongly Typed Mapping 原理 Part3 : 動態建立方法重要概念「結果反推程式碼」優化效率 ](#page6) 15 | 7. [Strongly Typed Mapping 原理 Part4 : Expression版本 ](#page7) 16 | 8. [ Strongly Typed Mapping 原理 Part5 : Emit IL反建立C#代碼 ](#page8) 17 | 9. [Strongly Typed Mapping 原理 Part6 : Emit版本 ](#page9) 18 | 10. [Dapper 效率快關鍵之一 : Cache 緩存原理 ](#page10) 19 | 11. [錯誤SQL字串拼接方式,會導致效率慢、內存洩漏](#page11) 20 | 12. [Dapper SQL正確字串拼接方式 : Literal Replacement ](#page12) 21 | 13. [Query Multi Mapping 使用方式 ](#page13) 22 | 14. [Query Multi Mapping 底層原理 ](#page14) 23 | 15. [QueryMultiple 底層原理 ](#page15) 24 | 16. [TypeHandler 自訂Mapping邏輯使用、底層邏輯 ](#page16) 25 | 17. [ CommandBehavior的細節處理 ](#page17) 26 | 18. [Parameter 參數化底層原理 ](#page18) 27 | 19. [ IN 多集合參數化底層原理 ](#page19) 28 | 20. [DynamicParameter 底層原理、自訂實作 ](#page20) 29 | 21. [ 單次、多次 Execute 底層原理 ](#page21) 30 | 22. [ ExecuteScalar應用](#page22) 31 | 23. [總結](#page23) 32 | 33 | --- 34 | 35 | 36 | 　 37 | 　 38 | ## 1.前言、目錄、安裝環境 39 | 40 | 41 | 經過業界前輩、StackOverflow多年推廣,「Dapper搭配Entity Framework」成為一種功能強大的組合,它滿足`「安全、方便、高效、好維護」`需求。 42 | 43 | 但目前中文網路文章,雖然有很多關於Dapper的文章但都停留在如何使用,沒人系統性解說底層原理。所以有了此篇「深入Dapper源碼」想帶大家進入Dapper底層,了解Dapper的精美細節設計、高效原理,並`學起來`實際應用在工作當中。 44 | 45 | --- 46 | 47 | #### 建立Dapper Debug環境 48 | 49 | 1. 到[Dapper Github 首頁](https://github.com/StackExchange/Dapper) Clone最新版本到自己本機端 50 | 2. 建立.NET Core Console專案 51 | ![20191003173131.png](https://i.loli.net/2019/10/03/z3ohdWlMpUscwbG.png) 52 | 3. 需要安裝NuGet SqlClient套件、添加Dapper Project Reference 53 | ![20191003173438.png](https://i.loli.net/2019/10/03/X6xYvkwandAMiRC.png) 54 | 4. 下中斷點運行就可以Runtime查看邏輯 55 | ![20191003215021.png](https://i.loli.net/2019/10/03/1yaCfgGsUmY7nX5.png) 56 | 57 | #### 個人環境 58 | - 數據庫 : MSSQLLocalDB 59 | - Visaul Studio版本 : 2019 60 | - LINQ Pad 5 版本 61 | - Dapper版本 : V2.0.30 62 | - 反編譯 : ILSpy 63 | 64 | 65 | 　 66 | 　 67 | ## 2.Dynamic Query 原理 Part1 68 | 69 | 70 | 71 | 在前期開發階段因為表格結構還在`調整階段`,或是不值得額外宣告類別輕量需求,使用Dapper dynamic Query可以節省下來回修改class屬性的時間。當表格穩定下來後使用POCO生成器快速生成Class轉成`強型別`維護。 72 | 73 | #### 為何Dapper可以如此方便,支援dynamic? 74 | 75 | 追溯`Query`方法源碼可以發現兩個重點 76 | 1. 實體類別其實是`DapperRow`再隱性轉型為dynamic。 77 | ![20191003180501.png](https://i.loli.net/2019/10/03/wmCaByiPftj7V6W.png) 78 | 2. DapperRow繼承`IDynamicMetaObjectProvider`並且實作對應方法。 79 | 80 | ![20191003044133.png](https://i.loli.net/2019/10/03/Bt71kJc3dnOforN.png) 81 | 82 | 此段邏輯我這邊做一個簡化版本的Dapper dynamic Query讓讀者了解轉換邏輯 : 83 | 84 | 1. 建立`dynamic`類別變量,實體類別是`ExpandoObject` 85 | 2. 因為有繼承關係可以轉型為`IDictionary` 86 | 3. 使用DataReader使用GetName取得欄位名稱,藉由欄位index取得值,並將兩者分別添加進Dictionary當作key跟value。 87 | 4. 因為ExpandoObject有實作IDynamicMetaObjectProvider介面可以轉換成dynamic 88 | 89 | ```C# 90 | public static class DemoExtension 91 | { 92 | public static IEnumerable Query(this IDbConnection cnn, string sql) 93 | { 94 | using (var command = cnn.CreateCommand()) 95 | { 96 | command.CommandText = sql; 97 | using (var reader = command.ExecuteReader()) 98 | { 99 | while (reader.Read()) 100 | { 101 | yield return reader.CastToDynamic(); 102 | } 103 | } 104 | } 105 | } 106 | 107 | private static dynamic CastToDynamic(this IDataReader reader) 108 | { 109 | dynamic e = new ExpandoObject(); 110 | var d = e as IDictionary; 111 | for (int i = 0; i < reader.FieldCount; i++) 112 | d.Add(reader.GetName(i),reader[i]); 113 | return e; 114 | } 115 | } 116 | ``` 117 | 118 | ![20191003044145.png](https://i.loli.net/2019/10/03/7b3vFaEQpnSVj1y.png) 119 | 120 | 　 121 | 　 122 | ## 3.Dynamic Query 原理 Part2 123 | 124 | 125 | 126 | 有了前面簡單ExpandoObject Dynamic Query例子的概念後,接著進到底層來了解Dapper如何細節處理,為何要自訂義DynamicMetaObjectProvider。 127 | 128 | #### 首先掌握Dynamic Query流程邏輯 : 129 | 130 | 假設使用下面代碼 131 | ```C# 132 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 133 | { 134 | var result = cn.Query("select N'暐翰' Name,26 Age").First(); 135 | Console.WriteLine(result.Name); 136 | } 137 | ``` 138 | 139 | 取值的過程會是 : 建立動態Func > 保存在緩存 > 使用`result.Name` > 轉成呼叫 `((DapperRow)result)["Name"]` > 從`DapperTable.Values陣列`中以`"Name"欄位對應的Index`取值 140 | 141 | 接著查看源碼GetDapperRowDeserializer方法,它掌管dynamic如何運行的邏輯,並動態建立成Func給上層API呼叫、緩存重複利用。 142 | ![20191003190836.png](https://i.loli.net/2019/10/03/x2Pre7EDdTjUhXq.png) 143 | 144 | 此段Func邏輯 : 145 | 1. DapperTable雖然是方法內的局部變量,但是被生成的Func引用,所以`不會被GC`一直保存在內存內重複利用。 146 | ![20191003182219.png](https://i.loli.net/2019/10/03/kmXF7f5v8wTNKib.png) 147 | 148 | 2. 因為是dynamic不需要考慮類別Mapping,這邊直接使用`GetValue(index)`向數據庫取值 149 | ``` 150 | var values = new object[select欄位數量]; 151 | for (int i = 0; i < values.Length; i++) 152 | { 153 | object val = r.GetValue(i); 154 | values[i] = val is DBNull ? null : val; 155 | } 156 | ``` 157 | 158 | 3. 將資料保存到DapperRow內 159 | ```C# 160 | public DapperRow(DapperTable table, object[] values) 161 | { 162 | this.table = table ?? throw new ArgumentNullException(nameof(table)); 163 | this.values = values ?? throw new ArgumentNullException(nameof(values)); 164 | } 165 | ``` 166 | 167 | 4. DapperRow 繼承 IDynamicMetaObjectProvider 並實作 GetMetaObject 方法,實作邏輯是返回DapperRowMetaObject物件。 168 | 169 | ```C# 170 | private sealed partial class DapperRow : System.Dynamic.IDynamicMetaObjectProvider 171 | { 172 | DynamicMetaObject GetMetaObject(Expression parameter) 173 | { 174 | return new DapperRowMetaObject(parameter, System.Dynamic.BindingRestrictions.Empty, this); 175 | } 176 | } 177 | ``` 178 | 179 | 5. DapperRowMetaObject主要功能是定義行為,藉由override `BindSetMember、BindGetMember`方法,Dapper定義了Get、Set的行為分別使用`IDictionary - GetItem方法`跟`DapperRow - SetValue方法` 180 | ![20191003210351.png](https://i.loli.net/2019/10/03/xn5YMfiXkAvBq2O.png) 181 | ![20191003210547.png](https://i.loli.net/2019/10/03/u45PmaRGMQBUz7k.png) 182 | 183 | 6. 最後Dapper利用DataReader的`欄位順序性`,先利用欄位名稱取得Index,再利用Index跟Values取得值 184 | 185 | ![20191003211448.png](https://i.loli.net/2019/10/03/cGNIzAjuMv4HkyF.png) 186 | 187 | 188 | #### 為何要繼承IDictionary? 189 | 190 | 可以思考一個問題 : 在DapperRowMetaObject可以自行定義Get跟Set行為,那麼不使用Dictionary - GetItem方法,改用其他方式,是否代表不需要繼承`IDictionary`? 191 | 192 | Dapper這樣做的原因之一跟開放原則有關,DapperTable、DapperRow都是底層實作類別,基於開放封閉原則`不應該開放給使用者`,所以設為`private`權限。 193 | ``` 194 | private class DapperTable{/*略*/} 195 | private class DapperRow :IDictionary, IReadOnlyDictionary,System.Dynamic.IDynamicMetaObjectProvider{/*略*/} 196 | ``` 197 | 198 | 那麼使用者`想要知道欄位名稱`怎麼辦? 199 | 因為DapperRow實作IDictionary所以可以`向上轉型為IDictionary`,利用它為`公開介面`特性取得欄位資料。 200 | ```C# 201 | public interface IDictionary : ICollection>, IEnumerable>, IEnumerable{/*略*/} 202 | ``` 203 | 204 | 舉個例子,筆者有做一個小工具[HtmlTableHelper](https://github.com/shps951023/HtmlTableHelper)就是利用這特性,自動將Dapper Dynamic Query轉成Table Html,如以下代碼跟圖片 205 | ```C# 206 | using (var cn = "Your Connection") 207 | { 208 | var sourceData = cn.Query(@"select 'ITWeiHan' Name,25 Age,'M' Gender"); 209 | var tablehtml = sourceData.ToHtmlTable(); //Result :
NameAgeGender
ITWeiHan25M
210 | } 211 | ``` 212 | 213 | ![20191003212846.png](https://i.loli.net/2019/10/03/ke5Ty8lVGJCQjKX.png) 214 | 215 | 216 | 　 217 | 　 218 | ## 4. Strongly Typed Mapping 原理 Part1 : ADO.NET對比Dapper 219 | 220 | 221 | 接下來是Dapper關鍵功能 `Strongly Typed Mapping`,因為難度高,這邊會切分成多篇來解說。 222 | 223 | 第一篇先以ADO.NET DataReader GetItem By Index跟Dapper Strongly Typed Query對比,查看兩者IL的差異,了解Dapper Query Mapping的主要邏輯。 224 | 225 | 有了邏輯後,如何實作,我這邊依序用三個技術 :`Reflection、Expression、Emit` 從頭實作三個版本Query方法來讓讀者漸進式了解。 226 | 227 | ---- 228 | 229 | 230 | ### ADO.NET對比Dapper 231 | 232 | 首先使用以下代碼來追蹤Dapper Query邏輯 233 | ```C# 234 | class Program 235 | { 236 | static void Main(string[] args) 237 | { 238 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 239 | { 240 | var result = cn.Query("select N'暐翰' Name , 25 Age").First(); 241 | Console.WriteLine(result.Name); 242 | Console.WriteLine(result.Age); 243 | } 244 | } 245 | } 246 | 247 | public class User 248 | { 249 | public string Name { get; set; } 250 | public int Age { get; set; } 251 | } 252 | ``` 253 | 254 | 這邊需要重點來看`Dapper.SqlMapper.GenerateDeserializerFromMap`方法,它負責Mapping的邏輯,可以看到裡面大量使用Emit IL技術。 255 | 256 | ![20191004012713.png](https://i.loli.net/2019/10/04/DCVHTaOGBhcL59s.png) 257 | 258 | 要了解這段IL邏輯,我的方式 :`「不應該直接進到細節,而是先查看完整生成的IL」`,至於如何查看,這邊需要先準備 [il-visualizer](https://github.com/drewnoakes/il-visualizer) 開源工具,它可以在Runtime查看DynamicMethod生成的IL。 259 | 260 | 它預設支持vs 2015、2017,假如跟我一樣使用vs2019的讀者,需要注意 261 | 1. 需要手動解壓縮到 262 | `%USERPROFILE%\Documents\Visual Studio 2019`路徑下面 263 | ![20191004005622.png](https://i.loli.net/2019/10/04/wMHI1xCPQfV2rZg.png) 264 | 2. `.netstandard2.0`專案,需要建立`netstandard2.0`並解壓縮到該資料夾 265 | ![20191003044307.png](https://i.loli.net/2019/10/03/TvctsVmLJFXh43O.png) 266 | 267 | 268 | 最後重開visaul studio並debug運行,進到GetTypeDeserializerImpl方法,對DynamicMethod點擊放大鏡 > IL visualizer > 查看`Runtime`生成的IL代碼 269 | ![20191003044320.png](https://i.loli.net/2019/10/03/wyrcNSdnOloFICQ.png) 270 | 271 | 可以得出以下IL 272 | ```C# 273 | IL_0000: ldc.i4.0 274 | IL_0001: stloc.0 275 | IL_0002: newobj Void .ctor()/Demo.User 276 | IL_0007: stloc.1 277 | IL_0008: ldloc.1 278 | IL_0009: dup 279 | IL_000a: ldc.i4.0 280 | IL_000b: stloc.0 281 | IL_000c: ldarg.0 282 | IL_000d: ldc.i4.0 283 | IL_000e: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 284 | IL_0013: dup 285 | IL_0014: stloc.2 286 | IL_0015: dup 287 | IL_0016: isinst System.DBNull 288 | IL_001b: brtrue.s IL_0029 289 | IL_001d: unbox.any System.String 290 | IL_0022: callvirt Void set_Name(System.String)/Demo.User 291 | IL_0027: br.s IL_002b 292 | IL_0029: pop 293 | IL_002a: pop 294 | IL_002b: dup 295 | IL_002c: ldc.i4.1 296 | IL_002d: stloc.0 297 | IL_002e: ldarg.0 298 | IL_002f: ldc.i4.1 299 | IL_0030: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 300 | IL_0035: dup 301 | IL_0036: stloc.2 302 | IL_0037: dup 303 | IL_0038: isinst System.DBNull 304 | IL_003d: brtrue.s IL_004b 305 | IL_003f: unbox.any System.Int32 306 | IL_0044: callvirt Void set_Age(Int32)/Demo.User 307 | IL_0049: br.s IL_004d 308 | IL_004b: pop 309 | IL_004c: pop 310 | IL_004d: stloc.1 311 | IL_004e: leave IL_0060 312 | IL_0053: ldloc.0 313 | IL_0054: ldarg.0 314 | IL_0055: ldloc.2 315 | IL_0056: call Void ThrowDataException(System.Exception, Int32, System.Data.IDataReader, System.Object)/Dapper.SqlMapper 316 | IL_005b: leave IL_0060 317 | IL_0060: ldloc.1 318 | IL_0061: ret 319 | ``` 320 | 321 | 要了解這段IL之前需要先了解`ADO.NET DataReader快速讀取資料方式`會使用`GetItem By Index`方式,如以下代碼 322 | ```C# 323 | public static class DemoExtension 324 | { 325 | private static User CastToUser(this IDataReader reader) 326 | { 327 | var user = new User(); 328 | var value = reader[0]; 329 | if(!(value is System.DBNull)) 330 | user.Name = (string)value; 331 | var value = reader[1]; 332 | if(!(value is System.DBNull)) 333 | user.Age = (int)value; 334 | return user; 335 | } 336 | 337 | public static IEnumerable Query(this IDbConnection cnn, string sql) 338 | { 339 | if (cnn.State == ConnectionState.Closed) cnn.Open(); 340 | using (var command = cnn.CreateCommand()) 341 | { 342 | command.CommandText = sql; 343 | using (var reader = command.ExecuteReader()) 344 | while (reader.Read()) 345 | yield return reader.CastToUser(); 346 | } 347 | } 348 | } 349 | ``` 350 | 351 | 接著查看此Demo - CastToUser方法生成的IL代碼 352 | ```C# 353 | DemoExtension.CastToUser: 354 | IL_0000: nop 355 | IL_0001: newobj User..ctor 356 | IL_0006: stloc.0 // user 357 | IL_0007: ldarg.0 358 | IL_0008: ldc.i4.0 359 | IL_0009: callvirt System.Data.IDataRecord.get_Item 360 | IL_000E: stloc.1 // value 361 | IL_000F: ldloc.1 // value 362 | IL_0010: isinst System.DBNull 363 | IL_0015: ldnull 364 | IL_0016: cgt.un 365 | IL_0018: ldc.i4.0 366 | IL_0019: ceq 367 | IL_001B: stloc.2 368 | IL_001C: ldloc.2 369 | IL_001D: brfalse.s IL_002C 370 | IL_001F: ldloc.0 // user 371 | IL_0020: ldloc.1 // value 372 | IL_0021: castclass System.String 373 | IL_0026: callvirt User.set_Name 374 | IL_002B: nop 375 | IL_002C: ldarg.0 376 | IL_002D: ldc.i4.1 377 | IL_002E: callvirt System.Data.IDataRecord.get_Item 378 | IL_0033: stloc.1 // value 379 | IL_0034: ldloc.1 // value 380 | IL_0035: isinst System.DBNull 381 | IL_003A: ldnull 382 | IL_003B: cgt.un 383 | IL_003D: ldc.i4.0 384 | IL_003E: ceq 385 | IL_0040: stloc.3 386 | IL_0041: ldloc.3 387 | IL_0042: brfalse.s IL_0051 388 | IL_0044: ldloc.0 // user 389 | IL_0045: ldloc.1 // value 390 | IL_0046: unbox.any System.Int32 391 | IL_004B: callvirt User.set_Age 392 | IL_0050: nop 393 | IL_0051: ldloc.0 // user 394 | IL_0052: stloc.s 04 395 | IL_0054: br.s IL_0056 396 | IL_0056: ldloc.s 04 397 | IL_0058: ret 398 | 399 | ``` 400 | 401 | 跟Dapper生成的IL比對可以`發現大致是一樣的`(差異部分後面會講解),代表兩者在運行的邏輯、效率上都會是差不多的,這也是為何`Dapper效率接近原生ADO.NET`的原因之一。 402 | 403 | 404 | 　 405 | 　 406 | ## 5. Strongly Typed Mapping 原理 Part2 : Reflection版本 407 | 408 | 409 | 410 | 在前面ADO.NET Mapping例子可以發現嚴重問題`「沒辦法多類別共用方法,每新增一個類別就需要重寫代碼」`。要解決這個問題,可以寫一個共用方法在Runtime時期針對不同的類別做不同的邏輯處理。 411 | 412 | 實作方式做主要有三種Reflection、Expression、Emit,這邊首先介紹最簡單方式:「Reflection」,我這邊會使用反射方式從零模擬Query寫代碼,讓讀者初步了解動態處理概念。(假如有經驗的讀者可以跳過本篇) 413 | 414 | 邏輯 : 415 | 1. 使用泛型傳遞動態類別 416 | 2. 使用`泛型的條件約束new()`達到動態建立物件 417 | 3. DataReader需要使用`屬性字串名稱當Key`,可以使用Reflection取得動態類別的屬性名稱,在藉由`DataReader this[string parameter]`取得數據庫資料 418 | 4. 使用PropertyInfo.SetValue方式動態將數據庫資料賦予物件 419 | 420 | 最後得到以下代碼 : 421 | 422 | ```C# 423 | public static class DemoExtension 424 | { 425 | public static IEnumerable Query(this IDbConnection cnn, string sql) where T : new() 426 | { 427 | using (var command = cnn.CreateCommand()) 428 | { 429 | command.CommandText = sql; 430 | using (var reader = command.ExecuteReader()) 431 | while (reader.Read()) 432 | yield return reader.CastToType(); 433 | } 434 | } 435 | 436 | //1.使用泛型傳遞動態類別 437 | private static T CastToType(this IDataReader reader) where T : new() 438 | { 439 | //2.使用泛型的條件約束new()達到動態建立物件 440 | var instance = new T(); 441 | 442 | //3.DataReader需要使用屬性字串名稱當Key,可以使用Reflection取得動態類別的屬性名稱,在藉由DataReader this[string parameter]取得數據庫資料 443 | var type = typeof(T); 444 | var props = type.GetProperties(); 445 | foreach (var p in props) 446 | { 447 | var val = reader[p.Name]; 448 | 449 | //4.使用PropertyInfo.SetValue方式動態將數據庫資料賦予物件 450 | if( !(val is System.DBNull) ) 451 | p.SetValue(instance, val); 452 | } 453 | 454 | return instance; 455 | } 456 | } 457 | ``` 458 | 459 | Reflection版本優點是代碼`簡單`,但它有以下問題 460 | 1. 不應該重複屬性查詢,沒用到就要忽略 461 | 舉例 : 假如類別有N個屬性,SQL指查詢3個欄位,土炮ORM每次PropertyInfo foreach還是N次不是3次。而Dapper在Emit IL當中特別優化此段邏輯 : `「查多少用多少,不浪費」`(這段之後講解)。 462 | ![https://ithelp.ithome.com.tw/upload/images/20191003/20105988Y7jmgF76Wd.png](https://ithelp.ithome.com.tw/upload/images/20191003/20105988Y7jmgF76Wd.png) 463 | ![https://ithelp.ithome.com.tw/upload/images/20191003/20105988nHZMb3Copc.png](https://ithelp.ithome.com.tw/upload/images/20191003/20105988nHZMb3Copc.png) 464 | 2. 效率問題 : 465 | - 反射效率會比較慢,這點之後會介紹解決方式 : `「查表法 + 動態建立方法」`以空間換取時間。 466 | - 使用字串Key取值會多呼叫了`GetOrdinal`方法,可以查看MSDN官方解釋,`效率比Index取值差`。 467 | ![https://ithelp.ithome.com.tw/upload/images/20191003/20105988ABufu55xes.png](https://ithelp.ithome.com.tw/upload/images/20191003/20105988ABufu55xes.png) 468 | ![https://ithelp.ithome.com.tw/upload/images/20191003/20105988TqMlMbAIls.png](https://ithelp.ithome.com.tw/upload/images/20191003/20105988TqMlMbAIls.png) 469 | 470 | 471 | 472 | 473 | 474 | 　 475 | 　 476 | ## 6.Strongly Typed Mapping 原理 Part3 : 動態建立方法重要概念「結果反推程式碼」優化效率 477 | 478 | 479 | 接著使用Expression來解決Reflection版本問題,主要是利用Expression特性 : `「可以在Runtime時期動態建立方法」`來解決問題。 480 | 481 | 在這之前需要先有一個重要概念 : `「從結果反推最簡潔代碼」`優化效率,舉個例子 : 以前初學程式時一個經典題目「打印正三角型星星」做出一個長度為3的正三角,常見作法會是迴圈+遞迴方式 482 | ```C# 483 | void Main() 484 | { 485 | Print(3,0); 486 | } 487 | 488 | static void Print(int length, int spaceLength) 489 | { 490 | if (length < 0) 491 | return; 492 | else 493 | Print(length - 1, spaceLength + 1); 494 | for (int i = 0; i < spaceLength; i++) 495 | Console.Write(" "); 496 | for (int i = 0; i < length; i++) 497 | Console.Write("* "); 498 | Console.WriteLine(""); 499 | } 500 | ``` 501 | 502 | 但其實這個題目在已經知道長度的情況下,可以被改成以下代碼 503 | ```C# 504 | Console.WriteLine(" * "); 505 | Console.WriteLine(" * * "); 506 | Console.WriteLine("* * * "); 507 | ``` 508 | 509 | 這個概念很重要,因為是從結果反推代碼,所以邏輯直接、`效率快`,而Dapper就是使用此概念來動態建立方法。 510 | 511 | 舉例 : 假設有一段代碼如下,我們可以從結果得出 512 | - User Class的Name屬性對應Reader Index 0 、類別是String 、 預設值是null 513 | - User Class的Age屬性對應Reader Index 1 、類別是int 、 預設值是0 514 | ```C# 515 | void Main() 516 | { 517 | using (var cn = Connection) 518 | { 519 | var result = cn.Query("select N'暐翰' Name,26 Age").First(); 520 | } 521 | } 522 | 523 | class User 524 | { 525 | public string Name { get; set; } 526 | public int Age { get; set; } 527 | } 528 | ``` 529 | 530 | 假如系統能幫忙生成以下邏輯方法,那麼效率會是最好的 531 | ```C# 532 | User 動態方法(IDataReader reader) 533 | { 534 | var user = new User(); 535 | var value = reader[0]; 536 | if( !(value is System.DBNull) ) 537 | user.Name = (string)value; 538 | value = reader[1]; 539 | if( !(value is System.DBNull) ) 540 | user.Age = (int)value; 541 | return user; 542 | } 543 | ``` 544 | 545 | 另外上面例子可以看出對Dapper來說`SQL Select對應Class屬性順序很重要`,所以後面會講解Dapper在緩存的算法特別針對此優化。 546 | 547 | 548 | 549 | 　 550 | 　 551 | ## 7.Strongly Typed Mapping 原理 Part4 : Expression版本 552 | 553 | 554 | 有了前面的邏輯,就著使用Expression實作動態建立方法。 555 | 556 | #### 為何先使用 Expression 實作而不是 Emit ? 557 | 除了有能力動態建立方法,相比Emit有以下優點 : 558 | - `可讀性好`,可用熟悉的關鍵字,像是變量Variable對應Expression.Variable、建立物件New對應Expression.New 559 | ![https://ithelp.ithome.com.tw/upload/images/20190920/20105988rkSmaILTw7.png](https://ithelp.ithome.com.tw/upload/images/20190920/20105988rkSmaILTw7.png) 560 | - `方便Runtime Debug`,可以在Debug模式下看到Expression對應邏輯代碼 561 | ![https://ithelp.ithome.com.tw/upload/images/20190920/201059882EODD9OdnD.png](https://ithelp.ithome.com.tw/upload/images/20190920/201059882EODD9OdnD.png) 562 | ![https://ithelp.ithome.com.tw/upload/images/20190920/201059882gSYyfUduS.png](https://ithelp.ithome.com.tw/upload/images/20190920/201059882gSYyfUduS.png) 563 | 564 | 所以特別適合介紹動態方法建立,但Expression相比Emit無法作一些細節操作,這點會在後面Emit講解到。 565 | 566 | 567 | #### 改寫Expression版本 568 | 569 | 邏輯 : 570 | 1. 取得sql select所有欄位名稱 571 | 2. 取得mapping類別的屬性資料 > 將index,sql欄位,class屬性資料做好對應封裝在一個變量內方便後面使用 572 | 3. 動態建立方法 : 從數據庫Reader按照順序讀取我們要的資料,其中代碼邏輯 : 573 | ```C# 574 | User 動態方法(IDataReader reader) 575 | { 576 | var user = new User(); 577 | var value = reader[0]; 578 | if( !(value is System.DBNull) ) 579 | user.Name = (string)value; 580 | value = reader[1]; 581 | if( !(value is System.DBNull) ) 582 | user.Age = (int)value; 583 | return user; 584 | } 585 | ``` 586 | 587 | 最後得出以下Exprssion版本代碼 588 | ```C# 589 | public static class DemoExtension 590 | { 591 | public static IEnumerable Query(this IDbConnection cnn, string sql) where T : new() 592 | { 593 | using (var command = cnn.CreateCommand()) 594 | { 595 | command.CommandText = sql; 596 | using (var reader = command.ExecuteReader()) 597 | { 598 | var func = CreateMappingFunction(reader, typeof(T)); 599 | while (reader.Read()) 600 | { 601 | var result = func(reader as DbDataReader); 602 | yield return result is T ? (T)result : default(T); 603 | } 604 | 605 | } 606 | } 607 | } 608 | 609 | private static Func CreateMappingFunction(IDataReader reader, Type type) 610 | { 611 | //1. 取得sql select所有欄位名稱 612 | var names = Enumerable.Range(0, reader.FieldCount).Select(index => reader.GetName(index)).ToArray(); 613 | 614 | //2. 取得mapping類別的屬性資料 > 將index,sql欄位,class屬性資料做好對應封裝在一個變量內方便後面使用 615 | var props = type.GetProperties().ToList(); 616 | var members = names.Select((columnName, index) => 617 | { 618 | var property = props.Find(p => string.Equals(p.Name, columnName, StringComparison.Ordinal)) 619 | ?? props.Find(p => string.Equals(p.Name, columnName, StringComparison.OrdinalIgnoreCase)); 620 | return new 621 | { 622 | index, 623 | columnName, 624 | property 625 | }; 626 | }); 627 | 628 | //3. 動態建立方法 : 從數據庫Reader按照順序讀取我們要的資料 629 | /*方法邏輯 : 630 | User 動態方法(IDataReader reader) 631 | { 632 | var user = new User(); 633 | var value = reader[0]; 634 | if( !(value is System.DBNull) ) 635 | user.Name = (string)value; 636 | value = reader[1]; 637 | if( !(value is System.DBNull) ) 638 | user.Age = (int)value; 639 | return user; 640 | } 641 | */ 642 | var exBodys = new List(); 643 | { 644 | // 方法(IDataReader reader) 645 | var exParam = Expression.Parameter(typeof(DbDataReader), "reader"); 646 | 647 | // Mapping類別 物件 = new Mapping類別(); 648 | var exVar = Expression.Variable(type, "mappingObj"); 649 | var exNew = Expression.New(type); 650 | { 651 | exBodys.Add(Expression.Assign(exVar, exNew)); 652 | } 653 | 654 | // var value = defalut(object); 655 | var exValueVar = Expression.Variable(typeof(object), "value"); 656 | { 657 | exBodys.Add(Expression.Assign(exValueVar, Expression.Constant(null))); 658 | } 659 | 660 | 661 | var getItemMethod = typeof(DbDataReader).GetMethods().Where(w => w.Name == "get_Item") 662 | .First(w => w.GetParameters().First().ParameterType == typeof(int)); 663 | foreach (var m in members) 664 | { 665 | //reader[0] 666 | var exCall = Expression.Call( 667 | exParam, getItemMethod, 668 | Expression.Constant(m.index) 669 | ); 670 | 671 | // value = reader[0]; 672 | exBodys.Add(Expression.Assign(exValueVar, exCall)); 673 | 674 | //user.Name = (string)value; 675 | var exProp = Expression.Property(exVar, m.property.Name); 676 | var exConvert = Expression.Convert(exValueVar, m.property.PropertyType); //(string)value 677 | var exPropAssign = Expression.Assign(exProp, exConvert); 678 | 679 | //if ( !(value is System.DBNull)) 680 | // (string)value 681 | var exIfThenElse = Expression.IfThen( 682 | Expression.Not(Expression.TypeIs(exValueVar, typeof(System.DBNull))) 683 | , exPropAssign 684 | ); 685 | 686 | exBodys.Add(exIfThenElse); 687 | } 688 | 689 | 690 | // return user; 691 | exBodys.Add(exVar); 692 | 693 | // Compiler Expression 694 | var lambda = Expression.Lambda>( 695 | Expression.Block( 696 | new[] { exVar, exValueVar }, 697 | exBodys 698 | ), exParam 699 | ); 700 | 701 | return lambda.Compile(); 702 | } 703 | } 704 | } 705 | ``` 706 | 707 | 查詢效果圖 : 708 | ![20191004205645.png](https://i.loli.net/2019/10/04/7Sbtr6gRdm4DyLs.png) 709 | 710 | 711 | 最後查看Expression.Lambda > DebugView(注意是非公開屬性)驗證代碼 : 712 | ```C# 713 | .Lambda #Lambda1(System.Data.Common.DbDataReader $reader) { 714 | .Block( 715 | UserQuery+User $mappingObj, 716 | System.Object $value) { 717 | $mappingObj = .New UserQuery+User(); 718 | $value = null; 719 | $value = .Call $reader.get_Item(0); 720 | .If ( 721 | !($value .Is System.DBNull) 722 | ) { 723 | $mappingObj.Name = (System.String)$value 724 | } .Else { 725 | .Default(System.Void) 726 | }; 727 | $value = .Call $reader.get_Item(1); 728 | .If ( 729 | !($value .Is System.DBNull) 730 | ) { 731 | $mappingObj.Age = (System.Int32)$value 732 | } .Else { 733 | .Default(System.Void) 734 | }; 735 | $mappingObj 736 | } 737 | } 738 | ``` 739 | 740 | ![20191005035640.png](https://i.loli.net/2019/10/05/3KVFvHyYS6kjDAs.png) 741 | 742 | 　 743 | 　 744 | ## 8. Strongly Typed Mapping 原理 Part5 : Emit IL反建立C#代碼 745 | 746 | 747 | 有了前面Expression版本概念後,接著可以進到Dapper底層最核心的技術 : Emit。 748 | 749 | 首先要有個概念,MSIL(CIL)目的是給JIT編譯器看的,所以可讀性會很差、難Debug,但比起Expression來說可以做到更細節的邏輯操作。 750 | 751 | 在實際環境開發使用Emit,一般會`先寫好C#代碼後 > 反編譯查看IL > 使用Emit建立動態方法`,舉例 : 752 | 753 | 1.首先建立一個簡單打印例子 : 754 | ```C# 755 | void SyaHello() 756 | { 757 | Console.WriteLine("Hello World"); 758 | } 759 | ``` 760 | 2.反編譯查看IL 761 | ``` 762 | SyaHello: 763 | IL_0000: nop 764 | IL_0001: ldstr "Hello World" 765 | IL_0006: call System.Console.WriteLine 766 | IL_000B: nop 767 | IL_000C: ret 768 | ``` 769 | 3.使用DynamicMethod + Emit建立動態方法 770 | ```C# 771 | void Main() 772 | { 773 | // 1. 建立 void 方法() 774 | DynamicMethod methodbuilder = new DynamicMethod("Deserialize" + Guid.NewGuid().ToString(),typeof(void),null); 775 | 776 | // 2. 建立方法Body內容,藉由Emit 777 | var il = methodbuilder.GetILGenerator(); 778 | il.Emit(OpCodes.Ldstr, "Hello World"); 779 | Type[] types = new Type[1] 780 | { 781 | typeof(string) 782 | }; 783 | MethodInfo method = typeof(Console).GetMethod("WriteLine", types); 784 | il.Emit(OpCodes.Call,method); 785 | il.Emit(OpCodes.Ret); 786 | 787 | // 3. 轉換指定類型的Func or Action 788 | var action = (Action)methodbuilder.CreateDelegate(typeof(Action)); 789 | 790 | action(); 791 | } 792 | ``` 793 | 794 | ![https://ithelp.ithome.com.tw/upload/images/20190924/20105988bD9GSXyjNt.png](https://ithelp.ithome.com.tw/upload/images/20190924/20105988bD9GSXyjNt.png) 795 | 796 | 797 | 但是對已經寫好的專案來說就不是這樣流程了,開發者不一定會好心的告訴你當初設計的邏輯,所以接著討論此問題。 798 | 799 | #### 如果像是Dapper只有Emit IL沒有C# Source Code專案怎麼辦? 800 | 我的解決方式是 : `「既然只有Runtime才能知道IL,那麼將IL保存成靜態檔案再反編譯查看」` 801 | 802 | 這邊可以使用`MethodBuild + Save`方法將`IL保存成靜態exe檔案 > 反編譯查看`,但需要特別注意 803 | 1. 請對應好參數跟返回類別,否則會編譯錯誤。 804 | 2. netstandard不支援此方式,Dapper需要使用`region if 指定版本`來做區分,否則不能使用,如圖片 805 | ![20191004230125.png](https://i.loli.net/2019/10/04/ZUCKIcpnjmPB5Fb.png) 806 | 807 | 代碼如下 : 808 | ```C# 809 | //使用MethodBuilder查看別人已經寫好的Emit IL 810 | //1. 建立MethodBuilder 811 | AppDomain ad = AppDomain.CurrentDomain; 812 | AssemblyName am = new AssemblyName(); 813 | am.Name = "TestAsm"; 814 | AssemblyBuilder ab = ad.DefineDynamicAssembly(am, AssemblyBuilderAccess.Save); 815 | ModuleBuilder mb = ab.DefineDynamicModule("Testmod", "TestAsm.exe"); 816 | TypeBuilder tb = mb.DefineType("TestType", TypeAttributes.Public); 817 | MethodBuilder dm = tb.DefineMethod("TestMeThod", MethodAttributes.Public | 818 | MethodAttributes.Static, type, new[] { typeof(IDataReader) }); 819 | ab.SetEntryPoint(dm); 820 | 821 | // 2. 填入IL代碼 822 | //..略 823 | 824 | // 3. 生成靜態檔案 825 | tb.CreateType(); 826 | ab.Save("TestAsm.exe"); 827 | ``` 828 | 829 | 接著使用此方式在GetTypeDeserializerImpl方法反編譯Dapper Query Mapping IL,可以得出C#代碼 : 830 | 831 | ```C# 832 | public static User TestMeThod(IDataReader P_0) 833 | { 834 | int index = 0; 835 | User user = new User(); 836 | object value = default(object); 837 | try 838 | { 839 | User user2 = user; 840 | index = 0; 841 | object obj = value = P_0[0]; 842 | if (!(obj is DBNull)) 843 | { 844 | user2.Name = (string)obj; 845 | } 846 | index = 1; 847 | object obj2 = value = P_0[1]; 848 | if (!(obj2 is DBNull)) 849 | { 850 | user2.Age = (int)obj2; 851 | } 852 | user = user2; 853 | return user; 854 | } 855 | catch (Exception ex) 856 | { 857 | SqlMapper.ThrowDataException(ex, index, P_0, value); 858 | return user; 859 | } 860 | } 861 | ``` 862 | 863 | ![20191004230548.png](https://i.loli.net/2019/10/04/YBcnfCz2FtudaWX.png) 864 | 865 | 有了C#代碼後再來了解Emit邏輯會快很多,接著就可以進到Emit版本Query實作部分。 866 | 867 | 　 868 | 　 869 | ## 9.Strongly Typed Mapping 原理 Part6 : Emit版本 870 | 871 | 872 | 以下代碼是Emit版本,我把C#對應IL部分都寫在註解。 873 | ```C# 874 | public static class DemoExtension 875 | { 876 | public static IEnumerable Query(this IDbConnection cnn, string sql) where T : new() 877 | { 878 | using (var command = cnn.CreateCommand()) 879 | { 880 | command.CommandText = sql; 881 | using (var reader = command.ExecuteReader()) 882 | { 883 | var func = GetTypeDeserializerImpl(typeof(T), reader); 884 | 885 | while (reader.Read()) 886 | { 887 | var result = func(reader as DbDataReader); 888 | yield return result is T ? (T)result : default(T); 889 | } 890 | } 891 | 892 | } 893 | } 894 | 895 | private static Func GetTypeDeserializerImpl(Type type, IDataReader reader, int startBound = 0, int length = -1, bool returnNullIfFirstMissing = false) 896 | { 897 | var returnType = type.IsValueType ? typeof(object) : type; 898 | 899 | var dm = new DynamicMethod("Deserialize" + Guid.NewGuid().ToString(), returnType, new[] { typeof(IDataReader) }, type, true); 900 | var il = dm.GetILGenerator(); 901 | 902 | //C# : User user = new User(); 903 | //IL : 904 | //IL_0001: newobj 905 | //IL_0006: stloc.0 906 | var constructor = returnType.GetConstructors(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic)[0]; //這邊簡化成只會有預設constructor 907 | il.Emit(OpCodes.Newobj, constructor); 908 | var returnValueLocal = il.DeclareLocal(type); 909 | il.Emit(OpCodes.Stloc, returnValueLocal); //User user = new User(); 910 | 911 | // C# : 912 | //object value = default(object); 913 | // IL : 914 | //IL_0007: ldnull 915 | //IL_0008: stloc.1 // value 916 | var valueLoacl = il.DeclareLocal(typeof(object)); 917 | il.Emit(OpCodes.Ldnull); 918 | il.Emit(OpCodes.Stloc, valueLoacl); 919 | 920 | 921 | int index = startBound; 922 | var getItem = typeof(IDataRecord).GetProperties(BindingFlags.Instance | BindingFlags.Public) 923 | .Where(p => p.GetIndexParameters().Length > 0 && p.GetIndexParameters()[0].ParameterType == typeof(int)) 924 | .Select(p => p.GetGetMethod()).First(); 925 | 926 | foreach (var p in type.GetProperties()) 927 | { 928 | //C# : value = P_0[0]; 929 | //IL: 930 | //IL_0009: ldarg.0 931 | //IL_000A: ldc.i4.0 932 | //IL_000B: callvirt System.Data.IDataRecord.get_Item 933 | //IL_0010: stloc.1 // value 934 | il.Emit(OpCodes.Ldarg_0); //取得reader參數 935 | EmitInt32(il, index); 936 | il.Emit(OpCodes.Callvirt, getItem); 937 | il.Emit(OpCodes.Stloc, valueLoacl); 938 | 939 | 940 | //C#: if (!(value is DBNull)) user.Name = (string)value; 941 | //IL: 942 | // IL_0011: ldloc.1 // value 943 | // IL_0012: isinst System.DBNull 944 | // IL_0017: ldnull 945 | // IL_0018: cgt.un 946 | // IL_001A: ldc.i4.0 947 | // IL_001B: ceq 948 | // IL_001D: stloc.2 949 | // IL_001E: ldloc.2 950 | // IL_001F: brfalse.s IL_002E 951 | // IL_0021: ldloc.0 // user 952 | // IL_0022: ldloc.1 // value 953 | // IL_0023: castclass System.String 954 | // IL_0028: callvirt UserQuery+User.set_Name 955 | il.Emit(OpCodes.Ldloc, valueLoacl); 956 | il.Emit(OpCodes.Isinst, typeof(System.DBNull)); 957 | il.Emit(OpCodes.Ldnull); 958 | 959 | var tmpLoacl = il.DeclareLocal(typeof(int)); 960 | il.Emit(OpCodes.Cgt_Un); 961 | il.Emit(OpCodes.Ldc_I4_0); 962 | il.Emit(OpCodes.Ceq); 963 | 964 | il.Emit(OpCodes.Stloc,tmpLoacl); 965 | il.Emit(OpCodes.Ldloc,tmpLoacl); 966 | 967 | 968 | var labelFalse = il.DefineLabel(); 969 | il.Emit(OpCodes.Brfalse_S,labelFalse); 970 | il.Emit(OpCodes.Ldloc, returnValueLocal); 971 | il.Emit(OpCodes.Ldloc, valueLoacl); 972 | if (p.PropertyType.IsValueType) 973 | il.Emit(OpCodes.Unbox_Any, p.PropertyType); 974 | else 975 | il.Emit(OpCodes.Castclass, p.PropertyType); 976 | il.Emit(OpCodes.Callvirt, p.SetMethod); 977 | 978 | il.MarkLabel(labelFalse); 979 | 980 | index++; 981 | } 982 | 983 | // IL_0053: ldloc.0 // user 984 | // IL_0054: stloc.s 04 //不需要 985 | // IL_0056: br.s IL_0058 986 | // IL_0058: ldloc.s 04 //不需要 987 | // IL_005A: ret 988 | il.Emit(OpCodes.Ldloc, returnValueLocal); 989 | il.Emit(OpCodes.Ret); 990 | 991 | var funcType = System.Linq.Expressions.Expression.GetFuncType(typeof(IDataReader), returnType); 992 | return (Func)dm.CreateDelegate(funcType); 993 | } 994 | 995 | private static void EmitInt32(ILGenerator il, int value) 996 | { 997 | switch (value) 998 | { 999 | case -1: il.Emit(OpCodes.Ldc_I4_M1); break; 1000 | case 0: il.Emit(OpCodes.Ldc_I4_0); break; 1001 | case 1: il.Emit(OpCodes.Ldc_I4_1); break; 1002 | case 2: il.Emit(OpCodes.Ldc_I4_2); break; 1003 | case 3: il.Emit(OpCodes.Ldc_I4_3); break; 1004 | case 4: il.Emit(OpCodes.Ldc_I4_4); break; 1005 | case 5: il.Emit(OpCodes.Ldc_I4_5); break; 1006 | case 6: il.Emit(OpCodes.Ldc_I4_6); break; 1007 | case 7: il.Emit(OpCodes.Ldc_I4_7); break; 1008 | case 8: il.Emit(OpCodes.Ldc_I4_8); break; 1009 | default: 1010 | if (value >= -128 && value <= 127) 1011 | { 1012 | il.Emit(OpCodes.Ldc_I4_S, (sbyte)value); 1013 | } 1014 | else 1015 | { 1016 | il.Emit(OpCodes.Ldc_I4, value); 1017 | } 1018 | break; 1019 | } 1020 | } 1021 | } 1022 | ``` 1023 | 1024 | 這邊Emit的細節概念非常的多,這邊無法全部都講解,先挑出重要概念講解 1025 | 1026 | #### Emit Label 1027 | 在Emit if/else需要使用Label定位,告知編譯器條件為true/false時要跳到哪個位子,舉例 : 「boolean轉整數」,假設要簡單將Boolean轉換成Int,C#代碼可以用「如果是True返回1否則返回0」邏輯來寫: 1028 | ```C# 1029 | public static int BoolToInt(bool input) => input ? 1 : 0; 1030 | ``` 1031 | 1032 | 當轉成Emit寫法的時候,需要以下邏輯 : 1033 | 1. 考慮Label動態定位問題 1034 | 2. 先要建立好Label讓Brtrue_S知道符合條件時要去哪個Label位子 `(注意,這時候Label位子還沒確定)` 1035 | 3. 繼續按順序由上而下建立IL 1036 | 4. 等到了`符合條件`要運行區塊的`前一行`,使用`MarkLabel方法標記Label的位子`。 1037 | 1038 | 最後寫出的C# Emit代碼 : 1039 | ```C# 1040 | public class Program 1041 | { 1042 | public static void Main(string[] args) 1043 | { 1044 | var func = CreateFunc(); 1045 | Console.WriteLine(func(true)); //1 1046 | Console.WriteLine(func(false)); //0 1047 | } 1048 | 1049 | static Func CreateFunc() 1050 | { 1051 | var dm = new DynamicMethod("Test" + Guid.NewGuid().ToString(), typeof(int), new[] { typeof(bool) }); 1052 | 1053 | var il = dm.GetILGenerator(); 1054 | var labelTrue = il.DefineLabel(); 1055 | 1056 | il.Emit(OpCodes.Ldarg_0); 1057 | il.Emit(OpCodes.Brtrue_S, labelTrue); 1058 | il.Emit(OpCodes.Ldc_I4_0); 1059 | il.Emit(OpCodes.Ret); 1060 | il.MarkLabel(labelTrue); 1061 | il.Emit(OpCodes.Ldc_I4_1); 1062 | il.Emit(OpCodes.Ret); 1063 | 1064 | var funcType = System.Linq.Expressions.Expression.GetFuncType(typeof(bool), typeof(int)); 1065 | return (Func)dm.CreateDelegate(funcType); 1066 | } 1067 | } 1068 | ``` 1069 | 1070 | --- 1071 | 1072 | 這邊可以發現Emit版本 1073 | 優點 : 1074 | 1. 能做更多細節的操作 1075 | 2. 因為細節顆粒度小,可以優化的效率更好 1076 | 1077 | 缺點 : 1078 | 1. 難以Debug 1079 | 2. 可讀性差 1080 | 3. 代碼量變大、複雜度增加 1081 | 1082 | 接著來看Dapper作者的建議,現在一般專案當中沒有必要使用Emit,使用Expression + Func/Action已經可以解決大部分動態方法的需求,尤其是Expression支援Block等方法情況。連結 [c# - What's faster: expression trees or manually emitting IL ](https://stackoverflow.com/questions/16530539/whats-faster-expression-trees-or-manually-emitting-il) 1083 | 1084 | ![20190927163441.png](https://i.loli.net/2019/09/27/Y4WeGHu8amVdIwz.png) 1085 | 1086 | 話雖如此,但有一些厲害的開源專案就是使用Emit管理細節,`如果想看懂它們,就需要基礎的Emit IL概念`。 1087 | 1088 | 1089 | 1090 | 1091 | 　 1092 | 　 1093 | ## 10.Dapper 效率快關鍵之一 : Cache 緩存原理 1094 | 1095 | 1096 | #### 為何Dapper可以這麼快? 1097 | 前面介紹到動態使用 Emit IL 建立 ADO.NET Mapping 方法,但單就這功能無法讓 Dapper 被稱為輕量ORM效率之王。 1098 | 1099 | 因為動態建立方法是`需要成本、並耗費時間`的動作,單純使用反而會拖慢速度。但當配合 Cache 後就不一樣,將建立好的方法保存在 Cache 內,可以用`『空間換取時間』`概念加快查詢的效率,也就是俗稱`查表法`。 1100 | 1101 | 接著追蹤Dapper源碼,這次需要特別關注的是QueryImpl方法下的Identity、GetCacheInfo 1102 | ![https://ithelp.ithome.com.tw/upload/images/20191005/20105988cCwaS7ejnY.png](https://ithelp.ithome.com.tw/upload/images/20191005/20105988cCwaS7ejnY.png) 1103 | 1104 | 1105 | #### Identity、GetCacheInfo 1106 | 1107 | Identity主要封裝各緩存的比較Key屬性 : 1108 | - sql : 區分不同SQL字串 1109 | - type : 區分Mapping類別 1110 | - commandType : 負責區分不同數據庫 1111 | - gridIndex : 主用用在QueryMultiple,後面講解。 1112 | - connectionString : 主要區分同數據庫廠商但是不同DB情況 1113 | - parametersType : 主要區分參數類別 1114 | - typeCount : 主要用在Multi Query多映射,需要搭配override GetType方法,後面講解 1115 | 1116 | 接著搭配GetCacheInfo方法內Dapper使用的緩存類別`ConcurrentDictionary`,使用`TryGetValue`方法時會去先比對HashCode接著比對Equals特性,如圖片源碼。 1117 | ![https://ithelp.ithome.com.tw/upload/images/20191005/20105988tOgZiBCwly.png](https://ithelp.ithome.com.tw/upload/images/20191005/20105988tOgZiBCwly.png) 1118 | 1119 | 1120 | 將Key類別Identity藉由`override Equals`方法實現緩存比較算法,可以看到以下Dapper實作邏輯,只要一個屬性不一樣就會建立一個新的動態方法、緩存。 1121 | ```C# 1122 | public bool Equals(Identity other) 1123 | { 1124 | if (ReferenceEquals(this, other)) return true; 1125 | if (ReferenceEquals(other, null)) return false; 1126 | 1127 | int typeCount; 1128 | return gridIndex == other.gridIndex 1129 | && type == other.type 1130 | && sql == other.sql 1131 | && commandType == other.commandType 1132 | && connectionStringComparer.Equals(connectionString, other.connectionString) 1133 | && parametersType == other.parametersType 1134 | && (typeCount = TypeCount) == other.TypeCount 1135 | && (typeCount == 0 || TypesEqual(this, other, typeCount)); 1136 | } 1137 | ``` 1138 | 1139 | 1140 | 以此概念拿之前Emit版本修改成一個簡單Cache Demo讓讀者感受: 1141 | ```C# 1142 | public class Identity 1143 | { 1144 | public string sql { get; set; } 1145 | public CommandType? commandType { get; set; } 1146 | public string connectionString { get; set; } 1147 | public Type type { get; set; } 1148 | public Type parametersType { get; set; } 1149 | public Identity(string sql, CommandType? commandType, string connectionString, Type type, Type parametersType) 1150 | { 1151 | this.sql = sql; 1152 | this.commandType = commandType; 1153 | this.connectionString = connectionString; 1154 | this.type = type; 1155 | this.parametersType = parametersType; 1156 | unchecked 1157 | { 1158 | hashCode = 17; // we *know* we are using this in a dictionary, so pre-compute this 1159 | hashCode = (hashCode * 23) + commandType.GetHashCode(); 1160 | hashCode = (hashCode * 23) + (sql?.GetHashCode() ?? 0); 1161 | hashCode = (hashCode * 23) + (type?.GetHashCode() ?? 0); 1162 | hashCode = (hashCode * 23) + (connectionString == null ? 0 : StringComparer.Ordinal.GetHashCode(connectionString)); 1163 | hashCode = (hashCode * 23) + (parametersType?.GetHashCode() ?? 0); 1164 | } 1165 | } 1166 | 1167 | public readonly int hashCode; 1168 | public override int GetHashCode() => hashCode; 1169 | 1170 | public override bool Equals(object obj) => Equals(obj as Identity); 1171 | public bool Equals(Identity other) 1172 | { 1173 | if (ReferenceEquals(this, other)) return true; 1174 | if (ReferenceEquals(other, null)) return false; 1175 | 1176 | return type == other.type 1177 | && sql == other.sql 1178 | && commandType == other.commandType 1179 | && StringComparer.Ordinal.Equals(connectionString, other.connectionString) 1180 | && parametersType == other.parametersType; 1181 | } 1182 | } 1183 | 1184 | public static class DemoExtension 1185 | { 1186 | private static readonly Dictionary> readers = new Dictionary>(); 1187 | 1188 | public static IEnumerable Query(this IDbConnection cnn, string sql,object param=null) where T : new() 1189 | { 1190 | using (var command = cnn.CreateCommand()) 1191 | { 1192 | command.CommandText = sql; 1193 | using (var reader = command.ExecuteReader()) 1194 | { 1195 | var identity = new Identity(command.CommandText, command.CommandType, cnn.ConnectionString, typeof(T), param?.GetType()); 1196 | 1197 | // 2. 如果cache有資料就使用,沒有資料就動態建立方法並保存在緩存內 1198 | if (!readers.TryGetValue(identity, out Func func)) 1199 | { 1200 | //動態建立方法 1201 | func = GetTypeDeserializerImpl(typeof(T), reader); 1202 | readers[identity] = func; 1203 | Console.WriteLine("沒有緩存,建立動態方法放進緩存"); 1204 | }else{ 1205 | Console.WriteLine("使用緩存"); 1206 | } 1207 | 1208 | 1209 | // 3. 呼叫生成的方法by reader,讀取資料回傳 1210 | while (reader.Read()) 1211 | { 1212 | var result = func(reader as DbDataReader); 1213 | yield return result is T ? (T)result : default(T); 1214 | } 1215 | } 1216 | 1217 | } 1218 | } 1219 | 1220 | private static Func GetTypeDeserializerImpl(Type type, IDataReader reader, int startBound = 0, int length = -1, bool returnNullIfFirstMissing = false) 1221 | { 1222 | //..略 1223 | } 1224 | } 1225 | ``` 1226 | 1227 | 效果圖 : 1228 | ![https://ithelp.ithome.com.tw/upload/images/20191005/20105988mKud6Ejzqe.png](https://ithelp.ithome.com.tw/upload/images/20191005/20105988mKud6Ejzqe.png) 1229 | 1230 | 1231 | 1232 | 　 1233 | 　 1234 | ## 11.錯誤SQL字串拼接方式,會導致效率慢、內存洩漏 1235 | 1236 | 1237 | 了解實作邏輯後,接著延伸一個Dapper使用的重要觀念,`SQL字串`為緩存重要Key值之一,假如不同的SQL字串,Dapper會為此建立新的動態方法、緩存,所以使用不當情況下就算使用StringBuilder`也會造成效率慢、內存洩漏問題`。 1238 | 1239 | ![https://ithelp.ithome.com.tw/upload/images/20190916/20105988lAFBAWbhS6.png](https://ithelp.ithome.com.tw/upload/images/20190916/20105988lAFBAWbhS6.png) 1240 | 1241 | 至於為何要以SQL字串當其中一個關鍵Key,而不是單純使用Mapping類別的Handle,其中原因之一是跟`查詢欄位順序`有關,在前面有講到,Dapper使用`「結果反推程式碼」`方式建立動態方法,代表說順序跟資料都必須要是`固定`的,避免SQL Select欄位順序不一樣又使用同一組動態方法,會有`A欄位值給B屬性`錯值大問題。 1242 | 1243 | 最直接解決方式,對每個不同SQL字串建立不同的動態方法,並保存在不同的緩存。 1244 | 1245 | 舉例,以下代碼只是簡單的查詢動作,查看Dapper Cache數量卻達到999999個,如Gif動畫顯示 1246 | ```C# 1247 | using (var cn = new SqlConnection(@"connectionString")) 1248 | { 1249 | for (int i = 0; i < 999999; i++) 1250 | { 1251 | var guid = Guid.NewGuid(); 1252 | for (int i2 = 0; i2 < 2; i2++) 1253 | { 1254 | var result = cn.Query($"select '{guid}' ").First(); 1255 | } 1256 | } 1257 | } 1258 | ``` 1259 | 1260 | ![zeiAPVJ](https://i.loli.net/2019/10/03/R8piq1uN4UZWg6t.gif) 1261 | 1262 | 要避免此問題,只需要保持一個原則`重複利用SQL字串`,而最簡單方式就是`參數化`, 舉例 : 將上述代碼改成以下代碼,緩存數量降為`1`,達到重複利用目的 : 1263 | ```C# 1264 | using (var cn = new SqlConnection(@"connectionString")) 1265 | { 1266 | for (int i = 0; i < 999999; i++) 1267 | { 1268 | var guid = Guid.NewGuid(); 1269 | for (int i2 = 0; i2 < 2; i2++) 1270 | { 1271 | var result = cn.Query($"select @guid ",new { guid}).First(); 1272 | } 1273 | } 1274 | } 1275 | ``` 1276 | 1277 | ![4IR5M47](https://i.loli.net/2019/10/03/3ZhQa5dwAE9Cmty.gif) 1278 | 1279 | 　 1280 | 　 1281 | ## 12.Dapper SQL正確字串拼接方式 : Literal Replacement 1282 | 1283 | 1284 | 假如遇到必要拼接SQL字串需求的情況下,舉例 : 有時候值使用字串拼接會比不使用參數化效率好,特別是該欄位值只會有幾種`固定值`。 1285 | 1286 | 這時候Dapper可以使用`Literal Replacements`功能,使用方式 : 將要拼接的值字串以`{=屬性名稱}`取代,並將值保存在Parameter參數內,舉例 : 1287 | ```C# 1288 | void Main() 1289 | { 1290 | using (var cn = Connection) 1291 | { 1292 | var result = cn.Query("select N'暐翰' Name,26 Age,{=VipLevel} VipLevel", new User{ VipLevel = 1}).First(); 1293 | } 1294 | } 1295 | ``` 1296 | 1297 | #### 為什麼Literal Replacement可以避免緩存問題 1298 | 首先追蹤源碼GetCacheInfo下GetLiteralTokens方法,可以發現Dapper在建立`緩存之前`會抓取`SQL字串`內符合`{=變量名稱}`規格的資料。 1299 | 1300 | ```C# 1301 | private static readonly Regex literalTokens = new Regex(@"(? GetLiteralTokens(string sql) 1303 | { 1304 | if (string.IsNullOrEmpty(sql)) return LiteralToken.None; 1305 | if (!literalTokens.IsMatch(sql)) return LiteralToken.None; 1306 | 1307 | var matches = literalTokens.Matches(sql); 1308 | var found = new HashSet(StringComparer.Ordinal); 1309 | List list = new List(matches.Count); 1310 | foreach (Match match in matches) 1311 | { 1312 | string token = match.Value; 1313 | if (found.Add(match.Value)) 1314 | { 1315 | list.Add(new LiteralToken(token, match.Groups[1].Value)); 1316 | } 1317 | } 1318 | return list.Count == 0 ? LiteralToken.None : list; 1319 | } 1320 | ``` 1321 | 1322 | 接著在CreateParamInfoGenerator方法生成Parameter參數化動態方法,此段方法IL如下 : 1323 | ``` 1324 | IL_0000: ldarg.1 1325 | IL_0001: castclass <>f__AnonymousType1`1[System.Int32] 1326 | IL_0006: stloc.0 1327 | IL_0007: ldarg.0 1328 | IL_0008: callvirt System.Data.IDataParameterCollection get_Parameters()/System.Data.IDbCommand 1329 | IL_000d: pop 1330 | IL_000e: ldarg.0 1331 | IL_000f: ldarg.0 1332 | IL_0010: callvirt System.String get_CommandText()/System.Data.IDbCommand 1333 | IL_0015: ldstr "{=VipLevel}" 1334 | IL_001a: ldloc.0 1335 | IL_001b: callvirt Int32 get_VipLevel()/<>f__AnonymousType1`1[System.Int32] 1336 | IL_0020: stloc.1 1337 | IL_0021: ldloca.s V_1 1338 | 1339 | IL_0023: call System.Globalization.CultureInfo get_InvariantCulture()/System.Globalization.CultureInfo 1340 | IL_0028: call System.String ToString(System.IFormatProvider)/System.Int32 1341 | IL_002d: callvirt System.String Replace(System.String, System.String)/System.String 1342 | IL_0032: callvirt Void set_CommandText(System.String)/System.Data.IDbCommand 1343 | IL_0037: ret 1344 | ``` 1345 | 1346 | 接著再生成Mapping動態方法,要了解此段邏輯我這邊做一個模擬例子方便讀者理解 : 1347 | ```C# 1348 | public static class DbExtension 1349 | { 1350 | public static IEnumerable Query(this DbConnection cnn, string sql, User parameter) 1351 | { 1352 | using (var command = cnn.CreateCommand()) 1353 | { 1354 | command.CommandText = sql; 1355 | CommandLiteralReplace(command, parameter); 1356 | using (var reader = command.ExecuteReader()) 1357 | while (reader.Read()) 1358 | yield return Mapping(reader); 1359 | } 1360 | } 1361 | 1362 | private static void CommandLiteralReplace(IDbCommand cmd, User parameter) 1363 | { 1364 | cmd.CommandText = cmd.CommandText.Replace("{=VipLevel}", parameter.VipLevel.ToString(System.Globalization.CultureInfo.InvariantCulture)); 1365 | } 1366 | 1367 | private static User Mapping(IDataReader reader) 1368 | { 1369 | var user = new User(); 1370 | var value = default(object); 1371 | value = reader[0]; 1372 | if(!(value is System.DBNull)) 1373 | user.Name = (string)value; 1374 | value = reader[1]; 1375 | if (!(value is System.DBNull)) 1376 | user.Age = (int)value; 1377 | value = reader[2]; 1378 | if (!(value is System.DBNull)) 1379 | user.VipLevel = (int)value; 1380 | return user; 1381 | } 1382 | } 1383 | ``` 1384 | 1385 | 看完以上例子,可以發現Dapper Literal Replacements底層原理就是`字串取代`,同樣屬於字串拼接方式,為何可以避免緩存問題? 1386 | 1387 | 這是因為取代的時機點在SetParameter動態方法內,所以Cache的`SQL Key是沒有變動過的`,可以重複利用同樣的SQL字串、緩存。 1388 | 1389 | 也因為是字串取代方式,所以`只支持基本Value類別`,假如使用String類別系統會告知`The type String is not supported for SQL literals.`,避免SQL Injection問題。 1390 | 1391 | 1392 | 　 1393 | 　 1394 | ## 13.Query Multi Mapping 使用方式 1395 | 1396 | 1397 | 接著講解`Dapper Multi Mapping`(多對應)實作跟底層邏輯,畢竟工作當中不可能都是一對一概念。 1398 | 1399 | 使用方式 : 1400 | - 需要自己編寫Mapping邏輯,使用方式 : `Query(SQL,Parameter,Mapping邏輯Func)` 1401 | - 需要指定泛型參數類別,規則為`Query` (最多支持六組泛型參數) 1402 | - 指定切割欄位名稱,預設使用`ID`,假如不一樣需要特別指定 (這段後面特別講解) 1403 | - 以上順序都是`由左至右` 1404 | 1405 | 舉例 : 有訂單(Order)跟會員(User)表格,關係是一對多關係,一個會員可以有多個訂單,以下是C# Demo代碼 : 1406 | ```C# 1407 | void Main() 1408 | { 1409 | using (var ts = new TransactionScope()) 1410 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 1411 | { 1412 | cn.Execute(@" 1413 | CREATE TABLE [User]([ID] int, [Name] nvarchar(10)); 1414 | INSERT INTO [User]([ID], [Name])VALUES(1, N'大雄'),(2, N'小明'); 1415 | 1416 | CREATE TABLE [Order]([ID] int, [OrderNo] varchar(13), [UserID] int); 1417 | INSERT INTO [Order]([ID], [OrderNo], [UserID])VALUES(1, 'SO20190900001', 1),(2, 'SO20190900002', 1),(3, 'SO20190900003', 2),(4, 'SO20190900004', 2); 1418 | "); 1419 | 1420 | var result = cn.Query(@" 1421 | select * from [order] T1 1422 | left join [User] T2 on T1.UserId = T2.ID 1423 | ", (order, user) => { 1424 | order.User = user; 1425 | return order; 1426 | } 1427 | ); 1428 | 1429 | ts.Dispose(); 1430 | } 1431 | } 1432 | 1433 | public class Order 1434 | { 1435 | public int ID { get; set; } 1436 | public string OrderNo { get; set; } 1437 | public User User { get; set; } 1438 | } 1439 | 1440 | public class User 1441 | { 1442 | public int ID { get; set; } 1443 | public string Name { get; set; } 1444 | } 1445 | ``` 1446 | 1447 | ![20191001145311.png](https://i.loli.net/2019/10/01/vEXgOINbojLs6yT.png) 1448 | 1449 | 1450 | ---- 1451 | 1452 | #### 支援dynamic Multi Mapping 1453 | 在初期常變動表格結構或是一次性功能不想宣告Class,Dapper Multi Mapping也支援dynamic方式 1454 | 1455 | ```C# 1456 | void Main() 1457 | { 1458 | using (var ts = new TransactionScope()) 1459 | using (var connection = Connection) 1460 | { 1461 | const string createSql = @" 1462 | create table Users (Id int, Name nvarchar(20)) 1463 | create table Posts (Id int, OwnerId int, Content nvarchar(20)) 1464 | 1465 | insert Users values(1, N'小明') 1466 | insert Users values(2, N'小智') 1467 | 1468 | insert Posts values(101, 1, N'小明第1天日記') 1469 | insert Posts values(102, 1, N'小明第2天日記') 1470 | insert Posts values(103, 2, N'小智第1天日記') 1471 | "; 1472 | connection.Execute(createSql); 1473 | 1474 | const string sql = 1475 | @"select * from Posts p 1476 | left join Users u on u.Id = p.OwnerId 1477 | Order by p.Id 1478 | "; 1479 | 1480 | var data = connection.Query(sql, (post, user) => { post.Owner = user; return post; }).ToList(); 1481 | } 1482 | } 1483 | ``` 1484 | 1485 | ![20191002023135.png](https://i.loli.net/2019/10/02/cOP2mrlRTGgVit1.png) 1486 | 1487 | 1488 | ### SplitOn區分類別Mapping組別 1489 | 1490 | Split預設是用來切割主鍵,所以預設切割字串是`Id`,假如當表格結構PK名稱為`Id`可以省略參數,舉例 1491 | ![20191001151715.png](https://i.loli.net/2019/10/01/4UzltBks7ae3TDA.png) 1492 | ```C# 1493 | var result = cn.Query(@" 1494 | select * from [order] T1 1495 | left join [User] T2 on T1.UserId = T2.ID 1496 | ", (order, user) => { 1497 | order.User = user; 1498 | return order; 1499 | } 1500 | ); 1501 | ``` 1502 | 1503 | 1504 | 假如主鍵名稱是其他名稱,`請指定splitOn字串名稱`,並且對應多個可以使用`,`做區隔,舉例,添加商品表格做Join : 1505 | ``` 1506 | var result = cn.Query(@" 1507 | select * from [order] T1 1508 | left join [User] T2 on T1.UserId = T2.ID 1509 | left join [Item] T3 on T1.ItemId = T3.ID 1510 | " 1511 | 1512 | ,map : (order, user,item) => { 1513 | order.User = user; 1514 | order.Item = item; 1515 | return order; 1516 | } 1517 | ,splitOn : "Id,Id" 1518 | ); 1519 | ``` 1520 | 1521 | 　 1522 | 　 1523 | ## 14.Query Multi Mapping 底層原理 1524 | 1525 | 1526 | ### Multiple Mapping 底層原理 1527 | 1528 | 這邊先以一個簡單Demo帶讀者了解Dapper Multi Mapping 概念 1529 | 1. 按照泛型類別參數數量建立對應數量的Mapping Func集合 1530 | 2. Mapping Func建立邏輯跟Query Emit IL一樣 1531 | 3. 呼叫使用者的Custom Mapping Func,其中參數由前面動態生成的Mapping Func而來 1532 | 1533 | ```C# 1534 | public static class MutipleMappingDemo 1535 | { 1536 | public static IEnumerable Query(this IDbConnection connection, string sql, Func map) 1537 | where T1 : Order, new() 1538 | where T2 : User, new() //這兩段where單純為了Demo方便 1539 | { 1540 | //1. 按照泛型類別參數數量建立對應數量的Mapping Func集合 1541 | var deserializers = new List>(); 1542 | { 1543 | //2. Mapping Func建立邏輯跟Query Emit IL一樣 1544 | deserializers.Add((reader) => 1545 | { 1546 | var newObj = new T1(); 1547 | var value = default(object); 1548 | value = reader[0]; 1549 | newObj.ID = value is DBNull ? 0 : (int)value; 1550 | value = reader[1]; 1551 | newObj.OrderNo = value is DBNull ? null : (string)value; 1552 | return newObj; 1553 | }); 1554 | 1555 | deserializers.Add((reader) => 1556 | { 1557 | var newObj = new T2(); 1558 | var value = default(object); 1559 | value = reader[2]; 1560 | newObj.ID = value is DBNull ? 0 : (int)value; 1561 | value = reader[4]; 1562 | newObj.Name = value is DBNull ? null : (string)value; 1563 | return newObj; 1564 | }); 1565 | } 1566 | 1567 | 1568 | using (var command = connection.CreateCommand()) 1569 | { 1570 | command.CommandText = sql; 1571 | using (var reader = command.ExecuteReader()) 1572 | { 1573 | while (reader.Read()) 1574 | { 1575 | //3. 呼叫使用者的Custom Mapping Func,其中參數由前面動態生成的Mapping Func而來 1576 | yield return map(deserializers[0](reader) as T1, deserializers[1](reader) as T2); 1577 | } 1578 | } 1579 | } 1580 | } 1581 | } 1582 | ``` 1583 | 1584 | 以上概念就是此方法的主要邏輯,接著講其他細節部分 1585 | 1586 | #### 支持多組類別 + 強型別返回值 1587 | 1588 | Dapper為了`強型別多類別Mapping`使用`多組泛型參數方法`方式,這方式有個小缺點就是`沒辦法動態調整`,需要以寫死方式來處理。 1589 | 1590 | 舉例,可以看到圖片GenerateMapper方法,依照泛型參數數量,寫死強轉型邏輯,這也是為何Multiple Query有最大組數限制,只能支持最多6組的原因。 1591 | ![20191001173320.png](https://i.loli.net/2019/10/01/3BKcPCzdnMeGRV6.png) 1592 | 1593 | #### 多類別泛型緩存算法 1594 | - 這邊Dapper使用`泛型類別`來`強型別`保存多類別的資料 1595 | ![20191001175139.png](https://i.loli.net/2019/10/01/5eSfkoaQiIPXvz4.png) 1596 | - 並配合繼承共用Identity大部分身分驗證邏輯 1597 | - 提供可`override`的GetType方法,來客製泛型比較邏輯,避免造成跟Non Multi Query`緩存衝突`。 1598 | 1599 | ![20191001175600.png](https://i.loli.net/2019/10/01/ypEj12lzDFVsMuo.png) 1600 | ![20191001175707.png](https://i.loli.net/2019/10/01/nIbHarVcyNxfpqj.png) 1601 | 1602 | 1603 | #### Dapper Query Multi Mapping的Select順序很重要 1604 | 因為SplitOn`分組基礎依賴於Select的順序`,所以順序一錯就有可能`屬性值錯亂`情況。 1605 | 1606 | 舉例 : 假如上面例子的SQL改成以下,會發生User的ID變成Order的ID;Order的ID會變成User的ID。 1607 | ```sql 1608 | select T2.[ID],T1.[OrderNo],T1.[UserID],T1.[ID],T2.[Name] from [order] T1 1609 | left join [User] T2 on T1.UserId = T2.ID 1610 | ``` 1611 | 1612 | 原因可以追究到Dapper的切割算法 1613 | 1. 首先`倒序`方式處理欄位分組(GetNextSplit方法可以看到從DataReader Index`大到小`查詢) 1614 | ![20191002022109.png](https://i.loli.net/2019/10/02/p6mzVJEGZtL84C7.png) 1615 | 2. 接著`倒序`方式處理類別的Mapping Emit IL Func 1616 | 3. 最後反轉為`正序`,方便後面Call Func對應泛型使用 1617 | ![20191002021750.png](https://i.loli.net/2019/10/02/RjEizNoSFBgHODr.png) 1618 | ![20191002022208.png](https://i.loli.net/2019/10/02/Qu1MENwDGWi9cCS.png) 1619 | ![20191002022214.png](https://i.loli.net/2019/10/02/IBWfdvzLwbqX1GP.png) 1620 | 1621 | 1622 | 　 1623 | 　 1624 | ## 15.QueryMultiple 底層原理 1625 | 1626 | 1627 | 使用方式例子 : 1628 | ```C# 1629 | using (var cn = Connection) 1630 | { 1631 | using (var gridReader = cn.QueryMultiple("select 1; select 2;")) 1632 | { 1633 | Console.WriteLine(gridReader.Read()); //result : 1 1634 | Console.WriteLine(gridReader.Read()); //result : 2 1635 | } 1636 | } 1637 | ``` 1638 | 1639 | 使用QueryMultiple優點 : 1640 | - 主要`減少Reqeust次數` 1641 | - 可以將多個查詢`共用同一組Parameter參數` 1642 | 1643 | QueryMultiple的底層實作邏輯 : 1644 | 1. 底層技術是ADO.NET - DataReader - MultipleResult 1645 | 2. QueryMultiple取得DataReader並封裝進GridReader 1646 | 3. 呼叫Read方法時才會建立Mapping動態方法,Emit IL動作跟Query方法一樣 1647 | 4. 接著使用ADO.NET技術呼叫`DataReader NextResult`取得下一組查詢結果 1648 | 5. 假如`沒有`下一組查詢結果才會將`DataReader釋放` 1649 | 1650 | ---- 1651 | 1652 | 1653 | #### 緩存算法 1654 | 緩存的算法多增加gridIndex判斷,主要對每個result mapping動作做一個緩存,Emit IL的邏輯跟Query一樣。 1655 | 1656 | ![20190930183038.png](https://i.loli.net/2019/09/30/b1aKNx2qQ5lSufF.png) 1657 | 1658 | #### 沒有延遲查詢特性 1659 | 注意Read方法使用的是buffer = true = 返回結果直接ToList保存在內存,所以沒有延遲查詢特性。 1660 | 1661 | ![20190930183212.png](https://i.loli.net/2019/09/30/mIVDex8rqk5Z46E.png) 1662 | ![20190930183219.png](https://i.loli.net/2019/09/30/CdqjTZyI1BnuDzi.png) 1663 | 1664 | #### 記得管理DataReader的釋放 1665 | Dapper 呼叫QueryMultiple方法時會將DataReader封裝在GridReader物件內,只有當`最後一次Read`動作後才會回收DataReader 1666 | 1667 | ![20190930183447.png](https://i.loli.net/2019/09/30/eB9Z5dgCtF87PRI.png) 1668 | 1669 | 所以`沒有讀取完`再開一個GridReader > Read會出現錯誤:`已經開啟一個與這個 Command 相關的 DataReader，必須先將它關閉`。 1670 | 1671 | ![20190930183532.png](https://i.loli.net/2019/09/30/ByrTuOf6jzD7w21.png) 1672 | 1673 | 要避免以上情況,可以改成`using`區塊方式,運行完區塊代碼後就會自動釋放DataReader 1674 | ```C# 1675 | using (var gridReader = cn.QueryMultiple("select 1; select 2;")) 1676 | { 1677 | //略.. 1678 | } 1679 | ``` 1680 | 1681 | #### 閒話 : 1682 | 感覺Dapper GridReader好像有機會可以實作`是否有NextResult`方法,這樣就可以`配合while`方法`一次讀取完多組查詢資料`,等之後有空來想想有沒有機會做成。 1683 | 1684 | 概念代碼 : 1685 | ```C# 1686 | public static class DbExtension 1687 | { 1688 | public static IEnumerable> GetMultipleResult(this IDbConnection cn,string sql, object paramters) 1689 | { 1690 | using (var reader = cn.QueryMultiple(sql,paramters)) 1691 | { 1692 | while(reader.NextResult()) 1693 | { 1694 | yield return reader.Read(); 1695 | } 1696 | } 1697 | } 1698 | } 1699 | ``` 1700 | 1701 | 　 1702 | 　 1703 | ## 16.TypeHandler 自訂Mapping邏輯使用、底層邏輯 1704 | 1705 | 1706 | 遇到想要客製某些屬性Mapping邏輯時,在Dapper可以使用`TypeHandler` 1707 | 1708 | 使用方式 : 1709 | - 建立類別繼承`SqlMapper.TypeHandler` 1710 | - 將要客製的類別指定給`泛型`,e.g : `JsonTypeHandler<客製類別> : SqlMapper.TypeHandler<客製類別> ` 1711 | - `查詢`的邏輯使用override實作`Parse`方法,`增刪改`邏輯實作`SetValue`方法 1712 | - 假如多個類別Parse、SetValue共用同樣邏輯,可以將實作類別改為`泛型`方式,客製類別在`AddTypeHandler`時指定就可以,可以避免建立一堆類別,e.g : `JsonTypeHandler : SqlMapper.TypeHandler where T : class` 1713 | 1714 | 舉例 : 1715 | 想要特定屬性成員在數據庫保存Json,在AP端自動轉成對應Class類別,這時候可以使用`SqlMapper.AddTypeHandler<繼承實作TypeHandler的類別>`。 1716 | 1717 | 以下例子是User資料變更時會自動在Log欄位紀錄變更動作。 1718 | ```C# 1719 | public class JsonTypeHandler : SqlMapper.TypeHandler 1720 | where T : class 1721 | { 1722 | public override T Parse(object value) 1723 | { 1724 | return JsonConvert.DeserializeObject((string)value); 1725 | } 1726 | 1727 | public override void SetValue(IDbDataParameter parameter, T value) 1728 | { 1729 | parameter.Value = JsonConvert.SerializeObject(value); 1730 | } 1731 | } 1732 | 1733 | public void Main() 1734 | { 1735 | SqlMapper.AddTypeHandler(new JsonTypeHandler>()); 1736 | 1737 | using (var ts = new TransactionScope()) 1738 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=master;")) 1739 | { 1740 | 1741 | cn.Execute("create table [User] (Name nvarchar(200),Age int,Level int,Logs nvarchar(max))"); 1742 | 1743 | var user = new User() 1744 | { 1745 | Name = "暐翰", 1746 | Age = 26, 1747 | Level = 1, 1748 | Logs = new List() { 1749 | new Log(){Time=DateTime.Now,Remark="CreateUser"} 1750 | } 1751 | }; 1752 | 1753 | //新增資料 1754 | { 1755 | cn.Execute("insert into [User] (Name,Age,Level,Logs) values (@Name,@Age,@Level,@Logs);", user); 1756 | 1757 | var result = cn.Query("select * from [User]"); 1758 | Console.WriteLine(result); 1759 | } 1760 | 1761 | //升級Level動作 1762 | { 1763 | user.Level = 9; 1764 | user.Logs.Add(new Log() {Remark="UpdateLevel"}); 1765 | cn.Execute("update [User] set Level = @Level,Logs = @Logs where Name = @Name", user); 1766 | var result = cn.Query("select * from [User]"); 1767 | Console.WriteLine(result); 1768 | } 1769 | 1770 | ts.Dispose(); 1771 | 1772 | } 1773 | } 1774 | 1775 | public class User 1776 | { 1777 | public string Name { get; set; } 1778 | public int Age { get; set; } 1779 | public int Level { get; set; } 1780 | public List Logs { get; set; } 1781 | 1782 | } 1783 | public class Log 1784 | { 1785 | public DateTime Time { get; set; } = DateTime.Now; 1786 | public string Remark { get; set; } 1787 | } 1788 | ``` 1789 | 1790 | 效果圖 : 1791 | ![20190929231937.png](https://i.loli.net/2019/09/29/F7xWKHPaSGuo3kB.png) 1792 | 1793 | ---- 1794 | 1795 | 1796 | 接著追蹤TypeHandler源碼邏輯,需要分兩個部份來追蹤 : SetValue,Parse 1797 | 1798 | ### SetValue底層原理 1799 | 1. AddTypeHandlerImpl方法管理緩存的添加 1800 | 2. 在CreateParamInfoGenerator方法Emit建立動態AddParameter方法時,假如該Mapping類別TypeHandler緩存內有資料,Emit添加呼叫SetValue方法動作。 1801 | ```C# 1802 | if (handler != null) 1803 | { 1804 | il.Emit(OpCodes.Call, typeof(TypeHandlerCache<>).MakeGenericType(prop.PropertyType).GetMethod(nameof(TypeHandlerCache.SetValue))); // stack is now [parameters] [[parameters]] [parameter] 1805 | } 1806 | ``` 1807 | 3. 在Runtime呼叫AddParameters方法時會使用LookupDbType,判斷是否有自訂TypeHandler 1808 | ![20191006151723.png](https://i.loli.net/2019/10/06/Jq5gomaEnb7kRTO.png) 1809 | ![20191006151614.png](https://i.loli.net/2019/10/06/kjlbsXBWAFoJ72I.png) 1810 | 4. 接著將建立好的Parameter傳給自訂TypeHandler.SetValue方法 1811 | ![20191006151901.png](https://i.loli.net/2019/10/06/xehnlaP4NDJ6wLg.png) 1812 | 1813 | 最後查看IL轉成的C#代碼 1814 | ```C# 1815 | public static void TestMeThod(IDbCommand P_0, object P_1) 1816 | { 1817 | User user = (User)P_1; 1818 | IDataParameterCollection parameters = P_0.Parameters; 1819 | //略... 1820 | IDbDataParameter dbDataParameter3 = P_0.CreateParameter(); 1821 | dbDataParameter3.ParameterName = "Logs"; 1822 | dbDataParameter3.Direction = ParameterDirection.Input; 1823 | SqlMapper.TypeHandlerCache>.SetValue(dbDataParameter3, ((object)user.Logs) ?? ((object)DBNull.Value)); 1824 | parameters.Add(dbDataParameter3); 1825 | //略... 1826 | } 1827 | ``` 1828 | 1829 | 可以發現生成的Emit IL會去從TypeHandlerCache取得我們實作的TypeHandler,接著`呼叫實作SetValue方法`運行設定的邏輯,並且TypeHandlerCache特別使用`泛型類別`依照不同泛型以`Singleton`方式保存不同handler,這樣有以下優點 : 1830 | 1. 只要傳遞泛型類別參數就可以取得同一個handler`避免重複建立物件` 1831 | 2. 因為是泛型類別,取handler時可以避免了反射動作,`提升效率` 1832 | 1833 | ![https://ithelp.ithome.com.tw/upload/images/20190929/20105988x970H6xWXC.png](https://ithelp.ithome.com.tw/upload/images/20190929/20105988x970H6xWXC.png) 1834 | ![https://ithelp.ithome.com.tw/upload/images/20190929/20105988S7VZLLXLZo.png](https://ithelp.ithome.com.tw/upload/images/20190929/20105988S7VZLLXLZo.png) 1835 | ![https://ithelp.ithome.com.tw/upload/images/20190929/20105988Q1mWkL0GP6.png](https://ithelp.ithome.com.tw/upload/images/20190929/20105988Q1mWkL0GP6.png) 1836 | 1837 | 1838 | ---- 1839 | 1840 | ### Parse對應底層原理 1841 | 1842 | 主要邏輯是在GenerateDeserializerFromMap方法Emit建立動態Mapping方法時,假如判斷TypeHandler緩存有資料,以Parse方法取代原本的Set屬性動作。 1843 | ![https://ithelp.ithome.com.tw/upload/images/20190930/20105988JvCw5z207s.png](https://ithelp.ithome.com.tw/upload/images/20190930/20105988JvCw5z207s.png) 1844 | 1845 | 查看動態Mapping方法生成的IL代碼 : 1846 | ``` 1847 | IL_0000: ldc.i4.0 1848 | IL_0001: stloc.0 1849 | IL_0002: newobj Void .ctor()/Demo.User 1850 | IL_0007: stloc.1 1851 | IL_0008: ldloc.1 1852 | IL_0009: dup 1853 | IL_000a: ldc.i4.0 1854 | IL_000b: stloc.0 1855 | IL_000c: ldarg.0 1856 | IL_000d: ldc.i4.0 1857 | IL_000e: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1858 | IL_0013: dup 1859 | IL_0014: stloc.2 1860 | IL_0015: dup 1861 | IL_0016: isinst System.DBNull 1862 | IL_001b: brtrue.s IL_0029 1863 | IL_001d: unbox.any System.String 1864 | IL_0022: callvirt Void set_Name(System.String)/Demo.User 1865 | IL_0027: br.s IL_002b 1866 | IL_0029: pop 1867 | IL_002a: pop 1868 | IL_002b: dup 1869 | IL_002c: ldc.i4.1 1870 | IL_002d: stloc.0 1871 | IL_002e: ldarg.0 1872 | IL_002f: ldc.i4.1 1873 | IL_0030: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1874 | IL_0035: dup 1875 | IL_0036: stloc.2 1876 | IL_0037: dup 1877 | IL_0038: isinst System.DBNull 1878 | IL_003d: brtrue.s IL_004b 1879 | IL_003f: unbox.any System.Int32 1880 | IL_0044: callvirt Void set_Age(Int32)/Demo.User 1881 | IL_0049: br.s IL_004d 1882 | IL_004b: pop 1883 | IL_004c: pop 1884 | IL_004d: dup 1885 | IL_004e: ldc.i4.2 1886 | IL_004f: stloc.0 1887 | IL_0050: ldarg.0 1888 | IL_0051: ldc.i4.2 1889 | IL_0052: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1890 | IL_0057: dup 1891 | IL_0058: stloc.2 1892 | IL_0059: dup 1893 | IL_005a: isinst System.DBNull 1894 | IL_005f: brtrue.s IL_006d 1895 | IL_0061: unbox.any System.Int32 1896 | IL_0066: callvirt Void set_Level(Int32)/Demo.User 1897 | IL_006b: br.s IL_006f 1898 | IL_006d: pop 1899 | IL_006e: pop 1900 | IL_006f: dup 1901 | IL_0070: ldc.i4.3 1902 | IL_0071: stloc.0 1903 | IL_0072: ldarg.0 1904 | IL_0073: ldc.i4.3 1905 | IL_0074: callvirt System.Object get_Item(Int32)/System.Data.IDataRecord 1906 | IL_0079: dup 1907 | IL_007a: stloc.2 1908 | IL_007b: dup 1909 | IL_007c: isinst System.DBNull 1910 | IL_0081: brtrue.s IL_008f 1911 | IL_0083: call System.Collections.Generic.List`1[Demo.Log] Parse(System.Object)/Dapper.SqlMapper+TypeHandlerCache`1[System.Collections.Generic.List`1[Demo.Log]] 1912 | IL_0088: callvirt Void set_Logs(System.Collections.Generic.List`1[Demo.Log])/Demo.User 1913 | IL_008d: br.s IL_0091 1914 | IL_008f: pop 1915 | IL_0090: pop 1916 | IL_0091: stloc.1 1917 | IL_0092: leave IL_00a4 1918 | IL_0097: ldloc.0 1919 | IL_0098: ldarg.0 1920 | IL_0099: ldloc.2 1921 | IL_009a: call Void ThrowDataException(System.Exception, Int32, System.Data.IDataReader, System.Object)/Dapper.SqlMapper 1922 | IL_009f: leave IL_00a4 1923 | IL_00a4: ldloc.1 1924 | IL_00a5: ret 1925 | ``` 1926 | 1927 | 轉成C#代碼來驗證 : 1928 | ```C# 1929 | public static User TestMeThod(IDataReader P_0) 1930 | { 1931 | int index = 0; 1932 | User user = new User(); 1933 | object value = default(object); 1934 | try 1935 | { 1936 | User user2 = user; 1937 | index = 0; 1938 | object obj = value = P_0[0]; 1939 | //..略 1940 | index = 3; 1941 | object obj4 = value = P_0[3]; 1942 | if (!(obj4 is DBNull)) 1943 | { 1944 | user2.Logs = SqlMapper.TypeHandlerCache>.Parse(obj4); 1945 | } 1946 | user = user2; 1947 | return user; 1948 | } 1949 | catch (Exception ex) 1950 | { 1951 | SqlMapper.ThrowDataException(ex, index, P_0, value); 1952 | return user; 1953 | } 1954 | } 1955 | ``` 1956 | 1957 | 　 1958 | 　 1959 | ## 17. CommandBehavior的細節處理 1960 | 1961 | 1962 | 這篇將帶讀者了解Dapper如何在底層利用CommandBehavior優化查詢效率,如何選擇正確Behavior在特定時機。 1963 | 1964 | 我這邊整理了各方法對應的Behavior表格 : 1965 | 1966 | |方法|Behavior| 1967 | |-|-| 1968 | |Query | CommandBehavior.SequentialAccess & CommandBehavior.SingleResult | 1969 | |QueryFirst | CommandBehavior.SequentialAccess & CommandBehavior.SingleResult & CommandBehavior.SingleRow | 1970 | |QueryFirstOrDefault | CommandBehavior.SequentialAccess & CommandBehavior.SingleResult & CommandBehavior.SingleRow | 1971 | |QuerySingle | CommandBehavior.SingleResult & CommandBehavior.SequentialAccess | 1972 | |QuerySingleOrDefault | CommandBehavior.SingleResult & CommandBehavior.SequentialAccess | 1973 | |QueryMultiple|CommandBehavior.SequentialAccess| 1974 | 1975 | --- 1976 | 1977 | #### SequentialAccess、SingleResult優化邏輯 1978 | 1979 | 首先可以看到每個方法都使用`CommandBehavior.SequentialAccess`,該標籤主要功能 `使DataReader順序讀取行和列,行和列不緩衝,讀取一列後,它會從內存中刪除。`,有以下優點 : 1980 | 1. 可按順序分次讀取資源,`避免二進制大資源一次性讀取到內存`,尤其是Blob或是Clob會配合GetBytes 或 GetChars 方法限制緩衝區大小,微軟官方也特別標註注意 : 1981 | ![20191003014421.png](https://i.loli.net/2019/10/03/PVSuO8xwfhzgaFH.png) 1982 | 1. 實際環境測試,可以`加快查詢效率` 1983 | 1984 | 但它卻`不是`DataReader的預設行為,系統預設是`CommandBehavior.Default` 1985 | ![20191003015853.png](https://i.loli.net/2019/10/03/23Zs87l9aGCnR4E.png) 1986 | CommandBehavior.Default有著以下特性 : 1987 | 1. 可傳回`多個`結果集(Multi Result) 1988 | 2. 一次性讀取行資料到內存 1989 | 1990 | 這兩個特性跟生產環境情況差滿多,畢竟大多時刻是`只需要一組結果集配合有限的內存`,所以除了SequentialAccess外Dapper還特別在大多方法使用了`CommandBehavior.SingleResult`,滿足只需一組結果就好避免浪費資源。 1991 | 1992 | 這段還有一段細節的處理,查看源碼可以發現除了標記SingleResult外,Dapper還特別加上一段代碼在結尾`while (reader.NextResult()){}`,而不是直接Return(如圖片) 1993 | 1994 | ![20191003021109.png](https://i.loli.net/2019/10/03/2VSKACmruvobanI.png) 1995 | 1996 | 早些前我有特別發Issue([連結#1210](https://github.com/StackExchange/Dapper/issues/1210))詢問過作者,這邊是回答 : `主要避免忽略錯誤,像是在DataReader提早關閉情況` 1997 | 1998 | --- 1999 | 2000 | #### QueryFirst搭配SingleRow, 2001 | 2002 | 有時候我們會遇到`select top 1`知道只會讀取一行資料的情況,這時候可以使用`QueryFirst`。它使用`CommandBehavior.SingleRow`可以避免浪費資源只讀取一行資料。 2003 | 2004 | 另外可以發現此段除了`while (reader.NextResult()){}`外還有`while (reader.Read()) {}`,同樣是避免忽略錯誤,這是一些公司自行土炮ORM會忽略的地方。 2005 | ![20191003024206.png](https://i.loli.net/2019/10/03/PRsrNoeJOInKB8u.png) 2006 | 2007 | ### 與QuerySingle之間的差別 2008 | 兩者差別在QuerySingle沒有使用CommandBehavior.SingleRow,至於為何沒有使用,是因為需要有`多行資料才能判斷是否不符合條件並拋出Exception告知使用者`。 2009 | 2010 | 這段有一個特別好玩小技巧可以學,錯誤處理直接沿用對應LINQ的Exception,舉例:超過一行資料錯誤,使用`new int[2].Single()`,這樣不用另外維護Exceptiono類別,還可以擁有i18N多國語言化。 2011 | ![20191003025631.png](https://i.loli.net/2019/10/03/S38mD9LElTgUYwd.png) 2012 | ![20191003025334.png](https://i.loli.net/2019/10/03/9vMXH8D1muWdYzQ.png) 2013 | 2014 | 2015 | 2016 | 　 2017 | 　 2018 | ## 18.Parameter 參數化底層原理 2019 | 2020 | 2021 | 2022 | 接著進到Dapper的另一個關鍵功能 : 「Parameter 參數化」 2023 | 2024 | 主要邏輯 : 2025 | GetCacheInfo檢查是否緩存內有動態方法 > 假如沒有緩存,使用CreateParamInfoGenerator方法Emit IL建立AddParameter動態方法 > 建立完後保存在緩存內 2026 | 2027 | 接著重點來看CreateParamInfoGenerator方法內的底成邏輯跟「精美細節處理」,使用了結果反推代碼方法,忽略`「沒使用的欄位」`不生成對應IL代碼,避免資源浪費情況。這也是前面緩存算法要去判斷不同SQL字串的原因。 2028 | 2029 | 以下是我挑出的源碼重點部分 : 2030 | ```C# 2031 | internal static Action CreateParamInfoGenerator(Identity identity, bool checkForDuplicates, bool removeUnused, IList literals) 2032 | { 2033 | //...略 2034 | if (filterParams) 2035 | { 2036 | props = FilterParameters(props, identity.sql); 2037 | } 2038 | 2039 | var callOpCode = isStruct ? OpCodes.Call : OpCodes.Callvirt; 2040 | foreach (var prop in props) 2041 | { 2042 | //Emit IL動作 2043 | } 2044 | //...略 2045 | } 2046 | 2047 | 2048 | private static IEnumerable FilterParameters(IEnumerable parameters, string sql) 2049 | { 2050 | var list = new List(16); 2051 | foreach (var p in parameters) 2052 | { 2053 | if (Regex.IsMatch(sql, @"[?@:]" + p.Name + @"([^\p{L}\p{N}_]+|$)", RegexOptions.IgnoreCase | RegexOptions.Multiline | RegexOptions.CultureInvariant)) 2054 | list.Add(p); 2055 | } 2056 | return list; 2057 | } 2058 | ``` 2059 | 2060 | 接著查看IL來驗證,查詢代碼如下 2061 | ```C# 2062 | var result = connection.Query("select @Name name ", new { Name = "暐翰", Age = 26}).First(); 2063 | ``` 2064 | 2065 | CreateParamInfoGenerator AddParameter 動態方法IL代碼如下 : 2066 | ```C# 2067 | IL_0000: ldarg.1 2068 | IL_0001: castclass <>f__AnonymousType1`2[System.String,System.Int32] 2069 | IL_0006: stloc.0 2070 | IL_0007: ldarg.0 2071 | IL_0008: callvirt System.Data.IDataParameterCollection get_Parameters()/System.Data.IDbCommand 2072 | IL_000d: dup 2073 | IL_000e: ldarg.0 2074 | IL_000f: callvirt System.Data.IDbDataParameter CreateParameter()/System.Data.IDbCommand 2075 | IL_0014: dup 2076 | IL_0015: ldstr "Name" 2077 | IL_001a: callvirt Void set_ParameterName(System.String)/System.Data.IDataParameter 2078 | IL_001f: dup 2079 | IL_0020: ldc.i4.s 16 2080 | IL_0022: callvirt Void set_DbType(System.Data.DbType)/System.Data.IDataParameter 2081 | IL_0027: dup 2082 | IL_0028: ldc.i4.1 2083 | IL_0029: callvirt Void set_Direction(System.Data.ParameterDirection)/System.Data.IDataParameter 2084 | IL_002e: dup 2085 | IL_002f: ldloc.0 2086 | IL_0030: callvirt System.String get_Name()/<>f__AnonymousType1`2[System.String,System.Int32] 2087 | IL_0035: dup 2088 | IL_0036: brtrue.s IL_0042 2089 | IL_0038: pop 2090 | IL_0039: ldsfld System.DBNull Value/System.DBNull 2091 | IL_003e: ldc.i4.0 2092 | IL_003f: stloc.1 2093 | IL_0040: br.s IL_005a 2094 | IL_0042: dup 2095 | IL_0043: callvirt Int32 get_Length()/System.String 2096 | IL_0048: ldc.i4 4000 2097 | IL_004d: cgt 2098 | IL_004f: brtrue.s IL_0058 2099 | IL_0051: ldc.i4 4000 2100 | IL_0056: br.s IL_0059 2101 | IL_0058: ldc.i4.m1 2102 | IL_0059: stloc.1 2103 | IL_005a: callvirt Void set_Value(System.Object)/System.Data.IDataParameter 2104 | IL_005f: ldloc.1 2105 | IL_0060: brfalse.s IL_0069 2106 | IL_0062: dup 2107 | IL_0063: ldloc.1 2108 | IL_0064: callvirt Void set_Size(Int32)/System.Data.IDbDataParameter 2109 | IL_0069: callvirt Int32 Add(System.Object)/System.Collections.IList 2110 | IL_006e: pop 2111 | IL_006f: pop 2112 | IL_0070: ret 2113 | ``` 2114 | 2115 | IL轉成對應C#代碼: 2116 | ```C# 2117 | public class TestType 2118 | { 2119 | public static void TestMeThod(IDataReader P_0, object P_1) 2120 | { 2121 | var anon = (<>f__AnonymousType1)P_1; 2122 | IDataParameterCollection parameters = ((IDbCommand)P_0).Parameters; 2123 | IDbDataParameter dbDataParameter = ((IDbCommand)P_0).CreateParameter(); 2124 | dbDataParameter.ParameterName = "Name"; 2125 | dbDataParameter.DbType = DbType.String; 2126 | dbDataParameter.Direction = ParameterDirection.Input; 2127 | object obj = anon.Name; 2128 | int num; 2129 | if (obj == null) 2130 | { 2131 | obj = DBNull.Value; 2132 | num = 0; 2133 | } 2134 | else 2135 | { 2136 | num = ((((string)obj).Length > 4000) ? (-1) : 4000); 2137 | } 2138 | dbDataParameter.Value = obj; 2139 | if (num != 0) 2140 | { 2141 | dbDataParameter.Size = num; 2142 | } 2143 | parameters.Add(dbDataParameter); 2144 | } 2145 | } 2146 | ``` 2147 | 2148 | 可以發現雖然傳遞Age參數,但是SQL字串沒有用到,Dapper不會去生成該欄位的SetParameter動作IL。這個細節處理真的要給Dapper一個讚! 2149 | 2150 | 　 2151 | 　 2152 | ## 19. IN 多集合參數化底層原理 2153 | 2154 | 2155 | 2156 | #### 為何ADO.NET不支援`IN 參數化`,Dapper支援 ? 2157 | 原理 2158 | 1. 判斷參數的屬性是否為IEnumerable類別子類別 2159 | 2. 假如是,以該參數名稱為主 + Parameter正則格式找尋SQL內的參數字串 (正則格式 : `([?@:]參數名)(?!\w)(\s+(?i)unknown(?-i))?`) 2160 | 3. 將找到的字串以`()` + 多個`屬性名稱+流水號`方式替換 2161 | 4. 依照流水號順序依序CreateParameter > SetValue 2162 | 2163 | 關鍵程式部分 2164 | ![https://ithelp.ithome.com.tw/upload/images/20190925/20105988ouMJ6GRB7F.png](https://ithelp.ithome.com.tw/upload/images/20190925/20105988ouMJ6GRB7F.png) 2165 | 2166 | 以下用sys.objects來查SQL Server的表格跟視圖當追蹤例子 : 2167 | ```C# 2168 | var result = cn.Query(@"select * from sys.objects where type_desc In @type_descs", new { type_descs = new[] { "USER_TABLE", "VIEW" } }); 2169 | ``` 2170 | 2171 | Dapper會將SQL字串改成以下方式執行 2172 | ```C# 2173 | select * from sys.objects where type_desc In (@type_descs1,@type_descs2) 2174 | -- @type_descs1 = nvarchar(4000) - 'USER_TABLE' 2175 | -- @type_descs2 = nvarchar(4000) - 'VIEW' 2176 | ``` 2177 | 2178 | 查看Emit IL可以發現跟之前的參數化IL很不一樣,非常的簡短 2179 | ```C# 2180 | IL_0000: ldarg.1 2181 | IL_0001: castclass <>f__AnonymousType0`1[System.String[]] 2182 | IL_0006: stloc.0 2183 | IL_0007: ldarg.0 2184 | IL_0008: callvirt System.Data.IDataParameterCollection get_Parameters()/System.Data.IDbCommand 2185 | IL_000d: ldarg.0 2186 | IL_000e: ldstr "type_descs" 2187 | IL_0013: ldloc.0 2188 | IL_0014: callvirt System.String[] get_type_descs()/<>f__AnonymousType0`1[System.String[]] 2189 | IL_0019: call Void PackListParameters(System.Data.IDbCommand, System.String, System.Object)/Dapper.SqlMapper 2190 | IL_001e: pop 2191 | IL_001f: ret 2192 | ``` 2193 | 2194 | 轉成C#代碼來看,會很驚訝地發現:`「這段根本不需要使用Emit IL簡直多此一舉」` 2195 | ```C# 2196 | public static void TestMeThod(IDbCommand P_0, object P_1) 2197 | { 2198 | var anon = (<>f__AnonymousType0)P_1; 2199 | IDataParameterCollection parameter = P_0.Parameters; 2200 | SqlMapper.PackListParameters(P_0, "type_descs", anon.type_descs); 2201 | } 2202 | ``` 2203 | 2204 | 沒錯,是多此一舉,甚至` IDataParameterCollection parameter = P_0.Parameters;`這段代碼根本不會用到。 2205 | 2206 | Dapper這邊做法是有原因的,因為要`能跟非集合參數配合使用`,像是前面例子加上找出訂單Orders名稱的資料邏輯 2207 | ```C# 2208 | var result = cn.Query(@"select * from sys.objects where type_desc In @type_descs and name like @name" 2209 | , new { type_descs = new[] { "USER_TABLE", "VIEW" }, @name = "order%" }); 2210 | ``` 2211 | 對應生成的IL轉換C#代碼就會是以下代碼,達到能搭配使用目的 : 2212 | ```C# 2213 | public static void TestMeThod(IDbCommand P_0, object P_1) 2214 | { 2215 | <>f__AnonymousType0 val = P_1; 2216 | IDataParameterCollection parameters = P_0.Parameters; 2217 | SqlMapper.PackListParameters(P_0, "type_descs", val.get_type_descs()); 2218 | IDbDataParameter dbDataParameter = P_0.CreateParameter(); 2219 | dbDataParameter.ParameterName = "name"; 2220 | dbDataParameter.DbType = DbType.String; 2221 | dbDataParameter.Direction = ParameterDirection.Input; 2222 | object obj = val.get_name(); 2223 | int num; 2224 | if (obj == null) 2225 | { 2226 | obj = DBNull.Value; 2227 | num = 0; 2228 | } 2229 | else 2230 | { 2231 | num = ((((string)obj).Length > 4000) ? (-1) : 4000); 2232 | } 2233 | dbDataParameter.Value = obj; 2234 | if (num != 0) 2235 | { 2236 | dbDataParameter.Size = num; 2237 | } 2238 | parameters.Add(dbDataParameter); 2239 | } 2240 | ``` 2241 | 2242 | 另外為何Dapper這邊Emit IL會直接呼叫工具方法`PackListParameters`,是因為IN的`參數化數量是不固定`,所以`不能由固定結果反推程式碼`方式動態生成方法。 2243 | 2244 | 該方法裡面包含的主要邏輯: 2245 | 1. 判斷集合參數的類型是哪一種 (假如是字串預設使用4000大小) 2246 | 2. 正則判斷SQL參數以流水號參數字串取代 2247 | 3. DbCommand的Paramter的創建 2248 | 2249 | ![https://ithelp.ithome.com.tw/upload/images/20190925/20105988KgYZmlciZJ.png](https://ithelp.ithome.com.tw/upload/images/20190925/20105988KgYZmlciZJ.png) 2250 | SQL參數字串的取代邏輯也寫在這邊,如圖片 2251 | ![https://ithelp.ithome.com.tw/upload/images/20190925/20105988Rhner7LZPA.png](https://ithelp.ithome.com.tw/upload/images/20190925/20105988Rhner7LZPA.png) 2252 | 2253 | 2254 | 2255 | 　 2256 | 　 2257 | ## 20.DynamicParameter 底層原理、自訂實作 2258 | 2259 | 2260 | 這邊用個例子帶讀者了解DynamicParameter原理,舉例現在有一段代碼如下 : 2261 | ```C# 2262 | using (var cn = Connection) 2263 | { 2264 | var paramter = new { Name = "John", Age = 25 }; 2265 | var result = cn.Query("select @Name Name,@Age Age", paramter).First(); 2266 | } 2267 | ``` 2268 | 2269 | 前面已經知道String型態Dapper會自動將轉成數據庫`Nvarchar`並且`長度為4000`的參數,數據庫實際執行的SQL如下 : 2270 | ```sql 2271 | exec sp_executesql N'select @Name Name,@Age Age',N'@Name nvarchar(4000),@Age int',@Name=N'John',@Age=25 2272 | ``` 2273 | 2274 | 這是一個方便快速開發的貼心設計,但假如遇到欄位是`varchar`型態的情況,有可能會因為隱性轉型導致`索引失效`,導致查詢效率變低。 2275 | 2276 | 這時解決方式可以使用Dapper DynamicParamter指定數據庫型態跟大小,達到優化效能目的 2277 | ```C# 2278 | using (var cn = Connection) 2279 | { 2280 | var paramters = new DynamicParameters(); 2281 | paramters.Add("Name","John",DbType.AnsiString,size:4); 2282 | paramters.Add("Age",25,DbType.Int32); 2283 | var result = cn.Query("select @Name Name,@Age Age", paramters).First(); 2284 | } 2285 | ``` 2286 | 2287 | ---- 2288 | 2289 | 2290 | 接著往底層來看如何實現,首先關注GetCacheInfo方法,可以看到DynamicParameters建立動態方法方式代碼很簡單,就只是呼叫AddParameters方法 2291 | ```C# 2292 | Action reader; 2293 | if (exampleParameters is IDynamicParameters) 2294 | { 2295 | reader = (cmd, obj) => ((IDynamicParameters)obj).AddParameters(cmd, identity); 2296 | } 2297 | ``` 2298 | 2299 | 代碼可以這麼簡單的原因,是Dapper在這邊特別使用`「依賴於介面」`設計,增加`程式的彈性`,讓使用者可以客制自己想要的實作邏輯。這點下面會講解,首先來看Dapper預設的實作類別`DynamicParameters`中`AddParameters`方法的實作邏輯 2300 | 2301 | ```C# 2302 | public class DynamicParameters : SqlMapper.IDynamicParameters, SqlMapper.IParameterLookup, SqlMapper.IParameterCallbacks 2303 | { 2304 | protected void AddParameters(IDbCommand command, SqlMapper.Identity identity) 2305 | { 2306 | var literals = SqlMapper.GetLiteralTokens(identity.sql); 2307 | 2308 | foreach (var param in parameters.Values) 2309 | { 2310 | if (param.CameFromTemplate) continue; 2311 | 2312 | var dbType = param.DbType; 2313 | var val = param.Value; 2314 | string name = Clean(param.Name); 2315 | var isCustomQueryParameter = val is SqlMapper.ICustomQueryParameter; 2316 | 2317 | SqlMapper.ITypeHandler handler = null; 2318 | if (dbType == null && val != null && !isCustomQueryParameter) 2319 | { 2320 | #pragma warning disable 618 2321 | dbType = SqlMapper.LookupDbType(val.GetType(), name, true, out handler); 2322 | #pragma warning disable 618 2323 | } 2324 | if (isCustomQueryParameter) 2325 | { 2326 | ((SqlMapper.ICustomQueryParameter)val).AddParameter(command, name); 2327 | } 2328 | else if (dbType == EnumerableMultiParameter) 2329 | { 2330 | #pragma warning disable 612, 618 2331 | SqlMapper.PackListParameters(command, name, val); 2332 | #pragma warning restore 612, 618 2333 | } 2334 | else 2335 | { 2336 | bool add = !command.Parameters.Contains(name); 2337 | IDbDataParameter p; 2338 | if (add) 2339 | { 2340 | p = command.CreateParameter(); 2341 | p.ParameterName = name; 2342 | } 2343 | else 2344 | { 2345 | p = (IDbDataParameter)command.Parameters[name]; 2346 | } 2347 | 2348 | p.Direction = param.ParameterDirection; 2349 | if (handler == null) 2350 | { 2351 | #pragma warning disable 0618 2352 | p.Value = SqlMapper.SanitizeParameterValue(val); 2353 | #pragma warning restore 0618 2354 | if (dbType != null && p.DbType != dbType) 2355 | { 2356 | p.DbType = dbType.Value; 2357 | } 2358 | var s = val as string; 2359 | if (s?.Length <= DbString.DefaultLength) 2360 | { 2361 | p.Size = DbString.DefaultLength; 2362 | } 2363 | if (param.Size != null) p.Size = param.Size.Value; 2364 | if (param.Precision != null) p.Precision = param.Precision.Value; 2365 | if (param.Scale != null) p.Scale = param.Scale.Value; 2366 | } 2367 | else 2368 | { 2369 | if (dbType != null) p.DbType = dbType.Value; 2370 | if (param.Size != null) p.Size = param.Size.Value; 2371 | if (param.Precision != null) p.Precision = param.Precision.Value; 2372 | if (param.Scale != null) p.Scale = param.Scale.Value; 2373 | handler.SetValue(p, val ?? DBNull.Value); 2374 | } 2375 | 2376 | if (add) 2377 | { 2378 | command.Parameters.Add(p); 2379 | } 2380 | param.AttachedParam = p; 2381 | } 2382 | } 2383 | 2384 | // note: most non-priveleged implementations would use: this.ReplaceLiterals(command); 2385 | if (literals.Count != 0) SqlMapper.ReplaceLiterals(this, command, literals); 2386 | } 2387 | } 2388 | ``` 2389 | 2390 | 可以發現Dapper在AddParameters為了方便性跟兼容其他功能,像是Literal Replacement、EnumerableMultiParameter功能,做了許多判斷跟動作,所以代碼量會比以前使用ADO.NET版本多,所以效率也會比較慢。 2391 | 2392 | 假如有效率苛求的需求,可以自己實作想要的邏輯,因為Dapper此段特別設計成`「依賴於介面」`,只需要實作`IDynamicParameters`介面就可以。 2393 | 2394 | 以下是我做的一個Demo,可以使用ADO.NET SqlParameter建立參數跟Dapper配合 2395 | ```C# 2396 | public class CustomPraameters : SqlMapper.IDynamicParameters 2397 | { 2398 | private SqlParameter[] parameters; 2399 | public void Add(params SqlParameter[] mParameters) 2400 | { 2401 | parameters = mParameters; 2402 | } 2403 | 2404 | void SqlMapper.IDynamicParameters.AddParameters(IDbCommand command, SqlMapper.Identity identity) 2405 | { 2406 | if (parameters != null && parameters.Length > 0) 2407 | foreach (var p in parameters) 2408 | command.Parameters.Add(p); 2409 | } 2410 | } 2411 | ``` 2412 | 2413 | ![https://ithelp.ithome.com.tw/upload/images/20191005/20105988qzCAsa5KZu.png](https://ithelp.ithome.com.tw/upload/images/20191005/20105988qzCAsa5KZu.png) 2414 | 2415 | 　 2416 | 　 2417 | ## 21. 單次、多次 Execute 底層原理 2418 | 2419 | 2420 | 查詢、Mapping、參數講解完後,接著講解在`增、刪、改`情況Dapper我們會使用Execute方法,其中Execute Dapper分為`單次執行、多次執行`。 2421 | 2422 | #### 單次Execute 2423 | 以單次執行來說Dapper Execute底層是ADO.NET的ExecuteNonQuery的封裝,封裝目的為了跟Dapper的`Parameter、緩存`功能搭配使用,代碼邏輯簡潔明瞭這邊就不做多說明,如圖片 2424 | ![20191002144453.png](https://i.loli.net/2019/10/02/DPUYt5mLzFvoaQw.png) 2425 | 2426 | 2427 | #### 「多次」Execute 2428 | 2429 | 這是Dapper一個特色功能,它簡化了集合操作Execute之間的操作,簡化了代碼,只需要 : `connection.Execute("sql",集合參數);`。 2430 | 2431 | 至於為何可以這麼方便,以下是底層的邏輯 : 2432 | 1. 確認是否為集合參數 2433 | ![20191002150155.png](https://i.loli.net/2019/10/02/alGyqdNxSkg3EWz.png) 2434 | 2. 建立`一個共同DbCommand`提供foreach迭代使用,避免重複建立浪費資源 2435 | ![20191002151237.png](https://i.loli.net/2019/10/02/HfTID8jpE1nZyXF.png) 2436 | 3. 假如是集合參數,建立Emit IL動態方法,並放在緩存內利用 2437 | ![20191002150349.png](https://i.loli.net/2019/10/02/s1aNBUEfhnFcMbw.png) 2438 | 4. 動態方法邏輯是`CreateParameter > 對Parameter賦值 > 使用Parameters.Add添加新建的參數`,以下是Emit IL轉成的C#代碼 : 2439 | ```C# 2440 | public static void ParamReader(IDbCommand P_0, object P_1) 2441 | { 2442 | var anon = (<>f__AnonymousType0)P_1; 2443 | IDataParameterCollection parameters = P_0.Parameters; 2444 | IDbDataParameter dbDataParameter = P_0.CreateParameter(); 2445 | dbDataParameter.ParameterName = "V"; 2446 | dbDataParameter.DbType = DbType.Int32; 2447 | dbDataParameter.Direction = ParameterDirection.Input; 2448 | dbDataParameter.Value = anon.V; 2449 | parameters.Add(dbDataParameter); 2450 | } 2451 | ``` 2452 | 5. `foreach`該集合參數 > 除了第一次外,每次迭代清空DbCommand的Parameters > 重新呼叫`同一個`動態方法添加Parameter > 送出SQL查詢 2453 | 2454 | --- 2455 | 2456 | 實作方式簡潔明瞭,並且細節考慮共用資源避免浪費(e.g`共用同一個DbCommand、Func`),但遇到大量執行追求效率需求情況,需要特別注意此方法`每跑一次對數據庫送出一次reqesut`,效率會被網路傳輸拖慢,所以這功能被稱為`「多次執行」而不是「批量執行」`的主要原因。 2457 | 2458 | 舉例,簡單Execute插入十筆資料,查看SQL Profiler可以看到系統接到10次Reqeust: 2459 | ```C# 2460 | using (var cn = new SqlConnection(@"Data Source=(localdb)\MSSQLLocalDB;Integrated Security=SSPI;Initial Catalog=Northwind;")) 2461 | { 2462 | cn.Open(); 2463 | using (var tx = cn.BeginTransaction()) 2464 | { 2465 | cn.Execute("create table #T (V int);", transaction: tx); 2466 | cn.Execute("insert into #T (V) values (@V)", Enumerable.Range(1, 10).Select(val => new { V = val }).ToArray() , transaction:tx); 2467 | 2468 | var result = cn.Query("select * from #T", transaction: tx); 2469 | Console.WriteLine(result); 2470 | } 2471 | } 2472 | ``` 2473 | 2474 | ![20191002151658.png](https://i.loli.net/2019/10/02/WZ5w1gMFG2ESzDm.png) 2475 | 2476 | 2477 | 　 2478 | 　 2479 | ## 22. ExecuteScalar應用 2480 | 2481 | 2482 | ExecuteScalar因為其`只能讀取第一組結果、第一筆列、第一筆資料`特性,是一個常被遺忘的功能,但它在特定需求下還是能派上用場,底下用「查詢資料是否存在」例子來做說明。 2483 | 2484 | ### 首先,Entity Framwork如何高效率判斷資料是否存在? 2485 | 2486 | 假如有EF經驗的讀者會答使用`Any`而不是`Count() > 1`。 2487 | 2488 | 使用Count系統會幫轉換SQL為 : 2489 | ```sql 2490 | SELECT COUNT(*) AS [value] FROM [表格] AS [t0] 2491 | ``` 2492 | SQL Count 是一個匯總函數,會迭代符合條件的資料行`判斷每列該資料是否為null`,並返回其行數。 2493 | 2494 | 而Any語法轉換SQL使用`EXISTS`,它只在乎`是否有沒有資料`,代表`不用檢查到每列`,只需要其中一筆有資料就有結果,所以效率快。 2495 | ```sql 2496 | SELECT 2497 | (CASE 2498 | WHEN EXISTS( 2499 | SELECT NULL AS [EMPTY] 2500 | FROM [表格] AS [t0] 2501 | ) THEN 1 2502 | ELSE 0 2503 | END) AS [value] 2504 | ``` 2505 | 2506 | #### Dapper如何做到同樣效果? 2507 | 2508 | SQL Server可以使用SQL格式`select top 1 1 from [表格] where 條件` 搭配 ExecuteScalar 方法,接著在做一個擴充方法,如下 : 2509 | ```C# 2510 | public static class DemoExtension 2511 | { 2512 | public static bool Any(this IDbConnection cn,string sql,object paramter = null) 2513 | { 2514 | return cn.ExecuteScalar(sql,paramter); 2515 | } 2516 | } 2517 | ``` 2518 | 效果圖 : 2519 | ![20191003043825.png](https://i.loli.net/2019/10/03/iSzgBUmkuOFoa4w.png) 2520 | 2521 | 使用如此簡單原因,是利用Dapper ExecuteScalar會去呼叫ExecuteScalarImpl其底層Parse邏輯 2522 | ```C# 2523 | private static T ExecuteScalarImpl(IDbConnection cnn, ref CommandDefinition command) 2524 | { 2525 | //..略 2526 | object result; 2527 | //..略 2528 | result = cmd.ExecuteScalar(); 2529 | //..略 2530 | return Parse(result); 2531 | } 2532 | 2533 | private static T Parse(object value) 2534 | { 2535 | if (value == null || value is DBNull) return default(T); 2536 | if (value is T) return (T)value; 2537 | var type = typeof(T); 2538 | //..略 2539 | return (T)Convert.ChangeType(value, type, CultureInfo.InvariantCulture); 2540 | } 2541 | ``` 2542 | 使用 Convert.ChangeType 轉成 bool : `「0=false,非0=true」` 特性,讓系統可以簡單轉型為bool值。 2543 | 2544 | #### 注意 2545 | 不要QueryFirstOrDefault代替,因為它需要在SQL額外做Null的判斷,否則會出現「NullReferenceException」。 2546 | ![20191003043931.png](https://i.loli.net/2019/10/03/XoRu7Y3nA58wKpe.png) 2547 | 2548 | 這原因是兩者Parse實作方式不一樣,QueryFirstOrDefault判斷結果為null時直接強轉型 2549 | ![20191003043941.png](https://i.loli.net/2019/10/03/TjHpdwxDMWuVaA5.png) 2550 | 2551 | 而ExecuteScalar的Parce實作多了`為空時使用default值`的判斷 2552 | ![20191003043953.png](https://i.loli.net/2019/10/03/hcoO1sEbaIzHlKS.png) 2553 | 2554 | 2555 | 　 2556 | 　 2557 | ## 23.總結 2558 | 2559 | 2560 | Dapper系列到這邊,重要底層原理差不多都講完了,這系列總共花了筆者連續25天的時間,除了想幫助讀者外,最大的收穫就是我自己在這期間更了解Dapper底層原理,並且學習Dapper精心的細節、框架處理。 2561 | 2562 | 另外想提Dapper作者之一Marc Gravell,真的非常熱心,在寫文章的期間有幾個概念疑問,發issue詢問,他都會熱心、詳細的回覆。並且也發現他對代碼的品質要求之高,舉例 : 在S.O發問,遇到他在底下留言 : `「他對目前Dapper IL的架構其實是不滿意的,甚至覺得粗糙,想搭配protobuf-net技術打掉重寫」` (謎之聲 : 真令人敬佩 ) 2563 | 2564 | 連結 : [c# - How to remove the last few segments of Emit IL at runtime - Stack Overflow](https://stackoverflow.com/questions/58094242/how-to-remove-the-last-few-segments-of-emit-il-at-runtime) 2565 | ![https://ithelp.ithome.com.tw/upload/images/20190925/201059884hfaioQATW.png](https://ithelp.ithome.com.tw/upload/images/20190925/201059884hfaioQATW.png) 2566 | 2567 | 最後筆者想說 : 2568 | 寫這篇的初衷,是希望本系列可以幫助到讀者 2569 | 1. 了解底層邏輯,知其所以然,避免寫出吃掉效能的怪獸,更進一步完整的利用Dapper優點開發專案 2570 | 2. 可以輕鬆面對Dapper的面試,比起一般使用Dapper工程師回答出更深層的概念 2571 | 3. 從最簡單Reflection到常用Expression到最細節Emit從頭建立Mapping方法,帶讀者`漸進式`了解Dapper底層強型別Mapping邏輯 2572 | 4. 了解動態建立方法的重要概念`「結果反推程式碼」` 2573 | 5. 有基本IL能力,可以利用IL反推C#代碼方式看懂其他專案的底層Emit邏輯 2574 | 6. 了解Dapper因為緩存的算法邏輯,所以`不能使用錯誤字串拼接SQL` 2575 | 2576 | 2577 | --------------------------------------------------------------------------------