├── README.md
└── LICENSE


/README.md:
--------------------------------------------------------------------------------
 1 | # Awesome-Tricks-in-SOTA-ML
 2 | Interesting & promising & widely adopted tricks for SOTA performance in machine learning community.
 3 | 
 4 | Any wonderful tricks are encouraged! Take your PR!
 5 | 
 6 | ## Image Classification:
 7 | 
 8 | ### Semi-Supervised & Self-Supervised Learning
 9 | 
10 | - Mean Teacher https://arxiv.org/abs/1703.01780
11 | - Using EMA version of Batch Normalization in Teacher model (EMAN) https://arxiv.org/pdf/2101.08482.pdf
12 | - Byol: Self-Supervised Learning without negative sampling https://arxiv.org/abs/2006.07733
13 | 
14 | ### Transformer in Image
15 | - Fix the patch projection with random initialization can improve the traning stability https://arxiv.org/pdf/2104.02057.pdf
16 | 
17 | 
18 | ## Image Generation:
19 | 
20 | ### GAN
21 | - Zero or Negative momentum in the optimizer https://arxiv.org/abs/1807.04740 https://arxiv.org/abs/1909.13188
22 | 
23 | ### Energy-based model
24 | - Image Buffer for PCD https://arxiv.org/abs/1903.08689
25 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2021 Kun Xu
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------