├── .gitignore ├── README.md ├── 人智 ├── Helin_AI.pdf ├── Helin_AI_notability.note └── readme.md ├── 任选课 └── readme.md ├── 信号系统 ├── Helin_SS_cheatsheet.pdf ├── Helin_SS_notability.note └── readme.md ├── 数值分析 ├── Helin_NA_cheatsheet.pdf ├── Helin_NA_project1.pdf ├── readme.md └── 数值分析大作业2021-1.pptx ├── 数据结构 └── readme.md ├── 智能优化算法 ├── Intelligent_Optimization_Algorithms_and_Its_Applications_Final_Project_(2021_Fall).pdf ├── readme.md └── 智能优化考核作业2021.ppt ├── 智能检测 └── readme.md ├── 模式识别 ├── Helin_PRML.pdf ├── Helin_PRML_notability.note ├── Helin_PRML_project.md ├── readme.md └── 大作业说明v0_4.pdf ├── 系统工程导论 ├── 1解释性结构.pdf ├── 1黑箱建模.pdf ├── 2主成分分析.pdf ├── 2因子分析.pdf ├── 2聚类分析.pdf ├── 3不确定性分析.pdf └── readme.md ├── 自控 ├── readme.md ├── 自动控制理论-现代部分知识点总结.pdf └── 自控教学安排2021年秋20210910.pdf ├── 计网 └── readme.md └── 运筹学 └── readme.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 清华大学自动化系课程 2 | 3 | [在线地址](https://github.com/HelinXu/THUDA-9) 4 | 5 | ## 专业内课程 6 | 7 | | | 复习/小抄 | 项目/论文 | 实用信息 | 社区资源 | 8 | | -------------- | --------- | --------- | -------- | ---------- | 9 | | 人智 | ✅ 🈶 | ✅ 🈶 | | 不太有用 | 10 | | 信号系统 | ✅ 🈶 | 🈶 | | 🈶 | 11 | | 数值分析 | ✅ 🈶 | ✅ 🈶 | | 🈶 不太有用 | 12 | | 数据结构（邓） | | NA | ✅ | 🈶 | 13 | | 智能优化算法 | NA | ✅ | P/F | NA | 14 | | 智能检测 | 🈶 | | | 🈶 | 15 | | 模式识别 | ✅ 🈶 | ✅ 🈶 | | 不太有用 | 16 | | 系统工程导论 | ✅ 🈶 | 🈶 | ✅ | 🈶 | 17 | | 自控 | ✅ | | ✅ | 🈶 | 18 | | 计网 | | 🈶 | ✅ | 🈶 | 19 | | 运筹学 | | | | 🈶 | 20 | 21 | 注： 22 | 23 | - 🈶 表示应该能找到一些社区资源，但是本仓库不会包含这些内容。 24 | - ✅ 表示本仓库已经包含了的内容，来自本仓库的贡献者们。 25 | 26 | 27 | 28 | ## 任选课 29 | 30 | - 电影文化 31 | - 管乐合奏 32 | - 文学研究中的当代英语 33 | - 词汇的力量 34 | - 西方古典音乐文化 35 | - 艺术欣赏导论 36 | - 京剧艺术赏析 37 | - 大学生创新创业基础 38 | - 新生导引课 39 | - 新生专题研讨 40 | 41 | 42 | 43 | ## 其他 44 | 45 | 欢迎分享/贡献！ 46 | 47 | 欢迎 issue/PR/⭐ etc. :) 48 | 49 | 更新于 2022/09/23 50 | 51 | [在线地址](https://github.com/HelinXu/THUDA-9) -------------------------------------------------------------------------------- /人智/Helin_AI.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/人智/Helin_AI.pdf -------------------------------------------------------------------------------- /人智/Helin_AI_notability.note: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/人智/Helin_AI_notability.note -------------------------------------------------------------------------------- /人智/readme.md: -------------------------------------------------------------------------------- 1 | # Foundation of Artificial Intelligence 2 | 3 | Instructor: Rui Jiang 4 | 5 | Grading: 6 | 7 | - 3 credits. 8 | - 50% (Homework * 10, program assignment * 5, course project * 2), 50% Final Exam (A4 cheat sheet allowed). 9 | 10 | Helin's Notes: 11 | 12 | - Thankfully, this year, the final exam was composed of ~7 problems, covering the main topics from lectures, similar to those covered in homework. I personally feel that with a fair understanding of topics from homework, you should be comfortable with the exam. However, this was not the case last year, when there were 2 instructors for this course. 13 | 14 | - My cheat sheet is [here](./Helin_AI.pdf). Use notability to [edit yourself](./Helin_AI_notability.note) if you wish. 15 | 16 | - My code for project 1: [Chinese Poker Player](https://github.com/HelinXu/ChinesePokerPlayer) 17 | - My code for project 2: [Covid Classification](https://github.com/HelinXu/covid_classification) -------------------------------------------------------------------------------- /任选课/readme.md: -------------------------------------------------------------------------------- 1 | # Other Courses 2 | 3 | Contributor: Helin Xu 4 | 5 | 6 | 7 | ## 电影文化 8 | 9 | - 教师：尹鸿 10 | 11 | - 3学分。 12 | 13 | - 我上的这一年，是P/F，文核，并且计入培养方案自主发展课程。 14 | - 课程任务包括：期中影评一篇，期末开卷考试。 15 | 16 | - 期末开卷，可使用纸质参考书（比如去图书馆借几本影评书）、笔记本，不可使用任何电子设备。 17 | - 开卷考了若干开放性问题，好像是七个问题里任选五个回答。例如，分析一部令你印象深刻的中国电影，分析上课讲到的某电影中的声音特点，举例说明不同景别的作用，等等。 18 | 19 | 20 | 21 | ## 管乐合奏（THUMB限选课） 22 | 23 | - 就是 THUMB 大合排。 24 | - 文素课。 25 | - 每学期 1 学分，基本都会给到 4.0。 26 | - 欢迎入队然后选课！ 27 | 28 | 29 | 30 | ## 文学研究中的当代英语 31 | 32 | - 教师：吕中舌 33 | - 2学分。 34 | - 文核课。英语限定选课组课。 35 | - 分析英文诗歌，修辞手法，和有关的文学作品。 36 | - 期末有一篇相关话题的essay。 37 | - 吕老师的口音非常British、非常纯正，人很和蔼。可以与她讨论你的期末essay，给分优秀。 38 | 39 | 40 | 41 | ## 词汇的力量 42 | 43 | - 教师：王英 44 | - 2学分，个人感觉是4学分。有人认为是6学分，但我认为也没那么夸张。 45 | - 英语限定课组课。 46 | - 硬课。背诵任务多，随堂练习多，英文电影观影任务多，但是完成任务给分优秀，我认为可以保证 4.0。 47 | 48 | 49 | 50 | ## 西方古典音乐文化 51 | 52 | - 教师：刘欣欣 53 | - 2学分。 54 | - 文核课。 55 | - 随堂音乐赏析，有较多雨课堂弹幕、一节课一两个雨课堂题目。老师很有激情，同时很注重互动。 56 | - 期末 3k 字论文，可以与老师讨论你的期末论文获得修改意见。一说论文需要卷，但是我也就写了规定的字数，给分优秀。 57 | 58 | 59 | 60 | ## 艺术欣赏导论 61 | 62 | - 艺教中心开课 63 | - 1学分，P/F 64 | - 讲座课，兴趣课，选就对了。任务是期末论文。 65 | 66 | 67 | 68 | ## 京剧艺术赏析 69 | 70 | - 艺教中心开课 71 | - 1学分，P/F 72 | - 讲座课，兴趣课，选就对了。任务是期末论文。 73 | 74 | 75 | 76 | ## 大学生创新创业基础 77 | 78 | - 教师：韩文弢 79 | - 1学分，P/F 80 | - 校友讲座，自主行业调研，组队期末pre。选就对了。 81 | 82 | 83 | 84 | ## 新生导引课 02070301 85 | 86 | - 教师：大量老师都开课 87 | - 1学分，P/F 88 | - 参观学校，去工字厅秋游、去天文台观测博雅塔、去荷塘喂蚊子，以及一些介绍学校的讲座。一个学期写了一篇感想。 89 | - 限新生选课，新生不要错过。 90 | 91 | 92 | 93 | ## 新生专题研讨 30511021 94 | 95 | - 教师：张涛 96 | - 1学分，P/F 97 | - 外文认定课 98 | - 八次专业相关的讲座课，科普性质。作业是随便写写有关的note。 -------------------------------------------------------------------------------- /信号系统/Helin_SS_cheatsheet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/信号系统/Helin_SS_cheatsheet.pdf -------------------------------------------------------------------------------- /信号系统/Helin_SS_notability.note: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/信号系统/Helin_SS_notability.note -------------------------------------------------------------------------------- /信号系统/readme.md: -------------------------------------------------------------------------------- 1 | # Signals and System Analysis 2 | 3 | Instructor: Qing (Joking) Zhuo 4 | 5 | Grading: 6 | 7 | - 4 credits. 8 | - Homework, Midterm and Final Exams. Bonus points are given if you write a final paper. 9 | 10 | Helin's Notes: 11 | 12 | - Joking's classes are very interactive. 13 | - After releasing the final score, you still have a chance to write & submit a final paper, which can save your grade after the final exam. 14 | - Past exams can be found and are very informative. 15 | - Past cheat sheets can be found, and can save you a lot of time. 16 | - My cheat sheet is [here](./Helin_SS_cheatsheet.pdf). Use notability to [edit yourself](./Helin_SS_notability.note) if you wish. 17 | 18 | -------------------------------------------------------------------------------- /数值分析/Helin_NA_cheatsheet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/数值分析/Helin_NA_cheatsheet.pdf -------------------------------------------------------------------------------- /数值分析/Helin_NA_project1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/数值分析/Helin_NA_project1.pdf -------------------------------------------------------------------------------- /数值分析/readme.md: -------------------------------------------------------------------------------- 1 | # Numerical Analysis and Algorithms 2 | 3 | Instructor: ZHAO HONG 4 | 5 | Grading: 6 | 7 | - 3 credits 8 | - Homework, Course Projects, Final Exam (A4 cheat sheet allowed) 9 | 10 | Helin's Notes: 11 | 12 | - Though past exams can be found, there is NO GUARANTEE how well they cover your exam. In my case, there were at least 2 new problems. 13 | - However, past exams still tell a lot about the key topics to appear in your exam. Make sure you really understand them. Discuss with TA about homework and examples from the lectures. 14 | - My cheat sheet is [here](./Helin_NA_cheatsheet.pdf). Editable versions can be found elsewhere. 15 | - My project: [Report](./Helin_NA_project1.pdf), [Code Release](https://github.com/HelinXu/NA_project1). 16 | 17 | -------------------------------------------------------------------------------- /数值分析/数值分析大作业2021-1.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/数值分析/数值分析大作业2021-1.pptx -------------------------------------------------------------------------------- /数据结构/readme.md: -------------------------------------------------------------------------------- 1 | # Data Structure 2 | 3 | Instructor: Junhui Deng 4 | 5 | Grading: 6 | 7 | - 4 credits. 8 | - 72% OJ, 28% Final Exam (A4 cheat sheet allowed). 9 | 10 | Course Description: 11 | 12 | > 亲爱的同学你好： 13 | > 14 | > 15 | > 16 | > 欢迎选修“数据结构”，我们的“数据结构与算法”之旅今天就启程了！以下就是你朝着这个方向需要跨出的前几步： 17 | > 18 | > 19 | > 20 | > +) 打开教材服务页面： 21 | > 22 | > https://dsa.cs.tsinghua.edu.cn/~deng/ds/dsacpp/ 23 | > 24 | > 并借助上面的链接下载以下资料。 25 | > 26 | > 27 | > 28 | > +) 下载课程讲义 29 | > 30 | > 其中，电子阅读版讲义按章、节组织为260个独立的PDF文件，在电子阅读器中你可以通过文档名称来辨识它们。 31 | > 32 | > 打印版讲义则提供了数种拼页的制式，你可根据偏好选择其一。相对而言八拼页制式是更多同学的选择，既能控制装订厚度、便于携带，同时也有一定的留白便于补充笔记。如选择四拼页制式，建议分成上、下两卷装订（别忘了让复印室给下册也加一份目录）。所有讲稿均已统一地做过编号，因此无论选用何种制式，你我在学习交流过程中都能方便而准确地在其中定位。 33 | > 34 | > 课程大纲（syllabus）一节中，有关于这门课程的很多有用信息，请大家务必仔细阅读。比如，其中解释了这门课程到底需要哪些先修的“基础知识”，建议本校同学主要借助讲义来学习而“不必购买教材”，同时还就编程实验、实验平台、课程纪律等做了说明。 35 | > 36 | > 37 | > 38 | > +) 下载示例代码包 39 | > 40 | > 课程中讲授的数据结构基本上都涵盖在这个代码包中，它按照Visual Studio 2008/2015格式，将60个项目组织为一个解决方案，可以直接编译、运行。这些代码不仅能够演示各数据结构与算法的原理与机制，同时也能形象地展示其运行与演化的过程。 41 | > 42 | > 43 | > 44 | > +) 从网络学堂“课程文件”区下载演示程序 45 | > 46 | > 按其运行平台划分，演示程序主要有Excel、Java Applet两种类型。前一类其实就是Excel文件，打开之后你就会看到关于使用方法的提示。通常，只需要不断按下F9键，就能观看对应算法的执行过程；有些复杂的算法，可能还需要首先解锁一个开关。在新式的电脑上，传统的F1~12键往往会默认地当做多媒体以及上网操作的快捷键，此时你可使用Fn键来切换（甚至使用Fn Lock来锁定），或直接在BIOS中将F1~12键设为Legacy模式。这类演示程序尽管在手机上也可能打开，但很遗憾往往并不能正常运转，因此还是建议你在电脑上运行它们。 47 | > 48 | > 为运行Java Applet类型的演示程序，你需要确认已安装Java AppletViewer。它是JDK或JRE中的一个小组件，通常放在bin目录下。你可以用AppletViewer打开各演示所对应的index.applet文件，也可以将*.applet文件统一设定为用AppletViewer打开，从而只需双击*.applet即可快捷地启动演示。 49 | > 50 | > 51 | > 52 | > +) 注册慕课课堂，在开学之前自学第一章和第二章 53 | > 54 | > 同学们在数据结构与算法方面的基础，近年来有差异日渐扩大的趋势，因此我们将属于基础性的前两章交由同学们，灵活地根据自己的具体情况进行自学。开学后我们只会对这部分内容简略地做一回顾和概述，便从第三章开始正式讲授。 55 | > 56 | > 通过这个页面： 57 | > 58 | > https://dsa.cs.tsinghua.edu.cn/~deng/ds/mooc/ 59 | > 60 | > 你可找到课程组制作的同名慕课（分为基础、高级两个课堂）并免费注册，然后通过网络在线学习。其中的“基础”课堂，覆盖了上面建议自学的前两章的大部分内容。 61 | > 62 | > 63 | > 64 | > 经历过新冠疫情的冲击，老师与你一样更加期待着在校园里、课堂上见到你，加油！ 65 | > 66 | > 67 | > 68 | > 清华大学计算机科学与技术系“数据结构”教学团队 69 | > 70 | > 2020年8月22日 71 | 72 | 73 | 74 | Helin's Notes: 75 | 76 | - I chose Mr. Deng's DSA class, and he explained difficult things in a clear and easy way. 77 | - C++ skills are needed for OJ. Otherwise, those programming assignments can be very challenging. 78 | - Final exams are easy once you understand the main data structures and algorithms. -------------------------------------------------------------------------------- /智能优化算法/Intelligent_Optimization_Algorithms_and_Its_Applications_Final_Project_(2021_Fall).pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/智能优化算法/Intelligent_Optimization_Algorithms_and_Its_Applications_Final_Project_(2021_Fall).pdf -------------------------------------------------------------------------------- /智能优化算法/readme.md: -------------------------------------------------------------------------------- 1 | # Intelligent Optimization Algorithms and Its Applications 2 | 3 | Instructor: Ling Wang 4 | 5 | Grading: 6 | 7 | - P/F 8 | - 100% final project, see 智能优化考核作业2021.ppt 9 | 10 | Helin's final project: 11 | 12 | - Code: https://github.com/HelinXu/optimization 13 | - Report: Intelligent_Optimization_Algorithms_and_Its_Applications_Final_Project_(2021_Fall).pdf 14 | 15 | - Additional code for #3: https://github.com/HelinXu/SRT-4DAssociation-Louvain -------------------------------------------------------------------------------- /智能优化算法/智能优化考核作业2021.ppt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/智能优化算法/智能优化考核作业2021.ppt -------------------------------------------------------------------------------- /智能检测/readme.md: -------------------------------------------------------------------------------- 1 | # Smart Sensing and Measuring 2 | 3 | Instructor: Li Cao, Lihui Peng 4 | 5 | Course Description: 6 | 7 | > This course meets the sensing and measurement requirements for data and information acquisition in automatic and intelligent unmanned systems. It gives the students basic knowledge of sensors and instrumentation technology, as well as professional knowledge of data analysis and processing in intelligent measurement systems. Focusing on the measurement methods of force, displacement, speed, acceleration, temperature, pressure, flow, and other physical parameters, the lecture introduces the working principles and signal conditioning methods of resistance, capacitance, inductance, photoelectric, piezoelectric, semiconductor, MEMS and other sensors; performance analysis of measurement system and evaluation of measurement uncertainty; theory and application of weak signal detection and multi-sensor fusion. Using the examples of scientific research, industrial intelligence, and automation, combined with the latest topics such as artificial intelligence and big data, the lecture will introduce new intelligent sensing and detection methods and strengthen the student's ability to solve problems comprehensively using the knowledge they have learned. 8 | 9 | Grading: 10 | 11 | - 3 credits. 12 | - 15% Homework, 15% Experiments, 10% Attendance, and 60% Final exam (A4 cheat sheet allowed). 13 | 14 | Helin's Notes: 15 | 16 | - Past exams can be found and are very informative. 17 | - Past cheat sheets can be found and should suffice your need. 18 | 19 | -------------------------------------------------------------------------------- /模式识别/Helin_PRML.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/模式识别/Helin_PRML.pdf -------------------------------------------------------------------------------- /模式识别/Helin_PRML_notability.note: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/模式识别/Helin_PRML_notability.note -------------------------------------------------------------------------------- /模式识别/Helin_PRML_project.md: -------------------------------------------------------------------------------- 1 | # EMIT: Embedding Matching for Image-Text pairing 2 | 3 | Helin Xu (xuhelin1911@gmail.com) 4 | 5 | ## Requirements 6 | 7 | - Pytorch 1.11 8 | - CUDA 11.3 9 | 10 | All the packages can be installed by `pip install`. 11 | 12 | ## How to run 13 | 14 | 1. Download the data. 15 | 16 | 2. Unzip the data, and either put the data to `../medium/` or set the `DATA_ROOT` to the unzipped data directory. 17 | 18 | 3. Split the data into train, val by `python split_data.py`. 19 | 20 | 4. Train the model: `python train.py --tune_full_model`. 21 | 22 | 5. Inferer the model: `python inference.py`. The final prediction will be saved to `final_json.json`. 23 | 24 | ## Abstract 25 | 26 | In this work, we tackle the problem of image-text pairing, i.e. pairing images of various kinds of clothes and their corresponding text tags scrapped from Taobao. We propose to extract feature embeddings from both the images and the text tags and then match the embeddings of images and texts in the feature space. Our Embedding Matching for Image-Text pairing method achieves an impressive improvement compared to TAs' baseline methods. Moreover, to better leverage the constraint of images covering all optional tags, we design a heuristic-based bagging process that further boosts our performance. 27 | Codes are available at [GitHub](https://www.github.com/HelinXu/EMIT). 28 | 29 | 30 | 31 | ## Introduction 32 | 33 | Taobao, the biggest online store in China, hosts billions of products on sale, among which clothes are a typical case. There are a million kinds of cloth (i.e. products) on Taobao, each of which comes with a couple of styles (i.e. options). We hereby wish to develop a method that streamlines the process of manually labeling all the images, and thus comes the task of image-text pairing. 34 | 35 | We propose a novel approach that pairs images and text tags through embedding matching. (a) We adopt an image feature extractor for image feature embedding and compute word embedding for each piece of text tag. (b) Cross entropy is used to match the image and text embedding together during training, and for selecting the best-matching image-text pairs during inference. (c) Heuristic-based bagging is designed to overcome the drawback of the embedding matching method. 36 | 37 | 38 | 39 | ## Task Formulation 40 | 41 | In this section, we further formulate the task. 42 | 43 | Image-text pairing aims to match all the images from the same product to all the text tags. In this paper, we use **product** to refer to a single product on sale on Taobao, which often comes in several styles or options with typically several images under each style or option. **Optional tags** refers to the text-based tags that tell different styles from each other within the same product. For example, a certain kind of skirt is a product, and this product comes in three different styles, pink, white, and black. The text labels *pink, white, black* are the optional tags for this certain product. 44 | 45 | For a certain product, we have $n$ images in total 46 | $$ 47 | \mathcal { I } = \{I_1, I_2, ..., I_n\} 48 | $$ 49 | with $m$ different optional tags, 50 | $$ 51 | \mathcal { T } = \{T_1, T_2, ..., T_m\} 52 | $$ 53 | For each product, we output the image-text pairs for all images in the product, 54 | $$ 55 | (I_1, T_{I_1}), (I_2, T_{I_2}), ..., (I_n, T_{I_n}) 56 | $$ 57 | Where tags are selected from its optional tags, 58 | $$ 59 | \forall i \in \{ 1,2 , \ldots , n \} , I _ { i } \in \mathcal { I } , T _ { I _ { i } } \in \mathcal { T } 60 | $$ 61 | 62 | ## Task Analysis 63 | 64 | With careful inspection, we find this task challenging especially due to the messy optional tags as well as corner cases. In this section, we provide a detailed analysis of the challenges in this task, followed by our novel designs tailored for all of them. 65 | 66 | First and foremost, the dataset is raw in terms of optional tags. After enumerating the dataset, we find optional tags messy. There are over 5,000 kinds of unique optional tags among all products in the dataset. Treating the task as a naive classification task with optional tags as labels is not ideal. A selection of optional tags is shown in the quote below: 67 | 68 | ```text 69 | '牛油果绿预售15天', '【杏色】单件西装', '蓝色T恤+蓝色长裤', '烟雾粉181', '白色上衣+黑色套装裙子', '2801XCY紫色', '粉色上衣黑色阔腿裤套装', '军绿风衣', '丈青色上衣+卡其色裤子', '灰色套装', '黑白格923', '黑红', '323_灰色', '蓝色T恤', '黑白条纹', '紫红色连衣裙_', '松雪红', 'F5J7-红色', '单件灰白格中袖西装', '嫩黄色3082', '9665#红色', '[E]红色', '豆绿色', '粉橘色', '0505_驼色', '变色绿', '浅沙', '人字纹驼色', '粉色裤子', '杏白色开衫', 'BZ1609焦糖色', '218红色', '黑色-019', '暗紫', '粉色鹿角卫衣' 70 | ``` 71 | 72 | There are these challenges that lie in the dealing with optional tags: 73 | 74 | - The inclusion of meaningless characters within the optional tags (e.g. 'BZ1609焦糖色'); 75 | - The inclusion of unrelated semantic information, such as cloth styles (e.g. ''单件灰白格中袖西装''), sales-related information (e.g. '牛油果绿预售15天'); 76 | - The cases where there is more than one single item in one product (e.g. when a product includes a full suite of cloth, such as '白色上衣+黑色套装裙子'); 77 | - The internal complexity of text-based color information. Color-related information can be further decomposed into three components: 78 | - The lighting of a certain color (e.g. 深，浅，暗etc.); 79 | - The texture (e.g. 人字，格子etc.); 80 | - The color itself. 81 | 82 | We hereby propose to first decompose the optional tags into separate words using Jieba, to treat all kinds of information in a decomposed manner, so that it's possible for us to filter out the unrelated semantic information while keeping the information that matters. In this way, we obtain the following advantages: 83 | 84 | - We get the number of labels under control; 85 | - We keep the labels that matter by sorting the labels; 86 | - Our design of encoding cover several modalities of an image (lighting, texture, color, etc.) so that we made learning explainable by guiding the network to understand the separate concepts of different modalities. 87 | 88 | 89 | 90 | ## Method 91 | 92 | ### Method Overview 93 | 94 | ![method](figs/method.png) 95 | 96 | We propose a novel approach that pairs images and text tags through embedding matching, Embedding Matching for Image-Text pairing (EMIT). 97 | 98 | We first decompose the optional tags into separate words using Jieba and encode the optional tags and extract the text embeddings, which is a 0,1 vector. Then, a DenseNet-based feature extractor is trained on the text embeddings in a multi-label classification fashion, to match the image embedding to the text embedding. This is where the network learns about the separate semantic labels on this image. During inference, we extract the image embedding from the testing image and all text embeddings from the optional tags, and we match them by selecting the text embedding that has the smallest cross-entropy with the image embedding. 99 | 100 | Last, we further propose a heuristic-based bagging method that takes into account the constraint that all images from one product should make a full cover over the optional tags of this product. What we do is that we train several different networks using a different selection of semantic labels, data dropouts, etc., and obtain several models for inference. For each product, we reject the predictions from the models where this constraint was not met while keeping the predictions that meet this constraint. We show that by considering this constraint, we further boost the performance of our method. 101 | 102 | ### Method Details 103 | 104 | **Data processing.** The data processing procedure is crucial in our method. 105 | 106 | 1. Jieba word cutting. We decompose the messy optional tags into words (semantic labels) so that semantic information is decomposed. 107 | 2. Label sorting and filtering. We sort the semantic labels on the full dataset. Semantic labels that appear less than 5 times over the full dataset are considered noise and thus filtered. The dominating semantic labels are: "黑", "白", "蓝", "红", "绿", "灰" etc. However, we also get labels covering other semantic information such as "深", "浅", "格", "衣", "西装" etc. 108 | 3. Multi-label assigning. One image can have multiple labels, so that network will understand and disentangle different semantic information. For example, “灰白格中袖” gets {灰，白，格，中袖}. After this, our text embeddings look like the following: 109 | 110 | ```python 111 | "img_labels": {"543462160412_0.jpg": [9, 11], "543462160412_1.jpg": [9, 11], "543462160412_2.jpg": [24, 31, 41], "543462160412_3.jpg": [24, 31, 41], "543462160412_4.jpg": [3, 15], "543462160412_5.jpg": [3, 15], "543462160412_6.jpg": [4, 28, 59, 107], "543462160412_7.jpg": [4, 28, 59, 107], "543462160412_8.jpg": [49, 113]} 112 | ``` 113 | 114 | This process is rather fast. In practice, we do the Jieba word cutting, sorting, and filtering beforehand, save the top 64 dominating semantic labels to label set $ \mathcal S $, and generate the vector-like text-embedding on the fly. 115 | 116 | **Text embedding.** We retrieve text embedding $F_T$ by looking into the $\mathcal S$, and assigning 1 to $F_t [i]$ if $\mathcal S [i]$ is a substring in the optional tag. 117 | 118 | **Image embedding.** We reshape all images to 224*224 and use a DenseNet-like neural network as the backbone. In the last layer, we add a fully connected layer with 64 output channels indicating 64 different semantic labels that we are learning, and we would like the image embedding $F_I$ to match the text embedding on each corresponding channel. 119 | 120 | **Loss function.** We use cross-entropy loss. 121 | 122 | **Optimizer.** We use Adam optimizer with a learning rate of 1e-3. 123 | 124 | 125 | 126 | ## Experiments & Ablation Studies 127 | 128 | We train our method on the medium-sized dataset with batch size = 64. We use LR = 1e-3 and our method can converge when trained on a single GeForce 3090 GPU for about 30 minutes. 129 | 130 | With no bells and whistles, EMIT outperforms the baseline method by a large margin, with results shown in the table below. 131 | 132 | | Method | Acc | EM | 133 | | ----------- | ---------- | ---------- | 134 | | TA_001 | 0.8673 | 0.6155 | 135 | | TA_000 | 0.8839 | 0.6243 | 136 | | EMIT (Ours) | **0.9477** | **0.8040** | 137 | 138 | We further do ablation experiments to validate the effectiveness of our design of multi-label text-embedding and our design of heuristic-based bagging. 139 | 140 | | Experiment | Semantic Label | Bagging | Acc | EM | 141 | | ---------- | -------------- | ------- | ---------- | ---------- | 142 | | 1 | Manual design | No | 0.8429 | 0.5519 | 143 | | 2 | Text embedding | No | 0.9231 | 0.7526 | 144 | | 3 (Ours) | Text embedding | Yes | **0.9477** | **0.8040** | 145 | 146 | (1) Experiment 1 is done as described in the assignment. We adopt the same DenseNet-like network structure as used in our method and only switched to manually designed color labels. In particular, we keep all the color labels that we adopt in our EMIT version and do a single-class classification on the dataset. Experiments 1 and 2 show that keeping not only the color labels but also other information like "深", "浅", "格", "衣", "西装" etc. and treating the problem as a multi-label classification benefits the performance. 147 | 148 | (2) We run experiment 2 several times with different selection of semantic labels, data dropouts, etc. The top-performing model reaches the accuracy in the table. However, from experiments 2 and 3 we can see that putting each model together by our heuristic-based bagging method further improves the accuracy by 2 percent and the EM by 5 percent. -------------------------------------------------------------------------------- /模式识别/readme.md: -------------------------------------------------------------------------------- 1 | # Pattern Recognition and Machine Learning 2 | 3 | Instructor: Xiaowo Wang, Xuegong Zhang 4 | 5 | Course Description: 6 | 7 | > This course introduces the basic concepts and methodology framework of pattern recognition and machine learning. It starts from an angle of statistical learning. The course will cover major methods of traditional regression, classification, and clustering, as well as support vector machines, neural networks, and introductory deep learning. Feature selection, representation as well as model assessment will also be covered. 8 | 9 | Grading: 10 | 11 | - 3 Credits. 12 | - Homework + Course Project + Final Exam (A4 cheat sheet allowed). 13 | 14 | Helin's Note: 15 | 16 | - My course project: EMIT_Embedding_Matching_for_Image_Text_pairing([Code release](https://github.com/HelinXu/EMIT); [Report](https://github.com/HelinXu/EMIT/blob/main/EMIT_Embedding_Matching_for_Image_Text_pairing.pdf)) 17 | - My cheat sheet: [Helin_PRML.pdf](./Helin_PRML.pdf). Use notability.app to [edit yourself](./Helin_PRML_notability.note) if you wish. -------------------------------------------------------------------------------- /模式识别/大作业说明v0_4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/模式识别/大作业说明v0_4.pdf -------------------------------------------------------------------------------- /系统工程导论/1解释性结构.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/系统工程导论/1解释性结构.pdf -------------------------------------------------------------------------------- /系统工程导论/1黑箱建模.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/系统工程导论/1黑箱建模.pdf -------------------------------------------------------------------------------- /系统工程导论/2主成分分析.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/系统工程导论/2主成分分析.pdf -------------------------------------------------------------------------------- /系统工程导论/2因子分析.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/系统工程导论/2因子分析.pdf -------------------------------------------------------------------------------- /系统工程导论/2聚类分析.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/系统工程导论/2聚类分析.pdf -------------------------------------------------------------------------------- /系统工程导论/3不确定性分析.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/系统工程导论/3不确定性分析.pdf -------------------------------------------------------------------------------- /系统工程导论/readme.md: -------------------------------------------------------------------------------- 1 | # Introduction to Systems Engineering 2 | 3 | Instructor: Jianming Hu 4 | 5 | Course Description: 6 | 7 | > The main contents of this course are composed of the following four parts. (1) Overview of the fundamentals and methods of systems engineering; (2) Qualitative modeling method, mainly teaching interpretive structure modeling (ISM) method; (3) Quantitative modeling method: mainly teaching black box modeling method, the basic principles of polynomial model and neural network model; (4) System analysis methods, mainly teaching univariate and multivariate regression analysis method, ill-linear regression analysis, principal component analysis, factor analysis and cluster analysis methods, and extensively introducing some new methods, such as SOM, MARS, STARMA, RPCA, deep learning, and so forth; (5) system decision-making analysis method, mainly teaching risk decision-making analysis method, multi-objective decision-making method, analytic hierarchy process (AHP) and group decision method; (6) overview of several modern optimization methods, including genetic algorithm, simulated annealing, particle swarm and ant algorithm, are introduced in combination with some examples. 8 | 9 | Grading: 10 | 11 | - 2 credits 12 | - 30% Homework, 70% Final Exam (in-class, open-book). 13 | 14 | Helin's Notes: 15 | 16 | - Exam contents have not varied much for years. Though you can find a collection of past exam problems that are very informative, please DO NOT print them or write them down in your notebook and bring them to your exam, otherwise, this will be noticed during your exam. 17 | - I have formatted all slides and released the printable pdf here. They have trustworthy coverage of the final exam. Only bring the printed slides and your actual course notes to your exam. -------------------------------------------------------------------------------- /自控/readme.md: -------------------------------------------------------------------------------- 1 | # Theory of Automatic Control 2 | 3 | Instructor: Qianchuan Zhao, Chao Shang 4 | 5 | Grading: 6 | 7 | - 30% Homework, 70% Midterm + Final Exams. 8 | - 5 credits. 9 | 10 | Helin's Notes: 11 | 12 | - Plenty of past exams can be found. Very informative. -------------------------------------------------------------------------------- /自控/自动控制理论-现代部分知识点总结.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/自控/自动控制理论-现代部分知识点总结.pdf -------------------------------------------------------------------------------- /自控/自控教学安排2021年秋20210910.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HelinXu/THUDA-9/c13a573fea8cb5345878364895c9f42e24435e7a/自控/自控教学安排2021年秋20210910.pdf -------------------------------------------------------------------------------- /计网/readme.md: -------------------------------------------------------------------------------- 1 | # Computer Networks and Applications 2 | 3 | Instructor: Qingshan Jia 4 | 5 | Grading: 6 | 7 | - 3 credits. 8 | - 20% Homework, 20% Experiments, 10% Project, 50% Final Exam (close-book). 9 | 10 | Helin's Notes: 11 | 12 | - Past exams can be found and are very informative. 13 | - Yes, there are a lot of things to memorize, and they will appear in the exam! 14 | - I feel my final grade has been curved up. Thanks, Mr. Jia! 15 | 16 | -------------------------------------------------------------------------------- /运筹学/readme.md: -------------------------------------------------------------------------------- 1 | # Operations Research 2 | 3 | Instructor: Huangang Wang 4 | 5 | Grading: 6 | 7 | - 3 credits 8 | - Homework + Midterm and Final Exams (close-book). 9 | 10 | Helin's Note: 11 | 12 | - Past exams can be found and are informative. --------------------------------------------------------------------------------