├── README-zh.md
├── README-zh.org
└── README.md
/README-zh.md:
--------------------------------------------------------------------------------
1 |
2 | # Table of Contents
3 |
4 | 1. [programming languages](#orgf967f91)
5 | 2. [基础知识](#org08afe60)
6 | 1. [SuperCowPowers/data\_hacking: Data Hacking Project](#org34a1e9c)
7 | 2. [机器学习库](#orgea9c3c4)
8 | 3. [相关学习和教程](#org462cdfa)
9 | 4. [研究](#orgcf723c7)
10 | 1. [JohnLaTwC/Shared: Shared Blogs and Notebooks](#org6c54c28):research:windows:
11 | 2. [lbnl-cybersecurity/ddos-detection](#orgad75342):DDOS:
12 | 5. [Osuqery](#org740ed40)
13 | 6. [Sysmon/syslog/or any system's logs](#orgee69060)
14 | 7. [Zeek](#orga622085)
15 | 1. [SuperCowPowers/zat: Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, and Spark](#orge0dcd1e)
16 | 2. [tenzir/threatbus: 🚌 The missing link to connect open-source threat intelligence tools.](#org727acfc)
17 | 3. [mitre-attack/bzar: A set of Zeek scripts to detect ATT&CK techniques.](#orgf975e0c)
18 | 8. [搜索和展示平台](#org0e92733)
19 | 1. [Elk](#org5632b2b)
20 | 2. [Grafana: The open observability platform | Grafana Labs](#org14538c9)
21 |
22 |
23 |
24 |
25 |
26 | # programming languages
27 |
28 | - Python
29 |
30 |
31 |
32 |
33 |
34 |
35 |
36 |
37 |
38 |
39 |
40 | | machine learn |
41 | |
42 |
43 |
44 |
45 |
46 |
47 | | scipy |
48 | |
49 |
50 |
51 |
52 |
53 | | Numpy |
54 | |
55 |
56 |
57 |
58 |
59 | | tensorflow |
60 | |
61 |
62 |
63 |
64 |
65 | | pandas |
66 | |
67 |
68 |
69 |
70 |
71 | | matplotlib |
72 | plot |
73 |
74 |
75 |
76 |
77 | | clx |
78 | Important |
79 |
80 |
81 |
82 |
83 | | fastai |
84 | |
85 |
86 |
87 |
88 |
89 | - Julia
90 |
91 |
92 |
93 |
94 |
95 |
96 |
97 |
98 |
99 |
100 |
101 | | machine learn |
102 | |
103 |
104 |
105 |
106 |
107 |
108 | | flux |
109 | |
110 |
111 |
112 |
113 |
114 | | cudaArray |
115 | GPU |
116 |
117 |
118 |
119 |
120 | - Haskell
121 | - Data parse/query
122 |
123 | - R
124 |
125 |
126 |
127 |
128 |
129 |
130 |
131 |
132 |
133 |
134 |
135 | | package name |
136 | role |
137 |
138 |
139 |
140 |
141 |
142 | | ggplot2 |
143 | plot |
144 |
145 |
146 |
147 |
148 | | tidyverse |
149 | |
150 |
151 |
152 |
153 |
154 | | xts |
155 | timeline |
156 |
157 |
158 |
159 |
160 | | timetk |
161 | timeline |
162 |
163 |
164 |
165 |
166 |
167 |
168 |
169 | # 基础知识
170 |
171 | - 须知
172 | 这里不仅仅为 `logs 日志作为分析源头` 可以使用脚本分析 PDF binary 任何分析可以使用机器学习相关的库。如 [Maldoc Analysis With xlm-deobfuscator - YouTube](https://www.youtube.com/watch?v=Y7IP0pksEb8) 可以利用任何脚本 进行安全相关的 `数值` 分析。这里不单一局限在日志。
173 | - [Didier Stevens | (blog 'DidierStevens)](https://blog.didierstevens.com/) 分析脚本模范展示
174 |
175 | 1. 相关输出脚本以 jupyter notebook 为展示页面
176 |
177 | 1. Notebook 要有相关分析的说明
178 |
179 | - 数据范畴不限 SIEM 支持的。可以参考 SIEM 要求数据分析范围类型
180 | - [TonyPhipps/SIEM: SIEM Tactics, Techiques, and Procedures](https://github.com/TonyPhipps/SIEM)
181 |
182 |
183 |
184 |
185 | ## ✰ Important [SuperCowPowers/data\_hacking: Data Hacking Project](https://github.com/SuperCowPowers/data_hacking)
186 |
187 | 可任选一个主题作为当前的分析主题。提交内容结构如上 repo 一样
188 |
189 | 1. data logs
190 |
191 | 1. scripts
192 |
193 | 1. environment requirements
194 |
195 | 1. Notebook overview
196 |
197 | 阅读掌握:
198 |
199 | - [infosec-jupyterthon/docs at master · OTRF/infosec-jupyterthon](https://github.com/OTRF/infosec-jupyterthon/tree/master/docs)
200 |
201 | - [hunters-forge/ThreatHunter-Playbook: A Threat hunter's playbook to aid the development of techniques and hypothesis for hunting campaigns.](https://github.com/hunters-forge/ThreatHunter-Playbook)
202 |
203 |
204 |
205 |
206 | ## 机器学习库
207 |
208 | - [rapidsacademy/tutorials/security/tour at master · RAPIDSAcademy/rapidsacademy](https://github.com/RAPIDSAcademy/rapidsacademy/tree/master/tutorials/security/tour) :ML:GPU:
209 |
210 | - [frapidsai/clx: A collection of RAPIDS examples for security analysts, data scientists, and engineers to quickly get started applying RAPIDS and GPU acceleration to real-world cybersecurity use cases.](https://github.com/rapidsai/clx)
211 |
212 | 基于 GPU 的攻击安全分析，建议使用的库
213 |
214 | - [clx/notebooks at branch-0.15 · rapidsai/clx](https://github.com/rapidsai/clx/tree/branch-0.15/notebooks)
215 |
216 | 提交标准参考 notebooks
217 |
218 |
219 |
220 |
221 | # ✰ Important 相关学习和教程
222 |
223 | 不限如下：
224 |
225 | - [Kayzaks/HackingNeuralNetworks: A small course on exploiting and defending neural networks](https://github.com/Kayzaks/HackingNeuralNetworks)
226 |
227 | - [course-v4/nbs at master · fastai/course-v4](https://github.com/fastai/course-v4/tree/master/nbs)
228 |
229 | - [ogrisel/parallel\_ml\_tutorial: Tutorial on scikit-learn and IPython for parallel machine learning](https://github.com/ogrisel/parallel_ml_tutorial)
230 |
231 | - [fastai/courses/dl1 at master · fastai/fastai](https://github.com/fastai/fastai/tree/master/courses/dl1)
232 |
233 | - [https://tools.netsa.cert.org/silk/analysis-handbook.pdf](https://tools.netsa.cert.org/silk/analysis-handbook.pdf) [网络分析基础]
234 |
235 |
236 |
237 |
238 | # 研究
239 |
240 |
241 |
242 |
243 | ## [JohnLaTwC/Shared: Shared Blogs and Notebooks](https://github.com/JohnLaTwC/Shared) :research:windows:
244 |
245 |
246 |
247 |
248 | ## [lbnl-cybersecurity/ddos-detection](https://github.com/lbnl-cybersecurity/ddos-detection) :DDOS:
249 |
250 |
251 |
252 |
253 | # Osuqery
254 |
255 | 提交说明：
256 |
257 | 1. osquery logs
258 |
259 | 1. osquery config files
260 |
261 | 1. notebook
262 |
263 | 1. environment requirements(Python or R or something else)
264 |
265 |
266 |
267 |
268 | # Sysmon/syslog/or any system's logs
269 |
270 | 提交要求：
271 |
272 | 可以分析任何日志相关的安全数值的分析。
273 |
274 | 1. data logs
275 |
276 | 1. notebook
277 |
278 | 1. environment requirements(Python or R or something else)
279 | - parsing script
280 |
281 | - query rules
282 |
283 |
284 |
285 |
286 | # Zeek
287 |
288 | zeek 分析提交规范样例 [stratosphereips/IRC-Behavioral-Analysis](https://github.com/stratosphereips/IRC-Behavioral-Analysis)
289 |
290 | 1. Zeek script
291 |
292 | 1. environment dependences
293 |
294 | 1. analysis Notebook
295 |
296 |
297 |
298 |
299 | ## ☞ TODO [SuperCowPowers/zat: Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, and Spark](https://github.com/SuperCowPowers/zat)
300 |
301 | - 目的： 利用 zat 实现各种 zeek logs 的分析和解析
302 |
303 |
304 |
305 |
306 | ## ☞ TODO [tenzir/threatbus: 🚌 The missing link to connect open-source threat intelligence tools.](https://github.com/tenzir/threatbus)
307 |
308 |
309 |
310 |
311 | ## [mitre-attack/bzar: A set of Zeek scripts to detect ATT&CK techniques.](https://github.com/mitre-attack/bzar)
312 |
313 |
314 |
315 |
316 | # 搜索和展示平台
317 |
318 |
319 |
320 |
321 | ## Elk
322 |
323 | 提交需求：
324 |
325 | 1. 解析日志的 elk config files
326 |
327 | 1. dashboard json (import output)
328 |
329 | 1. dashboard plugin
330 | 能够编写 kibana 的图表插件
331 |
332 | 1. [Yelp/elastalert: Easy & Flexible Alerting With ElasticSearch](https://github.com/Yelp/elastalert)
333 | 提供基于 alert 实现的部分事件报警。给相关数值说明
334 |
335 | - [HASecuritySolutions/VulnWhisperer: Create actionable data from your Vulnerability Scans](https://github.com/HASecuritySolutions/VulnWhisperer)
336 |
337 | 可脱离 Kibana 实现展示功能
338 |
339 |
340 |
341 |
342 | ## [Grafana: The open observability platform | Grafana Labs](https://grafana.com/)
343 |
344 | 提交需求：
345 |
346 | 1. 解析日志的 config files
347 |
348 | 1. dashboard json (import output)
349 |
350 | 1. dashboard plugin
351 | 能够编写 grafana 的图表插件
352 |
353 |
--------------------------------------------------------------------------------
/README-zh.org:
--------------------------------------------------------------------------------
1 | #+TITLE: Infosec-jupyter-lab
2 | #+AUTHOR: GTrunSec
3 | #+EMAIL: gtrunsec@hardenedlinux.org
4 | #+DATE: 13 June 2020
5 | #+DESCRIPTION:
6 | #+KEYWORDS:
7 | #+LANGUAGE: en cn
8 | #+SETUPFILE: ~/Documents/my-blog/public/theme/org-html-themes/setup/theme-readtheorg.setup
9 | #+hugo_publishdate: (0 5)
10 | #+hugo_auto_set_lastmod: t
11 | #+HUGO_categories:
12 | #+HUGO_tags:
13 | #+OPTIONS: H:3 num:t toc:t \n:nil @:t ::t |:t ^:nil -:t f:t *:t <:t
14 | #+SELECT_TAGS: export
15 | #+EXCLUDE_TAGS: noexport
16 | #+TAGS: research windows linux DDOS
17 | * programming languages
18 | - Python
19 |
20 |
21 | | machine learn | |
22 | |---------------+-----------|
23 | | scipy | |
24 | | Numpy | |
25 | | tensorflow | |
26 | | pandas | |
27 | | matplotlib | plot |
28 | | clx | Important |
29 | | fastai | |
30 |
31 | - Julia
32 |
33 |
34 | | machine learn | |
35 | |---------------+---|
36 | | flux | |
37 | | cudaArray | GPU |
38 |
39 | - Haskell
40 |
41 | - Data parse/query
42 |
43 |
44 |
45 | - R
46 |
47 |
48 | | package name | role |
49 | |--------------+----------|
50 | | ggplot2 | plot |
51 | | tidyverse | |
52 | | xts | timeline |
53 | | timetk | timeline |
54 |
55 | * 基础知识
56 | - 须知
57 | 这里不仅仅为 ~logs 日志作为分析源头~ 可以使用脚本分析 PDF binary 任何分析可以使用机器学习相关的库。如 [[https://www.youtube.com/watch?v=Y7IP0pksEb8][Maldoc Analysis With xlm-deobfuscator - YouTube]] 可以利用任何脚本 进行安全相关的 ~数值~ 分析。这里不单一局限在日志。
58 |
59 | - [[https://blog.didierstevens.com/][Didier Stevens | (blog 'DidierStevens)]] 分析脚本模范展示
60 |
61 |
62 |
63 | 1. 相关输出脚本以 jupyter notebook 为展示页面
64 |
65 |
66 | 2. Notebook 要有相关分析的说明
67 |
68 |
69 | - 数据范畴不限 SIEM 支持的。可以参考 SIEM 要求数据分析范围类型
70 |
71 | - [[https://github.com/TonyPhipps/SIEM][TonyPhipps/SIEM: SIEM Tactics, Techiques, and Procedures]]
72 |
73 |
74 |
75 | ** ✰ Important [[https://github.com/SuperCowPowers/data_hacking][SuperCowPowers/data_hacking: Data Hacking Project]]
76 | 可任选一个主题作为当前的分析主题。提交内容结构如上 repo 一样
77 |
78 | 1. data logs
79 |
80 |
81 | 2. scripts
82 |
83 |
84 | 3. environment requirements
85 |
86 |
87 | 4. Notebook overview
88 |
89 |
90 | 阅读掌握:
91 |
92 | - [[https://github.com/OTRF/infosec-jupyterthon/tree/master/docs][infosec-jupyterthon/docs at master · OTRF/infosec-jupyterthon]]
93 |
94 |
95 | - [[https://github.com/hunters-forge/ThreatHunter-Playbook][hunters-forge/ThreatHunter-Playbook: A Threat hunter's playbook to aid the development of techniques and hypothesis for hunting campaigns.]]
96 |
97 |
98 | ** 机器学习库
99 | - [[https://github.com/RAPIDSAcademy/rapidsacademy/tree/master/tutorials/security/tour][rapidsacademy/tutorials/security/tour at master · RAPIDSAcademy/rapidsacademy]] :ML:GPU:
100 |
101 |
102 | - [[https://github.com/rapidsai/clx][frapidsai/clx: A collection of RAPIDS examples for security analysts, data scientists, and engineers to quickly get started applying RAPIDS and GPU acceleration to real-world cybersecurity use cases.]]
103 |
104 |
105 | 基于 GPU 的攻击安全分析，建议使用的库
106 |
107 | - [[https://github.com/rapidsai/clx/tree/branch-0.15/notebooks][clx/notebooks at branch-0.15 · rapidsai/clx]]
108 |
109 |
110 | 提交标准参考 notebooks
111 |
112 | * ✰ Important 相关学习和教程
113 | 不限如下：
114 |
115 | - [[https://github.com/Kayzaks/HackingNeuralNetworks][Kayzaks/HackingNeuralNetworks: A small course on exploiting and defending neural networks]]
116 |
117 |
118 | - [[https://github.com/fastai/course-v4/tree/master/nbs][course-v4/nbs at master · fastai/course-v4]]
119 |
120 |
121 | - [[https://github.com/ogrisel/parallel_ml_tutorial][ogrisel/parallel_ml_tutorial: Tutorial on scikit-learn and IPython for parallel machine learning]]
122 |
123 |
124 | - [[https://github.com/fastai/fastai/tree/master/courses/dl1][fastai/courses/dl1 at master · fastai/fastai]]
125 |
126 |
127 | - [[https://tools.netsa.cert.org/silk/analysis-handbook.pdf][https://tools.netsa.cert.org/silk/analysis-handbook.pdf]] [网络分析基础]
128 |
129 |
130 | * 研究
131 | ** [[https://github.com/JohnLaTwC/Shared][JohnLaTwC/Shared: Shared Blogs and Notebooks]] :research:windows:
132 | ** [[https://github.com/lbnl-cybersecurity/ddos-detection][lbnl-cybersecurity/ddos-detection]] :DDOS:
133 |
134 | * Osuqery
135 | 提交说明：
136 |
137 | 1. osquery logs
138 |
139 |
140 | 2. osquery config files
141 |
142 |
143 | 3. notebook
144 |
145 |
146 | 4. environment requirements(Python or R or something else)
147 |
148 |
149 | * Sysmon/syslog/or any system's logs
150 | 提交要求：
151 |
152 | 可以分析任何日志相关的安全数值的分析。
153 |
154 | 1. data logs
155 |
156 |
157 | 2. notebook
158 |
159 |
160 | 3. environment requirements(Python or R or something else)
161 |
162 | - parsing script
163 |
164 |
165 |
166 | - query rules
167 |
168 |
169 | * Zeek
170 | zeek 分析提交规范样例 [[https://github.com/stratosphereips/IRC-Behavioral-Analysis][stratosphereips/IRC-Behavioral-Analysis]]
171 |
172 | 1. Zeek script
173 |
174 |
175 | 2. environment dependences
176 |
177 |
178 | 3. analysis Notebook
179 |
180 |
181 | ** ☞ TODO [[https://github.com/SuperCowPowers/zat][SuperCowPowers/zat: Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, and Spark]]
182 | - 目的： 利用 zat 实现各种 zeek logs 的分析和解析
183 |
184 |
185 | ** ☞ TODO [[https://github.com/tenzir/threatbus][tenzir/threatbus: 🚌 The missing link to connect open-source threat intelligence tools.]]
186 |
187 | ** [[https://github.com/mitre-attack/bzar][mitre-attack/bzar: A set of Zeek scripts to detect ATT&CK techniques.]]
188 |
189 | * 搜索和展示平台
190 | ** Elk
191 | 提交需求：
192 |
193 | 1. 解析日志的 elk config files
194 |
195 |
196 | 2. dashboard json (import output)
197 |
198 |
199 | 3. dashboard plugin
200 | 能够编写 kibana 的图表插件
201 |
202 |
203 | 2. [[https://github.com/Yelp/elastalert][Yelp/elastalert: Easy & Flexible Alerting With ElasticSearch]]
204 | 提供基于 alert 实现的部分事件报警。给相关数值说明
205 |
206 |
207 | - [[https://github.com/HASecuritySolutions/VulnWhisperer][HASecuritySolutions/VulnWhisperer: Create actionable data from your Vulnerability Scans]]
208 |
209 |
210 | 可脱离 Kibana 实现展示功能
211 |
212 | ** [[https://grafana.com/][Grafana: The open observability platform | Grafana Labs]]
213 | 提交需求：
214 |
215 | 1. 解析日志的 config files
216 |
217 |
218 | 2. dashboard json (import output)
219 |
220 |
221 | 3. dashboard plugin
222 | 能够编写 grafana 的图表插件
223 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Infosec-jupyter-lab
--------------------------------------------------------------------------------