├── .gitignore ├── .gitlab-ci.yml ├── .markdownlint.json ├── CacheMappingDemo.py ├── CacheReplacementDemo.py ├── LICENSE ├── README.md ├── big-data-storage-experiment.md ├── bio.md ├── ceph-deployment-simple.md ├── computer-system-design-lab2.md ├── data-center-intro.md ├── data-center-pre.md ├── image.png ├── images ├── 01-Global-Overview-DataReportal-20200422-Digital-2020-April-Global-Statshot-Report-Slide-7.png ├── 05-Changes-in-Connected-Behaviours-DataReportal-20200422-Digital-2020-April-Global-Statshot-Report-Slide-11.png ├── 2u-rack-server.jpg ├── 3-replica.png ├── 3-tier-memory.png ├── 3steps.png ├── 439Microsoft_Project-Natick-servers-1068x601.jpg ├── 451figures.original.png ├── 690px-SETI-home_ScrSaver.jpg ├── ANSI_TIA-942_Standard.png ├── APM-Diagram-1.webp ├── Basic-structure-of-queueing-models.png ├── Block-Storage-vs-File-Storage.jpg ├── CIS_Q320.jpg ├── CPU-Utilization-on-a-vSphere-Host-Before-and-After-Consolidation.jpg ├── Cloud-Computing-as-a-Service-Revenue.png ├── Cloudfront-Map_9.24_2x.png ├── Data-Centers-top10.webp ├── Erklarung_M10_72ppi_EN.png ├── Facebook-s-Arctic-Server-Farm-1.jpg ├── Fraction-of-US-data-center-electricity-use-in-2014.png ├── Globa-and-China-DC-trend.png ├── Global-electricity-demand-of-consumer-devices-2010-2030.png ├── Global-electricity-demand-of-data-centers-2010-2030.png ├── GooglePUE-800.png ├── Historical-energy-usage.jpeg ├── IDC_DataSphere.png ├── IDC_edge_to_core.png ├── Latency-Numbers-Every-Programmer-Should-Know.png ├── Massively-parallel-processing.webp ├── MemoryHierarchy.png ├── Micron-Computex-2021-Keynote-Memory-Pyramid.jpg ├── OSD-Metadata.webp ├── OSD-vs-block-based.webp ├── Object-Storage.webp ├── PI-Controller.png ├── QoS-managed-distributed-storage-system.gif ├── Replication-and-Erasure-Coding.jpg ├── Restful-Web-Services.png ├── S3-decision-flow-chart_1.png ├── SCSI-Architecture-OSD.webp ├── SNIA-tail-latency.webp ├── SQL-and-Oracle-Database-Consolidation-Reduces-Software-Costs-by-70-Percent.jpg ├── Screen-Shot-2015-10-04-at-6.15.24-PM-small.jpg ├── Screen-Shot-2015-10-04-at-6.15.24-PM.png ├── Source-of-Latency-Solutions.png ├── Source-of-Latency.png ├── Stochastic-Fairness-Queuing.png ├── Tahoe-Reno-1.2e16d0ba.fill-1200x630.jpg ├── Telecom-Fraud.png ├── Thelongtailshift.webp ├── Top-Four-Cloud-Infrastructure-Providers.jpg ├── WSC-in-CAQA6e.png ├── WYG0047-0179d.png ├── Web-Application-Architecture.png ├── access-pattern-real.png ├── access-pattern.png ├── active-object-storage-award.png ├── active-object-storage.png ├── ai-market-2024.jpg ├── ali-qiandaohu-small.jpg ├── ali-qiandaohu.jpg ├── aliyun-top5.jpg ├── aliyun-zhangbei-aidc.jpg ├── amdahl-law.png ├── approximation-by-betweenness-centrality.jpg ├── architecture-hpc-cfd.png ├── autoscaling-architecture5.png ├── aws-region.png ├── aws_proxy_s3_create_methods_on_folder.png ├── azure-data-center-global-map.webp ├── baidu-index-datacenter.jpg ├── beyond-server-consolidation.png ├── block-diagram-of-feedback-control-system.drawio ├── block-diagram-of-feedback-control-system.svg ├── byte-dance-hedged-read.webp ├── cache-block-address.png ├── cache-memory-storage.png ├── cache-working-flow.png ├── caqa5e-ch6-wsc-p0.gif ├── caqa5e-ch6-wsc-p1.gif ├── caqa5e-ch6-wsc-p2.gif ├── caqa6e.jpg ├── ceph-graphic.png ├── ceph-stack.png ├── cephDiagramm.webp ├── cfq-cgroup.png ├── cgroup-vs-qemu.png ├── chaos.jpg ├── china-computing-power.png ├── china-datacenter-map.png ├── china-datacenter-pue-area.png ├── china-datacenter-pue.png ├── china-multi-cloud.png ├── china-new-construction.png ├── class-survey.jpg ├── clip-art-robotic-arm.png ├── cloud-dc-locations@2x.png ├── cloud-dc-numbers@2x.png ├── cloud-everywhere.jpg ├── cloud-fog-edge_infographic.jpg ├── cloud-fog-edge_infographic.webp ├── cloud-provider-location-map.png ├── cloud-storage-performance-factors.png ├── cloud-storage-performance-prediction.png ├── cloud-storage-request-analysis.png ├── cloud-storage.jpg ├── cmp-consolidated-servers.png ├── comp-storage-01.png ├── comp-storage-02.png ├── compressed-sparse-row.jpg ├── constrained-optimization-l.jpg ├── course-barcode.png ├── cpu-operations.png ├── csapp3e-cover.jpg ├── dac23-bg1.png ├── dac23-bg2.png ├── dac23-bg3.png ├── data-centre-fuel-cell-efficiency.png ├── datacenter-scale-evaluation.png ├── datacenter-trend.png ├── deepwalk.png ├── dft-acc7-small.jpg ├── dft-acc7.jpg ├── dilemma-on-data-loading.jpg ├── dilemma-on-skewness.jpg ├── direct-mapping.png ├── distributed-storage.png ├── edge-centric.jpg ├── emergency.jpg ├── erasure-code.png ├── evaluation-on-losc.jpg ├── evolving-graph-apps.png ├── evolving-graph-logic.png ├── evolving-graph-research.png ├── example-sssp-parallel-bfs-in-pregel-l.jpg ├── failure-trends.png ├── flops-bytes-unbalance.jpg ├── fund-tracking.png ├── github-object-storage.png ├── google-cloud-platform-infra-map-small.jpg ├── google-cloud-platform-infra-map.jpg ├── google-maps2-576x1024-small.jpg ├── google-maps2-576x1024.png ├── graph-app-1.jpg ├── graph-processing-systems.png ├── graph_logbase10_ms.png ├── graphlab-consistency-model.jpg ├── graphlab-framework.jpg ├── graphs-are-everywhere.jpg ├── green-sustainable-data-center-with-earth.jpg ├── hedged-requests.png ├── here-are-the-10-largest-data-center-providers-in-the-world.jpg ├── homogeneous-server-queue.jpg ├── hpc_schematic.png ├── huge-evaluation-0.jpg ├── huge-evaluation-1.jpg ├── huge-evaluation-2.jpg ├── huge-hrw.jpg ├── huge-workflow.jpg ├── iccd22-bg1.png ├── iccd22-bg2.png ├── iccd22-bg3.png ├── iccd22-bg4.png ├── idc-computing-power-2021.jpg ├── idc-datasphere-2020.webp ├── intel-xeon-2-small.jpg ├── intel-xeon-2.jpg ├── it-was-nature-for-change.png ├── keep-calm.png ├── krongen256bfs.jpg ├── latency-gap.png ├── latency-sla.png ├── law-of-the-minimum.jpg ├── m1-m2.png ├── mclock.png ├── measuring-percentile-latency.svg ├── memory-hierarchy.jpg ├── memory-hierarchy.png ├── minio_architecture_diagram.svg ├── model-size.png ├── model-system-size.jpg ├── multi-tenancy-degrees.png ├── multitenancy-web.png ├── nanjing-aidc.jpg ├── neo4j-intro.png ├── network-representative-learning.png ├── object-vs-traditional.png ├── odcc-chart1.png ├── odcc-chart5.png ├── openstack-frontend-qos.png ├── osd-2007.png ├── osd-commands.PNG ├── osd-commands.png ├── pagerank-pregel.jpg ├── pagerank-result.jpg ├── pagerank.webp ├── performance-isolation-1.png ├── performance-isolation-2.png ├── placement-of-inactive-vertexes.jpg ├── power-law-internet.jpg ├── power-law-twitter.jpg ├── prometheus-architecture.png ├── provisioning.png ├── qq-group-2021.png ├── qq-group-2022.jpg ├── qq-group-2023.jpg ├── qq-group-2024.jpg ├── rack-full.jpg ├── random-access-problem.jpg ├── range-intl-langfang.jpg ├── reading-list-2023.png ├── reading-list-2024.png ├── red-hat-ceph-storage.png ├── reinforcement-learning.drawio ├── reinforcement-learning.svg ├── s3-storage-classes.png ├── sc23-bg1.png ├── sc23-bg2.png ├── scaleout-diagram.jpg ├── sequential-vs-random.jpg ├── server-consolidation-scenario.drawio ├── server-consolidation-scenario.svg ├── server-consolidation.drawio ├── server-consolidation.svg ├── server-motherboard-small.jpg ├── server-motherboard.jpg ├── server-racks.png ├── set-associative-mapping.png ├── sfk-big-english.jpg ├── shangtang-aidc.webp ├── shortest-path.jpg ├── simulated-sessions.png ├── size-affects-latency.png ├── sla-slo-sli-relation.png ├── slo-spec.png ├── slo-vs-sla-vs-sli-1.jpg ├── snapshot-vs-log.png ├── sowalker-background.jpg ├── sowalker-contribution.jpg ├── sowalker-evaluation-1.jpg ├── sowalker-evaluation-2.jpg ├── sowalker-motivation.jpg ├── spark-arch.png ├── spark-graphx-app.gif ├── spark-graphx-property_graph.png ├── spark-graphx-uml.jpg ├── subgraph-construction-is-important.jpg ├── swift_network_diagram-1.png ├── tail-latency-with-scaling-and-outliers.png ├── tencent-cloud-2021.png ├── tencent-fact-2016.png ├── tencent-guizhou.jpg ├── tencent-qingyuan.jpg ├── text-book.jpg ├── the-long-tail.jpg ├── toolbox-small.png ├── toolbox.png ├── understanding-disk-failure.png ├── us_datacenters.jpg ├── utah-data-center-entrance.jpg ├── val-nas18.jpg ├── vertex-activity-trend.jpg ├── vertex-centric.jpg ├── vertex-computing.png ├── vertex-edge-computing.png ├── vhost-target.png ├── web-request-tail.png ├── what-happens-online-in-60-seconds.png ├── white-flower-icon-5.jpg ├── white-flower-icon-6.png ├── who-moved-my-cheese.jpg ├── workload-plot-1.jpg ├── workload-plot-2.jpg ├── wsc-arch.drawio ├── wsc-arch.svg ├── xAI-Colossus-net.jpg ├── xAI-Colossus-rack.jpg ├── xAI-Colossus-url.jpg ├── xAI-Colossus-web.jpg ├── xAI-Colossus.jpeg ├── yi-zhuan.jpg └── 计算机系统结构（大数据2201-2202）.jpg ├── memory-hierarchy.md ├── multi-processor.md ├── object-storage.md ├── qos-guarantee.md ├── storage-system.md ├── system-for-cloud.md └── system-for-graph.md /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | media 3 | -------------------------------------------------------------------------------- /.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | stages: 2 | - marp 3 | - deploy 4 | 5 | marp-prepare: 6 | stage: marp 7 | image: busybox:uclibc 8 | script: 9 | - wget https://github.com/marp-team/marp-cli/releases/download/v4.0.3/marp-cli-v4.0.3-linux.tar.gz 10 | - tar xzvf marp-cli-v4.0.3-linux.tar.gz 11 | - mkdir bin 12 | - mv marp ./bin/ 13 | artifacts: 14 | paths: 15 | - bin/ 16 | 17 | pages: 18 | stage: deploy 19 | image: ubuntu:focal-20211006 20 | script: 21 | - bin/marp *.md 22 | - mkdir public 23 | - mv *.html public 24 | - mv images public 25 | artifacts: 26 | paths: 27 | - public 28 | rules: 29 | - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH 30 | -------------------------------------------------------------------------------- /.markdownlint.json: -------------------------------------------------------------------------------- 1 | { 2 | "MD013": false, 3 | "MD025": false, 4 | "MD033": false 5 | } -------------------------------------------------------------------------------- /CacheMappingDemo.py: -------------------------------------------------------------------------------- 1 | from manim import * 2 | 3 | class CacheMappingDemo(Scene): 4 | def construct(self): 5 | # 定义颜色主题 6 | colors = { 7 | "tag": YELLOW, 8 | "index": BLUE, 9 | "offset": GREEN, 10 | "cache_block": PURPLE, 11 | "memory_block": ORANGE, 12 | } 13 | 14 | # 第一部分：地址解析的通用结构 15 | title = Text("缓存地址解析", font_size=36).to_edge(UP) 16 | address_structure = self.show_address_structure(colors) 17 | self.play(Write(title), run_time=1) 18 | self.play(Create(address_structure), run_time=2) 19 | self.wait(2) 20 | 21 | # 第二部分：直接映射缓存 22 | self.play(FadeOut(title), FadeOut(address_structure)) 23 | direct_mapped_scene = self.direct_mapped_cache(colors) 24 | self.play(*direct_mapped_scene) 25 | self.wait(3) 26 | 27 | # 第三部分：组相联缓存 28 | self.clear_scene() 29 | set_associative_scene = self.set_associative_cache(colors) 30 | self.play(*set_associative_scene) 31 | self.wait(3) 32 | 33 | # 第四部分：全相联缓存 34 | self.clear_scene() 35 | fully_associative_scene = self.fully_associative_cache(colors) 36 | self.play(*fully_associative_scene) 37 | self.wait(3) 38 | 39 | # 最终对比总结 40 | self.clear_scene() 41 | self.show_comparison_table() 42 | 43 | def show_address_structure(self, colors): 44 | # 地址位分解示意图 45 | address_bits = VGroup( 46 | Rectangle(width=6, height=1, color=WHITE), 47 | Text("虚拟地址", font_size=24).next_to(Rectangle(), UP) 48 | ) 49 | tag = Rectangle(width=2, height=1, fill_color=colors["tag"], fill_opacity=0.5) 50 | index = Rectangle(width=2, height=1, fill_color=colors["index"], fill_opacity=0.5).next_to(tag, RIGHT, buff=0) 51 | offset = Rectangle(width=2, height=1, fill_color=colors["offset"], fill_opacity=0.5).next_to(index, RIGHT, buff=0) 52 | labels = VGroup( 53 | Text("Tag", color=colors["tag"]).scale(0.6).move_to(tag), 54 | Text("Index", color=colors["index"]).scale(0.6).move_to(index), 55 | Text("Offset", color=colors["offset"]).scale(0.6).move_to(offset) 56 | ) 57 | return VGroup(address_bits, tag, index, offset, labels) 58 | 59 | def direct_mapped_cache(self, colors): 60 | # 直接映射缓存结构 61 | cache = VGroup(*[Square(side_length=1, color=colors["cache_block"]) for _ in range(8)]).arrange(RIGHT, buff=0.2) 62 | cache.to_edge(LEFT) 63 | arrows = VGroup() 64 | mappings = [] 65 | for i in range(8): 66 | mem_block = Text(f"M{i}", color=colors["memory_block"]).scale(0.8) 67 | mem_block.next_to(cache[i], UP) 68 | arrow = Arrow(mem_block.get_bottom(), cache[i].get_top(), color=WHITE) 69 | mappings.append(VGroup(mem_block, arrow)) 70 | return [ 71 | Create(cache), 72 | LaggedStart(*[Create(m) for m in mappings], lag_ratio=0.2), 73 | Write(Text("直接映射：每个内存块固定映射到一个缓存行", font_size=24).to_edge(DOWN)) 74 | ] 75 | 76 | def set_associative_cache(self, colors): 77 | # 组相联缓存结构（以2路为例） 78 | sets = VGroup(*[VGroup(*[Square(side_length=1, color=colors["cache_block"]) for _ in range(2)]).arrange(RIGHT, buff=0.5) for _ in range(4)]) 79 | sets.arrange(DOWN, buff=0.5).to_edge(LEFT) 80 | mappings = [] 81 | for set_idx in range(4): 82 | for way in range(2): 83 | mem_block = Text(f"M{set_idx*2+way}", color=colors["memory_block"]).scale(0.6) 84 | mem_block.next_to(sets[set_idx][way], UP) 85 | arrow = Arrow(mem_block.get_bottom(), sets[set_idx][way].get_top(), color=WHITE) 86 | mappings.append(VGroup(mem_block, arrow)) 87 | return [ 88 | Create(sets), 89 | LaggedStart(*[Create(m) for m in mappings], lag_ratio=0.1), 90 | Write(Text("组相联：内存块可映射到指定组内的任意位置", font_size=24).to_edge(DOWN)) 91 | ] 92 | 93 | def fully_associative_cache(self, colors): 94 | # 全相联缓存结构 95 | cache = VGroup(*[Square(side_length=1, color=colors["cache_block"]) for _ in range(8)]).arrange_in_grid(4, 2, buff=0.2) 96 | cache.to_edge(LEFT) 97 | mappings = [] 98 | for i in range(8): 99 | mem_block = Text(f"M{i}", color=colors["memory_block"]).scale(0.8) 100 | mem_block.move_to(UP*2 + RIGHT*i) 101 | arrow = CurvedArrow(mem_block.get_bottom(), cache[i].get_top(), color=WHITE) 102 | mappings.append(VGroup(mem_block, arrow)) 103 | return [ 104 | Create(cache), 105 | LaggedStart(*[Create(m) for m in mappings], lag_ratio=0.2), 106 | Write(Text("全相联：内存块可以映射到任意缓存位置", font_size=24).to_edge(DOWN)) 107 | ] 108 | 109 | def show_comparison_table(self): 110 | # 对比表格 111 | table = Table( 112 | [["直接映射", "组相联", "全相联"], 113 | ["固定映射位置", "组内灵活映射", "完全自由映射"], 114 | ["速度快", "平衡速度与灵活性", "灵活性最高"], 115 | ["冲突率高", "中等冲突率", "无冲突"]], 116 | row_labels=[Text("类型"), Text("映射方式"), Text("速度"), Text("冲突率")] 117 | ).scale(0.6) 118 | self.play(Create(table), run_time=3) 119 | self.wait(3) 120 | 121 | def clear_scene(self): 122 | self.play(*[FadeOut(mob) for mob in self.mobjects]) -------------------------------------------------------------------------------- /CacheReplacementDemo.py: -------------------------------------------------------------------------------- 1 | from manim import * 2 | import random 3 | 4 | class CacheReplacementDemo(Scene): 5 | def __init__(self, **kwargs): 6 | super().__init__(**kwargs) 7 | # 颜色定义 8 | self.COLORS = { 9 | "hit": GREEN, 10 | "miss": RED, 11 | "cache_block": BLUE, 12 | "text": WHITE 13 | } 14 | # 缓存大小定义 15 | self.cache_size = 4 16 | 17 | def construct(self): 18 | # 全局参数 19 | access_sequence = [1, 2, 3, 4, 1, 5, 1, 2] # 可替换为其他测试序列 20 | 21 | # 创建初始缓存状态 22 | cache = self.create_cache(self.cache_size) 23 | access_flow = self.create_access_flow(access_sequence) 24 | self.play(Create(cache), Create(access_flow)) 25 | self.wait(1) 26 | 27 | # 三种策略对比演示 28 | strategies = ["LRU", "FIFO", "Random"] 29 | strategy_scenes = [ 30 | self.lru_strategy(cache.copy(), access_sequence.copy()), 31 | self.fifo_strategy(cache.copy(), access_sequence.copy()), 32 | self.random_strategy(cache.copy(), access_sequence.copy()) 33 | ] 34 | 35 | for i, strategy in enumerate(strategy_scenes): 36 | title = Text(f"策略: {strategies[i]}", font_size=36).to_edge(UP) 37 | self.play(Write(title)) 38 | self.play(*strategy["animations"]) # 使用 * 解包动画列表 39 | self.show_performance_stats(access_sequence, strategy["hits"], strategy["replacements"]) 40 | self.wait(2) 41 | self.clear_scene() 42 | 43 | # 实际应用场景分析 44 | self.real_world_scenarios() 45 | 46 | def create_cache(self, size): 47 | # 创建缓存可视化模块 48 | blocks = VGroup() 49 | for i in range(size): 50 | block = Square(side_length=1, color=self.COLORS["cache_block"]) 51 | label = Text(f"Block {i+1}", font_size=20).move_to(block) 52 | blocks.add(VGroup(block, label)) 53 | blocks.arrange(RIGHT, buff=0.5).shift(UP) 54 | return blocks 55 | 56 | def create_access_flow(self, sequence): 57 | # 创建访问序列可视化 58 | elements = [Text(str(num), font_size=24) for num in sequence] 59 | flow = VGroup(*elements).arrange(RIGHT, buff=0.3).shift(DOWN) 60 | return flow 61 | 62 | def lru_strategy(self, cache, sequence): 63 | # LRU策略动画 64 | used_order = [] 65 | hits = 0 66 | replacements = 0 67 | animations = [] 68 | 69 | for i, num in enumerate(sequence): 70 | # 命中检测动画 71 | hit, anims = self.highlight_access(cache, num) 72 | animations.extend(anims) 73 | if hit: 74 | used_order.remove(num) 75 | used_order.append(num) 76 | hits += 1 77 | else: 78 | # 替换动画 79 | if len(used_order) >= self.cache_size: 80 | lru = used_order.pop(0) 81 | anims = self.replace_block(cache, lru, num) 82 | animations.extend(anims) 83 | replacements += 1 84 | else: 85 | anims = self.add_new_block(cache, num) 86 | animations.extend(anims) 87 | used_order.append(num) 88 | return {"hits": hits, "replacements": replacements, "animations": animations} 89 | 90 | def fifo_strategy(self, cache, sequence): 91 | # FIFO策略动画 92 | queue = [] 93 | hits = 0 94 | animations = [] 95 | 96 | for num in sequence: 97 | hit, anims = self.highlight_access(cache, num) 98 | animations.extend(anims) 99 | if hit: 100 | hits += 1 101 | else: 102 | if len(queue) >= self.cache_size: 103 | oldest = queue.pop(0) 104 | anims = self.replace_block(cache, oldest, num) 105 | animations.extend(anims) 106 | else: 107 | anims = self.add_new_block(cache, num) 108 | animations.extend(anims) 109 | queue.append(num) 110 | return {"hits": hits, "replacements": len(queue)-self.cache_size, "animations": animations} 111 | 112 | def random_strategy(self, cache, sequence): 113 | # 随机策略动画 114 | current_blocks = [] 115 | hits = 0 116 | animations = [] 117 | 118 | for num in sequence: 119 | hit, anims = self.highlight_access(cache, num) 120 | animations.extend(anims) 121 | if hit: 122 | hits += 1 123 | else: 124 | if len(current_blocks) >= self.cache_size: 125 | victim = random.choice(current_blocks) 126 | anims = self.replace_block(cache, victim, num) 127 | animations.extend(anims) 128 | current_blocks.remove(victim) 129 | else: 130 | anims = self.add_new_block(cache, num) 131 | animations.extend(anims) 132 | current_blocks.append(num) 133 | return {"hits": hits, "replacements": len(current_blocks)-self.cache_size, "animations": animations} 134 | 135 | def highlight_access(self, cache, num): 136 | # 高亮显示访问过程 137 | animations = [] 138 | for block in cache: 139 | if block[1].text == str(num): 140 | animations.append(block.animate.set_color(self.COLORS["hit"])) 141 | animations.append(block.animate.set_color(self.COLORS["cache_block"])) 142 | return True, animations 143 | animations.append(Flash(cache[0], color=self.COLORS["miss"])) # 假设 cache 不为空 144 | return False, animations 145 | 146 | def replace_block(self, cache, old_num, new_num): 147 | # 执行替换动画 148 | animations = [] 149 | for block in cache: 150 | if block[1].text == str(old_num): 151 | new_label = Text(str(new_num), font_size=20).move_to(block[1]) 152 | animations.append(Transform(block[1], new_label)) 153 | return animations 154 | return animations 155 | 156 | def add_new_block(self, cache, num): 157 | # 添加新块动画 158 | animations = [] 159 | for block in cache: 160 | if block[1].text.startswith("Block"): # 假设初始块标签为 "Block 1", "Block 2", ... 161 | new_label = Text(str(num), font_size=20).move_to(block[1]) 162 | animations.append(Transform(block[1], new_label)) 163 | return animations 164 | return animations 165 | 166 | def show_performance_stats(self, sequence, hits, replacements): 167 | # 显示性能指标 168 | stats = VGroup( 169 | Text(f"访问次数: {len(sequence)}", font_size=24), 170 | Text(f"命中次数: {hits}", color=self.COLORS["hit"]), 171 | Text(f"替换次数: {replacements}", color=self.COLORS["miss"]) 172 | ).arrange(DOWN).to_edge(RIGHT) 173 | self.play(Write(stats)) 174 | self.wait(2) 175 | 176 | def real_world_scenarios(self): 177 | # 实际应用场景分析 178 | title = Text("现代应用中的替换策略挑战", font_size=36).to_edge(UP) 179 | self.play(Write(title)) 180 | 181 | scenarios = VGroup( 182 | Text("1. 图遍历算法（随机访问模式）", font_size=24), 183 | Text("2. 矩阵运算（顺序访问模式）", font_size=24), 184 | Text("3. 流式数据处理（时序相关性）", font_size=24) 185 | ).arrange(DOWN, aligned_edge=LEFT).shift(UP*0.5) 186 | 187 | questions = VGroup( 188 | Text("- LRU是否适合突发访问模式？", color=YELLOW), 189 | Text("- 如何设计混合替换策略？", color=YELLOW), 190 | Text("- 机器学习能否预测访问模式？", color=YELLOW) 191 | ).arrange(DOWN, aligned_edge=LEFT).shift(DOWN*0.5) 192 | 193 | self.play(Write(scenarios), run_time=2) 194 | self.wait(1) 195 | self.play(Write(questions), run_time=2) 196 | self.wait(3) 197 | 198 | def clear_scene(self): 199 | self.play(*[FadeOut(mob) for mob in self.mobjects]) -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright 2022 Shi Zhan 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 计算机系统结构相关课程资料 2 | 3 | CS210.566 4 | -------------------------------------------------------------------------------- /big-data-storage-experiment.md: -------------------------------------------------------------------------------- 1 | --- 2 | marp: true 3 | theme: gaia 4 | title: 大数据存储系统与管理实验 5 | # size: 4:3 6 | math: katex 7 | --- 8 | 9 | 10 | 11 | # 大数据存储系统与管理
对象存储技术实验 12 | 13 | **施展** 14 | 武汉光电国家研究中心 & 计算机学院 15 | 华中科技大学 16 | 2025-05-09 (第12周周五) 至 2025-05-30 (第15周周五) 17 | 19:00~22:20 @ 南一楼803 18 | 19 | --- 20 | 21 | ## 授课教师 22 | 23 | - **施展** 副教授，武汉光电国家研究中心，存储部，C523 24 | - 联系方式 25 | - EMail 26 | - 电话 027-87792450 27 | - 主页 28 | - 29 | - 30 | 31 | --- 32 | 33 | 34 | 35 | ## 背景知识 36 | 37 | ```python 38 | if not 学习过"缺失的一课"，至少第6讲: 39 | if not 独立维护过Github或Gitee等开放Git仓库: 40 | 随我进行课堂练习 41 | ``` 42 | 43 | [计算机教育中缺失的一课](https://missing-semester-cn.github.io/)，[官网](https://missing.csail.mit.edu/)，[B站](https://www.bilibili.com/video/BV1x7411H7wa/) 44 | 45 | 与本次课程相关的基础训练 46 | 47 | - 1/13: 课程概览与 shell 48 | - 1/14: Shell 工具和脚本 49 | - 1/22: 版本控制([Git](https://git-scm.com/download)) 50 | 51 | --- 52 | 53 | ## 学习内容 54 | 55 | - **Lab 0** (05-09) **准备** 56 | - 按照要求建立实验环境，熟悉基本工具环境 57 | - **Lab 1** (05-16) **实践** 58 | - 搭建使用对象存储系统，实践命令行与API基本功能 59 | - **Lab 2** (05-23) **分析** 60 | - 观测对象存储基本功能，尝试分析经典性能问题 61 | - **Lab 3** (05-30) **预测** 62 | - 采集观测典型性能指标，尝试建模进行预测 63 | 64 | --- 65 | 66 | ### Lab 0 建立实验环境 67 | 68 | 73 | 74 | - [背景知识](object-storage) 75 | - [实验参考](https://github.com/cs-course/obs-tutorial) 76 | - 系统配置苦手？ 77 | - 推荐用LLM工具辅助 78 | - 79 | - 80 | 81 | --- 82 | 83 | #### 实验报告（<学号>-lab.md）模板 84 | 85 | ```Markdown 86 | 姓名：XXX 87 | 班级：XXXX 班 88 | 学号：^U2022\d{5}$ 89 | 日期：2025-05-dd 90 | 91 | --- 92 | 93 | # 实验简介 94 | 95 | …介绍实验软硬件环境，实验目的和主要实验内容条目… 96 | 97 | # 实验内容 98 | 99 | ## 实验1：搭建对象存储 100 | 101 | …实验过程，基本功能测试… 102 | 103 | …(可选) 应用集成测试… 104 | 105 | ## 实验2：观测分析性能 106 | 107 | …实验案例，实验负载生成，实验观测，数据分析… 108 | 109 | ## (可选)实验3：尝试建模预测 110 | 111 | …求解实际问题，达成额外成就… 112 | 113 | # 实验小结 114 | 115 | …实验结论和收获… 116 | ``` 117 | 118 | --- 119 | 120 | ### Lab 1 搭建对象存储 121 | 122 | - 目标系统 123 | - [Minio](https://minio.io/)， 简便易上手 124 | - [mock-s3](https://github.com/ShiZhan/mock-s3)，从后台认识基本功能 125 | - [Ceph](https://ceph.com/)，生产环境标杆 126 | - 搭建示范 127 | - [Ceph单机版](https://gitee.com/shi_zhan/data-center-course/blob/master/ceph-deployment-simple.md) 128 | 129 | --- 130 | 131 | #### 实践基本功能 132 | 133 | 138 | 139 | - 在计算机领域中，[create, read, update, and delete (缩写为 CRUD)](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) 是访问持久存储的4项基本操作。 140 | 141 | | Operation | SQL | HTTP | 142 | | :-------------- | :---- | :---------------- | 143 | | Create | INSERT | PUT / POST | 144 | | Read (Retrieve) | SELECT | GET | 145 | | Update (Modify) | UPDATE | PUT / POST / PATCH | 146 | | Delete (Destroy) | DELETE | DELETE | 147 | 148 | - 通过典型界面访问在搭建的系统，执行上述典型操作 149 | - 客户端: [MinIO Client (mc)](https://github.com/minio/mc), [OSM](https://github.com/appscode/osm), [s3cmd](https://github.com/s3tools/s3cmd), [aws-{cli\|shell}](https://github.com/awslabs/aws-shell) 150 | - 编程API: [boto3](https://github.com/boto/boto3), [aws-c-s3](https://github.com/awslabs/aws-c-s3), [awssdk-rust](https://github.com/awslabs/aws-sdk-rust) 151 | 152 | --- 153 | 154 | #### 【扩展探索】应用集成 155 | 156 | 通过部署实际应用来了解对象存储在云环境中的角色 157 | 158 | - 个人云 [Nextcloud](https://github.com/nextcloud), [Seafile](https://www.seafile.com/home/), [zfile](https://github.com/zhaojun1998/zfile) 159 | - 图片管理 [Thumbor](http://thumbor.org/), [picfit](https://github.com/thoas/picfit) 160 | - …更丰富应用请同学们自行探索… 161 | 162 | --- 163 | 164 | ### Lab 2 观测分析性能 165 | 166 | 实验测试工具 167 | 168 | - [Warp](https://github.com/minio/warp)，[Minio官网参考](https://blog.min.io/how-to-benchmark-minio-warp-speedtest/#step-3-run-warp-and-launch-a-mixed-benchmark) 169 | - [s3bench](https://github.com/cs-course/s3bench), [benchio](https://github.com/giacomoguiulfo/benchio), [s3-benchmark](https://github.com/chinglinwen/s3-benchmark) , Go语言环境 170 | - [OSSperf](https://github.com/christianbaun/ossperf), 内部包装[s3cmd](https://github.com/s3tools/s3cmd) 171 | - [s3-bench-rs](https://github.com/SKTT1Ryze/s3-bench-rs) , Rust语言环境 172 | - 感谢 [@SKTT1Ryze](https://github.com/SKTT1Ryze) 同学提供，欢迎更多同学参与！ 173 | - [COSBench](https://github.com/intel-cloud/cosbench) , Java语言环境，功能全面但过于古老，仅供参考。 174 | 175 | --- 176 | 177 | #### 影响性能的主要因素 178 | 179 | 184 | 185 | 典型问题举例 186 | 187 | - **对象尺寸**如何影响性能? 188 | - 对于熟悉的某类应用，根据其数据访问特性，怎样适配对象存储最合适? 189 | - 如何全面观测**I/O延迟**? 190 | - 平均值与统计分布 191 | - 如果客户端爆满将怎样? 192 | - **请求并发数**如何同时影响延迟分布和吞吐率？如何保障服务质量？ 193 | - 横向扩展系统 (Scaling Out) 效果如何? 194 | - 向系统中追加**更多存储设备**或服务器 195 | 196 | --- 197 | 198 | #### 尾延迟问题 199 | 200 | 205 | 206 | - 尾延迟为什么很重要？ 207 | - [The Tail at Scale](https://dl.acm.org/doi/10.1145/2408776.2408794). Commun. ACM, 2013, 56(2): 74–80. 208 | - 可以用什么方法来分析？ 209 | - [Amdahl’s Law for Tail Latency](https://cacm.acm.org/research/amdahls-law-for-tail-latency/). Commun. ACM, 2018, 61(8): 65–72. 210 | - 读论文苦手？ 211 | - 继续用LLM工具辅助 212 | - 213 | - 214 | 215 | --- 216 | 217 | ### Lab 3 尝试建模预测 218 | 219 | - 为**一个应用**，结合其参数 220 | - 预测总体**尾延迟分布**，确定与其访存行为适配的对冲百分位 221 | - 为**一段负载**，结合其请求序列 222 | - 预测未来一段时间的**IOPS**，确定是否在未来使用对冲 223 | - 高IOPS尽量避免，低IOPS积极发起 224 | - 为**一段负载**，结合其参数序列 225 | - 预测未来一段时间的**尾延迟分布**，用于指导灵活对冲的百分位阈值 226 | - 范例: [构建多模态模型，生成主机观测指标学习赛](https://tianchi.aliyun.com/competition/entrance/532270) 227 | 228 | --- 229 | 230 | ## 评分构成 231 | 232 | 244 | 245 | | $组成部分$ | $基础★$ | $进阶_1★★$ | $困难_2★★★$ | 246 | | :--- | :--- | :--- | :--- | 247 | | $Server$ | Minio Server | mock-s3 | Ceph | 248 | | $Client$ | Minio Client | osm/s3cmd/aws-{cli\|shell} | boto3/awssdk | 249 | | $Benchmark_3$ | 预置负载范例 | 选题观察分析 | 编制实验程序研究 | 250 | | **评分标准** | 每项**25分** | 每项**30分** | 每项**35分** | 251 | 252 | 1. 提供**有限课堂帮助**，[实验参考](https://github.com/cs-course/obs-tutorial)库内提供在线资源传送门。 253 | 2. 超过本资料库所提供导引之外部分**请自己探索**。 254 | 3. s3bench/benchio/warp/ossperf…… 255 | 256 | **实验方案示例1**：Minio Server (25) + boto3 (35) + 延迟分布随并发数的变化趋势分析 (35)=95分 257 | **实验方案示例2**：mock-s3 (30) + osm (30) + run-s3bench.cmd (25)=85分 258 | 259 | --- 260 | 261 | ## 作业提交 262 | 263 | 274 | 275 | 在微助教中完成**实验报告**。 276 | 277 | 每周的**课堂学习也会要求交一份作业**，请上传进微助教。 278 | 279 | ### 提交日期 280 | 281 | 第16周周五：**2025-06-06** 282 | 283 | ### 提交内容 284 | 285 | 课堂部分，华老师会在05-22课堂给出报告要求。 286 | 287 | 实验部分，请同学们按照前述要求用Markdown编写，上传微助教账号，提交作业。 288 | 289 | --- 290 | 291 | ### 往年作业参考 292 | 293 | 298 | 299 | - 300 | - 301 | - 302 | - 303 | - 304 | - 305 | 306 | #### 模范作业 307 | 308 | - [bigdata-storage-experiment-assignment-2023/U202015630](https://github.com/cs-course/bigdata-storage-experiment-assignment-2023/tree/main/U202015630/) 309 | - [bigdata-storage-experiment-assignment-2023/U202015628](https://github.com/cs-course/bigdata-storage-experiment-assignment-2023/tree/main/U202015628/) 310 | - [bigdata-storage-experiment-assignment-2022/U201916202](https://github.com/cs-course/bigdata-storage-experiment-assignment-2022/tree/master/U201916202/) 311 | - [bigdata-storage-experiment-assignment-2022/U201915084](https://github.com/cs-course/bigdata-storage-experiment-assignment-2022/tree/master/U201915084/) 312 | - [iot-storage-experiment-assignment-2021/U201816030](https://github.com/cs-course/iot-storage-experiment-assignment-2021/tree/master/U201816030) 313 | -------------------------------------------------------------------------------- /bio.md: -------------------------------------------------------------------------------- 1 | --- 2 | marp: true 3 | theme: gaia 4 | title: 个人简介 5 | # size: 4:3 6 | math: katex 7 | footer: '[个人简介](bio#个人简介)，[教育及工作经历](bio#教育及工作经历)，[教学工作](bio#教学工作)，[科研工作](bio#科研工作)，[社会服务](bio#社会服务)' 8 | --- 9 | 10 | # 个人简介 11 | 12 | **施展** 13 | 14 | 副教授，武汉光电国家研究中心，华中科技大学 15 | 16 | - 联系方式 17 | - 电邮 zshi (AT) hust.edu.cn 18 | - 电话 13971459597 19 | - 主页 20 | 21 | --- 22 | 23 | 28 | 29 | ## 教育及工作经历 30 | 31 | ### 教育 32 | 33 | - **华中科技大学（原华中理工大学）** 本科(1998)、硕士(2001)、博士(2011) 34 | 35 | ### 工作 36 | 37 | - **武汉光电国家研究中心，华中科技大学**、副教授(2017) 38 | - **内布拉斯加林肯州立大学**，美国，访问学者(2013-2014) 39 | - **计算机学院，华中科技大学**，助教、讲师 40 | 41 | --- 42 | 43 | 48 | 49 | ## 教学工作 50 | 51 | - **讲授课程** 52 | - 本科：计算机系统结构、大数据存储系统与管理实验 53 | - 研究生：数据中心技术、计算机系统设计 54 | - **研究生培养** 55 | - 硕士、博士在读12人，毕业43人 56 | - 多赴华为、腾讯、阿里等头部企业，也有优秀毕业生本地创业 57 | - **曾获奖项** 58 | - 聚焦计算机系统创新能力的“一基两翼全链”研究生培养体系构建与实践，**国家级教学成果二等奖**，2023 59 | 60 | --- 61 | 62 | 67 | 68 | ## 科研工作 69 | 70 | - **研究方向**：海量数据存储、存储管理、大数据、图数据处理等 71 | - **主持项目** 72 | - 面向新型计算模式的AI使能分布式存储架构及关键技术，**国家重点研发(子课题在研)** 73 | - 分布式存储系统的性能建模，**阿里AIR校企合作(在研)** 74 | - 曾主持完成**科技支撑计划、预研**等多项课题 75 | - 曾围绕分布式文件系统、知识图谱推理等方向与华为、搜狐、706等完成多项合作 76 | - **曾获奖项** 77 | - 主动对象海量存储系统及关键技术，**国家技术发明二等奖**，2014 78 | - 超级计算大会(SC)**国际存储挑战决赛奖**，2006 79 | - 电子学会电子信息科学技术二等奖(2013)，湖北省技术发明一等奖(2011、2018) 80 | - 在计算机学会A类期刊会议(ATC、DAC、SC等)上发表论文30余篇 81 | 82 | --- 83 | 84 | ## 社会服务 85 | 86 | - 教育部平台外审论文 87 | - 研究中心科普展评审 88 | - 研究中心暑期夏令营报告 89 | - 曾任基层支部书记、支委 90 | -------------------------------------------------------------------------------- /ceph-deployment-simple.md: -------------------------------------------------------------------------------- 1 | # Ubuntu20.04使用cephadm部署单节点Ceph集群 2 | 3 | ## 准备工作 4 | 5 | #### 1. 配置hosts解析 6 | 7 | ```bash 8 | vim /etc/hosts 9 | 192.168.1.1 host 10 | 这里在最后添加自己机器的ip 11 | 12 | ``` 13 | 14 | #### 2. 安装基础组件 15 | 16 | - 安装 docker/podman (cephadm基于容器运行所有ceph组件)，这里以podman为例： 17 | ```bash 18 | . /etc/os-release 19 | 20 | echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list 21 | 22 | curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/Release.key | sudo apt-key add - 23 | 24 | sudo apt-get update 25 | apt install -y podman 26 | ``` 27 | - 安装时间服务器: 28 | ```bash 29 | sudo apt install -y chrony && sudo systemctl enable --now chrony 30 | ``` 31 | 32 | #### 3. 配置SSH访问 33 | 34 | ```bash 35 | # 允许root用户登录 36 | sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/g' /etc/ssh/sshd_config 37 | 38 | # 重启ssh服务 39 | systemctl restart ssh 40 | 41 | # 设置root密码 42 | sudo passwd 43 | ``` 44 | 45 | ## 部署步骤 46 | 47 | ### 1. 安装cephadm (主节点) 48 | 49 | ```bash 50 | sudo apt install -y cephadm 51 | ``` 52 | 53 | ### 2. 引导新集群 54 | 55 | ```bash 56 | mkdir -p /etc/ceph 57 | cephadm bootstrap --mon-ip 192.168.1.1 (自己机器的ip) 58 | ``` 59 | 60 | 该命令执行以下操作： 61 | 62 | - 创建monitor和manager daemon 63 | - 生成SSH密钥 64 | - 添加密钥到authorized_keys 65 | - 配置必要文件 66 | - 生成管理密钥 67 | 68 | 69 | 此时已运行的组件： 70 | 71 | - ceph-mgr：管理程序 72 | - ceph-monitor：监视器 73 | - ceph-crash：崩溃数据收集 74 | - prometheus：监控组件 75 | - grafana：监控数据展示 76 | - alertmanager：告警组件 77 | - node_exporter：节点数据收集 78 | 79 | ### 3. 启用CEPH命令 80 | 81 | ```bash 82 | # 进入命令状态 83 | sudo cephadm shell 84 | 85 | # 查看集群状态 86 | ceph -s 87 | 88 | # 查看版本 89 | ceph -v 90 | 91 | #安装ceph-common，不用进入cephadm shell也可以使用ceph命令 92 | apt install ceph-common 93 | 94 | # 查看集群状态 95 | ceph -s 96 | ``` 97 | ### 4. 部署OSD 98 | 99 | 这里以sdb为例 100 | 101 | ```bash 102 | #取消磁盘挂载 103 | umount /dev/sdb 104 | ``` 105 | 106 | 如果有已经用过的磁盘或清理之前ceph使用的磁盘可使用一下脚本（官方文档引用） 107 | ```bash 108 | #!/bin/bash 109 | DISK="/dev/sdb" 110 | 111 | # Zap the disk to a fresh, usable state (zap-all is important, b/c MBR has to be clean) 112 | sgdisk --zap-all $DISK 113 | 114 | # Wipe a large portion of the beginning of the disk to remove more LVM metadata that may be present 115 | dd if=/dev/zero of="$DISK" bs=1M count=100 oflag=direct,dsync 116 | 117 | # SSDs may be better cleaned with blkdiscard instead of dd 118 | blkdiscard $DISK 119 | 120 | # Inform the OS of partition table changes 121 | partprobe $DISK 122 | 123 | ``` 124 | 或者可使用以下命令清理磁盘 125 | ```bash 126 | #查看磁盘 127 | lsblk 128 | 129 | #创建物理卷，以sdb为例 130 | pvcreate -ff /dev/sdb 131 | 132 | Physical volume "/dev/sdb" successfully created. 133 | #之后即可创建osd 134 | ``` 135 | 136 | 137 | ```bash 138 | 139 | # 创建OSD 140 | ceph orch daemon add osd host:/dev/sdb 141 | ``` 142 | 143 | ### 5.部署RGW服务 144 | 注意，只有在运行ceph -s时，health OK的情况下才能部署RGW服务 145 | ```bash 146 | # 创建rgw服务 147 | ceph orch apply rgw myorg us-east-1 --placement="1 host" 148 | 149 | #创建用户(可以指定uid、display_name、access_key、secret_key) 150 | radosgw-admin user create --uid="admin" --display-name="admin" --access-key="admin" --secret-key="123456" 151 | #之后就可以通过http://host:80和admin用户来访问对象存储 152 | ``` 153 | 154 | 155 | 156 | ## 常见问题处理 157 | 158 | ### 出错后完整删除Ceph集群 159 | 160 | ```bash 161 | #查看fsid 162 | ceph -s 163 | #清除集群 164 | cephadm rm-cluster --force --fsid 165 | #删除服务 166 | systemctl stop ceph.target 167 | systemctl reset-failed 168 | 169 | #查看磁盘 170 | lsblk 171 | #清理磁盘 172 | #之前安装过osd的磁盘上会有ceph--6100837c--0117--4b29--9cbf--00c0c6c13994-osd--block--2e5b167a--7416--4d16--8be8--92e0d82af345这样的逻辑卷 173 | dmsetup remove ceph--6100837c--0117--4b29--9cbf--00c0c6c13994-osd--block--2e5b167a--7416--4d16--8be8--92e0d82af345 174 | #创建物理卷 175 | pvcreate -ff /dev/sdb 176 | #之后即可部署新的OSD 177 | ``` 178 | -------------------------------------------------------------------------------- /computer-system-design-lab2.md: -------------------------------------------------------------------------------- 1 | # 图学习案例实验 2 | 3 | ## 实验背景 4 | 5 | 随着图神经网络（GNNs）在图数据挖掘研究中的广泛应用，研究者开始关注其在异构图（Heterogeneous Graphs）上的潜力。异构图由多种类型的节点和边组成，携带丰富的侧信息，连接了新颖有效的图学习算法与复杂多变的工业场景，如推荐系统。为了应对异构性的挑战，提出了多种异构图神经网络（HGNNs）模型来处理节点分类、链接预测和知识感知推荐等任务。然而，由于每个模型采用独特的数据处理和设置，我们对这些模型实际进展的理解仍然有限。 6 | 7 | 为了促进健壮且开放的HGNN发展，构建了异构图基准（HGB）。HGB目前包含11个异构图数据集，这些数据集在异构性（节点和边类型的数目）、任务（节点分类、链接预测和知识感知推荐）和领域（如学术图、用户-项目图和知识图）方面各不相同。HGB提供了统一的接口用于数据加载、特征处理和评估，为比较HGNN模型提供了便捷一致的方式。 8 | 9 | ## 实验要求 10 | 11 | - **组队要求**：每3名同学一组。各组需共同完成模型的选择、实现、调优、评估和报告撰写等任务。 12 | - **算法实现**：基于HGB提供的异构图数据集，实现一个或多个异构图神经网络模型，进行[节点分类任务](https://www.biendata.xyz/competition/hgb-1/)。 13 | - **性能评估**：使用Micro-F1和Macro-F1作为评价指标，对模型性能进行评估。 14 | - **代码提交**：按照HGB官方排行榜的要求，通过HGB在线表格提交代码，并在验证后加入官方排行榜。 15 | - **结果提交**：提交一个包含4个数据集预测结果的zip文件，每个数据集运行5次，共20个文件。文件命名格式为数据集名_序号.txt。 16 | - **实验报告**：介绍实验采用的模型、过程、结果分析及结论。明确标注小组成员的贡献。 17 | 18 | ### 时间安排 19 | 20 | - 开始日期：**2024年10月10日** 21 | - 提交截止：**2024年10月24日** 22 | 23 | 请在规定时间内完成实验，并按照要求提交代码、预测结果和实验报告。各组需确保在实验过程中积极协作，充分发挥团队成员的专长和创意。 24 | 25 | ## 参考资料 26 | 27 | - [HGB官网](https://www.biendata.xyz/hgb/)：提供HGB的详细介绍、数据集和使用指南。 28 | - [HGB节点分类任务](https://www.biendata.xyz/competition/hgb-1/)：介绍节点分类任务的背景和数据集。 29 | - [HGB竞赛规则](https://www.biendata.xyz/competition/hgb-1/rules/)：详细说明如何提交算法代码和论文，以及如何加入官方排行榜。 30 | - [HGB评估标准](https://www.biendata.xyz/competition/hgb-1/evaluation/)：介绍Micro-F1和Macro-F1评估指标，以及提交格式要求。 31 | - [HGB GitHub仓库](https://github.com/THUDM/HGB)：提供数据加载、特征处理和评估的脚本，以及如何生成预测结果文件的指南。 32 | -------------------------------------------------------------------------------- /data-center-pre.md: -------------------------------------------------------------------------------- 1 | --- 2 | marp: true 3 | theme: gaia 4 | title: 数据中心技术 5 | # size: 4:3 6 | --- 7 | 8 | 9 | 10 | # 数据中心技术 11 | 12 | **施展，童薇，胡燏翀，谭支鹏** 13 | 武汉光电国家研究中心，计算机学院 14 | 2024-11-06 至 2024-12-27 15 | 16 | --- 17 | 18 | ## 授课教师 19 | 20 | - **施展** 副教授，武汉光电国家研究中心，存储部，C523 21 | - 联系方式 22 | - EMail 23 | - 主页 24 | - 25 | - 26 | 27 | --- 28 | 29 | ## 授课教师… 30 | 31 | - **童薇** 副教授，武汉光电国家研究中心，存储部，C522 32 | - 联系方式 33 | - EMail 34 | - 主页 35 | - 36 | 37 | --- 38 | 39 | ## 授课教师…… 40 | 41 | - **胡燏翀** 教授，计算机科学与技术学院，存储所，B518 42 | - 联系方式 43 | - EMail 44 | - 主页 45 | - 46 | - 47 | 48 | --- 49 | 50 | ## 授课教师……… 51 | 52 | - **谭支鹏** 教授，武汉光电国家研究中心，存储部，C525 53 | - 联系方式 54 | - EMail 55 | - 主页 56 | - 57 | 58 | --- 59 | 60 | ## 基本信息 61 | 62 | 63 | 64 | 69 | 70 | - 课程资料 71 | - 课程平台 72 | - 参考书 73 | - [John Hennessy](https://hennessy.stanford.edu/), [David Patterson](https://www2.eecs.berkeley.edu/Faculty/Homepages/patterson.html), [**Computer Architecture: A Quantitative Approach**, 6th Edition.](https://www.elsevier.com/books/computer-architecture/hennessy/978-0-12-811905-1), 2017 74 | - [Luiz André Barroso](https://www.barroso.org/), Urs Hölzle, and Parthasarathy Ranganathan, [**The Datacenter as a Computer: Designing Warehouse-Scale Machines**, Third Edition.](https://www.morganclaypool.com/doi/10.2200/S00874ED3V01Y201809CAC046), 2019 75 | - [**数据中心一体化最佳实践：设计仓储级计算机**（原书第3版）](https://book.douban.com/subject/34950732/), [美] [路易斯·安德烈·巴罗索](https://www.barroso.org/) 著，徐凌杰 译, 2020 76 | - [计算机系统结构教程（第3版）](http://www.tup.tsinghua.edu.cn/booksCenter/book_09019101.html)， 清华大学出版社，2021 77 | 78 | --- 79 | 80 | ## 计算机系统结构拓展 81 | 82 | 87 | 88 | ![bg right fit](images/caqa6e.jpg) 89 | 90 | - [**计算机体系结构：量化研究方法**（第6版）](https://item.jd.com/13427803.html), 人民邮电出版社, 2022 91 | 92 | ![w:270](images/WSC-in-CAQA6e.png) 93 | 94 | --- 95 | 96 | ![bg fit](images/caqa5e-ch6-wsc-p0.gif) 97 | 98 | 99 | 100 | --- 101 | 102 | ![bg fit](images/caqa5e-ch6-wsc-p1.gif) 103 | 104 | 105 | 106 | --- 107 | 108 | ![bg fit](images/caqa5e-ch6-wsc-p2.gif) 109 | 110 | 111 | 112 | --- 113 | 114 | ## 仓储级计算机 115 | 116 | 126 | 127 | - [路易斯·安德烈·巴罗索（Luiz André Barroso）](https://www.barroso.org/)![h:33](images/white-flower-icon-5.jpg)，谷歌研究员、ACM会士、AAAS会士，领导着谷歌的工程基础设施工作。 128 | - [Warehouse-scale Computing](https://dl.acm.org/doi/10.1145/1807167.1837133), SIGMOD '10 129 | - [Warehouse-Scale Computing: Entering the Teenage Decade](https://dl.acm.org/doi/10.1145/2000064.2019527), ISCA '11 130 | - [A Brief History of Warehouse-Scale Computing](https://barroso.org/publications/IEEEMicro2021.pdf), 2020 [Eckert-Mauchly Award](https://awards.acm.org/eckert-mauchly) 131 | 132 | Before the onset of the current pandemic, some of us may have underappreciated how important computing technology and cloud-based services have become to our society. In this last year, these technologies have allowed many of us to continue to work, to connect with loved ones, and to support each other. I am grateful to all of those at Google and everywhere in our industry who have built such essential technologies, and I am inspired to be working in a field with still so much potential to improve people’s lives. 133 | 134 | 139 | 140 | --- 141 | 142 | ## 一些关于先驱的故事 143 | 144 | Barroso was born in Brazil and had a bachelor’s and master’s degree in electrical engineering from the Pontifical Catholic University of Rio de Janeiro. 145 | 146 | In the United States, he did a doctorate in **computer engineering** at the University of Southern California and worked with processors at Compaq and **Digital Equipment Corporation**. In 2001, he joined Google as a **software engineer**. 147 | 148 | --- 149 | 150 | According to an article in Wired, Barroso had never designed a datacenter until he received this request from Google. He came up with the concept of **"datacenter as a computer"**, building data centres with low-cost components, as we know them today. 151 | 152 | He comments that **the lack of experience in datacenter design may have been an advantage**, as we questioned almost every aspect of how these facilities were designed. Perhaps the most important thing was having **the opportunity to look at the entire design**, from the cooling towers to the compilers, and this quickly revealed important opportunities for improvement. Barroso’s idea quickly spread throughout Silicon Valley, among the datacenters of other Internet giants. 153 | 154 | --- 155 | 156 | 161 | 162 | Barroso shared three lessons he learnt in the first half of his career: 163 | 164 | 1. **Consider the winding road**: Although there are always risks when embarking on somethingnew, the upside of being adventurous in your professional career can be incredibly rewarding. 165 | 166 | 2. **Develop respect for the obvious**: Big problems and important issues have one characteristic incommon: they tend to be simple to understand but difficult to solve. They are obvious and deserveattention. 167 | 168 | 3. **Even success has an expiry date**: Some of the most intellectual moments in Barrosa’s careercame when he was forced to abandon his original position, in which he had invested significanttime and effort and achieved some success. 169 | 170 | --- 171 | 172 | ## 授课目标 173 | 174 | 179 | 180 | - **工程实践**方面 181 | - 跟踪一线应用**思考方向** 182 | - 华为（黄大年茶思屋）、腾讯（犀牛鸟）、阿里（AIR）、浪潮…… 183 | - 积累工程基础**动手实践** 184 | - [Kaggle](https://www.kaggle.com/competitions), [Codalab](https://codalab.lisn.upsaclay.fr/) 185 | - 阿里[天池](https://tianchi.aliyun.com/), 腾讯[DataFountain](https://www.datafountain.cn/), [和鲸](https://www.heywhale.com/home/competition), 百度[飞桨AI Studio](https://aistudio.baidu.com/competition), [讯飞开放平台](https://challenge.xfyun.cn/)…… 186 | - **学术探索**方面 187 | - 相关领域**研讨前沿**技术与进展 188 | - 数据中心扩展性、性能、服务质量、可靠性…… 189 | - 建立独立研究技能**解决问题** 190 | - 选题汇报与研讨、应用实践与检验…… 191 | 192 | --- 193 | 194 | ## 评分构成 195 | 196 | 201 | 202 | - **论文研讨** *30%* 203 | - 制作胶片汇报一篇相关论文 204 | - 第一周确认计划安排，39位同学每人选择1篇Paper准备汇报 205 | - 每位同学有**12分钟汇报和2~3分钟讨论** 206 | - 请严格守时（开PPT排练计时），可自备讲稿辅助 207 | - 论坛内就汇报内容回答相关问题，**积极参与问答可以加分** 208 | - **实验作业** *30%* 209 | - [构建多模态模型，生成主机观测指标学习赛](https://tianchi.aliyun.com/competition/entrance/532270/introduction) 210 | - 以完成提交为准，不追求PVP，重在以日常赛促**实践** 211 | - 日常赛作为入门学习，学有余力可以遍寻各大竞赛平台PVP收获成就 212 | - **开卷考试** 40% 213 | - TBA 214 | 215 | 216 | --- 217 | 218 | ### 说在选读论文之前 219 | 220 | 225 | 226 | - **传统教育模式的颠覆**：自2022年11月ChatGPT发布以来，基于LLM的AI工具在各种考试中表现出色，能够即时生成与学术研究人员撰写的论文相媲美的内容，学生可以轻松完成作业。 227 | - **作弊启示录**：ChatGPT的出现使得传统的评估模式变得过时，因为学生的成绩可能不再反映他们的真实能力。 228 | - **作弊的本质**：当任务被视为外在的、与个人身份无关时，使用工具完成任务并不被视为作弊。 229 | - **前进之路**：为了适应人工智能时代，大学教育需要从以教学为中心转变为以学习为中心，利用人工智能作为实时、自适应的导师、辅导员和助手。 230 | - **个性化辅导**：ChatGPT可以作为个性化辅导工具，帮助学生提高（非母语）写作和其他学术技能。 231 | 232 | **人工智能时代的大学教学变革**，[中国教育网络](https://cqvip.com/journal/37548/37548) 2024年8月刊 233 | 来源：美国高等教育信息化协会 [EDUCAUSE](https://www.educause.edu/) 234 | 作者：丹·萨罗菲安-布廷 235 | 编译：李想 236 | 责编：项阳 237 | 238 | 239 | 240 | --- 241 | 242 | ### 示范 243 | 244 | 245 | 246 | --- 247 | 248 | 255 | 256 | **要学会站在AI的肩膀上** 257 | 而不是靠着TA 258 | 259 | --- 260 | 261 | ## 研讨论文列表 262 | 263 | ![bg right](images/reading-list-2024.png) 264 | 265 | 扫码在线填表 266 | 267 | - 每人选择1篇拟汇报论文 268 | - 选择范围：CCF-A相关论文 269 | - [FAST](https://www.usenix.org/conference/fast24/technical-sessions), [ATC](https://www.usenix.org/conference/atc24/technical-sessions), [OSDI](https://www.usenix.org/conference/osdi24/technical-sessions), [SOSP](https://sosp2023.mpi-sws.org/program.html), [ISCA](https://www.iscaconf.org/isca2024/program/), [HPCA](https://www.hpca-conf.org/2024/program/main.php) (2023~24) 270 | 271 | 272 | 273 | 也可以点击 274 | 275 | - [【腾讯文档】数据中心技术课程论文研讨2024](https://docs.qq.com/sheet/DRHJZV01EbWhwaHlZ) 276 | 277 | --- 278 | 279 | ### 补充说明 280 | 281 | 286 | 287 | 本次课堂在限定上述6个计算机系统结构方向主要会议之后，不再进行麻烦的列表选择，同学们也可以自选在前面课堂中讨论过的，自己已经比较熟悉的论文进行汇报，**学习重点在于研讨过程** 288 | 289 | **举例**：[FAST24](https://www.usenix.org/conference/fast24/technical-sessions) 290 | 291 | - **Distributed Storage**（这个分论坛就属于课堂主题相关，其中论文优先挑选） 292 | - Session Chair: Raju Rangaswami, Florida International University 293 | - TeRM: Extending RDMA-Attached Memory with SSD 294 | - Combining Buffered I/O and Direct I/O in Distributed File Systems 295 | - OmniCache: Collaborative Caching for Near-storage Accelerators 296 | 297 | --- 298 | 299 | ## 课程计划 300 | 301 | 312 | 313 | | | 讲座主题 | 日期 | 地点 | 314 | |:-:|:--------|:---| :--- | 315 | | 1 | [数据中心技术概述](data-center-intro) | 11-06(**周三5-6**) 11-08(**周五3-4**) | C12-S204 | 316 | | 2 | [对象存储技术](object-storage) 及 [存储服务质量保障技术](qos-guarantee) | 11-13, 11-15 | C12-S204 | 317 | | 3 | 数据中心固态存储技术 (童老师) | 11-20, 11-22 | C12-S204 | 318 | | 4 | 数据中心磁盘故障预测技术 (谭老师) | 11-27, 11-29 | C12-S204 | 319 | | 5 | 数据中心可靠性保障技术 (胡老师) | 12-04, 12-06 | C12-S204 | 320 | | 6 | 论文研讨* | 12-11, 12-13 | C12-S204 | 321 | | 7 | 论文研讨 | 12-18, 12-20 | C12-S204 | 322 | | 8 | 论文研讨 | 12-25, 12-27 | C12-S204 | 323 | 324 | \* *每周13名同学，每人12分钟报告，2~3分钟问答，相关问题还应**在课程平台讨论里作答**。* 325 | 326 | --- 327 | 328 | ## 课堂背景调查 329 | 330 | 346 | 347 | 扫码进群，参与[调查](https://docs.qq.com/form/page/DRFlyQ2xnaG9xY3Zk?_w_tencentdocx_form=1)，**合计25复选项** 348 | 349 | ![bg right:32% fit](images/qq-group-2024.jpg) 350 | 351 | ||| 352 | |:-|:-| 353 | |**动手实践**
- 在裸机上安装过系统
- 自己用零件装配台式机
- 实验室的机群是我配的|**系统环境**
- 只用桌面(GNOME, KDE…)
- 命令行(bash, zsh, fish…)
- 5个以上GNU工具(grep, sed, awk…)
- 远程操作(ssh, tmux, screen…)
- 拥有VPS、云主机| 354 | |**C/C++**
- 作业程序
- 多模块工程(make, autoconf…)
- 知名程序库(STL, Boost, MPI…)|**Java**
- 课程作业
- 构建工具(Ant, Maven, SBT…)
- 知名工程(Hadoop, Spark, Giraph…)| 355 | |**Python**
- 一般脚本
- 管理过系统、运行过网站
- 跑过大数据或ML任务(Tensorflow, torch…)|**还接触过**
- 专业语言(Matlab, R)
- 流行语言(Go, JS, Rust)
- 特长语言(Scala, Julia)| 356 | |**资料来源**
- Stackoverflow
- Github, Bitbucket, Gitee, GitLab
- Linkedin
- LeetCode
- Release Mirrors, Vagrant, Docker…|| 357 | 358 | 359 | 360 | --- 361 | 362 | ## 或者扫描问卷条码 363 | 364 | ![bg right:32% fit](images/qq-group-2024.jpg) 365 | 366 | ![h:450](images/class-survey.jpg) 367 | 368 | --- 369 | 370 | 377 | 378 | [数据中心技术总体介绍](data-center-intro) 379 | -------------------------------------------------------------------------------- /image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/image.png -------------------------------------------------------------------------------- /images/01-Global-Overview-DataReportal-20200422-Digital-2020-April-Global-Statshot-Report-Slide-7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/01-Global-Overview-DataReportal-20200422-Digital-2020-April-Global-Statshot-Report-Slide-7.png -------------------------------------------------------------------------------- /images/05-Changes-in-Connected-Behaviours-DataReportal-20200422-Digital-2020-April-Global-Statshot-Report-Slide-11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/05-Changes-in-Connected-Behaviours-DataReportal-20200422-Digital-2020-April-Global-Statshot-Report-Slide-11.png -------------------------------------------------------------------------------- /images/2u-rack-server.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/2u-rack-server.jpg -------------------------------------------------------------------------------- /images/3-replica.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/3-replica.png -------------------------------------------------------------------------------- /images/3-tier-memory.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/3-tier-memory.png -------------------------------------------------------------------------------- /images/3steps.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/3steps.png -------------------------------------------------------------------------------- /images/439Microsoft_Project-Natick-servers-1068x601.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/439Microsoft_Project-Natick-servers-1068x601.jpg -------------------------------------------------------------------------------- /images/451figures.original.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/451figures.original.png -------------------------------------------------------------------------------- /images/690px-SETI-home_ScrSaver.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/690px-SETI-home_ScrSaver.jpg -------------------------------------------------------------------------------- /images/ANSI_TIA-942_Standard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/ANSI_TIA-942_Standard.png -------------------------------------------------------------------------------- /images/APM-Diagram-1.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/APM-Diagram-1.webp -------------------------------------------------------------------------------- /images/Basic-structure-of-queueing-models.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Basic-structure-of-queueing-models.png -------------------------------------------------------------------------------- /images/Block-Storage-vs-File-Storage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Block-Storage-vs-File-Storage.jpg -------------------------------------------------------------------------------- /images/CIS_Q320.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/CIS_Q320.jpg -------------------------------------------------------------------------------- /images/CPU-Utilization-on-a-vSphere-Host-Before-and-After-Consolidation.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/CPU-Utilization-on-a-vSphere-Host-Before-and-After-Consolidation.jpg -------------------------------------------------------------------------------- /images/Cloud-Computing-as-a-Service-Revenue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Cloud-Computing-as-a-Service-Revenue.png -------------------------------------------------------------------------------- /images/Cloudfront-Map_9.24_2x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Cloudfront-Map_9.24_2x.png -------------------------------------------------------------------------------- /images/Data-Centers-top10.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Data-Centers-top10.webp -------------------------------------------------------------------------------- /images/Erklarung_M10_72ppi_EN.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Erklarung_M10_72ppi_EN.png -------------------------------------------------------------------------------- /images/Facebook-s-Arctic-Server-Farm-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Facebook-s-Arctic-Server-Farm-1.jpg -------------------------------------------------------------------------------- /images/Fraction-of-US-data-center-electricity-use-in-2014.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Fraction-of-US-data-center-electricity-use-in-2014.png -------------------------------------------------------------------------------- /images/Globa-and-China-DC-trend.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Globa-and-China-DC-trend.png -------------------------------------------------------------------------------- /images/Global-electricity-demand-of-consumer-devices-2010-2030.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Global-electricity-demand-of-consumer-devices-2010-2030.png -------------------------------------------------------------------------------- /images/Global-electricity-demand-of-data-centers-2010-2030.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Global-electricity-demand-of-data-centers-2010-2030.png -------------------------------------------------------------------------------- /images/GooglePUE-800.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/GooglePUE-800.png -------------------------------------------------------------------------------- /images/Historical-energy-usage.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Historical-energy-usage.jpeg -------------------------------------------------------------------------------- /images/IDC_DataSphere.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/IDC_DataSphere.png -------------------------------------------------------------------------------- /images/IDC_edge_to_core.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/IDC_edge_to_core.png -------------------------------------------------------------------------------- /images/Latency-Numbers-Every-Programmer-Should-Know.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Latency-Numbers-Every-Programmer-Should-Know.png -------------------------------------------------------------------------------- /images/Massively-parallel-processing.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Massively-parallel-processing.webp -------------------------------------------------------------------------------- /images/MemoryHierarchy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/MemoryHierarchy.png -------------------------------------------------------------------------------- /images/Micron-Computex-2021-Keynote-Memory-Pyramid.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Micron-Computex-2021-Keynote-Memory-Pyramid.jpg -------------------------------------------------------------------------------- /images/OSD-Metadata.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/OSD-Metadata.webp -------------------------------------------------------------------------------- /images/OSD-vs-block-based.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/OSD-vs-block-based.webp -------------------------------------------------------------------------------- /images/Object-Storage.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Object-Storage.webp -------------------------------------------------------------------------------- /images/PI-Controller.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/PI-Controller.png -------------------------------------------------------------------------------- /images/QoS-managed-distributed-storage-system.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/QoS-managed-distributed-storage-system.gif -------------------------------------------------------------------------------- /images/Replication-and-Erasure-Coding.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Replication-and-Erasure-Coding.jpg -------------------------------------------------------------------------------- /images/Restful-Web-Services.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Restful-Web-Services.png -------------------------------------------------------------------------------- /images/S3-decision-flow-chart_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/S3-decision-flow-chart_1.png -------------------------------------------------------------------------------- /images/SCSI-Architecture-OSD.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/SCSI-Architecture-OSD.webp -------------------------------------------------------------------------------- /images/SNIA-tail-latency.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/SNIA-tail-latency.webp -------------------------------------------------------------------------------- /images/SQL-and-Oracle-Database-Consolidation-Reduces-Software-Costs-by-70-Percent.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/SQL-and-Oracle-Database-Consolidation-Reduces-Software-Costs-by-70-Percent.jpg -------------------------------------------------------------------------------- /images/Screen-Shot-2015-10-04-at-6.15.24-PM-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Screen-Shot-2015-10-04-at-6.15.24-PM-small.jpg -------------------------------------------------------------------------------- /images/Screen-Shot-2015-10-04-at-6.15.24-PM.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Screen-Shot-2015-10-04-at-6.15.24-PM.png -------------------------------------------------------------------------------- /images/Source-of-Latency-Solutions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Source-of-Latency-Solutions.png -------------------------------------------------------------------------------- /images/Source-of-Latency.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Source-of-Latency.png -------------------------------------------------------------------------------- /images/Stochastic-Fairness-Queuing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Stochastic-Fairness-Queuing.png -------------------------------------------------------------------------------- /images/Tahoe-Reno-1.2e16d0ba.fill-1200x630.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Tahoe-Reno-1.2e16d0ba.fill-1200x630.jpg -------------------------------------------------------------------------------- /images/Telecom-Fraud.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Telecom-Fraud.png -------------------------------------------------------------------------------- /images/Thelongtailshift.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Thelongtailshift.webp -------------------------------------------------------------------------------- /images/Top-Four-Cloud-Infrastructure-Providers.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Top-Four-Cloud-Infrastructure-Providers.jpg -------------------------------------------------------------------------------- /images/WSC-in-CAQA6e.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/WSC-in-CAQA6e.png -------------------------------------------------------------------------------- /images/WYG0047-0179d.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/WYG0047-0179d.png -------------------------------------------------------------------------------- /images/Web-Application-Architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/Web-Application-Architecture.png -------------------------------------------------------------------------------- /images/access-pattern-real.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/access-pattern-real.png -------------------------------------------------------------------------------- /images/access-pattern.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/access-pattern.png -------------------------------------------------------------------------------- /images/active-object-storage-award.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/active-object-storage-award.png -------------------------------------------------------------------------------- /images/active-object-storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/active-object-storage.png -------------------------------------------------------------------------------- /images/ai-market-2024.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/ai-market-2024.jpg -------------------------------------------------------------------------------- /images/ali-qiandaohu-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/ali-qiandaohu-small.jpg -------------------------------------------------------------------------------- /images/ali-qiandaohu.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/ali-qiandaohu.jpg -------------------------------------------------------------------------------- /images/aliyun-top5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/aliyun-top5.jpg -------------------------------------------------------------------------------- /images/aliyun-zhangbei-aidc.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/aliyun-zhangbei-aidc.jpg -------------------------------------------------------------------------------- /images/amdahl-law.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/amdahl-law.png -------------------------------------------------------------------------------- /images/approximation-by-betweenness-centrality.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/approximation-by-betweenness-centrality.jpg -------------------------------------------------------------------------------- /images/architecture-hpc-cfd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/architecture-hpc-cfd.png -------------------------------------------------------------------------------- /images/autoscaling-architecture5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/autoscaling-architecture5.png -------------------------------------------------------------------------------- /images/aws-region.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/aws-region.png -------------------------------------------------------------------------------- /images/aws_proxy_s3_create_methods_on_folder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/aws_proxy_s3_create_methods_on_folder.png -------------------------------------------------------------------------------- /images/azure-data-center-global-map.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/azure-data-center-global-map.webp -------------------------------------------------------------------------------- /images/baidu-index-datacenter.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/baidu-index-datacenter.jpg -------------------------------------------------------------------------------- /images/beyond-server-consolidation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/beyond-server-consolidation.png -------------------------------------------------------------------------------- /images/block-diagram-of-feedback-control-system.drawio: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | -------------------------------------------------------------------------------- /images/block-diagram-of-feedback-control-system.svg: -------------------------------------------------------------------------------- 1 |
参考输入
参考输入
控制器
控制器
控制误差
控制误差
目标系统
目标系统
控制输入
控制输入
变换器
变换器
状态测量
状态测量Viewer does not support full SVG 1.1 -------------------------------------------------------------------------------- /images/byte-dance-hedged-read.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/byte-dance-hedged-read.webp -------------------------------------------------------------------------------- /images/cache-block-address.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cache-block-address.png -------------------------------------------------------------------------------- /images/cache-memory-storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cache-memory-storage.png -------------------------------------------------------------------------------- /images/cache-working-flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cache-working-flow.png -------------------------------------------------------------------------------- /images/caqa5e-ch6-wsc-p0.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/caqa5e-ch6-wsc-p0.gif -------------------------------------------------------------------------------- /images/caqa5e-ch6-wsc-p1.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/caqa5e-ch6-wsc-p1.gif -------------------------------------------------------------------------------- /images/caqa5e-ch6-wsc-p2.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/caqa5e-ch6-wsc-p2.gif -------------------------------------------------------------------------------- /images/caqa6e.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/caqa6e.jpg -------------------------------------------------------------------------------- /images/ceph-graphic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/ceph-graphic.png -------------------------------------------------------------------------------- /images/ceph-stack.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/ceph-stack.png -------------------------------------------------------------------------------- /images/cephDiagramm.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cephDiagramm.webp -------------------------------------------------------------------------------- /images/cfq-cgroup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cfq-cgroup.png -------------------------------------------------------------------------------- /images/cgroup-vs-qemu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cgroup-vs-qemu.png -------------------------------------------------------------------------------- /images/chaos.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/chaos.jpg -------------------------------------------------------------------------------- /images/china-computing-power.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/china-computing-power.png -------------------------------------------------------------------------------- /images/china-datacenter-map.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/china-datacenter-map.png -------------------------------------------------------------------------------- /images/china-datacenter-pue-area.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/china-datacenter-pue-area.png -------------------------------------------------------------------------------- /images/china-datacenter-pue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/china-datacenter-pue.png -------------------------------------------------------------------------------- /images/china-multi-cloud.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/china-multi-cloud.png -------------------------------------------------------------------------------- /images/china-new-construction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/china-new-construction.png -------------------------------------------------------------------------------- /images/class-survey.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/class-survey.jpg -------------------------------------------------------------------------------- /images/clip-art-robotic-arm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/clip-art-robotic-arm.png -------------------------------------------------------------------------------- /images/cloud-dc-locations@2x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-dc-locations@2x.png -------------------------------------------------------------------------------- /images/cloud-dc-numbers@2x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-dc-numbers@2x.png -------------------------------------------------------------------------------- /images/cloud-everywhere.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-everywhere.jpg -------------------------------------------------------------------------------- /images/cloud-fog-edge_infographic.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-fog-edge_infographic.jpg -------------------------------------------------------------------------------- /images/cloud-fog-edge_infographic.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-fog-edge_infographic.webp -------------------------------------------------------------------------------- /images/cloud-provider-location-map.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-provider-location-map.png -------------------------------------------------------------------------------- /images/cloud-storage-performance-factors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-storage-performance-factors.png -------------------------------------------------------------------------------- /images/cloud-storage-performance-prediction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-storage-performance-prediction.png -------------------------------------------------------------------------------- /images/cloud-storage-request-analysis.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-storage-request-analysis.png -------------------------------------------------------------------------------- /images/cloud-storage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cloud-storage.jpg -------------------------------------------------------------------------------- /images/cmp-consolidated-servers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cmp-consolidated-servers.png -------------------------------------------------------------------------------- /images/comp-storage-01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/comp-storage-01.png -------------------------------------------------------------------------------- /images/comp-storage-02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/comp-storage-02.png -------------------------------------------------------------------------------- /images/compressed-sparse-row.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/compressed-sparse-row.jpg -------------------------------------------------------------------------------- /images/constrained-optimization-l.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/constrained-optimization-l.jpg -------------------------------------------------------------------------------- /images/course-barcode.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/course-barcode.png -------------------------------------------------------------------------------- /images/cpu-operations.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/cpu-operations.png -------------------------------------------------------------------------------- /images/csapp3e-cover.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/csapp3e-cover.jpg -------------------------------------------------------------------------------- /images/dac23-bg1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dac23-bg1.png -------------------------------------------------------------------------------- /images/dac23-bg2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dac23-bg2.png -------------------------------------------------------------------------------- /images/dac23-bg3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dac23-bg3.png -------------------------------------------------------------------------------- /images/data-centre-fuel-cell-efficiency.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/data-centre-fuel-cell-efficiency.png -------------------------------------------------------------------------------- /images/datacenter-scale-evaluation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/datacenter-scale-evaluation.png -------------------------------------------------------------------------------- /images/datacenter-trend.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/datacenter-trend.png -------------------------------------------------------------------------------- /images/deepwalk.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/deepwalk.png -------------------------------------------------------------------------------- /images/dft-acc7-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dft-acc7-small.jpg -------------------------------------------------------------------------------- /images/dft-acc7.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dft-acc7.jpg -------------------------------------------------------------------------------- /images/dilemma-on-data-loading.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dilemma-on-data-loading.jpg -------------------------------------------------------------------------------- /images/dilemma-on-skewness.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/dilemma-on-skewness.jpg -------------------------------------------------------------------------------- /images/direct-mapping.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/direct-mapping.png -------------------------------------------------------------------------------- /images/distributed-storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/distributed-storage.png -------------------------------------------------------------------------------- /images/edge-centric.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/edge-centric.jpg -------------------------------------------------------------------------------- /images/emergency.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/emergency.jpg -------------------------------------------------------------------------------- /images/erasure-code.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/erasure-code.png -------------------------------------------------------------------------------- /images/evaluation-on-losc.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/evaluation-on-losc.jpg -------------------------------------------------------------------------------- /images/evolving-graph-apps.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/evolving-graph-apps.png -------------------------------------------------------------------------------- /images/evolving-graph-logic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/evolving-graph-logic.png -------------------------------------------------------------------------------- /images/evolving-graph-research.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/evolving-graph-research.png -------------------------------------------------------------------------------- /images/example-sssp-parallel-bfs-in-pregel-l.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/example-sssp-parallel-bfs-in-pregel-l.jpg -------------------------------------------------------------------------------- /images/failure-trends.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/failure-trends.png -------------------------------------------------------------------------------- /images/flops-bytes-unbalance.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/flops-bytes-unbalance.jpg -------------------------------------------------------------------------------- /images/fund-tracking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/fund-tracking.png -------------------------------------------------------------------------------- /images/github-object-storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/github-object-storage.png -------------------------------------------------------------------------------- /images/google-cloud-platform-infra-map-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/google-cloud-platform-infra-map-small.jpg -------------------------------------------------------------------------------- /images/google-cloud-platform-infra-map.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/google-cloud-platform-infra-map.jpg -------------------------------------------------------------------------------- /images/google-maps2-576x1024-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/google-maps2-576x1024-small.jpg -------------------------------------------------------------------------------- /images/google-maps2-576x1024.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/google-maps2-576x1024.png -------------------------------------------------------------------------------- /images/graph-app-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/graph-app-1.jpg -------------------------------------------------------------------------------- /images/graph-processing-systems.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/graph-processing-systems.png -------------------------------------------------------------------------------- /images/graph_logbase10_ms.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/graph_logbase10_ms.png -------------------------------------------------------------------------------- /images/graphlab-consistency-model.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/graphlab-consistency-model.jpg -------------------------------------------------------------------------------- /images/graphlab-framework.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/graphlab-framework.jpg -------------------------------------------------------------------------------- /images/graphs-are-everywhere.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/graphs-are-everywhere.jpg -------------------------------------------------------------------------------- /images/green-sustainable-data-center-with-earth.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/green-sustainable-data-center-with-earth.jpg -------------------------------------------------------------------------------- /images/hedged-requests.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/hedged-requests.png -------------------------------------------------------------------------------- /images/here-are-the-10-largest-data-center-providers-in-the-world.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/here-are-the-10-largest-data-center-providers-in-the-world.jpg -------------------------------------------------------------------------------- /images/homogeneous-server-queue.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/homogeneous-server-queue.jpg -------------------------------------------------------------------------------- /images/hpc_schematic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/hpc_schematic.png -------------------------------------------------------------------------------- /images/huge-evaluation-0.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/huge-evaluation-0.jpg -------------------------------------------------------------------------------- /images/huge-evaluation-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/huge-evaluation-1.jpg -------------------------------------------------------------------------------- /images/huge-evaluation-2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/huge-evaluation-2.jpg -------------------------------------------------------------------------------- /images/huge-hrw.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/huge-hrw.jpg -------------------------------------------------------------------------------- /images/huge-workflow.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/huge-workflow.jpg -------------------------------------------------------------------------------- /images/iccd22-bg1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/iccd22-bg1.png -------------------------------------------------------------------------------- /images/iccd22-bg2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/iccd22-bg2.png -------------------------------------------------------------------------------- /images/iccd22-bg3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/iccd22-bg3.png -------------------------------------------------------------------------------- /images/iccd22-bg4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/iccd22-bg4.png -------------------------------------------------------------------------------- /images/idc-computing-power-2021.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/idc-computing-power-2021.jpg -------------------------------------------------------------------------------- /images/idc-datasphere-2020.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/idc-datasphere-2020.webp -------------------------------------------------------------------------------- /images/intel-xeon-2-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/intel-xeon-2-small.jpg -------------------------------------------------------------------------------- /images/intel-xeon-2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/intel-xeon-2.jpg -------------------------------------------------------------------------------- /images/it-was-nature-for-change.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/it-was-nature-for-change.png -------------------------------------------------------------------------------- /images/keep-calm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/keep-calm.png -------------------------------------------------------------------------------- /images/krongen256bfs.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/krongen256bfs.jpg -------------------------------------------------------------------------------- /images/latency-gap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/latency-gap.png -------------------------------------------------------------------------------- /images/latency-sla.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/latency-sla.png -------------------------------------------------------------------------------- /images/law-of-the-minimum.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/law-of-the-minimum.jpg -------------------------------------------------------------------------------- /images/m1-m2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/m1-m2.png -------------------------------------------------------------------------------- /images/mclock.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/mclock.png -------------------------------------------------------------------------------- /images/memory-hierarchy.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/memory-hierarchy.jpg -------------------------------------------------------------------------------- /images/memory-hierarchy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/memory-hierarchy.png -------------------------------------------------------------------------------- /images/model-size.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/model-size.png -------------------------------------------------------------------------------- /images/model-system-size.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/model-system-size.jpg -------------------------------------------------------------------------------- /images/multi-tenancy-degrees.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/multi-tenancy-degrees.png -------------------------------------------------------------------------------- /images/multitenancy-web.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/multitenancy-web.png -------------------------------------------------------------------------------- /images/nanjing-aidc.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/nanjing-aidc.jpg -------------------------------------------------------------------------------- /images/neo4j-intro.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/neo4j-intro.png -------------------------------------------------------------------------------- /images/network-representative-learning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/network-representative-learning.png -------------------------------------------------------------------------------- /images/object-vs-traditional.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/object-vs-traditional.png -------------------------------------------------------------------------------- /images/odcc-chart1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/odcc-chart1.png -------------------------------------------------------------------------------- /images/odcc-chart5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/odcc-chart5.png -------------------------------------------------------------------------------- /images/openstack-frontend-qos.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/openstack-frontend-qos.png -------------------------------------------------------------------------------- /images/osd-2007.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/osd-2007.png -------------------------------------------------------------------------------- /images/osd-commands.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/osd-commands.PNG -------------------------------------------------------------------------------- /images/osd-commands.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/osd-commands.png -------------------------------------------------------------------------------- /images/pagerank-pregel.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/pagerank-pregel.jpg -------------------------------------------------------------------------------- /images/pagerank-result.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/pagerank-result.jpg -------------------------------------------------------------------------------- /images/pagerank.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/pagerank.webp -------------------------------------------------------------------------------- /images/performance-isolation-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/performance-isolation-1.png -------------------------------------------------------------------------------- /images/performance-isolation-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/performance-isolation-2.png -------------------------------------------------------------------------------- /images/placement-of-inactive-vertexes.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/placement-of-inactive-vertexes.jpg -------------------------------------------------------------------------------- /images/power-law-internet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/power-law-internet.jpg -------------------------------------------------------------------------------- /images/power-law-twitter.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/power-law-twitter.jpg -------------------------------------------------------------------------------- /images/prometheus-architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/prometheus-architecture.png -------------------------------------------------------------------------------- /images/provisioning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/provisioning.png -------------------------------------------------------------------------------- /images/qq-group-2021.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/qq-group-2021.png -------------------------------------------------------------------------------- /images/qq-group-2022.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/qq-group-2022.jpg -------------------------------------------------------------------------------- /images/qq-group-2023.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/qq-group-2023.jpg -------------------------------------------------------------------------------- /images/qq-group-2024.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/qq-group-2024.jpg -------------------------------------------------------------------------------- /images/rack-full.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/rack-full.jpg -------------------------------------------------------------------------------- /images/random-access-problem.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/random-access-problem.jpg -------------------------------------------------------------------------------- /images/range-intl-langfang.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/range-intl-langfang.jpg -------------------------------------------------------------------------------- /images/reading-list-2023.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/reading-list-2023.png -------------------------------------------------------------------------------- /images/reading-list-2024.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/reading-list-2024.png -------------------------------------------------------------------------------- /images/red-hat-ceph-storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/red-hat-ceph-storage.png -------------------------------------------------------------------------------- /images/s3-storage-classes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/s3-storage-classes.png -------------------------------------------------------------------------------- /images/sc23-bg1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sc23-bg1.png -------------------------------------------------------------------------------- /images/sc23-bg2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sc23-bg2.png -------------------------------------------------------------------------------- /images/scaleout-diagram.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/scaleout-diagram.jpg -------------------------------------------------------------------------------- /images/sequential-vs-random.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sequential-vs-random.jpg -------------------------------------------------------------------------------- /images/server-consolidation-scenario.drawio: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | -------------------------------------------------------------------------------- /images/server-consolidation-scenario.svg: -------------------------------------------------------------------------------- 1 |
物理硬件
物理硬件
操作系统
操作系统
应用
应用
物理硬件
物理硬件
虚拟化
虚拟化
操作系统
操作系统
应用
应用
虚拟硬件
虚拟硬件
虚拟硬件
虚拟硬件
虚拟硬件
虚拟硬件
操作系统
操作系统
应用
应用
操作系统
操作系统
应用
应用Viewer does not support full SVG 1.1 -------------------------------------------------------------------------------- /images/server-consolidation.drawio: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | -------------------------------------------------------------------------------- /images/server-consolidation.svg: -------------------------------------------------------------------------------- 1 |
Cloud
Cloud
PaaS
PaaS
IaaS
IaaS
SaaS
SaaS
Multicore
Multicore
Virtualization
Virtualization
Consolidation
ConsolidationViewer does not support full SVG 1.1 -------------------------------------------------------------------------------- /images/server-motherboard-small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/server-motherboard-small.jpg -------------------------------------------------------------------------------- /images/server-motherboard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/server-motherboard.jpg -------------------------------------------------------------------------------- /images/server-racks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/server-racks.png -------------------------------------------------------------------------------- /images/set-associative-mapping.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/set-associative-mapping.png -------------------------------------------------------------------------------- /images/sfk-big-english.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sfk-big-english.jpg -------------------------------------------------------------------------------- /images/shangtang-aidc.webp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/shangtang-aidc.webp -------------------------------------------------------------------------------- /images/shortest-path.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/shortest-path.jpg -------------------------------------------------------------------------------- /images/simulated-sessions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/simulated-sessions.png -------------------------------------------------------------------------------- /images/size-affects-latency.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/size-affects-latency.png -------------------------------------------------------------------------------- /images/sla-slo-sli-relation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sla-slo-sli-relation.png -------------------------------------------------------------------------------- /images/slo-spec.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/slo-spec.png -------------------------------------------------------------------------------- /images/slo-vs-sla-vs-sli-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/slo-vs-sla-vs-sli-1.jpg -------------------------------------------------------------------------------- /images/snapshot-vs-log.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/snapshot-vs-log.png -------------------------------------------------------------------------------- /images/sowalker-background.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sowalker-background.jpg -------------------------------------------------------------------------------- /images/sowalker-contribution.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sowalker-contribution.jpg -------------------------------------------------------------------------------- /images/sowalker-evaluation-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sowalker-evaluation-1.jpg -------------------------------------------------------------------------------- /images/sowalker-evaluation-2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sowalker-evaluation-2.jpg -------------------------------------------------------------------------------- /images/sowalker-motivation.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/sowalker-motivation.jpg -------------------------------------------------------------------------------- /images/spark-arch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/spark-arch.png -------------------------------------------------------------------------------- /images/spark-graphx-app.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/spark-graphx-app.gif -------------------------------------------------------------------------------- /images/spark-graphx-property_graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/spark-graphx-property_graph.png -------------------------------------------------------------------------------- /images/spark-graphx-uml.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/spark-graphx-uml.jpg -------------------------------------------------------------------------------- /images/subgraph-construction-is-important.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/subgraph-construction-is-important.jpg -------------------------------------------------------------------------------- /images/swift_network_diagram-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/swift_network_diagram-1.png -------------------------------------------------------------------------------- /images/tail-latency-with-scaling-and-outliers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/tail-latency-with-scaling-and-outliers.png -------------------------------------------------------------------------------- /images/tencent-cloud-2021.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/tencent-cloud-2021.png -------------------------------------------------------------------------------- /images/tencent-fact-2016.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/tencent-fact-2016.png -------------------------------------------------------------------------------- /images/tencent-guizhou.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/tencent-guizhou.jpg -------------------------------------------------------------------------------- /images/tencent-qingyuan.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/tencent-qingyuan.jpg -------------------------------------------------------------------------------- /images/text-book.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/text-book.jpg -------------------------------------------------------------------------------- /images/the-long-tail.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/the-long-tail.jpg -------------------------------------------------------------------------------- /images/toolbox-small.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/toolbox-small.png -------------------------------------------------------------------------------- /images/toolbox.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/toolbox.png -------------------------------------------------------------------------------- /images/understanding-disk-failure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/understanding-disk-failure.png -------------------------------------------------------------------------------- /images/us_datacenters.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/us_datacenters.jpg -------------------------------------------------------------------------------- /images/utah-data-center-entrance.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/utah-data-center-entrance.jpg -------------------------------------------------------------------------------- /images/val-nas18.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/val-nas18.jpg -------------------------------------------------------------------------------- /images/vertex-activity-trend.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/vertex-activity-trend.jpg -------------------------------------------------------------------------------- /images/vertex-centric.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/vertex-centric.jpg -------------------------------------------------------------------------------- /images/vertex-computing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/vertex-computing.png -------------------------------------------------------------------------------- /images/vertex-edge-computing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/vertex-edge-computing.png -------------------------------------------------------------------------------- /images/vhost-target.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/vhost-target.png -------------------------------------------------------------------------------- /images/web-request-tail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/web-request-tail.png -------------------------------------------------------------------------------- /images/what-happens-online-in-60-seconds.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/what-happens-online-in-60-seconds.png -------------------------------------------------------------------------------- /images/white-flower-icon-5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/white-flower-icon-5.jpg -------------------------------------------------------------------------------- /images/white-flower-icon-6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/white-flower-icon-6.png -------------------------------------------------------------------------------- /images/who-moved-my-cheese.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/who-moved-my-cheese.jpg -------------------------------------------------------------------------------- /images/workload-plot-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/workload-plot-1.jpg -------------------------------------------------------------------------------- /images/workload-plot-2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/workload-plot-2.jpg -------------------------------------------------------------------------------- /images/xAI-Colossus-net.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/xAI-Colossus-net.jpg -------------------------------------------------------------------------------- /images/xAI-Colossus-rack.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/xAI-Colossus-rack.jpg -------------------------------------------------------------------------------- /images/xAI-Colossus-url.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/xAI-Colossus-url.jpg -------------------------------------------------------------------------------- /images/xAI-Colossus-web.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/xAI-Colossus-web.jpg -------------------------------------------------------------------------------- /images/xAI-Colossus.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/xAI-Colossus.jpeg -------------------------------------------------------------------------------- /images/yi-zhuan.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/yi-zhuan.jpg -------------------------------------------------------------------------------- /images/计算机系统结构（大数据2201-2202）.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cs-course/data-center-course/ab6e68370138a74e555e4df7b072c90965e9aad4/images/计算机系统结构（大数据2201-2202）.jpg -------------------------------------------------------------------------------- /memory-hierarchy.md: -------------------------------------------------------------------------------- 1 | --- 2 | marp: true 3 | theme: gaia 4 | title: 计算机系统结构 5 | # size: 4:3 6 | math: katex 7 | --- 8 | 9 | 10 | 11 | # 存储系统 12 | 13 | **施展** 14 | 武汉光电国家研究中心 15 | 光电信息存储研究部 16 | 17 | 18 | 19 | 20 | --- 21 | 22 | ## 内容大纲 23 | 24 | 25 | 26 | - 存储系统基础 27 | - 存储系统量化分析 28 | - 缓存基础 29 | - 缓存优化方法 30 | - 虚拟存储器 31 | 32 | --- 33 | 34 | ## 存储系统基础 35 | 36 | ### 何谓存储？ 37 | 38 | - 信息的传递 39 | - 跨越**空间**，是为**通讯** 40 | - 通讯介质，是为**网络** 41 | - 物理状态：声、光、电、量子…… 42 | - 跨越**时间**，是为**记忆** 43 | - 记忆介质，是为**存储** 44 | - 物理状态：形、影、磁、相…… 45 | 46 | 47 | 48 | --- 49 | 50 | ### 处理器和内存的性能差越来越大 51 | 52 | 58 | 59 | ![latency-gap h:500](images/latency-gap.png) 60 | 61 | Four-issue 3GHz superscalar accessing 100ns DRAM could execute 1,200 instructions during time for one memory access! 62 | 63 | 64 | 65 | --- 66 | 67 | 74 | 75 | ![bg fit](images/flops-bytes-unbalance.jpg) 76 | 77 | 78 | 79 | 80 | 81 | --- 82 | 83 | ### 处理器物理尺寸影响延迟时间 84 | 85 | 91 | 92 | ![latency-gap h:500](images/size-affects-latency.png) 93 | 94 | --- 95 | 96 | ### 访问数据的代价 97 | 98 | 106 | 107 | |操作|用时|||| 108 | |:-|-:|-:|-:|:-| 109 | | L1 cache reference | 0.5 ns | | | | 110 | | Branch mispredict | 5 ns | | | | 111 | | L2 cache reference | 7 ns | | | 14x L1 cache | 112 | | Mutex lock/unlock | 25 ns | | | | 113 | | Main memory reference | 100 ns | | | 20x L2 cache, 200x L1 cache | 114 | | Compress 1K bytes with Zippy | 3,000 ns | 3 us | | | 115 | | Send 1K bytes over 1 Gbps network | 10,000 ns | 10 us | | | 116 | | Read 4K randomly from SSD* | 150,000 ns | 150 us | | ~1GB/sec SSD | 117 | | Read 1 MB sequentially from memory | 250,000 ns | 250 us | | | 118 | | Round trip within same datacenter | 500,000 ns | 500 us | | | 119 | | Read 1 MB sequentially from SSD* | 1,000,000 ns | 1,000 us | 1 ms | ~1GB/sec SSD, 4X memory | 120 | | Disk seek | 10,000,000 ns | 10,000 us | 10 ms | 20x datacenter roundtrip | 121 | | Read 1 MB sequentially from disk | 20,000,000 ns | 20,000 us | 20 ms | 80x memory, 20X SSD | 122 | | Send packet CA->Netherlands->CA |150,000,000 ns | 150,000 us | 150 ms | | 123 | 124 | Notes 125 | 126 | $1 ns = 10^{-9} seconds$ 127 | $1 us = 10^{-6} seconds = 1,000 ns$ 128 | $1 ms = 10^{-3} seconds = 1,000 us = 1,000,000 ns$ 129 | 130 | Credit 131 | 132 | By Jeff Dean: 133 | Originally by Peter Norvig: 134 | 135 | --- 136 | 137 | 143 | 144 | ![bg fit](images/Latency-Numbers-Every-Programmer-Should-Know.png) 145 | 146 | Source: 147 | 148 | --- 149 | 150 | ## 存储系统的基础知识 151 | 152 | 153 | 154 | --- 155 | 156 | ### 存储系统层次结构 157 | 158 | - 1. 计算机系统结构设计关键问题之一 159 | - **如何以合理的价格，设计容量和速度都满足计算机系统要求的存储器系统？** 160 | - 2. 人们对三个指标的要求 161 | - 容量**大**、速度**快**、成本**低** 162 | - 3. 三个要求之间的矛盾 163 | - 速度越快，每位价格越高； 164 | - 容量越大，每位价格越低； 165 | - 容量越大，速度越慢。 166 | 167 | --- 168 | 169 | ### 矛盾如何应对？ 170 | 171 | - 4. 解决方法：**采用多种存储器技术，构成多级存储层次结构**。 172 | - 程序访问的**局部性原理**：对于绝大多数程序来说，程序所访问的指令和数据在地址上不是均匀分布的，而是相对**簇聚**的。 173 | - 程序访问的局部性包含两个方面 174 | - **时间局部性**：程序马上将要用到的信息很可能就是现在正在使用的信息。 175 | - **空间局部性**：程序马上将要用到的信息很可能与现在正在使用的信息在存储空间上是相邻的。 176 | 177 | --- 178 | 179 | ### 典型内存存取模式 180 | 181 | ![access-pattern h:500](images/access-pattern.png) 182 | 183 | --- 184 | 185 | ![bg fit](images/access-pattern-real.png) 186 | 187 | --- 188 | 189 | ### 多级层次结构 190 | 191 | 196 | 197 | - 5. 存储系统的多级层次结构 198 | 199 | ![bg fit](images/memory-hierarchy.png) 200 | 201 | --- 202 | 203 | 209 | 210 | ![bg fit](images/memory-hierarchy.jpg) 211 | 212 | 213 | 214 | --- 215 | 216 | - 假设第$i$个存储器Mi的访问时间为$T_i$，容量为$S_i$，平均每位价格为$C_i$ 217 | - 访问时间: $T_1 < T_2 < \dots < T_n$ 218 | - 容量: $S_1 < S_2 < \dots < S_n$ 219 | - 平均每位价格: $C_1 > C_2 > \dots > C_n$ 220 | - 整个存储系统要达到的目标 221 | - 从CPU来看，速度接近于$M_1$，而容量和每位价格都接近于$M_n$。 222 | - 存储器越靠近CPU，则CPU对它的访问频度越高，而且最好大多数的访问都能在$M_1$完成。 223 | 224 | --- 225 | 226 | ### 存储层次的性能参数 227 | 228 | - 这里仅考虑由M1和M2构成的两级存储层次 229 | - $M_1$的参数: $S_1, T_1, C_1$ 230 | - $M_2$的参数: $S_2, T_2, C_2$ 231 | 232 | ![m1-m2 h:300](images/m1-m2.png) 233 | 234 | --- 235 | 236 | - 1. 存储容量 **S** 237 | - 一般来说，整个存储系统的容量即是第二级存储器$M_2$的容量，即$S=S_2$。 238 | - 2. 每位价格 **C** 239 | 240 | $$ 241 | C=\frac{C_1 S_1 + C_2 S_2}{S_1 + S_2} 242 | $$ 243 | 244 | 当$S1 \ll S2$时，$C \approx C_2$。 245 | 246 | --- 247 | 248 | - 3. 命中率 **H** 和不命中率 **F** 249 | 250 | 命中率：CPU访问存储系统时，在$M_1$中找到所需信息的概率。 251 | 252 | $$ 253 | H = \frac{N_1}{N_1 + N_2} 254 | $$ 255 | 256 | - $N_1$: 访问 $M_1$ 的次数 257 | - $N_2$: 访问 $M_2$ 的次数 258 | 259 | 不命中率: $F=1-H$ 260 | 261 | --- 262 | 263 | - 4. 平均访问时间 $T_A$ 264 | 265 | $$ 266 | \begin{align*} 267 | T_A &= HT_1 + (1 - H)(T_1 + T_M) 268 | \\ 269 | &= T_1 + (1 - H)T_M 270 | \\ 271 | 或 \quad T_A &= T_1 + FT_M 272 | \end{align*} 273 | $$ 274 | 275 | --- 276 | 277 | - 分两种情况来考虑CPU的一次访存: 278 | - 若命中，则访问时间即为 $T_1$ (**命中时间**) 279 | - 若不命中，则访问时间为: 280 | 281 | $$ 282 | \begin{align*} 283 | T_2 + T_B + T_1 &= T_M + T_1 284 | \\ 285 | T_M &= T_2 + T_B 286 | \end{align*} 287 | $$ 288 | 289 | - **不命中开销** $T_M$: 从向 $M_2$ 发出访问请求到把整个数据块调入 $M_1$ 中所需的时间。 290 | - 传送一个信息块所需的时间为 $T_B$。 291 | 292 | --- 293 | 294 | ### 三级存储系统 295 | 296 | - 构成 297 | - Cache（高速缓冲存储器） 298 | - 主存储器 299 | - 磁盘存储器（辅存） 300 | - 可以看成是由"**Cache—主存**"层次和"**主存—辅存**"层次构成的系统。 301 | 302 | ![3-tier-memory h:200](images/3-tier-memory.png) 303 | 304 | --- 305 | 306 | ![bg right fit](images/cache-memory-storage.png) 307 | 308 | - 从主存的角度来看 309 | - 弥补主存速度的不足 310 | - **Cache－主存**层次 311 | - 弥补主存容量的不足 312 | - **主存－辅存**层次 313 | 314 | --- 315 | 316 | "**Cache－主存**"与"**主存－辅存**"层次的区别 317 | 318 | ||Cache－主存|主存－辅存| 319 | |:-|:-|:-| 320 | |目的|弥补主存速度的不足|弥补主存容量的不足| 321 | |存储管理实现|主要由专用硬件实现|主要由软件实现| 322 | |访问速度的比值|几比一|几万比一| 323 | |典型的块(页)大小|几十个字节|几百到几千个字节| 324 | |CPU对次级的访问方式|可直接访问|均通过第一级| 325 | |不命中时CPU是否切换|不切换|切换到其他进程| 326 | 327 | --- 328 | 329 | ### 存储层次的四个问题 330 | 331 | - 当把一个块调入高一层(靠近CPU)存储器时，可以放在哪些位置上? 332 | - **映象规则** 333 | - 当所要访问的块在高一层存储器中时，如何找到该块? 334 | - **查找算法** 335 | - 当发生不命中时，应替换哪一块？ 336 | - **替换算法** 337 | - 当进行写访问时，应进行哪些操作? 338 | - **写策略** 339 | 340 | --- 341 | 342 | ## Cache基本知识 343 | 344 | 345 | 346 | --- 347 | 348 | ### 基本结构和原理 349 | 350 | - 存储空间分割与地址计算 351 | - Cache和主存分块 352 | - Cache是按块进行管理的。Cache和主存均被分割成大小相同的块。信息以块为单位调入Cache。 353 | - **主存块地址 (块号)** 用于查找该块在Cache中的位置。 354 | - **块内位移**用于确定所访问的数据在该块中的位置。 355 | 356 | ![cache-block-address](images/cache-block-address.png) 357 | 358 | --- 359 | 360 | - Cache的基本工作原理示意图 361 | 362 | ![cache-working-flow h:500](images/cache-working-flow.png) 363 | 364 | --- 365 | 366 | ### 映像规则 367 | 368 | - 全相联 369 | - 主存中的任一块可以被放置到Cache中的**任意一个位置** 370 | - 直接相联 371 | - 主存中的每一块只能被放置到Cache中**唯一的一个位置** 372 | - 组相联 373 | - 主存中的每一块可以被放置到Cache中**唯一的一个组**中的任何一个位置 374 | 375 | --- 376 | 377 | 385 | 386 | ![bg right](images/direct-mapping.png) 387 | 388 | - Word Size $1B$ 389 | - Cache Size $C = 256B$ 390 | - Number of Blocks in Cache $B = 32$ 391 | - Bits to reference a block $b = log_{2}B = 5$ 392 | - Size of one Block/Frame = $\frac{256B}{32} = 8B$ 393 | - Words per Block/Frame $W = \frac{8B}{1B} = 8$ 394 | - Bits to reference Word within a block 395 | - $w = log_{2}W = 3$ 396 | - Primary Memory $P = 64KB$ 397 | - Bits to reference Primary Memory 398 | - $p = log_{2}(\frac{P}{w}) = 16$ 399 | - Number of Frames in Primary Memory 400 | - $F = 64KB/8B = 8192$ 401 | - Tag length $p - w - b = 8$. 402 | 403 | 404 | 405 | --- 406 | 407 | 415 | 416 | ![bg right](images/set-associative-mapping.png) 417 | 418 | - Word Size $1B$ 419 | - Bits to reference Word within a block $w = 3$ 420 | - Words per Block/Frame $W = 2^3 = 8$ 421 | - Size of one Block/Frame = $8 \times 1B = 8B$ 422 | - Number of slots in a set $N = 4$ 423 | - Size of one set = $4 * 8B = 32B$ 424 | - Number of Sets in Cache = 32 425 | - Bits to reference a set 426 | - $b - n = log_{2}32 = 5$ 427 | - Cache Size $C = 32 \times 32B = 1KB$ 428 | - Primary Memory P = 64KB 429 | - Bits to reference Primary Memory 430 | - $p = log_{2}(\frac{P}{W}) = 16$ 431 | - Number of Frames in Primary Memory 432 | - $F = 64KB/8B = 8192$ 433 | - Tag length $p - w - (b - n) = 8$. 434 | 435 | --- 436 | 437 | - n路组相联 438 | 439 | || n (路数) | G (组数) | 440 | |:-|:-|:-| 441 | |全相联|M|1| 442 | |直接映象|1|M| 443 | |组相联|$1 10 | 11 | # 多处理器 12 | 13 | **施展** 14 | 武汉光电国家研究中心 15 | 光电信息存储研究部 16 | 17 | 18 | 19 | 20 | --- 21 | 22 | ## 内容大纲 23 | 24 | 25 | 26 | - 多处理器基础 27 | - 互联网络 28 | - 互联函数 29 | - 静态互联网络 30 | - 动态互联网络 31 | - 缓存一致性问题 32 | - 同步问题 33 | 34 | --- 35 | 36 | ## 多处理器基础 37 | 38 | --- 39 | 40 | ## 互联函数 41 | 42 | --- 43 | 44 | ## 静态互联网络 45 | 46 | --- 47 | 48 | ## 动态互联网络 49 | 50 | --- 51 | 52 | ## 缓存一致性问题 53 | 54 | --- 55 | 56 | ## 同步问题 57 | 58 | --- 59 | 60 | ## 参考文献 61 | 62 | 67 | 68 | 1. [...](https://link.springer.com/chapter/10.1007%2F978-3-030-68110-4_17), MASCOTS 2020. 69 | 2. [...](https://ieeexplore.ieee.org/document/9043685), TPDS 2020. 70 | -------------------------------------------------------------------------------- /qos-guarantee.md: -------------------------------------------------------------------------------- 1 | --- 2 | marp: true 3 | theme: gaia 4 | title: 数据中心技术 5 | # size: 4:3 6 | math: katex 7 | --- 8 | 9 | 10 | 11 | # 存储服务质量保障问题 12 | 13 | **施展** 14 | 武汉光电国家研究中心 15 | 光电信息存储研究部 16 | 17 | 18 | 19 | 20 | --- 21 | 22 | ## 内容大纲 23 | 24 | 25 | 26 | - 存储服务质量保障背景 27 | - 资源决策问题 28 | - 经典方法和实践 29 | 30 | --- 31 | 32 | ## 广泛应用的云 33 | 34 | ![bg](images/cloud-storage.jpg) 35 | 36 | --- 37 | 38 | 44 | 45 | ![bg](images/Cloud-Computing-as-a-Service-Revenue.png) 46 | 47 | Source: 48 | 49 | --- 50 | 51 | 57 | 58 | ![bg](images/Top-Four-Cloud-Infrastructure-Providers.jpg) 59 | 60 | Source: 61 | 62 | --- 63 | 64 | 71 | 72 | ![bg opacity:.3](images/Top-Four-Cloud-Infrastructure-Providers.jpg) 73 | 74 | Pandemic boosts cloud consumption by a third in Q3 2020 75 | 76 | --- 77 | 78 | 84 | 85 | ![bg fit](images/CIS_Q320.jpg) 86 | 87 | Source: 88 | 89 | 90 | 91 | 92 | 93 | --- 94 | 95 | ## 服务器整合 96 | 97 | 102 | 103 | ![h:450](images/cmp-consolidated-servers.png) 104 | 105 | Source: [Virtual hierarchies to support server consolidation, ISCA '07](https://dl.acm.org/doi/10.1145/1250662.1250670) 106 | 107 | --- 108 | 109 | ### 机遇 110 | 111 | ![h:400](images/server-consolidation.svg) 112 | 113 | 118 | 119 | --- 120 | 121 | ### 意义 122 | 123 | ![bg fit](images/server-consolidation-scenario.svg) 124 | 125 | --- 126 | 127 | ### 更丰富意义 128 | 129 | 134 | 135 | ![bg right fit](images/beyond-server-consolidation.png) 136 | 137 | 1) 规模化应用部署 138 | 2) 扩展、可靠和安全 139 | 3) 使能服务质量保障 140 | 141 | Source: [Beyond Server Consolidation: Server consolidation helps companies improve resource utilization, but virtualization can help in other ways, too. Queue 2008](https://dl.acm.org/doi/10.1145/1348583.1348590) 142 | 143 | --- 144 | 145 | ### 案例 146 | 147 | 153 | 154 | ![bg fit](images/CPU-Utilization-on-a-vSphere-Host-Before-and-After-Consolidation.jpg) 155 | 156 | Source: [Update on Virtualizing SQL](https://blogs.vmware.com/apps/2012/11/update-on-virtualizing-sql.html) 157 | 158 | --- 159 | 160 | 166 | 167 | ![bg](images/SQL-and-Oracle-Database-Consolidation-Reduces-Software-Costs-by-70-Percent.jpg) 168 | 169 | Source: [Update on Virtualizing SQL](https://blogs.vmware.com/apps/2012/11/update-on-virtualizing-sql.html) 170 | 171 | --- 172 | 173 | ## 多租户存储 174 | 175 | 180 | 181 | ![h:450](images/multi-tenancy-degrees.png) 182 | 183 | Source: 184 | [What is a multi-tenant system?](https://www.prudentdevs.club/mts/) 185 | 186 | 187 | 188 | 189 | --- 190 | 191 | 197 | 198 | ![bg](images/multitenancy-web.png) 199 | 200 | Source: 201 | 202 | --- 203 | 204 | ## 服务等级协议、服务等级目标、服务等级指标 205 | 206 | - **服务等级协议**(SLA)：协议双方(服务提供者和用户)签订的一个合约或协议，这个合约**规范了双方的商务关系或部分商务关系**。 207 | - SLA是服务提供者与用户之间协商并签订的一个具有**法律约束力**的合同，合同规定了在服务提供过程中双方所承担的商务条款。 208 | - **服务等级目标**(SLO)：指定服务所提供功能的一种**期望状态**。 209 | - **服务等级指标**(SLI)：经过仔细定义的**测量指标**，它根据不同系统特点确定要测量什么。 210 | 211 | Source: [SLO（服务等级目标）与 SLA（服务等级协议）](https://xie.infoq.cn/article/eda3b32806bc800173793118e) 212 | 213 | --- 214 | 215 | 221 | 222 | ![bg](images/slo-vs-sla-vs-sli-1.jpg) 223 | 224 | Source: [SLA vs. SLO vs. SLI: What’s the difference?](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) 225 | 226 | --- 227 | 228 | 234 | 235 | ![bg](images/sla-slo-sli-relation.png) 236 | 237 | Source: [The Key Differences Between SLI, SLO, and SLA in SRE](https://dzone.com/articles/the-key-differences-between-sli-slo-and-sla-in-sre) 238 | 239 | --- 240 | 241 | ### 归纳一下 242 | 243 | 249 | 250 | - SLA是法律文书而非技术文档 251 | - SLO是细分后的具体目标承诺 252 | - SLI是监控采集的实际观测值 253 | 254 | --- 255 | 256 | ### 归纳一下… 257 | 258 | 264 | 265 | - SLA是法律文书而非技术文档，**重在严格约束而非技术实现** 266 | - SLO是细分后的具体目标承诺，**重在明确量化而非如何测量** 267 | - SLI是监控采集的实际观测值，**需要精辟选择合适指标** 268 | 269 | --- 270 | 271 | ### 归纳一下…… 272 | 273 | 279 | 280 | - SLA是法律文书而非技术文档，重在严格约束而非技术实现，**难在协商** 281 | - SLO是细分后的具体目标承诺，重在明确量化而非如何测量，**难在提炼** 282 | - SLI是监控采集的实际观测值，需要精辟选择合适指标，**难在精准观测** 283 | 284 | --- 285 | 286 | ### 范例 —— Web 服务器可用性和延迟 287 | 288 | - 考察Web服务器**可用性**。指标为收到的HTTP请求数和成功响应的HTTP请求数。若要了解Web服务器是否胜任用户需求，可以分析成功请求数与请求总数的比率。 289 | - 用成功请求数除以请求总数，可得一个比率。乘以100得到百分比。例如，Web服务器收到100个请求，成功响应了80个，则比率为0.8，即可用性为80%。 290 | - 另例，考察Web**服务延迟**相关的度量值。需观测不到10毫秒内完成的操作与操作总数的比率。 291 | - 若阈值时间内返回80个请求，请求总数100，则延迟满足率80%。 292 | 293 | Source: [服务级别指标(SLI)和服务级别目标(SLO)示例](https://docs.microsoft.com/zh-cn/learn/modules/improve-reliability-monitoring/7-sli-slo) 294 | 295 | --- 296 | 297 | ### 一般评价标准 298 | 299 | ![h:450](images/slo-spec.png) 300 | 301 | --- 302 | 303 | ## 问题：过度供应 304 | 305 | 310 | 311 | ![w:1150](images/provisioning.png) 312 | 313 | Source: [A View of Cloud Computing. CACM 2010](https://dl.acm.org/doi/10.1145/1721654.1721672) 314 | 315 | --- 316 | 317 | ## 问题：性能干扰 318 | 319 | 324 | 325 | ![h:500](images/performance-isolation-1.png) ![h:500](images/performance-isolation-2.png) 326 | 327 | Source: [Enforcing performance isolation across virtual machines in Xen, Middleware '06](https://dl.acm.org/doi/10.5555/1515984.1516011) 328 | 329 | --- 330 | 331 | ## 经典机制 332 | 333 | - **操作系统**：I/O管理器 334 | - **虚拟化环境**：虚拟机管理器 335 | - **存储系统**：I/O调度模块 336 | 337 | --- 338 | 339 | ### 公平排队的悠久历史 340 | 341 | ![bg right fit](images/Stochastic-Fairness-Queuing.png) 342 | 343 | 网络领域很早就开始探索…… 344 | 345 | - [Stochastic fairness queueing, INFOCOM '90](https://ieeexplore.ieee.org/document/91316) 346 | - [On Measuring Fairness in Queues, Advances in Applied Probability 2004](https://www.jstor.org/stable/4140415) 347 | 348 | 349 | 350 | --- 351 | 352 | ### 案例1：cgroup (Linux内核) 353 | 354 | 363 | 364 | - [Block IO Controller](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/blkio-controller.html) 365 | - [BFQ (Budget Fair Queueing)](https://www.kernel.org/doc/html/latest/block/bfq-iosched.html), [BFQ I/O Scheduler](https://lwn.net/Articles/306641/), [Comparison among 6 schedulers](http://algo.ing.unimo.it/people/paolo/disk_sched/comparison.php) 366 | 367 | ![h:350](images/cfq-cgroup.png) 368 | 369 | Source: [Variations on fair I/O schedulers](https://lwn.net/Articles/309400/), 2008 370 | 371 | 386 | 387 | --- 388 | 389 | ### 案例2：libvirt (KVM, Xen, VMware, QEMU) 390 | 391 | 399 | 400 | - [virsh blkiotune](https://www.libvirt.org/manpages/virsh.html#blkiotune) 401 | - [Quality of Service (QoS) in OpenStack](https://wiki.openstack.org/wiki/QoS) 402 | 403 | ![h:300](images/cgroup-vs-qemu.png) ![h:300](images/openstack-frontend-qos.png) 404 | 405 | Source: [IO Throttling in QEMU](https://www.linux-kvm.org/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf), [KVM Forum 2011](https://www.linux-kvm.org/page/KVM_Forum_2011); [OpenStack - Basic volume quality of service](https://docs.openstack.org/cinder/latest/admin/blockstorage-basic-volume-qos.html); 406 | 407 | --- 408 | 409 | ### 案例3：Object Storage (Ceph) 410 | 411 | 419 | 420 | - [QoS Study with mClock and WPQ Schedulers](https://ceph.com/en/news/blog/2021/qos-study-with-mclock-and-wpq-schedulers/) 421 | - [The dmclock distributed quality of service algorithm](https://github.com/ceph/dmclock) 422 | 423 | ![h:330](images/mclock.png) ![h:330](images/vhost-target.png) 424 | 425 | Source: [mClock: handling throughput variability for hypervisor IO scheduling](https://dl.acm.org/doi/10.5555/1924943.1924974), [OSDI'10](https://www.usenix.org/legacy/events/osdi10/tech/); 426 | [Using Ceph RBD as a QEMU Storage](https://insujang.github.io/2021-03-04/using-ceph-rbd-as-a-qemu-storage/); 427 | 428 | --- 429 | 430 | ## 决策过程 431 | 432 | 440 | 441 | ![bg right fit](images/QoS-managed-distributed-storage-system.gif) 442 | 443 | - **MAPE**-loop steps 444 | - **Monitoring** QoS metrics 445 | - **Analyzing** divergence of current performance from goals and updating a performance model 446 | - **Planning** decisions about actions needed to achieve performance goals 447 | - **Executing** those actions 448 | 449 | Source: [Decision-Making Approaches for Performance QoS in Distributed Storage Systems: A Survey, TPDS 2019](https://ieeexplore.ieee.org/document/8618414) 450 | 451 | --- 452 | 453 | ## 问题描述 454 | 455 | - 目标 456 | - $(r_i, l^{r}_i, l^{w}_i)$ 457 | - ${Average\ latency\ over\ time}\ w \leq f_r\cdot l^{r}_i + (1 - f_r)\cdot l^{w}_i$ 458 | - 方法 459 | - 静态：任务初始资源如何分配 460 | - 调整资源配置 461 | - 动态：负载、系统变化如何动态适应 462 | - 流量控制 463 | 464 | --- 465 | 466 | ### 方法1：控制论 467 | 468 | ![w:1150](images/block-diagram-of-feedback-control-system.svg) 469 | 470 | --- 471 | 472 | ### 方法2：约束优化 473 | 474 | ![h:500](images/constrained-optimization-l.jpg) 475 | 476 | --- 477 | 478 | ### 方法3：机器学习 479 | 480 | ![w:1150](images/reinforcement-Learning.svg) 481 | 482 | --- 483 | 484 | ## 实践环境 485 | 486 | - 大数据存储实验课 487 | - 对象存储入门实验 488 | 489 | --- 490 | 491 | ### 实验思路 492 | 493 | - 调节并发数，观察性能约束与提升的空间 494 | 495 | ```python 496 | with ThreadPoolExecutor(max_workers=1) as executor: 497 | futures = [ executor.submit(access_obs for i in range(100) ] 498 | ``` 499 | 500 | - 尝试反馈控制并发数 501 | 502 | ```bash 503 | s3bench ... -numClients=8 ... 504 | ``` 505 | 506 | - Minio API 子系统节流实践 507 | - 508 | 509 | --- 510 | 511 | ## 进一步思考：如何精确控制？ 512 | 513 | - 波动性与突发性问题 514 | - 比例积分控制？ 515 | - 机器学习序列预测？ 516 | - …… 517 | 518 | --- 519 | 520 | ### 我们的工作 521 | 522 | 531 | 532 | - [Storage Sharing Optimization Under Constraints of SLO Compliance and Performance Variability](https://ieeexplore.ieee.org/document/7498602), ToSC 2019. 533 | - [Customizable SLO and Its Near-Precise Enforcement for Storage Bandwidth](https://dl.acm.org/doi/10.1145/2998454), ToS 2017. 534 | - [PSLO: enforcing the Xth percentile latency and throughput SLOs for consolidated VM storage](https://dl.acm.org/doi/10.1145/2901318.2901330), EuroSys 2016. 535 | 536 | ![h:270](images/PI-Controller.png) 537 | 538 | Source: [PID Controllers Explained](https://blog.opticontrols.com/archives/344) 539 | 540 | --- 541 | 542 | ## 参考文献 543 | 544 | 549 | 550 | 1. [Decision-Making Approaches for Performance QoS in Distributed Storage Systems: A Survey](https://ieeexplore.ieee.org/document/8618414), TPDS 2019. 551 | 2. [Server consolidation techniques in virtualized data centers of cloud environments: A systematic literature review](https://onlinelibrary.wiley.com/doi/abs/10.1002/spe.2582), SPE 2018. 552 | 3. [MittOS: Supporting Millisecond Tail Tolerance with Fast Rejecting SLO-Aware OS Interface](https://dl.acm.org/doi/10.1145/3132747.3132774), SOSP 2017 553 | 4. [Crystal: software-defined storage for multi-tenant object stores](https://www.usenix.org/conference/fast17/technical-sessions/presentation/gracia-tinedo), FAST 2017 554 | 5. [Argon: Performance Insulation for Shared Storage Servers](https://www.usenix.org/legacy/events/fast07/tech/wachs.html), FAST 2007. 555 | 6. [Façade: Virtual Storage Devices with Performance Guarantees](https://www.usenix.org/legacy/events/fast03/tech/lumb.html), FAST 2003. 556 | -------------------------------------------------------------------------------- /storage-system.md: -------------------------------------------------------------------------------- 1 | --- 2 | marp: true 3 | theme: gaia 4 | title: 计算机系统结构 5 | # size: 4:3 6 | math: katex 7 | --- 8 | 9 | 10 | 11 | # I/O系统 12 | 13 | **施展** 14 | 武汉光电国家研究中心 15 | 光电信息存储研究部 16 | 17 | 18 | 19 | 20 | --- 21 | 22 | ## 内容大纲 23 | 24 | 25 | 26 | - I/O系统基础 27 | - I/O系统评价 28 | - I/O系统实例——磁盘阵列 29 | 30 | --- 31 | 32 | ## I/O系统基础 33 | 34 | --- 35 | 36 | ## I/O系统评价 37 | 38 | --- 39 | 40 | ## I/O系统实例——磁盘阵列 41 | 42 | --- 43 | 44 | ## 参考文献 45 | 46 | 51 | 52 | 1. [...](https://link.springer.com/chapter/10.1007%2F978-3-030-68110-4_17), MASCOTS 2020. 53 | 2. [...](https://ieeexplore.ieee.org/document/9043685), TPDS 2020. 54 | --------------------------------------------------------------------------------