├── .gitattributes ├── .gitignore ├── README.md ├── examples ├── README.md └── asio │ ├── Makefile │ ├── async_timer.cc │ ├── async_timer_parallel.cc │ ├── async_timer_with_args.cc │ ├── async_timer_with_member_function.cc │ └── sync_timer.cc └── sources ├── README.md ├── ebooks ├── Large Scale C++ Software Design.pdf ├── 《Redis设计与实现》.pdf ├── 《代码大全》(第2版).pdf ├── 《汇编语言》(第4版) .pdf └── 《深入理解计算机系统》（第3版）.pdf ├── emulator_dos ├── DEBUG.EXE ├── EDIT.COM ├── EXE2BIN.EXE ├── LIB.EXE ├── LINK.EXE ├── MASM.EXE ├── README.md └── masm.IMg └── papers ├── Google-Bigtable中文版_1.0.pdf ├── Google-File-System中文版_1.0.pdf ├── Google-MapReduce中文版_1.0.pdf ├── Large-scale cluster management at Google with Borg.pdf ├── NanoLog- A Nanosecond Scale Logging System.pdf ├── The Log-Structured Merge-Tree (LSM-Tree).pdf ├── The ϕ Accrual Failure Detector.pdf └── What Every Programmer Should Know About Memory.pdf /.gitattributes: -------------------------------------------------------------------------------- 1 | *.pdf filter=lfs diff=lfs merge=lfs -text 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .vscode/ 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 我的分布式存储之路 2 | 3 | * sources/：一些pdf资源和常用工具 4 | * examples/：学习一些框架或者其他知识时写的一些demo代码 5 | 6 | ## 文档目录 7 | 8 | * [C/CPP](#ccpp) 9 | * [书籍文档](#书籍文档) 10 | * [代码框架](#代码框架) 11 | * [基础知识](#基础知识) 12 | * [汇编语言](#汇编语言) 13 | * [分布式存储](#分布式存储) 14 | * [单机引擎](#单机引擎) 15 | 16 | ## C/CPP 17 | 18 | ### 书籍文档 19 | 20 | * [《现代C++教程》](https://changkun.de/modern-cpp/) - 国人写的一本关于现代C++语法的导读 21 | * [Large Scale C++ Software Design](https://github.com/wonter/learning-distributed-storage/blob/master/sources/ebooks/Large%20Scale%20C%2B%2B%20Software%20Design.pdf) - 需要有一定的开发经验后才适合阅读，是一本很老的书了（96年出版的），不过里面有部分思想现在还可以学习，`物理设计概念`这一部分。中文名字叫《大规模c++程序设计》，但中文版翻译的据说非常烂，所以尽量还是看英文版吧 22 | * [《代码大全》（第2版）](https://github.com/wonter/learning-distributed-storage/blob/master/sources/ebooks/%E3%80%8A%E4%BB%A3%E7%A0%81%E5%A4%A7%E5%85%A8%E3%80%8B(%E7%AC%AC2%E7%89%88).pdf) - 也需要有一定的开发经验后阅读才能有收获，主要是逻辑设计概念 23 | 24 | ### 代码框架 25 | 26 | * [NanoLog](https://github.com/PlatformLab/NanoLog) - 一个纳秒级别的c++日志库，性能很极限，对原理感兴趣的话可以研究下它的[论文](https://www.usenix.org/conference/atc18/presentation/yang-stephen) 27 | * [Asio](https://think-async.com/Asio/) - 比较主流的C++异步网络框架 28 | 29 | ## 基础知识 30 | 31 | ### 汇编语言 32 | 33 | * [王爽《汇编语言》(第四版)](https://github.com/wonter/learning-distributed-storage/blob/master/sources/ebooks/%E3%80%8A%E6%B1%87%E7%BC%96%E8%AF%AD%E8%A8%80%E3%80%8B(%E7%AC%AC4%E7%89%88)%20.pdf) - 最好的汇编语言入门书之一，搭建实验环境可以参考我写的[这篇文章](http://blog.wonter.net/posts/2821c29c/) 34 | 35 | ## 分布式存储 36 | 37 | ### 分布式导论 38 | 39 | * [《设计数据密集型应用》](https://github.com/Vonng/ddia) - 神书了，入门必看 40 | * [MIT 6.824](https://pdos.csail.mit.edu/6.824/schedule.html) - 也是分布式系统入门必看，里面的实验课一定要做 41 | 42 | ### 单机引擎 43 | 44 | * [Redis](https://redis.io/) - 应该是最常见的KV服务了，内部实现原理不算复杂，可以结合源码和[《Redis设计与实现》](https://github.com/wonter/learning-distributed-storage/blob/master/sources/ebooks/%E3%80%8ARedis%E8%AE%BE%E8%AE%A1%E4%B8%8E%E5%AE%9E%E7%8E%B0%E3%80%8B.pdf)一起看 45 | * [LevelDB](https://github.com/google/leveldb) - 基于LSM树的引擎，出自Jeff Dean之手，**源码非常值得一读**，可以结合着[架构剖析](https://leveldb-handbook.readthedocs.io/zh/latest/basic.html)一起看 46 | * [RocksDB](https://github.com/facebook/rocksdb/) - Facebook在LevelDB基础上优化出来的一个引擎，应该是业界最常用的单机KV引擎了吧 47 | -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | ## 示例代码 2 | 3 | 学习一些框架或者其他知识时写的一些demo代码 4 | 5 | 代码风格遵循[Google 开源项目风格指南](https://zh-google-styleguide.readthedocs.io/en/latest/contents/)，构建使用Makefile 6 | 7 | * asio/：学习asio时写的一些小demo，需要提前安装Boost库才能成功构建 8 | -------------------------------------------------------------------------------- /examples/asio/Makefile: -------------------------------------------------------------------------------- 1 | .PHONY : all clean 2 | all: sync_timer async_timer async_timer_with_args async_timer_with_member_function async_timer_parallel 3 | clean: 4 | rm -rf sync_timer async_timer async_timer_with_args async_timer_with_member_function async_timer_parallel 5 | 6 | sync_timer: sync_timer.cc 7 | g++ -std=c++11 sync_timer.cc -lboost_chrono -o sync_timer 8 | 9 | async_timer: async_timer.cc 10 | g++ -std=c++11 async_timer.cc -lboost_chrono -o async_timer 11 | 12 | async_timer_with_args: async_timer_with_args.cc 13 | g++ -std=c++11 async_timer_with_args.cc -lboost_chrono -o async_timer_with_args 14 | 15 | async_timer_with_member_function: async_timer_with_member_function.cc 16 | g++ -std=c++11 async_timer_with_member_function.cc -lboost_chrono -o async_timer_with_member_function 17 | 18 | async_timer_parallel: async_timer_parallel.cc 19 | g++ -std=c++11 async_timer_parallel.cc -lboost_chrono -o async_timer_parallel 20 | -------------------------------------------------------------------------------- /examples/asio/async_timer.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include "boost/asio.hpp" 4 | 5 | void end_handler(const boost::system::error_code& /*ec*/) { 6 | std::cout << "end timer" << std::endl; 7 | } 8 | 9 | int main() { 10 | boost::asio::io_context io_context; 11 | boost::asio::steady_timer timer(io_context, 12 | boost::asio::chrono::seconds(3)); 13 | 14 | std::cout << "start timer" << std::endl; 15 | timer.async_wait(end_handler); 16 | 17 | io_context.run(); 18 | return 0; 19 | } 20 | -------------------------------------------------------------------------------- /examples/asio/async_timer_parallel.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include "boost/asio.hpp" 4 | #include "boost/bind/bind.hpp" 5 | #include "boost/thread/thread.hpp" 6 | 7 | class Printer { 8 | public: 9 | Printer(boost::asio::io_context &io) 10 | : strand_(boost::asio::make_strand(io)), 11 | timer1_(io, boost::asio::chrono::seconds(1)), 12 | timer2_(io, boost::asio::chrono::seconds(1)) { 13 | timer1_.async_wait(boost::asio::bind_executor( 14 | strand_, 15 | boost::bind(&Printer::print1, this, boost::placeholders::_1))); 16 | timer2_.async_wait(boost::asio::bind_executor( 17 | strand_, 18 | boost::bind(&Printer::print2, this, boost::placeholders::_1))); 19 | } 20 | 21 | void print1(const boost::system::error_code & /*ec*/) { 22 | if (counter_ < 10) { 23 | std::cout << "Timer 1: counter = " << counter_ << std::endl; 24 | ++counter_; 25 | timer1_.expires_at(timer1_.expiry() + 26 | boost::asio::chrono::seconds(1)); 27 | timer1_.async_wait(boost::asio::bind_executor( 28 | strand_, 29 | boost::bind(&Printer::print1, this, boost::placeholders::_1))); 30 | } 31 | } 32 | 33 | void print2(const boost::system::error_code & /*ec*/) { 34 | if (counter_ < 10) { 35 | std::cout << "Timer 2: counter = " << counter_ << std::endl; 36 | ++counter_; 37 | timer2_.expires_at(timer2_.expiry() + 38 | boost::asio::chrono::seconds(1)); 39 | timer2_.async_wait(boost::asio::bind_executor( 40 | strand_, 41 | boost::bind(&Printer::print2, this, boost::placeholders::_1))); 42 | } 43 | } 44 | 45 | private: 46 | boost::asio::strand strand_; 47 | boost::asio::steady_timer timer1_; 48 | boost::asio::steady_timer timer2_; 49 | int counter_; 50 | }; 51 | 52 | int main() { 53 | boost::asio::io_context io_context; 54 | 55 | std::cout << "start timer" << std::endl; 56 | Printer printer(io_context); 57 | 58 | std::thread t(boost::bind(&boost::asio::io_context::run, &io_context)); 59 | io_context.run(); 60 | t.join(); 61 | return 0; 62 | } 63 | -------------------------------------------------------------------------------- /examples/asio/async_timer_with_args.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include "boost/asio.hpp" 4 | #include "boost/bind/bind.hpp" 5 | 6 | void end_handler(const boost::system::error_code & /*ec*/, 7 | boost::asio::steady_timer *timer, int *counter) { 8 | std::cout << "counter=" << *counter << std::endl; 9 | if (*counter < 5) { 10 | ++(*counter); 11 | timer->expires_at(timer->expiry() + std::chrono::seconds(1)); 12 | timer->async_wait(boost::bind(end_handler, boost::placeholders::_1, 13 | timer, counter)); 14 | } 15 | } 16 | 17 | int main() { 18 | boost::asio::io_context io_context; 19 | 20 | int counter = 0; 21 | boost::asio::steady_timer timer(io_context, 22 | boost::asio::chrono::seconds(1)); 23 | 24 | std::cout << "start timer" << std::endl; 25 | timer.async_wait( 26 | boost::bind(end_handler, boost::placeholders::_1, &timer, &counter)); 27 | io_context.run(); 28 | return 0; 29 | } 30 | -------------------------------------------------------------------------------- /examples/asio/async_timer_with_member_function.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include "boost/asio.hpp" 4 | #include "boost/bind/bind.hpp" 5 | 6 | class Printer { 7 | public: 8 | Printer(boost::asio::io_context &io) 9 | : timer_(io, boost::asio::chrono::seconds(1)) { 10 | timer_.async_wait( 11 | boost::bind(&Printer::end_handler, this, boost::placeholders::_1)); 12 | } 13 | 14 | void end_handler(const boost::system::error_code & /*ec*/) { 15 | std::cout << "counter=" << counter_ << std::endl; 16 | if (counter_ < 5) { 17 | ++counter_; 18 | timer_.expires_at(timer_.expiry() + std::chrono::seconds(1)); 19 | timer_.async_wait(boost::bind(&Printer::end_handler, this, 20 | boost::placeholders::_1)); 21 | } 22 | } 23 | 24 | private: 25 | boost::asio::steady_timer timer_; 26 | int counter_; 27 | }; 28 | 29 | int main() { 30 | boost::asio::io_context io_context; 31 | 32 | std::cout << "start timer" << std::endl; 33 | Printer printer(io_context); 34 | 35 | io_context.run(); 36 | return 0; 37 | } 38 | -------------------------------------------------------------------------------- /examples/asio/sync_timer.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include "boost/asio.hpp" 4 | 5 | int main() { 6 | boost::asio::io_context io_context; 7 | boost::asio::steady_timer timer(io_context, 8 | boost::asio::chrono::seconds(3)); 9 | 10 | std::cout << "start timer" << std::endl; 11 | timer.wait(); 12 | std::cout << "end timer" << std::endl; 13 | return 0; 14 | } 15 | -------------------------------------------------------------------------------- /sources/README.md: -------------------------------------------------------------------------------- 1 | ## 资源 2 | 3 | 一些pdf资源和常用工具 4 | 5 | * ebooks/：电子书 6 | * papers/：论文或者其他pdf资料 7 | * emulator_dos/：DOS模拟环境及配套软件，主要是王爽《汇编语言》一书里的实验环境 8 | -------------------------------------------------------------------------------- /sources/ebooks/Large Scale C++ Software Design.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:ae2c252a6658629158ff5b9821770c6f2c0af942d0d9feacdf294e5fec059024 3 | size 46297324 4 | -------------------------------------------------------------------------------- /sources/ebooks/《Redis设计与实现》.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:5fcff12465b7764dc58deb1d6edb5390c7b40ac1d28ad3d8d295b9b482b8e44d 3 | size 68661636 4 | -------------------------------------------------------------------------------- /sources/ebooks/《代码大全》(第2版).pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:4e530214f4af4975bb52e5887fd92cf28efb30f969e79722839b5b26ee262cc2 3 | size 9629813 4 | -------------------------------------------------------------------------------- /sources/ebooks/《汇编语言》(第4版) .pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:56c14cc3cba3538a5624d79bc90cf71a3c30dcad76aeb2af1c547f0256c9eb73 3 | size 47918305 4 | -------------------------------------------------------------------------------- /sources/ebooks/《深入理解计算机系统》（第3版）.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:4400c3388ca17e6b97a335624edd8689f64e6f379cbf416bc1e670268ae27366 3 | size 524927879 4 | -------------------------------------------------------------------------------- /sources/emulator_dos/DEBUG.EXE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/DEBUG.EXE -------------------------------------------------------------------------------- /sources/emulator_dos/EDIT.COM: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/EDIT.COM -------------------------------------------------------------------------------- /sources/emulator_dos/EXE2BIN.EXE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/EXE2BIN.EXE -------------------------------------------------------------------------------- /sources/emulator_dos/LIB.EXE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/LIB.EXE -------------------------------------------------------------------------------- /sources/emulator_dos/LINK.EXE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/LINK.EXE -------------------------------------------------------------------------------- /sources/emulator_dos/MASM.EXE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/MASM.EXE -------------------------------------------------------------------------------- /sources/emulator_dos/README.md: -------------------------------------------------------------------------------- 1 | ## DOS模拟环境 2 | 3 | 搭建过程参考：http://blog.wonter.net/posts/2821c29c/ 4 | 5 | -------------------------------------------------------------------------------- /sources/emulator_dos/masm.IMg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wonter/learning-distributed-storage/005d90c4694bb5fd1a9474aa9b9272e37251f9ab/sources/emulator_dos/masm.IMg -------------------------------------------------------------------------------- /sources/papers/Google-Bigtable中文版_1.0.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:26b8c6794d4284c1ecb238caf12d78f79b5c20fb830b5fe4bb7a9a8b9dd4837a 3 | size 857800 4 | -------------------------------------------------------------------------------- /sources/papers/Google-File-System中文版_1.0.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:f27331c96ae22d21a4e2a1ad31714caeff1e0d3ff37c3d7715d26bf9404df981 3 | size 1239210 4 | -------------------------------------------------------------------------------- /sources/papers/Google-MapReduce中文版_1.0.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:33a648581a9683a4a146a5fac9d997d4907f87136d146f219c513c96441c6356 3 | size 669544 4 | -------------------------------------------------------------------------------- /sources/papers/Large-scale cluster management at Google with Borg.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:2fdacd3b69f8af91477412fc91d1d858a43e764929a4edb646bd517ededdad94 3 | size 856540 4 | -------------------------------------------------------------------------------- /sources/papers/NanoLog- A Nanosecond Scale Logging System.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:c18119d27cbbef0e946c5595297899cb729c99f35ac48046173f0094cbb307cc 3 | size 513864 4 | -------------------------------------------------------------------------------- /sources/papers/The Log-Structured Merge-Tree (LSM-Tree).pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:9d508a97b15dfa0bc0fd6c218ae733d1ea6ef35dd43b3e20856c982a2b19879f 3 | size 122526 4 | -------------------------------------------------------------------------------- /sources/papers/The ϕ Accrual Failure Detector.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:860ce07f5c23fde43d07e5d73ccb94cf25e144d5b81c613f48a07c25c7e8c1ea 3 | size 324294 4 | -------------------------------------------------------------------------------- /sources/papers/What Every Programmer Should Know About Memory.pdf: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:2902dddcadb1ec97eeabd338bfaabf80f1fa2ff4e4d769b4052da88bfbb20387 3 | size 934051 4 | --------------------------------------------------------------------------------