├── README.md ├── SUMMARY.md ├── advanced ├── deno_dir-code-fetch-and-cache.md ├── interaction-with-v8.md ├── process-lifecycle.md └── under-the-call-site-hood.md ├── chinese ├── README.md ├── SUMMARY.md ├── advanced │ ├── deno_dir-code-fetch-and-cache.md │ ├── interaction-with-v8.md │ ├── process-lifecycle.md │ └── under-the-call-site-hood.md ├── codebase-basics │ ├── example-adding-a-deno-api.md │ ├── infrastructure.md │ ├── more-components.md │ └── repo-structure.md ├── contributing-guide.md └── installing-deno.md ├── codebase-basics ├── example-adding-a-deno-api.md ├── infrastructure.md ├── more-components.md └── repo-structure.md ├── contributing-guide.md └── installing-deno.md /README.md: -------------------------------------------------------------------------------- 1 | # A Guide to Deno Core 2 | 3 | ## What is this? 4 | 5 | This is a guide to the design and structure of Deno, a secure server-side TypeScript runtime. 6 | 7 | This guide is created and maintained by some contributors to Deno. 8 | 9 | ### Authors 10 | 11 | 1. [@kevinkassimo](https://github.com/kevinkassimo) 12 | 2. [@monkingxue](https://github.com/monkingxue) 13 | 14 | ### Translation 15 | 1. [Chinese](https://github.com/denolib/guide/tree/master/chinese) 16 | + By [@caijw](https://github.com/caijw) 17 | 18 | ## Before You Start 19 | 20 | If you are not familiar with Deno yet, we recommend checking out the following resources: 21 | 22 | 1. [10 Things I Regret About Node.js - Ryan Dahl - JSConf EU 2018](https://www.youtube.com/watch?v=M3BM9TB-8yA) 23 | 2. [JSDC 2018\#A01 - Deno, A New Server-Side Runtime By Ryan Dahl](https://www.youtube.com/watch?v=FlTG0UXRAkE) 24 | 3. [Official Deno Documentation](https://github.com/denoland/deno/blob/master/Docs.md) 25 | 26 | ## DISCLAIMER 27 | 28 | **THIS IS NOT AN OFFICIAL DOCUMENT.** 29 | 30 | This document has no affiliation with the `denoland` organization and its collaborators. 31 | -------------------------------------------------------------------------------- /SUMMARY.md: -------------------------------------------------------------------------------- 1 | # Table of contents 2 | 3 | * [A Guide to Deno Core](README.md) 4 | * [Installing Deno](installing-deno.md) 5 | * [Contribution Guide](contributing-guide.md) 6 | 7 | ## Codebase Basics 8 | 9 | * [Infrastructure](codebase-basics/infrastructure.md) 10 | * [More Components](codebase-basics/more-components.md) 11 | * [Repo Structure](codebase-basics/repo-structure.md) 12 | * [Example: Adding to Deno API](codebase-basics/example-adding-a-deno-api.md) 13 | 14 | ## Advanced 15 | 16 | * [Under the call site's hood](advanced/under-the-call-site-hood.md) 17 | * [Process Lifecycle](advanced/process-lifecycle.md) 18 | * [Interaction with V8](advanced/interaction-with-v8.md) 19 | * [DENO\_DIR, Code Fetch and Cache](advanced/deno_dir-code-fetch-and-cache.md) 20 | 21 | -------------------------------------------------------------------------------- /advanced/deno_dir-code-fetch-and-cache.md: -------------------------------------------------------------------------------- 1 | --- 2 | description: >- 3 | In this section, we will discuss DENO_DIR, a folder that caches compiled or 4 | remote files, REPL history, etc. It is to some extent similar to GOPATH of Go. 5 | We will also check out code fetch logic. 6 | --- 7 | 8 | # DENO\_DIR, Code Fetch and Cache 9 | 10 | ## DENO\_DIR Structure 11 | 12 | `DENO_DIR` contains the following files and directories: 13 | 14 | ```text 15 | # DIRECTORIES 16 | gen/: Cache for files compiled to JavaScript 17 | deps/: Cache for remote url imported files 18 | |__ http/: For http imports 19 | |__ https/: For https imports 20 | 21 | # FILES 22 | deno_history.txt: History of Deno REPL 23 | ``` 24 | 25 | By default, `DENO_DIR` is located in `$HOME/.deno`. However, the user could also change its location by modifying the `$DENO_DIR` environment variable. Explicitly setting `DENO_DIR` is recommended in production. 26 | 27 | ### gen/ 28 | 29 | `$DENO_DIR/gen/` is used to store JavaScript files that are compiled from source TypeScript files. Such compilation is necessary since V8 does not recognize TypeScript syntax beyond the JS subset. 30 | 31 | Each JS file inside of the `gen/` folder has a filename that is the hash of its TypeScript source. It also comes with a source map ending with `.map`. 32 | 33 | This cache exists to avoid repeated recompilation each run while the source file is not updated by the user. For example, suppose we have a file `hello-world.ts` which contains nothing but `console.log("Hello world")`. During the first run, we will see the message about compilation: 34 | 35 | ```text 36 | $ deno hello-world.ts 37 | Compiling /Users/kevinqian/my-folder/hello-world.ts 38 | Hello world 39 | ``` 40 | 41 | However, without changing the file, when you rerun the code: 42 | 43 | ```text 44 | $ deno hello-world.ts 45 | Hello world 46 | ``` 47 | 48 | The compilation message is gone. This is because for this run, instead of compiling again, Deno is directly using the cached version in `gen/`. 49 | 50 | The cache load and save code could be found in `DenoDir::load_cache` and `DenoDir::code_cache`, from `src/deno_dir.rs`. 51 | 52 | To force Deno to recompile your TypeScript code instead of using the cached version, you could use the `--recompile` flag. 53 | 54 | ### deps/ 55 | 56 | `$DENO_DIR/deps` is used to store files fetched through remote url import. It contains subfolders based on url scheme \(currently only `http` and `https`\), and store files to locations based on the URL path. For example, for the following import \(notice that Deno requires the user to specify extensions explicitly\): 57 | 58 | ```typescript 59 | import { serve } from "https://deno.land/x/std/net/http.ts"; 60 | ``` 61 | 62 | The downloaded `http.ts` will be locally stored in 63 | 64 | ```text 65 | $DENO_DIR/deps/https/deno.land/x/std/net/http.ts 66 | ``` 67 | 68 | Notice that unless the user run the code using `--reload` flag, `http.ts` in our case would NEVER be re-downloaded in future runs. 69 | 70 | Currently \(**WARNING: MIGHT BE CHANGED IN THE FUTURE**\), Deno would keep an eye on the content MIME type of downloaded remote files. In the cases where a file is missing an extension, or having an extension that does not match the content type, Deno would create an extra file ended with `.mime` to store the MIME type as provided by HTTP response headers. Therefore, if we download a file named `a.ts` while having `Content-Type: text/javascript` in the response header, a file `a.ts.mime` would be created to its side, containing `text/javascript`. Due to the presence of this `.mime` file, `a.ts` would later be imported as if it is a JavaScript file. 71 | 72 | ## Code Fetch Logic 73 | 74 | Let's now look into how Deno resolves the code for an import. 75 | 76 | Go to `src/deno_dir.rs` and find the following function: 77 | 78 | {% code-tabs %} 79 | {% code-tabs-item title="src/deno\_dir.rs" %} 80 | ```rust 81 | fn get_source_code( 82 | self: &Self, 83 | module_name: &str, 84 | filename: &str, 85 | ) -> DenoResult { 86 | let is_module_remote = is_remote(module_name); 87 | // We try fetch local. Two cases: 88 | // 1. This is a remote module, but no reload provided 89 | // 2. This is a local module 90 | if !is_module_remote || !self.reload { 91 | debug!( 92 | "fetch local or reload {} is_module_remote {}", 93 | module_name, is_module_remote 94 | ); 95 | match self.fetch_local_source(&module_name, &filename)? { 96 | Some(output) => { 97 | debug!("found local source "); 98 | return Ok(output); 99 | } 100 | None => { 101 | debug!("fetch_local_source returned None"); 102 | } 103 | } 104 | } 105 | 106 | // If not remote file, stop here! 107 | if !is_module_remote { 108 | debug!("not remote file stop here"); 109 | return Err(DenoError::from(std::io::Error::new( 110 | std::io::ErrorKind::NotFound, 111 | format!("cannot find local file '{}'", filename), 112 | ))); 113 | } 114 | 115 | debug!("is remote but didn't find module"); 116 | 117 | // not cached/local, try remote 118 | let maybe_remote_source = 119 | self.fetch_remote_source(&module_name, &filename)?; 120 | if let Some(output) = maybe_remote_source { 121 | return Ok(output); 122 | } 123 | return Err(DenoError::from(std::io::Error::new( 124 | std::io::ErrorKind::NotFound, 125 | format!("cannot find remote file '{}'", filename), 126 | ))); 127 | } 128 | ``` 129 | {% endcode-tabs-item %} 130 | {% endcode-tabs %} 131 | 132 | `module_name` is a string representing the module to be loaded, created based on the paths of the path of the file to be imported, and the file that contains the `import` statement. Therefore, `import "./b.ts"` inside of the file imported by `import "https://example.com/a.ts"` would be treated as if it is `import "https://example.com/b.ts"`. 133 | 134 | `fetch_local_source` goes into the filesystem and try resolving the content based on the filename and its type: 135 | 136 | {% code-tabs %} 137 | {% code-tabs-item title="src/deno\_dir.rs" %} 138 | ```rust 139 | fn fetch_local_source( 140 | self: &Self, 141 | module_name: &str, 142 | filename: &str, 143 | ) -> DenoResult> { 144 | let p = Path::new(&filename); 145 | let media_type_filename = [&filename, ".mime"].concat(); 146 | let mt = Path::new(&media_type_filename); 147 | let source_code = match fs::read(p) { 148 | Err(e) => { 149 | if e.kind() == std::io::ErrorKind::NotFound { 150 | return Ok(None); 151 | } else { 152 | return Err(e.into()); 153 | } 154 | } 155 | Ok(c) => c, 156 | }; 157 | // .mime file might not exists 158 | let maybe_content_type_string = fs::read_to_string(&mt).ok(); 159 | // Option -> Option<&str> 160 | let maybe_content_type_str = 161 | maybe_content_type_string.as_ref().map(String::as_str); 162 | Ok(Some(CodeFetchOutput { 163 | module_name: module_name.to_string(), 164 | filename: filename.to_string(), 165 | media_type: map_content_type(&p, maybe_content_type_str), 166 | source_code, 167 | maybe_output_code: None, 168 | maybe_source_map: None, 169 | })) 170 | } 171 | ``` 172 | {% endcode-tabs-item %} 173 | {% endcode-tabs %} 174 | 175 | while `fetch_remote_source` goes and downloads the remote file and caches it in `$DENO_DIR/deps`, possibly also creating the `.mime` files: 176 | 177 | ```rust 178 | fn fetch_remote_source( 179 | self: &Self, 180 | module_name: &str, 181 | filename: &str, 182 | ) -> DenoResult> { 183 | let p = Path::new(&filename); 184 | // We write a special ".mime" file into the `.deno/deps` directory along side the 185 | // cached file, containing just the media type. 186 | let media_type_filename = [&filename, ".mime"].concat(); 187 | let mt = Path::new(&media_type_filename); 188 | eprint!("Downloading {}...", &module_name); // no newline 189 | let maybe_source = http_util::fetch_sync_string(&module_name); 190 | if let Ok((source, content_type)) = maybe_source { 191 | eprintln!(""); // next line 192 | match p.parent() { 193 | Some(ref parent) => fs::create_dir_all(parent), 194 | None => Ok(()), 195 | }?; 196 | deno_fs::write_file(&p, &source, 0o666)?; 197 | // Remove possibly existing stale .mime file 198 | let _ = std::fs::remove_file(&media_type_filename); 199 | // Create .mime file only when content type different from extension 200 | let resolved_content_type = map_content_type(&p, Some(&content_type)); 201 | let ext = p 202 | .extension() 203 | .map(|x| x.to_str().unwrap_or("")) 204 | .unwrap_or(""); 205 | let media_type = extmap(&ext); 206 | if media_type == msg::MediaType::Unknown 207 | || media_type != resolved_content_type 208 | { 209 | deno_fs::write_file(&mt, content_type.as_bytes(), 0o666)? 210 | } 211 | return Ok(Some(CodeFetchOutput { 212 | module_name: module_name.to_string(), 213 | filename: filename.to_string(), 214 | media_type: map_content_type(&p, Some(&content_type)), 215 | source_code: source, 216 | maybe_output_code: None, 217 | maybe_source_map: None, 218 | })); 219 | } else { 220 | eprintln!(" NOT FOUND"); 221 | } 222 | Ok(None) 223 | } 224 | ``` 225 | 226 | Thus, the resolution logic goes as follows: 227 | 228 | * If `module_name` starts with a remote url scheme: 229 | * If `--reload` flag is present, force download the file and use it. 230 | * Otherwise 231 | * If the local cached file is present, use it. 232 | * Otherwise, download the file to `$DENO_DIR/deps` and use it. 233 | * If `module_name` represents a local source, use the local file. 234 | 235 | 236 | 237 | 238 | 239 | -------------------------------------------------------------------------------- /advanced/interaction-with-v8.md: -------------------------------------------------------------------------------- 1 | --- 2 | description: >- 3 | In this section, we will explore how libdeno is used to interact with V8, the 4 | underlying JavaScript engine of Deno. 5 | --- 6 | 7 | # Interaction with V8 8 | 9 | ## Creating V8 Platform 10 | 11 | From `src/main.rs`, we would find these 2 lines of code: 12 | 13 | {% code-tabs %} 14 | {% code-tabs-item title="src/main.rs" %} 15 | ```rust 16 | let snapshot = snapshot::deno_snapshot(); 17 | let isolate = isolate::Isolate::new(snapshot, state, ops::dispatch); 18 | ``` 19 | {% endcode-tabs-item %} 20 | {% endcode-tabs %} 21 | 22 | This is where the V8 isolate is created. Going into the definition of `Isolate::new` in `src/isolate.rs`: 23 | 24 | {% code-tabs %} 25 | {% code-tabs-item title="src/isolate.rs" %} 26 | ```rust 27 | pub fn new( 28 | snapshot: libdeno::deno_buf, 29 | state: Arc, 30 | dispatch: Dispatch, 31 | ) -> Self { 32 | DENO_INIT.call_once(|| { 33 | unsafe { libdeno::deno_init() }; 34 | }); 35 | let config = libdeno::deno_config { 36 | will_snapshot: 0, 37 | load_snapshot: snapshot, 38 | shared: libdeno::deno_buf::empty(), 39 | recv_cb: pre_dispatch, // callback to invoke when Rust receives a message 40 | }; 41 | let libdeno_isolate = unsafe { libdeno::deno_new(config) }; 42 | // This channel handles sending async messages back to the runtime. 43 | let (tx, rx) = mpsc::channel::<(i32, Buf)>(); 44 | 45 | Self { 46 | libdeno_isolate, 47 | dispatch, 48 | rx, 49 | tx, 50 | ntasks: Cell::new(0), 51 | timeout_due: Cell::new(None), 52 | state, 53 | } 54 | } 55 | ``` 56 | {% endcode-tabs-item %} 57 | {% endcode-tabs %} 58 | 59 | Here, we would find 2 calls on `libdeno`: `deno_init` and `deno_new`. These two functions, unlike many other functions used, are defined from `libdeno/api.cc`, and are made available thanks to FFI and `libc`. Their Rust interface is provided through `src/libdeno.rs`. 60 | 61 | Go to `libdeno/api.cc`: 62 | 63 | {% code-tabs %} 64 | {% code-tabs-item title="libdeno/api.cc" %} 65 | ```c 66 | void deno_init() { 67 | auto* p = v8::platform::CreateDefaultPlatform(); 68 | v8::V8::InitializePlatform(p); 69 | v8::V8::Initialize(); 70 | } 71 | 72 | Deno* deno_new(deno_config config) { 73 | // ... code omitted 74 | deno::DenoIsolate* d = new deno::DenoIsolate(config); 75 | // ... code omitted 76 | v8::Isolate* isolate = v8::Isolate::New(params); 77 | // ... code omitted 78 | 79 | v8::Locker locker(isolate); 80 | v8::Isolate::Scope isolate_scope(isolate); 81 | { 82 | v8::HandleScope handle_scope(isolate); 83 | auto context = 84 | v8::Context::New(isolate, nullptr, v8::MaybeLocal(), 85 | v8::MaybeLocal(), 86 | v8::DeserializeInternalFieldsCallback( 87 | deno::DeserializeInternalFields, nullptr)); 88 | if (!config.load_snapshot.data_ptr) { 89 | // If no snapshot is provided, we initialize the context with empty 90 | // main source code and source maps. 91 | deno::InitializeContext(isolate, context); 92 | } 93 | d->context_.Reset(isolate, context); 94 | } 95 | 96 | return reinterpret_cast(d); 97 | } 98 | ``` 99 | {% endcode-tabs-item %} 100 | {% endcode-tabs %} 101 | 102 | From these two functions, we would find that `deno_init` is used to initialize the V8 platform, while `deno_new` is used to create a new isolated VM instance on this platform. \(A few V8 embedding APIs are used here. To learn more about V8 embedding, check out [https://v8.dev/docs/embed](https://v8.dev/docs/embed) for concepts, and [https://denolib.github.io/v8-docs/](https://denolib.github.io/v8-docs/) for API Reference.\) 103 | 104 | ## Adding Bindings 105 | 106 | There are 2 important functions/constructors used in `deno_new` that might not be immediately clear: `DenoIsolate` and `InitializeContext`. It turns out `DenoIsolate` serves more or less as a collection of Isolate information. Instead, `InitializeContext` is the more interesting one. \(It seems to be invoked here only when there is no snapshot provided. However, you'll also find the function being used in `deno_new_snapshotter` in `libdeno/api.cc` to create a new snapshot, so it is always an inevitable step\): 107 | 108 | {% code-tabs %} 109 | {% code-tabs-item title="libdeno/binding.cc" %} 110 | ```c 111 | void InitializeContext(v8::Isolate* isolate, v8::Local context) { 112 | v8::HandleScope handle_scope(isolate); 113 | v8::Context::Scope context_scope(context); 114 | 115 | auto global = context->Global(); 116 | 117 | auto deno_val = v8::Object::New(isolate); 118 | CHECK(global->Set(context, deno::v8_str("libdeno"), deno_val).FromJust()); 119 | 120 | auto print_tmpl = v8::FunctionTemplate::New(isolate, Print); 121 | auto print_val = print_tmpl->GetFunction(context).ToLocalChecked(); 122 | CHECK(deno_val->Set(context, deno::v8_str("print"), print_val).FromJust()); 123 | 124 | auto recv_tmpl = v8::FunctionTemplate::New(isolate, Recv); 125 | auto recv_val = recv_tmpl->GetFunction(context).ToLocalChecked(); 126 | CHECK(deno_val->Set(context, deno::v8_str("recv"), recv_val).FromJust()); 127 | 128 | auto send_tmpl = v8::FunctionTemplate::New(isolate, Send); 129 | auto send_val = send_tmpl->GetFunction(context).ToLocalChecked(); 130 | CHECK(deno_val->Set(context, deno::v8_str("send"), send_val).FromJust()); 131 | 132 | CHECK(deno_val->SetAccessor(context, deno::v8_str("shared"), Shared) 133 | .FromJust()); 134 | } 135 | ``` 136 | {% endcode-tabs-item %} 137 | {% endcode-tabs %} 138 | 139 | If you are familiar with the `libdeno` API on the TypeScript end, the above names wrapped in `deno::v8_str` might sound familiar to you. In fact, this is where the very few C++ bindings onto JavaScript are attached: we get the global object of current `Context` \(a V8 execution environment that allows separate code to run\) and add some extra properties onto it, from C++. 140 | 141 | Based on the code above, whenever you call `libdeno.send(...)` from TypeScript \(you'll find such usage in `sendInternal` of `js/dispatch.ts`\), you are actually calling into a C++ function called `Send`. Similar things happens to `libdeno.print`. 142 | 143 | Another example: `console.log(...)` is defined as 144 | 145 | {% code-tabs %} 146 | {% code-tabs-item title="js/console.ts" %} 147 | ```typescript 148 | log = (...args: any[]): void => { 149 | this.printFunc(stringifyArgs(args)); 150 | }; 151 | ``` 152 | {% endcode-tabs-item %} 153 | {% endcode-tabs %} 154 | 155 | This `printFunc` is a private field in class `Console`, and its initial value is provided through 156 | 157 | {% code-tabs %} 158 | {% code-tabs-item title="js/globals.ts" %} 159 | ```typescript 160 | window.console = new consoleTypes.Console(libdeno.print); 161 | ``` 162 | {% endcode-tabs-item %} 163 | {% endcode-tabs %} 164 | 165 | Okay, we find `libdeno.print` here. From the above bindings code, we know that calling to `libdeno.print` from TypeScript is equivalent to calling `Print` in `libdeno/binding.cc`, inside of which we would discover 166 | 167 | ```cpp 168 | void Print(const v8::FunctionCallbackInfo& args) { 169 | // ... code omitted 170 | v8::String::Utf8Value str(isolate, args[0]); 171 | bool is_err = 172 | args.Length() >= 2 ? args[1]->BooleanValue(context).ToChecked() : false; 173 | FILE* file = is_err ? stderr : stdout; 174 | fwrite(*str, sizeof(**str), str.length(), file); 175 | fprintf(file, "\n"); 176 | fflush(file); 177 | } 178 | ``` 179 | 180 | Nothing more surprising than some familiar standard C print formula. Therefore, calling `console.log` is in fact just indirectly calling `fwrite/fprintf`! 181 | 182 | Check out the source code of `Print`, `Send` and `Recv` in `libdeno/binding.cc` to understand what is happening behind the scene. 183 | 184 | ## Executing Code on V8 185 | 186 | From `src/main.rs`, immediately following isolate creation, we find 187 | 188 | {% code-tabs %} 189 | {% code-tabs-item title="src/main.rs" %} 190 | ```rust 191 | isolate 192 | .execute("denoMain();") 193 | .unwrap_or_else(print_err_and_exit); 194 | ``` 195 | {% endcode-tabs-item %} 196 | {% endcode-tabs %} 197 | 198 | From the previous section, we know `denoMain` is a TypeScript side function. `isolate.execute` is used to run the code. Let's extract its definition from `src/isolate.rs`: 199 | 200 | {% code-tabs %} 201 | {% code-tabs-item title="src/isolate.rs" %} 202 | ```rust 203 | /// Same as execute2() but the filename defaults to "". 204 | pub fn execute(&self, js_source: &str) -> Result<(), JSError> { 205 | self.execute2("", js_source) 206 | } 207 | 208 | /// Executes the provided JavaScript source code. The js_filename argument is 209 | /// provided only for debugging purposes. 210 | pub fn execute2( 211 | &self, 212 | js_filename: &str, 213 | js_source: &str, 214 | ) -> Result<(), JSError> { 215 | let filename = CString::new(js_filename).unwrap(); 216 | let source = CString::new(js_source).unwrap(); 217 | let r = unsafe { 218 | libdeno::deno_execute( 219 | self.libdeno_isolate, 220 | self.as_raw_ptr(), 221 | filename.as_ptr(), 222 | source.as_ptr(), 223 | ) 224 | }; 225 | if r == 0 { 226 | let js_error = self.last_exception().unwrap(); 227 | return Err(js_error); 228 | } 229 | Ok(()) 230 | } 231 | ``` 232 | {% endcode-tabs-item %} 233 | {% endcode-tabs %} 234 | 235 | The only interesting function we care in this section is `libdeno::deno_execute`. We can find its actual definition in `libdeno/api.cc` again: 236 | 237 | {% code-tabs %} 238 | {% code-tabs-item title="libdeno/api.cc" %} 239 | ```c 240 | int deno_execute(Deno* d_, void* user_data, const char* js_filename, 241 | const char* js_source) { 242 | auto* d = unwrap(d_); 243 | // ... code omitted 244 | auto context = d->context_.Get(d->isolate_); 245 | CHECK(!context.IsEmpty()); 246 | return deno::Execute(context, js_filename, js_source) ? 1 : 0; 247 | } 248 | ``` 249 | {% endcode-tabs-item %} 250 | {% endcode-tabs %} 251 | 252 | Eventually, let's find `deno::Execute`: 253 | 254 | ```cpp 255 | bool Execute(v8::Local context, const char* js_filename, 256 | const char* js_source) { 257 | // ... code omitted 258 | auto source = v8_str(js_source); 259 | return ExecuteV8StringSource(context, js_filename, source); 260 | } 261 | 262 | bool ExecuteV8StringSource(v8::Local context, 263 | const char* js_filename, 264 | v8::Local source) { 265 | // ... code omitted 266 | 267 | v8::TryCatch try_catch(isolate); 268 | 269 | auto name = v8_str(js_filename); 270 | 271 | v8::ScriptOrigin origin(name); 272 | 273 | auto script = v8::Script::Compile(context, source, &origin); 274 | 275 | if (script.IsEmpty()) { 276 | DCHECK(try_catch.HasCaught()); 277 | HandleException(context, try_catch.Exception()); 278 | return false; 279 | } 280 | 281 | auto result = script.ToLocalChecked()->Run(context); 282 | 283 | if (result.IsEmpty()) { 284 | DCHECK(try_catch.HasCaught()); 285 | HandleException(context, try_catch.Exception()); 286 | return false; 287 | } 288 | 289 | return true; 290 | } 291 | ``` 292 | 293 | As we see here, `Execute` is eventually submitting the code to `v8::Script::Compile`, which compiles the JavaScript code and call `Run` on it. Any exceptions, compile time or runtime, are further processed through `HandleException`: 294 | 295 | {% code-tabs %} 296 | {% code-tabs-item title="libdeno/binding.cc" %} 297 | ```cpp 298 | void HandleException(v8::Local context, 299 | v8::Local exception) { 300 | v8::Isolate* isolate = context->GetIsolate(); 301 | DenoIsolate* d = FromIsolate(isolate); 302 | std::string json_str = EncodeExceptionAsJSON(context, exception); 303 | CHECK(d != nullptr); 304 | d->last_exception_ = json_str; 305 | } 306 | ``` 307 | {% endcode-tabs-item %} 308 | {% endcode-tabs %} 309 | 310 | It sets `d->last_exception_` to be an error message formatted in JSON, which was read in `deno_last_exception`: 311 | 312 | {% code-tabs %} 313 | {% code-tabs-item title="libdeno/api.cc" %} 314 | ```c 315 | const char* deno_last_exception(Deno* d_) { 316 | auto* d = unwrap(d_); 317 | if (d->last_exception_.length() > 0) { 318 | return d->last_exception_.c_str(); 319 | } else { 320 | return nullptr; 321 | } 322 | } 323 | ``` 324 | {% endcode-tabs-item %} 325 | {% endcode-tabs %} 326 | 327 | which is used in `Isolate::last_exception` from Rust: 328 | 329 | {% code-tabs %} 330 | {% code-tabs-item title="src/isolate.rs" %} 331 | ```rust 332 | pub fn last_exception(&self) -> Option { 333 | let ptr = unsafe { libdeno::deno_last_exception(self.libdeno_isolate) }; 334 | // ... code omitted 335 | } 336 | ``` 337 | {% endcode-tabs-item %} 338 | {% endcode-tabs %} 339 | 340 | and is checked inside of `Isolate::execute` to decide if error handling is necessary. 341 | 342 | Whew! That's a long trip across TypeScript, C/C++ and Rust. However, it should be very clear to you how Deno is interacting with V8 now. Nothing magical. 343 | 344 | -------------------------------------------------------------------------------- /advanced/process-lifecycle.md: -------------------------------------------------------------------------------- 1 | # Process Lifecycle 2 | 3 | ## Rust main\(\) Entry Point 4 | 5 | A Deno process starts with the `main` function in `src/main.rs`: 6 | 7 | {% code-tabs %} 8 | {% code-tabs-item title="src/main.rs" %} 9 | ```rust 10 | fn main() { 11 | log::set_logger(&LOGGER).unwrap(); 12 | let args = env::args().collect(); 13 | let (flags, rest_argv, usage_string) = 14 | flags::set_flags(args).unwrap_or_else(|err| { 15 | eprintln!("{}", err); 16 | std::process::exit(1) 17 | }); 18 | 19 | if flags.help { 20 | println!("{}", &usage_string); 21 | std::process::exit(0); 22 | } 23 | 24 | log::set_max_level(if flags.log_debug { 25 | log::LevelFilter::Debug 26 | } else { 27 | log::LevelFilter::Warn 28 | }); 29 | 30 | let state = Arc::new(isolate::IsolateState::new(flags, rest_argv)); 31 | let snapshot = snapshot::deno_snapshot(); 32 | let isolate = isolate::Isolate::new(snapshot, state, ops::dispatch); 33 | tokio_util::init(|| { 34 | isolate 35 | .execute("denoMain();") 36 | .unwrap_or_else(print_err_and_exit); 37 | isolate.event_loop().unwrap_or_else(print_err_and_exit); 38 | }); 39 | } 40 | ``` 41 | {% endcode-tabs-item %} 42 | {% endcode-tabs %} 43 | 44 | After processing the flags, Deno first creates a V8 isolate \(an isolated V8 VM instance with its own heap\). We also pass in the snapshot here, which contains serialized heap data about TypeScript compiler and frontend code. This allows much faster startup speed. 45 | 46 | Immediately afterwards, we setup Tokio, and ask the isolate to execute a function called `denoMain`. Only after it has finished running this function does Deno starting its event loop. Once the event loop stops, the Deno process dies. 47 | 48 | ## denoMain\(\) 49 | 50 | `denoMain` is located in `js/main.ts`. 51 | 52 | {% code-tabs %} 53 | {% code-tabs-item title="js/main.ts" %} 54 | ```typescript 55 | export default function denoMain() { 56 | libdeno.recv(handleAsyncMsgFromRust); 57 | 58 | const startResMsg = sendStart(); 59 | setLogDebug(startResMsg.debugFlag()); 60 | 61 | const compiler = Compiler.instance(); 62 | 63 | // ... code omitted 64 | 65 | const cwd = startResMsg.cwd(); 66 | 67 | // ... code omitted 68 | 69 | const runner = new Runner(compiler); 70 | 71 | if (inputFn) { 72 | runner.run(inputFn, `${cwd}/`); 73 | } else { 74 | replLoop(); 75 | } 76 | } 77 | ``` 78 | {% endcode-tabs-item %} 79 | {% endcode-tabs %} 80 | 81 | First, we tell `libdeno`, which is the exposed API from the middle-end, that whenever there is a message sent from the Rust side, please forward the buffer to a function called `handleAsyncMsgFromRust`. Then, a `Start` message is sent to Rust side, signaling that we are starting and receiving information including the current working directory, or `cwd`. We then decide whether the user is running a script, or using the REPL. If we have an input file name, Deno would then try to let the `runner`, which contains the TypeScript `compiler`, to try running the file \(going deep into its definition, you'll eventually find a secret `eval/globalEval` called somewhere\). `denoMain` only exits when the `runner` finish running the file. 82 | 83 | ## isolate.event\_loop\(\) 84 | 85 | With `denoMain` running to its end, a set of asynchronous calls might have been made without any of them being responded. Deno would then call the `isolate.event_loop()` to start the loop. 86 | 87 | Code for `event_loop` is located in `src/isolate.rs`: 88 | 89 | {% code-tabs %} 90 | {% code-tabs-item title="src/isolate.rs" %} 91 | ```rust 92 | pub fn event_loop(&self) -> Result<(), JSError> { 93 | // Main thread event loop. 94 | while !self.is_idle() { 95 | match recv_deadline(&self.rx, self.get_timeout_due()) { 96 | Ok((req_id, buf)) => self.complete_op(req_id, buf), 97 | Err(mpsc::RecvTimeoutError::Timeout) => self.timeout(), 98 | Err(e) => panic!("recv_deadline() failed: {:?}", e), 99 | } 100 | self.check_promise_errors(); 101 | if let Some(err) = self.last_exception() { 102 | return Err(err); 103 | } 104 | } 105 | // Check on done 106 | self.check_promise_errors(); 107 | if let Some(err) = self.last_exception() { 108 | return Err(err); 109 | } 110 | Ok(()) 111 | } 112 | ``` 113 | {% endcode-tabs-item %} 114 | {% endcode-tabs %} 115 | 116 | This is literally a loop. Every single time, we check if there is still some events we haven't responded, with `self.is_idle()`. If yes, we will wait for `self.rx`, which is the receiving end of a message channel, to receive some message, or wait for a timer, created by `setTimeout`, to expire. 117 | 118 | If `self.rx` received some message, this means that one of the task in the Tokio thread pool has completed its execution. We would then process its execution result, which is a pair of `req_id` \(the task id\) and `buf` \(the buffer containing serialized response\), by sending them to `self.complete_op`, which forwards the result based on request id and invoke the code that is waiting for this output, running to the end. Notice that the code invoked here might schedule for more async operation. 119 | 120 | If timeout happens, code scheduled to execute by `setTimeout` would similarly be invoked and running to the end. 121 | 122 | The event loop would continue to wait for output and run more code, until `self.is_idle` becomes `true`. This happens when there are no more pending tasks. Then, the event loop exits and Deno quits. 123 | 124 | It is possible that an error might happen when executing code, and Deno would exit if such error exists. If it is a promise error \(e.g. uncaught rejection\), Deno would invoke `self.check_promise_errors` to check. 125 | 126 | ## Promise Table 127 | 128 | In the previous section, we mentioned that `req_id` is provided to `self.complete_op` as a task id. Actually, it is used to retrieve the correct promise created by the past async operation from a promise table, which holds all the promises that are pending resolve. 129 | 130 | The promise table is located in `src/dispatch.ts` 131 | 132 | {% code-tabs %} 133 | {% code-tabs-item title="src/dispatch.ts" %} 134 | ```typescript 135 | const promiseTable = new Map>(); 136 | ``` 137 | {% endcode-tabs-item %} 138 | {% endcode-tabs %} 139 | 140 | Promise table is populated by `dispatch.sendAsync`, which is used to forward async requests to Rust. 141 | 142 | {% code-tabs %} 143 | {% code-tabs-item title="src/dispatch.ts" %} 144 | ```typescript 145 | // @internal 146 | export function sendAsync( 147 | builder: flatbuffers.Builder, 148 | innerType: msg.Any, 149 | inner: flatbuffers.Offset, 150 | data?: ArrayBufferView 151 | ): Promise { 152 | const [cmdId, resBuf] = sendInternal(builder, innerType, inner, data, false); 153 | util.assert(resBuf == null); 154 | const promise = util.createResolvable(); 155 | promiseTable.set(cmdId, promise); 156 | return promise; 157 | } 158 | ``` 159 | {% endcode-tabs-item %} 160 | {% endcode-tabs %} 161 | 162 | The `cmdId` here is essentially the `req_id` on the Rust side. It is returned by `sendInternal`, which calls into V8 Send binding and increments a global request id counter. For each async operation, we create a deferred/resolvable and place it into the promise table with `cmdId` as the key. 163 | 164 | From the `denoMain` section, we saw that we attach `handleAsyncMsgFromRust` to `libdeno` as the receive callback whenever a new message arrives. Here is its source code: 165 | 166 | ```typescript 167 | export function handleAsyncMsgFromRust(ui8: Uint8Array) { 168 | // If a the buffer is empty, recv() on the native side timed out and we 169 | // did not receive a message. 170 | if (ui8.length) { 171 | const bb = new flatbuffers.ByteBuffer(ui8); 172 | const base = msg.Base.getRootAsBase(bb); 173 | const cmdId = base.cmdId(); 174 | const promise = promiseTable.get(cmdId); 175 | util.assert(promise != null, `Expecting promise in table. ${cmdId}`); 176 | promiseTable.delete(cmdId); 177 | const err = errors.maybeError(base); 178 | if (err != null) { 179 | promise!.reject(err); 180 | } else { 181 | promise!.resolve(base); 182 | } 183 | } 184 | // Fire timers that have become runnable. 185 | fireTimers(); 186 | } 187 | ``` 188 | 189 | Notice that it grabs the `cmdId` from the response and find the corresponding Promise in the table. The promise is then resolved, allowing code awaiting the response to be executed as microtasks. 190 | 191 | ## Lifecycle Example 192 | 193 | For a file looks like the following: 194 | 195 | ```typescript 196 | // RUN FILE 197 | import * as deno from 'deno'; 198 | 199 | const dec = new TextDecoder(); 200 | 201 | (async () => { 202 | // prints "A" 203 | console.log(dec.decode(await deno.readFile('a.c'))); 204 | // prints "B" 205 | console.log(dec.decode(deno.readFileSync('b.c'))); 206 | // prints "C" 207 | console.log(dec.decode(await deno.readFile('c.c'))); 208 | })(); 209 | 210 | // prints "D" 211 | console.log(dec.decode(deno.readFileSync('d.c'))); 212 | 213 | // END FILE 214 | ``` 215 | 216 | The Deno process lifecycle control flow graph looks like this: 217 | 218 | ```text 219 | Rust TypeScript (V8) 220 | 221 | main 222 | | 223 | create isolate 224 | | 225 | | execute 226 | |--------------> denoMain 227 | | 228 | startMsg | 229 | |<-------------------| 230 | |------------------->| 231 | startResMsg | 232 | | 233 | RUN FILE 234 | readFile('a.c') | 235 | (thread pool) |<-------------------| 236 | | 237 | | 238 | readFileSync('d.c')| 239 | |<-------------------| 240 | | 241 | blocking 242 | execution 243 | | 244 | |------------------->| 245 | (Uint8Array) "D" | 246 | console.log("D") 247 | | 248 | END FILE | 249 | |....................| # no more sync code to exec 250 | | 251 | event loop 252 | | 253 | more tasks 254 | | 255 | | 'a.c' done reading 256 | (thread pool) |------------------->| 257 | (Uint8Array) "A" | 258 | console.log("A") 259 | | 260 | readFileSync('b.c')| 261 | |<-------------------| 262 | | 263 | blocking 264 | execution 265 | | 266 | |------------------->| 267 | (Uint8Array) "B" | 268 | console.log("B") 269 | | 270 | readFile('c.c') | 271 | (thread pool) |<-------------------| 272 | | 273 | |....................| # no more sync code to exec 274 | | 275 | more tasks 276 | | 277 | | 'c.c' done reading 278 | (thread pool) |------------------->| 279 | (Uint8Array) "C" | 280 | console.log("C") 281 | | 282 | | 283 | |....................| # no more sync code to exec 284 | | 285 | no task 286 | | 287 | | 288 | v 289 | EXIT 290 | ``` 291 | 292 | 293 | 294 | -------------------------------------------------------------------------------- /advanced/under-the-call-site-hood.md: -------------------------------------------------------------------------------- 1 | --- 2 | description: >- 3 | In this section, we'll dive into the source code to get an idea of how 4 | TypeScript functions communicate with Rust implementations. 5 | --- 6 | 7 | # Under the call site's hood 8 | 9 | In the following, we'll use the `readFileSync` and `readFile` to illustrate the underlying principles. 10 | 11 | ## TypeScript call site 12 | 13 | In Deno, we can read files like this: 14 | 15 | ```typescript 16 | import * as deno from "deno"; 17 | 18 | async function readFileTest() { 19 | // Read file asynchronously 20 | const data1 = await deno.readFile("test.json"); 21 | 22 | // Read file synchronously 23 | // Note that we don't need `await` keyword. 24 | const data2 = deno.readFile("test.json"); 25 | } 26 | ``` 27 | 28 | The above code is very simple and straightforward, but how does Typescript read the file actually? And in the absence of `libuv`, how does Deno implement asynchronous calls? 29 | 30 | In fact, the Deno APIs that need to use system calls are converted to messaging process like the following: 31 | 32 | ```typescript 33 | function readFileSync(filename: string): Uint8Array { 34 | return res(dispatch.sendSync(...req(filename))); 35 | } 36 | 37 | async function readFile(filename: string): Promise { 38 | return res(await dispatch.sendAsync(...req(filename))); 39 | } 40 | ``` 41 | 42 | Both the `sendSync` and `sendAsync` methods call the `libdeno.send` method on the underlying, and this method is a C++ binding function injected by V8. After this processing, the function call information is passed to C++. 43 | 44 | In addition to dispatching messages, TypeScript end also needs to register a callback function to get the asynchronous result sent back from the Rust side: 45 | 46 | ```typescript 47 | // js/main.ts denoMain() 48 | 49 | libdeno.recv(handleAsyncMsgFromRust); 50 | ``` 51 | 52 | `libdeno.recv` is also a binding function injected in V8 to register callback for receiving messages. For details on this function, please refer to [the next section](https://denolib.gitbook.io/guide/~/drafts/-LUtKIIrxNaQMXGyMSgw/primary/advanced/process-lifecycle#rust-main-entry-point). 53 | 54 | Let's take a look at how C++ consumes received messages. 55 | 56 | ## C++ converter 57 | 58 | Note: The following is pseudo `libdeno.send` code, just for the sake of easy understanding, the actual code is not like this. 59 | 60 | ```cpp 61 | // libdeno/binding.cc 62 | 63 | void Send(const pseudo::Args args) { 64 | // get the call inforamtion, such as function name etc. 65 | pseudo::Value control = args[0]; 66 | // get args with a large amount of data. 67 | pseudo::Value data; 68 | // get the request id 69 | int32_t req_id = isolate->next_req_id_++; 70 | 71 | if (args.Length() == 2) { 72 | data = args[1]; 73 | } 74 | 75 | isolate->current_args_ = &args; 76 | 77 | isolate->recv_cb_(req_id, control, data); 78 | } 79 | ``` 80 | 81 | The `Send` function get args and invoke the `recv_cb_` . Notice that the `recv_cb_` is defined in the Rust code. 82 | 83 | The return value of the `libdeno.send` will be set by `deno_respond`: 84 | 85 | ```cpp 86 | // libdeno/api.cc 87 | // it's also pseudo code 88 | 89 | int deno_respond(DenoIsolate* d, int32_t req_id, deno_buf buf) { 90 | // Synchronous response. 91 | if (d->current_args_ != nullptr) { 92 | // get return value as Uint8Array 93 | auto ab = deno::ImportBuf(d, buf); 94 | // set return value 95 | d->current_args_->GetReturnValue().Set(ab); 96 | d->current_args_ = nullptr; 97 | return 0; 98 | } 99 | 100 | // Asynchronous response. 101 | // get receive callback function defined in TypeScript 102 | auto recv_ = d->recv_.Get(d->isolate_); 103 | if (recv_.IsEmpty()) { 104 | d->last_exception_ = "libdeno.recv_ has not been called."; 105 | return 1; 106 | } 107 | 108 | pseudo::Value args[1]; 109 | // get return value as Uint8Array 110 | args[0] = deno::ImportBuf(d, buf); 111 | // call the TypeScript function 112 | auto v = recv_->Call(context, context->Global(), 1, args); 113 | 114 | return 0; 115 | } 116 | ``` 117 | 118 | The `deno_respond` is called by the `recv_cb_` and will be distinguished between synchronous and asynchronous when executed. When the call is synchronous, `deno_respond` returns the result directly. And when the call is asynchronously, the function is triggered asynchronously and calls the callback function defined in TypeScript. 119 | 120 | ## Rust executor 121 | 122 | The journey of the function call comes to the last station. The Rust code maps the type of function call to the corresponding handler and handles synchronous and asynchronous calls based on event-loop. For details on event-loop, see [the next section](https://denolib.gitbook.io/guide/~/drafts/-LUtyQuHAr1yFSGlQdF7/primary/advanced/process-lifecycle#isolate-event_loop). 123 | 124 | -------------------------------------------------------------------------------- /chinese/README.md: -------------------------------------------------------------------------------- 1 | # Deno Core 指南 2 | 3 | ## 这是什么？ 4 | 5 | 本指南介绍了 Deno（一个安全的服务端的 TypeScript 运行时）的设计和架构。 6 | 7 | 本指南是由 Deno 的贡献者创建和维护的。 8 | 9 | ### 作者 10 | 11 | 1. [@kevinkassimo](https://github.com/kevinkassimo) 12 | 2. [@monkingxue](https://github.com/monkingxue) 13 | 3. [@caijw](https://github.com/caijw) 14 | 15 | ## 在你开始之前 16 | 17 | 如果你对 Deno 还不是很熟悉，我们建议你先查看下面的资源： 18 | 19 | 1. [10 Things I Regret About Node.js - Ryan Dahl - JSConf EU 2018](https://www.youtube.com/watch?v=M3BM9TB-8yA) 20 | 2. [JSDC 2018\#A01 - Deno, A New Server-Side Runtime By Ryan Dahl](https://www.youtube.com/watch?v=FlTG0UXRAkE) 21 | 3. [Official Deno Documentation](https://github.com/denoland/deno/blob/master/Docs.md) 22 | 23 | ## 免责声明 24 | 25 | **这不是一个官方的文档.** 26 | 27 | 这个文档跟 `denoland` 组织和他的成员没有从属关系。 28 | 29 | 30 | ## 多语言支持 31 | - [英文](https://github.com/denolib/guide) 32 | - [中文](https://github.com/denolib/guide/chinese) 33 | -------------------------------------------------------------------------------- /chinese/SUMMARY.md: -------------------------------------------------------------------------------- 1 | # 内容列表 2 | 3 | * [Deno Core 指南](./README.md) 4 | * [安装 Deno](./installing-deno.md) 5 | * [如何贡献源码](./contributing-guide.md) 6 | 7 | ## 代码库的基本概念 8 | 9 | * [Deno 基础架构](./codebase-basics/infrastructure.md) 10 | * [更多的模块](./codebase-basics/more-components.md) 11 | * [源码一览](./codebase-basics/repo-structure.md) 12 | * [例子: 给 Deno 添加一个新的 api](./codebase-basics/example-adding-a-deno-api.md) 13 | 14 | ## 高级指南 15 | 16 | * [从调用的角度](./advanced/under-the-call-site-hood.md) 17 | * [进程生命周期](./advanced/process-lifecycle.md) 18 | * [与 v8 交互](./advanced/interaction-with-v8.md) 19 | * [DENO\_DIR, Code Fetch 和 Cache](./advanced/deno_dir-code-fetch-and-cache.md) 20 | 21 | -------------------------------------------------------------------------------- /chinese/advanced/deno_dir-code-fetch-and-cache.md: -------------------------------------------------------------------------------- 1 | --- 2 | 描述: >- 3 | 在这一部分，我们将会讨论DENO_DIR，这是一个缓存编译后的或者远程的文件，REPL历史记录等等的目录。他跟Go的GOPATH是一样的。我们也将会讨论代码获取逻辑。 4 | --- 5 | 6 | # DENO\_DIR, Code Fetch 和 Cache 7 | 8 | ## DENO\_DIR 结构 9 | 10 | `DENO_DIR`包含下面的文件和目录： 11 | 12 | ```text 13 | # DIRECTORIES 14 | gen/: 缓存编译为JavaScript的文件 15 | deps/: 缓存导入的远程url的文件 16 | |__ http/: http方式导入的文件 17 | |__ https/: https方式导入的文件 18 | 19 | # FILES 20 | deno_history.txt: Deno REPL历史记录缓存 21 | ``` 22 | 23 | 默认的，`DENO_DIR`位于`$HOME/.deno`。但是，用户可以通过修改`$DENO_DIR`环境变量来修改该位置。在生产环境建议显式的设置`DENO_DIR`。 24 | 25 | ### gen/ 26 | 27 | `$DENO_DIR/gen/`被用来存放JavaScript文件，这些文件是从TypeScript源码编译来的。这样的编译是必要的，因为V8不识别JS子集之外的TypeScript语法。 28 | 29 | `gen/`目录下的每一个JS文件的文件名是他的TypeScript源码的hash值。同时JS文件也对应一个`.map`为后缀的source map文件。 30 | 31 | 缓存存在的原因是为了避免在用户没有修改代码的情况下，每次运行时不断的重新编译文件。比如我们有一个`hello-world.ts`文件，他只是包含了代码`console.log("Hello world")`。在第一次运行时，我们会看到编译信息： 32 | 33 | 34 | ```text 35 | $ deno hello-world.ts 36 | Compiling /Users/kevinqian/my-folder/hello-world.ts 37 | Hello world 38 | ``` 39 | 40 | 但是在没有修改文件内容的情况下，当你重新运行代码： 41 | 42 | ```text 43 | $ deno hello-world.ts 44 | Hello world 45 | ``` 46 | 47 | 不会再有编译信息的提示。这是因为在这一次运行中，Deno直接使用了`gen/`中缓存版本，而不是再次编译。 48 | 49 | 缓存加载和保存的代码，可以从文件`src/deno_dir.rs`中的`DenoDir::load_cache`和`DenoDir::code_cache`中找到。 50 | 51 | 为了强制Deno重新编译你的代码而不是使用缓存的版本，你需要使用`--recompile`标志。 52 | 53 | ### deps/ 54 | 55 | `$DENO_DIR/deps`被用来保存远端url import 获得的文件。根据url的模式，他包含了子目录\(现在只有`http`和`https`\)，并且保存文件的位置由URL path决定。比如，对于下面的的import\(请注意，Deno要求用户显式地指定扩展\)。 56 | 57 | ```typescript 58 | import { serve } from "https://deno.land/x/std/net/http.ts"; 59 | ``` 60 | 61 | 下载的`http.ts`文件将会被存储在： 62 | 63 | ```text 64 | $DENO_DIR/deps/https/deno.land/x/std/net/http.ts 65 | ``` 66 | 67 | 需要注意，除非用户用`--reload`标志运行代码，否则我们的`http.ts`文件在接下来的运行中不会被重新下载。 68 | 69 | 当前\(**警告：将来可能改变**\)，Deno会关注从远端下载的文件的内容的MIME类型。在文件缺少扩展名或扩展名与内容类型不匹配的情况下，Deno将创建一个以`.mime`结尾的额外文件，来存储HTTP响应头提供的mime类型。如果我们下载的文件名是`a.ts`，然而响应头里面是`Content-Type: text/javascript`，一个包含`text/javascript`内容的`a.ts.mime`文件将会在他旁边被创建。由于`.mime`文件的存在，`a.ts`后面将会被当做一个JavaScript文件被import。 70 | 71 | ## 代码下载逻辑 72 | 73 | 现在让我们看看Deno如何解析import的代码。 74 | 75 | 进入`src/deno_dir.rs`文件，然后找到下面的函数： 76 | 77 | {% code-tabs %} 78 | {% code-tabs-item title="src/deno\_dir.rs" %} 79 | ```rust 80 | fn get_source_code( 81 | self: &Self, 82 | module_name: &str, 83 | filename: &str, 84 | ) -> DenoResult { 85 | let is_module_remote = is_remote(module_name); 86 | // 我们尝试加载本地文件. 两个例子: 87 | // 1. 这是一个远程的模块, 但是没有提供reload标志 88 | // 2. 这是一个本地的模块 89 | if !is_module_remote || !self.reload { 90 | debug!( 91 | "fetch local or reload {} is_module_remote {}", 92 | module_name, is_module_remote 93 | ); 94 | match self.fetch_local_source(&module_name, &filename)? { 95 | Some(output) => { 96 | debug!("found local source "); 97 | return Ok(output); 98 | } 99 | None => { 100 | debug!("fetch_local_source returned None"); 101 | } 102 | } 103 | } 104 | 105 | // 如果不是远程文件，停止运行！ 106 | if !is_module_remote { 107 | debug!("not remote file stop here"); 108 | return Err(DenoError::from(std::io::Error::new( 109 | std::io::ErrorKind::NotFound, 110 | format!("cannot find local file '{}'", filename), 111 | ))); 112 | } 113 | 114 | debug!("is remote but didn't find module"); 115 | 116 | // 不是缓存/本地，尝试远程加载 117 | let maybe_remote_source = 118 | self.fetch_remote_source(&module_name, &filename)?; 119 | if let Some(output) = maybe_remote_source { 120 | return Ok(output); 121 | } 122 | return Err(DenoError::from(std::io::Error::new( 123 | std::io::ErrorKind::NotFound, 124 | format!("cannot find remote file '{}'", filename), 125 | ))); 126 | } 127 | ``` 128 | {% endcode-tabs-item %} 129 | {% endcode-tabs %} 130 | 131 | `module_name`是一个字符串，表示需要加载的模块，它是根据要导入的文件的路径和包含“import”语句的文件的路径创建的。因此，一个通过`import "https://example.com/a.ts"`来加载的文件的内部包含了`import "./b.ts"`，将会被当做`import "https://example.com/b.ts"`。 132 | 133 | `fetch_local_source`进入了文件系统，然后尝试基于文件名和文件类型来获取文件内容。 134 | 135 | {% code-tabs %} 136 | {% code-tabs-item title="src/deno\_dir.rs" %} 137 | ```rust 138 | fn fetch_local_source( 139 | self: &Self, 140 | module_name: &str, 141 | filename: &str, 142 | ) -> DenoResult> { 143 | let p = Path::new(&filename); 144 | let media_type_filename = [&filename, ".mime"].concat(); 145 | let mt = Path::new(&media_type_filename); 146 | let source_code = match fs::read(p) { 147 | Err(e) => { 148 | if e.kind() == std::io::ErrorKind::NotFound { 149 | return Ok(None); 150 | } else { 151 | return Err(e.into()); 152 | } 153 | } 154 | Ok(c) => c, 155 | }; 156 | // .mime文件可能不存在 157 | let maybe_content_type_string = fs::read_to_string(&mt).ok(); 158 | // Option -> Option<&str> 159 | let maybe_content_type_str = 160 | maybe_content_type_string.as_ref().map(String::as_str); 161 | Ok(Some(CodeFetchOutput { 162 | module_name: module_name.to_string(), 163 | filename: filename.to_string(), 164 | media_type: map_content_type(&p, maybe_content_type_str), 165 | source_code, 166 | maybe_output_code: None, 167 | maybe_source_map: None, 168 | })) 169 | } 170 | ``` 171 | {% endcode-tabs-item %} 172 | {% endcode-tabs %} 173 | 174 | 当`fetch_remote_source`运行和下载远程的文件然后缓存在`$DENO_DIR/deps`时，他可能也会创建`.mime`文件： 175 | 176 | ```rust 177 | fn fetch_remote_source( 178 | self: &Self, 179 | module_name: &str, 180 | filename: &str, 181 | ) -> DenoResult> { 182 | let p = Path::new(&filename); 183 | // 我们在`.deno/deps`目录中，缓存文件旁边的写入一个特殊的".mime"文件 184 | // 这个文件仅仅包含了媒体类型。 185 | let media_type_filename = [&filename, ".mime"].concat(); 186 | let mt = Path::new(&media_type_filename); 187 | eprint!("Downloading {}...", &module_name); // no newline 188 | let maybe_source = http_util::fetch_sync_string(&module_name); 189 | if let Ok((source, content_type)) = maybe_source { 190 | eprintln!(""); // 下一行 191 | match p.parent() { 192 | Some(ref parent) => fs::create_dir_all(parent), 193 | None => Ok(()), 194 | }?; 195 | deno_fs::write_file(&p, &source, 0o666)?; 196 | // 删除可能存在的旧的.mime文件 197 | let _ = std::fs::remove_file(&media_type_filename); 198 | // 只在内容类型与扩展名不同时创建.mime文件 199 | let resolved_content_type = map_content_type(&p, Some(&content_type)); 200 | let ext = p 201 | .extension() 202 | .map(|x| x.to_str().unwrap_or("")) 203 | .unwrap_or(""); 204 | let media_type = extmap(&ext); 205 | if media_type == msg::MediaType::Unknown 206 | || media_type != resolved_content_type 207 | { 208 | deno_fs::write_file(&mt, content_type.as_bytes(), 0o666)? 209 | } 210 | return Ok(Some(CodeFetchOutput { 211 | module_name: module_name.to_string(), 212 | filename: filename.to_string(), 213 | media_type: map_content_type(&p, Some(&content_type)), 214 | source_code: source, 215 | maybe_output_code: None, 216 | maybe_source_map: None, 217 | })); 218 | } else { 219 | eprintln!(" NOT FOUND"); 220 | } 221 | Ok(None) 222 | } 223 | ``` 224 | 225 | 因此，解析逻辑如下所示： 226 | 227 | * 如果`module_name`以一个远程url模式开始： 228 | * 如果带有`--reload`标志，强制下载文件和使用它。 229 | * 否则 230 | * 如果本地缓存文件存在，使用它。 231 | * 否则，下载文件到`$DENO_DIR/deps`目录，然后使用它。 232 | * 如果`module_name`表示一个本地源文件，使用本地文件。 233 | 234 | 235 | 236 | 237 | 238 | -------------------------------------------------------------------------------- /chinese/advanced/interaction-with-v8.md: -------------------------------------------------------------------------------- 1 | --- 2 | 描述: >- 3 | 在这一部分，我们将会探索libdeno是如何用来跟Deno底层的JavaScript引擎V8交互的。 4 | --- 5 | 6 | # 跟V8的交互 7 | 8 | ## 创建 V8 Platform 9 | 10 | 从`src/main.rs`文件，我们可以找到如下两行代码： 11 | 12 | {% code-tabs %} 13 | {% code-tabs-item title="src/main.rs" %} 14 | ```rust 15 | let snapshot = snapshot::deno_snapshot(); 16 | let isolate = isolate::Isolate::new(snapshot, state, ops::dispatch); 17 | ``` 18 | {% endcode-tabs-item %} 19 | {% endcode-tabs %} 20 | 21 | 这里是V8 isolate被创建的地方。进入到文件`src/isolate.rs`中`Isolate::new`的定义： 22 | 23 | {% code-tabs %} 24 | {% code-tabs-item title="src/isolate.rs" %} 25 | ```rust 26 | pub fn new( 27 | snapshot: libdeno::deno_buf, 28 | state: Arc, 29 | dispatch: Dispatch, 30 | ) -> Self { 31 | DENO_INIT.call_once(|| { 32 | unsafe { libdeno::deno_init() }; 33 | }); 34 | let config = libdeno::deno_config { 35 | will_snapshot: 0, 36 | load_snapshot: snapshot, 37 | shared: libdeno::deno_buf::empty(), 38 | recv_cb: pre_dispatch, // 当Rust接受到消息，需要执行的回调 39 | }; 40 | let libdeno_isolate = unsafe { libdeno::deno_new(config) }; 41 | // 这个通道处理发送异步消息回运行时 42 | let (tx, rx) = mpsc::channel::<(i32, Buf)>(); 43 | 44 | Self { 45 | libdeno_isolate, 46 | dispatch, 47 | rx, 48 | tx, 49 | ntasks: Cell::new(0), 50 | timeout_due: Cell::new(None), 51 | state, 52 | } 53 | } 54 | ``` 55 | {% endcode-tabs-item %} 56 | {% endcode-tabs %} 57 | 58 | 在这里，我们找到了`libdeno`上的两个调用：`deno_init`和`deno_new`。跟其他的函数不同，这两个函数是在文件`libdeno/api.cc`中定义的，感谢FFI 和 `libc`，Rust中可以调用这两个C/C++函数。他们的Rust接口是由`src/libdeno.rs`提供的。 59 | 60 | 进入`libdeno/api.cc`文件： 61 | 62 | {% code-tabs %} 63 | {% code-tabs-item title="libdeno/api.cc" %} 64 | ```c 65 | void deno_init() { 66 | auto* p = v8::platform::CreateDefaultPlatform(); 67 | v8::V8::InitializePlatform(p); 68 | v8::V8::Initialize(); 69 | } 70 | 71 | Deno* deno_new(deno_config config) { 72 | // ... 忽略的代码 73 | deno::DenoIsolate* d = new deno::DenoIsolate(config); 74 | // ... 忽略的代码 75 | v8::Isolate* isolate = v8::Isolate::New(params); 76 | // ... 忽略的代码 77 | 78 | v8::Locker locker(isolate); 79 | v8::Isolate::Scope isolate_scope(isolate); 80 | { 81 | v8::HandleScope handle_scope(isolate); 82 | auto context = 83 | v8::Context::New(isolate, nullptr, v8::MaybeLocal(), 84 | v8::MaybeLocal(), 85 | v8::DeserializeInternalFieldsCallback( 86 | deno::DeserializeInternalFields, nullptr)); 87 | if (!config.load_snapshot.data_ptr) { 88 | // 如果没有提供snapshot，我们将会用空的main source和source map初始化context 89 | deno::InitializeContext(isolate, context); 90 | } 91 | d->context_.Reset(isolate, context); 92 | } 93 | 94 | return reinterpret_cast(d); 95 | } 96 | ``` 97 | {% endcode-tabs-item %} 98 | {% endcode-tabs %} 99 | 100 | 从这两个函数，我们可以发现，`deno_init`是用来初始化V8 platform，而`deno_new`是用来在该platform上创建一个新的独立的虚拟机实例的。\(一些V8嵌入APIs在这里被使用。为了了解更多的V8嵌入，查看[https://v8.dev/docs/embed](https://v8.dev/docs/embed)相关概念和[https://denolib.github.io/v8-docs/](https://denolib.github.io/v8-docs/)API 手册\) 101 | 102 | ## 添加扩展 103 | 104 | 有两个函数/构造函数`DenoIsolate`和`InitializeContext`在`deno_new`中被使用，可能目前还不清楚。或多或少的，`DenoIsolate`是用来收集Isolate的信息的。而`InitializeContext`是更加有意思的一个。\(好像没有snapshot被提供他才会在这里被调用。但是，你也会发现`libdeno/api.cc`中的`deno_new_snapshotter`函数会被调用来生成一个新的snapshot，所以`InitializeContext`总是会被调用的\) 105 | 106 | {% code-tabs %} 107 | {% code-tabs-item title="libdeno/binding.cc" %} 108 | ```c 109 | void InitializeContext(v8::Isolate* isolate, v8::Local context) { 110 | v8::HandleScope handle_scope(isolate); 111 | v8::Context::Scope context_scope(context); 112 | 113 | auto global = context->Global(); 114 | 115 | auto deno_val = v8::Object::New(isolate); 116 | CHECK(global->Set(context, deno::v8_str("libdeno"), deno_val).FromJust()); 117 | 118 | auto print_tmpl = v8::FunctionTemplate::New(isolate, Print); 119 | auto print_val = print_tmpl->GetFunction(context).ToLocalChecked(); 120 | CHECK(deno_val->Set(context, deno::v8_str("print"), print_val).FromJust()); 121 | 122 | auto recv_tmpl = v8::FunctionTemplate::New(isolate, Recv); 123 | auto recv_val = recv_tmpl->GetFunction(context).ToLocalChecked(); 124 | CHECK(deno_val->Set(context, deno::v8_str("recv"), recv_val).FromJust()); 125 | 126 | auto send_tmpl = v8::FunctionTemplate::New(isolate, Send); 127 | auto send_val = send_tmpl->GetFunction(context).ToLocalChecked(); 128 | CHECK(deno_val->Set(context, deno::v8_str("send"), send_val).FromJust()); 129 | 130 | CHECK(deno_val->SetAccessor(context, deno::v8_str("shared"), Shared) 131 | .FromJust()); 132 | } 133 | ``` 134 | {% endcode-tabs-item %} 135 | {% endcode-tabs %} 136 | 137 | 如果你对TypeScript端的`libdeno`API已经不陌生了，上面的被`deno::v8_str`包裹的名字你可能也不会陌生。事实上，这里是一些V8的C++扩展被绑定到JavaScript的地方，我们获取当前`Context`\(一个运行分隔的代码的V8的执行环境\)的全局变量，然后从C++添加一些额外的属性到全局变量。 138 | 139 | 基于上面的代码，当你从TypeScript调用`libdeno.send(...)`\(你会在`js/dispatch.ts`文件的`sendInternal`函数找到该调用\)，你实际上是调用了`Send`这个C++函数。对于`libdeno.print`调用也是同样的。 140 | 141 | 另一个例子：`console.log(...)`定义如下： 142 | 143 | {% code-tabs %} 144 | {% code-tabs-item title="js/console.ts" %} 145 | ```typescript 146 | log = (...args: any[]): void => { 147 | this.printFunc(stringifyArgs(args)); 148 | }; 149 | ``` 150 | {% endcode-tabs-item %} 151 | {% endcode-tabs %} 152 | 153 | `printFunc`是`Console`类的一个私有的属性，他的初始化值是这样的： 154 | 155 | {% code-tabs %} 156 | {% code-tabs-item title="js/globals.ts" %} 157 | ```typescript 158 | window.console = new consoleTypes.Console(libdeno.print); 159 | ``` 160 | {% endcode-tabs-item %} 161 | {% endcode-tabs %} 162 | 163 | ok，我们在这里找到了`libdeno.print`。从上面的扩展代码中，我们知道从TypeScript调用`libdeno.print`实际上将会调用`libdeno/binding.cc`中的`Print`。 164 | 165 | ```cpp 166 | void Print(const v8::FunctionCallbackInfo& args) { 167 | // ... 忽略的代码 168 | v8::String::Utf8Value str(isolate, args[0]); 169 | bool is_err = 170 | args.Length() >= 2 ? args[1]->BooleanValue(context).ToChecked() : false; 171 | FILE* file = is_err ? stderr : stdout; 172 | fwrite(*str, sizeof(**str), str.length(), file); 173 | fprintf(file, "\n"); 174 | fflush(file); 175 | } 176 | ``` 177 | 178 | 一点也不惊讶，跟C语言的print是一样的。因此，调用`console.log`最终也是间接调用了`fwrite/fprintf`！ 179 | 180 | 查阅`libdeno/binding.cc`中的`Print`，`Send`和`Recv`来了解更多发生在底层的细节。 181 | 182 | ## 在V8中执行代码 183 | 184 | 从`src/main.rs`文件中，紧接着isolate的创建，我们发现 185 | 186 | {% code-tabs %} 187 | {% code-tabs-item title="src/main.rs" %} 188 | ```rust 189 | isolate 190 | .execute("denoMain();") 191 | .unwrap_or_else(print_err_and_exit); 192 | ``` 193 | {% endcode-tabs-item %} 194 | {% endcode-tabs %} 195 | 196 | 从前面的部分，我们知道`denoMain`是一个TypeScript端的函数。`isolate.execute`被用来执行代码。让我们提取出他在`src/isolate.rs`文件中的定义： 197 | 198 | {% code-tabs %} 199 | {% code-tabs-item title="src/isolate.rs" %} 200 | ```rust 201 | /// 跟execute2()一样， 但是文件名默认实际是 ""。 202 | pub fn execute(&self, js_source: &str) -> Result<(), JSError> { 203 | self.execute2("", js_source) 204 | } 205 | 206 | /// 执行提供的JavaScript源码。 207 | /// js_filename参数仅仅当调试时提供。 208 | pub fn execute2( 209 | &self, 210 | js_filename: &str, 211 | js_source: &str, 212 | ) -> Result<(), JSError> { 213 | let filename = CString::new(js_filename).unwrap(); 214 | let source = CString::new(js_source).unwrap(); 215 | let r = unsafe { 216 | libdeno::deno_execute( 217 | self.libdeno_isolate, 218 | self.as_raw_ptr(), 219 | filename.as_ptr(), 220 | source.as_ptr(), 221 | ) 222 | }; 223 | if r == 0 { 224 | let js_error = self.last_exception().unwrap(); 225 | return Err(js_error); 226 | } 227 | Ok(()) 228 | } 229 | ``` 230 | {% endcode-tabs-item %} 231 | {% endcode-tabs %} 232 | 233 | 我们最感兴趣的部分是`libdeno::deno_execute`。我们可以再次在`libdeno/api.cc`找到他的实际定义： 234 | 235 | {% code-tabs %} 236 | {% code-tabs-item title="libdeno/api.cc" %} 237 | ```c 238 | int deno_execute(Deno* d_, void* user_data, const char* js_filename, 239 | const char* js_source) { 240 | auto* d = unwrap(d_); 241 | // ... 忽略的代码 242 | auto context = d->context_.Get(d->isolate_); 243 | CHECK(!context.IsEmpty()); 244 | return deno::Execute(context, js_filename, js_source) ? 1 : 0; 245 | } 246 | ``` 247 | {% endcode-tabs-item %} 248 | {% endcode-tabs %} 249 | 250 | 最后，我们找到`deno::Execute`： 251 | 252 | ```cpp 253 | bool Execute(v8::Local context, const char* js_filename, 254 | const char* js_source) { 255 | // ... 忽略的代码 256 | auto source = v8_str(js_source); 257 | return ExecuteV8StringSource(context, js_filename, source); 258 | } 259 | 260 | bool ExecuteV8StringSource(v8::Local context, 261 | const char* js_filename, 262 | v8::Local source) { 263 | // ... 忽略的代码 264 | 265 | v8::TryCatch try_catch(isolate); 266 | 267 | auto name = v8_str(js_filename); 268 | 269 | v8::ScriptOrigin origin(name); 270 | 271 | auto script = v8::Script::Compile(context, source, &origin); 272 | 273 | if (script.IsEmpty()) { 274 | DCHECK(try_catch.HasCaught()); 275 | HandleException(context, try_catch.Exception()); 276 | return false; 277 | } 278 | 279 | auto result = script.ToLocalChecked()->Run(context); 280 | 281 | if (result.IsEmpty()) { 282 | DCHECK(try_catch.HasCaught()); 283 | HandleException(context, try_catch.Exception()); 284 | return false; 285 | } 286 | 287 | return true; 288 | } 289 | ``` 290 | 291 | 正如我们看到的，`Execute`最终把代码提交给了`v8::Script::Compile`，`v8::Script::Compile`编译了JavaScript代码然后调用他的`Run`方法。编译或者运行时，如果任何异常发生，都会被传递给`HandleException`： 292 | 293 | {% code-tabs %} 294 | {% code-tabs-item title="libdeno/binding.cc" %} 295 | ```cpp 296 | void HandleException(v8::Local context, 297 | v8::Local exception) { 298 | v8::Isolate* isolate = context->GetIsolate(); 299 | DenoIsolate* d = FromIsolate(isolate); 300 | std::string json_str = EncodeExceptionAsJSON(context, exception); 301 | CHECK(d != nullptr); 302 | d->last_exception_ = json_str; 303 | } 304 | ``` 305 | {% endcode-tabs-item %} 306 | {% endcode-tabs %} 307 | 308 | 他设置了`d->last_exception_`为一个JSON格式的错误信息，该消息会被`deno_last_exception`读取： 309 | 310 | 311 | {% code-tabs %} 312 | {% code-tabs-item title="libdeno/api.cc" %} 313 | ```c 314 | const char* deno_last_exception(Deno* d_) { 315 | auto* d = unwrap(d_); 316 | if (d->last_exception_.length() > 0) { 317 | return d->last_exception_.c_str(); 318 | } else { 319 | return nullptr; 320 | } 321 | } 322 | ``` 323 | {% endcode-tabs-item %} 324 | {% endcode-tabs %} 325 | 326 | Rust中会在`Isolate::last_exception`使用该错误信息： 327 | 328 | {% code-tabs %} 329 | {% code-tabs-item title="src/isolate.rs" %} 330 | ```rust 331 | pub fn last_exception(&self) -> Option { 332 | let ptr = unsafe { libdeno::deno_last_exception(self.libdeno_isolate) }; 333 | // ... 忽略的代码 334 | } 335 | ``` 336 | {% endcode-tabs-item %} 337 | {% endcode-tabs %} 338 | 339 | 并且在`Isolate::execute`内会检查是否错误处理是必需的。 340 | 341 | 哟！在TypeScript，C/C++和Rust之间我们经历了一个漫长的旅行。现在你应该非常清楚Deno是如何跟V8交互的了。其实没什么特别的！ 342 | -------------------------------------------------------------------------------- /chinese/advanced/process-lifecycle.md: -------------------------------------------------------------------------------- 1 | # 进程生命周期 2 | 3 | ## Rust main\(\) 入口 4 | 5 | 一个Deno进程的启动入口是`src/main.rs`文件的`main`函数： 6 | 7 | {% code-tabs %} 8 | {% code-tabs-item title="src/main.rs" %} 9 | ```rust 10 | fn main() { 11 | log::set_logger(&LOGGER).unwrap(); 12 | let args = env::args().collect(); 13 | let (flags, rest_argv, usage_string) = 14 | flags::set_flags(args).unwrap_or_else(|err| { 15 | eprintln!("{}", err); 16 | std::process::exit(1) 17 | }); 18 | 19 | if flags.help { 20 | println!("{}", &usage_string); 21 | std::process::exit(0); 22 | } 23 | 24 | log::set_max_level(if flags.log_debug { 25 | log::LevelFilter::Debug 26 | } else { 27 | log::LevelFilter::Warn 28 | }); 29 | 30 | let state = Arc::new(isolate::IsolateState::new(flags, rest_argv)); 31 | let snapshot = snapshot::deno_snapshot(); 32 | let isolate = isolate::Isolate::new(snapshot, state, ops::dispatch); 33 | tokio_util::init(|| { 34 | isolate 35 | .execute("denoMain();") 36 | .unwrap_or_else(print_err_and_exit); 37 | isolate.event_loop().unwrap_or_else(print_err_and_exit); 38 | }); 39 | } 40 | ``` 41 | {% endcode-tabs-item %} 42 | {% endcode-tabs %} 43 | 44 | 在处理了flags之后，Deno接着创建了一个V8 isolate\(一个独立的拥有自己的堆的V8虚拟机实例\)。我们同时也传入了snapshot，snapshot包含了序列化的堆数据，而堆数据则是关于TypeScript编译器和一些前台代码的。这让Deno的启动更加快。 45 | 46 | 紧跟着，我们启动Tokio，并且让isolate去执行一个叫做`denoMain`的函数。只有在完成了这个函数的运行之后，Deno开始了他的事件循环。一旦事件循环结束，Deno进程结束。 47 | 48 | 49 | ## denoMain\(\) 50 | 51 | `denoMain`位于文件`js/main.ts`。 52 | 53 | {% code-tabs %} 54 | {% code-tabs-item title="js/main.ts" %} 55 | ```typescript 56 | export default function denoMain() { 57 | libdeno.recv(handleAsyncMsgFromRust); 58 | 59 | const startResMsg = sendStart(); 60 | setLogDebug(startResMsg.debugFlag()); 61 | 62 | const compiler = Compiler.instance(); 63 | 64 | // ... 被省略的代码 65 | 66 | const cwd = startResMsg.cwd(); 67 | 68 | // ... 被省略的代码 69 | 70 | const runner = new Runner(compiler); 71 | 72 | if (inputFn) { 73 | runner.run(inputFn, `${cwd}/`); 74 | } else { 75 | replLoop(); 76 | } 77 | } 78 | ``` 79 | {% endcode-tabs-item %} 80 | {% endcode-tabs %} 81 | 82 | 83 | 首先，我们告诉`libdeno`，当从Rust端有消息发送过来，就把buffer转发给`handleAsyncMsgFromRust`函数，`libdeno`是从中台暴露出来的API。然后，一个`Start`消息被发送给Rust端，表明我们正在启动和接受包含当前工作目录`cwd`的信息。然后我们我们决定用户是运行脚本还是使用REPL。如果我们传入了一个文件名，Deno会尝试让`runner`运行这个文件， `runner`包含了TypeScript `compiler`\(当你深入该函数的定义，你会发现在某处一个神秘的`eval/globalEval`被调用\)。仅当`runner`运行完这个文件，`denoMain`才会结束。 84 | 85 | 86 | ## isolate.event\_loop\(\) 87 | 88 | 当`denoMain`运行结束，一系列的异步调用被启动，并且还没有得到响应。因此Deno接着调用`isolate.event_loop()`来开始事件循环。 89 | 90 | `event_loop`的代码是在`src/isolate.rs`文件中： 91 | 92 | {% code-tabs %} 93 | {% code-tabs-item title="src/isolate.rs" %} 94 | ```rust 95 | pub fn event_loop(&self) -> Result<(), JSError> { 96 | // 主线程事件循环 97 | while !self.is_idle() { 98 | match recv_deadline(&self.rx, self.get_timeout_due()) { 99 | Ok((req_id, buf)) => self.complete_op(req_id, buf), 100 | Err(mpsc::RecvTimeoutError::Timeout) => self.timeout(), 101 | Err(e) => panic!("recv_deadline() failed: {:?}", e), 102 | } 103 | self.check_promise_errors(); 104 | if let Some(err) = self.last_exception() { 105 | return Err(err); 106 | } 107 | } 108 | // 结束后的检查 109 | self.check_promise_errors(); 110 | if let Some(err) = self.last_exception() { 111 | return Err(err); 112 | } 113 | Ok(()) 114 | } 115 | ``` 116 | {% endcode-tabs-item %} 117 | {% endcode-tabs %} 118 | 119 | 这就是一个循环，每一次，我们调用`self.is_idle()`检查是否还有事件没有得到响应。如果是，我们将会等待`self.rx`来接受一些消息或者一个由`setTimeout`创建的定时器超时，`self.rx`是消息通道的接收端。 120 | 121 | 如果`self.rx`接受到一些消息，这意味着Tokio线程池里面有一个任务已经完成了。我们接下来将会把执行结果发送给`self.complete_op`来处理，执行结果是由`req_id`\(任务id\)和`buf`\(包含序列化响应的buffer\)两个组成的。`self.complete_op`根据`req_id`调用正在等待结果的代码，运行到结束。需要注意的是，这里被调用的代码，可能会加入更多的异步操作。 122 | 123 | 如果发生了超时，由`setTimeout`加入的代码将会被调用，然后运行到结束。 124 | 125 | 事件循环将会继续等待输出和运行更多的代码，直到`self.is_idle`返回`true`为止。当没有挂起的任务时，`self.is_idle`返回true，然后，事件循环退出，Deno结束运行。 126 | 127 | 当执行代码的时候，有可能会有错误发生，如果错误发生了，Deno将会退出。如果是一个promise错误\(比如未捕获的错误\)，Deno将会调用`self.check_promise_errors`来进行检查。 128 | 129 | ## Promise表 130 | 131 | 在上面，我们提到，`req_id`作为一个任务id提供给了`self.complete_op`。事实上，他会被用来从一个promise表中查找获取相应的promise，promise表中的promise是由异步操作创建的，promise表包含了所有等待resolve的promise。 132 | 133 | promise表位于文件`src/dispatch.ts` 134 | 135 | {% code-tabs %} 136 | {% code-tabs-item title="src/dispatch.ts" %} 137 | ```typescript 138 | const promiseTable = new Map>(); 139 | ``` 140 | {% endcode-tabs-item %} 141 | {% endcode-tabs %} 142 | 143 | Promise表是由`dispatch.sendAsync`函数填充的，`dispatch.sendAsync`被用来转发异步请求给Rust。 144 | 145 | {% code-tabs %} 146 | {% code-tabs-item title="src/dispatch.ts" %} 147 | ```typescript 148 | // @internal 149 | export function sendAsync( 150 | builder: flatbuffers.Builder, 151 | innerType: msg.Any, 152 | inner: flatbuffers.Offset, 153 | data?: ArrayBufferView 154 | ): Promise { 155 | const [cmdId, resBuf] = sendInternal(builder, innerType, inner, data, false); 156 | util.assert(resBuf == null); 157 | const promise = util.createResolvable(); 158 | promiseTable.set(cmdId, promise); 159 | return promise; 160 | } 161 | ``` 162 | {% endcode-tabs-item %} 163 | {% endcode-tabs %} 164 | 165 | 这里的`cmdId`其实就是Rust端的`req_id`。他是由`sendInternal`返回的。`sendInternal`调用了V8的扩展并且递增了一个全局的请求id计数。对于每一个异步操作，我们创建了一个deferred/resolvable，并以`cmdId`为key存入promise表中。 166 | 167 | 在`denoMain`部分，我们知道，`libdeno`绑定了`handleAsyncMsgFromRust`作为接受新消息的回调。下面是他的源码： 168 | 169 | ```typescript 170 | export function handleAsyncMsgFromRust(ui8: Uint8Array) { 171 | // If a the buffer is empty, recv() on the native side timed out and we 172 | // did not receive a message. 173 | if (ui8.length) { 174 | const bb = new flatbuffers.ByteBuffer(ui8); 175 | const base = msg.Base.getRootAsBase(bb); 176 | const cmdId = base.cmdId(); 177 | const promise = promiseTable.get(cmdId); 178 | util.assert(promise != null, `Expecting promise in table. ${cmdId}`); 179 | promiseTable.delete(cmdId); 180 | const err = errors.maybeError(base); 181 | if (err != null) { 182 | promise!.reject(err); 183 | } else { 184 | promise!.resolve(base); 185 | } 186 | } 187 | // Fire timers that have become runnable. 188 | fireTimers(); 189 | } 190 | ``` 191 | 192 | 我们注意到，他从响应里面取出`cmdId`，然后从表里面取出相应的promise。这个promise接着被resolve，允许等待该任务响应的代码执行。 193 | 194 | ## 生命周期例子 195 | 196 | 对于像下面这样的例子： 197 | 198 | ```typescript 199 | // RUN FILE 200 | import * as deno from 'deno'; 201 | 202 | const dec = new TextDecoder(); 203 | 204 | (async () => { 205 | // 打印 "A" 206 | console.log(dec.decode(await deno.readFile('a.c'))); 207 | // 打印 "B" 208 | console.log(dec.decode(deno.readFileSync('b.c'))); 209 | // 打印 "C" 210 | console.log(dec.decode(await deno.readFile('c.c'))); 211 | })(); 212 | 213 | // 打印 "D" 214 | console.log(dec.decode(deno.readFileSync('d.c'))); 215 | 216 | // END FILE 217 | ``` 218 | 219 | Deno进程生命周期控制流图看起来像这样： 220 | 221 | ```text 222 | Rust TypeScript (V8) 223 | 224 | main 225 | | 226 | 创建 isolate 227 | | 228 | | 执行 229 | |--------------> denoMain 230 | | 231 | startMsg | 232 | |<-------------------| 233 | |------------------->| 234 | startResMsg | 235 | | 236 | RUN FILE 237 | readFile('a.c') | 238 | (线程池) |<-------------------| 239 | | 240 | | 241 | readFileSync('d.c')| 242 | |<-------------------| 243 | | 244 | 堵塞的执行 245 | | 246 | |------------------->| 247 | (Uint8Array) "D" | 248 | console.log("D") 249 | | 250 | END FILE | 251 | |....................| # 没有更多的同步代码需要执行 252 | | 253 | 事件循环 254 | | 255 | 更多的任务 256 | | 257 | | 'a.c' 结束读取 258 | (线程池) |------------------->| 259 | (Uint8Array) "A" | 260 | console.log("A") 261 | | 262 | readFileSync('b.c')| 263 | |<-------------------| 264 | | 265 | 堵塞的执行 266 | | 267 | |------------------->| 268 | (Uint8Array) "B" | 269 | console.log("B") 270 | | 271 | readFile('c.c') | 272 | (线程池) |<-------------------| 273 | | 274 | |....................| # 没有更多的同步代码需要执行 275 | | 276 | 更多的任务 277 | | 278 | | 'c.c' 结束读取 279 | (线程池) |------------------->| 280 | (Uint8Array) "C" | 281 | console.log("C") 282 | | 283 | | 284 | |....................| # 没有更多的同步代码需要执行 285 | | 286 | 没有任务 287 | | 288 | | 289 | v 290 | 退出 291 | ``` 292 | 293 | 294 | 295 | -------------------------------------------------------------------------------- /chinese/advanced/under-the-call-site-hood.md: -------------------------------------------------------------------------------- 1 | --- 2 | 描述: >- 3 | 在本节中，我们将深入源代码，了解TypeScript函数是如何与Rust实现进行通信的。 4 | --- 5 | 6 | # 从调用的角度 7 | 8 | 接下来，我们将会使用`readFileSync`和`readFile`来揭示底层的原理。 9 | 10 | ## TypeScript调用 11 | 12 | 在Deno，我们可以像下面这样读取文件： 13 | 14 | ```typescript 15 | import * as deno from "deno"; 16 | 17 | async function readFileTest() { 18 | // 异步读取文件 19 | const data1 = await deno.readFile("test.json"); 20 | 21 | // 同步读取文件 22 | // 需要注意的是，我们不需要`await`关键字。 23 | const data2 = deno.readFile("test.json"); 24 | } 25 | ``` 26 | 27 | 上面的代码是非常简单和直观的，但是究竟Typescript是如何读取文件的呢？没有了`libuv`，到底Deno是如何实现异步调用的呢？ 28 | 29 | 事实上，需要进行系统调用的Deno APIs会像下面这样，被转换为消息处理： 30 | 31 | ```typescript 32 | function readFileSync(filename: string): Uint8Array { 33 | return res(dispatch.sendSync(...req(filename))); 34 | } 35 | 36 | async function readFile(filename: string): Promise { 37 | return res(await dispatch.sendAsync(...req(filename))); 38 | } 39 | ``` 40 | 41 | `sendSync`和`sendAsync`方法底层都调用了`libdeno.send`方法，这个方法是添加到V8的C++扩展。通过这样的处理，函数调用信息被传递给了C++。 42 | 43 | 除了分发消息，TypeScript端也需要注册回调函数来从Rust端获得异步回调的结果： 44 | 45 | ```typescript 46 | // js/main.ts denoMain() 47 | 48 | libdeno.recv(handleAsyncMsgFromRust); 49 | ``` 50 | 51 | `libdeno.recv`也是一个注入到V8的C++扩展函数，他用来注册接受消息的回调函数。了解更多的信息，请查看[下一节](https://github.com/denolib/guide/chinese/advanced/process-lifecycle.md)。 52 | 53 | 让我们看看C++是如何处理接受到的的消息的。 54 | 55 | ## C++转换器 56 | 57 | 注意：下面不是真实的`libdeno.send`代码，只是为了方便我们更好的理解而已，真实的代码不是这个样子的。 58 | 59 | ```cpp 60 | // libdeno/binding.cc 61 | 62 | void Send(const pseudo::Args args) { 63 | // 获取调用信息，比如调用函数名字等。 64 | pseudo::Value control = args[0]; 65 | // 获取带有许多数据的参数。 66 | pseudo::Value data; 67 | // 获取请求id 68 | int32_t req_id = isolate->next_req_id_++; 69 | 70 | if (args.Length() == 2) { 71 | data = args[1]; 72 | } 73 | 74 | isolate->current_args_ = &args; 75 | 76 | isolate->recv_cb_(req_id, control, data); 77 | } 78 | ``` 79 | 80 | `Send`函数获取参数然后调用`recv_cb_`。需要注意的是`recv_cb_`是定义在Rust代码中的。 81 | 82 | `libdeno.send`的返回值将会被`deno_respond`设置： 83 | 84 | ```cpp 85 | // libdeno/api.cc 86 | // 这也不是真实的代码 87 | 88 | int deno_respond(DenoIsolate* d, int32_t req_id, deno_buf buf) { 89 | // 同步响应。 90 | if (d->current_args_ != nullptr) { 91 | // 获取作为Uint8Array类型的返回值 92 | auto ab = deno::ImportBuf(d, buf); 93 | // 设置返回值 94 | d->current_args_->GetReturnValue().Set(ab); 95 | d->current_args_ = nullptr; 96 | return 0; 97 | } 98 | 99 | // 异步响应。 100 | // 获取定义在TypeScript中的回调函数。 101 | auto recv_ = d->recv_.Get(d->isolate_); 102 | if (recv_.IsEmpty()) { 103 | d->last_exception_ = "libdeno.recv_ has not been called."; 104 | return 1; 105 | } 106 | 107 | pseudo::Value args[1]; 108 | // 获取作为Uint8Array类型的返回值 109 | args[0] = deno::ImportBuf(d, buf); 110 | // 调用TypeScript函数 111 | auto v = recv_->Call(context, context->Global(), 1, args); 112 | 113 | return 0; 114 | } 115 | ``` 116 | 117 | `deno_respond`函数是被`recv_cb_`调用的，并且当被执行的时候，会区分同步和异步。当调用是同步的，`deno_respond`直接返回结果。当调用是异步的，函数是被异步触发的，然后调用定义在TypeScript中的函数。 118 | 119 | ## Rust执行器 120 | 121 | 函数调用之旅，进入到了最后的阶段。Rust代码映射不同类型的函数调用到相应的处理操作，并且同步和异步调用是基于事件循环的。对于事件循环的更多内容，请查看[下一节](https://github.com/denolib/guide/chinese/advanced/process-lifecycle.md) 122 | 123 | -------------------------------------------------------------------------------- /chinese/codebase-basics/example-adding-a-deno-api.md: -------------------------------------------------------------------------------- 1 | # 例子: 贡献一个新的api给Deno 2 | 3 | 在这个部分，我们将会演示一个简单的例子，这个例子给Deno增加了产生某个范围内的一个随机数的功能。 4 | 5 | \(这仅仅是一个例子，你完全可以用JavaScript的`Math.random`api完成。\) 6 | 7 | ## 定义消息 8 | 9 | 正如在前面几节我们提到的，Deno依靠消息传递在TypeScript和Rust之间进行通信来完成所有的工作。因此，我们必须首先定义要发送和接收的消息。 10 | 11 | 打开`src/msg.fbs`文件，然后做如下操作： 12 | 13 | {% code-tabs %} 14 | {% code-tabs-item title="src/msg.fbs" %} 15 | ```text 16 | union Any { 17 | // ... 其他消息，忽略 18 | // 添加如下几行 19 | RandRange, 20 | RandRangeRes 21 | } 22 | 23 | // 添加下面的tables 24 | table RandRange { 25 | from: int32; 26 | to: int32; 27 | } 28 | 29 | table RandRangeRes { 30 | result: int32; 31 | } 32 | ``` 33 | {% endcode-tabs-item %} 34 | {% endcode-tabs %} 35 | 36 | 通过这些代码，我们告诉FlatBuffers我们想要两个新的消息类型，`RandRange`和`RandRangeRes`。`RandRange`包含`from`到`to`的范围。`RandRangeRes`包含一个值，表示我们从Rsut中产生的一个随机数。 37 | 38 | 现在，运行`./tools/build.py`来更新相应的序列化和反序列化代码。自动生成的代码被保存在`target/debug/gen/msg_generated.ts`和`target/debug/gen/msg_generated.rs`文件，如果你感兴趣，可以看看这两个文件。 39 | 40 | ## 添加前台代码 41 | 42 | 进入`js/`目录。我们将会新增TypeScript前台接口。 43 | 44 | 创建一个新文件，`js/rand_range.ts`。增加下面的imports。 45 | 46 | {% code-tabs %} 47 | {% code-tabs-item title="js/rand\_range.ts" %} 48 | ```typescript 49 | import * as msg from "gen/msg_generated"; 50 | import * as flatbuffers from "./flatbuffers"; 51 | import { assert } from "./util"; 52 | import * as dispatch from "./dispatch"; 53 | ``` 54 | {% endcode-tabs-item %} 55 | {% endcode-tabs %} 56 | 57 | 我们将会使用`dispatch.sendAsync`和`dispatch.sendSync`来分发异步或者同步操作。 58 | 59 | 因为我们处理的是消息，所以我们想要做一些序列化和反序列化的工作。对于请求，我们需要提供`from`和`to`给`RandRange`表；对于响应，我们需要从表`RandRangeRes`中提取`result`。 60 | 61 | 因此，让我们定义两个函数，`req`和`res`，来完成这些工作： 62 | 63 | {% code-tabs %} 64 | {% code-tabs-item title="js/rand\_range.ts" %} 65 | ```typescript 66 | function req( 67 | from: number, 68 | to: number, 69 | ): [flatbuffers.Builder, msg.Any, flatbuffers.Offset] { 70 | // 获取一个builder来创建一个序列化的buffer 71 | const builder = flatbuffers.createBuilder(); 72 | msg.RandRange.startRandRange(builder); 73 | // 把数据存入buffer！ 74 | msg.RandRange.addFrom(builder, from); 75 | msg.RandRange.addTo(builder, to); 76 | const inner = msg.RandRange.endRandRange(builder); 77 | // 我们返回这三个信息。 78 | // dispatch.sendSync/sendAsync将会需要这些参数！ 79 | // （可以把这些操作当做是模板） 80 | return [builder, msg.Any.RandRange, inner]; 81 | } 82 | 83 | function res(baseRes: null | msg.Base): number { 84 | // 一些检查 85 | assert(baseRes !== null); 86 | // 确保我们确实得到正确的响应类型 87 | assert(msg.Any.RandRangeRes === baseRes!.innerType()); 88 | // 创建RandRangeRes模板 89 | const res = new msg.RandRangeRes(); 90 | // Deserialize! 91 | // 反序列化！ 92 | assert(baseRes!.inner(res) !== null); 93 | // 提取出result 94 | return res.result(); 95 | } 96 | ``` 97 | {% endcode-tabs-item %} 98 | {% endcode-tabs %} 99 | 100 | 非常棒！定义了`req`和`res`，我们可以非常容易的定义实际的同步/异步APIs： 101 | 102 | {% code-tabs %} 103 | {% code-tabs-item title="js/rand\_range.ts" %} 104 | ```typescript 105 | // 同步 106 | export function randRangeSync(from: number, to: number): number { 107 | return res(dispatch.sendSync(...req(from, to))); 108 | } 109 | 110 | // 异步 111 | export async function randRange(from: number, to: number): Promise { 112 | return res(await dispatch.sendAsync(...req(from, to))); 113 | } 114 | ``` 115 | {% endcode-tabs-item %} 116 | {% endcode-tabs %} 117 | 118 | ## 暴露出接口 119 | 120 | 进入`js/deno.ts`然后增加下面一行： 121 | 122 | {% code-tabs %} 123 | {% code-tabs-item title="js/deno.ts" %} 124 | ```typescript 125 | export { randRangeSync, randRange } from "./rand_range"; 126 | ``` 127 | {% endcode-tabs-item %} 128 | {% endcode-tabs %} 129 | 130 | 同样, 进入`BUILD.gn`，然后把我们的文件添加到`ts_sources`： 131 | 132 | {% code-tabs %} 133 | {% code-tabs-item title="BUILD.gn" %} 134 | ```text 135 | ts_sources = [ 136 | "js/assets.ts", 137 | "js/blob.ts", 138 | "js/buffer.ts", 139 | # ... 140 | "js/rand_range.ts" 141 | ] 142 | ``` 143 | {% endcode-tabs-item %} 144 | {% endcode-tabs %} 145 | 146 | ## 增加后台代码 147 | 148 | 进入，`src/ops.rs`。这是唯一一个Rust后台代码文件，我们需要进行修改来增加这个功能的。 149 | 150 | 在`ops.rs`，让我们首先导入产生随机数的工具。非常幸运的，Deno已经包含了一个叫做`rand`的Rust crate，他可以用来完成这个任务。可以查阅这个API[here](https://docs.rs/rand/0.6.1/rand/)。 151 | 152 | 让我们导入他： 153 | 154 | {% code-tabs %} 155 | {% code-tabs-item title="src/ops.rs" %} 156 | ```rust 157 | use rand::{Rng, thread_rng}; 158 | ``` 159 | {% endcode-tabs-item %} 160 | {% endcode-tabs %} 161 | 162 | 现在我们可以创建一个函数\(通过一些模板代码\)来完成随机数范围逻辑： 163 | 164 | {% code-tabs %} 165 | {% code-tabs-item title="src/ops" %} 166 | ```rust 167 | fn op_rand_range( 168 | _state: &IsolateState, 169 | base: &msg::Base, 170 | data: libdeno::deno_buf, 171 | ) -> Box { 172 | assert_eq!(data.len(), 0); 173 | // 把消息解码为RandRange 174 | let inner = base.inner_as_rand_range().unwrap(); 175 | // 获取 command id, 用来响应异步调用。 176 | let cmd_id = base.cmd_id(); 177 | // 从buffer中获取`from`和`to` 178 | let from = inner.from(); 179 | let to = inner.to(); 180 | 181 | // 把我们的慢代码和响应代码包在这里 182 | // 基于dispatch.sendSync和dispatch.sendAsync， 183 | // base.sync()将会是true或false。 184 | // 如果是true，blocking()将会在主线程产生任务 185 | // 否则，blocking()将会在Tokio线程池产生任务 186 | blocking(base.sync(), move || -> OpResult { 187 | // 真正的随机数生成代码！ 188 | let result = thread_rng().gen_range(from, to); 189 | 190 | // 准备响应消息序列化 191 | // 现在把这些当做是模板化的代码 192 | let builder = &mut FlatBufferBuilder::new(); 193 | // 将消息类型设置为RandRangeRes 194 | let inner = msg::RandRangeRes::create( 195 | builder, 196 | &msg::RandRangeResArgs { 197 | result, // 把我们的结果放在这里 198 | }, 199 | ); 200 | // 把消息序列化 201 | Ok(serialize_response( 202 | cmd_id, // 用来回应给TypeScript，如果这是一个异步调用 203 | builder, 204 | msg::BaseArgs { 205 | inner: Some(inner.as_union_value()), 206 | inner_type: msg::Any::RandRangeRes, 207 | ..Default::default() 208 | }, 209 | )) 210 | }) 211 | } 212 | ``` 213 | {% endcode-tabs-item %} 214 | {% endcode-tabs %} 215 | 216 | 在完成我们的实现后，我们需要告诉Rust我们该何时调用他。 217 | 218 | 我们注意到在`src/ops.rs`中由一个`dispatch`函数。在这个函数里面，我们可以找到一个对于消息类型的`match`语句来设置对应的处理操作。让我们设置我们的函数作为`msg::Any::RandRange`类型的处理函数，插入下面这一行： 219 | 220 | {% code-tabs %} 221 | {% code-tabs-item title="src/ops.rs" %} 222 | ```rust 223 | pub fn dispatch( 224 | // ... 225 | ) -> (bool, Box) { 226 | // ... 227 | let op_creator: OpCreator = match inner_type { 228 | msg::Any::Accept => op_accept, 229 | msg::Any::Chdir => op_chdir, 230 | // ... 231 | /* 增加下面一行！ */ 232 | msg::Any::RandRange => op_rand_range, 233 | // ... 234 | _ => panic!(format!( 235 | "Unhandled message {}", 236 | msg::enum_name_any(inner_type) 237 | )), 238 | } 239 | } 240 | ``` 241 | {% endcode-tabs-item %} 242 | {% endcode-tabs %} 243 | 244 | 好了！这就是你需要为这个调用添加的所有代码！ 245 | 246 | 运行`./tools/build.py`来编译你的项目！现在你已经能够调用`deno.randRangeSync`和`deno.randRange`。 247 | 248 | 在终端中尝试运行！ 249 | 250 | ```bash 251 | $ ./target/debug/deno 252 | > deno.randRangeSync(0, 100) 253 | 96 254 | > (async () => console.log(await deno.randRange(0, 100)))() 255 | Promise {} 256 | > 74 257 | ``` 258 | 259 | 祝贺！ 260 | 261 | ## 增加测试 262 | 263 | 现在让我们为我们的代码增加基本的测试。 264 | 265 | 新建一个文件`js/rand_range_test.ts`： 266 | 267 | {% code-tabs %} 268 | {% code-tabs-item title="js/rand\_range\_ts.ts" %} 269 | ```typescript 270 | import { test, assert } from "./test_util.ts"; 271 | import * as deno from "deno"; 272 | 273 | test(function randRangeSync() { 274 | const v = deno.randRangeSync(0, 100); 275 | assert(0 <= v && v < 100); 276 | }); 277 | 278 | test(async function randRange() { 279 | const v = await deno.randRange(0, 100); 280 | assert(0 <= v && v < 100); 281 | }); 282 | ``` 283 | {% endcode-tabs-item %} 284 | {% endcode-tabs %} 285 | 286 | 把这个测试文件添加到`js/unit_tests.ts`： 287 | 288 | {% code-tabs %} 289 | {% code-tabs-item title="js/unit\_tests.ts" %} 290 | ```typescript 291 | // 增加这一行 292 | import "./rand_range_test.ts"; 293 | ``` 294 | {% endcode-tabs-item %} 295 | {% endcode-tabs %} 296 | 297 | 现在，运行`./tools/test.py`。对于所有的测试，你需要确认你的测试是通过的： 298 | 299 | ```text 300 | test randRangeSync_permW0N0E0R0 301 | ... ok 302 | test randRange_permW0N0E0R0 303 | ... ok 304 | ``` 305 | 306 | ## 提交䘝PR! 307 | 308 | 让我们假设Deno确实需要这个功能，并且您决心为Deno做出贡献。你可以提交一个PR！ 309 | 310 | 记得运行下面的命令来确保你的代码是可以提交的！ 311 | 312 | ```bash 313 | ./tools/format.py 314 | ./tools/lint.py 315 | ./tools/build.py 316 | ./tools/test.py 317 | ``` 318 | 319 | 320 | 321 | -------------------------------------------------------------------------------- /chinese/codebase-basics/infrastructure.md: -------------------------------------------------------------------------------- 1 | # Deno基础架构 2 | 3 | Deno主要由三个独立的部分构成： 4 | 5 | ## 1. TypeScript前台 6 | 7 | 那些没有直接的系统调用的公开的接口，APIs和大多数的函数功能，是由TypeScript前台实现的。当前使用了[https://deno.land/typedoc](https://deno.land/typedoc)工具来自动的生成TypeScript API文档\(后面可能回改用其他的工具\)。 8 | 9 | TypeScript/JavaScript被当做“没有特权方”，默认的，他们没有文件系统很网络的访问权限\(因为他们是运行在v8沙箱环境的\)。如果想要进行文件和网络访问，他们需要跟“特权方”，即rust后台进行通信。 10 | 11 | 12 | 因此许多的Deno APIs\(特别是系统调用\)是TypeScript前台通过创建buffers数据，然后发送给`libdeno`中台插件，然后等待\(同步或者异步\)结果回传回来。 13 | 14 | ## 2. C++ libdeno中台 15 | 16 | `libdeno`是处于TypeScript前台和Rust后台直接比较小的一层，他用来跟v8进行通信并且暴露出一些必须的插件。他是由C/C++实现的。 17 | 18 | 19 | `libdeno`暴露了在TypeScript和Rust之间发送和接受消息的传递接口。他也用来初始化v8 platforms/isolates 和创建或加载V8 snapshot\(用了保存序列化得堆信息的数据块。详细请参考[https://v8.dev/blog/custom-startup-snapshots](https://v8.dev/blog/custom-startup-snapshots)\)。值得注意的是，Deno的snapshot保护了一个完整的TypeScript编译器。这很大的缩短了编译器启动时间。 20 | 21 | ## 3. Rust后台 22 | 23 | 当前，有文件系统，网络和环境访问权限的后端或者“特权方”，是由Rust实现的。 24 | 25 | 对于不太熟悉Rust这门语言的人来说，Rust是由Mozilla开发出来的一门系统编程语言，具备内存和并发安全的特性。他也在[Servo](https://servo.org/)项目中被使用。 26 | 27 | 在2018年6月，Deno原形后端的技术选型是Go语言，后面从Go迁移到了Rust。主要原因是双GC。具体详情请查看[this thread](https://github.com/denoland/deno/issues/205)。 28 | 29 | -------------------------------------------------------------------------------- /chinese/codebase-basics/more-components.md: -------------------------------------------------------------------------------- 1 | # 更多的模块 2 | 3 | 在这一部分，我们将会简短的讨论Deno依赖的其他重要的模块。 4 | 5 | ## V8 6 | 7 | [V8](https://v8.dev/)是Google出品的JavaScript/WebAssembly引擎。他是由C++编写的，被Google Chrome浏览器 和 Node.js所使用。 8 | 9 | V8并不是原生支持TypeScript。实际上，所有你运行在Deno上的TypeScript代码会被保存为快照形式的TS 编译器编译成JavaScript代码，同时生成的文件会被缓存在`.deno`目录。除非用户更新代码，否则缓存的js文件在编译后会被一直使用。 10 | 11 | ## FlatBuffers 12 | 13 | [Flatbuffers](https://google.github.io/flatbuffers/)是一个由Google开发的，高效的跨平台的序列化库。他允许在不同的语言之间传递和接受消息，这其中没有打包和解包的性能损耗。 14 | 15 | 对于Deno来说，FlatBuffers被用来在进程内部的“特权方”和“没有特权方”之间进行消息通信。很多公开的Deno APIs内部，在TypeScript前台创建了包含序列化数据的buffers，然后这些buffers传递到Rust，使得Rust可以处理这些请求。在处理完这些请求后，Rust后台也创建了包含序列化数据的buffers回传给TypeScript，其中，反序列化这些buffers使用的是FlatBuffers编译器产生的文件。 16 | 17 | 相比较于Node，Node对于每一个特权调用创建了很多v8的扩展，而借助于FlatBuffers，Deno仅仅需要暴露出在TypeScript和Rust之间进行消息发送和消息接受的方法。 18 | 19 | 在Go原型中，Flatbuffers被用来取代[Protocol Buffers](https://developers.google.com/protocol-buffers/)，来避免序列化的性能损耗。具体可以查看[this thread](https://github.com/denoland/deno/issues/269)来获得更多的信息。 20 | 21 | ### 例子：通过FlatBuffers传递readFile数据 22 | 23 | TypeScript 代码: 24 | 25 | {% code-tabs %} 26 | {% code-tabs-item title="read\_file.ts" %} 27 | ```typescript 28 | import * as msg from "gen/msg_generated"; 29 | import * as flatbuffers from "./flatbuffers"; 30 | // dispatch被用来传递一个消息给Rust 31 | import * as dispatch from "./dispatch"; 32 | 33 | export async function readFile(filename: string): Promise { 34 | return res(await dispatch.sendAsync(...req(filename))); 35 | } 36 | 37 | function req( 38 | filename: string 39 | ): [flatbuffers.Builder, msg.Any, flatbuffers.Offset] { 40 | // 用来序列化消息的Builder 41 | const builder = flatbuffers.createBuilder(); 42 | const filename_ = builder.createString(filename); 43 | msg.ReadFile.startReadFile(builder); 44 | // 文件名被存入底层的ArrayBuffer 45 | msg.ReadFile.addFilename(builder, filename_); 46 | const inner = msg.ReadFile.endReadFile(builder); 47 | return [builder, msg.Any.ReadFile, inner]; 48 | } 49 | 50 | function res(baseRes: null | msg.Base): Uint8Array { 51 | // ... 52 | const inner = new msg.ReadFileRes(); 53 | // ... 54 | // Taking data out of FlatBuffers 55 | const dataArray = inner.dataArray(); 56 | // ... 57 | return new Uint8Array(dataArray!); 58 | } 59 | ``` 60 | {% endcode-tabs-item %} 61 | {% endcode-tabs %} 62 | 63 | Rust 代码: 64 | 65 | {% code-tabs %} 66 | {% code-tabs-item title="ops.rs" %} 67 | ```rust 68 | fn op_read_file( 69 | _config: &IsolateState, 70 | base: &msg::Base, 71 | data: libdeno::deno_buf, 72 | ) -> Box { 73 | // ... 74 | let inner = base.inner_as_read_file().unwrap(); 75 | let cmd_id = base.cmd_id(); 76 | // 从序列化buffer中取出filename 77 | let filename = PathBuf::from(inner.filename().unwrap()); 78 | // ... 79 | blocking(base.sync(), move || { 80 | // 真正的fs调用 81 | let vec = fs::read(&filename)?; 82 | // 序列化输出然后回传给TypeScript 83 | let builder = &mut FlatBufferBuilder::new(); 84 | let data_off = builder.create_vector(vec.as_slice()); 85 | let inner = msg::ReadFileRes::create( 86 | builder, 87 | &msg::ReadFileResArgs { 88 | data: Some(data_off), 89 | }, 90 | ); 91 | Ok(serialize_response( 92 | cmd_id, 93 | builder, 94 | msg::BaseArgs { 95 | inner: Some(inner.as_union_value()), 96 | inner_type: msg::Any::ReadFileRes, 97 | ..Default::default() 98 | }, 99 | )) 100 | }) 101 | } 102 | ``` 103 | {% endcode-tabs-item %} 104 | {% endcode-tabs %} 105 | 106 | ## Tokio 107 | 108 | [Tokio](https://tokio.rs/) 是一个Rust编写的异步运行时。他被用来创建和处理事件。他允许Deno在一个内部的线程池创建任务，然后当任务完成后，接收到输出的消息通知。 109 | 110 | Tokio依赖于Rust的`Future`特性，这个特性类似于JavaScript的Promises。 111 | 112 | ### 例子: 产生异步任务 113 | 114 | 在上面的`readFile`例子中，有一个`blocking`函数。他被用来决定一个任务被生成在主线程中运行还是交给Tokio的线程池。 115 | 116 | {% code-tabs %} 117 | {% code-tabs-item title="ops.rs" %} 118 | ```rust 119 | fn blocking(is_sync: bool, f: F) -> Box 120 | where 121 | F: 'static + Send + FnOnce() -> DenoResult, 122 | { 123 | if is_sync { 124 | // 在主线程中运行任务 125 | Box::new(futures::future::result(f())) 126 | } else { 127 | // 把任务交给Tokio 128 | Box::new(tokio_util::poll_fn(move || convert_blocking(f))) 129 | } 130 | } 131 | ``` 132 | {% endcode-tabs-item %} 133 | {% endcode-tabs %} 134 | 135 | 136 | 137 | -------------------------------------------------------------------------------- /chinese/codebase-basics/repo-structure.md: -------------------------------------------------------------------------------- 1 | # 源码一览 2 | 3 | 为了不让你感到迷茫，在这一部分中，我们将会探索当前Deno仓库的源码结构。 4 | 5 | 代码仓库的链接：[https://github.com/denoland/deno](https://github.com/denoland/deno) 6 | 7 | 我们将会列举出你可能会使用到的一些重要的目录或文件，然后简短的介绍他们： 8 | 9 | ```text 10 | # 目录 11 | build_extra/: 一些定制化的Deno构建规则 12 | js/: TypeScript前台 13 | libdeno/: C/C++中台 14 | src/: Rust后台 15 | tests/: 集成测试 16 | third_party/: 第三方代码 17 | tools/: 用来测试和构建Deno的工具 18 | 19 | # 文件 20 | BUILD.gn: 主要的Deno构建规则 21 | Cargo.toml: Rust后台的一些依赖信息 22 | package.json: Node依赖(主要是TypeScript的编译器) 23 | build.rs: Cargo构建规则 24 | ``` 25 | 26 | ## build\_extra/ 27 | 28 | 一些定制化的构建Deno的规则，当前是由FlatBuffers和Rust crates的构建规则组成。 29 | 30 | 你可能需要特别关注的一个文件是`build_extra/rust/BUILD.gn`。当在`third_party/`中添加一个新的Rust crate，我们需要在这个文件添加一个项。例如，我们添加了一个构建`time`crate的规则： 31 | 32 | {% code-tabs %} 33 | {% code-tabs-item title="build\_extra/rust/BUILD.gn" %} 34 | ``` 35 | rust_crate("time") { 36 | # carate存放在哪里 37 | source_root = "$registry_github/time-0.1.40/src/lib.rs" 38 | # 他的依赖是什么？ 39 | extern = [ 40 | ":libc", 41 | ":winapi", 42 | ] 43 | } 44 | ``` 45 | {% endcode-tabs-item %} 46 | {% endcode-tabs %} 47 | 48 | \(`rust_crate` GN 模板 是定义在 `build_extra/rust/rust.gni`\) 49 | 50 | ## js/ 51 | 52 | TypeScript前台代码存放的目录。大多数的APIs被定义在他们的文件，同时有一个关联的测试文件， `readFile` 被定义在 `read_file.ts`, 同时，他的单元测试代码是在 `read_file_test.ts`. 53 | 54 | 除了定义API的文件，还有其他一些特殊的文件： 55 | 56 | 1. `main.ts`: `denoMain` 被定义在这个文件, Rust调用这个方法作为启动TypeScript代码的入口。 57 | 2. `globals.ts`: 所有不需要imports的全局变量收归在这个文件。 58 | 3. `deno.ts`: All `deno` 命名空间的APIs从这里被导出。 59 | 4. `libdeno.ts`: 暴露给TypeScript的定制化的V8 API的类型定义。 60 | 5. `unit_tests.ts`: 所有的单元测试从这里被导入并且被执行。 61 | 6. `compiler.ts`: 为Deno定制化的TypeScript编译逻辑。 62 | 63 | ## libdeno/ 64 | 65 | C/C++ libdeno中台代码存放在这里 66 | 67 | 1. `api.cc`: 暴露给Rust的libdeno APIs 68 | 2. `binding.cc`: 处理跟V8之间的交互并且绑定V8 C++扩展。这些APIs被暴露给了Rust和TypeScript。 69 | 3. `snapshot_creator.cc`: 构建的时候创建V8 snapshot的逻辑。 70 | 71 | ## src/ 72 | 73 | Rust后端代码存放在这里 74 | 75 | 1. `libdeno.rs`: 暴露`libdeno/api.cc`的libdeno APIs给Rust。 76 | 2. `isolate.rs`: 获取V8 isolate \(一个V8引擎的isolated实例\)和使用了libdeno APIs的事件循环创建逻辑。 77 | 3. `deno_dir.rs`: Deno模块解析和缓存的逻辑。 78 | 4. `ops.rs`: 处理Deno Op \(operation\)。真正的文件系统和网络访问是在这里完成的。 79 | 5. `main.rs`: 包含了Rust的main函数。当你允许Deno 二进制文件，这里就是**真正的**入口。 80 | 6. `msg.fbs`: FlatBuffers消息结构的定义文件， 被用来进行消息传递。 当你需要添加或者修改消息结构的时候，你需要修改该文件。 81 | 82 | ## tests/ 83 | 84 | Deno的集成测试 85 | 86 | `*.js`或 `*.ts` 文件定义了每个测试样例的代码。`*.test`定义了这个测试样例该如何执行。通常的，`*.out`文件定义了一个测试样例的正常输出\(具体的文件名是在`*.test`文件中指定的\)。 87 | 88 | 请查阅[tests/README.md](https://github.com/denoland/deno/blob/master/tests/README.md)来获得更多的细节。 89 | 90 | ## third\_party/ 91 | 92 | Deno的第三方代码库。包含了node模块，v8，Rust crate代码等等。目录的位置是[https://github.com/denoland/deno\_third\_party](https://github.com/denoland/deno_third_party)。 93 | 94 | ## tools/ 95 | 96 | 用来执行一系列任务的大部分的Python脚本。 97 | 98 | 1. `build.py`: 构建Deno 99 | 2. `setup.py`: 设置构建环境和下载必须的代码 100 | 3. `format.py`: 代码格式化 101 | 4. `lint.py`: 代码提示 102 | 5. `sync_third_party.py`: 在用户修改`package.json`, `Cargo.toml`等文件之后，同步 third\_party 代码 103 | 104 | ## BUILD.gn 105 | 106 | 包含了Deno最重要的构建逻辑。在接下来的为Deno添加新的调用接口的例子中，我们将会讲解到该文件。 107 | 108 | ## build.rs 109 | 110 | 定义了`cargo build`的逻辑。内部调用的是`BUILD.gn`。 111 | 112 | -------------------------------------------------------------------------------- /chinese/contributing-guide.md: -------------------------------------------------------------------------------- 1 | # 源码贡献指南 2 | 3 | ## 开始 4 | 5 | 在开始阅读该文档前，**请先查看官方的文档** [CONTRIBUTING.md](https://github.com/denoland/deno/blob/master/.github/CONTRIBUTING.md) **Deno 团队出品**。本文档主要是做为官方文档的一个补充。 6 | 7 | ## 测试 release 版本 8 | 9 | 对于release版本\(非debug版本\)的测试，请先确保你构建的是release版本，然后 10 | 11 | ```bash 12 | ./tools/test.py target/release 13 | ``` 14 | 15 | ## 新增一个第三方包 16 | 17 | 根据包类型的不同，你需要确保正确的在 `package.json` 或 `Cargo.toml` 文件中新增包。在 `third_party/` 目录新增一个包后，请提交一个 PR 到[deno\_third\_party](https://github.com/denoland/deno_third_party)。一旦第三方包的 PR 被接收后，你才可以在 Deno 的仓库中提交 PR，Deno 中的PR对应你在第三方包仓库的提交并且会从CI工具中获取该包。 18 | 19 | 20 | 如果你提交的是 Rust crate，你需要在 `build_extra/rust/BUILD.gn` 文件中新增一个 `rust_crate` 项。你也需要在`BUILD.gn`文件的[main\_extern](https://github.com/denoland/deno/blob/73fb98ce70b327d1236b9540f3839a89c5d22ef0/BUILD.gn#L20-L48)中插入该项。 21 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /chinese/installing-deno.md: -------------------------------------------------------------------------------- 1 | # 安装 Deno 2 | 3 | ## 获取Deno的release二进制文件 4 | 5 | Deno现在还处在非常初级的发展阶段。当前的release版本是0.2.x，奉行“适度可用”原则，并且主要是给开发者预览的。 6 | 7 | 如果想要安装Deno的release二进制文件，需要在\*nix系统运行下面的命令 8 | 9 | ```bash 10 | curl -L https://deno.land/x/install/install.py | python 11 | export PATH=$HOME/.deno/bin:$PATH 12 | ``` 13 | 14 | 在windows系统上，你可能需要用Powershell来安装： 15 | 16 | ```text 17 | iex (iwr https://deno.land/x/install/install.ps1) 18 | ``` 19 | 20 | Deno是通过脚本[deno\_install](https://github.com/denoland/deno_install)安装的。如果你在安装过程 21 | 中遇到问题，请在那里提issue。 22 | 23 | ## 用源码编译Deno 24 | 25 | 如果你对贡献源码给Deno非常感兴趣，那么你可能想要从源码编译Deno。 26 | 27 | 运行下面的命令来编译生成一个debug版本 28 | 29 | ```bash 30 | # 获取源码. 31 | git clone --recurse-submodules https://github.com/denoland/deno.git 32 | cd deno 33 | # 设置和安装第三方依赖 34 | ./tools/setup.py 35 | # 构建. 36 | ./tools/build.py 37 | ``` 38 | 39 | {% hint style="info" %} 40 | 需要注意的是，有些域名，比如google.com，是被墙的。当运行`./tools/setup.py`时，你可能需要用VPN代理或者其他替代工具。 41 | {% endhint %} 42 | 43 | Deno是用GN和Ninja工具构建的，这两个工具也是Chromium团队的构建工具。生成的构建文件放在了`target/debug/`目录。 44 | 45 | 另外，如果想要构建一个release版本，在构建的时候，你需要设置你的环境DENO\_BUILD\_MODE为release： 46 | 47 | ```bash 48 | DENO_BUILD_MODE=release ./tools/build.py 49 | ``` 50 | 51 | 生成的构建文件将会存放在`target/release/`目录 52 | 53 | Deno 也可以用Cargo进行构建: 54 | 55 | ```bash 56 | cargo build 57 | ``` 58 | 59 | -------------------------------------------------------------------------------- /codebase-basics/example-adding-a-deno-api.md: -------------------------------------------------------------------------------- 1 | # Example: Adding to Deno API 2 | 3 | In this example, we will walk through a simple example that adds the functionality of generating a random number inside of a range. 4 | 5 | \(This is just an example. You can definitely implement with JavaScript `Math.random` if you want.\) 6 | 7 | ## Define Messages 8 | 9 | As mentioned in previous sections, Deno relies on message passing to communicate between TypeScript to Rust and do all the fancy stuff. Therefore, we must first define the messages we want to send and receive. 10 | 11 | Go to `src/msg.fbs` and do the following: 12 | 13 | {% code-tabs %} 14 | {% code-tabs-item title="src/msg.fbs" %} 15 | ```text 16 | union Any { 17 | // ... Other message types, omitted 18 | // ADD THE FOLLOWING LINES 19 | RandRange, 20 | RandRangeRes 21 | } 22 | 23 | // ADD THE FOLLOWING TABLES 24 | table RandRange { 25 | from: int32; 26 | to: int32; 27 | } 28 | 29 | table RandRangeRes { 30 | result: int32; 31 | } 32 | ``` 33 | {% endcode-tabs-item %} 34 | {% endcode-tabs %} 35 | 36 | With such code, we tell FlatBuffers that we want 2 new message types, `RandRange` and `RandRangeRes`. `RandRange` contains the range `from` and `to` we want. `RandRangeRes` contains a single value `result` that represents the random number we will get from the Rust side. 37 | 38 | Now, run `./tools/build.py` to update the corresponding serialization and deserialization code. Such auto-generated code are store in `target/debug/gen/msg_generated.ts` and `target/debug/gen/msg_generated.rs`, if you are interested. 39 | 40 | ## Add Frontend Code 41 | 42 | Go to `js/` folder. We will now create the TypeScript frontend interface. 43 | 44 | Create a new file, `js/rand_range.ts`. Add the following imports: 45 | 46 | {% code-tabs %} 47 | {% code-tabs-item title="js/rand\_range.ts" %} 48 | ```typescript 49 | import * as msg from "gen/msg_generated"; 50 | import * as flatbuffers from "./flatbuffers"; 51 | import { assert } from "./util"; 52 | import * as dispatch from "./dispatch"; 53 | ``` 54 | {% endcode-tabs-item %} 55 | {% endcode-tabs %} 56 | 57 | We will be using `dispatch.sendAsync` and `dispatch.sendSync` to dispatch the request as async or sync operation. 58 | 59 | Since we are dealing with messages, we need to do some serialization and deserialization work. For request, we need to put user provided `from` and `to` to the table `RandRange` \(as defined above\); for responses, we need to extract the `result` from the table `RandRangeRes`. 60 | 61 | Therefore, let's define 2 functions, `req` and `res`, that does the work: 62 | 63 | {% code-tabs %} 64 | {% code-tabs-item title="js/rand\_range.ts" %} 65 | ```typescript 66 | function req( 67 | from: number, 68 | to: number, 69 | ): [flatbuffers.Builder, msg.Any, flatbuffers.Offset] { 70 | // Get a builder to create a serialized buffer 71 | const builder = flatbuffers.createBuilder(); 72 | msg.RandRange.startRandRange(builder); 73 | // Put stuff inside the buffer! 74 | msg.RandRange.addFrom(builder, from); 75 | msg.RandRange.addTo(builder, to); 76 | const inner = msg.RandRange.endRandRange(builder); 77 | // We return these 3 pieces of information. 78 | // dispatch.sendSync/sendAsync will need these as arguments! 79 | // (treat such as boilerplate) 80 | return [builder, msg.Any.RandRange, inner]; 81 | } 82 | 83 | function res(baseRes: null | msg.Base): number { 84 | // Some checks 85 | assert(baseRes !== null); 86 | // Make sure we actually do get a correct response type 87 | assert(msg.Any.RandRangeRes === baseRes!.innerType()); 88 | // Create the RandRangeRes template 89 | const res = new msg.RandRangeRes(); 90 | // Deserialize! 91 | assert(baseRes!.inner(res) !== null); 92 | // Extract the result 93 | return res.result(); 94 | } 95 | ``` 96 | {% endcode-tabs-item %} 97 | {% endcode-tabs %} 98 | 99 | Great! With `req` and `res` defined, we can very easily define the actual sync/async APIs: 100 | 101 | {% code-tabs %} 102 | {% code-tabs-item title="js/rand\_range.ts" %} 103 | ```typescript 104 | // Sync 105 | export function randRangeSync(from: number, to: number): number { 106 | return res(dispatch.sendSync(...req(from, to))); 107 | } 108 | 109 | // Async 110 | export async function randRange(from: number, to: number): Promise { 111 | return res(await dispatch.sendAsync(...req(from, to))); 112 | } 113 | ``` 114 | {% endcode-tabs-item %} 115 | {% endcode-tabs %} 116 | 117 | ## Expose the Interface 118 | 119 | Go to `js/deno.ts` and add the following line: 120 | 121 | {% code-tabs %} 122 | {% code-tabs-item title="js/deno.ts" %} 123 | ```typescript 124 | export { randRangeSync, randRange } from "./rand_range"; 125 | ``` 126 | {% endcode-tabs-item %} 127 | {% endcode-tabs %} 128 | 129 | Also, go to `BUILD.gn` and add our file to `ts_sources`: 130 | 131 | {% code-tabs %} 132 | {% code-tabs-item title="BUILD.gn" %} 133 | ```text 134 | ts_sources = [ 135 | "js/assets.ts", 136 | "js/blob.ts", 137 | "js/buffer.ts", 138 | # ... 139 | "js/rand_range.ts" 140 | ] 141 | ``` 142 | {% endcode-tabs-item %} 143 | {% endcode-tabs %} 144 | 145 | ## Add Backend Code 146 | 147 | Go to `src/ops.rs`. This is the only Rust backend file we need to modify to add this functionality. 148 | 149 | Inside `ops.rs`, let's first import the utility for generating random numbers. Fortunately, Deno already includes a Rust crate called `rand` that could handle the job. See more about its API [here](https://docs.rs/rand/0.6.1/rand/). 150 | 151 | Let's import it: 152 | 153 | {% code-tabs %} 154 | {% code-tabs-item title="src/ops.rs" %} 155 | ```rust 156 | use rand::{Rng, thread_rng}; 157 | ``` 158 | {% endcode-tabs-item %} 159 | {% endcode-tabs %} 160 | 161 | Now, we can create a function \(with some boilerplate code\) that implements the random range logic: 162 | 163 | {% code-tabs %} 164 | {% code-tabs-item title="src/ops" %} 165 | ```rust 166 | fn op_rand_range( 167 | _state: &IsolateState, 168 | base: &msg::Base, 169 | data: libdeno::deno_buf, 170 | ) -> Box { 171 | assert_eq!(data.len(), 0); 172 | // Decode the message as RandRange 173 | let inner = base.inner_as_rand_range().unwrap(); 174 | // Get the command id, used to respond to async calls 175 | let cmd_id = base.cmd_id(); 176 | // Get `from` and `to` out of the buffer 177 | let from = inner.from(); 178 | let to = inner.to(); 179 | 180 | // Wrap our potentially slow code and respond code here 181 | // Based on dispatch.sendSync and dispatch.sendAsync, 182 | // base.sync() will be true or false. 183 | // If true, blocking() will spawn the task on the main thread 184 | // Else, blocking() would spawn it in the Tokio thread pool 185 | blocking(base.sync(), move || -> OpResult { 186 | // Actual random number generation code! 187 | let result = thread_rng().gen_range(from, to); 188 | 189 | // Prepare respond message serialization 190 | // Treat these as boilerplate code for now 191 | let builder = &mut FlatBufferBuilder::new(); 192 | // We want the message type to be RandRangeRes 193 | let inner = msg::RandRangeRes::create( 194 | builder, 195 | &msg::RandRangeResArgs { 196 | result, // put in our result here 197 | }, 198 | ); 199 | // Get message serialized 200 | Ok(serialize_response( 201 | cmd_id, // Used to reply to TypeScript if this is an async call 202 | builder, 203 | msg::BaseArgs { 204 | inner: Some(inner.as_union_value()), 205 | inner_type: msg::Any::RandRangeRes, 206 | ..Default::default() 207 | }, 208 | )) 209 | }) 210 | } 211 | ``` 212 | {% endcode-tabs-item %} 213 | {% endcode-tabs %} 214 | 215 | After completing our implementation, let's tell Rust when we should invoke it. 216 | 217 | Notice there is a function called `dispatch` in `src/ops.rs`. Inside, we could find a `match` statement on message types to set corresponding handlers. Let's set our function to be the handler of the `msg::Any::RandRange` type, by inserting this line: 218 | 219 | {% code-tabs %} 220 | {% code-tabs-item title="src/ops.rs" %} 221 | ```rust 222 | pub fn dispatch( 223 | // ... 224 | ) -> (bool, Box) { 225 | // ... 226 | let op_creator: OpCreator = match inner_type { 227 | msg::Any::Accept => op_accept, 228 | msg::Any::Chdir => op_chdir, 229 | // ... 230 | /* ADD THE FOLLOWING LINE! */ 231 | msg::Any::RandRange => op_rand_range, 232 | // ... 233 | _ => panic!(format!( 234 | "Unhandled message {}", 235 | msg::enum_name_any(inner_type) 236 | )), 237 | } 238 | } 239 | ``` 240 | {% endcode-tabs-item %} 241 | {% endcode-tabs %} 242 | 243 | Done! This is all the code you need to add for this call! 244 | 245 | Run `./tools/build.py` to build your project! You would now be able to call `deno.randRangeSync` and `deno.randRange` . 246 | 247 | Try it out in the terminal! 248 | 249 | ```bash 250 | $ ./target/debug/deno 251 | > deno.randRangeSync(0, 100) 252 | 96 253 | > (async () => console.log(await deno.randRange(0, 100)))() 254 | Promise {} 255 | > 74 256 | ``` 257 | 258 | Congrats! 259 | 260 | ## Add Tests 261 | 262 | Now let's add a basic test for our code. 263 | 264 | Create a file `js/rand_range_test.ts`: 265 | 266 | {% code-tabs %} 267 | {% code-tabs-item title="js/rand\_range\_ts.ts" %} 268 | ```typescript 269 | import { test, assert } from "./test_util.ts"; 270 | import * as deno from "deno"; 271 | 272 | test(function randRangeSync() { 273 | const v = deno.randRangeSync(0, 100); 274 | assert(0 <= v && v < 100); 275 | }); 276 | 277 | test(async function randRange() { 278 | const v = await deno.randRange(0, 100); 279 | assert(0 <= v && v < 100); 280 | }); 281 | ``` 282 | {% endcode-tabs-item %} 283 | {% endcode-tabs %} 284 | 285 | Include this test in `js/unit_tests.ts`: 286 | 287 | {% code-tabs %} 288 | {% code-tabs-item title="js/unit\_tests.ts" %} 289 | ```typescript 290 | // ADD THIS LINE 291 | import "./rand_range_test.ts"; 292 | ``` 293 | {% endcode-tabs-item %} 294 | {% endcode-tabs %} 295 | 296 | Now, run `./tools/test.py`. From all the tests, you will be able to find that your tests are passing: 297 | 298 | ```text 299 | test randRangeSync_permW0N0E0R0 300 | ... ok 301 | test randRange_permW0N0E0R0 302 | ... ok 303 | ``` 304 | 305 | ## Submit a PR! 306 | 307 | Let's suppose that Deno really needs this functionality and you are resolved to contribute to Deno. You can submit a PR! 308 | 309 | Remember to run the following commands to ensure your code is ready for submit! 310 | 311 | ```bash 312 | ./tools/format.py 313 | ./tools/lint.py 314 | ./tools/build.py 315 | ./tools/test.py 316 | ``` 317 | 318 | 319 | 320 | -------------------------------------------------------------------------------- /codebase-basics/infrastructure.md: -------------------------------------------------------------------------------- 1 | # Infrastructure 2 | 3 | Deno is composed of mainly 3 separate parts: 4 | 5 | ## 1. TypeScript Frontend 6 | 7 | Public interfaces, APIs and important most functionalities that do not directly require syscalls are implemented here. An automatically generated TypeScript API doc is available at [https://deno.land/typedoc](https://deno.land/typedoc) at this moment \(which might be changed in the future\). 8 | 9 | TypeScript/JavaScript is considered as the "unprivileged side" which does not, by default, have access to the file system or the network \(as they are run on V8, which is a sandboxed environment\). These are only made possible through message passing to the Rust backend, which is "privileged". 10 | 11 | Therefore, many Deno APIs \(especially file system calls\) are implement on the TypeScript end as purely creating buffers for data, sending them to the Rust backend through `libdeno` middle-end bindings, and waiting \(synchronously or asynchronously\) for the result to be sent back. 12 | 13 | ## 2. C++ libdeno Middle-end 14 | 15 | `libdeno` is a thin layer between the TypeScript frontend and Rust backend, serving to interact with V8 and expose only necessary bindings. It is implemented with C/C++. 16 | 17 | `libdeno` exposes Send and Receive message passing APIs to both TypeScript and Rust sides. It is also used to initiate V8 platforms/isolates and create/load V8 snapshot \(a data blob that represents serialized heap information. See [https://v8.dev/blog/custom-startup-snapshots](https://v8.dev/blog/custom-startup-snapshots) for details\). Worth noting, the snapshot for Deno also contains a complete TypeScript compiler. This allows much shorter compiler startup time. 18 | 19 | ## 3. Rust Backend 20 | 21 | Currently, the backend, or the "privileged side" that has file system, network and environment access, is implemented in Rust. 22 | 23 | For those who are not familiar with the language, Rust is a systems programming language developed by Mozilla, focusing on memory and concurrency safeties. It is also used in projects like [Servo](https://servo.org/). 24 | 25 | The Rust backend is migrated from Go, which served to create the original Deno prototype present in June 2018. Reasons for the switch is due to concerns about double GC. Read more in [this thread](https://github.com/denoland/deno/issues/205). 26 | 27 | -------------------------------------------------------------------------------- /codebase-basics/more-components.md: -------------------------------------------------------------------------------- 1 | # More Components 2 | 3 | In this section, we will be briefly discuss about other important components which are relied by Deno. 4 | 5 | ## V8 6 | 7 | [V8](https://v8.dev/) is a JavaScript/WebAssembly engine by Google. Written in C++, it is also used most notably in Google Chrome and Node.js. 8 | 9 | V8 does not support TypeScript. Instead, all TypeScript code you run in Deno are compiled to JavaScript by a snapshotted TS compiler, while the generated files are stored under `.deno` folder. Unless the user updates the code, only the cached JS files would be run after the initial compilation. 10 | 11 | ## FlatBuffers 12 | 13 | [Flatbuffers](https://google.github.io/flatbuffers/) is an efficient cross platform serialization library, developed by Google. Flatbuffers allows messages to be passed and accessed across languages without the overhead of parsing and unpacking. 14 | 15 | In the case of Deno, FlatBuffers is used to allow intra-process message communication between the privileged and unprivileged sides. Many public Deno APIs internally create buffers that contain serialized data on the TypeScript frontend, and make these buffer available for Rust so that the Rust end could process the request. After completing or scheduling the requests, the Rust end similarly creates buffers for serialized results and sends them back to TypeScript, of which it deserializes them using files generated by FlatBuffers compiler. 16 | 17 | In comparison to Node, which creates many V8 bindings for each privileged calls, Deno with FlatBuffers only need to expose message send and receive methods on TypeScript and Rust. This makes adding a new call much easier, and avoids direct interaction with V8. 18 | 19 | Flatbuffers is introduced to replace [Protocol Buffers](https://developers.google.com/protocol-buffers/) in the Go prototype to avoid overhead. See [this thread](https://github.com/denoland/deno/issues/269) for more information. 20 | 21 | ### Example: Passing readFile Data Through FlatBuffers 22 | 23 | TypeScript side: 24 | 25 | {% code-tabs %} 26 | {% code-tabs-item title="read\_file.ts" %} 27 | ```typescript 28 | import * as msg from "gen/msg_generated"; 29 | import * as flatbuffers from "./flatbuffers"; 30 | // dispatch is used to dispatch a message to Rust 31 | import * as dispatch from "./dispatch"; 32 | 33 | export async function readFile(filename: string): Promise { 34 | return res(await dispatch.sendAsync(...req(filename))); 35 | } 36 | 37 | function req( 38 | filename: string 39 | ): [flatbuffers.Builder, msg.Any, flatbuffers.Offset] { 40 | // Builder for serializing a message 41 | const builder = flatbuffers.createBuilder(); 42 | const filename_ = builder.createString(filename); 43 | msg.ReadFile.startReadFile(builder); 44 | // Filename is placed into the underlying ArrayBuffer 45 | msg.ReadFile.addFilename(builder, filename_); 46 | const inner = msg.ReadFile.endReadFile(builder); 47 | return [builder, msg.Any.ReadFile, inner]; 48 | } 49 | 50 | function res(baseRes: null | msg.Base): Uint8Array { 51 | // ... 52 | const inner = new msg.ReadFileRes(); 53 | // ... 54 | // Taking data out of FlatBuffers 55 | const dataArray = inner.dataArray(); 56 | // ... 57 | return new Uint8Array(dataArray!); 58 | } 59 | ``` 60 | {% endcode-tabs-item %} 61 | {% endcode-tabs %} 62 | 63 | Rust side: 64 | 65 | {% code-tabs %} 66 | {% code-tabs-item title="ops.rs" %} 67 | ```rust 68 | fn op_read_file( 69 | _config: &IsolateState, 70 | base: &msg::Base, 71 | data: libdeno::deno_buf, 72 | ) -> Box { 73 | // ... 74 | let inner = base.inner_as_read_file().unwrap(); 75 | let cmd_id = base.cmd_id(); 76 | // Extract filename from serialized buffer 77 | let filename = PathBuf::from(inner.filename().unwrap()); 78 | // ... 79 | blocking(base.sync(), move || { 80 | // Actual fs operation happens here! 81 | let vec = fs::read(&filename)?; 82 | // Serialize the output and send back to TypeScript 83 | let builder = &mut FlatBufferBuilder::new(); 84 | let data_off = builder.create_vector(vec.as_slice()); 85 | let inner = msg::ReadFileRes::create( 86 | builder, 87 | &msg::ReadFileResArgs { 88 | data: Some(data_off), 89 | }, 90 | ); 91 | Ok(serialize_response( 92 | cmd_id, 93 | builder, 94 | msg::BaseArgs { 95 | inner: Some(inner.as_union_value()), 96 | inner_type: msg::Any::ReadFileRes, 97 | ..Default::default() 98 | }, 99 | )) 100 | }) 101 | } 102 | ``` 103 | {% endcode-tabs-item %} 104 | {% endcode-tabs %} 105 | 106 | ## Tokio 107 | 108 | [Tokio](https://tokio.rs/) is an asynchronous runtime for Rust. It is used for creating and handling events. It allows Deno to spawn tasks in a internal thread pool and receive notifications to process the output after the task is complete. 109 | 110 | Tokio relies on Rust `Future`, which is a construct similar to JavaScript Promises. 111 | 112 | ### Example: Async Task Spawning 113 | 114 | In the example of `readFile` above, there is a `blocking` function. It is used to decide whether a task should be spawned on the main thread or forwarded to the Tokio thread pool. 115 | 116 | {% code-tabs %} 117 | {% code-tabs-item title="ops.rs" %} 118 | ```rust 119 | fn blocking(is_sync: bool, f: F) -> Box 120 | where 121 | F: 'static + Send + FnOnce() -> DenoResult, 122 | { 123 | if is_sync { 124 | // Runs task on the main thread 125 | Box::new(futures::future::result(f())) 126 | } else { 127 | // Forward the task to Tokio 128 | Box::new(tokio_util::poll_fn(move || convert_blocking(f))) 129 | } 130 | } 131 | ``` 132 | {% endcode-tabs-item %} 133 | {% endcode-tabs %} 134 | 135 | 136 | 137 | -------------------------------------------------------------------------------- /codebase-basics/repo-structure.md: -------------------------------------------------------------------------------- 1 | # Repo Structure 2 | 3 | In this section, we will explore the current Deno repository structure, such that you won't feel lost. 4 | 5 | Here is the link to the repo: [https://github.com/denoland/deno](https://github.com/denoland/deno) 6 | 7 | We will list some of the important directories/files you might need to interact with, and briefly introduce each of them: 8 | 9 | ```text 10 | # DIRECTORIES 11 | build_extra/: Some customized rules of building Deno 12 | js/: TypeScript Frontend 13 | libdeno/: C/C++ Middle-end 14 | src/: Rust Backend 15 | tests/: Integration Tests 16 | third_party/: Third party code 17 | tools/: Deno tools for testing and building 18 | 19 | # FILES 20 | BUILD.gn: Main Deno build rules 21 | Cargo.toml: Dependency info for Rust end 22 | package.json: Node dependencies (mostly for TypeScript compiler and Rollup) 23 | build.rs: Cargo build rules 24 | ``` 25 | 26 | ## build\_extra/ 27 | 28 | Some customized rules for building Deno, consisting of rules for FlatBuffers and Rust crates at the moment. 29 | 30 | One important file you might want to pay attention to is `build_extra/rust/BUILD.gn`. When adding a new Rust crate in `third_party/`, we need to add an entry into this file. For example, this is the rule for building a crate called `time`: 31 | 32 | {% code-tabs %} 33 | {% code-tabs-item title="build\_extra/rust/BUILD.gn" %} 34 | ``` 35 | rust_crate("time") { 36 | # Where is this crate? 37 | source_root = "$registry_github/time-0.1.40/src/lib.rs" 38 | # What are its dependencies? 39 | extern = [ 40 | ":libc", 41 | ":winapi", 42 | ] 43 | } 44 | ``` 45 | {% endcode-tabs-item %} 46 | {% endcode-tabs %} 47 | 48 | \(`rust_crate` GN template is defined in `build_extra/rust/rust.gni`\) 49 | 50 | ## js/ 51 | 52 | This is where code for TypeScript frontend is located. Most APIs are defined in its own file, accompanied with a test file. For example, `readFile` is defined in `read_file.ts`, while its unit tests are in `read_file_test.ts`. 53 | 54 | Besides files for API, here are some other special files: 55 | 56 | 1. `main.ts`: `denoMain` is defined in this file, which is invoked by Rust backend as the entry for bootstrapping TypeScript code. 57 | 2. `globals.ts`: All globals that do not need imports are attached here. 58 | 3. `deno.ts`: All `deno` namespace APIs are exported here. 59 | 4. `libdeno.ts`: Type definition for customized V8 API exposed to TypeScript. 60 | 5. `unit_tests.ts`: All unit tests are imported here and executed. 61 | 6. `compiler.ts`: Customized TypeScript compilation logic for Deno. 62 | 63 | ## libdeno/ 64 | 65 | The C/C++ libdeno middle-end lives here. 66 | 67 | 1. `api.cc`: libdeno APIs that are exposed and callable from the Rust side 68 | 2. `binding.cc`: code that handles interaction with V8 and where V8 C++ bindings are added. These APIs are exposed Rust and TypeScript side both. 69 | 3. `snapshot_creator.cc`: logic for creating V8 snapshot during build. 70 | 71 | ## src/ 72 | 73 | The Rust backend lives here. 74 | 75 | 1. `libdeno.rs`: exposes libdeno APIs from `libdeno/api.cc` to Rust 76 | 2. `isolate.rs`: extracts V8 isolate \(an isolated instance of V8 engine\) and event loop creation logic using libdeno APIs 77 | 3. `deno_dir.rs`: contains logic for Deno module file resolution and caching 78 | 4. `ops.rs`: where each Deno Op \(operation\) is handled. This is where the actual file system and network accesses are done. 79 | 5. `main.rs`: contains main function for Rust. This is the **ACTUAL** entry point when you run the Deno binary. 80 | 6. `msg.fbs`: FlatBuffers definition file for message structure, used for message passing. Modify this file when you want to add or modify a message structure. 81 | 82 | ## tests/ 83 | 84 | Integration tests for Deno. 85 | 86 | `*.js`or `*.ts` files define the code to run for each test. `*.test` defined how should this test be executed. Usually, `*.out` defines the expected output of a test \(the actual name is specified in `*.test` files\) 87 | 88 | See [tests/README.md](https://github.com/denoland/deno/blob/master/tests/README.md) for more details. 89 | 90 | ## third\_party/ 91 | 92 | Third party code for Deno. Contains actual node modules, v8, Rust crate code and so on. It is placed in [https://github.com/denoland/deno\_third\_party](https://github.com/denoland/deno_third_party). 93 | 94 | ## tools/ 95 | 96 | Mostly Python scripts for a range of purposes. 97 | 98 | 1. `build.py`: Builds Deno 99 | 2. `setup.py`: Setup and necessary code fetching 100 | 3. `format.py`: Code formatting 101 | 4. `lint.py`: Code linting 102 | 5. `sync_third_party.py`: Sync third\_party code after user modifies `package.json`, `Cargo.toml` , etc. 103 | 104 | ## BUILD.gn 105 | 106 | Contains the most important Deno build logic. We will visit this file later in the example of adding a new Deno call. 107 | 108 | ## build.rs 109 | 110 | Defines `cargo build` logic. Internally invokes `BUILD.gn`. 111 | 112 | -------------------------------------------------------------------------------- /contributing-guide.md: -------------------------------------------------------------------------------- 1 | # Contribution Guide 2 | 3 | ## Getting Started 4 | 5 | Before reading this document, **please check out the official** [CONTRIBUTING.md](https://github.com/denoland/deno/blob/master/.github/CONTRIBUTING.md) **by Deno team**. This document aims to be a complement to the above guidelines. 6 | 7 | ## Test Release Build 8 | 9 | To run tests on release \(instead of debug\) build, ensure you have built the release version, and 10 | 11 | ```bash 12 | ./tools/test.py target/release 13 | ``` 14 | 15 | ## Adding a New Third Party Package 16 | 17 | Based on the type of the package, you might want to ensure the package is correctly recorded in either `package.json` or `Cargo.toml` . After the third party package is added in `third_party/`, submit a PR to [deno\_third\_party](https://github.com/denoland/deno_third_party). Once the third party PR is present, you will be able to submit PRs in deno repo that references to the corresponding commit and access the packages in CI. 18 | 19 | For adding a new Rust crate, you might want to look into `build_extra/rust/BUILD.gn` and adding a new `rust_crate` entry there. You might also need to insert the entry in [main\_extern](https://github.com/denoland/deno/blob/73fb98ce70b327d1236b9540f3839a89c5d22ef0/BUILD.gn#L20-L48) inside of `BUILD.gn`. 20 | 21 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /installing-deno.md: -------------------------------------------------------------------------------- 1 | # Installing Deno 2 | 3 | ## Get Deno Release Binary 4 | 5 | Deno is still at very early development stage. The current releases are 0.2.x, which are defined to be "mildly usable" and is mostly for developer preview. 6 | 7 | To install Deno release binary, run the following commands on \*nix 8 | 9 | ```bash 10 | curl -L https://deno.land/x/install/install.py | python 11 | export PATH=$HOME/.deno/bin:$PATH 12 | ``` 13 | 14 | On windows, you might want to install through Powershell: 15 | 16 | ```text 17 | iex (iwr https://deno.land/x/install/install.ps1) 18 | ``` 19 | 20 | Deno is installed through scripts from [deno\_install](https://github.com/denoland/deno_install). If you encountered any installation problems, submit an issue there. 21 | 22 | ## Compile Deno From Source 23 | 24 | If you are interested in contributing to Deno, you might want to compile Deno from source yourself. 25 | 26 | Run the following commands to get make a debug build 27 | 28 | ```bash 29 | # Fetch deps. 30 | git clone --recurse-submodules https://github.com/denoland/deno.git 31 | cd deno 32 | # Setup and download important third party tools 33 | ./tools/setup.py 34 | # Build. 35 | ./tools/build.py 36 | ``` 37 | 38 | {% hint style="info" %} 39 | Notice that in some countries, domains, such as google.com, are blocked from access. You might want to use a VPN or alternative tools when running `./tools/setup.py`. 40 | {% endhint %} 41 | 42 | Deno is built with GN and Ninja, tools that are used also by the Chromium team. The built files will be located at `target/debug/` 43 | 44 | Alternatively, to create a release build for evaluation, set DENO\_BUILD\_MODE to release on build: 45 | 46 | ```bash 47 | DENO_BUILD_MODE=release ./tools/build.py 48 | ``` 49 | 50 | The built files will be located at `target/release/` 51 | 52 | Deno also comes with Cargo build support: 53 | 54 | ```bash 55 | cargo build 56 | ``` 57 | 58 | --------------------------------------------------------------------------------