├── .gitignore ├── Cargo.toml ├── LICENSE ├── README.md ├── src ├── default_macros.rs ├── lib.rs ├── memberfn.rs ├── mmap_allocator.rs ├── net_stream.rs ├── processorext.rs ├── processorimpl.rs ├── protocol.rs ├── publisherext.rs ├── publisherimpl.rs ├── reactive.rs ├── reactor.rs ├── scheduler.rs ├── sendable.rs └── subscriber.rs └── test └── test.rs /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled files 2 | *.o 3 | *.so 4 | *.rlib 5 | *.dll 6 | 7 | # Executables 8 | *.exe 9 | 10 | # Generated by Cargo 11 | /target/ 12 | 13 | Cargo.lock 14 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | 3 | name = "rx" 4 | version = "0.0.2" 5 | authors = ["Rick Richardson "] 6 | 7 | [dependencies] 8 | log="*" 9 | time="*" 10 | quickcheck="*" 11 | regex="*" 12 | 13 | [dependencies.nix] 14 | git = "https://github.com/carllerche/nix-rust" 15 | 16 | [dependencies.mio] 17 | 18 | git = "https://github.com/carllerche/mio" 19 | 20 | [dependencies.iobuf] 21 | 22 | git = "https://github.com/cgaebel/iobuf" 23 | 24 | [dependencies.lazy_static] 25 | 26 | git = "https://github.com/Kimundi/lazy-static.rs" 27 | 28 | [[test]] 29 | 30 | name = "test" 31 | path = "test/test.rs" 32 | 33 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 ReactiveX 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | This is an implementation of reactive streams, which, at the high level, is patterned off of the interfaces and 3 | protocols defined in http://reactive-streams.org. 4 | 5 | It also draws a lot of inspiration from clojures Transducers, in that it attempts to decouple the functions doing the 6 | work from their plumbing. It does this with Strategies. These effect how a publisher/processor will consume and output 7 | data. 8 | 9 | The high level API built on top of the interfaces (the customer facing portions) draw inspiration more from Elm than the java based Rx libraries. 10 | 11 | The core types are as follows: 12 | 13 | ``` 14 | Producer 15 | ``` 16 | 17 | Producer is where most of the action happens. It generates data and supplies a datum at a time to its subscribers via 18 | the Subscriber::on_next method. If it has multiple subscribers, how it supplies each subscriber with data is up to the 19 | Strategy. It also limits its output using a pushback mechanism by being supplied a request amount by the subscriber 20 | (via its subscription). The hope is that a producer can be as simple as a proc, relying on the strategy and the 21 | subscription objects to do most of its heavy lifting, while the proc does only data transformation/generation. 22 | 23 | ``` 24 | OutputStrategy 25 | ``` 26 | 27 | There are many ways a producer might distribute data. When request limiting is present, there are a few options a 28 | strategy might take: 29 | 1. Keep all subscribers in-sync by broadcasting to all only when they all have remaining requests. 30 | 2. Eagerly send to any subscribers with available requests, discarding the messages to those without. 31 | 3. Round-Robin across subscribers with available requests. 32 | 4. Keep a queue for each subscriber which is currently without requests, in an attempt to keep all subscribers in sync. 33 | 34 | ``` 35 | InputStrategy<...> 36 | ``` 37 | 38 | An input Strategy is what copes with various types of input, Iterators, data structures, multiple subscriptions, etc. It 39 | can choose to execute one when it receives the other, or selects, or anything, really. 40 | 41 | 42 | ``` 43 | Subscriber 44 | ``` 45 | 46 | The subscriber subscribes to a Producer. Its only responsibility is to supply to the producer a reqeust number (via the subscription 47 | object) which acts as a backpressure mechanism. 48 | 49 | ``` 50 | Processor : Producer + Subscriber 51 | ``` 52 | 53 | A processor is simply a Producer + a Subscriber. It is a link in the execution chain. Among its other responsibilities, 54 | it must pass down request backpressure values from its subscribers. It would also propagate errors downstream, and 55 | propagate unsubscribes upstream. 56 | 57 | 58 | ## LICENSE ## 59 | 60 | The MIT License (MIT) 61 | 62 | Copyright (c) 2015 ReactiveX 63 | 64 | Permission is hereby granted, free of charge, to any person obtaining a copy 65 | of this software and associated documentation files (the "Software"), to deal 66 | in the Software without restriction, including without limitation the rights 67 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 68 | copies of the Software, and to permit persons to whom the Software is 69 | furnished to do so, subject to the following conditions: 70 | 71 | The above copyright notice and this permission notice shall be included in all 72 | copies or substantial portions of the Software. 73 | 74 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 75 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 76 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 77 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 78 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 79 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 80 | SOFTWARE. 81 | -------------------------------------------------------------------------------- /src/default_macros.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | 6 | 7 | #[macro_export] 8 | macro_rules! default_pass_error( 9 | () => ( 10 | fn on_error(&mut self, err: &str) { 11 | error!("Error: {:?}", err); 12 | match self.subscriber.as_mut() { 13 | Some(s) => s.on_error(err), 14 | None => {panic!("on_error called but I don't have a subscriber")} 15 | } 16 | } 17 | ) 18 | ); 19 | 20 | #[macro_export] 21 | macro_rules! default_pass_complete ( 22 | () => ( 23 | fn on_complete(&mut self, force: bool) { 24 | match self.subscriber.as_mut() { 25 | Some(s) => s.on_complete(force), 26 | None => panic!("on_complete called but I don't have a subscriber") 27 | } 28 | } 29 | ) 30 | ); 31 | 32 | #[macro_export] 33 | macro_rules! default_pass_subscribe ( 34 | () => ( 35 | fn on_subscribe(&mut self, index: usize) { 36 | self.index = Some(index); 37 | } 38 | ) 39 | ); 40 | 41 | #[macro_export] 42 | macro_rules! protocol_size { 43 | ($($name:ident = $value:expr),+) => {$( 44 | #[derive(Debug)] 45 | struct $name; 46 | impl HasSize for $name { 47 | #[inline(always)] 48 | fn size() -> u32 { $value } 49 | } 50 | )*} 51 | } 52 | 53 | -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | //! # Reactive Rust 6 | //! A reactive streams library in the vein of [reactive-streams.org](http://reactive-streams.org) 7 | //! with inspiration taken from [Elm](http://elm-lang.org) 8 | //! Reactive streams are a mechanism for chaining computation in an event-driven way. 9 | //! If it helps, think of them as Iterators but Push instead of Pull 10 | //! 11 | //! This library is still early alpha, and surely has bugs. 12 | //! 13 | //! ## Goals 14 | //! 15 | //! * Speed - Rust is a systems library. Convieniences 16 | //! are foresaken for performance where necessary 17 | //! * 0-Copy - The core IO mechanisms pass around refcounted handles to buffers, 18 | //! data passing is typically move where possible, clone/copy is rarely required 19 | //! * Extensibility - It is assumed that the user of the API will want their own 20 | //! objects to be easily made reactive. 21 | //! * Familiarity - The "builder" API has a similar look and feel as the 22 | //! stdlibs Iterator API, but reactive streams themselves are much more 23 | //! loosely coupled 24 | //! 25 | //! The core abstraction of Reactive Streams is a [Publisher](reactive/trait.Publisher.html) / 26 | //! [Subscriber](reactive/trait.Subscriber) 27 | //! 28 | //! Objects which are both a Publisher and a Subscriber are a [Processor](reactive/trait.Processor.html) 29 | //! 30 | //! 31 | //! ## Example 32 | //! 33 | //! ```rust 34 | //! ``` 35 | #![doc(html_root_url = "http://www.rust-ci.org/rrichardson/reactive/doc/reactive/")] 36 | #![unstable] 37 | #![crate_id = "rx"] 38 | #![crate_type="lib"] 39 | 40 | #![allow(unstable)] 41 | #![feature(slicing_syntax)] 42 | #![feature(unboxed_closures)] 43 | #![feature(unsafe_destructor)] 44 | #![feature(libc)] 45 | #![feature(core)] 46 | #![feature(io)] 47 | #![feature(collections)] 48 | 49 | extern crate core; 50 | extern crate mio; 51 | extern crate collections; 52 | extern crate iobuf; 53 | extern crate time; 54 | extern crate rand; 55 | extern crate nix; 56 | extern crate libc; 57 | extern crate quickcheck; 58 | 59 | #[macro_use] 60 | extern crate log; 61 | 62 | #[macro_use] 63 | extern crate lazy_static; 64 | 65 | #[macro_use] 66 | pub mod default_macros; 67 | pub mod subscriber; 68 | pub mod reactive; 69 | pub mod reactor; 70 | pub mod net_stream; 71 | pub mod sendable; 72 | pub mod mmap_allocator; 73 | pub mod scheduler; 74 | #[macro_use] 75 | pub mod protocol; 76 | mod processorimpl; 77 | mod publisherimpl; 78 | mod processorext; 79 | mod publisherext; 80 | 81 | pub mod publisher { 82 | pub use publisherimpl::*; 83 | pub use publisherext::*; 84 | } 85 | 86 | pub mod processor { 87 | pub use processorimpl::*; 88 | pub use processorext::*; 89 | } 90 | 91 | #[test] 92 | fn main() { 93 | use publisher::{IterPublisher, Coupler}; 94 | use processor::{Map}; 95 | use subscriber::{StdoutSubscriber, Decoupler}; 96 | use reactive::{Publisher, Subscriber}; 97 | use std::old_io::Timer; 98 | use std::time::Duration; 99 | use std::sync::mpsc::{channel}; 100 | use std::thread::Thread; 101 | let (dtx, drx) = channel(); 102 | 103 | let out = move |:| { 104 | let sub = Box::new(StdoutSubscriber::new()); 105 | let mut rec = Box::new(Coupler::new(drx)); 106 | rec.subscribe(sub); 107 | }; 108 | 109 | let gen = move |:| { 110 | let it = range(0is, 20is); 111 | let q = Box::new(Decoupler::new(dtx.clone())); 112 | let mut map1 = Box::new(Map::new(|i : isize| {i * 10})); 113 | let mut map2 = Box::new(Map::new(|i : isize| {i + 2})); 114 | let mut iter = Box::new(IterPublisher::new(it)); 115 | 116 | 117 | map2.subscribe(q); 118 | map1.subscribe(map2); 119 | iter.subscribe(map1); 120 | }; 121 | 122 | Thread::spawn(out); 123 | Thread::spawn(gen); 124 | 125 | let mut timer = Timer::new().unwrap(); 126 | timer.sleep(Duration::milliseconds(1000)); 127 | 128 | Thread::spawn(|| { 129 | let it = range(0is, 20is); 130 | let sub = Box::new(StdoutSubscriber::new()); 131 | let mut map1 = Box::new(Map::new(|i : isize| {i * 10})); 132 | let mut map2 = Box::new(Map::new(|i : isize| {i + 2})); 133 | let mut iter = Box::new(IterPublisher::new(it)); 134 | 135 | 136 | map2.subscribe(sub); 137 | map1.subscribe(map2); 138 | iter.subscribe(map1); 139 | 140 | }); 141 | 142 | 143 | timer.sleep(Duration::milliseconds(1000)); 144 | 145 | } 146 | -------------------------------------------------------------------------------- /src/memberfn.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | #![feature(overloaded_calls)] 6 | 7 | struct MemberFn0<'a, T, R> where T : 'a { 8 | fun: |&T|:'static -> R, 9 | obj: &'a T 10 | } 11 | 12 | impl<'a, T, R> MemberFn0<'a, T, R> where T : 'a { 13 | fn new(o: &'a T, f: |&T|:'static -> R) -> MemberFn0<'a, T, R> { 14 | MemberFn0 { 15 | fun: f, 16 | obj: o 17 | } 18 | } 19 | 20 | fn call(&mut self) -> R { 21 | (self.fun)(self.obj) 22 | } 23 | } 24 | 25 | struct MemberFn1<'a, T, R, A0> where T : 'a { 26 | fun: |&T, A0|:'static -> R, 27 | obj: &'a T 28 | } 29 | 30 | impl<'a, T, R, A0> MemberFn1<'a, T, R, A0> where T : 'a { 31 | fn new(o: &'a T, f: |&T, A0|:'static -> R) -> MemberFn1<'a, T, R, A0> { 32 | MemberFn1 { 33 | fun: f, 34 | obj: o 35 | } 36 | } 37 | 38 | fn call(&mut self, arg: A0) -> R { 39 | (self.fun)(self.obj, arg) 40 | } 41 | } 42 | 43 | struct MemberFn2<'a, T, R, A0, A1> where T : 'a { 44 | fun: |&T, A0, A1|:'static -> R, 45 | obj: &'a T 46 | } 47 | 48 | impl<'a, T, R, A0, A1> MemberFn2<'a, T, R, A0, A1> where T : 'a { 49 | fn new(o: &'a T, f: |&T, A0, A1|:'static -> R) -> MemberFn2<'a, T, R, A0, A1> { 50 | MemberFn2 { 51 | fun: f, 52 | obj: o 53 | } 54 | } 55 | 56 | fn call(&mut self, arg0: A0, arg1: A1) -> R { 57 | (self.fun)(self.obj, arg0, arg1) 58 | } 59 | } 60 | 61 | // there should be an Impl for FnMut, but it is currently broken, so we'll have to invoke with 62 | // .call(...) 63 | 64 | struct Foo { 65 | x: usize 66 | } 67 | 68 | impl Foo { 69 | 70 | fn winning(&self, a: usize) -> usize { 71 | a + self.x 72 | } 73 | 74 | } 75 | 76 | #[test] 77 | fn main() { 78 | let bar = Foo { x : 10 }; 79 | 80 | let mut binding = MemberFn1::new(&bar, |f: &Foo, a: usize| { f.winning(a) } ); 81 | println!("result: {:?}", binding.call(10)); 82 | } 83 | -------------------------------------------------------------------------------- /src/mmap_allocator.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | 6 | use std::mem; 7 | use std::ops::Drop; 8 | use std::old_io::FilePermission; 9 | use std::fmt; 10 | use std::sync::atomic::{Ordering, AtomicUint}; 11 | use std::path::Path; 12 | use nix::sys::{mman, stat}; 13 | use nix::{fcntl, unistd}; 14 | use nix::sys::stat::{Mode, S_IRWXU, S_IRWXG, S_IRWXO }; 15 | use libc::{c_int, c_void}; 16 | 17 | //when rust has allocator traits, this won't be necessary 18 | use iobuf::Allocator; 19 | 20 | lazy_static! { 21 | static ref ALIGN: usize = mem::size_of_val(&0us); 22 | static ref MAGIC: usize = 0x42424242; 23 | static ref MODE_ALL : Mode = S_IRWXU | S_IRWXG | S_IRWXO; 24 | } 25 | 26 | impl Allocator for MappedRegion { 27 | fn allocate(&self, size: usize, align: usize) -> *mut u8 { 28 | self.allocate(size, align) 29 | } 30 | 31 | fn deallocate(&self, _: *mut u8, _: usize, _: usize) { 32 | // NO DISASSEMBLE! JOHNNY FIVE ALIVE! 33 | } 34 | } 35 | 36 | 37 | pub struct MappedRegion { 38 | addr: *const c_void, 39 | total_size: u64, 40 | fd: c_int, 41 | header: *mut MMapHeader, 42 | count: AtomicUint 43 | } 44 | 45 | unsafe impl Send for MappedRegion {} 46 | unsafe impl Sync for MappedRegion {} 47 | 48 | pub struct MMapHeader { 49 | magic: usize, 50 | current: AtomicUint, 51 | total_size: u64 52 | } 53 | 54 | impl fmt::Show for MMapHeader { 55 | fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { 56 | write!(f, "MMapHeader[current: {:?}, total_size: {:?}]", self.current.load(Ordering::Relaxed), self.total_size) 57 | } 58 | } 59 | 60 | impl MappedRegion { 61 | pub fn new(basepath: &str, total_size: u64) -> Result { 62 | let fpath = Path::new(basepath); 63 | let fd = fcntl::open(&fpath, fcntl::O_CREAT | fcntl::O_RDWR, *MODE_ALL).unwrap(); 64 | unistd::ftruncate(fd, total_size as i64).unwrap(); 65 | let ptr = mman::mmap(0 as *mut c_void, total_size, mman::PROT_READ | mman::PROT_WRITE, mman::MAP_SHARED, fd, 0).unwrap(); 66 | match mman::madvise(ptr as *const c_void, total_size, mman::MADV_SEQUENTIAL) { 67 | Ok(..) => {}, 68 | Err(e) => { error!("Failed to advise mmap to alloc {:?} bytes at {:?} - error: {:?}", total_size, ptr, e); 69 | mman::munmap(ptr, total_size).unwrap(); 70 | unistd::close(fd).unwrap(); } 71 | } 72 | let offset = ((mem::size_of::() - 1) | (*ALIGN -1)) + 1; 73 | let headerptr : *mut MMapHeader = unsafe { mem::transmute(ptr) }; 74 | unsafe { *headerptr = MMapHeader {magic: *MAGIC, current: AtomicUint::new(offset), total_size: total_size } }; 75 | Ok(MappedRegion{addr : ptr as *const c_void, 76 | total_size : total_size, 77 | fd: fd, 78 | count: AtomicUint::new(0), 79 | header: headerptr}) 80 | } 81 | 82 | pub fn load(basepath: &str) -> Result { 83 | let fpath = Path::new(basepath); 84 | let fd = fcntl::open(&fpath, fcntl::O_CREAT | fcntl::O_RDWR, *MODE_ALL).unwrap(); 85 | let fstat = stat::fstat(fd).unwrap(); 86 | let ptr = mman::mmap(0 as *mut c_void, fstat.st_size as u64, mman::PROT_READ | mman::PROT_WRITE, mman::MAP_SHARED, fd, 0).unwrap(); 87 | let headerptr : *mut MMapHeader = unsafe { mem::transmute(ptr) }; 88 | let ref mut header = unsafe { &(*headerptr) }; 89 | debug!("Mmap File loaded: {:?}", header); 90 | 91 | if header.magic != *MAGIC && 92 | header.total_size != fstat.st_size as u64 { // quick sanity check 93 | mman::munmap(ptr, fstat.st_size as u64 ).unwrap(); 94 | unistd::close(fd).unwrap(); 95 | Err(format!("Failed to load data file, {:?}. Reported sizes do not match: file size: {:?}, header size {:?}", basepath, fstat.st_size, header.total_size)) 96 | } 97 | else { 98 | Ok(MappedRegion{addr : ptr as *const c_void, 99 | total_size : header.total_size, 100 | fd: fd, 101 | count: AtomicUint::new(0), 102 | header: headerptr }) 103 | } 104 | } 105 | /* 106 | pub fn alt_new(basepath: &str, count: usize, size: size_t, align: usize) -> Result { 107 | let fpath = Path::new(format!("{:?}-{:?}", basepath, count)); 108 | fcntl::open(&fpath, fcntl::O_CREAT | fcntl::O_RDWR, FilePermission::all()).map( 109 | |fd| Destructed::new(fd, Box::new(|f| {unistd::close(*f);})).and_then( 110 | |fd| { unistd::ftruncate(*fd.inner(), size as i64).map(|_| fd )}).and_then( 111 | |fd| { mman::mmap(0 as *mut c_void, size, mman::PROT_READ | mman::PROT_WRITE, mman::MAP_SHARED, *fd.inner(), 0).map( 112 | |ptr| (fd, Destructed::new(ptr, Box::new(|p| { mman::munmap(*p, size);})) )}).and_then( 113 | |(fd,ptr)| { mman::madvise(*ptr.inner() as *const c_void, size, mman::MADV_SEQUENTIAL).map(|_| (fd, ptr)) }).map( 114 | |(fd,ptr)| { ptr.release(); fd.release(); 115 | MappedRegion{addr : *ptr.inner() as *const c_void, total_size : size, current: AtomicUint::new(*ptr.inner() as usize), fd: *fd.inner(), align: align} 116 | }).map_err(|e| e.desc()) 117 | } 118 | */ 119 | 120 | /// returns the number of allocations since this object was created 121 | /// for statistical purposes, does not start from beginning of the 122 | /// journal file, only the instantiation of MappedRegion by load or new 123 | pub fn num_allocated(&self) -> usize { 124 | self.count.load(Ordering::Relaxed) 125 | } 126 | //current is guaranteed to start off as aligned, so we'll ensure it stays that way 127 | //get the current value to return, then calculate the next value to be supplied by 128 | //the next call to this function 129 | pub fn allocate(&self, size: usize, align: usize) -> *mut u8 { 130 | if align != *ALIGN { panic!("User requested alignment of {:?} but we only have {:?}", align, *ALIGN); } 131 | let ref mut header = unsafe { &(*self.header) }; 132 | let mut offset : usize; 133 | loop { // attempt to fetch the next available slot, if it is taken, as evidenced by the CAS, then try again 134 | offset = header.current.load(Ordering::SeqCst); 135 | let newval = (((offset + size) - 1) | (*ALIGN - 1)) + 1; 136 | let oldval = header.current.compare_and_swap(offset, newval, Ordering::SeqCst); 137 | if offset == oldval { break } 138 | } 139 | if offset > self.total_size as usize { 140 | error!("we have gone past our allocated file space for the allocator : offset {:?}", offset); 141 | return unsafe { mem::transmute::(0)} 142 | } 143 | self.count.fetch_add(1, Ordering::SeqCst); 144 | unsafe { mem::transmute((self.addr as *mut u8).offset(offset as isize)) } 145 | } 146 | } 147 | 148 | impl Drop for MappedRegion { 149 | fn drop(&mut self) { 150 | if mman::msync(self.addr as *const c_void, self.total_size, mman::MS_SYNC).is_err() {} 151 | if mman::munmap(self.addr as *mut c_void, self.total_size).is_err() {} 152 | if unistd::close(self.fd).is_err() {} 153 | } 154 | } 155 | 156 | #[cfg(test)] 157 | 158 | mod test { 159 | 160 | use std::old_io::fs::mkdir_recursive; 161 | use super::MappedRegion; 162 | use std::path::posix::Path; 163 | use std::old_io::FilePermission; 164 | use std::mem; 165 | use super::ALIGN; 166 | struct TestObject { 167 | v1 : isize, 168 | v2 : isize, 169 | v3 : isize, 170 | v4 : isize, 171 | v5 : isize, 172 | v6 : u16, 173 | v7 : u8, 174 | } 175 | 176 | 177 | 178 | #[test] 179 | fn mmap_allocator_basic() { 180 | use std::old_io::fs; 181 | let two_jigabytes = 1024 * 1024 * 1024 * 2; 182 | 183 | mkdir_recursive(&"target/data".parse().unwrap(), FilePermission::from_bits(0o775).unwrap()); 184 | println!("TestObject is {:?} bytes or {:?} rounded", mem::size_of::(),((mem::size_of::() - 1) | (*ALIGN - 1)) + 1); 185 | { 186 | let region = MappedRegion::new("./target/data/test_alloc.db", two_jigabytes ).unwrap(); 187 | 188 | for i in range(0, 100is) { 189 | let foo : *mut TestObject = unsafe { mem::transmute(region.allocate(mem::size_of::(), 8)) }; 190 | unsafe { *foo = TestObject { v1: i, v2: i, v3: i, v4: i, v5: i, v6: i as u16, v7: i as u8 } }; 191 | } 192 | } 193 | 194 | { 195 | let region = MappedRegion::load("./target/data/test_alloc.db").unwrap(); 196 | for i in range(0, 100is) { 197 | let foo : *mut TestObject = unsafe { mem::transmute(region.allocate(mem::size_of::(), 8)) }; 198 | unsafe { *foo = TestObject { v1: i*i, v2: i*i, v3: i*i, v4: i*i, v5: i*i, v6: (i*i) as u16, v7: (i*i) as u8 } }; 199 | } 200 | assert!(region.total_size == two_jigabytes); 201 | } 202 | fs::unlink(&Path::new("./target/data/test_alloc.db")); 203 | } 204 | 205 | #[test] 206 | fn mmap_allocator_boundaries() { 207 | use std::old_io::fs; 208 | mkdir_recursive(&"target/data".parse().unwrap(), FilePermission::from_bits(0o775).unwrap()); 209 | 210 | let one_k = 1024; 211 | { 212 | let region = MappedRegion::new("./target/data/test_alloc2.db", one_k).unwrap(); 213 | for i in range(0, one_k / mem::size_of::() as u64) { 214 | let foo : *mut TestObject = unsafe { mem::transmute(region.allocate(mem::size_of::(), 8)) }; 215 | assert!(foo as usize != 0); 216 | } 217 | let foo : *mut TestObject = unsafe { mem::transmute(region.allocate(mem::size_of::(), 8)) }; 218 | assert!(foo as usize == 0); 219 | } 220 | let region = MappedRegion::load("./target/data/test_alloc2.db").unwrap(); 221 | let foo : *mut TestObject = unsafe { mem::transmute(region.allocate(mem::size_of::(), 8)) }; 222 | assert!(foo as usize == 0); 223 | 224 | fs::unlink(&Path::new("./target/data/test_alloc2.db")); 225 | } 226 | 227 | } 228 | -------------------------------------------------------------------------------- /src/net_stream.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | //! A handler for both incoming and outgoing connections, 6 | //! all incoming connections' data is sent through a standard sync_channel 7 | //! which is returned from EngineInner::new 8 | //! all outgoing connections produce their own sync channel through which 9 | //! their incoming data is received. Outgoing messages are all sent 10 | //! back through the sender provided by EngineInner::channel or via the 11 | //! StreamConneciton send_all function for Traversals 12 | 13 | use reactor::{Reactor, StreamBuf, Sender, ProtoMsg}; 14 | use mio::Token; 15 | use publisherimpl::Coupler; 16 | use reactive::{Publisher, Subscriber}; 17 | use iobuf::{AROIobuf}; 18 | 19 | use std::sync::mpsc::{Receiver,SyncSender}; 20 | use std::sync::Arc; 21 | use std::cell::RefCell; 22 | 23 | type Superbox = Arc>>; 24 | 25 | /// A Traversal style representation of a socket 26 | #[derive(Clone)] 27 | pub struct NetStream<'a, U : Send> { 28 | pub dtx: Sender, 29 | pub drx: Arc>>, 30 | pub tok: Token, 31 | } 32 | 33 | 34 | impl<'a, U : Send> NetStream<'a, U> 35 | { 36 | pub fn new(tok: Token, 37 | drx: Receiver>, 38 | dtx: Sender) -> NetStream<'a, U> { 39 | NetStream { tok: tok, drx: Arc::new(drx), dtx: dtx.clone() } 40 | } 41 | } 42 | 43 | pub struct NetStreamer<'a, U : Send> 44 | { 45 | stream: NetStream<'a, U>, 46 | subscriber: Option> + 'a >> 47 | //subscriber: Option as Publisher<'a>>::Output> + 'a >> 48 | } 49 | 50 | impl<'a, U : Send> Subscriber for NetStreamer<'a, U> 51 | { 52 | type Input = StreamBuf; 53 | fn on_next(&mut self, StreamBuf (buf, _) : StreamBuf) -> bool { 54 | 55 | //TODO better handle queue failure, maybe put the returned buf 56 | //isizeo a recovery queue 57 | match self.stream.dtx.send(StreamBuf(buf, self.stream.tok)) { 58 | Ok(()) => true, 59 | Err(_) => false 60 | } 61 | } 62 | } 63 | 64 | impl<'a, U : Send> Publisher<'a> for NetStreamer<'a, U> 65 | { 66 | type Output = ProtoMsg; 67 | 68 | //fn subscribe(&mut self, s: Box>::Output > + 'a>) { 69 | fn subscribe(&mut self, s: Box> + 'a>) { 70 | //let t: Box>::Output> + 'a> = s; 71 | self.subscriber = Some(s); 72 | self.subscriber.as_mut().unwrap().on_subscribe(0); 73 | } 74 | 75 | fn next (&mut self) -> bool { 76 | match self.subscriber.as_mut() { 77 | Some(s) => match self.stream.drx.recv() { 78 | Ok(d) => s.on_next(d), 79 | Err(..) => { s.on_complete(false); false } 80 | }, 81 | None => { error!("My subscriber went away"); false } 82 | } 83 | } 84 | } 85 | 86 | #[cfg(test)] 87 | mod test { 88 | use reactor::Reactor; 89 | use reactor::{StreamBuf, ProtoMsg, NetEngine}; 90 | use std::thread::Thread; 91 | use std::vec::Vec; 92 | use std::mem; 93 | use std::num::Int; 94 | use std::raw; 95 | use std::old_io::timer::sleep; 96 | use mio::Token; 97 | use iobuf::{Iobuf, RWIobuf, AROIobuf}; 98 | use std::time::Duration; 99 | use protocol::Protocol; 100 | use publisher::{Repeat, Coupler}; 101 | use processor::{Map, Take, DoDebug}; 102 | use subscriber::{Decoupler, Collect}; 103 | use reactive::{Publisher, Subscriber}; 104 | 105 | unsafe fn to_bytes(t: &T) -> &[u8] { 106 | mem::transmute(raw::Slice:: { 107 | data: t as *const T as *const u8, 108 | len: mem::size_of::(), 109 | }) 110 | } 111 | 112 | fn isize_to_strbuf(t: &T) -> StreamBuf { unsafe { 113 | StreamBuf (RWIobuf::from_slice_copy(to_bytes(t)).atomic_read_only().unwrap(), Token(0)) 114 | }} 115 | 116 | fn strbuf_to_isize(buf: StreamBuf) -> T { unsafe { 117 | *(mem::transmute::<*mut u8, *const T>(buf.0.ptr())) 118 | }} 119 | 120 | pub struct U64Protocol; 121 | 122 | impl Protocol for U64Protocol { 123 | type Output = u64; 124 | 125 | fn new() -> U64Protocol { 126 | U64Protocol 127 | } 128 | 129 | fn append(&mut self, buf: &AROIobuf) -> Option<(::Output, AROIobuf, u32)> { 130 | if buf.len() >= 8 { 131 | let (a, b) = buf.split_at(8).unwrap(); 132 | let val = unsafe { *(mem::transmute::<*mut u8, *const u64>(a.ptr())) }; 133 | Some((val, b, 8)) 134 | } else { 135 | None 136 | } 137 | } 138 | } 139 | 140 | #[test] 141 | fn oneway_test() { 142 | 143 | let mut ne = NetEngine::::new(); 144 | let srv_rx = ne.listen("127.0.0.1", 10000).unwrap(); 145 | let cl = { ne.connect("127.0.0.1", 10000).unwrap().clone() }; 146 | 147 | ne.timeout(Duration::milliseconds(500), Box::new(|&: el : &mut Reactor| { el.shutdown(); true})); 148 | 149 | let tok = cl.tok.clone(); 150 | let dtx = cl.dtx.clone(); 151 | 152 | Thread::spawn(move|| { 153 | let mut rep = Box::new(Repeat::new(5u64)); 154 | let mut map1 = Box::new(Map::new(|x| isize_to_strbuf(&x))); 155 | let mut map2 = Box::new(Map::new(move | StreamBuf (buf, _) | StreamBuf (buf, tok))); 156 | let mut trc = Box::new(DoDebug::new()); 157 | let mut sen = Box::new(Decoupler::new(dtx)); 158 | 159 | trc.subscribe(sen); 160 | map2.subscribe(trc); 161 | map1.subscribe(map2); 162 | rep.subscribe(map1); 163 | 164 | for i in range(0, 5) { 165 | rep.next(); 166 | } 167 | 168 | let mut v = Box::new(Vec::::new()); 169 | { 170 | let mut recv = Box::new(Coupler::new(srv_rx)); 171 | let mut take = Box::new(Take::new(5)); 172 | let mut map3 = Box::new(Map::new(| ProtoMsg (x, _) | x )); 173 | let mut coll = Box::new(Collect::new(&mut v)); 174 | 175 | map3.subscribe(coll); 176 | take.subscribe(map3); 177 | recv.subscribe(take); 178 | recv.run(); 179 | } 180 | assert_eq!(*v, vec![5,5,5,5,5]); 181 | 182 | }); 183 | 184 | ne.run(); 185 | } 186 | /* 187 | #[test] 188 | fn roundtrip_test() { 189 | 190 | let mut ne = NetEngine::new(1500, 100, 100); 191 | let srv_rx = ne.listen("127.0.0.1", 10001).unwrap(); 192 | let cl = ne.connect("127.0.0.1", 10001).unwrap(); 193 | 194 | ne.timeout(Duration::milliseconds(100), Box::new(|el| { el.shutdown(); true}); 195 | 196 | Thread::spawn(move|| { 197 | T::range::(0, 5).map(|x| isize_to_strbuf(&x)).send(cl.dtx); 198 | assert_eq!(recv(srv_rx).map(|x| strbuf_to_isize(x)).collect::>(), vec![0, 1, 2, 3, 4]); 199 | 200 | 201 | assert_eq!(recv(cl.drx).map(|x| strbuf_to_isize(x)).collect::>(), vec![0, 1, 2, 3, 4]); 202 | }).detach(); 203 | 204 | ne.run(); 205 | } 206 | */ 207 | } 208 | 209 | -------------------------------------------------------------------------------- /src/processorext.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | use reactive::{Publisher, Processor, Subscriber}; 6 | use processorimpl::*; 7 | 8 | pub trait ProcessorExt { 9 | } 10 | 11 | -------------------------------------------------------------------------------- /src/processorimpl.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | // 6 | 7 | use std::fmt::Debug; 8 | use reactive::{Publisher, Subscriber}; 9 | use sendable::Sendable; 10 | 11 | pub struct DoDebug<'a, I> where I : Debug { 12 | subscriber: Option + 'a>>, 13 | index: Option 14 | } 15 | 16 | impl<'a, I> DoDebug<'a, I> where I : Debug { 17 | 18 | pub fn new() -> DoDebug<'a, I> 19 | { 20 | DoDebug { 21 | subscriber: None, 22 | index: None 23 | } 24 | } 25 | } 26 | 27 | impl<'a, I> Publisher<'a> for DoDebug<'a, I> where I : Debug { 28 | type Output = I; 29 | fn subscribe(&mut self, s: Box + 'a>) { 30 | let t: Box+'a> = s; 31 | self.subscriber = Some(t); 32 | self.subscriber.as_mut().unwrap().on_subscribe(0); 33 | } 34 | } 35 | 36 | impl<'a, I> Subscriber for DoDebug<'a, I> where I : Debug { 37 | type Input = I; 38 | fn on_next(&mut self, t: I) -> bool { 39 | match self.subscriber.as_mut() { 40 | Some(s) => { println!("{:?}", t); s.on_next(t) }, 41 | None => {true} 42 | } 43 | } 44 | 45 | default_pass_subscribe!(); 46 | default_pass_complete!(); 47 | default_pass_error!(); 48 | } 49 | 50 | pub struct DebugWhile<'a, I, F> where I : Debug, F : Fn(&I) -> bool { 51 | fun: F, 52 | subscriber: Option + 'a>>, 53 | index: Option 54 | } 55 | 56 | 57 | impl<'a, I, F> DebugWhile<'a, I, F> where I : 'a + Debug, F : Fn(&I) -> bool{ 58 | 59 | pub fn new( f: F ) -> DebugWhile<'a, I, F> 60 | { 61 | DebugWhile { 62 | fun: f, 63 | subscriber: None, 64 | index: None 65 | } 66 | } 67 | } 68 | 69 | impl<'a, I, F> Publisher<'a> for DebugWhile<'a, I, F> where I : Debug, F : Fn(&I) -> bool { 70 | type Output = I; 71 | fn subscribe(&mut self, s: Box + 'a>) { 72 | let s: Box+'a> = s; 73 | self.subscriber = Some(s); 74 | self.subscriber.as_mut().unwrap().on_subscribe(0); 75 | } 76 | } 77 | 78 | impl<'a, I, F> Subscriber for DebugWhile<'a, I, F> where I : Debug, F : Fn(&I) -> bool { 79 | type Input = I; 80 | 81 | default_pass_subscribe!(); 82 | default_pass_complete!(); 83 | default_pass_error!(); 84 | 85 | fn on_next(&mut self, t: I) -> bool { 86 | match self.subscriber.as_mut() { 87 | Some(s) => { if (self.fun)(&t) { println!("{:?}", t) } 88 | s.on_next(t) } 89 | None => {true} 90 | } 91 | } 92 | } 93 | 94 | /// Do 95 | /// Applies a () function at each iteration to the supplied value 96 | /// which does not effect the result passed to the subscriber 97 | 98 | pub struct Do<'a, I, F> where F : Fn(&I) -> () { 99 | fun: F, 100 | subscriber: Option + 'a>>, 101 | index: Option 102 | } 103 | 104 | 105 | impl<'a, I, F> Do<'a, I, F> where F : Fn(&I) -> () { 106 | 107 | pub fn new(f: F) -> Do<'a, I, F> 108 | { 109 | Do { 110 | fun: f, 111 | subscriber: None, 112 | index: None 113 | } 114 | } 115 | } 116 | 117 | impl<'a, I, F> Publisher<'a> for Do<'a, I, F> where F : Fn(&I) -> () { 118 | type Output = I; 119 | fn subscribe(&mut self, s: Box + 'a>) { 120 | let s: Box+'a> = s; 121 | self.subscriber = Some(s); 122 | self.subscriber.as_mut().unwrap().on_subscribe(0); 123 | } 124 | } 125 | 126 | impl<'a, I, F> Subscriber for Do<'a, I, F> where F : Fn(&I) -> () { 127 | type Input = I; 128 | 129 | fn on_next(&mut self, t: I) -> bool { 130 | match self.subscriber.as_mut() { 131 | Some(s) => { (self.fun)(&t); s.on_next(t) } 132 | None => {true} 133 | } 134 | } 135 | 136 | default_pass_subscribe!(); 137 | default_pass_complete!(); 138 | default_pass_error!(); 139 | } 140 | 141 | 142 | 143 | /// Map 144 | /// 145 | /// 146 | 147 | pub struct Map<'a, I, O, F> where F : Fn(I) -> O { 148 | fun: F, 149 | subscriber: Option + 'a>>, 150 | index: Option 151 | } 152 | 153 | 154 | impl<'a, I, O, F> Map<'a, I, O, F> where F : Fn(I) -> O { 155 | 156 | pub fn new(f: F) -> Map<'a, I, O, F> 157 | { 158 | Map { 159 | fun: f, 160 | subscriber: None, 161 | index: None 162 | } 163 | } 164 | } 165 | 166 | impl<'a, I, O, F> Publisher<'a> for Map<'a, I, O, F> where F : Fn(I) -> O { 167 | type Output = O; 168 | fn subscribe(&mut self, s: Box + 'a>) { 169 | let s: Box+'a> = s; 170 | self.subscriber = Some(s); 171 | self.subscriber.as_mut().unwrap().on_subscribe(0); 172 | } 173 | } 174 | 175 | impl<'a, I, O, F> Subscriber for Map<'a, I, O, F> where F : Fn(I) -> O { 176 | type Input = I; 177 | 178 | fn on_next(&mut self, t: I) -> bool { 179 | match self.subscriber.as_mut() { 180 | Some(s) => s.on_next((self.fun)(t)), 181 | None => {true} 182 | } 183 | } 184 | 185 | default_pass_subscribe!(); 186 | default_pass_complete!(); 187 | default_pass_error!(); 188 | } 189 | 190 | 191 | pub struct MapVal1<'a, 'c, I, V, O, F> where O : 'c, V : Clone, F : Fn(I,&V) -> O { 192 | fun: F, 193 | subscriber: Option + 'a>>, 194 | index: Option, 195 | val: V 196 | } 197 | 198 | 199 | impl<'a, 'c, I, V, O, F> MapVal1<'a, 'c, I, V, O, F> where O : 'c, V : Clone, F : Fn(I,&V) -> O { 200 | 201 | pub fn new( v : V, f: F ) -> MapVal1<'a, 'c, I, V, O, F> 202 | { 203 | MapVal1 { 204 | fun: f, 205 | subscriber: None, 206 | index: None, 207 | val: v.clone() 208 | } 209 | } 210 | } 211 | 212 | impl<'a, 'c, I, V, O, F> Publisher<'a> for MapVal1<'a, 'c, I, V, O, F> where O : 'c, V : Clone, F : Fn(I,&V) -> O { 213 | type Output = O; 214 | 215 | fn subscribe(&mut self, s: Box + 'a>) { 216 | let s: Box+'a> = s; 217 | self.subscriber = Some(s); 218 | self.subscriber.as_mut().unwrap().on_subscribe(0); 219 | } 220 | } 221 | 222 | impl<'a, 'c, I, V, O, F> Subscriber for MapVal1<'a, 'c, I, V, O, F> where O : 'c, V : Clone, F : Fn(I,&V) -> O { 223 | type Input = I; 224 | 225 | default_pass_subscribe!(); 226 | default_pass_complete!(); 227 | default_pass_error!(); 228 | 229 | fn on_next(&mut self, t: I) -> bool { 230 | match self.subscriber.as_mut() { 231 | Some(s) => s.on_next((self.fun)(t, &self.val)), 232 | None => {true} 233 | } 234 | } 235 | 236 | } 237 | 238 | 239 | // 240 | // Reduce 241 | // 242 | 243 | pub struct Reduce<'a, 'c, I, V, O, F> where O : 'c, V : Copy, F : Fn(V,I) -> (V, O) { 244 | fun: F, 245 | subscriber: Option + 'a>>, 246 | index: Option, 247 | state: V 248 | } 249 | 250 | 251 | impl<'a, 'c, I, V, O, F> Reduce<'a, 'c, I, V, O, F> where O : 'c, V : Copy, F : Fn(V,I) -> (V, O) { 252 | 253 | pub fn new( initial : V, f: F ) -> Reduce<'a, 'c, I, V, O, F> 254 | { 255 | Reduce { 256 | fun: f, 257 | subscriber: None, 258 | index: None, 259 | state: initial 260 | } 261 | } 262 | } 263 | 264 | impl<'a, 'c, I, V, O, F> Publisher<'a> for Reduce<'a, 'c, I, V, O, F> where O : 'c, V : Copy, F : Fn(V,I) -> (V, O) { 265 | type Output = O; 266 | 267 | fn subscribe(&mut self, s: Box + 'a>) { 268 | let s: Box+'a> = s; 269 | self.subscriber = Some(s); 270 | self.subscriber.as_mut().unwrap().on_subscribe(0); 271 | } 272 | } 273 | 274 | impl<'a, 'c, I, V, O, F> Subscriber for Reduce<'a, 'c, I, V, O, F> where O : 'c, V : Copy, F : Fn(V,I) -> (V, O) { 275 | type Input = I; 276 | 277 | default_pass_subscribe!(); 278 | default_pass_complete!(); 279 | default_pass_error!(); 280 | 281 | fn on_next(&mut self, t: I) -> bool { 282 | let (newstate, outval) = (self.fun)(self.state, t); 283 | self.state = newstate; 284 | match self.subscriber.as_mut() { 285 | Some(s) => { s.on_next(outval) } 286 | None => {true} 287 | } 288 | } 289 | } 290 | 291 | // 292 | // Enumerate 293 | // 294 | // 295 | 296 | pub struct Enumerate<'a, I> { 297 | subscriber: Option + 'a>>, 298 | index: Option, 299 | count: u64 300 | } 301 | 302 | 303 | impl<'a, I> Enumerate<'a, I> { 304 | 305 | pub fn new() -> Enumerate<'a, I> 306 | { 307 | Enumerate { 308 | subscriber: None, 309 | index: None, 310 | count: 0 311 | } 312 | } 313 | } 314 | 315 | impl<'a, I> Publisher<'a> for Enumerate<'a, I> { 316 | type Output = (I, u64); 317 | 318 | fn subscribe(&mut self, s: Box + 'a>) { 319 | let s: Box+'a> = s; 320 | self.subscriber = Some(s); 321 | self.subscriber.as_mut().unwrap().on_subscribe(0); 322 | } 323 | } 324 | 325 | impl<'a, I> Subscriber for Enumerate<'a, I> { 326 | type Input = I; 327 | 328 | default_pass_subscribe!(); 329 | default_pass_complete!(); 330 | default_pass_error!(); 331 | 332 | fn on_next(&mut self, t: I) -> bool { 333 | match self.subscriber.as_mut() { 334 | Some(s) => { let c = self.count; 335 | self.count += 1; 336 | s.on_next( (t, c) ) } 337 | None => {true} 338 | } 339 | } 340 | } 341 | 342 | /// Tee 343 | /// Sends the value off to a supplied Sendable queue, 344 | /// then passes the data to its subscriber 345 | /// 346 | pub struct Tee<'a, Q, I> 347 | where I : Send + Clone, 348 | Q : Sendable 349 | { 350 | 351 | subscriber: Option + 'a>>, 352 | index: Option, 353 | data_tx: Q, 354 | } 355 | 356 | impl<'a, Q, I> Tee<'a, Q, I> 357 | where I : Send + Clone, 358 | Q : Sendable 359 | { 360 | 361 | pub fn new(tx: Q) -> Tee<'a, Q,I> { 362 | Tee { 363 | index: None, 364 | subscriber: None, 365 | data_tx: tx, 366 | } 367 | } 368 | } 369 | 370 | impl<'a, Q, I> Publisher<'a> for Tee<'a, Q, I> 371 | where I : Send + Clone, 372 | Q : Sendable { 373 | 374 | type Output = I; 375 | 376 | fn subscribe(&mut self, s: Box + 'a>) { 377 | let s: Box+'a> = s; 378 | self.subscriber = Some(s); 379 | self.subscriber.as_mut().unwrap().on_subscribe(0); 380 | } 381 | } 382 | 383 | impl<'a, Q, I> Subscriber for Tee<'a, Q, I> 384 | where I : Send + Clone, 385 | Q : Sendable { 386 | 387 | type Input = I; 388 | 389 | default_pass_subscribe!(); 390 | default_pass_complete!(); 391 | default_pass_error!(); 392 | 393 | fn on_next(&mut self, t: I) -> bool { 394 | match self.subscriber.as_mut() { 395 | Some(s) => { self.data_tx.send(t.clone()); s.on_next(t) } 396 | None => {true} 397 | } 398 | } 399 | } 400 | 401 | 402 | 403 | /// Unzip 404 | /// takes tuples of identical items as input 405 | /// unpacks them into their own message 406 | /// 407 | pub struct Unzip<'a, I> 408 | { 409 | subscriber: Option + 'a>>, 410 | index: Option, 411 | } 412 | 413 | impl<'a, I> Unzip<'a, I> { 414 | pub fn new() -> Unzip<'a, I> { 415 | Unzip { 416 | index: None, 417 | subscriber: None, 418 | } 419 | } 420 | } 421 | 422 | impl<'a, I> Publisher<'a> for Unzip<'a, I> 423 | { 424 | type Output = I; 425 | 426 | fn subscribe(&mut self, s: Box + 'a>) { 427 | let s: Box+'a> = s; 428 | self.subscriber = Some(s); 429 | self.subscriber.as_mut().unwrap().on_subscribe(0); 430 | } 431 | } 432 | 433 | impl<'a, I> Subscriber for Unzip<'a, I> 434 | { 435 | type Input = (I,I); 436 | default_pass_subscribe!(); 437 | default_pass_complete!(); 438 | default_pass_error!(); 439 | 440 | fn on_next(&mut self, t: (I,I)) -> bool { 441 | match self.subscriber.as_mut() { 442 | Some(s) => { 443 | s.on_next(t.0) | s.on_next(t.1) 444 | } 445 | None => {true} 446 | } 447 | } 448 | } 449 | 450 | 451 | /// Take 452 | /// takes tuples of identical items as input 453 | /// unpacks them into their own message 454 | /// 455 | pub struct Take<'a, O> 456 | { 457 | subscriber: Option + 'a>>, 458 | index: Option, 459 | count: usize, 460 | max: usize, 461 | notified: bool 462 | } 463 | 464 | impl<'a, O> Take<'a, O> { 465 | pub fn new(max: usize) -> Take<'a, O> { 466 | Take { 467 | index: None, 468 | subscriber: None, 469 | count: 0, 470 | max: max, 471 | notified: false 472 | } 473 | } 474 | } 475 | 476 | impl<'a, O> Publisher<'a> for Take<'a, O> 477 | { 478 | type Output = O; 479 | 480 | fn subscribe(&mut self, s: Box + 'a>) { 481 | let s: Box+'a> = s; 482 | self.subscriber = Some(s); 483 | self.subscriber.as_mut().unwrap().on_subscribe(0); 484 | } 485 | } 486 | 487 | impl<'a, O> Subscriber for Take<'a, O> 488 | { 489 | type Input = O; 490 | default_pass_subscribe!(); 491 | default_pass_complete!(); 492 | default_pass_error!(); 493 | 494 | fn on_next(&mut self, t: O) -> bool { 495 | match self.subscriber.as_mut() { 496 | Some(s) => { 497 | self.count += 1; 498 | if self.count > self.max { s.on_complete(false); self.notified = true; false } 499 | else { s.on_next(t) } 500 | }, 501 | None => {true} 502 | } 503 | } 504 | } 505 | 506 | 507 | -------------------------------------------------------------------------------- /src/protocol.rs: -------------------------------------------------------------------------------- 1 | 2 | use iobuf::{AROIobuf, Iobuf}; 3 | 4 | pub trait Protocol { 5 | type Output : Send + 'static; 6 | 7 | fn new() -> Self; 8 | 9 | fn append(&mut self, &AROIobuf) -> Option<(Self::Output, AROIobuf, u32)>; 10 | } 11 | 12 | pub trait HasSize{ fn size() -> u32; } 13 | 14 | /// A simple Protocol implementation which produces buffers of a fixed size 15 | #[derive(Debug)] 16 | pub struct BufProtocol; 17 | 18 | 19 | impl Protocol for BufProtocol { 20 | type Output = AROIobuf; 21 | 22 | fn new() -> BufProtocol { 23 | BufProtocol 24 | } 25 | 26 | fn append(&mut self, buf: &AROIobuf) -> Option<(::Output, AROIobuf, u32)> { 27 | let bufsz = ::size(); 28 | if buf.len() >= bufsz { 29 | let (a, b) = buf.split_at(bufsz).unwrap(); 30 | Some((a, b, bufsz)) 31 | } else { 32 | None 33 | } 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /src/publisherext.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | use reactive::{Publisher, Subscriber, Processor}; 6 | use processorimpl::*; 7 | use std::sync::Arc; 8 | use std::cell::RefCell; 9 | 10 | /* 11 | pub trait PublisherExt<'p> : Publisher<'p> + Sized { 12 | 13 | fn map<'a, O, F>(mut self, f: F) -> PublisherWrapper<'a, Self, O> 14 | where F : Fn(Self::Output) -> O 15 | { 16 | let mut m : Box + 'a> = Box::new(Map::new(f)); 17 | let t = Arc::new(RefCell::new(m)); 18 | self.subscribe(*t.borrow_mut()); 19 | PublisherWrapper { head: self, tail: t.clone() } 20 | } 21 | } 22 | 23 | impl<'a, P> PublisherExt<'a> for P where P : Publisher<'a> {} 24 | 25 | pub struct PublisherWrapper<'a, P, O> 26 | where P : Publisher<'a> + 'a 27 | { 28 | head: P, 29 | tail: Arc + 'a>>> 30 | } 31 | 32 | */ 33 | -------------------------------------------------------------------------------- /src/publisherimpl.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | 6 | use reactive::{Publisher, Subscriber}; 7 | 8 | use quickcheck::{Arbitrary, Gen, StdGen}; 9 | 10 | use std::sync::mpsc::{Sender, Receiver, TryRecvError}; 11 | use std::rand::Rng; 12 | 13 | use rand::isaac::Isaac64Rng as IRng; 14 | 15 | pub struct TestIncGen { 16 | current: u64 17 | } 18 | 19 | impl Rng for TestIncGen { 20 | fn next_u32(&mut self) -> u32 { 21 | let c = self.current; 22 | self.current += 1; 23 | c as u32 24 | } 25 | 26 | fn next_u64(&mut self) -> u64 { 27 | let c = self.current; 28 | self.current += 1; 29 | c 30 | } 31 | } 32 | 33 | pub struct RndGen<'a, 'b, O> where O : Arbitrary { 34 | subscriber: Option + 'a>>, 35 | gen: StdGen 36 | } 37 | 38 | impl<'a, 'b, O> RndGen<'a, 'b, O> where O : Arbitrary { 39 | pub fn new() -> RndGen<'a, 'b, O> 40 | { 41 | RndGen { 42 | subscriber: None, 43 | gen: StdGen::new(TestIncGen { current: 0 }, 1_000_000) 44 | } 45 | } 46 | 47 | } 48 | 49 | impl<'a,'b, O> Publisher<'a> for RndGen<'a, 'b, O> where O : Arbitrary { 50 | type Output = O; 51 | fn subscribe(&mut self, s: Box + 'a>) { 52 | let s: Box+'a> = s; 53 | self.subscriber = Some(s); 54 | self.subscriber.as_mut().unwrap().on_subscribe(0); 55 | } 56 | 57 | fn try_next(&mut self) -> bool { 58 | match self.subscriber.as_mut() { 59 | Some(s) => s.on_next(Arbitrary::arbitrary(&mut self.gen)), 60 | None => {error!("My subscriber went away");false} 61 | } 62 | } 63 | } 64 | 65 | 66 | // 67 | // Iterator Publisher 68 | // 69 | 70 | pub struct IterPublisher<'a, 'b, 'c, O, Iter> 71 | where Iter: Iterator + 'b , O : 'c 72 | { 73 | iter: Iter, 74 | subscriber: Option + 'a>> 75 | } 76 | 77 | impl<'a, 'b, 'c, O, Iter> IterPublisher<'a, 'b, 'c, O, Iter> 78 | where Iter: Iterator + 'b , 79 | O : 'c 80 | { 81 | pub fn new(iter: Iter) -> IterPublisher<'a, 'b, 'c, O, Iter> 82 | { 83 | IterPublisher { 84 | iter: iter, 85 | subscriber: None 86 | } 87 | } 88 | 89 | 90 | } 91 | 92 | impl<'a, 'b, 'c, O, Iter> Publisher<'a> for IterPublisher<'a, 'b, 'c, O, Iter> 93 | where Iter: Iterator + 'b, 94 | O : 'c 95 | { 96 | 97 | type Output = O; 98 | fn subscribe(&mut self, s: Box + 'a>) { 99 | let s: Box+'a> = s; 100 | self.subscriber = Some(s); 101 | self.subscriber.as_mut().unwrap().on_subscribe(0); 102 | } 103 | 104 | fn next(&mut self) -> bool { 105 | match self.subscriber.as_mut() { 106 | Some(s) => { 107 | match self.iter.next() { 108 | Some(x) => s.on_next(x), 109 | None => { s.on_complete(false); false } 110 | } 111 | }, 112 | None => {error!("My subscriber went away");false} 113 | } 114 | } 115 | } 116 | 117 | // 118 | // Coupler 119 | // 120 | pub struct Coupler<'a, O> where O : Send { 121 | data_rx: Receiver, 122 | subscriber: Option + 'a>> 123 | } 124 | 125 | 126 | impl<'a, O> Coupler<'a, O> where O : Send { 127 | pub fn new(rx: Receiver) -> Coupler<'a, O> { 128 | Coupler { 129 | data_rx: rx, 130 | subscriber: None, 131 | } 132 | } 133 | 134 | } 135 | 136 | 137 | impl<'a, O> Publisher<'a> for Coupler<'a, O> where O : Send { 138 | 139 | type Output = O; 140 | fn subscribe(&mut self, s: Box + 'a>) { 141 | let s: Box+'a> = s; 142 | self.subscriber = Some(s); 143 | self.subscriber.as_mut().unwrap().on_subscribe(0); 144 | } 145 | 146 | fn next (&mut self) -> bool { 147 | match self.subscriber.as_mut() { 148 | Some(s) => match self.data_rx.recv() { 149 | Ok(d) => s.on_next(d), 150 | Err(..) => { info!("The other end of the coupler queue went away"); s.on_complete(false); false } 151 | }, 152 | None => { error!("My subscriber went away"); false } 153 | } 154 | } 155 | 156 | // Does not block 157 | fn try_next(&mut self) -> bool { 158 | match self.subscriber.as_mut() { 159 | Some(s) => match self.data_rx.try_recv() { 160 | Ok(d) => s.on_next(d), 161 | Err(TryRecvError::Empty) => true, 162 | Err(TryRecvError::Disconnected) => { 163 | info!("The other end of the coupler queue went away"); s.on_complete(false); false 164 | } 165 | }, 166 | None => { error!("My subscriber went away"); false } 167 | } 168 | } 169 | } 170 | 171 | // 172 | // Repeat 173 | // 174 | pub struct Repeat<'a, O> where O : Send + Clone { 175 | val: O, 176 | subscriber: Option + 'a>> 177 | } 178 | 179 | 180 | impl<'a, O> Repeat<'a, O> where O : Send + Clone { 181 | pub fn new(o: O) -> Repeat<'a, O> { 182 | Repeat { 183 | val: o, 184 | subscriber: None, 185 | } 186 | } 187 | 188 | } 189 | 190 | 191 | impl<'a, O> Publisher<'a> for Repeat<'a, O> where O : Send + Clone { 192 | type Output = O; 193 | fn subscribe(&mut self, s: Box + 'a>) { 194 | let s: Box+'a> = s; 195 | self.subscriber = Some(s); 196 | self.subscriber.as_mut().unwrap().on_subscribe(0); 197 | } 198 | 199 | fn next (&mut self) -> bool{ 200 | match self.subscriber.as_mut() { 201 | Some(s) => s.on_next(self.val.clone()), 202 | None => { error!("My subscriber went away"); false } 203 | } 204 | } 205 | } 206 | 207 | 208 | -------------------------------------------------------------------------------- /src/reactive.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | 6 | 7 | 8 | pub trait Subscriber { 9 | type Input; 10 | 11 | fn on_next(&mut self, t: Self::Input) -> bool; 12 | fn on_subscribe(&mut self, usize) { 13 | debug!("on_subscribe called"); 14 | } 15 | fn on_error(&mut self, err: &str) { 16 | error!("on_error called: {:?}", err); 17 | } 18 | fn on_complete(&mut self, force: bool) { 19 | debug!("on_complete called"); 20 | } 21 | } 22 | 23 | pub trait Publisher<'a> { 24 | type Output; 25 | 26 | fn subscribe(&mut self, Box + 'a>); 27 | 28 | /// The basic message event generation function 29 | /// this is typically called in a loop 30 | /// This version of the function can block 31 | fn next(&mut self) -> bool { 32 | self.try_next() 33 | } 34 | 35 | /// The basic message event generation function 36 | /// this is typically called in a loop 37 | /// It is expected that this next will never block 38 | fn try_next(&mut self) -> bool { 39 | panic!("Unimplemented fn, presumably run() or next() is being attempted on a processor, not a publisher"); 40 | } 41 | 42 | /// Runs in a loop, expects that its publisher might block 43 | fn run(&mut self) { 44 | debug!("Starting loop"); 45 | loop { 46 | if ! self.next() { 47 | break 48 | } 49 | } 50 | debug!("Done with loop"); 51 | } 52 | 53 | } 54 | 55 | pub trait Processor<'a> : Subscriber + Publisher<'a> { } 56 | 57 | impl<'a, P> Processor<'a> for P 58 | where P : Subscriber + Publisher<'a> { } 59 | -------------------------------------------------------------------------------- /src/reactor.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | 6 | use net_stream::NetStream; 7 | 8 | use mio::{ 9 | EventLoop, 10 | EventLoopSender, 11 | EventLoopConfig, 12 | Handler, 13 | NonBlock, 14 | IoWriter, 15 | IoReader, 16 | IoAcceptor, 17 | Buf, MutBuf, 18 | Timeout, 19 | MioResult}; 20 | 21 | pub use mio::Token; 22 | use mio::net::{SockAddr, Socket}; 23 | use mio::net::tcp::{TcpAcceptor, TcpSocket}; 24 | use mio::util::Slab; 25 | use mio::event; 26 | 27 | use iobuf::{Iobuf, RWIobuf, AROIobuf, Allocator, AppendBuf}; 28 | 29 | use std::old_io::net::addrinfo::get_host_addresses; 30 | use std::result::Result; 31 | use std::sync::Arc; 32 | use std::sync::mpsc::{Receiver,SyncSender, sync_channel}; 33 | 34 | use std::time::Duration; 35 | 36 | use collections::dlist::DList; 37 | 38 | use reactive::Subscriber; 39 | use protocol::Protocol; 40 | 41 | /// The basic sendable buffer which also contains 42 | /// its own addressing. When the buffer is received, 43 | /// Token reflects the socket whence it came, 44 | /// when it is sent, Token is the socket out of which 45 | /// the buffer should be sent 46 | #[derive(Show)] 47 | pub struct StreamBuf (pub AROIobuf, pub Token); 48 | 49 | #[derive(Show)] 50 | pub struct ProtoMsg (pub T, pub Token); 51 | 52 | unsafe impl Send for StreamBuf {} 53 | 54 | impl Clone for StreamBuf { 55 | fn clone(&self) -> StreamBuf { 56 | StreamBuf (self.0.clone(), self.1) 57 | } 58 | } 59 | 60 | struct ReadBuf (AppendBuf<'static>); 61 | 62 | pub type TimerCB<'a> = FnMut(&mut Reactor)->bool + 'a; 63 | 64 | pub type Reactor = EventLoop; 65 | 66 | pub type Sender = EventLoopSender; 67 | 68 | impl Buf for StreamBuf { 69 | fn remaining(&self) -> usize { 70 | self.0.len() as usize 71 | } 72 | 73 | fn bytes<'a>(&'a self) -> &'a [u8] { unsafe { 74 | self.0.as_window_slice() 75 | } } 76 | 77 | fn advance(&mut self, cnt: usize) { 78 | self.0.advance(cnt as u32).unwrap(); 79 | } 80 | } 81 | 82 | impl Buf for ReadBuf { 83 | fn remaining(&self) -> usize { 84 | self.0.len() as usize 85 | } 86 | 87 | fn bytes<'b>(&'b self) -> &'b [u8] { 88 | self.0.as_window_slice() 89 | } 90 | 91 | fn advance(&mut self, cnt: usize) { 92 | self.0.advance(cnt as u32).unwrap(); 93 | } 94 | } 95 | 96 | impl MutBuf for ReadBuf { 97 | fn mut_bytes<'b>(&'b mut self) -> &'b mut [u8] { 98 | self.0.as_mut_window_slice() 99 | } 100 | } 101 | 102 | 103 | struct Connection 104 | where T : Protocol, ::Output : Send 105 | { 106 | sock: TcpSocket, 107 | outbuf: DList, 108 | interest: event::Interest, 109 | conn_tx: SyncSender::Output>>, 110 | marker: u32, 111 | proto: T, 112 | buf: ReadBuf 113 | } 114 | 115 | impl Connection 116 | where T : Protocol, ::Output : Send 117 | { 118 | pub fn new(s: TcpSocket, tx: SyncSender::Output>>, rbuf: ReadBuf) -> Connection { 119 | Connection { 120 | sock: s, 121 | outbuf: DList::new(), 122 | interest: event::HUP, 123 | conn_tx: tx, 124 | marker: 0, 125 | proto: ::new(), 126 | buf: rbuf 127 | } 128 | } 129 | 130 | fn drain_write_queue_to_socket(&mut self) -> usize { 131 | let mut writable = true; 132 | while writable && self.outbuf.len() > 0 { 133 | let (result, sz) = { 134 | let buf = self.outbuf.front_mut().unwrap(); //shouldn't panic because of len() check 135 | let sz = buf.0.len(); 136 | (self.sock.write(buf), sz as usize) 137 | }; 138 | match result { 139 | Ok(NonBlock::Ready(n)) => 140 | { 141 | debug!("Wrote {:?} out of {:?} bytes to socket", n, sz); 142 | if n == sz { 143 | self.outbuf.pop_front(); // we have written the contents of this buffer so lets get rid of it 144 | } 145 | }, 146 | Ok(NonBlock::WouldBlock) => { // this is also very unlikely, we got a writable message, but failed 147 | // to write anything at all. 148 | debug!("Got Writable event for socket, but failed to write any bytes"); 149 | writable = false; 150 | }, 151 | Err(e) => { error!("error writing to socket: {:?}", e); writable = false } 152 | } 153 | } 154 | self.outbuf.len() 155 | } 156 | 157 | fn read(&mut self) -> MioResult> { 158 | self.sock.read(&mut self.buf) 159 | } 160 | } 161 | 162 | 163 | /// Configuration for the Net Engine 164 | /// queue_size: All queues, both inbound and outbound 165 | /// read_buf_sz: The size of the read buffer allocatod 166 | pub struct NetEngineConfig { 167 | queue_size: usize, 168 | read_buf_sz: usize, 169 | min_read_buf_sz: usize, 170 | max_connections: usize, 171 | poll_timeout_ms: usize, 172 | allocator: Option>> 173 | } 174 | 175 | pub struct NetEngine<'a, T> 176 | where T : Protocol, ::Output : Send 177 | { 178 | inner: EngineInner<'a, T>, 179 | event_loop: Reactor 180 | } 181 | 182 | 183 | impl<'a, T> NetEngine<'a, T> 184 | where T : Protocol, ::Output : Send 185 | { 186 | 187 | /// Construct a new NetEngine with (hopefully) intelligent defaults 188 | /// 189 | pub fn new() -> NetEngine<'a, T> { 190 | let config = NetEngineConfig { 191 | queue_size: 524288, 192 | read_buf_sz: 1536, 193 | min_read_buf_sz: 64, 194 | allocator: None, 195 | max_connections: 10240, 196 | poll_timeout_ms: 100 197 | }; 198 | 199 | NetEngine::configured(config) 200 | } 201 | 202 | /// Construct a new engine with defaults specified by the user 203 | pub fn configured(cfg: NetEngineConfig) -> NetEngine<'a, T> { 204 | NetEngine { event_loop: EventLoop::configured(NetEngine::<'a, T>::event_loop_config(cfg.queue_size, cfg.poll_timeout_ms)).unwrap(), 205 | inner: EngineInner::new(cfg) 206 | } 207 | } 208 | 209 | fn event_loop_config(queue_sz : usize, timeout: usize) -> EventLoopConfig { 210 | EventLoopConfig { 211 | io_poll_timeout_ms: timeout, 212 | notify_capacity: queue_sz, 213 | messages_per_tick: 512, 214 | timer_tick_ms: 100, 215 | timer_wheel_size: 1_024, 216 | timer_capacity: 65_536, 217 | } 218 | } 219 | 220 | /// connect to the supplied hostname and port 221 | /// any data that arrives on the connection will be put into a Buf 222 | /// and sent down the supplied Sender channel along with the Token of the connection 223 | pub fn connect<'b>(&mut self, 224 | hostname: &str, 225 | port: usize) -> Result::Output>, String> { 226 | self.inner.connect(hostname, port, &mut self.event_loop) 227 | } 228 | 229 | /// listen on the supplied ip address and port 230 | /// any new connections will be accepted and polled for read events 231 | /// all datagrams that arrive will be put into StreamBufs with their 232 | /// corresponding token, and added to the default outbound data queue 233 | /// this can be called multiple times for different ips/ports 234 | pub fn listen<'b>(&mut self, 235 | addr: &'b str, 236 | port: usize) -> Result::Output>>, String> { 237 | self.inner.listen(addr, port, &mut self.event_loop) 238 | } 239 | 240 | /// fetch the event_loop channel for notifying the event_loop of new outbound data 241 | pub fn channel(&self) -> EventLoopSender { 242 | self.event_loop.channel() 243 | } 244 | 245 | /// Set a timeout to be executed by the event loop after duration 246 | /// Minimum expected resolution is the tick duration of the event loop 247 | /// poller, but it could be shorted depending on how many events are 248 | /// occurring 249 | pub fn timeout(&mut self, timeout: Duration, callback: Box>) { 250 | let tok = self.inner.timeouts.insert((callback, None)).map_err(|_|()).unwrap(); 251 | let handle = self.event_loop.timeout(tok, timeout).unwrap(); 252 | self.inner.timeouts.get_mut(tok).unwrap().1 = Some(handle); 253 | } 254 | 255 | /// process all incoming and outgoing events in a loop 256 | pub fn run(mut self) { 257 | self.event_loop.run(self.inner).map_err(|_| ()).unwrap(); 258 | } 259 | 260 | /// process all incoming and outgoing events in a loop 261 | pub fn run_once(mut self) { 262 | self.event_loop.run_once(self.inner).map_err(|_| ()).unwrap(); 263 | } 264 | 265 | /// calculates the 11th digit of pi 266 | pub fn shutdown(mut self) { 267 | self.event_loop.shutdown(); 268 | } 269 | } 270 | 271 | 272 | struct EngineInner<'a, T> 273 | where T : Protocol, ::Output : Send 274 | { 275 | listeners: Slab<(TcpAcceptor, SyncSender::Output>>)>, 276 | timeouts: Slab<(Box>, Option)>, 277 | conns: Slab>, 278 | config: NetEngineConfig, 279 | } 280 | 281 | impl<'a, T> EngineInner<'a, T> 282 | where T : Protocol, ::Output : Send 283 | { 284 | 285 | pub fn new(cfg: NetEngineConfig) -> EngineInner<'a, T> { 286 | 287 | EngineInner { 288 | listeners: Slab::new_starting_at(Token(0), 128), 289 | timeouts: Slab::new_starting_at(Token(129), 255), 290 | conns: Slab::new_starting_at(Token(256), cfg.max_connections + 256), 291 | config: cfg 292 | } 293 | } 294 | 295 | pub fn connect<'b>(&mut self, 296 | hostname: &str, 297 | port: usize, 298 | event_loop: &mut Reactor) -> Result::Output>, String> 299 | { 300 | let ip = get_host_addresses(hostname).unwrap()[0]; //TODO manage receiving multiple IPs per hostname, random sample or something 301 | match TcpSocket::v4() { 302 | Ok(s) => { 303 | //s.set_tcp_nodelay(true); TODO: re-add to mio 304 | let (tx, rx) = sync_channel(self.config.queue_size); 305 | let buf = new_buf(self.config.read_buf_sz, self.config.allocator.clone()); 306 | match self.conns.insert(Connection::new(s, tx, buf)) { 307 | Ok(tok) => match event_loop.register_opt(&self.conns.get(tok).unwrap().sock, tok, event::READABLE, event::PollOpt::edge()) { 308 | Ok(..) => match self.conns.get(tok).unwrap().sock.connect(&SockAddr::InetAddr(ip, port as u16)) { 309 | Ok(..) => { 310 | debug!("Connected to server for token {:?}", tok); 311 | Ok(NetStream::new(tok, rx, event_loop.channel().clone())) 312 | } 313 | Err(e) => Err(format!("Failed to connect to {:?}:{:?}, error: {:?}", hostname, port, e)) 314 | }, 315 | Err(e) => Err(format!("Failed to register with the event loop, error: {:?}", e)) 316 | }, 317 | _ => Err(format!("Failed to insert into connection slab")) 318 | } 319 | }, 320 | Err(e) => Err(format!("Failed to create new socket, error:{:?}", e)) 321 | } 322 | } 323 | 324 | pub fn listen<'b>(&mut self, 325 | addr: &'b str, 326 | port: usize, 327 | event_loop: &mut Reactor) -> Result::Output >>, String> 328 | { 329 | let ip = get_host_addresses(addr).unwrap()[0]; 330 | match TcpSocket::v4() { 331 | Ok(s) => { 332 | //s.set_tcp_nodelay(true); TODO: re-add to mio 333 | match s.bind(&SockAddr::InetAddr(ip, port as u16)) { 334 | Ok(l) => match l.listen(255) { 335 | Ok(a) => { 336 | let (tx, rx) = sync_channel(self.config.queue_size); 337 | match self.listeners.insert((a, tx)) { 338 | Ok(token) => { 339 | event_loop.register_opt(&self.listeners.get_mut(token).unwrap().0, 340 | token, 341 | event::READABLE, 342 | event::PollOpt::edge()). 343 | map_err(|e| format!("event registration failed: {:?}", e)). 344 | map(move |_| rx) 345 | }, 346 | Err(_) => Err(format!("failed to insert into listener slab")) 347 | } 348 | }, 349 | Err(e) => {Err(format!("Failed to listen to socket {:?}:{:?}, error:{:?}", addr, port, e)) } 350 | }, 351 | Err(e) => Err(format!("Failed to bind to {:?}:{:?}, error:{:?}", addr, port, e)) 352 | }}, 353 | Err(e) => Err(format!("Failed to create TCP socket, error:{:?}", e)) 354 | } 355 | } 356 | 357 | 358 | } 359 | 360 | impl<'a, T> Handler for EngineInner<'a, T> 361 | where T : Protocol, ::Output : Send 362 | { 363 | 364 | fn readable(&mut self, event_loop: &mut Reactor, token: Token, hint: event::ReadHint) { 365 | debug!("mio_processor::readable top, token: {:?}", token); 366 | let mut close = false; 367 | if self.listeners.contains(token) { 368 | let calloc = &self.config.allocator; 369 | let buf_sz = self.config.read_buf_sz; 370 | let (ref mut list, ref tx) = *self.listeners.get_mut(token).unwrap(); 371 | match list.accept() { 372 | Ok(NonBlock::Ready(sock)) => { 373 | let buf = new_buf(buf_sz, calloc.clone()); 374 | match self.conns.insert(Connection::new(sock, tx.clone(), buf)) { 375 | Ok(tok) => { 376 | event_loop.register_opt(&self.conns.get(tok).unwrap().sock, 377 | tok, event::READABLE | event::HUP, 378 | event::PollOpt::edge()).unwrap(); 379 | debug!("readable accepted socket for token {:?}", tok); } 380 | Err(..) => error!("Failed to insert into Slab") 381 | }; }, 382 | e => { error!("Failed to accept socket: {:?}", e);} 383 | } 384 | event_loop.reregister(list, token, event::READABLE, event::PollOpt::edge()).unwrap(); 385 | return; 386 | 387 | } else { 388 | 389 | match self.conns.get_mut(token) { 390 | None => error!("Got a readable event for token {:?}, 391 | but it is not present in MioHandler connections", token), 392 | Some(c) => { 393 | match c.read() { 394 | Ok(NonBlock::Ready(n)) => { 395 | debug!("read {:?} bytes", n); 396 | let mut abuf = c.buf.0.atomic_slice_pos_from_begin(c.marker, n as i64).unwrap(); 397 | loop { 398 | match c.proto.append(&abuf) { 399 | None => {break}, 400 | Some((item, remaining, consumed)) => { 401 | c.conn_tx.send( ProtoMsg(item, token)); 402 | abuf = remaining; 403 | c.marker += consumed; 404 | } 405 | } 406 | } 407 | if c.buf.0.len() < self.config.min_read_buf_sz as u32 { 408 | let mut newbuf = new_buf(self.config.read_buf_sz, self.config.allocator.clone()); 409 | if abuf.len() > 0 { 410 | // we didn't eat all of the bytes we just read 411 | // so we must move them to the new buffer 412 | unsafe { newbuf.0.fill(abuf.as_window_slice()) }; 413 | } 414 | c.buf = newbuf; 415 | c.marker = 0; 416 | } 417 | } 418 | Ok(NonBlock::WouldBlock) => { 419 | debug!("Got Readable event for socket, but failed to write any bytes"); 420 | }, 421 | Err(e) => error!("error reading from socket: {:?}", e) 422 | }; 423 | 424 | if hint.contains(event::HUPHINT) { 425 | close = true; 426 | } 427 | else { 428 | c.interest.insert(event::READABLE); 429 | event_loop.reregister(&c.sock, token, c.interest, event::PollOpt::edge()).unwrap(); 430 | } 431 | } 432 | } 433 | 434 | if close { 435 | self.conns.remove(token); 436 | } 437 | } 438 | } 439 | 440 | fn writable(&mut self, event_loop: &mut Reactor, token: Token) { 441 | debug!("mio_processor::writable, token: {:?}", token); 442 | if let Some(c) = self.conns.get_mut(token) { 443 | if c.drain_write_queue_to_socket() > 0 { 444 | c.interest.insert(event::WRITABLE); 445 | event_loop.reregister(&c.sock, token, c.interest, event::PollOpt::edge()).unwrap(); 446 | } 447 | } 448 | } 449 | 450 | 451 | fn notify(&mut self, event_loop: &mut Reactor, msg: StreamBuf) { 452 | let tok = msg.1; 453 | match self.conns.get_mut(tok) { 454 | Some(c) => { 455 | c.outbuf.push_back(msg); 456 | if c.drain_write_queue_to_socket() > 0 { 457 | c.interest.insert(event::WRITABLE); 458 | event_loop.reregister(&c.sock, tok, c.interest, event::PollOpt::edge()).unwrap(); 459 | } 460 | }, 461 | None => {} 462 | } 463 | } 464 | 465 | fn timeout(&mut self, event_loop: &mut Reactor, tok: Token) { 466 | let (ref mut cb, ref handle) = *self.timeouts.get_mut(tok).unwrap(); 467 | if !(*cb).call_mut((event_loop,)) { 468 | event_loop.clear_timeout(handle.unwrap()); 469 | } 470 | } 471 | } 472 | 473 | fn new_buf(sz: usize, calloc: Option>>) -> ReadBuf { 474 | if let Some(alloc) = calloc { 475 | ReadBuf(AppendBuf::new_with_allocator(sz, alloc)) 476 | } else { 477 | ReadBuf(AppendBuf::new(sz)) 478 | } 479 | } 480 | -------------------------------------------------------------------------------- /src/scheduler.rs: -------------------------------------------------------------------------------- 1 | 2 | 3 | use std::time::duration::Duration; 4 | use std::old_io::timer::Timer; 5 | use std::num::Int; 6 | use time::precise_time_ns; 7 | use reactive::Publisher; 8 | use reactor::StreamBuf; 9 | use libc::{timespec, nanosleep}; 10 | 11 | /// This spins in a loop at a specific frame rate 12 | /// it sleeps between tries 13 | /// rate is the max number of loops per second that 14 | /// it makes There is no minimum number of loops 15 | pub fn fixed_loop(rate: u64, mut f: F) where F : FnMut() -> bool { 16 | debug!("Starting loop"); 17 | let mut timer = Timer::new().unwrap(); 18 | let loop_time_ns = 1_000_000_000i64 / rate as i64; 19 | let mut run = true; 20 | let mut nein = timespec { tv_sec: 0, tv_nsec: 0 }; 21 | 22 | loop { 23 | let start = precise_time_ns() as i64; 24 | if !f() { 25 | break; 26 | } 27 | let span = precise_time_ns() as i64 - start; 28 | if let Some(diff) = loop_time_ns.checked_sub(span) { 29 | let ts = timespec { tv_sec: 0, tv_nsec: diff }; 30 | unsafe { nanosleep(&ts as *const timespec, &mut nein as *mut timespec) }; 31 | } 32 | } 33 | 34 | debug!("Done with loop"); 35 | } 36 | 37 | -------------------------------------------------------------------------------- /src/sendable.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | // 6 | use std::sync::mpsc::{SyncSender, SendError, Sender}; 7 | use mio::EventLoopSender; 8 | 9 | /// Sendable 10 | /// Wrapper trait for all types of queue senders 11 | pub trait Sendable : Sized + Clone { 12 | type Item; 13 | fn send(&self, a: Self::Item) -> Result<(), Self::Item>; 14 | } 15 | 16 | impl Sendable for Sender { 17 | type Item = A; 18 | fn send(&self, a: ::Item) -> Result<(), ::Item> { 19 | match self.send(a) { 20 | Ok(_) => Ok(()), 21 | Err(SendError(e)) => Err(e) 22 | } 23 | } 24 | } 25 | 26 | impl Sendable for SyncSender { 27 | type Item = A; 28 | fn send(&self, a: ::Item) -> Result<(), ::Item> { 29 | match self.send(a) { 30 | Ok(_) => Ok(()), 31 | Err(SendError(e)) => Err(e) 32 | } 33 | } 34 | } 35 | 36 | impl Sendable for EventLoopSender { 37 | type Item = A; 38 | fn send(&self, a: ::Item) -> Result<(), ::Item> { 39 | match self.send(a) { 40 | Ok(_) => Ok(()), 41 | e@Err(_) => e 42 | } 43 | } 44 | } 45 | 46 | -------------------------------------------------------------------------------- /src/subscriber.rs: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 2 | // 3 | // This software may be modified and distributed under the terms 4 | // of the MIT license. See the LICENSE file for details. 5 | 6 | use std::fmt::Display; 7 | use reactive::{Subscriber}; 8 | use sendable::Sendable; 9 | 10 | pub struct StdoutSubscriber where A : Display { 11 | index: Option 12 | } 13 | 14 | impl StdoutSubscriber where A : Display { 15 | 16 | pub fn new() -> StdoutSubscriber { 17 | StdoutSubscriber { 18 | index: None 19 | } 20 | } 21 | } 22 | 23 | impl Subscriber for StdoutSubscriber where A : Display { 24 | type Input = A; 25 | 26 | fn on_next(&mut self, t: A) -> bool { 27 | println!("{}", t); 28 | true 29 | } 30 | } 31 | 32 | pub struct Decoupler where I : Send, Q : Sendable { 33 | index: Option, 34 | data_tx: Q, 35 | } 36 | 37 | impl Decoupler where I : Send, Q : Sendable { 38 | 39 | pub fn new(tx: Q) -> Decoupler { 40 | Decoupler { 41 | index: None, 42 | data_tx: tx, 43 | } 44 | } 45 | } 46 | 47 | impl Subscriber for Decoupler 48 | where I : Send, 49 | Q : Sendable 50 | { 51 | type Input = I; 52 | 53 | fn on_next(&mut self, t: I) -> bool { 54 | //TODO better handle queue failure, maybe put the returned buf 55 | //isizeo a recovery queue 56 | match self.data_tx.send(t) { 57 | Ok(()) => true, 58 | Err(_) => false 59 | } 60 | } 61 | } 62 | 63 | pub struct Collect<'a, I> where I : 'a { 64 | index: Option, 65 | val: &'a mut Box> 66 | } 67 | 68 | impl<'a, I> Collect<'a, I> where I : 'a { 69 | 70 | pub fn new(v : &'a mut Box>) -> Collect<'a, I> { 71 | Collect { 72 | index: None, 73 | val: v 74 | } 75 | } 76 | } 77 | 78 | impl<'a, I> Subscriber for Collect<'a, I> where I : 'a { 79 | type Input = I; 80 | 81 | fn on_next(&mut self, t: I) -> bool { 82 | self.val.push(t); 83 | true 84 | } 85 | } 86 | 87 | -------------------------------------------------------------------------------- /test/test.rs: -------------------------------------------------------------------------------- 1 | #![feature(unboxed_closures)] 2 | 3 | extern crate mio; 4 | extern crate iobuf; 5 | extern crate alloc; 6 | extern crate time; 7 | 8 | #[macro_use] 9 | extern crate rx; 10 | 11 | #[macro_use] 12 | extern crate log; 13 | 14 | #[macro_use] 15 | extern crate lazy_static; 16 | 17 | use iobuf::{RWIobuf, AROIobuf, Iobuf, Allocator}; 18 | use mio::{EventLoop, EventLoopSender, Token}; 19 | 20 | use alloc::heap as a; 21 | use std::sync::Arc; 22 | 23 | use rx::reactive::{Publisher, Subscriber}; 24 | use rx::publisher::{IterPublisher, Coupler}; 25 | use rx::subscriber::{StdoutSubscriber, Decoupler}; 26 | use rx::processor::{MapVal1, Map, DebugWhile, Enumerate}; 27 | use rx::sendable::{Sendable}; 28 | use rx::reactor::{StreamBuf, ProtoMsg, NetEngine, Reactor}; 29 | use rx::protocol::{BufProtocol, HasSize}; 30 | 31 | use std::mem; 32 | use std::str; 33 | use std::old_io::Timer; 34 | use std::time::Duration; 35 | use std::fmt; 36 | use time::precise_time_ns; 37 | use std::sync::mpsc::channel; 38 | use std::thread::Thread; 39 | //static BUF_SIZE : uisize = 1600; 40 | 41 | struct MyAllocator; 42 | 43 | 44 | impl Allocator for MyAllocator { 45 | fn allocate(&self, size: usize, align: usize) -> *mut u8 { 46 | unsafe { a::allocate(size, align) } 47 | } 48 | 49 | fn deallocate(&self, ptr: *mut u8, len: usize, align: usize) { 50 | unsafe { a::deallocate(ptr, len, align) } 51 | } 52 | } 53 | 54 | lazy_static! { 55 | static ref MYALLOC: Arc> = { 56 | Arc::new( (Box::new(MyAllocator) as Box)) 57 | }; 58 | } 59 | 60 | protocol_size!(SixtyFour = 64); 61 | 62 | #[test] 63 | fn main() { 64 | 65 | println!("You are here"); 66 | 67 | let mut ne = NetEngine::>::new(); 68 | type Msg = ProtoMsg; 69 | 70 | let srv_rx = ne.listen("127.0.0.1", 10000).unwrap(); 71 | let cli = ne.connect("127.0.0.1", 10000).unwrap(); 72 | 73 | let token = cli.tok.clone(); 74 | let dtx = cli.dtx.clone(); 75 | 76 | ne.timeout(Duration::milliseconds(1000), Box::new(|&: el : &mut Reactor| { el.shutdown(); true})); 77 | 78 | let out = move |:| { 79 | let mut rec = Box::new(Coupler::new(srv_rx)); 80 | let mut map1 = Box::new(Map::new(| tup | -> (Msg, u64) { (tup, precise_time_ns()) })); 81 | let mut red = Box::new(Enumerate::new()); 82 | let mut trace = Box::new(DebugWhile::new(| &(_,count) : &((Msg, u64), u64) | { count % 10000 == 0 })); 83 | let mut map2 = Box::new(MapVal1::new(token, |((ProtoMsg (buf, _),_),_) : ((Msg, u64), u64), t: &Token| -> StreamBuf { StreamBuf (buf, *t) })); 84 | // change the token from t to the token of the connection, that way Decoupler 85 | // will tell eventloop to send it back out of the tcp connections's pipe 86 | let mut sen = Box::new(Decoupler::new(dtx)); 87 | 88 | println!("before the thingy"); 89 | map2.subscribe(sen); 90 | trace.subscribe(map2); 91 | red.subscribe(trace); 92 | map1.subscribe(red); 93 | rec.subscribe(map1); 94 | 95 | rec.run(); 96 | }; 97 | 98 | let guard = Thread::spawn(out); 99 | 100 | let mut buf = RWIobuf::from_str_copy_with_allocator("AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz012345678901", MYALLOC.clone()); 101 | cli.dtx.send( StreamBuf (buf.atomic_read_only().unwrap(), token)).unwrap(); 102 | // Start the event loop 103 | ne.run(); 104 | } 105 | --------------------------------------------------------------------------------