├── README.md ├── scripts ├── make_hpp └── simple_rpc ├── setup.py ├── simple_rpc ├── __init__.py ├── include │ ├── SimpleRPC.cpp │ ├── SimpleRPC.hpp │ ├── Socket.hpp │ ├── stdlib.cpp │ └── stdlib.hpp ├── interface_generator.py ├── main.py ├── parser.py ├── templates.py └── utils.py └── tests ├── Makefile ├── example_dot.cpp ├── example_dot.hpp ├── example_dot_app.cpp ├── std_scalars.cpp ├── std_scalars.hpp ├── std_scalars_app.cpp └── stdlib_app.cpp /README.md: -------------------------------------------------------------------------------- 1 | This project initiated from the following practical problem. To control experimental equipment via computers, manufactures provide software drivers with SDK written in C/C++. Unfortunately, Windows platform is often the only supported platform or support for other platforms stays years behind. In order to control such equipment from a Linux platform, for instance, Remote Procedure Calling (RPC) techniques can be used. 2 | 3 | _Simple RPC C++_ project provides a Python script that generates wrappers to C/C++ functions and RPC server applications so that the wrapped C/C++ functions could be called from an application (local host) but the C/C++ functions are executed on a server application (remote host). 4 | 5 | The usage of word "simple" in the name of this project is due to the following: 6 | 7 | * _RPC technology can be simple to use_ --- editing existing code should be minimal in order to remotely call C/C++ functions. Just changing the name of a header file should be sufficient to adapt an application to use RPC for certain C/C++ functions. Note that existing RPC libraries require RPC specific client applications; the current project aims at requiring minimal modification to client applications so that switching from calling C/C++ functions directly or via RPC, would be straightforward. 8 | 9 | * _Simplification can be sufficient for practical applications_ --- C/C++ functions that can be wrapped for RPC, are assumed to have "simple" prototypes, that is, the arguments should have types that Boost serialization library can handle. This includes all standard C/C++ scalar types and standard STL collections. These types can be defined in arguments as is, or as references, or as pointers. Notice that pointer arguments are treated in application program (in local host) as scalars that can be passed to RPC server program where they point to actual memory location of the remote host. 10 | 11 | * _Simplicity goes with restrictions_ --- For instance, wrapping C++ template functions is not supported. In addition, when running an application and RPC server on hosts with different architecture or operating system, special care must be taken to ensure that the data exchange between local and remote host is correct. For instance, the size of standard C/C++ `int` type may be different on local and remote hosts leading to possible corruption of data. To avoid this, always use scalar types with fixed byte width, for instance, `int32_t`, `uint64_t`, etc. Finally, no care is taken when the endian of local and remote hosts differ.. 12 | 13 | # Installation # 14 | 15 | The project assumes availability of Python 2.x. Python 3.x support might require minor modifications in the project source code, patches are welcome. 16 | 17 | Checkout the project source code, see [Source/Checkout](https://code.google.com/p/simple-rpc-cpp/source/checkout) for instructions. In the source code directory, run 18 | ``` 19 | sudo python setup.py install 20 | ``` 21 | Windows users should run `python setup.py install` from command prompt. 22 | 23 | 24 | To verify success, execute 25 | ``` 26 | cd tests 27 | make 28 | ``` 29 | If the tests pass, you can try out the "Example" project below. 30 | 31 | # Example # 32 | 33 | A typical C/C++ project consists of three parts: application source file containing the `main` function, source file(s) of computational functions, and header file(s) of the corresponding prototypes. 34 | All source files are compiled and linked together resulting an application program. 35 | When running this application program, the `main` function calls computational functions 36 | and outputs the results. Our aim is that when `main` function calls computational functions then they are actually executed in remote host. For that we use RPC technique to send function arguments and then receive function results over network. The current project _Simple RPC C++_ provides a Python script `simple_rpc` that constructs RPC wrapper functions to computational functions and generates the corresponding RPC source code. By design, modifications to existing source files are minimal. 37 | In fact, in a typical usage case only the application source file requires minor editing while no modifications are needed to the rest of source or header files, as will be demonstrated below. 38 | 39 | To illustrate this, consider the following example project consisting of three files: 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 61 | 75 | 89 | 90 |
Source file: example_dot.cppHeader file: example_dot.hppApplication source file: example_dot_app.cpp
48 | 
#include "example_dot.hpp"


 49 | 


 50 | double dot_product(const vector<double> & a,


 51 | 		   const vector<double> & b


 52 | 		   )


 53 | {


 54 |   double sum = 0;


 55 |   for (int i = 0; i < a.size(); i++)


 56 |     sum += a[i] * b[i];


 57 |   return sum;


 58 | }


 59 |
60 | 		62 | 
#ifndef EXAMPLE_DOT_HPP_DEFINED


 63 | #define EXAMPLE_DOT_HPP_DEFINED


 64 | 


 65 | #include <vector>


 66 | using namespace std;


 67 | 


 68 | double dot_product(const vector<double> & a,


 69 | 		   const vector<double> & b


 70 | 		   );


 71 | 


 72 | #endif


 73 |
74 | 		76 | 
#include <iostream>


 77 | 


 78 | #include "example_dot.hpp"


 79 | 


 80 | main()


 81 | {


 82 |   vector<double> a(3);


 83 |   a[0] = 1.2; a[1] = 3.4; a[2] = 5.6;


 84 |   cout << "dot_product(a,a) -> ";


 85 |   cout << dot_product(a,a) << endl;


 86 | }


 87 |
88 |
91 | The corresponding application can be compiled and executed: 92 | ``` 93 | $ c++ example_dot.cpp example_dot_app.cpp -o example_dot_app 94 | $ ./example_dot_app 95 | dot_product(a,a) -> 44.36 96 | ``` 97 | 98 | In order to call `dot_product` via RPC server, first, we generate wrapper codes: 99 | ``` 100 | $ simple_rpc example_dot.hpp 101 | Creating RPC wrappers to functions: 102 | double dot_product(const vector & a, 103 | const vector & b 104 | ) 105 | creating file example_dot-rpc.hpp 106 | creating file example_dot-rpc.cpp 107 | creating file example_dot-rpc-server.cpp 108 | ``` 109 | Notice that the `simple_rpc` script takes header files for input and generates 110 | three files as shown above. These files will be used to compile and build two 111 | executable programs: one for application and one for RPC server. 112 | 113 | Next, we will modify the application source code as follows. 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 135 | 149 | 168 | 169 |
Source file: example_dot.cppHeader file: example_dot.hppApplication source file: example_dot_app.cpp
122 | 
#include "example_dot.hpp"


123 | 


124 | double dot_product(const vector<double> & a,


125 | 		   const vector<double> & b


126 | 		   )


127 | {


128 |   double sum = 0;


129 |   for (int i = 0; i < a.size(); i++)


130 |     sum += a[i] * b[i];


131 |   return sum;


132 | }


133 |
134 | 		136 | 
#ifndef EXAMPLE_DOT_HPP_DEFINED


137 | #define EXAMPLE_DOT_HPP_DEFINED


138 | 


139 | #include <vector>


140 | using namespace std;


141 | 


142 | double dot_product(const vector<double> & a,


143 | 		   const vector<double> & b


144 | 		   );


145 | 


146 | #endif


147 |
148 | 		150 | 
#include <iostream>


151 | 


152 | #include "example_dot-rpc.hpp"              // <-- (1)


153 | 


154 | #ifdef SimpleRPC                            // <-- (2)


155 | using namespace simple_rpc::example_dot;    // <-- (2)


156 | #endif                                      // <-- (2)


157 | 


158 | main()


159 | {


160 |   SIMPLE_RPC_CONNECT("127.0.0.1", 2340, 2); // <-- (3)


161 |   vector<double> a(3);


162 |   a[0] = 1.2; a[1] = 3.4; a[2] = 5.6;


163 |   cout << "dot_product(a,a) -> ";


164 |   cout << dot_product(a,a) << endl;


165 | }


166 |
167 |
170 | The application source code is modified in three places indicated with `// <-- (#)`: 171 | 1. The name of the header file is appended with `-rpc` suffix. The generated header file contains RPC wrapper functions to functions found in the original header file. 172 | 1. `ifdef SimpleRPC` block is inserted. This is done for convenience as it will make easy to disable (use `-DDISABLE_SIMPLE_RPC` when compiling application code) and enable RPC in the application code. Following `using namespace` declaration exposes the RPC wrapper of `compute_dot` function to current namespace. In general, RPC wrappers are defined in namespace `simple_rpc::`. 173 | 1. `SIMPLE_RPC_CONNECT` macro is used to specify the RPC server host IP, port number and debug level (0 means no debug messages are shown, increasing this number will increase verbosity). Here we use debug level 2 in order to illustrate the connection between the application code and RPC server, see below. 174 | 175 | Next, the application and the RPC server programs must be built. In the given example 176 | we use RPC server local host (the corresponding host IP is 127.0.0.1) but, in general, 177 | the application program must be built on local host while the RPC server program on 178 | server host, especially, if these hosts run different operating systems. 179 | The following table illustrates the build process: 180 | 181 | 182 | 183 | 184 | 185 | 186 | 193 | 200 | 201 |
Remote hostLocal host
187 | 
$ c++ example_dot.cpp example_dot-rpc-server.cpp \


188 |    -o example_dot-rpc-server -pthread \


189 |    -lboost_system -lboost_serialization \


190 |    -I`simple_rpc --include-dir`


191 |
192 | 		194 | 
$ c++ example_dot-rpc.cpp example_dot_app.cpp \


195 |    -o example_dot_app-rpc -pthread \


196 |    -lboost_system -lboost_serialization \


197 |    -I`simple_rpc --include-dir`


198 |
199 |
202 | Notice that all source codes of functions must be compiled on remote host while the application source code with RPC wrapper source is compiled on local host. 203 | 204 | Finally, we run the RPC server in remote host, and subsequently, run the application program: 205 | 206 | 207 | 208 | 209 | 210 | 211 | 222 | 230 | 231 |
Remote hostLocal host
212 | 
$ ./example_dot-rpc-server


213 | rpc-server[2] waits connection via port 2340...connected!


214 | rpc-server:read_scalar<j>(a.size) <- 4 bytes


215 | rpc-server:read_vector<d>(a) <- 24 bytes


216 | rpc-server:read_scalar<j>(b.size) <- 4 bytes


217 | rpc-server:read_vector<d>(b) <- 24 bytes


218 | rpc-server:write_buffer_list(dot_product(a, b)) -> 16 bytes


219 | rpc-server[3] waits connection via port 2340...


220 |
221 | 		223 | 
$ ./example_dot_app-rpc


224 | set_debug_level:write_buffer_list(set_debug_level(debug_level)) -> 4 bytes


225 | dot_product:write_buffer_list(dot_product(a, b)) -> 56 bytes


226 | dot_product:read_scalar<d>(return_value) <- 8 bytes


227 | dot_product(a,a) -> 44.36


228 |
229 |
232 | 233 | Notice that RPC server can run continuously and different application programs can 234 | execute functions from the server. This will work only when different application 235 | programs will not execute the server functions at the same time. Server will serve the first application and during the time of execution, connections to the server by other applications will be declined. 236 | -------------------------------------------------------------------------------- /scripts/make_hpp: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- python-mode -*- 3 | 4 | import re 5 | import os 6 | import sys 7 | 8 | re_prototype = re.compile(r'(?P([\w_:<>*\s]+|))\s+(?P[\w_:]+)\s*\((?P[^)]*)\)\s*({|;)', re.I | re.S | re.M) 9 | re_include = re.compile(r'\s*#\s*include\s*[<][\w_-]+[>]') 10 | filename = sys.argv[1] 11 | assert filename.endswith ('cpp'),`filename` 12 | 13 | prototypes = [] 14 | includes = [] 15 | text = '' 16 | f = open (filename, 'r') 17 | for line in f.readlines (): 18 | i = line.find('//') 19 | if i!=-1: 20 | line = line[:i] + '\n' 21 | text += line 22 | f.close () 23 | for m in re_prototype.finditer(text): 24 | i = m.start () 25 | prev = text[max (0,i-100):i].rstrip(' ') 26 | prototypes.append(m.group (0)[:-1].rstrip()) 27 | for m in re_include.finditer(text): 28 | includes.append(m.group(0).strip()) 29 | 30 | prototypes = ';\n'.join(prototypes) + ';' 31 | includes = '\n'.join(includes) 32 | 33 | name = os.path.splitext (filename)[0] 34 | headername = name + '.hpp' 35 | NAME = name.upper () 36 | 37 | print 'writing', headername 38 | f = open (headername, 'w') 39 | f.write(''' 40 | #ifndef %(NAME)s_HPP_DEFINED 41 | #define %(NAME)s_HPP_DEFINED 42 | 43 | %(includes)s 44 | 45 | %(prototypes)s 46 | 47 | #endif 48 | ''' % locals ()) 49 | f.close () 50 | -------------------------------------------------------------------------------- /scripts/simple_rpc: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- python-mode -*- 3 | from simple_rpc import main 4 | main() -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from glob import glob 4 | import os 5 | if os.path.exists('MANIFEST'): os.remove('MANIFEST') 6 | 7 | def getfile(): 8 | try: 9 | return __file__ 10 | except NameError: 11 | import inspect 12 | return inspect.getsourcefile(getfile) 13 | setup_dir = os.path.dirname(getfile()) 14 | print setup_dir 15 | from numpy.distutils.core import setup 16 | 17 | setup(name='simple_rpc', 18 | version='0.1', 19 | description='Simple RPC C++ --- a simple RPC wrapper generator to C/C++ functions', 20 | author='Pearu Peterson', 21 | author_email='pearu.peterson@gmail.com', 22 | url='http://code.google.com/p/simple-rpc-cpp/', 23 | packages=['simple_rpc'], 24 | scripts = [os.path.join(setup_dir, 'scripts/simple_rpc')], 25 | data_files=[('simple_rpc/include', glob(os.path.join(setup_dir,'simple_rpc','include','*.hpp')))] 26 | ) 27 | -------------------------------------------------------------------------------- /simple_rpc/__init__.py: -------------------------------------------------------------------------------- 1 | """ simple_rpc --- a simple RPC wrapper generator to C/C++ functions. 2 | 3 | See http://code.google.com/p/simple-rpc-cpp/ for more information. 4 | """ 5 | # Author: Pearu Peterson 6 | # Created: August 2012 7 | 8 | __all__ = ['main'] 9 | 10 | from main import main 11 | -------------------------------------------------------------------------------- /simple_rpc/include/SimpleRPC.cpp: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /simple_rpc/include/SimpleRPC.hpp: -------------------------------------------------------------------------------- 1 | /* 2 | Provides SimpleRPC class to hold host:port information and to 3 | provide establish_connection method. 4 | 5 | Author: Pearu Peterson 6 | Created: September 2012 7 | */ 8 | 9 | #ifndef SIMPLE_RPC_HPP_DEFINED 10 | #define SIMPLE_RPC_HPP_DEFINED 11 | 12 | #include 13 | #include 14 | #include 15 | #include "Socket.hpp" 16 | 17 | #ifndef SimpleRPC 18 | #error "Must define -DSimpleRPC=SimpleRPC_" 19 | #endif 20 | 21 | #define SIMPLE_RPC_CONNECT(HOST, PORT, DEBUG_LEVEL) \ 22 | simple_rpc::SimpleRPC rpc(HOST, PORT, DEBUG_LEVEL); \ 23 | if (!rpc) \ 24 | { \ 25 | std::cout << "Failure in creating connection to RPC server. Quiting," << std::endl; \ 26 | exit(1); \ 27 | } 28 | 29 | namespace simple_rpc 30 | { 31 | 32 | void set_debug_level(const int & debug_level); 33 | 34 | class SimpleRPC 35 | { 36 | 37 | private: 38 | static std::string host; 39 | static unsigned short port; 40 | static bool success; 41 | static int debug_level; 42 | bool m_success; 43 | 44 | public: 45 | static int get_debug_level(void) { return debug_level; } 46 | 47 | static 48 | bool establish_connection(boost::asio::io_service& io_service, Socket & socket) 49 | { 50 | boost::asio::ip::tcp::resolver resolver(io_service); 51 | boost::asio::ip::tcp::resolver::query query(host, 52 | boost::lexical_cast(port)); 53 | success = true; 54 | try 55 | { 56 | boost::asio::ip::tcp::resolver::iterator endpoint_iterator = resolver.resolve(query); 57 | boost::asio::connect(socket, endpoint_iterator); 58 | } 59 | catch (std::exception& e) 60 | { 61 | success = false; 62 | std::cerr << "establish_connection["< 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | #include 25 | 26 | namespace simple_rpc 27 | { 28 | 29 | class Socket: public boost::asio::ip::tcp::socket 30 | { 31 | private: 32 | int m_debug_level; 33 | std::string m_appname; 34 | 35 | private: 36 | template size_t size_bytes(const std::vector& value) { return sizeof(T) * value.size(); } 37 | size_t size_bytes(const std::list& value) 38 | { 39 | size_t sz = 0; 40 | for (std::list::const_iterator it = value.begin(); it != value.end(); it++) 41 | sz += boost::asio::buffer_size(*it); 42 | return sz; 43 | } 44 | size_t size_bytes(const std::string& value) { return sizeof(char)*value.size(); } 45 | 46 | public: 47 | Socket(boost::asio::io_service& io_service, std::string appname = "", int debug_level = 0) : 48 | boost::asio::ip::tcp::socket(io_service), 49 | m_appname(appname), 50 | m_debug_level(debug_level) 51 | {} 52 | 53 | bool write_buffer_list(const std::list< boost::asio::const_buffer > &value, 54 | const std::string & name = "") 55 | { 56 | boost::system::error_code error; 57 | size_t len = boost::asio::write(*this, value, error); 58 | if (error) 59 | { 60 | std::cerr<1) 69 | std::cout< "< bool write_scalar(const T &value, const std::string & name = "", int delta_debug = 0) 74 | { 75 | boost::system::error_code error; 76 | size_t len = boost::asio::write(*this, boost::asio::buffer( &value, sizeof(T) ), error); 77 | if (error) 78 | { 79 | std::cerr<("<("<1) 88 | std::cout<("< "< bool read_scalar(T &value, const std::string & name = "", int delta_debug = 0) 93 | { 94 | boost::system::error_code error; 95 | size_t len = boost::asio::read( *this, boost::asio::buffer( &value, sizeof(T) ), error); 96 | if (error) 97 | { 98 | std::cerr<("<("<1) 107 | std::cout<("< bool write_serial(const T &value, const std::string & name = "") 112 | { 113 | std::ostringstream archive_stream; 114 | boost::archive::text_oarchive archive(archive_stream); 115 | archive << value; 116 | return write_string(archive_stream.str(), name + "_serialized"); 117 | 118 | } 119 | */ 120 | template bool read_serial(T &value, const std::string & name = "") 121 | { 122 | std::string inbound_data; 123 | if (!read_string(inbound_data, name + "_serialized")) 124 | return false; 125 | std::istringstream archive_stream(inbound_data); 126 | boost::archive::text_iarchive archive(archive_stream); 127 | archive >> value; 128 | return true; 129 | } 130 | 131 | /* 132 | template 133 | bool write_vector(const std::vector &value, 134 | const std::string & name = "") 135 | { 136 | std::list< boost::asio::const_buffer > buffers; 137 | uint32_t sz = value.size(); 138 | buffers.push_back( boost::asio::buffer( &sz, sizeof(sz) ) ); 139 | buffers.push_back( boost::asio::buffer( value ) ); 140 | return write_buffer_list(buffers, std::string("vector<") + typeid(T).name() + ">" + name); 141 | } 142 | */ 143 | 144 | template bool read_vector(std::vector &value, const std::string & name = "") 145 | { 146 | boost::system::error_code error; 147 | uint32_t sz = 0; 148 | if (!read_scalar(sz, name + ".size")) 149 | return false; 150 | value.resize(sz); 151 | size_t len = boost::asio::read( *this, boost::asio::buffer( value ), error); 152 | if (error) 153 | { 154 | std::cerr<("<("<1) 163 | std::cout<("< buffers; 172 | uint32_t sz = value.size(); 173 | buffers.push_back( boost::asio::buffer( &sz, sizeof(sz) ) ); 174 | buffers.push_back( boost::asio::buffer( value ) ); 175 | 176 | std::string new_name("string "); 177 | new_name += name; 178 | return write_buffer_list(buffers, "string " + name); 179 | } 180 | */ 181 | 182 | bool read_string(std::string &value, const std::string & name = "") 183 | { 184 | boost::system::error_code error; 185 | uint32_t sz = 0; 186 | if (!read_scalar(sz, name + ".size")) 187 | return false; 188 | std::vector data(sz); 189 | size_t len = boost::asio::read( *this, boost::asio::buffer( data, sz ), error); 190 | if (error) 191 | { 192 | std::cerr<1) 202 | std::cout< 3 | #include "stdlib.hpp" 4 | 5 | int system ( const std::string & command ) { return system(command.c_str()); } 6 | -------------------------------------------------------------------------------- /simple_rpc/include/stdlib.hpp: -------------------------------------------------------------------------------- 1 | /* 2 | stdlib.hpp 3 | 4 | Author: Pearu Peterson 5 | Created: September 2012 6 | */ 7 | 8 | 9 | #ifndef SIMPLE_RPC_STDLIB_HPP_DEFINED 10 | #define SIMPLE_RPC_STDLIB_HPP_DEFINED 11 | 12 | #include 13 | #include 14 | 15 | int system ( const std::string & command ); 16 | 17 | #ifdef SimpleRPC 18 | /* This is special hack to wrap standard C/C++ library functions, in 19 | general. Since the corresponding header files are convolved 20 | (prototypes are hard to find and may depend on CPP macro 21 | definitions), here we define few stdlib function prototypes 22 | that simple_rpc will use to construct the corresponding RPC 23 | wrapper functions. These prototypes should not be visibile 24 | to RPC server code in order to avoid declaration conflicts 25 | but must be exposed to RPC client code as they are prototypes 26 | of wrapper functions (living in simple_rpc::stdlib namespace). 27 | Hence the usage of `#ifdef SimpleRPC`. 28 | */ 29 | 30 | void exit ( int status ); 31 | int system ( const char * command ); 32 | 33 | void * malloc ( size_t size ); 34 | void * memset ( void * ptr, int value, size_t num ); 35 | void free ( void * ptr ); 36 | #endif 37 | 38 | #endif 39 | -------------------------------------------------------------------------------- /simple_rpc/interface_generator.py: -------------------------------------------------------------------------------- 1 | 2 | # Author: Pearu Peterson 3 | # Created: August 2012 4 | 5 | import re 6 | from utils import tabulate, collect, joinlist, uniquestr, nextint 7 | import templates 8 | 9 | 10 | 11 | def isvector (typespec): 12 | return re.search(r'(\A|(? %(srpc)sbuffers;' %(locals ())) 105 | send_arguments = '%(srpc)ssocket.write_buffer_list(%(srpc)sbuffers, "%(function_name)s(%(variables)s)")' % (locals()) 106 | 107 | 108 | if (buffers_add_results): 109 | buffers_add_results.insert(0, 'std::list< boost::asio::const_buffer > %(srpc)sbuffers;' %(locals ())) 110 | buffers_add_results.append ('%(srpc)sbuffers.push_back( boost::asio::buffer( &%(srpc)sconnection_magic, sizeof(%(srpc)sconnection_magic) ) );' % (locals())) 111 | send_results = ('%(srpc)ssocket.write_buffer_list(%(srpc)sbuffers, "%(function_name)s(%(variables)s)")' % (locals())) 112 | else: 113 | send_results = ('%(srpc)ssocket.write_scalar(%(srpc)sconnection_magic, "connection_magic", -1)' % (locals ())) 114 | 115 | if not recieve_results: recieve_results = 'true' 116 | if not recieve_arguments: recieve_arguments = 'true' 117 | 118 | function_prototype = '%(return_type)s %(namespace)s::%(function_name)s(%(typespec_names)s)' % locals() 119 | special_prototype = ' %(return_type)s %(function_name)s(%(typespec_names)s);' % locals() 120 | 121 | server_magic = uniquestr(str2magic(server_name)) 122 | function_magic = str2magic(function_prototype) 123 | if body is not None: 124 | function_call = body % (locals()) 125 | server_switch_case = templates.server_switch_case % (locals ()) 126 | 127 | return dict( 128 | srpc = srpc, # manglin prefix 129 | server_magic = server_magic, 130 | wrapper_function_prototype = function_prototype, 131 | function_implementation = templates.function_implementation % (locals()), 132 | server_switch_case = tabulate(server_switch_case, tabs=10), 133 | special_prototype = special_prototype, 134 | ) 135 | -------------------------------------------------------------------------------- /simple_rpc/main.py: -------------------------------------------------------------------------------- 1 | 2 | # Author: Pearu Peterson 3 | # Created: August 2012 4 | 5 | import argparse 6 | import sys 7 | import os 8 | from collections import defaultdict 9 | from parser import parse_prototypes 10 | from utils import collect 11 | from interface_generator import make_interface_source 12 | import templates 13 | 14 | def main(): 15 | parser = argparse.ArgumentParser(description='Simple RPC code generator.') 16 | parser.add_argument("-n", "--name", help="name of the RPC server code", 17 | type=str) 18 | parser.add_argument('filepath', type=str, nargs='*', 19 | help='path to C++ header files that are scanned for prototypes') 20 | parser.add_argument('--functions', type=str, 21 | help='comma separated list of functions to be wrapped') 22 | parser.add_argument('--include-dir', action='store_true') 23 | args = parser.parse_args() 24 | if args.include_dir: 25 | print os.path.join(os.path.dirname (__file__), 'include') 26 | return 27 | prototypes = defaultdict(list) 28 | for filename in args.filepath: 29 | original_filename = os.path.basename(filename) 30 | prototypes[original_filename] += parse_prototypes(filename, args.functions) 31 | 32 | special_list = [ 33 | ('set_debug_level', 'void', [('debug_level', 'const int &')], '%(srpc)sdebug_level = debug_level;'), 34 | ('get_counter', 'uint64_t', [('', 'void')], '%(srpc)sreturn_value = %(srpc)scounter;'), 35 | ] 36 | 37 | for original_filename, prototype_list in prototypes.iteritems(): 38 | source_info = dict(FILENAME=os.path.basename(original_filename).upper().replace('.','_'), 39 | original_filename=os.path.basename(original_filename), 40 | namespace = os.path.splitext(os.path.basename(original_filename))[0], 41 | ) 42 | server_name = source_info['namespace'] 43 | source_info['NAMESPACE'] = source_info['namespace'].upper() 44 | 45 | for prototype in special_list: 46 | interface_dict = make_interface_source(server_name, 'simple_rpc', prototype) 47 | collect(source_info, interface_dict) 48 | 49 | for prototype in prototype_list: 50 | interface_dict = make_interface_source(server_name, 'simple_rpc::' + source_info['namespace'], prototype) 51 | del interface_dict['special_prototype'] 52 | collect(source_info, interface_dict) 53 | 54 | client_filename = os.path.splitext(original_filename)[0] + '-rpc.hpp' 55 | print 'creating file', client_filename 56 | f = open(client_filename, 'w') 57 | f.write(templates.client_header % (source_info)) 58 | f.close() 59 | 60 | client_filename = os.path.splitext(original_filename)[0] + '-rpc.cpp' 61 | print 'creating file', client_filename 62 | f = open(client_filename, 'w') 63 | f.write(templates.client_source % (source_info)) 64 | f.close() 65 | 66 | server_filename = os.path.splitext(original_filename)[0] + '-rpc-server.cpp' 67 | print 'creating file', server_filename 68 | f = open(server_filename, 'w') 69 | f.write(templates.server_source % (source_info)) 70 | f.close() 71 | 72 | if __name__=='__main__': 73 | main () 74 | -------------------------------------------------------------------------------- /simple_rpc/parser.py: -------------------------------------------------------------------------------- 1 | 2 | # Author: Pearu Peterson 3 | # Created: August 2012 4 | 5 | import os 6 | import sys 7 | import re 8 | 9 | re_prototype = re.compile(r'(?P([\w_:<>*\s]+|))\s+(?P[\w_:]+)\s*\(\s*(?P[^)]*)\s*\)\s*({|;)', re.I | re.S) 10 | 11 | re_namespace = re.compile(r'namespace\s+(?P\w+)\s*{(?P.*)}', re.I | re.S) 12 | 13 | def parse_string (text, namespace = [], functions=None): 14 | prototypes = [] 15 | for m in re_namespace.finditer(text): 16 | prototypes += parse_string(m.group('content'), 17 | namespace = namespace + [m.group('namespace')], 18 | functions = functions) 19 | text = text[:m.start()] + text[m.end()] 20 | for m in re_prototype.finditer(text): 21 | fname = m.group('fname') 22 | if functions is not None and fname not in functions: 23 | continue 24 | rtype = m.group('rtype').strip() 25 | args = [] 26 | names = [] 27 | for a in m.group ('args').split(','): 28 | a = a.strip() 29 | aname = None 30 | for i,c in enumerate (reversed(a)): 31 | if not (c.isalnum () or c=='_'): 32 | aname = a[-i:].strip() 33 | atype = a[:-i].strip() 34 | break 35 | if aname is None: 36 | aname = '' 37 | atype = a 38 | args.append((aname, atype)) 39 | names.append(aname) 40 | ns = '::'.join (namespace) 41 | print ' ', ns, m.group(0)[:-1].strip()#, (fname, rtype, args, None) 42 | 43 | if ns.startswith('simple_rpc::'): 44 | if rtype=='void': 45 | body = '%s(%s);' % (fname, ', '.join (names)) 46 | else: 47 | body = '%%(srpc)sreturn_value = %s(%s);' % (fname, ', '.join (names)) 48 | fname = ns + '::' + fname 49 | prototypes.append ((fname, rtype, args, body)) 50 | else: 51 | prototypes.append ((fname, rtype, args, None)) 52 | return prototypes 53 | 54 | def parse_prototypes(filename, functions = None): 55 | """ Scan filename for C++ functions and prototypes. 56 | 57 | Parameters 58 | ---------- 59 | filename : str 60 | File name of a C++ source or header file. 61 | 62 | Return 63 | ------ 64 | prototypes : list 65 | A list of `(, , , )` tuples 66 | where `` is a list of `(, )` tuples. 67 | `` is None. 68 | """ 69 | text = '' 70 | f = open (filename, 'r') 71 | for line in f.readlines (): 72 | i = line.find('//') 73 | if i!=-1: 74 | line = line[:i] + '\n' 75 | if line.startswith ('#'): 76 | continue 77 | text += line 78 | f.close() 79 | 80 | # TODO: get rid of /*..*/ comment blocks 81 | print 'Creating RPC wrappers to functions:' 82 | return parse_string(text, [], functions = functions) 83 | 84 | if __name__=='__main__': 85 | filename = sys.argv[1] 86 | print parse_prototypes(filename) 87 | -------------------------------------------------------------------------------- /simple_rpc/templates.py: -------------------------------------------------------------------------------- 1 | client_header = ''' 2 | /* 3 | This file is generated using simple_rpc script. 4 | 5 | For more information, see 6 | http://code.google.com/p/simple-rpc-cpp/ 7 | */ 8 | 9 | #ifndef %(FILENAME)s_RPC_HPP_DEFINED 10 | #define %(FILENAME)s_RPC_HPP_DEFINED 11 | #ifdef DISABLE_SIMPLE_RPC 12 | #define SIMPLE_RPC_CONNECT(HOST, PORT, DEBUG_LEVEL) 13 | #include "%(original_filename)s" 14 | #else 15 | #define SimpleRPC SimpleRPC_%(namespace)s 16 | #include "SimpleRPC.hpp" 17 | namespace simple_rpc 18 | { 19 | %(special_prototypes)s 20 | namespace %(namespace)s 21 | { 22 | #include "%(original_filename)s" 23 | } 24 | } 25 | #endif 26 | #endif 27 | ''' 28 | 29 | client_source = ''' 30 | /* 31 | This file is generated using simple_rpc script. 32 | 33 | For more information, see 34 | http://code.google.com/p/simple-rpc-cpp/ 35 | */ 36 | 37 | #include "%(namespace)s-rpc.hpp" 38 | 39 | namespace simple_rpc 40 | { 41 | 42 | int SimpleRPC::debug_level = 0; 43 | unsigned short SimpleRPC::port = 0; 44 | std::string SimpleRPC::host = ""; 45 | bool SimpleRPC::success = false; 46 | 47 | } 48 | 49 | %(function_implementations)s 50 | 51 | ''' 52 | 53 | buffers_add_scalar = ''' 54 | %(srpc)sbuffers.push_back( boost::asio::buffer( &%(name)s, sizeof(%(name)s) ) ); 55 | ''' 56 | 57 | buffers_add_vector = ''' 58 | uint32_t %(srpc)ssz_%(name)s = %(name)s.size(); 59 | %(srpc)sbuffers.push_back( boost::asio::buffer( &%(srpc)ssz_%(name)s, sizeof(%(srpc)ssz_%(name)s) ) ); 60 | %(srpc)sbuffers.push_back( boost::asio::buffer( %(name)s ) ); 61 | ''' 62 | 63 | buffers_add_string = buffers_add_vector 64 | 65 | buffers_add_serial = ''' 66 | std::ostringstream %(srpc)sarchive_stream_%(name)s; 67 | boost::archive::text_oarchive %(srpc)sarchive_%(name)s(%(srpc)sarchive_stream_%(name)s); 68 | %(srpc)sarchive_%(name)s << %(name)s; 69 | const std::string& %(srpc)sstring_%(name)s = %(srpc)sarchive_stream_%(name)s.str(); 70 | uint32_t %(srpc)ssz_%(name)s = %(srpc)sstring_%(name)s.size(); 71 | %(srpc)sbuffers.push_back( boost::asio::buffer( &%(srpc)ssz_%(name)s, sizeof(%(srpc)ssz_%(name)s) ) ); 72 | %(srpc)sbuffers.push_back( boost::asio::buffer( %(srpc)sstring_%(name)s ) ); 73 | ''' 74 | 75 | function_implementation = ''' 76 | %(function_prototype)s 77 | { 78 | static const uint32_t %(srpc)sfunction_magic = %(function_magic)s; 79 | static const uint32_t %(srpc)sexpected_server_magic = %(server_magic)s; 80 | boost::asio::io_service %(srpc)sio_service; 81 | simple_rpc::Socket %(srpc)ssocket(%(srpc)sio_service, "%(function_name)s", simple_rpc::SimpleRPC::get_debug_level()); 82 | if (simple_rpc::SimpleRPC::establish_connection(%(srpc)sio_service, %(srpc)ssocket)) 83 | { 84 | uint32_t %(srpc)sserver_magic = 0; 85 | uint64_t %(srpc)sconnection_magic = 0; 86 | uint64_t %(srpc)send_connection_magic = 0; 87 | uint64_t %(srpc)scounter = 0; 88 | if (%(srpc)ssocket.read_scalar (%(srpc)sserver_magic, "server_magic", -1) 89 | && (%(srpc)sserver_magic == %(srpc)sexpected_server_magic) // server is ready 90 | && %(srpc)ssocket.write_scalar(%(srpc)sfunction_magic, "function_magic", -1) 91 | && %(srpc)ssocket.read_scalar(%(srpc)sconnection_magic, "connection_magic", -1) 92 | && (%(srpc)scounter = %(srpc)sconnection_magic >> 32) 93 | && ((%(srpc)sconnection_magic & 0xffffffff)==%(srpc)sfunction_magic)) // server is knowledgeable 94 | { 95 | %(buffers_add_arguments)s 96 | if (%(send_arguments)s) 97 | { 98 | %(return_declaration)s 99 | if (%(recieve_results)s 100 | && %(srpc)ssocket.read_scalar(%(srpc)send_connection_magic, "end_connection_magic", -1) 101 | && (%(srpc)sconnection_magic==%(srpc)send_connection_magic) // check for surprises 102 | ) 103 | { 104 | %(return_statement)s 105 | } 106 | else 107 | std::cerr << "%(function_name)s-rpc["<<%(srpc)scounter<<"] ERROR: failed to recieve results" <0) 155 | std::cout << "rpc-server["<<(%(srpc)scounter+1)<<"] waits connection via port "<<%(srpc)sport<<"..."; std::cout.flush (); 156 | %(srpc)sacceptor.accept(%(srpc)ssocket); 157 | %(srpc)scounter ++; 158 | if (%(srpc)sdebug_level>0) 159 | std::cout << "connected!" << std::endl; 160 | 161 | if (%(srpc)ssocket.write_scalar(%(srpc)sserver_magic, "server_magic", -1) 162 | && %(srpc)ssocket.read_scalar(%(srpc)sfunction_magic, "function_magic", -1) 163 | && ((%(srpc)sconnection_magic = (%(srpc)scounter << 32) | %(srpc)sfunction_magic)) 164 | && %(srpc)ssocket.write_scalar(%(srpc)sconnection_magic, "connection_magic", -1) 165 | ) 166 | { 167 | switch (%(srpc)sfunction_magic) 168 | { 169 | %(server_switch_cases)s 170 | default : 171 | std::cerr << "rpc-server["<<%(srpc)scounter<<"] ERROR: unknown function_magic=="<<%(srpc)sfunction_magic< & a, 4 | const vector & b 5 | ) 6 | { 7 | double sum = 0; 8 | for (int i = 0; i < a.size(); i++) 9 | sum += a[i] * b[i]; 10 | return sum; 11 | } 12 | -------------------------------------------------------------------------------- /tests/example_dot.hpp: -------------------------------------------------------------------------------- 1 | #ifndef EXAMPLE_DOT_HPP_DEFINED 2 | #define EXAMPLE_DOT_HPP_DEFINED 3 | 4 | #include 5 | using namespace std; 6 | 7 | double dot_product(const vector & a, 8 | const vector & b 9 | ); 10 | 11 | #endif 12 | -------------------------------------------------------------------------------- /tests/example_dot_app.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | using namespace std; 3 | 4 | #include "example_dot-rpc.hpp" 5 | #ifdef SimpleRPC 6 | using namespace simple_rpc::example_dot; 7 | #endif 8 | 9 | main() 10 | { 11 | SIMPLE_RPC_CONNECT("127.0.0.1", 2340, 2); 12 | vector a(3); a[0] = 1.2; a[1] = 3.4; a[2] = 5.6; 13 | cout << "dot_product(a,a) -> " << dot_product(a,a) << endl; 14 | } 15 | -------------------------------------------------------------------------------- /tests/std_scalars.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include "std_scalars.hpp" 5 | 6 | bool getarg(bool a) { return a; } 7 | char getarg(char a) { return a; } 8 | short getarg(short a) { return a; } 9 | int getarg(int a) { return a; } 10 | long getarg(long a) { return a; } 11 | float getarg(float a) { return a; } 12 | double getarg(double a) { return a; } 13 | std::string getarg(const std::string& a) { return a; } 14 | 15 | void passarg(const std::string& a) { ; } 16 | 17 | std::vector getarg(const std::vector& a) { return a; } 18 | std::vector getarg(const std::vector& a) { return a; } 19 | std::vector getarg(const std::vector& a) { return a; } 20 | std::vector getarg(const std::vector& a) { return a; } 21 | std::vector getarg(const std::vector& a) { return a; } 22 | std::vector getarg(const std::vector& a) { return a; } 23 | std::vector getarg(const std::vector& a) { return a; } 24 | std::list getarg(const std::list& a) { return a; } 25 | 26 | double sum(const std::vector& a, 27 | const std::vector& b, 28 | const std::vector& c, 29 | const std::vector& d, 30 | const std::vector& e, 31 | const std::vector& f 32 | ) 33 | { 34 | double sum = 0; 35 | for /**/(int i=0; i& a, 41 | const std::vector& b 42 | ) 43 | { 44 | double sum = 0; 45 | for /**/(int i=0; i& a 51 | ) 52 | { 53 | double sum = 0; 54 | for /**/(int i=0; i& a, 60 | std::vector& b 61 | ) 62 | { 63 | double x; 64 | for /**/(int i=0; i 6 | #include 7 | #include 8 | 9 | 10 | 11 | bool getarg(bool a); 12 | 13 | char getarg(char a); 14 | 15 | short getarg(short a); 16 | 17 | int getarg(int a); 18 | 19 | long getarg(long a); 20 | 21 | float getarg(float a); 22 | 23 | double getarg(double a); 24 | 25 | std::string getarg(const std::string& a); 26 | 27 | 28 | void passarg(const std::string& a); 29 | 30 | 31 | std::vector getarg(const std::vector& a); 32 | 33 | std::vector getarg(const std::vector& a); 34 | 35 | std::vector getarg(const std::vector& a); 36 | 37 | std::vector getarg(const std::vector& a); 38 | 39 | std::vector getarg(const std::vector& a); 40 | 41 | std::vector getarg(const std::vector& a); 42 | 43 | std::vector getarg(const std::vector& a); 44 | 45 | std::list getarg(const std::list& a); 46 | 47 | 48 | double sum(const std::vector& a, 49 | const std::vector& b, 50 | const std::vector& c, 51 | const std::vector& d, 52 | const std::vector& e, 53 | const std::vector& f 54 | ); 55 | 56 | 57 | double sum(const std::vector& a, 58 | const std::vector& b 59 | ); 60 | 61 | 62 | double sum(const std::vector& a 63 | ); 64 | 65 | 66 | bool swap(std::vector& a, 67 | std::vector& b 68 | ); 69 | 70 | #endif 71 | -------------------------------------------------------------------------------- /tests/std_scalars_app.cpp: -------------------------------------------------------------------------------- 1 | 2 | #include "std_scalars-rpc.hpp" 3 | #ifdef SimpleRPC 4 | using namespace simple_rpc::std_scalars; 5 | #endif 6 | 7 | #include 8 | #include 9 | #include 10 | 11 | static unsigned int check_getarg_success_count = 0; 12 | static unsigned int check_getarg_failure_count = 0; 13 | 14 | template bool check_getarg(T a) 15 | { 16 | T r = getarg(a); 17 | bool success = (r == a); 18 | if (!success) 19 | { 20 | std::cout << "check_getarg failed: getarg<"<("<"< bool check_getarg_container(T& a) 29 | { 30 | T r = getarg(a); 31 | bool success = (r == a); 32 | if (!success) 33 | { 34 | std::cout << "check_getarg failed: getarg<"<(["<["< v(3); v[0] = 1; v[1] = 2; v[2] = 3; 56 | check_getarg_container(v); 57 | } 58 | { 59 | std::vector v(3); v[0] = 1; v[1] = 2; v[2] = 3; 60 | check_getarg_container(v); 61 | } 62 | { 63 | std::vector v(3); v[0] = 1; v[1] = 2; v[2] = 3; 64 | check_getarg_container(v); 65 | } 66 | { 67 | std::vector v(3); v[0] = 1; v[1] = 2; v[2] = 3; 68 | check_getarg_container(v); 69 | } 70 | { 71 | std::vector v(3); v[0] = 1.1; v[1] = 2.2; v[2] = 3.3; 72 | check_getarg_container(v); 73 | } 74 | { 75 | std::vector v(3); v[0] = 1.1; v[1] = 2.2; v[2] = 3.3; 76 | check_getarg_container(v); 77 | } 78 | { 79 | std::vector v(3); v[0] = "hello"; v[1] = "hey"; v[2] = "hi"; 80 | check_getarg_container(v); 81 | } 82 | { 83 | std::list v; v.push_back(1); v.push_back(2); v.push_back(3); 84 | check_getarg_container(v); 85 | } 86 | 87 | // 88 | std::cout<<"check_getarg "< v(300); v[0] = 1.1;// v[1] = 2.2; v[2] = 3.3; 108 | 109 | TIMEIT(elapsed, 110 | for (int i=0; i> timing=" < timing=" < timing=" < timing=" < timing=" < a(3); 169 | a[0] = 1.1; a[1] = 2.2; a[2] = 3.3; 170 | TIMEIT(elapsed, 171 | for (int i=0; i a(3); 182 | a[0] = 1.1; a[1] = 2.2; a[2] = 3.3; 183 | TIMEIT(elapsed, 184 | for (int i=0; i a(3); 195 | a[0] = 1.1; a[1] = 2.2; a[2] = 3.3; 196 | TIMEIT(elapsed, 197 | for (int i=0; i a(3), b(3,0); 208 | a[0] = 1.1; a[1] = 2.2; a[2] = 3.3; 209 | swap(a,b); 210 | if (!(a[0] == 0 && b[0] == 1.1)) 211 | { 212 | std::cout << "swap failed" << std::endl; 213 | return 0; 214 | } 215 | TIMEIT(elapsed, 216 | for (int i=0; i 5 | #include 6 | #include 7 | 8 | 9 | main() 10 | { 11 | SIMPLE_RPC_CONNECT("127.0.0.1", 2340, 0); 12 | 13 | #ifdef SimpleRPC 14 | using namespace simple_rpc; 15 | /* 16 | std::cout << "Calling exit.." << std::endl; 17 | stdlib::exit(1); // this will force RPC server to exit with given return status 18 | */ 19 | 20 | char * command = (char*)stdlib::malloc(3); 21 | stdlib::memset(command, 'l', 1); 22 | stdlib::memset(command+1, 's', 1); 23 | stdlib::memset(command+2, 0, 1); 24 | stdlib::system(command); 25 | 26 | stdlib::free(command); 27 | 28 | stdlib::system(std::string("pwd")); 29 | #endif 30 | std::cout << "EOF main" << std::endl; 31 | } 32 | --------------------------------------------------------------------------------