├── .gitignore ├── LICENSE ├── README.md ├── rgp ├── __init__.py ├── graph.py ├── t.py └── test.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # C extensions 6 | *.so 7 | 8 | # Distribution / packaging 9 | .Python 10 | env/ 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | *.egg-info/ 23 | .installed.cfg 24 | *.egg 25 | 26 | # PyInstaller 27 | # Usually these files are written by a python script from a template 28 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 29 | *.manifest 30 | *.spec 31 | 32 | # Installer logs 33 | pip-log.txt 34 | pip-delete-this-directory.txt 35 | 36 | # Unit test / coverage reports 37 | htmlcov/ 38 | .tox/ 39 | .coverage 40 | .coverage.* 41 | .cache 42 | nosetests.xml 43 | coverage.xml 44 | *,cover 45 | 46 | # Translations 47 | *.mo 48 | *.pot 49 | 50 | # Django stuff: 51 | *.log 52 | 53 | # Sphinx documentation 54 | docs/_build/ 55 | 56 | # PyBuilder 57 | target/ 58 | 59 | 60 | #osx 61 | 62 | .DS_Store 63 | .AppleDouble 64 | .LSOverride 65 | 66 | # Icon must end with two \r 67 | Icon 68 | 69 | 70 | # Thumbnails 71 | ._* 72 | 73 | # Files that might appear in the root of a volume 74 | .DocumentRevisions-V100 75 | .fseventsd 76 | .Spotlight-V100 77 | .TemporaryItems 78 | .Trashes 79 | .VolumeIcon.icns 80 | 81 | # Directories potentially created on remote AFP share 82 | .AppleDB 83 | .AppleDesktop 84 | Network Trash Folder 85 | Temporary Items 86 | .apdisk -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Mark Henderson 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # R.G.P. (Redis Graph via Python) 2 | 3 | RGP provides a simple directed graph database built on top of Redis and utilizes a set of Python classes as its interface. Both vertices and edges can have data and can be queried when traversing the graph. 4 | 5 | *Note:* Still in early beta -- the interface may change, there are no tests written, and no performance analysis has been done. This was my way of learning Redis, one of you may find it useful (I think that I will). 6 | 7 | ## Installation 8 | 9 | pip install -e rgp 10 | 11 | ## Requrements 12 | 13 | * Python 2.7 < 3 (3 support should be simple to do) 14 | * Redis 15 | * [redis-py](https://github.com/andymccurdy/redis-py) 16 | 17 | ## Data Structure 18 | 19 | * 'rgp:index' -- holds the node id. Incremented when a new `Node` is added 20 | * 'rgp:vertex' -- hold the hash data for a `Vertex` 21 | * 'rgp:vertex_all' -- 22 | * 'rgp:vertex_out' -- 23 | * 'rgp:vertex_in' -- 24 | * 'rgp:edge' -- 25 | * 'rgp:edge_all' -- 26 | * 'rgp:index' -- *NOT IMPLEMENTED YET* 27 | 28 | ## Usage 29 | 30 | RGP is simple, it is made up of a few core components: 31 | 32 | * Nodes -- things that store data -- Vertices and Edges 33 | * Collections -- groupings of nodes 34 | * Traversals -- objects used query the graph. This is heavily insipred by [Tinkerpop's Gremlin](https://github.com/tinkerpop/gremlin/wiki) 35 | * Tokens -- logic used to filter the graph during a traversal 36 | 37 | ### Adding Data 38 | 39 | Here is a very simple graph of a father and son 40 | 41 | from rgp import Graph, Vertex, Edge 42 | import redis 43 | 44 | connection = redis.StrictRedis(host='localhost', port=6379, db=0) 45 | graph = Graph(connection) 46 | 47 | dad = Vertex({ 48 | 'name': 'Mark', 49 | 'age': 'old' 50 | }) 51 | son = Vertex({ 52 | 'name': 'Jr.', 53 | 'age': 'young' 54 | }) 55 | parent = Edge('parent', dad, son) 56 | child = Edge('child', son, dad) 57 | 58 | graph.save(parent) 59 | graph.save(child) 60 | 61 | What we have now is a simple graph with a `son` and `dad` vertices and `parent` and `child` edges. 62 | 63 | ####Graph 64 | 65 | The `Graph` object is the main interface into the database. 66 | 67 | ##### Methods 68 | 69 | * `e` -- Used to get either a edge by id or all edges in the graph. Returns a `Collection` 70 | * `v` -- Used to get a specific vertex by id or all vertices in the graph. Returns a `Collection` 71 | * `traverse` -- Used to create a `Traversal` object. This is registed with the `Graph` instance. Returns a `Traversal` 72 | * `query` -- Used to execute a `Traversal` object. When called without an argument, the last registred traversal will be used. 73 | * `save` -- Used to save a `Node`. If the `Node` is an `Edge`, it will save both `Vertex` objects associated with it. If the argument is a `Collection`, it will loop thorugh and save each `Node`. Retuns an id if the argument were a `Node` or `Collection` otherwise. 74 | 75 | ####Vertex 76 | 77 | A `Vertex` is the base unit of data in the graph. It is how data is stored 78 | 79 | ####Edge 80 | 81 | `Edge` objects are what connect `Vertex` objects in graph -- they make the graph possible. 82 | 83 | ### Traversing The Graph 84 | 85 | RGP makes graph traversals pretty easy. Each action taken (`Token` executed) during a traversal is esentially a filter against a `Collection` instance. 86 | 87 | #### Traversal 88 | 89 | All traversals start and end with a `Collection` object. It could be empty, could be fed one `Node`, or it could be a collection from a previous traversal. 90 | 91 | A common way of starting a traversal would be directly from the `Graph` instance: 92 | 93 | trav = graph.traversal(son).outE() 94 | 95 | The `traversal` method on `Graph` returns a `Traversal` instance, if called this way the instance is stored on the `Graph` instance. `Traversal` provides a fluid interface so that you can easily chain together `Token` objects to query the graph. 96 | 97 | `Traversal` objects can be instantiated directly. This allows for sub-traversals or even prepared statement-like behavior. 98 | 99 | my_trav = Traversal() 100 | my_trav.outE() 101 | 102 | When it is time to run the traversal that was created, you simply call the `query` method with or without a `Traversal` object. 103 | 104 | result = graph.query() #this will run the previous traversal from graph.traverse() 105 | result = graph.query(my_trav) 106 | 107 | #### Tokens 108 | 109 | Tokens 110 | 111 | * `outE` -- 112 | * `inE` -- 113 | * `bothE` -- 114 | * `outV` -- 115 | * `inV` -- 116 | * `bothV` -- 117 | * `has` -- 118 | * `alias` -- 119 | * `back` -- 120 | * `loop` -- 121 | * `map` -- 122 | * `filter` -- 123 | * `collect` -- 124 | 125 | #### Custom Tokens 126 | 127 | One of the stregths of RGP is the ability to extend the library by adding your own tokens. Tokens must follow a few rules: 128 | 129 | * Treat the collection member as immutable. We do this to ensure that we can walk through our traversal and rewind state. 130 | * Must have an `_operator` member. This defines how the `Token` is represented in traversal. 131 | * Must always return a new `Collection` instance 132 | * If antoher traversal must be run within the `Token`, a new `Traversal` instance is created. Simply calling `graph.traverse` will erase the parent traversal. 133 | * Must have a '__call__' method with a signature. 134 | 135 | 136 | ## License 137 | 138 | MIT 139 | -------------------------------------------------------------------------------- /rgp/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emehrkay/rgp/47b0969c8f0860d83879c4bfd7404c5a5d7322e2/rgp/__init__.py -------------------------------------------------------------------------------- /rgp/graph.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import copy 3 | 4 | 5 | THIS = sys.modules[__name__] 6 | GRAPH_VARIABLE = 'rgp:index' 7 | GRAPH_VERTEX = 'rgp:vertex' 8 | GRAPH_VERTEX_ALL = 'rgp:vertex_all' 9 | GRAPH_VERTEX_OUT = 'rgp:vertex_out' 10 | GRAPH_VERTEX_IN = 'rgp:vertex_in' 11 | GRAPH_VERTEX_INDICES = 'rgp:vertex_indices' 12 | GRAPH_EDGE = 'rgp:edge' 13 | GRAPH_EDGE_ALL = 'rgp:edge_all' 14 | GRAPH_EDGE_REF = 'rgp:edge_ref' 15 | GRAPH_EDGE_INDICES = 'rgp:vertex_indices' 16 | GRAPH_INDEX = 'rgp:index' 17 | GRAPH_PROPERTY_ID = '__id' 18 | GRAPH_PROPERTY_TYPE = '__type' 19 | GRAPH_PROPERTY_LABEL = '__label' 20 | GRAPH_PROPERTY_INDICES = '__indices' 21 | GRAPH_PROPERTY_OUT = '__out_v' 22 | GRAPH_PROPERTY_IN = '__in_v' 23 | OPERATORS = {} 24 | MEMO = {} 25 | 26 | 27 | def memo(element): 28 | """memoizes the element for future creation""" 29 | if element._id: 30 | MEMO[element._id] = element 31 | 32 | 33 | class RGPException(Exception): 34 | pass 35 | 36 | 37 | class RGPEdgeException(RGPException): 38 | pass 39 | 40 | 41 | class RGPTokenException(RGPException): 42 | pass 43 | 44 | 45 | class RGPTokenAliasException(RGPException): 46 | pass 47 | 48 | 49 | class _Collectionable(object): 50 | pass 51 | 52 | 53 | class Graph(_Collectionable): 54 | 55 | def __init__(self, redis): 56 | self.redis = redis 57 | 58 | if not self.redis.exists(GRAPH_VARIABLE): 59 | self.redis.set(GRAPH_VARIABLE, 0) 60 | 61 | def next_id(self): 62 | return self.redis.incr(GRAPH_VARIABLE) 63 | 64 | def get(self, _id): 65 | for meth in ['v', 'e']: 66 | col = getattr(self, meth)(_id) 67 | 68 | if len(col): 69 | return col 70 | 71 | return Colleciton([]) 72 | 73 | def v(self, _id=None): 74 | self.traverse().v(_id) 75 | 76 | return self.query() 77 | 78 | def e(self, _id=None): 79 | self.traverse().e(_id) 80 | 81 | return self.query() 82 | 83 | def traverse(self, element=None): 84 | self._traversal = Traversal(element) 85 | 86 | return self._traversal 87 | 88 | def query(self, traversal=None): 89 | if not traversal: 90 | traversal = self._traversal 91 | 92 | return traversal.start(self) 93 | 94 | def _add_edge(self, node_id, edge_id, direction='in'): 95 | key = GRAPH_VERTEX_IN if direction == 'in' else GRAPH_VERTEX_OUT 96 | key = '%s:%s' % (key, node_id) 97 | ref_key = '%s:%s' % (GRAPH_EDGE_REF, edge_id) 98 | 99 | self.redis.sadd(ref_key, edge_id) 100 | return self.redis.sadd(key, edge_id) 101 | 102 | def _index_node(self, node, indices=None): 103 | if indices: 104 | _id = node.id 105 | _nkey = GRAPH_VERTEX_INDICES if isinstance(node, Vertex) else\ 106 | GRAPH_EDGE_INDICES 107 | fields = [f.lower() for f in indices if node.data.get(f, None)] 108 | 109 | for f in fields: 110 | val = node.data.get(f) 111 | 112 | if val: 113 | ikey = '%s:%s:%s' % (GRAPH_INDEX, f, val) 114 | self.redis.sadd(ikey, _id) 115 | 116 | return self 117 | 118 | def save(self, element): 119 | try: 120 | if isinstance(element, Collection): 121 | map(self.save, element) 122 | 123 | return element 124 | else: 125 | _id = element.id if element.id else self.next_id() 126 | vertex = isinstance(element, Vertex) 127 | key = GRAPH_VERTEX if vertex else GRAPH_EDGE 128 | key = '%s:%s' % (key, _id) 129 | data = element.data 130 | indices = data.get(GRAPH_PROPERTY_INDICES, []) 131 | data[GRAPH_PROPERTY_TYPE] = 'vertex' if vertex else 'edge' 132 | 133 | if not vertex: 134 | out_v = element._out_v 135 | in_v = element._in_v 136 | 137 | if not out_v or not in_v: 138 | msg = """both the out and in vertices must be set 139 | before saving an edge""" 140 | raise RGPEdgeException(msg) 141 | 142 | out_v_id = self.save(out_v) 143 | in_v_id = self.save(in_v) 144 | data[GRAPH_PROPERTY_OUT] = out_v_id 145 | data[GRAPH_PROPERTY_IN] = in_v_id 146 | 147 | self._add_edge(out_v_id, _id, 'in') 148 | self._add_edge(in_v_id, _id, 'out') 149 | 150 | element.id = _id 151 | 152 | all_key = GRAPH_VERTEX_ALL if vertex else GRAPH_EDGE_ALL 153 | self.redis.hmset(key, data) 154 | self.redis.sadd(all_key, _id) 155 | self._index_node(element, indices) 156 | memo(element) 157 | 158 | return _id 159 | except Exception as e: 160 | raise e 161 | 162 | def delete(self, element): 163 | try: 164 | if isinstance(element, Collection): 165 | map(self.delete, element) 166 | elif isinstance(element, Vertex): 167 | # remove the vertex 168 | # remove all of the edges connected 169 | # remove the vertex's edge lists 170 | self.redis.delete(element.key) 171 | 172 | trav = Traversal(element).bothE() 173 | edges = self.query(trav) 174 | out_v = '%s:%s' % (GRAPH_VERTEX_OUT, element.id) 175 | in_v = '%s:%s' % (GRAPH_VERTEX_IN, element.id) 176 | 177 | self.redis.delete(out_v) 178 | self.redis.delete(in_v) 179 | self.delete(edges) 180 | elif isinstance(element, Edge): 181 | # remove the edge 182 | # remove all references to the edge 183 | self.redis.delete(element.key) 184 | 185 | ref_key = '%s:%s' % (GRAPH_EDGE_REF, element.id) 186 | refs = self.redis.smembers(ref_key) 187 | 188 | for k in refs: 189 | n_key = '%s:%s' % (GRAPH_VERTEX_OUT, k) 190 | self.redis.srem(n_key, element.id) 191 | else: 192 | raise ValueError("""the element %s is not 193 | a valid type""" % element) 194 | except Exception as e: 195 | raise e 196 | 197 | 198 | class Node(object): 199 | 200 | def __init__(self, data=None, indices=None): 201 | if data is None: 202 | data = {} 203 | 204 | self._id = data[GRAPH_PROPERTY_ID] if\ 205 | GRAPH_PROPERTY_ID in data else None 206 | self._data = data 207 | self._indices = indices if indices else\ 208 | data.get(GRAPH_PROPERTY_INDICES, []) 209 | 210 | memo(self) 211 | 212 | def __setitem__(self, name, value): 213 | self._data[name] = value 214 | self._dirty = True 215 | 216 | return self 217 | 218 | def __getitem__(self, name): 219 | return self._data.get(name, None) 220 | 221 | @property 222 | def id(self): 223 | return self._id 224 | 225 | @id.setter 226 | def id(self, _id): 227 | self._id = _id 228 | self.data[GRAPH_PROPERTY_ID] = _id 229 | 230 | @property 231 | def data(self): 232 | data = self._data 233 | data[GRAPH_PROPERTY_INDICES] = self._indices 234 | 235 | return data 236 | 237 | @property 238 | def key(self): 239 | _type = GRAPH_EDGE if isinstance(self, Edge) else GRAPH_VERTEX 240 | 241 | return '%s:%s' % (_type, self[GRAPH_PROPERTY_ID]) 242 | 243 | 244 | class Vertex(Node): 245 | 246 | @property 247 | def oute_key(self): 248 | return '%s:%s' % (GRAPH_VERTEX_OUT, self[GRAPH_PROPERTY_ID]) 249 | 250 | @property 251 | def ine_key(self): 252 | return '%s:%s' % (GRAPH_VERTEX_IN, self[GRAPH_PROPERTY_ID]) 253 | 254 | 255 | class Edge(Node): 256 | 257 | def __init__(self, label, out_v=None, in_v=None, data=None): 258 | super(Edge, self).__init__(data=data) 259 | 260 | self._out_v = out_v 261 | self._in_v = in_v 262 | self._label = label 263 | 264 | @property 265 | def data(self): 266 | data = super(Edge, self).data 267 | data[GRAPH_PROPERTY_LABEL] = self._label 268 | 269 | return data 270 | 271 | @property 272 | def inv_key(self): 273 | return '%s:%s' % (GRAPH_VERTEX, self[GRAPH_PROPERTY_IN]) 274 | 275 | @property 276 | def outv_key(self): 277 | return '%s:%s' % (GRAPH_VERTEX, self[GRAPH_PROPERTY_OUT]) 278 | 279 | 280 | class Collection(object): 281 | 282 | def __init__(self, data=None): 283 | self._data = data if data else [] 284 | self._elements = {} 285 | 286 | def __len__(self): 287 | return len(self._data) 288 | 289 | def __call__(self, *args): 290 | return Traversal(self) 291 | 292 | def __getitem__(self, key): 293 | element = self._elements.get(key, None) 294 | 295 | if not element: 296 | try: 297 | data = self._data[key] 298 | kwargs = { 299 | 'data': data, 300 | } 301 | etype = 'Vertex' if data[GRAPH_PROPERTY_TYPE] == 'vertex'\ 302 | else 'Edge' 303 | 304 | if etype is not 'Vertex': 305 | kwargs['label'] = data[GRAPH_PROPERTY_LABEL] 306 | 307 | element = getattr(THIS, etype)(**kwargs) 308 | self[key] = element 309 | except Exception as e: 310 | raise StopIteration() 311 | 312 | return element 313 | 314 | def __setitem__(self, key, value): 315 | self._elements[key] = value 316 | 317 | def __delitem__(self, key): 318 | if key in self._elements: 319 | del self._models[key] 320 | 321 | @property 322 | def data(self): 323 | return self._data 324 | 325 | def append(self, element): 326 | self[len(self._elements)] = element 327 | 328 | return self 329 | 330 | def copy(self): 331 | data = copy.deepcopy(self.data) 332 | 333 | return Collection(data) 334 | 335 | 336 | class Traversal(object): 337 | 338 | def __init__(self, collection=None): 339 | if isinstance(collection, Node): 340 | collection = Collection([collection.data]) 341 | 342 | self.collection = collection 343 | self.top = Token() 344 | self.bottom = self.top 345 | 346 | def __call__(self, *args, **kwargs): 347 | self.bottom._args = args 348 | self.bottom._kwargs = kwargs 349 | 350 | return self 351 | 352 | def __getattr__(self, name): 353 | token = OPERATORS.get(name, None) 354 | 355 | if not token: 356 | msg = '%s does not sub-class Token' % name 357 | raise RGPTokenException(msg) 358 | 359 | vertex = token(self.collection) 360 | 361 | return self.add_node(vertex) 362 | 363 | def __getitem__(self, val): 364 | if type(val) is not slice: 365 | val = slice(val, None, None) 366 | 367 | self.bottom._range = val 368 | 369 | return self 370 | 371 | def add_node(self, node): 372 | self.bottom.next = node 373 | node.previous = self.bottom 374 | self.bottom = node 375 | 376 | return self 377 | 378 | def start(self, graph): 379 | token = self.top.next 380 | collection = self.collection 381 | 382 | while token: 383 | token.collection = collection 384 | token.graph = graph 385 | collection = token(*token._args, **token._kwargs) 386 | token = token.next 387 | 388 | return collection 389 | 390 | 391 | class _MetaToken(type): 392 | 393 | def __new__(cls, name, bases, attrs): 394 | cls = super(_MetaToken, cls).__new__(cls, name, bases, attrs) 395 | _operator = attrs.pop('_operator', None) 396 | 397 | if not _operator: 398 | msg = '%s token does not nave an un _operator defined' % name 399 | raise RGPTokenException(msg) 400 | 401 | OPERATORS[_operator] = cls 402 | 403 | return cls 404 | 405 | 406 | class Token(object): 407 | __metaclass__ = _MetaToken 408 | _operator = '__\root\token\__' 409 | 410 | def __init__(self, value=None): 411 | self.value = value 412 | self.collection = None 413 | self.previous = None 414 | self.next = None 415 | self._args = () 416 | self._kwargs = {} 417 | self._range = slice(0, None, 1) 418 | 419 | def __call__(self, *args): 420 | error = '%s does is not callable' % self.__name__ 421 | raise NotImplementedError(error) 422 | 423 | def compare(self, field, value, comparsion='=='): 424 | if comparsion == '==': 425 | return field == value 426 | elif comparsion == '!=': 427 | return field != value 428 | elif comparsion == 'in': 429 | return field in value 430 | 431 | def get_alias(self, name): 432 | parent = self.previous 433 | 434 | while parent: 435 | if isinstance(parent, Alias) and\ 436 | parent.name == name: 437 | return parent 438 | else: 439 | parent = parent.previous 440 | 441 | msg = """There was no no alias with the name 442 | %s registered for use with %s""" %\ 443 | (name, self.__class__.__name__) 444 | 445 | raise RGPTokenAliasException(msg) 446 | 447 | 448 | class Get(Token): 449 | _operator = 'get' 450 | 451 | def _query(self, field, value, operator='=='): 452 | return '%s:%s:%s' % (GRAPH_INDEX, field, value) 453 | 454 | def __call__(self, field, value, operator='=='): 455 | chained = [self] 456 | token = self.next 457 | data = [] 458 | k = [] 459 | 460 | while token and isinstance(token, Get): 461 | chained.append(token) 462 | token = token.next 463 | 464 | self.next = chained[-1].next 465 | self._range = chained[-1]._range 466 | 467 | for token in chained: 468 | k.append(self._query(*token._args, **token._kwargs)) 469 | 470 | results = list(self.graph.redis.sinter(k)) 471 | 472 | [data.extend(self.graph.get(i).data) for i in results[self._range]] 473 | 474 | return Collection(data) 475 | 476 | 477 | class GetVertex(Token): 478 | _operator = 'v' 479 | 480 | def __call__(self, _id=None): 481 | if _id: 482 | key = '%s:%s' % (GRAPH_VERTEX, _id) 483 | data = [self.graph.redis.hgetall(key)] 484 | else: 485 | keys = self.graph.redis.smembers(GRAPH_VERTEX_ALL) 486 | data = [] 487 | 488 | for k in keys: 489 | key = '%s:%s' % (GRAPH_VERTEX, k) 490 | data.append(self.graph.redis.hgetall(key)) 491 | 492 | return Collection(data) 493 | 494 | 495 | class GetEdge(Token): 496 | _operator = 'e' 497 | 498 | def __call__(self, _id=None): 499 | if _id: 500 | key = '%s:%s' % (GRAPH_EDGE, _id) 501 | data = [self.graph.redis.hgetall(key)] 502 | else: 503 | keys = self.graph.redis.smembers(GRAPH_EDGE_ALL) 504 | data = [] 505 | 506 | for k in keys: 507 | key = '%s:%s' % (GRAPH_EDGE, k) 508 | data.append(self.graph.redis.hgetall(key)) 509 | 510 | return Collection(data) 511 | 512 | 513 | class Has(Token): 514 | _operator = 'has' 515 | 516 | def __call__(self, field, value, comparsion='=='): 517 | data = [] 518 | 519 | if field and value: 520 | for i, node in enumerate(self.collection.data): 521 | if field in node and\ 522 | self.compare(node[field], value, comparsion): 523 | data.append(node) 524 | 525 | return Collection(data) 526 | 527 | 528 | class Contains(Token): 529 | _operator = 'contains' 530 | 531 | 532 | class Alias(Token): 533 | _operator = 'alias' 534 | _aliases = {} 535 | 536 | def __call__(self, name): 537 | self.name = name 538 | self._aliases[name] = self 539 | 540 | return self.collection.copy() 541 | 542 | 543 | class Collect(Token): 544 | _operator = 'collect' 545 | 546 | def __call__(self, *names): 547 | self.names = names 548 | data = self.collection.copy().data 549 | 550 | for name in names: 551 | alias = self.get_alias(name) 552 | data.extend(alias.collection.copy().data) 553 | 554 | return Collection(data) 555 | 556 | 557 | class Back(Token): 558 | _operator = 'back' 559 | 560 | def __call__(self, name): 561 | self.name = name 562 | alias = self.get_alias(name) 563 | 564 | return alias.collection.copy() 565 | 566 | 567 | class Loop(Token): 568 | _operator = 'loop' 569 | _loops = {} 570 | 571 | def __call__(self, name, count): 572 | self.name = name 573 | self.count = count 574 | alias = self.get_alias(name) 575 | 576 | if name not in self._loops: 577 | self._loops[name] = { 578 | 'iter': 0, 579 | 'count': count, 580 | 'original_next': self.next, 581 | } 582 | 583 | loop = self._loops[name]['iter'] < \ 584 | self._loops[name]['count'] 585 | 586 | if loop: 587 | self._loops[name]['iter'] += 1 588 | self.next = alias.next 589 | else: 590 | self.next =\ 591 | self._loops[name]['original_next'] 592 | 593 | return self.collection 594 | 595 | 596 | class Filter(Token): 597 | _operator = 'filter' 598 | 599 | def __call__(self, callback): 600 | data = filter(callback, self.collection.copy().data) 601 | 602 | return Collecton(data) 603 | 604 | 605 | class Map(Token): 606 | _operator = 'map' 607 | 608 | def __call__(self, callback): 609 | data = map(callback, self.collection.copy().data) 610 | 611 | return Collecton(data) 612 | 613 | 614 | class OutE(Token): 615 | _operator = 'outE' 616 | 617 | def __call__(self): 618 | data = [] 619 | 620 | for i, node in enumerate(self.collection): 621 | if isinstance(node, Edge): 622 | self.graph.traverse(node).outV() 623 | data.extend(self.graph.query().data) 624 | else: 625 | edges = self.graph.redis.smembers(node.oute_key) 626 | 627 | for d in edges: 628 | data.extend(list(self.graph.e(d).data)) 629 | 630 | return Collection(data) 631 | 632 | 633 | class InE(Token): 634 | _operator = 'inE' 635 | 636 | def __call__(self): 637 | data = [] 638 | 639 | for i, node in enumerate(self.collection): 640 | if isinstance(node, Edge): 641 | self.graph.traverse(node).inV() 642 | data.extend(self.graph.query().data) 643 | else: 644 | edges = self.graph.redis.smembers(node.ine_key) 645 | 646 | for d in edges: 647 | data.extend(list(self.graph.e(d).data)) 648 | 649 | return Collection(data) 650 | 651 | 652 | class BothE(Token): 653 | _operator = 'bothE' 654 | 655 | def __call__(self): 656 | data = [] 657 | 658 | def get_edge(key): 659 | data = [] 660 | edges = self.graph.redis.smembers(key) 661 | 662 | for d in edges: 663 | data.extend(list(self.graph.e(d).data)) 664 | 665 | return data 666 | 667 | for i, node in enumerate(self.collection): 668 | if isinstance(node, Edge): 669 | self.graph.traverse(node).inV() 670 | data.extend(self.graph.query().data) 671 | self.graph.traverse(node).outV() 672 | data.extend(self.graph.query().data) 673 | else: 674 | data.extend(get_edge(node.ine_key)) 675 | data.extend(get_edge(node.oute_key)) 676 | 677 | return Collection(data) 678 | 679 | 680 | class OutV(Token): 681 | _operator = 'outV' 682 | 683 | def __call__(self): 684 | data = [] 685 | 686 | for i, node in enumerate(self.collection): 687 | if isinstance(node, Vertex): 688 | trav = Traversal(node) 689 | trav.outE() 690 | data.extend([self.graph.redis.hgetall(n.outv_key)\ 691 | for n in self.graph.query(trav)]) 692 | else: 693 | inv = self.graph.redis.hgetall(node.outv_key) 694 | 695 | data.append(inv) 696 | 697 | return Collection(data) 698 | 699 | 700 | class InV(Token): 701 | _operator = 'inV' 702 | 703 | def __call__(self): 704 | data = [] 705 | 706 | for i, node in enumerate(self.collection): 707 | if isinstance(node, Vertex): 708 | trav = Traversal(node) 709 | trav.inE() 710 | data.extend([self.graph.redis.hgetall(n.outv_key)\ 711 | for n in self.graph.query(trav)]) 712 | else: 713 | inv = self.graph.redis.hgetall(node.inv_key) 714 | 715 | data.append(inv) 716 | 717 | return Collection(data) 718 | 719 | 720 | class BothV(Token): 721 | _operator = 'bothV' 722 | 723 | def __call__(self): 724 | data = [] 725 | 726 | for i, node in enumerate(self.collection): 727 | if isinstance(node, Edge): 728 | trav = Traversal(node) 729 | trav.outE() 730 | out_v = [self.graph.redis.hgetall(n.outv_key)\ 731 | for n in self.graph.query(trav)] 732 | trav = Traversal(node) 733 | trav.inE() 734 | in_v = [self.graph.redis.hgetall(n.outv_key)\ 735 | for n in self.graph.query(trav)] 736 | data.extend(out_v) 737 | data.extend(in_v) 738 | else: 739 | data.extend(self.graph.redis.hgetall(node.inv_key)) 740 | data.extend(self.graph.redis.hgetall(node.outv_key)) 741 | 742 | return Collection(data) 743 | -------------------------------------------------------------------------------- /rgp/t.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/emehrkay/rgp/47b0969c8f0860d83879c4bfd7404c5a5d7322e2/rgp/t.py -------------------------------------------------------------------------------- /rgp/test.py: -------------------------------------------------------------------------------- 1 | import redis 2 | from graph import * 3 | import time 4 | 5 | 6 | start_time = time.time() 7 | r = redis.StrictRedis(host='localhost', port=6379, db=0) 8 | 9 | g = Graph(r) 10 | id = ['name', 'sex'] 11 | 12 | # r.flushdb() 13 | # for i in range(10000): 14 | # n = Vertex({ 15 | # 'name': 'mark', 16 | # 'role': 'dad', 17 | # 'sex': 'male', 18 | # '__indices': id 19 | # }) 20 | # n2 = Vertex({ 21 | # 'name': 'jr', 22 | # 'role': 'child', 23 | # 'sex': 'male', 24 | # '__indices': id 25 | # }) 26 | # n3 = Vertex({ 27 | # 'name': 'leanne', 28 | # 'role': 'mom', 29 | # 'sex': 'female', 30 | # '__indices': id 31 | # }) 32 | # s = Vertex({ 33 | # 'name': 'sade', 34 | # 'sex': 'female', 35 | # '__indices': id 36 | # }) 37 | # sam = Vertex({ 38 | # 'name': 'sam', 39 | # 'sex': 'female', 40 | # '__indices': id 41 | # }) 42 | # e = Edge('father', n, n2) 43 | # e2 = Edge('fson', n2, n) 44 | # e3 = Edge('mother', n3, n2) 45 | # e4 = Edge('mson', n2, n3) 46 | # e5 = Edge('so', n, s) 47 | # e6 = Edge('so', s, n) 48 | # e7 = Edge('sister', s, sam) 49 | # e8 = Edge('sister', sam, s) 50 | # col = Collection() 51 | # col.append(e).append(e2).append(e3).append(e4).append(e5).append(e6).append(n).append(n2).append(n3).append(s) 52 | # col.append(e7).append(e8) 53 | # g.save(col) 54 | 55 | print '+++++++++++++++++++++++++++++++' 56 | print '%s seconds to finish adding %s nodes' % (time.time() - start_time, len(r.keys())) 57 | 58 | # g.save(e) 59 | # g.save(e2) 60 | # g.save(e3) 61 | # g.save(e4) 62 | # g.save(e5) 63 | # g.save(e6) 64 | # print r.keys() 65 | # print g.v(9).data 66 | # 67 | # print r.smembers('rgp:vertex_out:3') 68 | # tr = g.traverse(n2) 69 | # tr.outE().outV().has('name', ['mark', 'leanne'], 'in') 70 | #print r.keys(), tr.outE(0, name='mark'), tr.bottom 71 | # col = g.query(tr) 72 | #g.traverse(n2).alias('t').outV().alias('x').has('name', 'mark').outE().outV().back('x').back('t') 73 | # print g.e().data 74 | # print g.v().data 75 | # g.traverse(n2).outV().outV().outV().has('name', 'sam', '==') 76 | # print 'first' 77 | # for i in g.query(): 78 | # print 'result::', i, i.data 79 | # 80 | # 81 | # g.traverse(n2).alias('s').outV().loop('s', 2).has('name', 'sam') 82 | # print 'with loop' 83 | # for i in g.query(): 84 | # print 'result::', i, i.data 85 | # 86 | # g.traverse(n2).alias('x').outV().loop('x', 2).has('name', 'sam').alias('y').collect('x', 'y') 87 | # print 'with collect' 88 | # for i in g.query(): 89 | # print 'result::', i, i.data 90 | # 91 | # 92 | # print '===============' 93 | # print n2.id 94 | # print len(r.keys()), r.keys() 95 | # 96 | # g.delete(n2) 97 | # 98 | # print '===============' 99 | # print len(r.keys()), r.keys() 100 | 101 | st = time.time() 102 | 103 | g.traverse().get('sex', 'male').get('name', 'jr')[40:80].alias('x').outV().loop('x', 2).has('name', 'sam') 104 | res = g.query() 105 | # for i in g.query(): 106 | # print i, i.data 107 | 108 | print '+++++++++++++++++++++++++++++++' 109 | print '%s seconds to query %s results' % (time.time() - st, len(res)) 110 | 111 | for it in res: 112 | print it, it.data 113 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright (c) 2007 Qtrac Ltd. All rights reserved. 3 | # This module is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or (at 6 | # your option) any later version. 7 | 8 | import os 9 | from setuptools import setup 10 | 11 | setup( 12 | name='rgp', 13 | version = '0.1.0', 14 | author="emehrkay@gmail.com", 15 | author_email="emehrkay@gmail.com", 16 | url="https://github.com/emehrkay/rgp", 17 | classifiers=[ 18 | 'License :: OSI Approved :: MIT License', 19 | 'Development Status :: 3 - Alpha', 20 | 'Programming Language :: Python :: 2.7', 21 | 'Environment :: Web Environment', 22 | 'Topic :: Database', 23 | 'Topic :: Database :: Front-Ends', 24 | 'Topic :: Internet :: WWW/HTTP', 25 | 'Topic :: Software Development', 26 | 'Topic :: Software Development :: Libraries :: Python Modules', 27 | 'Topic :: System :: Distributed Computing', 28 | 'Intended Audience :: Developers', 29 | 'Operating System :: POSIX :: Linux', 30 | 'Operating System :: MacOS', 31 | 'Operating System :: MacOS :: MacOS X', 32 | ], 33 | packages=['rgp'], 34 | license="MIT", 35 | keywords='python redis graph database', 36 | description="Python graph database implemented on top of Redis.", 37 | ) 38 | --------------------------------------------------------------------------------