├── README.md ├── easydata ├── __init__.py ├── database.py ├── dict.py └── graph.py ├── examples └── neural_network.py ├── img ├── 4927232E-8D0E-4211-9546-594CF7557A5B.jpeg └── D5C8A2F0-94B4-46C2-A9D7-BE7A3FE94563.jpeg └── setup.py /README.md: -------------------------------------------------------------------------------- 1 | # EasyData 2 | 3 | EasyData is a lightweight database library for handling complex graphical data seemlessly in python. 4 | 5 | ## Databases 6 | 7 | __I. Schemas__ 8 | 9 | Databases are dictionaries with additional structure built on top. 10 | 11 | A database consists of schemas, objects, and attributes, which are all related symbolically. 12 | 13 | A schema defines a label and a set of attributes, and is used to create new objects. 14 | 15 | ![Schema diagram](https://github.com/CarsonScott/easydata/blob/master/img/4927232E-8D0E-4211-9546-594CF7557A5B.jpeg) 16 | 17 | We can define a schema to represent positions in space: 18 | 19 | db = Database() 20 | db.create_schema('point', ['x', 'y']) 21 | 22 | It's useful to have some control over the acceptable values that can be assigned to the attributes of a schema. 23 | 24 | Constraints are proposition functions that are assigned with respect to a particular attribute of a given schema. 25 | 26 | We can define constraints on the x and y attributes of the point schema defined above. 27 | 28 | These constraints ensure that any value assigned to either x or y will be a non-negative integer. 29 | 30 | db.create_constraint('point', 'x', lambda val: isinstance(val, int) and val >= 0) 31 | db.create_constraint('point', 'y', lambda val: isinstance(val, int) and val >= 0) 32 | 33 | If a value is given for a particular attribute while creating an object, which fails to satisfy the corresponding constraints of that attribute, then an exception is raised. 34 | 35 | In other words, all constraints must be satisfied in order to create an instance of a particular schema. 36 | 37 | Attributes can be added to existing schemas as well: 38 | 39 | db.create_attribute('point', 'z', [lambda val: isinstance(val, int) and val >= 0]) 40 | 41 | __II. Objects__ 42 | 43 | As you can see the schema labeled 'point' defines two attributes: 'x', 'y', and 'z'. 44 | 45 | Objects created using the point schema are automatically assigned those attributes, and therefore require the same number of values to be defined when they are created. 46 | 47 | ![Object diagram](https://github.com/CarsonScott/easydata/blob/master/img/D5C8A2F0-94B4-46C2-A9D7-BE7A3FE94563.jpeg) 48 | 49 | We can create a new object using the point schema: 50 | 51 | db.create_object('point', 'p1', [5, 6, 7], 0) 52 | 53 | The database now contains an object called 'p1', equal to 0 (defaults to None), which has attributes 'x', 'y' and 'z', equal to 5, 6, and 7 respectively. 54 | 55 | The value of an object is stored in a database as a typical dictionary element: 56 | 57 | db['p1'] 58 | >>> 0 59 | 60 | __III. Attributes__ 61 | 62 | The individual attributes of an object are accessible through the following method: 63 | 64 | db.get_attr('p1', 'x) 65 | >>> 5 66 | 67 | db.get_attr('p1', 'y') 68 | >>> 6 69 | 70 | db.get_attr('p1', 'z') 71 | >>> 7 72 | 73 | ...or through the following short-hand: 74 | 75 | db['p1', 'x'] 76 | >>> 5 77 | 78 | db['p1', 'y'] 79 | >>> 6 80 | 81 | db['p1', 'z'] 82 | >>> 7 83 | 84 | All attributes of an object are accessible as a dictionary: 85 | 86 | db.get_attrs('p1') 87 | >>> {'x': 5, 'y': 6, 'z': 7} 88 | 89 | *** 90 | 91 | ## Graphs 92 | 93 | __I. Nodes__ 94 | 95 | Graphs are databases with a preset schema for defining links between objects. 96 | 97 | Graphs also come with a few additional methods for dealing with the link objects. 98 | 99 | graph = Graph() 100 | graph.create_schema('point', ['x', 'y']) 101 | graph.create_object('point', 'p1', [5, 6]) 102 | graph.create_object('point', 'p2', [3, 9]) 103 | 104 | Every object in a graph is assigned two additional attributes: 105 | 106 | graph.get_attrs('p1') 107 | >>> {'x': 5, 'y': 6, 'sources': [], 'targets': []) 108 | 109 | __II. Links__ 110 | 111 | Links are created using a 'source' and 'target' key, each pointing to an existing object: 112 | 113 | >>> graph.create_link('p1', 'p2') 114 | 115 | The resulting link is an object with 'source' and 'target' attributes. 116 | 117 | Every link is assigned a generated key based on the source and target keys used to define it: 118 | 119 | graph.get_key('p1', 'p2') 120 | >>> '(p1 p2)' 121 | 122 | The attributes of the source and target objects are updated when a new link is created: 123 | 124 | graph.get_attrs('p1') 125 | >>> {'x': 5, 'y': 6, 'sources': [], 'targets': ['p2']) 126 | 127 | graph.get_attrs('p2') 128 | >>> {'x': 3, 'y': 9, 'sources': ['p1'], 'targets': []) 129 | 130 | Additional attributes can be added to the link schema: 131 | 132 | graph.create_attribute('link', 'weight', [lambda val: isinstance(val, int) or isinstance(val, float)]) 133 | 134 | The additional attributes are assigned when a link is created: 135 | 136 | graph.create_link('p1', 'p2', [4]) 137 | 138 | The link between 'p1' and 'p2' now has an attribute called 'weight', equal to 4. 139 | 140 | __III. Hierarchies__ 141 | 142 | Every link is an object that may be the source or target of another link. 143 | 144 | Links between links may be defined in an infinite hierarchy. 145 | 146 | graph.get_attrs(graph.get_key('p1', 'p2')) 147 | >>> {'source': 'p1', 'target': 'p2', 'weight': 6, 'sources': [], 'targets': []) 148 | 149 | As you can see, the link between 'p1' and 'p2' also has attributes 'sources' and 'targets'. 150 | 151 | graph.create_object('point', 'p3', [1, 2]) 152 | graph.create_link('p2', 'p3', [2]) 153 | 154 | Now there is a new point called 'p3' that is connected with 'p2'. 155 | 156 | We can create a link between links since there are now two links in the graph: 157 | 158 | graph.create_link(graph.get_key('p1', 'p2'), graph.get_key('p1', 'p2'), [5]) 159 | 160 | The links from 'p1' to 'p2' and from 'p2' to 'p3' are now connected to one another. 161 | 162 | The key assigned to this link reflects the hierarchical nature: 163 | 164 | graph.get_key(graph.get_key('p1', 'p2'), graph.get_key('p1', 'p2')) 165 | >>> '((p1 p2) (p2 p3))' 166 | 167 | *** 168 | 169 | ## Notes 170 | 171 | The set of instances for a particular schema are accessible in list form: 172 | 173 | graph.get_instances('point') 174 | >>> ['p1', 'p2', 'p3'] 175 | 176 | Objects can be removed from a graph or database: 177 | 178 | graph.remove_object('p1') 179 | 180 | graph.get_instances('point') 181 | >>> ['p2', 'p3'] 182 | 183 | An object can be tested to see if it's an instance of a particular schema: 184 | 185 | graph.is_instance('p2', 'point') 186 | >>> True 187 | -------------------------------------------------------------------------------- /easydata/__init__.py: -------------------------------------------------------------------------------- 1 | from .graph import * 2 | 3 | -------------------------------------------------------------------------------- /easydata/database.py: -------------------------------------------------------------------------------- 1 | from .dict import * 2 | 3 | class Database(Dict): 4 | 5 | def __init__(self): 6 | self.schemas=dict() 7 | self.attributes=dict() 8 | self.constraints=dict() 9 | self.instances=dict() 10 | 11 | def __setitem__(self, key, value): 12 | if isinstance(key, tuple): 13 | self.set_attr(*key, value) 14 | else: 15 | super().__setitem__(key,value) 16 | if key not in self.attributes: 17 | self.attributes[key]=dict() 18 | 19 | def __getitem__(self, key): 20 | if isinstance(key, tuple): 21 | return self.get_attr(*key) 22 | else:return super().__getitem__(key) 23 | 24 | def create_schema(self, schema, attrs=[]): 25 | self.schemas[schema]=list() 26 | self.instances[schema]=list() 27 | self.constraints[schema]=dict() 28 | for attr in attrs: 29 | self.create_attribute(schema, attr, []) 30 | 31 | def create_constraint(self, schema, key, proposition): 32 | self.constraints[schema][key].append(proposition) 33 | 34 | def create_attribute(self, schema, attr, constraints=[]): 35 | self.schemas[schema].append(attr) 36 | self.constraints[schema][attr]=constraints 37 | 38 | def create_object(self, schema, key, values=[], value=None): 39 | self[key]=value 40 | attrs=self.schemas[schema] 41 | for i in range(len(attrs)): 42 | self.set_attr(key,attrs[i],values[i]) 43 | self.instances[schema].append(key) 44 | 45 | invalid_attrs=[] 46 | for attr in self.get_attrs(key): 47 | value=self.get_attr(key, attr) 48 | constraints=[] 49 | if attr in self.constraints[schema]: 50 | constraints=self.constraints[schema][attr] 51 | if not all(constraint(value) for constraint in constraints): 52 | invalid_attrs.append(attr) 53 | 54 | if len(invalid_attrs) > 0: 55 | self.remove_object(key) 56 | exception='Object "' + key + '" not created. ' 57 | if len(invalid_attrs) == 1: 58 | exception += 'Attribute ' 59 | else: exception += 'Attributes ' 60 | 61 | for i in range(len(invalid_attrs)): 62 | attr=invalid_attrs[i] 63 | exception += '"' + attr + '"' 64 | if i < len(invalid_attrs)-1: 65 | exception += ',' 66 | exception += ' ' 67 | exception += 'not satisfied.' 68 | raise Exception(exception) 69 | 70 | def remove_object(self, key): 71 | if key in self: 72 | del self[key] 73 | if key in self.attributes: 74 | del self.attributes[key] 75 | for schema in self.instances: 76 | if self.is_instance(key, schema): 77 | index=self.instances[schema].index(key) 78 | del self.instances[schema][index] 79 | 80 | def set_attr(self, key, attr, value):self.attributes[key][attr]=value 81 | def get_attr(self, key, attr):return self.attributes[key][attr] 82 | def has_attr(self, key, attr):return attr in self.attributes[key] 83 | def get_attrs(self, key):return self.attributes[key] 84 | def get_instances(self, schema):return self.instances[schema] 85 | def is_instance(self, key, schema):return key in self.get_instances(schema) 86 | -------------------------------------------------------------------------------- /easydata/dict.py: -------------------------------------------------------------------------------- 1 | class Reference(str):pass 2 | class Object(dict):pass 3 | 4 | class Dict(dict): 5 | def keys(self): 6 | return list(super().keys()) 7 | def values(self): 8 | return list(super().values()) 9 | -------------------------------------------------------------------------------- /easydata/graph.py: -------------------------------------------------------------------------------- 1 | from .database import * 2 | 3 | class Graph(Database): 4 | 5 | def __init__(self): 6 | super().__init__() 7 | self.create_schema('link', ['source', 'target']) 8 | 9 | def create_object(self, schema, key, values=[], value=None): 10 | super().create_object(schema, key, values, value) 11 | self.set_attr(key, 'sources', []) 12 | self.set_attr(key, 'targets', []) 13 | 14 | def create_link(self, source, target, values=[], value=None): 15 | key=self.get_key(source, target) 16 | self[source, 'targets'].append(target) 17 | self[target, 'sources'].append(source) 18 | self.create_object('link', key, [Reference(source), Reference(target)]+values, value) 19 | 20 | def get_key(self, source, target):return '('+source+' '+target+')' 21 | -------------------------------------------------------------------------------- /examples/neural_network.py: -------------------------------------------------------------------------------- 1 | from easydata import Graph 2 | import random 3 | import math 4 | 5 | # The following is an example of a graph used to implement a feedforward neural network. 6 | # The neural network produces outputs, given inputs from a randomly-generated sample set. 7 | # The goal is to show how the easydata library may be used to structure data for seemless 8 | # application, not as an example of machine-learning. 9 | # 10 | # For this reason, the neural network does not feed back (i.e. train). However, it 11 | # is obvious that this type of computation would fit into the framework shown below, as 12 | # the variables required to compute errors and gradients are potentially if not readily 13 | # accessible in the existing NN model. 14 | 15 | def logistic(x): 16 | return 1 / (1 + pow(math.e, -x)) 17 | 18 | # Create graph and define neuron schema 19 | nn=Graph() 20 | nn.create_schema('neuron', ['bias', 'output']) 21 | 22 | # Create neuron objects and links between layers 23 | layers=[] 24 | layer_sizes=(10, 5, 3) 25 | for i in range(len(layer_sizes)): 26 | layer_size=layer_sizes[i] 27 | layers.append([]) 28 | for j in range(layer_size): 29 | key=str(i) 30 | layers[i].append(key) 31 | nn.create_object('neuron', key, 0, [random.randrange(100)/100, 0]) 32 | if len(layers) > 1: 33 | for k in layers[i-1]: 34 | nn.create_link(k, key, random.randrange(100)/100) 35 | 36 | # Generate random samples 37 | samples=[] 38 | sample_size=3 39 | for i in range(sample_size): 40 | sample=[random.randrange(2) for j in range(len(layers[0]))] 41 | samples.append(sample) 42 | 43 | # Compute output for each sample 44 | outputs=[] 45 | for i in range(len(samples)): 46 | sample=samples[i] 47 | outputs.append([]) 48 | 49 | # Set input layer neurons 50 | input_keys=layers[0] 51 | for j in range(len(input_keys)): 52 | key=input_keys[j] 53 | value=sample[j] 54 | nn.set_attr(key, 'output', value) 55 | 56 | # Compute hidden and output layer neurons 57 | neuron_keys=nn.get_instances('neuron') 58 | for j in range(len(neuron_keys)): 59 | key=neuron_keys[j] 60 | if key not in input_keys: 61 | source_keys=nn.get_attr(key, 'sources') 62 | input_total=0 63 | for k in source_keys: 64 | link_key=nn.get_key(k, key) 65 | input_weight=nn[link_key] 66 | input_value=nn[k] 67 | input_total+=input_value*input_weight 68 | bias_value=nn.get_attr(key, 'bias') 69 | output_value=logistic(input_total+bias_value) 70 | nn.set_attr(key, 'output', output_value) 71 | 72 | # Update all neurons 73 | output_keys=layers[len(layers)-1] 74 | for j in range(len(neuron_keys)): 75 | key=neuron_keys[j] 76 | value=nn.get_attr(key, 'output') 77 | nn[key]=value 78 | if key in output_keys: 79 | outputs[i].append(value) 80 | 81 | # Display input-output pairs 82 | for i in range(len(samples)): 83 | sample=samples[i] 84 | output=outputs[i] 85 | print(sample, '\t', output, sep='') 86 | -------------------------------------------------------------------------------- /img/4927232E-8D0E-4211-9546-594CF7557A5B.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CarsonScott/easydata/7c4cce12c1750369d679d4f5e51fef367c00afe3/img/4927232E-8D0E-4211-9546-594CF7557A5B.jpeg -------------------------------------------------------------------------------- /img/D5C8A2F0-94B4-46C2-A9D7-BE7A3FE94563.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CarsonScott/easydata/7c4cce12c1750369d679d4f5e51fef367c00afe3/img/D5C8A2F0-94B4-46C2-A9D7-BE7A3FE94563.jpeg -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from distutils.core import setup 2 | 3 | setup( 4 | name='easydata', 5 | version='1.0', 6 | description='A simple and lightweight database/graph library in python.', 7 | url='https://github.com/CarsonScott/easydata', 8 | author='Carson Scott', 9 | author_email='carsonjscott14@gmail.com', 10 | packages=['easydata'], 11 | ) 12 | --------------------------------------------------------------------------------