├── .gitignore ├── LICENSE ├── README.md ├── img ├── Figure.png ├── Figure_1.png ├── Graph.png └── Memory.PNG └── src └── Network.py /.gitignore: -------------------------------------------------------------------------------- 1 | test.py 2 | *.pyc 3 | src/log.txt 4 | src/test1.py 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Carson Scott 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Conditional-Associative Logic Memory 2 | 3 | ## Learning 4 | The following is an unsupervised machine-learning algorithm that recognizes and predicts input patterns in real-time. The algorithm is essentially a system of information-processing inspired by the cognitive functions of the mind. The system has a collection of interacting memory systems akin to the spychological concepts of short-term and long-term memory, which allows it to learn by observation and adapt based on the frequency of patterns and their relationships through time. 5 | 6 | An association value is a representation of conditional probability which is calculated between the current observation and the rest of the elements in the buffer, i.e. the previous N observations, then stored in the association matrix. Each elements current place in the buffer determines the strength of the delta applied to its association value, so more recent observation are more strongly associated with the current observations than those further in the buffer. The association matrix enables the prediction of future observations based on current and previous observations. A significantly large association for the current observation to another signify that the other is likely to occur. The association value indicates the strength of the connection as well as the confidence interval related to the prediction. 7 | 8 | Therefore a system may be understood in terms of a Bayesian network, whose adaptations inherently create a foundation for inference, as well as provide a useful data structure, i.e. directed/labeled associations, which can be used to construct higher level objects like sequences (chains of associations) used to represent episodic memory, which in turn combine to form trees (hierarchies of sequences) used for high level reasoning like considering various alternatives in case a planned action fails, or for creating recursive models of observations that span multiple levels of complexity. 9 | 10 | ## Prediction 11 | The system makes predictions based on the current and previous observations. It takes the rows of the association matrix that correspond the with elements currently stored in the buffer and calculates a probability distribution over the possible events that could occur at the next observation. Each probability depends on the current observation as well as the previous N observations, and therefore accomplishes a type of chaining that emphasizes the context of the observation to have an impact when inferring about the future. 12 | 13 | ## Examples [To be updated] 14 | The following example was produced by selecting an observation at random from a set of 15 possible obervations. Each time a selection is ade, there is a small chance that the observation will not be random but instead will be from a "pattern", a pair of observations representing some event to be learned. The sampling process is repeated 1000 times and processed by the system to generate an association matrix. 15 | 16 | Pattern 1: {6, 13} 17 | Pattern 2: {9, 4) 18 | Pattern 3: {1, 5} 19 | 20 | ![](https://github.com/CarsonScott/AutoMLN/blob/master/img/Figure_1.png) 21 | 22 | Each pairing of x and y coordinates represent an association from an x element to a y element. The z axis contains the association values for each pairing. Each peak represents a statistically significant association between two obserbation 23 | 24 | -------------------------------------------------------------------------------- /img/Figure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CarsonScott/CALM/9269d749ae641274eceb3e2afd04fc21d8dcade0/img/Figure.png -------------------------------------------------------------------------------- /img/Figure_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CarsonScott/CALM/9269d749ae641274eceb3e2afd04fc21d8dcade0/img/Figure_1.png -------------------------------------------------------------------------------- /img/Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CarsonScott/CALM/9269d749ae641274eceb3e2afd04fc21d8dcade0/img/Graph.png -------------------------------------------------------------------------------- /img/Memory.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CarsonScott/CALM/9269d749ae641274eceb3e2afd04fc21d8dcade0/img/Memory.PNG -------------------------------------------------------------------------------- /src/Network.py: -------------------------------------------------------------------------------- 1 | #MIT License 2 | # 3 | #Copyright (c) 2017 Carson Scott 4 | # 5 | #Permission is hereby granted, free of charge, to any person obtaining a copy 6 | #of this software and associated documentation files (the "Software"), to deal 7 | #in the Software without restriction, including without limitation the rights 8 | #to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | #copies of the Software, and to permit persons to whom the Software is 10 | #furnished to do so, subject to the following conditions: 11 | # 12 | #The above copyright notice and this permission notice shall be included in all 13 | #copies or substantial portions of the Software. 14 | # 15 | #THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | #IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | #FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | #AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | #LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | #OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | #SOFTWARE. 22 | 23 | e = 2.718281828459 24 | 25 | class Network: 26 | 27 | def compute(self, x): 28 | return 1/(1 + pow(e, (x - self.bsize))) 29 | 30 | def __init__(self, msize, bsize): 31 | self.links = [] 32 | self.states = [] 33 | self.places = [] 34 | self.msize = msize 35 | self.bsize = bsize 36 | self.lrate = 0.001 37 | self.drate = 0.0001 38 | for i in range(msize): 39 | self.links.append([]) 40 | self.states.append(0) 41 | self.places.append(None) 42 | for j in range(msize): 43 | self.links[i].append(0) 44 | 45 | def update(self, x): 46 | for i in range(len(self.states)): 47 | if self.states[i] == 1: 48 | if self.places[i] >= self.bsize: 49 | self.places[i] = None 50 | self.states[i] = 0 51 | else: 52 | d = self.compute(self.places[i]) 53 | self.links[i][x] += self.lrate*d 54 | if self.links[i][x] > 1: 55 | self.links[i][x] = 1 56 | self.places[i] += 1 57 | else: 58 | self.links[i][x] -= self.drate 59 | if self.links[i][x] < 0: 60 | self.links[i][x] = 0 61 | self.states[x] = 1 62 | self.places[x] = 0 63 | 64 | def predict(self): 65 | y = [] 66 | total = 0 67 | for i in range(len(self.links)): 68 | if self.states[i] == 1: 69 | for j in range(len(self.links[i])): 70 | if len(y) == j: 71 | y.append(0) 72 | v = self.links[i][j] 73 | d = v*(1-self.places[i]/self.bsize) 74 | y[j] += d 75 | total += d 76 | if total != 0: 77 | for i in range(len(y)): 78 | y[i] /= total 79 | return y --------------------------------------------------------------------------------