├── LICENSE ├── README.md ├── multilayerGM ├── __init__.py ├── comparisons.py ├── dependency_tensors.py ├── distributions.py ├── export.py ├── networks.py └── partitions.py └── setup.py /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 2-Clause License 2 | 3 | Copyright (c) 2019, Lucas Jeub 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | 1. Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | 2. Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 17 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 18 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 19 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 20 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 21 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 22 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 23 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 24 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 25 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # multilayerGM (Version 0.1.0) 2 | 3 | This code implements the method for generating multilayer networks with 4 | planted mesoscale structure of 5 | 6 | >M. Bazzi, L. G. S. Jeub, A. Arenas, S. D. Howison, and M. A. Porter. 7 | Generative benchmark models for mesoscale structure in multilayer networks. 8 | arXiv:1608.06196 [cs.SI], 2016. 9 | 10 | If you use results based on this code in an academic publication please cite 11 | this paper and cite the code as 12 | >L. G. S. Jeub. A Python framework for generating multilayer networks with planted 13 | mesoscale structure. , 2019 14 | 15 | 16 | ## Installation 17 | 18 | This package supports installation with setuptools. The easiest way to 19 | install the latest version of this package and its dependencies is to use 20 | ``` 21 | pip install git+https://github.com/MultilayerGM/MultilayerGM-py.git@master 22 | ``` 23 | To install a particular version, replace `master` above by the appropriate commit 24 | identifier, e.g., use `v0.1.0` to install version 0.1.0. 25 | 26 | 27 | ## Basic usage 28 | 29 | The examples below assume that the package has been imported as 30 | ```python 31 | import multilayerGM as gm 32 | ``` 33 | 34 | The model generates a multilayer network in two steps: 35 | 1. Generate a multilayer partition with desired dependencies between layers 36 | 37 | 2. Generate a multilayer network that reflects this partition 38 | 39 | ### Generate partition 40 | 41 | The main interface for generating partitions is 42 | ```python 43 | partition = gm.sample_partition(dependency_tensor=dt, null_distribution=null) 44 | ``` 45 | and supports generating partitions and supports multiple aspects with a mix of 46 | ordered and unordered aspects. The type of multilayer network is determined by 47 | the choice of interlayer dependency tensor. Different dependency tensors are 48 | implemented in `gm.dependency_tensors`. 49 | 50 | The null distribution determines the random component of the generating process 51 | and is largely responsible for determining the expected mesoset sizes. `null` 52 | should be a function that takes a state node as input and returns a random mesoset 53 | assignment. 54 | 55 | Currently, the only implemented choice (which is the one we use for the numerical 56 | examples in the paper) for the null distribution independently samples the 57 | mesoset distribution for each layer from a symmetric dirichlet distribution 58 | with concentration parameter `theta` and number of mesosets `n_sets`. 59 | 60 | ```python 61 | null = gm.DirichletNull(layers=dt.shape[1:], theta=theta, n_sets=n_sets) 62 | ``` 63 | 64 | However, it is straight-forward to substitute different null distributions. 65 | 66 | 67 | #### Multiplex network with uniform dependencies: 68 | 69 | To set up a dependency tensor for generating multiplex networks with uniform 70 | interlayer dependencies use 71 | ```python 72 | dt = gm.dependency_tensors.UniformMultiplex(n_nodes, n_layers, p) 73 | ``` 74 | where `n_nodes` is the desired number of nodes, `n_layers` is the desired number 75 | of layers and `p` is the copying probability. With probability `p`, the state node 76 | that is being updated copies its mesoset assignment from a different state node 77 | representing the same physical node chosen uniformly at random. This is an unordered 78 | dependency tensor (i.e., `dt.aspect_types='r'`) and the partition will be sampled using pseudo-Gibbs sampling. 79 | 80 | 81 | #### Temporal network 82 | 83 | To set up a dependency tensor for generating temporal networks use 84 | ```python 85 | dt = gm.dependency_tensors.Temporal(n_nodes, n_layers, p) 86 | ``` 87 | where `n_nodes` is the desired number of nodes, `n_layers` is the desired number 88 | of layers and `p` specifies the copying probabilities. A state node in layer `l` that is being updated copies its 89 | mesoset assignment from the state node representing the same physical node in the previous layer with probability 90 | `p[l-1]`. Otherwise the state node 91 | gets a new mesoset assignment by sampling from the null distribution. Mesoset assignments 92 | for state nodes in the first layer are always sampled from the null distribution. One can also specify a single 93 | probability for `p` to generate a temporal network with uniform dependencies. This 94 | is an ordered dependency tensor (i.e., `dt.aspect_types='o'`) and the partition 95 | will be sampled sequentially. 96 | 97 | 98 | #### Networks with multiple aspects 99 | 100 | To generate multilayer networks with multiple aspects but without direct inter-aspect 101 | dependencies use 102 | ```python 103 | dt = gm.dependency_tensors.MultiAspect(tensors, weights) 104 | ``` 105 | where `tensors` is a list of single-aspect dependency tensors and `weights` is a list of 106 | corresponding weights. An update for this dependency tensor proceeds in two steps. First, 107 | an aspect is selected with probability proportional to its weight and then the update 108 | proceeds according to the dependency tensor for this aspect. The dependency tensors in 109 | `tensors` can be a mix of ordered and unordered tensors and accordingly the partition will be sampled 110 | using a mix of sequential and pseudo-Gibbs sampling. 111 | 112 | 113 | #### Custom dependency tensor 114 | 115 | To generate networks that do not fit in the above categories, it is possible 116 | to define a fully-custom dependency tensor. To do so, the custom dependency 117 | tensor should be a subclass of `gm.dependency_tensors.DependencyTensor` or 118 | otherwise make sure to implement the same interface. Subclasses of `gm.dependency_tensors.DependencyTensor` 119 | have to at a minimum define a `getrandneighbour` method which determines the copying 120 | process (see the inline documentation). In most cases one would also need to extent the 121 | constructor (i.e., define a custom `__init__` method) to store additional parameters. 122 | Note that `gm.dependency_tensors.DependencyTensor` makes it possible to specify 123 | the state nodes that are included in the tensor such that one construct networks 124 | that are not fully interconnected. However, if one wants to use this feature, one 125 | has to ensure that the `getrandneighbour` method can only return state nodes that 126 | are actually included in the tensor. 127 | 128 | 129 | ### Generate network 130 | 131 | Currently the only implemented network model is the DCSBM based benchmark model 132 | that we use for the numerical examples in the paper. To generate a network from 133 | a multilayer partition using this model use 134 | ```python 135 | multinet = gm.multilayer_DCSBM_network(partition, mu=0.1, k_min=5, k_max=70, t_k=-2) 136 | ``` 137 | Here `partition` is a multilayer partition defined as a mapping from state nodes to 138 | mesoset assignments (such as the output of `gm.sample_partition()`) and `mu` is a parameter 139 | that determines the strength of the planted community structure (for `mu=0` communities 140 | are disconnected from each other and for `mu=1` the network has no communities). The distribution 141 | of expected degrees for the network is a truncated powerlaw with minimum cutoff `k_min`, 142 | maximum cutoff `k_max`, and exponent `t_k`. 143 | 144 | The network is returned as `MultilayerGraph` which extends the NetworkX `Graph` class 145 | with some multilayer network functionality (see https://github.com/LJeub/nxMultilayerNet for more details). 146 | Each node has a `'mesoset'` attribute, such that the planted partition can be recovered 147 | using 148 | ```python 149 | partition = dict(multinet.nodes(data='mesoset')) 150 | ``` 151 | 152 | 153 | ### Compare partitions 154 | 155 | This package includes functionality to compare (multilayer) partitions using normalized mutual information (NMI). The 156 | variant of NMI implemented here uses joint entropy as the normalisation, i.e., 157 | `NMI(P1, P2) = I(P1, P2)/H(P1, P2)`, where `I` is the mutual information and `H` is the entropy. 158 | 159 | #### NMI 160 | 161 | To compare two partitions using NMI (this corresponds to computing multilayer NMI if the input partitions are 162 | multilayer) use 163 | ```python 164 | nmi_val = gm.comparisons.nmi(partition1, partition2) 165 | ``` 166 | 167 | #### mean NMI 168 | 169 | To compare two multilayer partitions using mean NMI between induced partitions for each layer use 170 | ```python 171 | nmi_val = gm.comparisons.mean_nmi(partition1, partition2) 172 | ``` 173 | 174 | #### NMI tensor 175 | 176 | To compute the NMI between induced partitions of a multilayer partition for all pairs of layers, use 177 | ```python 178 | nmi_map = gm.comparisons.nmi_tensor(partition) 179 | ``` 180 | 181 | #### Induced partitions 182 | 183 | To extract induced paritions for each layer from a multilayer partition use 184 | ```python 185 | lp = gm.comparisons.layer_partitions(partition) 186 | ``` 187 | 188 | This function is used by the other comparison methods internally but may be useful to implement custom 189 | comparisons, such as only comparing partitions on a subset of layers. 190 | 191 | 192 | ### Save results 193 | 194 | This package includes functions to export the results to JSON and MATLAB '.mat' format. 195 | 196 | To export a partition to JSON (note that state nodes are converted from tuple to string) use 197 | ```python 198 | gm.export.save_json_partition(partition, filename) 199 | ``` 200 | 201 | To load the partition data (converting state nodes back to tuples) use 202 | ```python 203 | partition = gm.export.load_json_partition(filename) 204 | ``` 205 | 206 | To export the multilayer network as a JSON edgelist use 207 | ```python 208 | gm.export.save_json_edgelist(multinet, filename) 209 | ``` 210 | 211 | To load the network data (optionally specify a filename for partition data 212 | to restore `'mesoset'` node attribute) use 213 | ```python 214 | multinet = gm.export.load_json_multinet(edgelist, partition=None) 215 | ``` -------------------------------------------------------------------------------- /multilayerGM/__init__.py: -------------------------------------------------------------------------------- 1 | from . import dependency_tensors 2 | from . import export 3 | from . import comparisons 4 | from .networks import multilayer_DCSBM_network 5 | from .partitions import sample_partition, DirichletNull, dirichlet_null 6 | from pkg_resources import get_distribution, DistributionNotFound 7 | 8 | 9 | try: 10 | __version__ = get_distribution(__name__).version 11 | except DistributionNotFound: 12 | # package is not installed 13 | pass 14 | -------------------------------------------------------------------------------- /multilayerGM/comparisons.py: -------------------------------------------------------------------------------- 1 | from collections import defaultdict as _dfdict 2 | from collections import Counter as _Counter 3 | from math import log2 4 | from statistics import mean 5 | 6 | 7 | def layer_partitions(partition): 8 | """ 9 | Construct induced partitions for each layer from an input multilayer partition. 10 | 11 | :param partition: Input partition as mapping of state-node to mesoset 12 | 13 | :return: mapping of layer to induced partition 14 | """ 15 | partitions = _dfdict(dict) 16 | for node, p in partition.items(): 17 | partitions[tuple(node[1:])][node[0]] = p 18 | return partitions 19 | 20 | 21 | def nmi(partition1, partition2): 22 | """ 23 | Compute NMI between two partitions. If the input partitions are multilayer, this function computes the multilayer 24 | NMI. 25 | 26 | :param partition1: first input partition as mapping of node to mesoset 27 | :param partition2: second input partition as mapping of node to mesoset 28 | 29 | :return: NMI value (normalised by joint entropy) 30 | """ 31 | n = len(partition1) 32 | if len(partition2) != n: 33 | raise ValueError("partitions need to have the same number of elements") 34 | p12 = _Counter((partition1[key], partition2[key]) for key in partition1) 35 | h12 = sum((p/n) * log2(p/n) for p in p12.values()) 36 | 37 | p1 = _Counter(partition1.values()) 38 | h1 = sum((p/n) * log2(p/n) for p in p1.values()) 39 | 40 | p2 = _Counter(partition2.values()) 41 | h2 = sum((p/n) * log2(p/n) for p in p2.values()) 42 | 43 | return (h1 + h2 - h12) / h12 44 | 45 | 46 | def mean_nmi(partition1, partition2): 47 | """ 48 | Compute mean NMI between induced partitions for a pair of multilayer partitions. 49 | 50 | :param partition1: first input partition as mapping of state-node to mesoset 51 | :param partition2: second input partition as mapping of state-node to mesoset 52 | 53 | :return: mean NMI value (normalised by joint entropy) 54 | """ 55 | layer_partitions1 = layer_partitions(partition1) 56 | layer_partitions2 = layer_partitions(partition2) 57 | return mean(nmi(layer_partitions1[layer], layer_partitions2[layer]) for layer in layer_partitions1) 58 | 59 | 60 | def nmi_tensor(partition): 61 | """ 62 | Compute NMI between all pairs of induced partitions for an input multilayer partition. 63 | 64 | :param partition: input partition as mapping of state-node to mesoset 65 | :return: nmi values as mapping dict(layer1: dict(layer2: nmi)) 66 | """ 67 | lp = layer_partitions(partition) 68 | return {l1: {l2: nmi(lp[l1], lp[l2]) for l2 in lp} for l1 in lp} 69 | -------------------------------------------------------------------------------- /multilayerGM/dependency_tensors.py: -------------------------------------------------------------------------------- 1 | import random as _rand 2 | from itertools import accumulate, chain 3 | from collections.abc import Iterable 4 | from abc import ABC, abstractmethod 5 | from .distributions import categorical 6 | 7 | # TODO: Implement generic dependency tensors that allow one to specify copying probabilities explicitly 8 | 9 | 10 | class DependencyTensor(ABC): 11 | """ 12 | Base class for implementing dependency tensors. 13 | 14 | At a minimum, a concrete implementation of a dependency tensor needs to override the `getrandneighbour` method which 15 | implicitly specifies the interlayer dependencies. 16 | 17 | """ 18 | def __init__(self, shape, aspect_types, state_nodes=None): 19 | """ 20 | Initialise dependency tensor 21 | 22 | :param shape: [n, a_1, ..., a_d] specify number of nodes and number of layers for each aspect 23 | :param aspect_types: specify sampling type for each aspect ('o' for ordered or 'r' for random). 24 | `aspect_types` should be a string such that aspect_types[i] specifies the 25 | type of the ith aspect. Note that `len(aspect_types) = len(shape)-1`. 26 | :param state_nodes: List of state-nodes (optional, defaults to all state nodes) 27 | """ 28 | self.shape = shape 29 | self.aspect_types = aspect_types 30 | if state_nodes is None: 31 | self._state_nodes = {l: [(n, *l) for n in range(shape[0])] for l in SubscriptIterator(self.shape[1:])} 32 | else: 33 | self._state_nodes = {l: [] for l in SubscriptIterator(self.shape[1:])} 34 | for n in state_nodes: 35 | if all(mi > ni >= 0 for mi, ni in zip(self.shape, n)): 36 | n_t = tuple(n) 37 | self._state_nodes[n[1:]].append(n) 38 | else: 39 | raise ValueError("Provided state nodes do not match `shape`") 40 | 41 | def state_nodes(self, layer=None): 42 | """ 43 | Return list of state nodes 44 | 45 | :param layer: (optional, if specified only return state-nodes for this layer) 46 | """ 47 | if layer is None: 48 | return chain.from_iterable(self._state_nodes.values()) 49 | else: 50 | return self._state_nodes[layer] 51 | 52 | def ordered_aspect_layers(self): 53 | """ 54 | List all layers corresponding to ordered aspects (indeces corresponding to unordered (random) aspects are fixed 55 | at 0. In particular, this method always includes the all-zero layer (0,...,0) even when there are no ordered 56 | aspects. 57 | :return: Iterator 58 | """ 59 | shape = list(self.shape[1:]) 60 | for i in range(len(shape)): 61 | if self.aspect_types[i] == 'r': 62 | shape[i] = 1 63 | return SubscriptIterator(shape) 64 | 65 | def random_aspect_layers(self): 66 | """ 67 | List all layers corresponding to random aspects (indeces corresponding to ordered aspects are fixed 68 | at 0. In particular, this method always includes the all-zero layer (0,...,0) even when there are no unordered 69 | (random) aspects. 70 | :return: Iterator 71 | """ 72 | shape = list(self.shape[1:]) 73 | for i in range(len(shape)): 74 | if self.aspect_types[i] == 'o': 75 | shape[i] = 1 76 | return SubscriptIterator(shape) 77 | 78 | def number_of_layers(self, aspect_type=None): 79 | """ 80 | Total number of layers 81 | :param aspect_type: optionally only count layers of a particular type 82 | ('o' to return the number of classes of order-equivalent layers and or 'r' to return the 83 | size of each class). 84 | :return: number of layers 85 | """ 86 | if aspect_type is None: 87 | def check(t): 88 | return True 89 | else: 90 | def check(t): 91 | return t == aspect_type 92 | 93 | n = 1 94 | for s, t in zip(self.shape[1:], self.aspect_types): 95 | if check(t): 96 | n *= s 97 | return n 98 | 99 | @abstractmethod 100 | def getrandneighbour(self, node): 101 | """ 102 | Return either a randomly sampled neighbour of the input state-node or the input state-node itself. This method 103 | is called to determine the updated meso-set assignment of a state-node. If the state-node returned by this 104 | method is not the same as the input state-node, the input state-node's meso-set assignment is updated by copying 105 | the assignment of the output state-node. If the input and output state-nodes are the same, the input state-node's 106 | assignment is updated by sampling from the null-distribution. 107 | :param node: input state-node 108 | :return: output state-node 109 | """ 110 | if _rand.random() > 0.5: 111 | # returning the same state node with probability 0.5 for resampling from null-distribution 112 | return node 113 | else: 114 | # return state-node sampled uniformly from all state-nodes in other layers (only an example) 115 | rnode = (_rand.randint(self.shape[0]), *(rn if rn < n else rn+1 116 | for rn, n in zip((_rand.randint(i-1) for i in self.shape[1:]), 117 | node[1:]))) 118 | return rnode 119 | 120 | 121 | class UniformMultiplex(DependencyTensor): 122 | """ 123 | Implements a dependency tensor for generating multiplex networks with uniform interlayer dependencies. 124 | 125 | This tensor is layer-coupled and has a single aspect. 126 | 127 | """ 128 | def __init__(self, n_nodes, n_layers, p_hat): 129 | """ 130 | :param n_nodes: number of physical nodes 131 | :param n_layers: number of layers 132 | :param p_hat: copying probability (probability that a state-node obtains a new mesoset assingment by copying the 133 | mesoset assignment of a different state-node during an update) 134 | """ 135 | super().__init__((n_nodes, n_layers), 'r') 136 | self.p_hat = p_hat 137 | 138 | def getrandneighbour(self, node): 139 | """ 140 | Return randomly sampled neighbour for the input state-node: 141 | with probability `self.p_hat`: sample the output state-node uniformly at random from all other state-nodes 142 | representing the same physical node as the input 143 | else: return the input state-node (it's mesoset assignment will be resampled from the null-distribution) 144 | 145 | :param node: input state-node 146 | :return: output state-node 147 | """ 148 | # returning the same state node for resampling from null-distribution 149 | if _rand.random() > self.p_hat: 150 | return node 151 | else: 152 | # sample a state-node uniformly at random from all other state-nodes representing the same physical node 153 | # as the input 154 | n = _rand.randrange(0, self.shape[1]-1) 155 | if n >= node[1]: 156 | n += 1 157 | return (node[0], n) 158 | 159 | 160 | class Temporal(DependencyTensor): 161 | """ 162 | Implements a dependency tensor for generating temporal networks where a layer only depends directly on the previous 163 | layer. 164 | 165 | This tensor is ordered, layer-coupled, and has a single aspect. 166 | 167 | """ 168 | def __init__(self, n_nodes, n_layers, p): 169 | """ 170 | :param n_nodes: number of physical nodes 171 | :param n_layers: number of layers 172 | :param p: copying probabilities (probability that a state-node obtains a new mesoset assingment by copying the 173 | mesoset assignment of a different state-node during an update). This can either be a single probability 174 | (in which case the copying probability is the same for each layer) or a list of probabilities of length 175 | `n_layers-1` (in which case p[i-1] is the probability that a state node in layer i copies its mesoset 176 | assignment from the corresponding state node in layer i-1). 177 | """ 178 | super().__init__((n_nodes, n_layers), 'o') 179 | if isinstance(p, Iterable): 180 | self.p = p 181 | else: 182 | self.p = [p] * (n_layers-1) 183 | 184 | def getrandneighbour(self, node): 185 | """ 186 | Return randomly sampled neighbour for the input state-node: 187 | 188 | If the input state-node is in layer 0, always return the input state-node (mesoset assignments for state-nodes in 189 | layer 0 are always resampled from the null-distribution), otherwise 190 | with probability `self.p[node[1]-1]`: return the state-node representing the same physical node as the input in the 191 | previous layer 192 | else: return the input state-node (it's mesoset assignment will be resampled from the null-distribution) 193 | 194 | :param node: input state-node 195 | :return: output state-node 196 | """ 197 | if node[1] == 0 or _rand.random() > self.p[node[1]-1]: 198 | # returning the same state node for resampling from null 199 | return node 200 | else: 201 | # return corresponding state-node from previous layer 202 | return (node[0], node[1]-1) 203 | 204 | 205 | class BlockMultiplex(DependencyTensor): 206 | def __init__(self, nodes, layers, n_blocks, p_in, p_out): 207 | super().__init__((nodes, layers), 'r') 208 | 209 | 210 | class MultiAspect(DependencyTensor): 211 | def __init__(self, tensors, weights): 212 | """ 213 | Construct multi-aspect dependency tensor by combining multiple single-aspect dependency tensors. 214 | 215 | :param tensors: list of single-aspect tensors 216 | :param weights: list of weights: tensors[i] is used with probability weights[i]/sum(weights) 217 | """ 218 | nodes = tensors[0].shape[0] 219 | for t in tensors: 220 | if t.shape[0] != nodes: 221 | raise ValueError("All tensors need to have the same number of nodes") 222 | if len(t.shape) != 2: 223 | raise ValueError("All tensors need to have exactly one aspect") 224 | shape =[nodes] 225 | shape += [t.shape[1] for t in tensors] 226 | super().__init__(tuple(shape), "".join(t.aspect_types for t in tensors)) 227 | self.calpha = list(accumulate(weights)) 228 | a_sum = self.calpha[-1] 229 | if a_sum != 1: 230 | self.calpha[:] = [a/a_sum for a in self.calpha] 231 | self.alpha = [a / a_sum for a in weights] 232 | self.tensors = tensors 233 | 234 | def getrandneighbour(self, node): 235 | """ 236 | An update for this tensor proceeds in two steps. First, we select an aspect such that the probability of 237 | selecting a particular aspect is proportional to its weight. Second, we use the tensor corresponding to the 238 | selected aspect to determine the output state-node. In particular, the output state-node differs from the input 239 | state-node in at most the node index and the index corresponding to the selected aspect. 240 | 241 | :param node: input state-node 242 | :return: output state-node 243 | """ 244 | i = categorical(self.calpha) 245 | n, a_i = self.tensors[i].getrandneighbour((node[0], node[i+1])) 246 | node = list(node) # convert to list for index assignment 247 | node[0] = n 248 | node[i+1] = a_i 249 | return tuple(node) 250 | 251 | 252 | class SubscriptIterator: 253 | def __init__(self, shape): 254 | """ 255 | Iterator over all subscripts of a multidimensional tensor of given shape in lexicographic order 256 | 257 | :param shape: shape of tensor 258 | """ 259 | self.shape = shape 260 | self.state = [0 for _ in self.shape] 261 | if self.state: 262 | self.state[-1] = -1 263 | 264 | def __iter__(self): 265 | return self 266 | 267 | def __next__(self): 268 | self._update_state(len(self.shape)-1) 269 | return tuple(self.state) 270 | 271 | def _update_state(self, index): 272 | if index < 0: 273 | raise StopIteration 274 | else: 275 | if self.state[index] < self.shape[index] - 1: 276 | self.state[index] += 1 277 | else: 278 | self.state[index] = 0 279 | self._update_state(index - 1) 280 | -------------------------------------------------------------------------------- /multilayerGM/distributions.py: -------------------------------------------------------------------------------- 1 | import random as _rand 2 | 3 | 4 | def categorical(cum_weights): 5 | """ 6 | Sample from a discrete distribution 7 | 8 | :param cum_weights: list of cumulative sums of probabilities (should satisfy cum_weights[-1] = 1) 9 | :return: sampled integer from `range(0,len(cum_weights))` 10 | """ 11 | p = _rand.random() 12 | i = 0 13 | while p > cum_weights[i]: 14 | i += 1 15 | return i 16 | 17 | 18 | def dirichlet(theta, n): 19 | """ 20 | Sample from a symmetric dirichlet distribution 21 | 22 | :param theta: concentration parameter (theta > 0) 23 | :param n: number of variables 24 | :return: list of sampled probabilities of length n 25 | """ 26 | weights = [_rand.gammavariate(theta,1) for _ in range(n)] 27 | s = sum(weights) 28 | weights[:] = [w/s for w in weights] 29 | return weights 30 | -------------------------------------------------------------------------------- /multilayerGM/export.py: -------------------------------------------------------------------------------- 1 | import json 2 | from scipy.io import savemat 3 | import numpy as np 4 | import nxmultilayer as nxm 5 | from ast import literal_eval 6 | 7 | 8 | def save_json_edgelist(multinet, filename): 9 | """ 10 | Export edgelist for multilayer network in JSON format 11 | 12 | :param multinet: Multilayer network in nxm.MultilayerGraph or nxm.MultilayerDiGraph format 13 | :param filename: filepath for output 14 | """ 15 | with open(filename, 'w') as f: 16 | json.dump(list(multinet.to_directed(as_view=True).edges()), f) 17 | 18 | 19 | def save_json_partition(partition, filename): 20 | """ 21 | Export partition in JSON format (note that node keys are converted to string) 22 | 23 | :param partition: map from state nodes to mesoset assignments 24 | :param filename: filepath for output 25 | """ 26 | partition = {repr(node): p for node, p in partition} 27 | with open(filename, 'w') as f: 28 | json.dump(partition, f) 29 | 30 | 31 | def load_json_multinet(edgelist, partition=None): 32 | """ 33 | Load multilayer network from JSON edgelist and optionally load partition data as well 34 | 35 | :param edgelist: filename for edgelist data 36 | :param partition: (optional) filename for partition data 37 | :return: nxm.MultilayerDiGraph (if partition is provided each node has a 'mesoset' attribute) 38 | """ 39 | with open(edgelist) as f: 40 | edges = json.load(f) 41 | if edges: 42 | mg = nxm.MultilayerDiGraph(n_aspects=len(edges[0][0])) 43 | for source, target in edges: 44 | mg.add_edge(source, target) 45 | if partition is not None: 46 | with open(partition) as f: 47 | partition_data = json.load(f) 48 | partition_data = {literal_eval(key): value for key, value in partition_data.items()} 49 | for node, p in partition_data: 50 | mg.add_node(node, mesoset=p) 51 | else: 52 | mg = nxm.MultilayerDiGraph() 53 | return mg 54 | 55 | 56 | def load_JSON_partition(filename): 57 | """ 58 | Load partition from JSON data 59 | :param filename: path 60 | :return: dict: Mapping from state nodes to mesosets 61 | """ 62 | with open(filename) as f: 63 | partition_data = json.load(f) 64 | partition_data = {literal_eval(key): value for key, value in partition_data.items()} 65 | return partition_data 66 | 67 | 68 | def save_matlab(multinet, file): 69 | """ 70 | Save multilayer network and partition in MATLAB format. Output file contains three variables: 71 | source: start node for each edge 72 | target: end node for each edge 73 | partition: mesoset assignment for each state node 74 | 75 | (Note that this function assumes that state-nodes are integer tuples) 76 | 77 | :param multinet: Multilayer network (each state node should have a 'mesoset' attribute) 78 | :param file: Output path 79 | :return: 80 | """ 81 | multinet = multinet.to_directed(as_view=True) 82 | source = np.row_stack([e[0] for e in multinet.edges()]) + 1 # one-based indeces for MATLAB 83 | target = np.row_stack([e[1] for e in multinet.edges()]) + 1 84 | 85 | partition = np.ndarray(shape=tuple(len(a) for a in multinet.aspects), dtype=float) 86 | for node, p in multinet.nodes(data='mesoset'): 87 | partition[node]=p 88 | savemat(file, {'source': source, 'target': target, 'partition': partition}) 89 | -------------------------------------------------------------------------------- /multilayerGM/networks.py: -------------------------------------------------------------------------------- 1 | import numpy as _np 2 | from collections import defaultdict as _dfdict 3 | import nxmultilayer as nxm 4 | 5 | 6 | def multilayer_DCSBM_network(partition, mu=0.1, k_min=5, k_max=70, t_k=-2, degrees=None): 7 | """ 8 | Generate multilayer benchmark networks with planted community structure using a DCSBM model. The mixing parameter 9 | `mu` determines the strength of the planted community structure. For `mu=0`, all edges are constrained to fall 10 | within communities. For `mu>0`, a fraction `mu` of edges is sampled independently of the planted communities. 11 | For `mu=1`, all edges are independent of the planted communities and the generated network has no community structure. 12 | 13 | The expected degrees are sampled from a truncated powerlaw distribution. 14 | 15 | The sampled multilayer network is returned as a MultilayerGraph which adds some functionality on top of a NetworkX 16 | Graph (see https://github.com/LJeub/nxMultilayerNet ) 17 | 18 | :param partition: Partition to plant 19 | :param mu: Fraction of random edges (default: 0.1) 20 | :param k_min: Minimum cutoff for distribution of expected degrees 21 | :param k_max: Maximum cutoff for distribution of expected degrees 22 | :param t_k: Exponent for distribution of expected degrees 23 | :param degrees: (Optional) specify degree sequence as mapping of state-node -> degree (if specified, `k_min`, 24 | `k_max`, and `t_k` have no effect and the specified degrees are used instead) 25 | 26 | :return: generated multilayer network 27 | """ 28 | layer_partitions = _dfdict(dict) 29 | neighbors = _dfdict(set) 30 | if isinstance(partition, _np.ndarray): 31 | for node, p in _np.ndenumerate(partition): 32 | layer_partitions[tuple(node[1:])][tuple(node)] = p 33 | else: 34 | for node, p in partition.items(): 35 | layer_partitions[tuple(node[1:])][tuple(node)] = p 36 | 37 | fixed_degrees = degrees 38 | 39 | for p in layer_partitions.values(): 40 | n_nodes = len(p) 41 | nodes_l = list(p.keys()) 42 | if fixed_degrees is None: 43 | degrees = power_law_sample(n_nodes, t=t_k, x_max=k_max, x_min=k_min) 44 | else: 45 | degrees = _np.array([fixed_degrees[node] for node in nodes_l]) 46 | # random edges 47 | no_partition = _np.zeros(n_nodes, dtype=int) 48 | block_matrix = build_block_matrix(no_partition, degrees * mu) 49 | neighbors = block_model_sampler(block_matrix=block_matrix, partition=no_partition, degrees=degrees * mu, 50 | neighbors=neighbors, nodes=nodes_l) 51 | # community edges 52 | p_l = _np.fromiter(p.values(), dtype=int) 53 | block_matrix = build_block_matrix(p_l, degrees * (1 - mu)) 54 | neighbors = block_model_sampler(block_matrix=block_matrix, partition=p_l, degrees=degrees * (1 - mu), 55 | neighbors=neighbors, nodes=nodes_l) 56 | multinet = nxm.MultilayerGraph(n_aspects=len(nodes_l[0])-1) 57 | if isinstance(partition, _np.ndarray): 58 | for node, p in _np.ndenumerate(partition): 59 | multinet.add_node(node, mesoset=p) 60 | else: 61 | for node, p in partition.items(): 62 | multinet.add_node(node, mesoset=p) 63 | 64 | for node1, adj in neighbors.items(): 65 | for node2 in adj: 66 | multinet.add_edge(node1, node2) 67 | 68 | return multinet 69 | 70 | 71 | class IdentityMap: 72 | def __getitem__(self, item): 73 | return item 74 | 75 | 76 | def block_model_sampler(block_matrix, partition, degrees, neighbors=None, nodes=IdentityMap()): 77 | max_reject = 100 78 | partition = _np.array(partition) 79 | degrees = _np.array(degrees) 80 | n_nodes = len(partition) 81 | if n_nodes != len(degrees): 82 | raise ValueError('length mismatch between partition and node degrees') 83 | 84 | n_groups = max(partition)+1 85 | 86 | # look up map for group memberships 87 | partition_map = [[] for _ in range(n_groups)] 88 | for i, g in enumerate(partition): 89 | partition_map[g].append(i) 90 | 91 | group_sizes = [len(g) for g in partition_map] 92 | 93 | sigma = [degrees[g] / sum(degrees[g]) for g in partition_map] 94 | 95 | if neighbors is None: 96 | neighbors = _dfdict(set) 97 | 98 | for g1 in range(n_groups): 99 | for g2 in range(g1,n_groups): 100 | w = block_matrix[g1,g2] 101 | if w > 0: 102 | m = _np.random.poisson(w) 103 | if g1 == g2: 104 | dense = 2*m > group_sizes[g1]*(group_sizes[g1]-1) 105 | m = int(_np.ceil(m / 2)) 106 | else: 107 | dense = 2*m > group_sizes[g1]*group_sizes[g2] 108 | 109 | if dense: 110 | probabilities = w * _np.outer(sigma[g1], sigma[g2]) 111 | probabilities[probabilities > 1] = 1 112 | if g1 == g2: 113 | for i in range(group_sizes[g1]): 114 | ni = _np.extract(probabilities[i, i + 1:] > _np.random.rand(1, group_sizes[g1] - i - 1), 115 | _np.arange(i + 1, group_sizes[g1])) 116 | for j in ni: 117 | neighbors[nodes[partition_map[g1][i]]].add(nodes[partition_map[g2][j]]) 118 | neighbors[nodes[partition_map[g2][j]]].add(nodes[partition_map[g1][i]]) 119 | else: 120 | for i in range(group_sizes[g1]): 121 | ni = _np.flatnonzero(probabilities[i, :] > _np.random.rand([1, group_sizes[g2]])) 122 | for j in ni: 123 | neighbors[nodes[partition_map[g1][i]]].add(nodes[partition_map[g2][j]]) 124 | neighbors[nodes[partition_map[g2][j]]].add(nodes[partition_map[g1][i]]) 125 | 126 | else: 127 | for e in range(m): 128 | is_neighbor = True 129 | reject_count = 0 130 | while is_neighbor and reject_count <= max_reject: 131 | node1 = nodes[_np.random.choice(partition_map[g1], 1, p=sigma[g1])[0]] 132 | node2 = nodes[_np.random.choice(partition_map[g2], 1, p=sigma[g2])[0]] 133 | is_neighbor = node1 in neighbors[node2] or node1 == node2 134 | reject_count += 1 135 | if reject_count > max_reject: 136 | print('rejection threshold reached') 137 | break 138 | neighbors[node1].add(node2) 139 | neighbors[node2].add(node1) 140 | 141 | return neighbors 142 | 143 | 144 | def build_block_matrix(partition, degrees): 145 | nc = max(partition) + 1 146 | block_matrix = _np.zeros([nc, nc]) 147 | for i, g in enumerate(partition): 148 | block_matrix[g, g] += degrees[i] 149 | return block_matrix 150 | 151 | 152 | def power_law_sample(n, t, x_min, x_max): 153 | y = _np.random.rand(n) 154 | if t != -1: 155 | return ((x_max**(t+1) - x_min**(t+1)) * y + x_min**(t+1))**(1/(t+1)) 156 | else: 157 | return x_min*(x_max/x_min)**y 158 | 159 | 160 | 161 | -------------------------------------------------------------------------------- /multilayerGM/partitions.py: -------------------------------------------------------------------------------- 1 | import random as _rand 2 | from .distributions import categorical, dirichlet 3 | from .dependency_tensors import SubscriptIterator 4 | from itertools import accumulate 5 | import numpy as _np 6 | 7 | 8 | def sample_partition(dependency_tensor, null_distribution, 9 | updates=100, 10 | initial_partition=None 11 | ): 12 | """ 13 | Sample partition for a multilayer network with specified interlayer dependencies 14 | 15 | :param dependency_tensor: dependency tensor 16 | :param null_distribution: null distribution (function that takes a state-node as input and returns a random mesoset 17 | assignment 18 | :param updates: expected number of (pseudo-)Gibbs-sampling updates per state-node (has no effect for fully ordered 19 | dependency tensor. (optional, default=100) 20 | :param initial_partition: mapping of state-nodes to initial meso-set assignment. 21 | (optional, default=sampled from null distribution) 22 | 23 | :return: sampled partition as a mapping (dict) from state-nodes to meso-set assignments. 24 | """ 25 | if initial_partition is None: 26 | partition = {node: null_distribution(node) for node in dependency_tensor.state_nodes()} 27 | else: 28 | partition = {node: initial_partition[node] for node in dependency_tensor.state_nodes()} 29 | 30 | random_layers = list(dependency_tensor.random_aspect_layers()) 31 | if len(random_layers) <= 1: 32 | n_updates = 1 33 | else: 34 | n_updates = updates * len(random_layers) 35 | for ordered_layer in dependency_tensor.ordered_aspect_layers(): 36 | for it in range(n_updates): 37 | random_layer = _rand.choice(random_layers) 38 | layer = tuple(o+r for o, r in zip(ordered_layer, random_layer)) 39 | for node in dependency_tensor.state_nodes(layer): 40 | update_node = dependency_tensor.getrandneighbour(node) 41 | if update_node == node: 42 | partition[node] = null_distribution(node) 43 | else: 44 | partition[node] = partition[update_node] 45 | 46 | return partition 47 | 48 | 49 | class DirichletNull: 50 | def __init__(self, layers, theta, n_sets): 51 | """ 52 | Samples meso-set assignment probabilities from a Dirichlet distribution and returns a categorical null-distribution 53 | based on these probabilities. 54 | 55 | :param layers: [a_1,...a_d] Number of layers for each aspect 56 | :param theta: concentration parameter for the dirichlet distribution 57 | :param n_sets: number of meso-sets 58 | :return: null distribution function: (state-node) -> random meso-set 59 | """ 60 | self.weights = dict() 61 | for layer in SubscriptIterator(layers): 62 | self.weights[layer] = list(accumulate(dirichlet(theta, n_sets))) 63 | 64 | def __call__(self, node): 65 | return categorical(self.weights[node[1:]]) 66 | 67 | 68 | def dirichlet_null(layers, theta, n_sets): 69 | """ 70 | Samples meso-set assignment probabilities from a Dirichlet distribution and returns a categorical null-distribution 71 | based on these probabilities. (Function form provided for backwards compatibility. New code should use the class 72 | version directly.) 73 | 74 | :param layers: [a_1,...a_d] Number of layers for each aspect 75 | :param theta: concentration parameter for the dirichlet distribution 76 | :param n_sets: number of meso-sets 77 | :return: null distribution function: (state-node) -> random meso-set 78 | """ 79 | return DirichletNull(layers, theta, n_sets) 80 | 81 | 82 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | setup( 4 | name='multilayerGM', 5 | description='A Python framework for generating multilayer networks with planted mesoscale structure.', 6 | url='https://github.com/MultilayerGM/MultilayerGM-py', 7 | use_scm_version=True, 8 | setup_requires=['setuptools_scm'], 9 | author='Lucas G. S. Jeub', 10 | python_requires='>=3.5', 11 | packages=find_packages(), 12 | install_requires=[ 13 | 'numpy', 14 | 'nxmultilayer @ git+https://github.com/LJeub/nxMultilayerNet.git#egg=nxmultilayer' 15 | ], 16 | ) 17 | --------------------------------------------------------------------------------