├── .gitignore ├── LICENSE ├── MANIFEST.in ├── README.rst ├── setup.py └── src ├── vconnector-cli └── vconnector ├── __init__.py ├── cache.py ├── core.py └── exceptions.py /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | \#* 3 | build/ 4 | dist/ 5 | src/vconnector.egg-info/ 6 | venv 7 | .idea 8 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2013-2018 Marin Atanasov Nikolov 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions 6 | are met: 7 | 8 | 1. Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer 10 | in this position and unchanged. 11 | 2. Redistributions in binary form must reproduce the above copyright 12 | notice, this list of conditions and the following disclaimer in the 13 | documentation and/or other materials provided with the distribution. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR 16 | IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 17 | OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 18 | IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, 19 | INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 20 | NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 21 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 22 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 24 | THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.* 2 | include LICENSE 3 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | vConnector - VMware vSphere Connector Module for Python 2 | ======================================================= 3 | 4 | .. image:: https://img.shields.io/pypi/v/vconnector.svg 5 | :target: https://pypi.python.org/pypi/vconnector/ 6 | :alt: Latest Version 7 | 8 | .. image:: https://img.shields.io/pypi/dm/vconnector.svg 9 | :target: https://pypi.python.org/pypi/vconnector/ 10 | :alt: Downloads 11 | 12 | vConnector is a wrapper module around 13 | `pyVmomi VMware vSphere bindings `_, 14 | which provides methods for connecting and retrieving of 15 | objects from a VMware vSphere server. 16 | 17 | The purpose of vConnector is to provide the basic primitives for 18 | building complex applications. vConnector can also be used for 19 | managing the user/pass/host credentials for your vSphere environment 20 | using an SQLite database, which in turn can be shared between 21 | multiple applications requiring access to your vSphere environment 22 | through a common interface. 23 | 24 | Requirements 25 | ============ 26 | 27 | * `Python 2.7.x, 3.2.x or later `_ 28 | * `docopt `_ 29 | * `pyVmomi `_ 30 | * `tabulate `_ 31 | 32 | Contributions 33 | ============= 34 | 35 | vConnector is hosted on 36 | `Github `_. Please contribute 37 | by reporting issues, suggesting features or by sending patches 38 | using pull requests. 39 | 40 | Installation 41 | ============ 42 | 43 | The easiest way to install vConnector is by using ``pip``: 44 | 45 | .. code-block:: bash 46 | 47 | $ pip install vconnector 48 | 49 | In order to install the latest version of vConnector from the 50 | Github repository simply execute these commands instead: 51 | 52 | .. code-block:: bash 53 | 54 | $ git clone https://github.com/dnaeon/py-vconnector.git 55 | $ cd py-vconnector 56 | $ python setup.py install 57 | 58 | Applications using vConnector module 59 | ==================================== 60 | 61 | * `vPoller - Distributed vSphere API Proxy `_ 62 | * `vEvents - VMware vSphere Events from the command-line `_ 63 | 64 | Using the vconnector-cli tool 65 | ============================= 66 | 67 | Using the ``vconnector-cli`` tool you can manage the user/pass/host 68 | credentials of your vSphere environment. The ``vconnector-cli`` tool 69 | stores this information in an SQLite database file, 70 | which also makes it easy to be shared between applications. 71 | 72 | First, initialize the vConnector database by executing the 73 | command below: 74 | 75 | .. code-block:: bash 76 | 77 | $ vconnector-cli init 78 | 79 | Here is how to add a new vSphere host to the vConnector database: 80 | 81 | .. code-block:: bash 82 | 83 | $ vconnector-cli -H vc01.example.org -U root -P p4ssw0rd add 84 | 85 | Here is how to update an already existing vSphere host 86 | from the vConnector database: 87 | 88 | .. code-block:: bash 89 | 90 | $ vconnector-cli -H vc01.example.org -U root -P newp4ssw0rd update 91 | 92 | Here is how to remove a vSphere host using vconnector-cli: 93 | 94 | .. code-block:: bash 95 | 96 | $ vconnector-cli -H vc01.example.org remove 97 | 98 | Here is how to enable a vSphere host using vconnector-cli: 99 | 100 | .. code-block:: bash 101 | 102 | $ vconnector-cli -H vc01.example.org enable 103 | 104 | Here this is how to disable a vSphere host: 105 | 106 | .. code-block:: bash 107 | 108 | $ vconnector-cli -H vc01.example.org disable 109 | 110 | And here is how to get the currently registered vSphere hosts from 111 | the vConnector database: 112 | 113 | .. code-block:: bash 114 | 115 | $ vconnector-cli get 116 | +---------------------------+---------------------+--------------+-----------+ 117 | | Hostname | Username | Password | Enabled | 118 | +===========================+=====================+==============+===========+ 119 | | vc01.example.org | root | p4ssw0rd | 0 | 120 | +---------------------------+---------------------+--------------+-----------+ 121 | 122 | Using the vConnector API 123 | ======================== 124 | 125 | Here are a few examples of using the ``vconnector`` module API. 126 | 127 | Connecting to a vSphere host: 128 | 129 | .. code-block:: python 130 | 131 | >>> from vconnector.core import VConnector 132 | >>> client = VConnector( 133 | ... user='root', 134 | ... pwd='p4ssw0rd', 135 | ... host='vc01.example.org' 136 | ...) 137 | >>> client.connect() 138 | 139 | Disconnecting from a vSphere host: 140 | 141 | .. code-block:: python 142 | 143 | >>> client.disconnect() 144 | 145 | Re-connecting to a vSphere host: 146 | 147 | .. code-block:: python 148 | 149 | >>> client.reconnect() 150 | 151 | How to get a ``VMware vSphere View`` of all ``VirtualMachine`` 152 | managed objects: 153 | 154 | .. code-block:: python 155 | 156 | >>> from __future__ import print_function 157 | >>> from vconnector.core import VConnector 158 | >>> client = VConnector( 159 | ... user='root', 160 | ... pwd='p4ssw0rd', 161 | ... host='vc01.example.org' 162 | ...) 163 | >>> client.connect() 164 | >>> vms = client.get_vm_view() 165 | >>> print(vms.view) 166 | (ManagedObject) [ 167 | 'vim.VirtualMachine:vm-36', 168 | 'vim.VirtualMachine:vm-129', 169 | 'vim.VirtualMachine:vm-162', 170 | 'vim.VirtualMachine:vm-146', 171 | 'vim.VirtualMachine:vm-67', 172 | 'vim.VirtualMachine:vm-147', 173 | 'vim.VirtualMachine:vm-134', 174 | 'vim.VirtualMachine:vm-88' 175 | ] 176 | >>> client.disconnect() 177 | 178 | How to get a ``Managed Object`` by a specific property, e.g. find the 179 | Managed Object of an ESXi host which name is ``esxi01.example.org``: 180 | 181 | .. code-block:: python 182 | 183 | >>> from __future__ import print_function 184 | >>> import pyVmomi 185 | >>> from vconnector.core import VConnector 186 | >>> client = VConnector( 187 | ... user='root', 188 | ... pwd='p4ssw0rd', 189 | ... host='vc01.example.org' 190 | ... ) 191 | >>> client.connect() 192 | >>> host = client.get_object_by_property( 193 | ... property_name='name', 194 | ... property_value='esxi01.example.org', 195 | ... obj_type=pyVmomi.vim.HostSystem 196 | ... ) 197 | >>> print(host.name) 198 | 'esxi01.example.org' 199 | >>> client.disconnect() 200 | 201 | How to collect properties for ``vSphere Managed Objects``, e.g. get 202 | the ``name`` and ``capacity`` properties for all ``Datastore`` 203 | managed objects: 204 | 205 | .. code-block:: python 206 | 207 | >>> from __future__ import print_function 208 | >>> import pyVmomi 209 | >>> from vconnector.core import VConnector 210 | >>> client = VConnector( 211 | ... user='root', 212 | ... pwd='p4ssw0rd', 213 | ... host='vc01.example.org' 214 | ... ) 215 | >>> client.connect() 216 | >>> datastores = client.get_datastore_view() 217 | >>> result = client.collect_properties( 218 | ... view_ref=datastores, 219 | ... obj_type=pyVmomi.vim.Datastore, 220 | ... path_set=['name', 'summary.capacity'] 221 | ...) 222 | >>> print(result) 223 | [{u'summary.capacity': 994821799936L, u'name': 'datastore1'}] 224 | >>> client.disconnect() 225 | 226 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import re 2 | import ast 3 | 4 | from setuptools import setup, find_packages 5 | 6 | _version_re = re.compile(r'__version__\s+=\s+(.*)') 7 | 8 | with open('src/vconnector/__init__.py', 'rb') as f: 9 | version = str(ast.literal_eval(_version_re.search( 10 | f.read().decode('utf-8')).group(1)) 11 | ) 12 | 13 | setup(name='vconnector', 14 | version=version, 15 | description='VMware vSphere Connector Module for Python', 16 | long_description=open('README.rst').read(), 17 | author='Marin Atanasov Nikolov', 18 | author_email='dnaeon@gmail.com', 19 | license='BSD', 20 | url='https://github.com/dnaeon/py-vconnector', 21 | download_url='https://github.com/dnaeon/py-vconnector/releases', 22 | package_dir={'': 'src'}, 23 | packages=find_packages('src'), 24 | scripts=[ 25 | 'src/vconnector-cli', 26 | ], 27 | install_requires=[ 28 | 'pyvmomi >= 6.7.3', 29 | 'docopt >= 0.6.2', 30 | 'tabulate >= 0.8.3', 31 | ] 32 | ) 33 | -------------------------------------------------------------------------------- /src/vconnector-cli: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 3 | # Copyright (c) 2013-2015 Marin Atanasov Nikolov 4 | # All rights reserved. 5 | # 6 | # Redistribution and use in source and binary forms, with or without 7 | # modification, are permitted provided that the following conditions 8 | # are met: 9 | # 1. Redistributions of source code must retain the above copyright 10 | # notice, this list of conditions and the following disclaimer 11 | # in this position and unchanged. 12 | # 2. Redistributions in binary form must reproduce the above copyright 13 | # notice, this list of conditions and the following disclaimer in the 14 | # documentation and/or other materials provided with the distribution. 15 | # 16 | # THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR 17 | # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 18 | # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 19 | # IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, 20 | # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 21 | # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 22 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 23 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 24 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 25 | # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | 27 | """ 28 | vconnector-cli is an application used for managing vSphere 29 | connection details using an SQLite database backend. 30 | 31 | """ 32 | 33 | from __future__ import print_function 34 | 35 | import logging 36 | 37 | from docopt import docopt 38 | from tabulate import tabulate 39 | from vconnector import __version__ 40 | from vconnector.core import VConnectorDatabase 41 | 42 | def init_db(db): 43 | """ 44 | Initialize the vConnector database 45 | 46 | Args: 47 | db (str): Path to the SQLite database file 48 | 49 | """ 50 | try: 51 | db = VConnectorDatabase(db) 52 | db.init_db() 53 | except Exception as e: 54 | raise SystemExit("Cannot initialize database: {}".format(e)) 55 | 56 | def add_update_agent(db, user, pwd, host): 57 | """ 58 | Add/update a vSphere Agent in the database 59 | 60 | Args: 61 | db (str): Path to the vConnector database file 62 | user (str): Username to use for this vSphere Agent 63 | pwd (str): Password to use for this vSphere Agent 64 | host (str): Hostname of the vSphere server 65 | 66 | """ 67 | try: 68 | db = VConnectorDatabase(db) 69 | db.add_update_agent( 70 | user=user, 71 | pwd=pwd, 72 | host=host, 73 | ) 74 | except Exception as e: 75 | raise SystemExit("Cannot update database: {}".format(e)) 76 | 77 | def remove_agent(db, host): 78 | """ 79 | Remove a vSphere Agent from the database 80 | 81 | Args: 82 | db (str): Path to the vConnector database file 83 | host (str): Hostname of the vSphere server 84 | 85 | """ 86 | try: 87 | db = VConnectorDatabase(db) 88 | db.remove_agent(host=host) 89 | except Exception as e: 90 | raise SystemExit("Cannot update database: {}".format(e)) 91 | 92 | def get_agents(db): 93 | """ 94 | Get the registered vSphere Agents 95 | 96 | Args: 97 | db (str): Path to the vConnector database file 98 | 99 | """ 100 | try: 101 | db = VConnectorDatabase(db) 102 | agents = db.get_agents() 103 | except Exception as e: 104 | raise SystemExit("Cannot read database: {}".format(e)) 105 | 106 | print(tabulate( 107 | agents, 108 | headers=['Hostname', 'Username', 'Password', 'Enabled'], 109 | tablefmt='grid' 110 | )) 111 | 112 | def enable_agent(db, host): 113 | """ 114 | Mark a vSphere Agent as enabled 115 | 116 | Args: 117 | db (str): Path to the vConnector database file 118 | host (str): Hostname of the vSphere Agent to enable 119 | 120 | """ 121 | try: 122 | db = VConnectorDatabase(db) 123 | db.enable_agent(host=host) 124 | except Exception as e: 125 | raise SystemExit("Cannot update database: {}".format(e)) 126 | 127 | def disable_agent(db, host): 128 | """ 129 | Mark a vSphere Agent as disabled 130 | 131 | Args: 132 | db (str): Path to the vConnector database file 133 | host (str): Hostname of the vSphere Agent to disable 134 | 135 | """ 136 | try: 137 | db = VConnectorDatabase(db) 138 | db.disable_agent(host=host) 139 | except Exception as e: 140 | raise SystemExit("Cannot update database: {}".format(e)) 141 | 142 | 143 | def main(): 144 | usage=""" 145 | Usage: vconnector-cli [-D] [-d ] init 146 | vconnector-cli [-D] [-d ] get 147 | vconnector-cli [-D] [-d ] -H (enable|disable|remove) 148 | vconnector-cli [-D] [-d ] -H -U -P (add|update) 149 | vconnector-cli (-h|-v) 150 | Arguments: 151 | add Add a vSphere Agent to the database 152 | update Update a vSphere Agent in the database 153 | remove Remove a vSphere Agent from the database 154 | get Get all registered vSphere Agents 155 | enable Mark this vSphere Agent as enabled 156 | disable Mark this vSphere Agent as disabled 157 | 158 | Options: 159 | -h, --help Display this usage info 160 | -v, --version Display version and exit 161 | -D, --debug Debug mode, be more verbose 162 | -d , --database Path to the SQLite database file 163 | [default: /var/lib/vconnector/vconnector.db] 164 | -H , --host Specify the hostname of the vSphere Agent 165 | -U , --user Username to use when connecting to the vSphere Agent 166 | -P , --pwd Password to use when connecting to the vSphere Agent 167 | 168 | """ 169 | 170 | args = docopt(usage, version=__version__) 171 | 172 | level = logging.DEBUG if args['--debug'] else logging.INFO 173 | 174 | logging.basicConfig( 175 | format='%(asctime)s - %(levelname)s - vconnector-cli[%(process)s]: %(message)s', 176 | level=level 177 | ) 178 | 179 | if args['init']: 180 | init_db(args['--database']) 181 | elif args['add'] or args['update']: 182 | add_update_agent( 183 | db=args['--database'], 184 | user=args['--user'], 185 | pwd=args['--pwd'], 186 | host=args['--host'], 187 | ) 188 | elif args['remove']: 189 | remove_agent( 190 | db=args['--database'], 191 | host=args['--host'] 192 | ) 193 | elif args['get']: 194 | get_agents(db=args['--database']) 195 | elif args['enable']: 196 | enable_agent(db=args['--database'], host=args['--host']) 197 | elif args['disable']: 198 | disable_agent(db=args['--database'], host=args['--host']) 199 | 200 | if __name__ == '__main__': 201 | main() 202 | 203 | -------------------------------------------------------------------------------- /src/vconnector/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = '0.6.0' 2 | -------------------------------------------------------------------------------- /src/vconnector/cache.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 Marin Atanasov Nikolov 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions 6 | # are met: 7 | # 1. Redistributions of source code must retain the above copyright 8 | # notice, this list of conditions and the following disclaimer 9 | # in this position and unchanged. 10 | # 2. Redistributions in binary form must reproduce the above copyright 11 | # notice, this list of conditions and the following disclaimer in the 12 | # documentation and/or other materials provided with the distribution. 13 | # 14 | # THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR 15 | # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 16 | # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 17 | # IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, 18 | # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 19 | # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 20 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 21 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 22 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 23 | # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 24 | 25 | """ 26 | The vConnector caching module 27 | 28 | """ 29 | 30 | import logging 31 | import threading 32 | 33 | from time import time 34 | from collections import OrderedDict, namedtuple 35 | 36 | from vconnector.exceptions import CacheException 37 | 38 | __all__ = ['CachedObject', 'CacheInventory'] 39 | 40 | _CachedObjectInfo = namedtuple('CachedObjectInfo', ['name', 'hits', 'ttl', 'timestamp']) 41 | 42 | 43 | class CachedObject(object): 44 | def __init__(self, name, obj, ttl): 45 | """ 46 | Initializes a new cached object 47 | 48 | Args: 49 | name (str): Human readable name for the cached entry 50 | obj (type): Object to be cached 51 | ttl (int): The TTL in seconds for the cached object 52 | 53 | """ 54 | self.hits = 0 55 | self.name = name 56 | self.obj = obj 57 | self.ttl = ttl 58 | self.timestamp = time() 59 | 60 | class CacheInventory(object): 61 | """ 62 | Inventory for cached objects 63 | 64 | """ 65 | def __init__(self, maxsize=0, housekeeping=0): 66 | """ 67 | Initializes a new cache inventory 68 | 69 | Args: 70 | maxsize (int): Upperbound limit on the number of items 71 | that will be stored in the cache inventory 72 | housekeeping (int): Time in minutes to perform periodic 73 | cache housekeeping 74 | 75 | Raises: 76 | CacheException 77 | 78 | """ 79 | if maxsize < 0: 80 | raise CacheException('Cache inventory size cannot be negative') 81 | 82 | if housekeeping < 0: 83 | raise CacheException('Cache housekeeping period cannot be negative') 84 | 85 | self._cache = OrderedDict() 86 | self.maxsize = maxsize 87 | self.housekeeping = housekeeping * 60.0 88 | self.lock = threading.RLock() 89 | self._schedule_housekeeper() 90 | 91 | def __len__(self): 92 | with self.lock: 93 | return len(self._cache) 94 | 95 | def __contains__(self, key): 96 | with self.lock: 97 | if key not in self._cache: 98 | return False 99 | 100 | item = self._cache[key] 101 | return not self._has_expired(item) 102 | 103 | def _has_expired(self, item): 104 | """ 105 | Checks if a cached item has expired and removes it if needed 106 | 107 | Args: 108 | item (CachedObject): A cached object to lookup 109 | 110 | Returns: 111 | bool: True if the item has expired, False otherwise 112 | 113 | """ 114 | with self.lock: 115 | if time() > item.timestamp + item.ttl: 116 | logging.debug( 117 | 'Object %s has expired and will be removed from cache', 118 | self.info(item.name) 119 | ) 120 | self._cache.pop(item.name) 121 | return True 122 | return False 123 | 124 | def _schedule_housekeeper(self): 125 | """ 126 | Schedules the next run of the housekeeper 127 | 128 | """ 129 | if self.housekeeping > 0: 130 | t = threading.Timer( 131 | interval=self.housekeeping, 132 | function=self._housekeeper 133 | ) 134 | t.setDaemon(True) 135 | t.start() 136 | 137 | def _housekeeper(self): 138 | """ 139 | Remove expired entries from the cache on regular basis 140 | 141 | """ 142 | with self.lock: 143 | expired = 0 144 | logging.info( 145 | 'Starting cache housekeeper [%d item(s) in cache]', 146 | len(self._cache) 147 | ) 148 | 149 | items = list(self._cache.values()) 150 | for item in items: 151 | if self._has_expired(item): 152 | expired += 1 153 | 154 | logging.info( 155 | 'Cache housekeeper completed [%d item(s) removed from cache]', 156 | expired 157 | ) 158 | self._schedule_housekeeper() 159 | 160 | def add(self, obj): 161 | """ 162 | Add an item to the cache inventory 163 | 164 | If the upperbound limit has been reached then the last item 165 | is being removed from the inventory. 166 | 167 | Args: 168 | obj (CachedObject): A CachedObject instance to be added 169 | 170 | Raises: 171 | CacheException 172 | 173 | """ 174 | if not isinstance(obj, CachedObject): 175 | raise CacheException('Need a CachedObject instance to add in the cache') 176 | 177 | with self.lock: 178 | if 0 < self.maxsize == len(self._cache): 179 | popped = self._cache.popitem(last=False) 180 | logging.debug( 181 | 'Cache maxsize reached, removing %s', 182 | self.info(name=popped.name) 183 | ) 184 | 185 | self._cache[obj.name] = obj 186 | logging.debug('Adding object to cache %s', self.info(name=obj.name)) 187 | 188 | def get(self, name): 189 | """ 190 | Retrieve an object from the cache inventory 191 | 192 | Args: 193 | name (str): Name of the cache item to retrieve 194 | 195 | Returns: 196 | The cached object if found, None otherwise 197 | 198 | """ 199 | with self.lock: 200 | if name not in self._cache: 201 | return None 202 | 203 | item = self._cache[name] 204 | if self._has_expired(item): 205 | return None 206 | 207 | item.hits += 1 208 | logging.debug( 209 | 'Returning object from cache %s', 210 | self.info(name=item.name) 211 | ) 212 | 213 | return item.obj 214 | 215 | def clear(self): 216 | """ 217 | Remove all items from the cache 218 | 219 | """ 220 | with self.lock: 221 | self._cache.clear() 222 | 223 | def info(self, name): 224 | """ 225 | Get statistics about a cached object 226 | 227 | Args: 228 | name (str): Name of the cached object 229 | 230 | """ 231 | with self.lock: 232 | if name not in self._cache: 233 | return None 234 | 235 | item = self._cache[name] 236 | return _CachedObjectInfo(item.name, item.hits, item.ttl, item.timestamp) 237 | -------------------------------------------------------------------------------- /src/vconnector/core.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2013-2015 Marin Atanasov Nikolov 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions 6 | # are met: 7 | # 1. Redistributions of source code must retain the above copyright 8 | # notice, this list of conditions and the following disclaimer 9 | # in this position and unchanged. 10 | # 2. Redistributions in binary form must reproduce the above copyright 11 | # notice, this list of conditions and the following disclaimer in the 12 | # documentation and/or other materials provided with the distribution. 13 | # 14 | # THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR 15 | # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 16 | # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 17 | # IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, 18 | # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 19 | # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 20 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 21 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 22 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 23 | # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 24 | 25 | """ 26 | VMware vSphere Connector Module for Python 27 | 28 | This module provides classes and methods for managing 29 | connections to VMware vSphere hosts and retrieving of 30 | object properties. 31 | 32 | """ 33 | 34 | import ssl 35 | import logging 36 | import sqlite3 37 | 38 | import pyVmomi 39 | import pyVim.connect 40 | 41 | from vconnector.cache import CachedObject 42 | from vconnector.cache import CacheInventory 43 | from vconnector.exceptions import VConnectorException 44 | 45 | __all__ = ['VConnector', 'VConnectorDatabase'] 46 | 47 | 48 | class VConnector(object): 49 | """ 50 | VConnector class 51 | 52 | The VConnector class defines methods for connecting, 53 | disconnecting and retrieving of objects from a 54 | VMware vSphere host. 55 | 56 | Returns: 57 | VConnector object 58 | 59 | Raises: 60 | VConnectorException 61 | 62 | """ 63 | def __init__(self, 64 | user, 65 | pwd, 66 | host, 67 | port=443, 68 | ssl_context=None, 69 | cache_maxsize=0, 70 | cache_enabled=False, 71 | cache_ttl=300, 72 | cache_housekeeping=0 73 | ): 74 | """ 75 | Initializes a new VConnector object 76 | 77 | Args: 78 | user (str): Username to use when connecting 79 | pwd (str): Password to use when connecting 80 | host (str): VMware vSphere host to connect to 81 | port (int): VMware vSphere port to connect to 82 | ssl_context (ssl.SSLContext): SSL context to use for the connection 83 | cache_maxsize (int): Upperbound limit on the number of 84 | items that will be stored in the cache 85 | cache_enabled (bool): If True use an expiring cache for the 86 | managed objects 87 | cache_ttl (int): Time in seconds after which a 88 | cached object is considered as expired 89 | cache_housekeeping (int): Time in minutes to perform periodic 90 | cache housekeeping 91 | 92 | """ 93 | self.user = user 94 | self.pwd = pwd 95 | self.host = host 96 | self.port = port 97 | self.ssl_context = None 98 | 99 | if not ssl_context: 100 | ctx = ssl.create_default_context() 101 | ctx.check_hostname = False 102 | ctx.verify_mode = ssl.CERT_NONE 103 | self.ssl_context = ctx 104 | else: 105 | self.ssl_context = ssl_context 106 | 107 | self._si = None 108 | self._perf_counter = None 109 | self._perf_interval = None 110 | self.cache_maxsize = cache_maxsize 111 | self.cache_enabled = cache_enabled 112 | self.cache_ttl = cache_ttl 113 | self.cache_housekeeping = cache_housekeeping 114 | self.cache = CacheInventory( 115 | maxsize=self.cache_maxsize, 116 | housekeeping=self.cache_housekeeping 117 | ) 118 | 119 | @property 120 | def si(self): 121 | if not self._si: 122 | self.connect() 123 | if not self._si.content.sessionManager.currentSession: 124 | logging.warning( 125 | '[%s] Lost connection to vSphere host, trying to reconnect', 126 | self.host 127 | ) 128 | self.connect() 129 | return self._si 130 | 131 | @property 132 | def perf_counter(self): 133 | if not self._perf_counter: 134 | self._perf_counter = self.si.content.perfManager.perfCounter 135 | return self._perf_counter 136 | 137 | @property 138 | def perf_interval(self): 139 | if not self._perf_interval: 140 | self._perf_interval = self.si.content.perfManager.historicalInterval 141 | return self._perf_interval 142 | 143 | def connect(self): 144 | """ 145 | Connect to the VMware vSphere host 146 | 147 | Raises: 148 | VConnectorException 149 | 150 | """ 151 | logging.info('Connecting vSphere Agent to %s', self.host) 152 | 153 | try: 154 | self._si = pyVim.connect.SmartConnect( 155 | host=self.host, 156 | user=self.user, 157 | pwd=self.pwd, 158 | port=self.port, 159 | sslContext=self.ssl_context 160 | ) 161 | except Exception as e: 162 | # TODO: Maybe retry connection after some time 163 | # before we finally give up on this vSphere host 164 | logging.error('Cannot connect to %s: %s', self.host, e) 165 | raise 166 | 167 | def disconnect(self): 168 | """ 169 | Disconnect from the VMware vSphere host 170 | 171 | """ 172 | if not self._si: 173 | return 174 | 175 | logging.info('Disconnecting vSphere Agent from %s', self.host) 176 | pyVim.connect.Disconnect(self.si) 177 | 178 | def reconnect(self): 179 | """ 180 | Reconnect to the VMware vSphere host 181 | 182 | """ 183 | self.disconnect() 184 | self.connect() 185 | 186 | def get_datacenter_view(self): 187 | """ 188 | Get a view ref to all vim.Datacenter managed objects 189 | 190 | """ 191 | return self.get_container_view( 192 | obj_type=[pyVmomi.vim.Datacenter] 193 | ) 194 | 195 | def get_cluster_view(self): 196 | """ 197 | Get a view ref to all vim.ClusterComputeResource managed objects 198 | 199 | """ 200 | return self.get_container_view( 201 | obj_type=[pyVmomi.vim.ClusterComputeResource] 202 | ) 203 | 204 | def get_host_view(self): 205 | """ 206 | Get a view ref to all vim.HostSystem managed objects 207 | 208 | """ 209 | return self.get_container_view( 210 | obj_type=[pyVmomi.vim.HostSystem] 211 | ) 212 | 213 | def get_vm_view(self): 214 | """ 215 | Get a view ref to all vim.VirtualMachine managed objects 216 | 217 | """ 218 | return self.get_container_view( 219 | obj_type=[pyVmomi.vim.VirtualMachine] 220 | ) 221 | 222 | def get_datastore_view(self): 223 | """ 224 | Get a view ref to all vim.Datastore managed objects 225 | 226 | """ 227 | return self.get_container_view( 228 | obj_type=[pyVmomi.vim.Datastore] 229 | ) 230 | 231 | def get_resource_pool_view(self): 232 | """ 233 | Get a view ref to all vim.ResourcePool managed objects 234 | 235 | """ 236 | return self.get_container_view( 237 | obj_type=[pyVmomi.vim.ResourcePool] 238 | ) 239 | 240 | def get_distributed_vswitch_view(self): 241 | """ 242 | Get a view ref to all vim.DistributedVirtualSwitch managed objects 243 | 244 | """ 245 | return self.get_container_view( 246 | obj_type=[pyVmomi.vim.DistributedVirtualSwitch] 247 | ) 248 | 249 | def collect_properties(self, 250 | view_ref, 251 | obj_type, 252 | path_set=[], 253 | include_mors=False): 254 | """ 255 | Collect properties for managed objects from a view ref 256 | 257 | Check the vSphere API documentation for example on 258 | retrieving object properties: 259 | 260 | - http://pubs.vmware.com/vsphere-50/index.jsp#com.vmware.wssdk.pg.doc_50/PG_Ch5_PropertyCollector.7.2.html 261 | 262 | Args: 263 | view_ref (pyVmomi.vim.view.*): Starting point of inventory navigation 264 | obj_type (pyVmomi.vim.*): Type of managed object 265 | path_set (list): List of properties to retrieve 266 | include_mors (bool): If True include the managed objects refs in the result 267 | 268 | Returns: 269 | A list of properties for the managed objects 270 | 271 | """ 272 | collector = self.si.content.propertyCollector 273 | 274 | logging.debug( 275 | '[%s] Collecting properties for %s managed objects', 276 | self.host, 277 | obj_type.__name__ 278 | ) 279 | 280 | # Create object specification to define the starting point of 281 | # inventory navigation 282 | obj_spec = pyVmomi.vmodl.query.PropertyCollector.ObjectSpec() 283 | obj_spec.obj = view_ref 284 | obj_spec.skip = True 285 | 286 | # Create a traversal specification to identify the 287 | # path for collection 288 | traversal_spec = pyVmomi.vmodl.query.PropertyCollector.TraversalSpec() 289 | traversal_spec.name = 'traverseEntities' 290 | traversal_spec.path = 'view' 291 | traversal_spec.skip = False 292 | traversal_spec.type = view_ref.__class__ 293 | obj_spec.selectSet = [traversal_spec] 294 | 295 | # Identify the properties to the retrieved 296 | property_spec = pyVmomi.vmodl.query.PropertyCollector.PropertySpec() 297 | property_spec.type = obj_type 298 | 299 | if not path_set: 300 | logging.warning( 301 | '[%s] Retrieving all properties for objects, this might take a while...', 302 | self.host 303 | ) 304 | property_spec.all = True 305 | 306 | property_spec.pathSet = path_set 307 | 308 | # Add the object and property specification to the 309 | # property filter specification 310 | filter_spec = pyVmomi.vmodl.query.PropertyCollector.FilterSpec() 311 | filter_spec.objectSet = [obj_spec] 312 | filter_spec.propSet = [property_spec] 313 | 314 | # Retrieve properties 315 | props = collector.RetrieveContents([filter_spec]) 316 | 317 | data = [] 318 | for obj in props: 319 | properties = {} 320 | for prop in obj.propSet: 321 | properties[prop.name] = prop.val 322 | 323 | if include_mors: 324 | properties['obj'] = obj.obj 325 | 326 | data.append(properties) 327 | 328 | return data 329 | 330 | def get_container_view(self, obj_type, container=None): 331 | """ 332 | Get a vSphere Container View reference to all 333 | objects of type 'obj_type' 334 | 335 | It is up to the caller to take care of destroying the View 336 | when no longer needed. 337 | 338 | Args: 339 | obj_type (list): A list of managed object types 340 | container (vim.ManagedEntity): Starting point of inventory search 341 | 342 | Returns: 343 | A container view ref to the discovered managed objects 344 | 345 | """ 346 | if not container: 347 | container = self.si.content.rootFolder 348 | 349 | logging.debug( 350 | '[%s] Getting container view ref to %s managed objects', 351 | self.host, 352 | [t.__name__ for t in obj_type] 353 | ) 354 | 355 | view_ref = self.si.content.viewManager.CreateContainerView( 356 | container=container, 357 | type=obj_type, 358 | recursive=True 359 | ) 360 | 361 | return view_ref 362 | 363 | def get_list_view(self, obj): 364 | """ 365 | Get a vSphere List View reference 366 | 367 | It is up to the caller to take care of destroying the View 368 | when no longer needed. 369 | 370 | Args: 371 | obj (list): A list of managed object to include in the View 372 | 373 | Returns: 374 | A list view ref to the managed objects 375 | 376 | """ 377 | view_ref = self.si.content.viewManager.CreateListView(obj=obj) 378 | 379 | logging.debug( 380 | '[%s] Getting list view ref for %s objects', 381 | self.host, 382 | [o.name for o in obj] 383 | ) 384 | 385 | return view_ref 386 | 387 | def get_object_by_property(self, property_name, property_value, obj_type): 388 | """ 389 | Find a Managed Object by a propery 390 | 391 | If cache is enabled then we search for the managed object from the 392 | cache first and if present we return the object from cache. 393 | 394 | Args: 395 | property_name (str): Name of the property to look for 396 | property_value (str): Value of the property to match 397 | obj_type (pyVmomi.vim.*): Type of the Managed Object 398 | 399 | Returns: 400 | The first matching object 401 | 402 | """ 403 | if not issubclass(obj_type, pyVmomi.vim.ManagedEntity): 404 | raise VConnectorException('Type should be a subclass of vim.ManagedEntity') 405 | 406 | if self.cache_enabled: 407 | cached_obj_name = '{}:{}'.format(obj_type.__name__, property_value) 408 | if cached_obj_name in self.cache: 409 | logging.debug('Using cached object %s', cached_obj_name) 410 | return self.cache.get(cached_obj_name) 411 | 412 | view_ref = self.get_container_view(obj_type=[obj_type]) 413 | props = self.collect_properties( 414 | view_ref=view_ref, 415 | obj_type=obj_type, 416 | path_set=[property_name], 417 | include_mors=True 418 | ) 419 | view_ref.DestroyView() 420 | 421 | obj = None 422 | for each_obj in props: 423 | if each_obj[property_name] == property_value: 424 | obj = each_obj['obj'] 425 | break 426 | 427 | if self.cache_enabled: 428 | cached_obj = CachedObject( 429 | name=cached_obj_name, 430 | obj=obj, 431 | ttl=self.cache_ttl 432 | ) 433 | self.cache.add(obj=cached_obj) 434 | 435 | return obj 436 | 437 | class VConnectorDatabase(object): 438 | """ 439 | VConnectorDatabase class 440 | 441 | Provides an SQLite database backend for storing information 442 | about vSphere Agents, such as hostname, username, password, etc. 443 | 444 | Returns: 445 | VConnectorDatabase object 446 | 447 | Raises: 448 | VConnectorException 449 | 450 | """ 451 | def __init__(self, db): 452 | """ 453 | Initializes a new VConnectorDatabase object 454 | 455 | Args: 456 | db (str): Path to the SQLite database file 457 | 458 | """ 459 | self.db = db 460 | self.conn = sqlite3.connect(self.db) 461 | 462 | def init_db(self): 463 | """ 464 | Initializes the vConnector Database backend 465 | 466 | """ 467 | logging.info('Initializing vConnector database at %s', self.db) 468 | 469 | self.cursor = self.conn.cursor() 470 | 471 | sql = """ 472 | CREATE TABLE hosts ( 473 | host TEXT UNIQUE, 474 | user TEXT, 475 | pwd TEXT, 476 | enabled INTEGER 477 | ) 478 | """ 479 | 480 | try: 481 | self.cursor.execute(sql) 482 | except sqlite3.OperationalError as e: 483 | raise VConnectorException('Cannot initialize database: {}'.format(e.message)) 484 | 485 | self.conn.commit() 486 | self.cursor.close() 487 | 488 | def add_update_agent(self, host, user, pwd, enabled=0): 489 | """ 490 | Add/update a vSphere Agent in the vConnector database 491 | 492 | Args: 493 | host (str): Hostname of the vSphere host 494 | user (str): Username to use when connecting 495 | pwd (str): Password to use when connecting 496 | enabled (int): If True mark this vSphere Agent as enabled 497 | 498 | """ 499 | logging.info( 500 | 'Adding/updating vSphere Agent %s in database', 501 | host 502 | ) 503 | 504 | self.cursor = self.conn.cursor() 505 | self.cursor.execute( 506 | 'INSERT OR REPLACE INTO hosts VALUES (?,?,?,?)', 507 | (host, user, pwd, enabled) 508 | ) 509 | self.conn.commit() 510 | self.cursor.close() 511 | 512 | def remove_agent(self, host): 513 | """ 514 | Remove a vSphere Agent from the vConnector database 515 | 516 | Args: 517 | host (str): Hostname of the vSphere Agent to remove 518 | 519 | """ 520 | logging.info('Removing vSphere Agent %s from database', host) 521 | 522 | self.cursor = self.conn.cursor() 523 | self.cursor.execute( 524 | 'DELETE FROM hosts WHERE host = ?', 525 | (host,) 526 | ) 527 | self.conn.commit() 528 | self.cursor.close() 529 | 530 | def get_agents(self, only_enabled=False): 531 | """ 532 | Get the vSphere Agents from the vConnector database 533 | 534 | Args: 535 | only_enabled (bool): If True return only the Agents which are enabled 536 | 537 | """ 538 | logging.debug('Getting vSphere Agents from database') 539 | 540 | self.conn.row_factory = sqlite3.Row 541 | self.cursor = self.conn.cursor() 542 | 543 | if only_enabled: 544 | sql = 'SELECT * FROM hosts WHERE enabled = 1' 545 | else: 546 | sql = 'SELECT * FROM hosts' 547 | 548 | self.cursor.execute(sql) 549 | result = self.cursor.fetchall() 550 | self.cursor.close() 551 | 552 | return result 553 | 554 | def enable_agent(self, host): 555 | """ 556 | Mark a vSphere Agent as enabled 557 | 558 | Args: 559 | host (str): Hostname of the vSphere Agent to enable 560 | 561 | """ 562 | logging.info('Enabling vSphere Agent %s', host) 563 | 564 | self.cursor = self.conn.cursor() 565 | self.cursor.execute( 566 | 'UPDATE hosts SET enabled = 1 WHERE host = ?', 567 | (host,) 568 | ) 569 | self.conn.commit() 570 | self.cursor.close() 571 | 572 | def disable_agent(self, host): 573 | """ 574 | Mark a vSphere Agent as disabled 575 | 576 | Args: 577 | host (str): Hostname of the vSphere Agent to disable 578 | 579 | """ 580 | logging.info('Disabling vSphere Agent %s', host) 581 | 582 | self.cursor = self.conn.cursor() 583 | self.cursor.execute( 584 | 'UPDATE hosts SET enabled = 0 WHERE host = ?', 585 | (host,) 586 | ) 587 | self.conn.commit() 588 | self.cursor.close() 589 | -------------------------------------------------------------------------------- /src/vconnector/exceptions.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 Marin Atanasov Nikolov 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions 6 | # are met: 7 | # 1. Redistributions of source code must retain the above copyright 8 | # notice, this list of conditions and the following disclaimer 9 | # in this position and unchanged. 10 | # 2. Redistributions in binary form must reproduce the above copyright 11 | # notice, this list of conditions and the following disclaimer in the 12 | # documentation and/or other materials provided with the distribution. 13 | # 14 | # THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR 15 | # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 16 | # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 17 | # IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, 18 | # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 19 | # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 20 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 21 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 22 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 23 | # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 24 | 25 | """ 26 | vConnector exceptions module 27 | 28 | """ 29 | 30 | __all__ = ['VConnectorException', 'CacheException'] 31 | 32 | 33 | class VConnectorException(Exception): 34 | """ 35 | Generic VConnector exception 36 | 37 | """ 38 | pass 39 | 40 | class CacheException(VConnectorException): 41 | """ 42 | Inventory cache exception 43 | 44 | """ 45 | pass 46 | --------------------------------------------------------------------------------