├── automation ├── api_usage_example │ ├── requirements.txt │ ├── configs │ │ ├── security_x509_backup_agent.json │ │ ├── security_x509_monitoring_agent.json │ │ ├── security_ldap_backup_agent.json │ │ ├── security_ldap_monitoring_agent.json │ │ ├── security_kerberos_backup_agent.json │ │ ├── security_kerberos_monitoring_agent.json │ │ ├── api_0_clean.json │ │ ├── api_1_define_versions.json │ │ ├── api_2_install_other_agents.json │ │ ├── mongo34_rs.json │ │ ├── api_3_create_replica_set.json │ │ ├── api_4_upgrade_replica_set.json │ │ ├── security_ssl.json │ │ ├── api_5_replica_set_to_cluster.json │ │ ├── security_ldap.json │ │ ├── security_x509.json │ │ ├── security_kerberos.json │ │ ├── mongo30_cluster.json │ │ ├── mongo32_cluster.json │ │ ├── security_ldap_cluster.json │ │ ├── mongo32_csrs_cluster.json │ │ ├── security_x509_cluster.json │ │ ├── security_kerberos_cluster.json │ │ └── api_6_enable_auth.json │ ├── test_single.py │ ├── test_automated_restore.py │ ├── api_base.py │ ├── test_enterprise_advanced.py │ ├── test_automation_api.py │ ├── test_restore_from_backup_replica_set.py │ ├── automation_api_base.py │ ├── test_restore_from_backup_sharded_cluster.py │ └── backup_api_base.py ├── example_configs │ └── custom_versions.json └── README.md ├── .gitignore └── README.md /automation/api_usage_example/requirements.txt: -------------------------------------------------------------------------------- 1 | requests 2 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_x509_backup_agent.json: -------------------------------------------------------------------------------- 1 | { 2 | "logPath": "/var/log/mongodb-mms-automation/backup-agent.log", 3 | "logRotate": { 4 | "sizeThresholdMB": 1000, 5 | "timeThresholdHrs": 24 6 | }, 7 | "sslPEMKeyFile": "/opt/backup-agent.pem" 8 | } 9 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_x509_monitoring_agent.json: -------------------------------------------------------------------------------- 1 | { 2 | "logPath": "/var/log/mongodb-mms-automation/monitoring-agent.log", 3 | "logRotate": { 4 | "sizeThresholdMB": 1000, 5 | "timeThresholdHrs": 24 6 | }, 7 | "sslPEMKeyFile": "/opt/monitoring-agent.pem" 8 | } 9 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_ldap_backup_agent.json: -------------------------------------------------------------------------------- 1 | { 2 | "logPath": "/var/log/mongodb-mms-automation/backup-agent.log", 3 | "logRotate": { 4 | "sizeThresholdMB": 1000, 5 | "timeThresholdHrs": 24 6 | }, 7 | "username": "backup-agent", 8 | "password": "bpassword" 9 | } 10 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_ldap_monitoring_agent.json: -------------------------------------------------------------------------------- 1 | { 2 | "logPath": "/var/log/mongodb-mms-automation/monitoring-agent.log", 3 | "logRotate": { 4 | "sizeThresholdMB": 1000, 5 | "timeThresholdHrs": 24 6 | }, 7 | "username": "monitoring-agent", 8 | "password": "mpassword" 9 | } 10 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_kerberos_backup_agent.json: -------------------------------------------------------------------------------- 1 | { 2 | "logPath": "/var/log/mongodb-mms-automation/backup-agent.log", 3 | "logRotate": { 4 | "sizeThresholdMB": 1000, 5 | "timeThresholdHrs": 24 6 | }, 7 | "kerberosPrincipal": "backup-agent/MACHINE_HOSTNAME@MMS.COM", 8 | "kerberosKeytab": "/opt/backup-agent.keytab" 9 | } 10 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_kerberos_monitoring_agent.json: -------------------------------------------------------------------------------- 1 | { 2 | "logPath": "/var/log/mongodb-mms-automation/monitoring-agent.log", 3 | "logRotate": { 4 | "sizeThresholdMB": 1000, 5 | "timeThresholdHrs": 24 6 | }, 7 | "kerberosPrincipal": "monitoring-agent/MACHINE_HOSTNAME@MMS.COM", 8 | "kerberosKeytab": "/opt/monitoring-agent.keytab" 9 | } 10 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_0_clean.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [], 7 | "monitoringVersions": [], 8 | "backupVersions": [], 9 | "processes": [], 10 | "replicaSets": [], 11 | "sharding": [] 12 | } 13 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_1_define_versions.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.0.1"}, 8 | {"name": "3.0.2"} 9 | ], 10 | "monitoringVersions": [], 11 | "backupVersions": [], 12 | "processes": [], 13 | "replicaSets": [], 14 | "sharding": [] 15 | } 16 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_2_install_other_agents.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.0.1"}, 8 | {"name": "3.0.2"} 9 | ], 10 | "monitoringVersions": [ 11 | { 12 | "hostname": "MACHINE_HOSTNAME" 13 | } 14 | ], 15 | "backupVersions": [ 16 | { 17 | "hostname": "MACHINE_HOSTNAME" 18 | } 19 | ], 20 | "processes": [], 21 | "replicaSets": [], 22 | "sharding": [] 23 | } 24 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # C extensions 6 | *.so 7 | 8 | # Distribution / packaging 9 | .Python 10 | env/ 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | lib/ 17 | lib64/ 18 | parts/ 19 | sdist/ 20 | var/ 21 | *.egg-info/ 22 | .installed.cfg 23 | *.egg 24 | 25 | # PyInstaller 26 | # Usually these files are written by a python script from a template 27 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 28 | *.manifest 29 | *.spec 30 | 31 | # Installer logs 32 | pip-log.txt 33 | pip-delete-this-directory.txt 34 | 35 | # Unit test / coverage reports 36 | htmlcov/ 37 | .tox/ 38 | .coverage 39 | .cache 40 | nosetests.xml 41 | coverage.xml 42 | 43 | # Translations 44 | *.mo 45 | *.pot 46 | 47 | # Django stuff: 48 | *.log 49 | 50 | # Sphinx documentation 51 | docs/_build/ 52 | 53 | # PyBuilder 54 | target/ 55 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | DISCLAIMER 2 | ========== 3 | Please note: all tools/ scripts in this repo are released for use "AS IS" without any warranties of any kind, including, but not limited to their installation, use, or performance. We disclaim any and all warranties, either express or implied, including but not limited to any warranty of noninfringement, merchantability, and/ or fitness for a particular purpose. We do not warrant that the technology will meet your requirements, that the operation thereof will be uninterrupted or error-free, or that any errors will be corrected. 4 | Any use of these scripts and tools is at your own risk. There is no guarantee that they have been through thorough testing in a comparable environment and we are not responsible for any damage or data loss incurred with their use. 5 | You are responsible for reviewing and testing any scripts you run thoroughly before use in any non-testing environment. 6 | 7 | #MMS API Examples# 8 | 9 | * [Automation API Examples](https://github.com/10gen-labs/mms-api-examples/blob/master/automation/README.md) 10 | 11 | 12 | -------------------------------------------------------------------------------- /automation/example_configs/custom_versions.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation" 4 | }, 5 | "mongoDbVersions": [ 6 | { 7 | "builds": [ 8 | { 9 | "bits": 64, 10 | "gitVersion": "e367a48c7ff9eca187712fcfba1322c37af69780", 11 | "platform": "osx", 12 | "url": "https://s3.amazonaws.com/mciuploads/mongodb-mongo-v2.6/osx-108/e367a48c7ff9eca187712fcfba1322c37af69780/binaries/binaries-mongodb_mongo_v2.6_osx_108_e367a48c7ff9eca187712fcfba1322c37af69780_15_01_16_02_19_24.tgz" 13 | }, 14 | { 15 | "bits": 64, 16 | "gitVersion": "e367a48c7ff9eca187712fcfba1322c37af69780", 17 | "platform": "linux", 18 | "url": "https://s3.amazonaws.com/mciuploads/mongodb-mongo-v2.6/linux-64/e367a48c7ff9eca187712fcfba1322c37af69780/binaries/binaries-mongodb_mongo_v2.6_linux_64_e367a48c7ff9eca187712fcfba1322c37af69780_15_01_16_02_19_24.tgz" 19 | } 20 | ], 21 | "name": "2.6.8-pre" 22 | } 23 | ], 24 | "monitoringVersions": [], 25 | "backupVersions": [], 26 | "processes": [ 27 | { 28 | "args2_6": { 29 | "net": { 30 | "port": 28000 31 | }, 32 | "storage": { 33 | "dbPath": "/data/blue_0" 34 | }, 35 | "systemLog": { 36 | "destination": "file", 37 | "path": "/tmp/data/blue_0/mongodb.log" 38 | } 39 | }, 40 | "hostname": "cailinmac", 41 | "logRotate": { 42 | "sizeThresholdMB": 1000, 43 | "timeThresholdHrs": 24 44 | }, 45 | "name": "blue_0", 46 | "processType": "mongod", 47 | "version": "2.6.8-pre", 48 | "authSchemaVersion": 3 49 | } 50 | ], 51 | "replicaSets": [], 52 | "sharding": [] 53 | } 54 | -------------------------------------------------------------------------------- /automation/api_usage_example/test_single.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | import os 4 | 5 | from automation_api_base import AutomationApiBase 6 | 7 | logging.basicConfig(level=logging.INFO, format='[%(asctime)s] [%(levelname)s] %(message)s') 8 | 9 | class TestAutomationApi(AutomationApiBase): 10 | 11 | def __init__(self, base_url, machine_hostname, group_id, api_user, api_key, config_name): 12 | AutomationApiBase.__init__(self, base_url, machine_hostname, group_id, api_user, api_key) 13 | self.config_name = config_name 14 | 15 | def clean(self): 16 | self.post_automation_config("configs/api_0_clean.json") 17 | 18 | def run(self): 19 | 20 | self.post_automation_config("configs/%s.json" % self.config_name) 21 | 22 | 23 | if __name__ == '__main__': 24 | 25 | # python test_single.py http://localhost:8080 cailinmac 541ed2009436399a1f54e01b cailin.nelson@10gen.com da5f6890-1ee4-49db-83ca-ece9b2dbe5c0 mongo32_csrs_cluster 26 | parser = argparse.ArgumentParser(description='Automation API Demo') 27 | parser.add_argument('base_url', help="Base URL") 28 | parser.add_argument('machine_hostname', help="Agent Hostname") 29 | parser.add_argument('group_id', help="Group ID") 30 | parser.add_argument('api_user', help="API User") 31 | parser.add_argument('api_key', help="API Key") 32 | parser.add_argument('config_name', help="ConfigName") 33 | parser.add_argument('--clean', action='store_true', required=False) 34 | args = parser.parse_args() 35 | 36 | test = TestAutomationApi( 37 | args.base_url, 38 | args.machine_hostname, 39 | args.group_id, 40 | args.api_user, 41 | args.api_key, 42 | args.config_name 43 | ) 44 | 45 | if args.clean: 46 | test.clean() 47 | else: 48 | test.run() 49 | 50 | -------------------------------------------------------------------------------- /automation/api_usage_example/test_automated_restore.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | import time 4 | 5 | from automation_api_base import AutomationApiBase 6 | from backup_api_base import BackupApiBase 7 | 8 | logging.basicConfig( 9 | level=logging.DEBUG, format='[%(asctime)s] [%(levelname)s] %(message)s') 10 | 11 | class TestRestoreFromBackupReplicaSet: 12 | 13 | def __init__(self, base_url, api_user, api_key, group_id, cluster_name, target_group_id, target_cluster_name): 14 | self.backup_api = BackupApiBase(base_url, group_id, api_user, api_key) 15 | self.backup_api_target = BackupApiBase(base_url, target_group_id, api_user, api_key) 16 | self.cluster_name = cluster_name 17 | self.target_group_id = target_group_id 18 | self.target_cluster_name = target_cluster_name 19 | 20 | def run(self): 21 | cluster_id = self.backup_api.get_cluster_id_from_cluster_name(self.cluster_name) 22 | target_cluster_id = self.backup_api_target.get_cluster_id_from_cluster_name(self.target_cluster_name) 23 | snapshots = self.backup_api._get_snapshots(cluster_id) 24 | snapshot_id = snapshots[0]['id'] 25 | self.backup_api.request_automated_restore(cluster_id, snapshot_id, self.target_group_id, target_cluster_id) 26 | 27 | if __name__ == '__main__': 28 | 29 | # python test_automated_restore.py http://opsmanager.example.com john@xmple.com f0f88855-d211-4d4f-9461-e53b0822b505 59f7b8f954559e0cc0146094 red 59f77d0254559e0cc0138ba0 blue 30 | parser = argparse.ArgumentParser(description='Automated Restore API Demo') 31 | parser.add_argument('base_url', help="Base URL") 32 | 33 | parser.add_argument('api_user', help="API User") 34 | parser.add_argument('api_key', help="API Key") 35 | 36 | parser.add_argument('group_id', help="Group ID") 37 | parser.add_argument('cluster_name', help="Cluster Name") 38 | parser.add_argument('target_group_id', help="Target Group ID") 39 | parser.add_argument('target_cluster_name', help="Target Cluster Name") 40 | 41 | args = parser.parse_args() 42 | 43 | test = TestRestoreFromBackupReplicaSet( 44 | args.base_url, 45 | args.api_user, 46 | args.api_key, 47 | args.group_id, 48 | args.cluster_name, 49 | args.target_group_id, 50 | args.target_cluster_name) 51 | 52 | test.run() 53 | 54 | -------------------------------------------------------------------------------- /automation/api_usage_example/api_base.py: -------------------------------------------------------------------------------- 1 | import json 2 | import logging 3 | import pprint 4 | import requests 5 | 6 | from requests.auth import HTTPDigestAuth 7 | 8 | class ApiBase: 9 | def __init__(self, base_url, group_id, api_user, api_key): 10 | self.base_url = base_url + "/api/public/v1.0" 11 | self.group_id = group_id 12 | self.api_user = api_user 13 | self.api_key = api_key 14 | 15 | def get(self, url): 16 | logging.info("Executing GET: %s" % url) 17 | r = requests.get(url, auth=HTTPDigestAuth(self.api_user, self.api_key)) 18 | self.check_response(r) 19 | logging.debug("%s" % pprint.pformat(r.json())) 20 | return r.json() 21 | 22 | def put(self, url, json_body): 23 | logging.info("Executing PUT: %s" % url) 24 | headers = {'content-type': 'application/json'} 25 | r = requests.put( 26 | url, 27 | auth=HTTPDigestAuth(self.api_user, self.api_key), 28 | data=json.dumps(json_body), 29 | headers=headers 30 | ) 31 | self.check_response(r) 32 | logging.debug("%s" % pprint.pformat(r.json())) 33 | 34 | return r.json() 35 | 36 | def patch(self, url, json_body): 37 | logging.info("Executing PATCH: %s" % url) 38 | headers = {'content-type': 'application/json'} 39 | r = requests.patch( 40 | url, 41 | auth=HTTPDigestAuth(self.api_user, self.api_key), 42 | data=json.dumps(json_body), 43 | headers=headers 44 | ) 45 | self.check_response(r) 46 | logging.debug("%s" % pprint.pformat(r.json())) 47 | 48 | return r.json() 49 | 50 | def post(self, url, json_body): 51 | logging.info("Executing POST: %s" % url) 52 | headers = {'content-type': 'application/json'} 53 | r = requests.post( 54 | url, 55 | auth=HTTPDigestAuth(self.api_user, self.api_key), 56 | data=json.dumps(json_body), 57 | headers=headers 58 | ) 59 | self.check_response(r) 60 | logging.debug("%s" % pprint.pformat(r.json())) 61 | 62 | return r.json() 63 | 64 | def check_response(self, r): 65 | if r.status_code not in [requests.codes.ok, 202]: 66 | logging.error("Response Error Code: %s Detail: %s" % (r.status_code, r.json()['detail'])) 67 | raise ValueError(r.json()['detail']) -------------------------------------------------------------------------------- /automation/api_usage_example/test_enterprise_advanced.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | import os 4 | 5 | from automation_api_base import AutomationApiBase 6 | 7 | logging.basicConfig(level=logging.INFO, format='[%(asctime)s] [%(levelname)s] %(message)s') 8 | 9 | class TestAutomationApi(AutomationApiBase): 10 | 11 | def __init__(self, base_url, machine_hostname, group_id, api_user, api_key, config_name, cluster): 12 | AutomationApiBase.__init__(self, base_url, machine_hostname, group_id, api_user, api_key) 13 | self.config_name = config_name 14 | self.cluster = cluster 15 | 16 | def clean(self): 17 | self.post_automation_config("configs/api_0_clean.json") 18 | 19 | def run(self): 20 | if self.cluster: 21 | self.post_automation_config("configs/%s_cluster.json" % self.config_name) 22 | else: 23 | self.post_automation_config("configs/%s.json" % self.config_name) 24 | 25 | if os.path.exists("configs/%s_monitoring_agent.json" % self.config_name): 26 | self.post_monitoring_agent_config("configs/%s_monitoring_agent.json" % self.config_name) 27 | 28 | if os.path.exists("configs/%s_backup_agent.json" % self.config_name): 29 | self.post_backup_agent_config("configs/%s_backup_agent.json" % self.config_name) 30 | 31 | #self.wait_for_goal_state() 32 | 33 | 34 | if __name__ == '__main__': 35 | 36 | # automation-agent/ec2-52-5-170-235.compute-1.amazonaws.com@MMS.COM 37 | # /opt/automation-agent.keytab 38 | 39 | # python test_enterprise_advanced.py http://localhost:8080 ec2-52-5-170-235.compute-1.amazonaws.com 541ed2009436399a1f54e01b cailin.nelson@10gen.com 0db80617-1cab-44d6-b858-a84496194f43 security_kerberos 40 | parser = argparse.ArgumentParser(description='Automation API Demo') 41 | parser.add_argument('base_url', help="Base URL") 42 | parser.add_argument('machine_hostname', help="Agent Hostname") 43 | parser.add_argument('group_id', help="Group ID") 44 | parser.add_argument('api_user', help="API User") 45 | parser.add_argument('api_key', help="API Key") 46 | parser.add_argument('config_name', help="ConfigName") 47 | parser.add_argument('--cluster', action='store_true', required=False) 48 | parser.add_argument('--clean', action='store_true', required=False) 49 | args = parser.parse_args() 50 | 51 | test = TestAutomationApi( 52 | args.base_url, 53 | args.machine_hostname, 54 | args.group_id, 55 | args.api_user, 56 | args.api_key, 57 | args.config_name, 58 | args.cluster 59 | ) 60 | 61 | if args.clean: 62 | test.clean() 63 | else: 64 | test.run() 65 | 66 | -------------------------------------------------------------------------------- /automation/api_usage_example/test_automation_api.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | 4 | from automation_api_base import AutomationApiBase 5 | 6 | logging.basicConfig( 7 | level=logging.INFO, format='[%(asctime)s] [%(levelname)s] %(message)s') 8 | 9 | 10 | class Step: 11 | 12 | def __init__(self, config_path, description): 13 | self.config_path = config_path 14 | self.description = description 15 | 16 | steps = [ 17 | Step("configs/api_0_clean.json", 18 | "Start with empty automation config"), 19 | 20 | Step("configs/api_1_define_versions.json", 21 | "Setup desired MongoDB versions"), 22 | 23 | Step("configs/api_2_install_other_agents.json", 24 | "Install Backup and Monitoring agents"), 25 | 26 | Step("configs/api_3_create_replica_set.json", 27 | "Create a Replica Set"), 28 | 29 | Step("configs/api_4_upgrade_replica_set.json", 30 | "Upgrade the Replica Set"), 31 | 32 | Step("configs/api_5_replica_set_to_cluster.json", 33 | "Convert Replica Set to Cluster"), 34 | 35 | Step("configs/api_6_enable_auth.json", 36 | "Enable Authentication"), 37 | ] 38 | 39 | class TestAutomationApi(AutomationApiBase): 40 | 41 | def __init__(self, base_url, machine_hostname, group_id, api_user, api_key, step_over): 42 | AutomationApiBase.__init__(self, base_url, machine_hostname, group_id, api_user, api_key) 43 | self.step_over = step_over 44 | 45 | def run(self): 46 | for (i, step) in enumerate(steps): 47 | self.run_step(step, i) 48 | 49 | def run_step(self, step, stepNumber, wait=True): 50 | step_info = self.build_step_info(step, stepNumber) 51 | logging.info("Running %s..." % step_info) 52 | self.post_automation_config(step.config_path) 53 | if wait: 54 | self.wait_for_goal_state() 55 | if self.step_over: 56 | raw_input("\nFinished %s. Hit enter to continue...\n" % step_info) 57 | 58 | def build_step_info(self, step, stepNumber): 59 | return "step {} ({})".format(stepNumber, step.description) 60 | 61 | 62 | if __name__ == '__main__': 63 | 64 | # python test_automation_api.py http://mms.example.com:8080 mongo-01.example.com 54b5e4df9436322466a89a3e apple@johnandcailin.com da5f6890-1ee4-49db-83ca-ece9b2dbe5c0 65 | parser = argparse.ArgumentParser(description='Automation API Demo') 66 | parser.add_argument('base_url', help="Base URL") 67 | parser.add_argument('machine_hostname', help="Agent Hostname") 68 | parser.add_argument('group_id', help="Group ID") 69 | parser.add_argument('api_user', help="API User") 70 | parser.add_argument('api_key', help="API Key") 71 | parser.add_argument('--clean', action='store_true', required=False) 72 | parser.add_argument('--step', action='store_true', required=False) 73 | args = parser.parse_args() 74 | 75 | test = TestAutomationApi(args.base_url, args.machine_hostname, args.group_id, args.api_user, args.api_key, args.step) 76 | if args.clean: 77 | test.clean() 78 | else: 79 | test.run() 80 | 81 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/mongo34_rs.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.4.9"} 8 | ], 9 | "backupVersions": [ 10 | { 11 | "hostname": "MACHINE_HOSTNAME" 12 | } 13 | ], 14 | 15 | "monitoringVersions": [ 16 | { 17 | "hostname": "MACHINE_HOSTNAME" 18 | } 19 | ], 20 | "processes": [ 21 | { 22 | "args2_6": { 23 | "net": { 24 | "port": 28000 25 | }, 26 | "replication": { 27 | "replSetName": "blue" 28 | }, 29 | "storage": { 30 | "dbPath": "/data/blue_0" 31 | }, 32 | "systemLog": { 33 | "destination": "file", 34 | "path": "/data/blue_0/mongodb.log" 35 | } 36 | }, 37 | "hostname": "MACHINE_HOSTNAME", 38 | "logRotate": { 39 | "sizeThresholdMB": 1000, 40 | "timeThresholdHrs": 24 41 | }, 42 | "name": "blue_0", 43 | "processType": "mongod", 44 | "version": "3.4.9", 45 | "authSchemaVersion": 3 46 | }, 47 | { 48 | "args2_6": { 49 | "net": { 50 | "port": 28001 51 | }, 52 | "replication": { 53 | "replSetName": "blue" 54 | }, 55 | "storage": { 56 | "dbPath": "/data/blue_1" 57 | }, 58 | "systemLog": { 59 | "destination": "file", 60 | "path": "/data/blue_1/mongodb.log" 61 | } 62 | }, 63 | "hostname": "MACHINE_HOSTNAME", 64 | "logRotate": { 65 | "sizeThresholdMB": 1000, 66 | "timeThresholdHrs": 24 67 | }, 68 | "name": "blue_1", 69 | "processType": "mongod", 70 | "version": "3.4.9", 71 | "authSchemaVersion": 3 72 | }, 73 | { 74 | "args2_6": { 75 | "net": { 76 | "port": 28002 77 | }, 78 | "replication": { 79 | "replSetName": "blue" 80 | }, 81 | "storage": { 82 | "dbPath": "/data/blue_2" 83 | }, 84 | "systemLog": { 85 | "destination": "file", 86 | "path": "/data/blue_2/mongodb.log" 87 | } 88 | }, 89 | "hostname": "MACHINE_HOSTNAME", 90 | "logRotate": { 91 | "sizeThresholdMB": 1000, 92 | "timeThresholdHrs": 24 93 | }, 94 | "name": "blue_2", 95 | "processType": "mongod", 96 | "version": "3.4.9", 97 | "authSchemaVersion": 3 98 | } 99 | ], 100 | "replicaSets": [ 101 | { 102 | "_id": "blue", 103 | "members": [ 104 | { 105 | "_id": 0, 106 | "host": "blue_0" 107 | }, 108 | { 109 | "_id": 1, 110 | "host": "blue_1" 111 | }, 112 | { 113 | "_id": 2, 114 | "arbiterOnly": true, 115 | "host": "blue_2", 116 | "priority": 0 117 | } 118 | ] 119 | } 120 | ], 121 | "roles": [], 122 | "sharding": [] 123 | } 124 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_3_create_replica_set.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.4.9"} 8 | ], 9 | "backupVersions": [ 10 | { 11 | "hostname": "MACHINE_HOSTNAME" 12 | } 13 | ], 14 | 15 | "monitoringVersions": [ 16 | { 17 | "hostname": "MACHINE_HOSTNAME" 18 | } 19 | ], 20 | "processes": [ 21 | { 22 | "args2_6": { 23 | "net": { 24 | "port": 28000 25 | }, 26 | "replication": { 27 | "replSetName": "blue" 28 | }, 29 | "storage": { 30 | "dbPath": "/data/blue_0" 31 | }, 32 | "systemLog": { 33 | "destination": "file", 34 | "path": "/data/blue_0/mongodb.log" 35 | } 36 | }, 37 | "hostname": "MACHINE_HOSTNAME", 38 | "logRotate": { 39 | "sizeThresholdMB": 1000, 40 | "timeThresholdHrs": 24 41 | }, 42 | "name": "blue_0", 43 | "processType": "mongod", 44 | "version": "3.4.9", 45 | "authSchemaVersion": 3 46 | }, 47 | { 48 | "args2_6": { 49 | "net": { 50 | "port": 28001 51 | }, 52 | "replication": { 53 | "replSetName": "blue" 54 | }, 55 | "storage": { 56 | "dbPath": "/data/blue_1" 57 | }, 58 | "systemLog": { 59 | "destination": "file", 60 | "path": "/data/blue_1/mongodb.log" 61 | } 62 | }, 63 | "hostname": "MACHINE_HOSTNAME", 64 | "logRotate": { 65 | "sizeThresholdMB": 1000, 66 | "timeThresholdHrs": 24 67 | }, 68 | "name": "blue_1", 69 | "processType": "mongod", 70 | "version": "3.4.9", 71 | "authSchemaVersion": 3 72 | }, 73 | { 74 | "args2_6": { 75 | "net": { 76 | "port": 28002 77 | }, 78 | "replication": { 79 | "replSetName": "blue" 80 | }, 81 | "storage": { 82 | "dbPath": "/data/blue_2" 83 | }, 84 | "systemLog": { 85 | "destination": "file", 86 | "path": "/data/blue_2/mongodb.log" 87 | } 88 | }, 89 | "hostname": "MACHINE_HOSTNAME", 90 | "logRotate": { 91 | "sizeThresholdMB": 1000, 92 | "timeThresholdHrs": 24 93 | }, 94 | "name": "blue_2", 95 | "processType": "mongod", 96 | "version": "3.4.9", 97 | "authSchemaVersion": 3 98 | } 99 | ], 100 | "replicaSets": [ 101 | { 102 | "_id": "blue", 103 | "members": [ 104 | { 105 | "_id": 0, 106 | "host": "blue_0" 107 | }, 108 | { 109 | "_id": 1, 110 | "host": "blue_1" 111 | }, 112 | { 113 | "_id": 2, 114 | "arbiterOnly": true, 115 | "host": "blue_2", 116 | "priority": 0 117 | } 118 | ] 119 | } 120 | ], 121 | "roles": [], 122 | "sharding": [] 123 | } 124 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_4_upgrade_replica_set.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.0.1"}, 8 | {"name": "3.0.2"} 9 | ], 10 | "backupVersions": [ 11 | { 12 | "hostname": "MACHINE_HOSTNAME" 13 | } 14 | ], 15 | 16 | "monitoringVersions": [ 17 | { 18 | "hostname": "MACHINE_HOSTNAME" 19 | } 20 | ], 21 | "processes": [ 22 | { 23 | "args2_6": { 24 | "net": { 25 | "port": 28000 26 | }, 27 | "replication": { 28 | "replSetName": "blue" 29 | }, 30 | "storage": { 31 | "dbPath": "/data/blue_0" 32 | }, 33 | "systemLog": { 34 | "destination": "file", 35 | "path": "/data/blue_0/mongodb.log" 36 | } 37 | }, 38 | "hostname": "MACHINE_HOSTNAME", 39 | "logRotate": { 40 | "sizeThresholdMB": 1000, 41 | "timeThresholdHrs": 24 42 | }, 43 | "name": "blue_0", 44 | "processType": "mongod", 45 | "version": "3.0.2", 46 | "authSchemaVersion": 3 47 | }, 48 | { 49 | "args2_6": { 50 | "net": { 51 | "port": 28001 52 | }, 53 | "replication": { 54 | "replSetName": "blue" 55 | }, 56 | "storage": { 57 | "dbPath": "/data/blue_1" 58 | }, 59 | "systemLog": { 60 | "destination": "file", 61 | "path": "/data/blue_1/mongodb.log" 62 | } 63 | }, 64 | "hostname": "MACHINE_HOSTNAME", 65 | "logRotate": { 66 | "sizeThresholdMB": 1000, 67 | "timeThresholdHrs": 24 68 | }, 69 | "name": "blue_1", 70 | "processType": "mongod", 71 | "version": "3.0.2", 72 | "authSchemaVersion": 3 73 | }, 74 | { 75 | "args2_6": { 76 | "net": { 77 | "port": 28002 78 | }, 79 | "replication": { 80 | "replSetName": "blue" 81 | }, 82 | "storage": { 83 | "dbPath": "/data/blue_2" 84 | }, 85 | "systemLog": { 86 | "destination": "file", 87 | "path": "/data/blue_2/mongodb.log" 88 | } 89 | }, 90 | "hostname": "MACHINE_HOSTNAME", 91 | "logRotate": { 92 | "sizeThresholdMB": 1000, 93 | "timeThresholdHrs": 24 94 | }, 95 | "name": "blue_2", 96 | "processType": "mongod", 97 | "version": "3.0.2", 98 | "authSchemaVersion": 3 99 | } 100 | ], 101 | "replicaSets": [ 102 | { 103 | "_id": "blue", 104 | "members": [ 105 | { 106 | "_id": 0, 107 | "host": "blue_0" 108 | }, 109 | { 110 | "_id": 1, 111 | "host": "blue_1" 112 | }, 113 | { 114 | "_id": 2, 115 | "arbiterOnly": true, 116 | "host": "blue_2", 117 | "priority": 0 118 | } 119 | ] 120 | } 121 | ], 122 | "roles": [], 123 | "sharding": [] 124 | } 125 | -------------------------------------------------------------------------------- /automation/api_usage_example/test_restore_from_backup_replica_set.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | import time 4 | 5 | from automation_api_base import AutomationApiBase 6 | from backup_api_base import BackupApiBase 7 | 8 | logging.basicConfig( 9 | level=logging.DEBUG, format='[%(asctime)s] [%(levelname)s] %(message)s') 10 | 11 | class TestRestoreFromBackupReplicaSet: 12 | 13 | def __init__(self, base_url, machine_hostname, group_id, api_user, api_key, step_over): 14 | self.automation_api = AutomationApiBase(base_url, machine_hostname, group_id, api_user, api_key) 15 | self.backup_api = BackupApiBase(base_url, group_id, api_user, api_key) 16 | self.step_over = step_over 17 | 18 | def run(self): 19 | config_file = 'configs/mongo34_rs.json' 20 | logging.info("Setting up a simple replica set") 21 | self.automation_api.post_automation_config(config_file) 22 | logging.info("Waiting for goal state") 23 | self.automation_api.wait_for_goal_state() 24 | logging.info("Enabling backup and waiting for restore link to be ready") 25 | restore_link = self.get_restore_link() 26 | 27 | logging.info("Retrieving the current automation configuration") 28 | automation_config = self.automation_api.load_config(config_file) 29 | 30 | logging.info("Adding the desired restore_link to each process") 31 | for process in automation_config['processes']: 32 | process['backupRestoreUrl'] = restore_link 33 | 34 | logging.info("Initializing restore") 35 | self.automation_api.post_automation_config(automation_config) 36 | logging.info("Waiting for goal state") 37 | self.automation_api.wait_for_goal_state() 38 | 39 | logging.info("Removing restore URLs from config") 40 | self.automation_api.post_automation_config(config_file) 41 | logging.info("Waiting for goal state") 42 | self.automation_api.wait_for_goal_state() 43 | 44 | def get_restore_link(self): 45 | replica_set_name = 'blue' 46 | self.backup_api.start_backup_replica_set(replica_set_name, "WIRED_TIGER") 47 | 48 | snapshot_id = None 49 | while True: 50 | snapshots = self.backup_api.get_snapshots_replica_set(replica_set_name) 51 | if snapshots: 52 | snapshot_id = snapshots[0]['id'] 53 | break 54 | 55 | logging.info("Snapshot not yet complete. Sleeping for 30 seconds") 56 | time.sleep(30) 57 | 58 | restore_job = self.backup_api.request_restore_http_replica_set('blue', snapshot_id, 6, 1) 59 | restore_link = None 60 | 61 | while True: 62 | restore_job_status = self.backup_api.get_restore_job_result_replica_set(replica_set_name, restore_job['id']) 63 | delivery_status = restore_job_status['delivery']['statusName'] 64 | if delivery_status == 'READY': 65 | restore_link = restore_job_status['delivery']['url'] 66 | break 67 | 68 | logging.info("Restore files not yet ready. Sleeping for 30 seconds") 69 | time.sleep(5) 70 | 71 | return restore_link 72 | 73 | def build_step_info(self, step, stepNumber): 74 | return "step {} ({})".format(stepNumber, step.description) 75 | 76 | if __name__ == '__main__': 77 | 78 | # python test_restore_from_backup_replica_set.py http://opsmanager.example.com somehostname 54b5e4df9436322466a89a3e john@example.com f0f88855-d211-4d4f-9461-e53b0822b505 79 | parser = argparse.ArgumentParser(description='Automation API Demo') 80 | parser.add_argument('base_url', help="Base URL") 81 | parser.add_argument('machine_hostname', help="Agent Hostname") 82 | parser.add_argument('group_id', help="Group ID") 83 | parser.add_argument('api_user', help="API User") 84 | parser.add_argument('api_key', help="API Key") 85 | parser.add_argument('--step', action='store_true', required=False) 86 | args = parser.parse_args() 87 | 88 | test = TestRestoreFromBackupReplicaSet(args.base_url, args.machine_hostname, args.group_id, args.api_user, args.api_key, args.step) 89 | test.run() 90 | 91 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_ssl.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "ssl": { 7 | "CAFilePath": "/opt/ca.pem", 8 | "clientCertificateMode": "OPTIONAL" 9 | }, 10 | "mongoDbVersions": [ 11 | { 12 | "name": "3.0.1-ent" 13 | } 14 | ], 15 | "backupVersions": [ 16 | { 17 | "hostname": "MACHINE_HOSTNAME" 18 | } 19 | ], 20 | 21 | "monitoringVersions": [ 22 | { 23 | "hostname": "MACHINE_HOSTNAME" 24 | } 25 | ], 26 | "processes": [ 27 | { 28 | "args2_6": { 29 | "net": { 30 | "port": 28000, 31 | "ssl": { 32 | "mode": "requireSSL", 33 | "PEMKeyFile": "/opt/mongodb-1.pem" 34 | } 35 | }, 36 | "replication": { 37 | "replSetName": "blue" 38 | }, 39 | "storage": { 40 | "dbPath": "/data/blue_0" 41 | }, 42 | "systemLog": { 43 | "destination": "file", 44 | "path": "/data/blue_0/mongodb.log" 45 | } 46 | }, 47 | "hostname": "MACHINE_HOSTNAME", 48 | "logRotate": { 49 | "sizeThresholdMB": 1000, 50 | "timeThresholdHrs": 24 51 | }, 52 | "name": "blue_0", 53 | "processType": "mongod", 54 | "version": "3.0.1-ent", 55 | "authSchemaVersion": 5 56 | }, 57 | { 58 | "args2_6": { 59 | "net": { 60 | "port": 28001, 61 | "ssl": { 62 | "mode": "requireSSL", 63 | "PEMKeyFile": "/opt/mongodb-2.pem" 64 | } 65 | }, 66 | "replication": { 67 | "replSetName": "blue" 68 | }, 69 | "storage": { 70 | "dbPath": "/data/blue_1" 71 | }, 72 | "systemLog": { 73 | "destination": "file", 74 | "path": "/data/blue_1/mongodb.log" 75 | } 76 | }, 77 | "hostname": "MACHINE_HOSTNAME", 78 | "logRotate": { 79 | "sizeThresholdMB": 1000, 80 | "timeThresholdHrs": 24 81 | }, 82 | "name": "blue_1", 83 | "processType": "mongod", 84 | "version": "3.0.1-ent", 85 | "authSchemaVersion": 5 86 | }, 87 | { 88 | "args2_6": { 89 | "net": { 90 | "port": 28002, 91 | "ssl": { 92 | "mode": "requireSSL", 93 | "PEMKeyFile": "/opt/mongodb-3.pem" 94 | } 95 | }, 96 | "replication": { 97 | "replSetName": "blue" 98 | }, 99 | "storage": { 100 | "dbPath": "/data/blue_2" 101 | }, 102 | "systemLog": { 103 | "destination": "file", 104 | "path": "/data/blue_2/mongodb.log" 105 | } 106 | }, 107 | "hostname": "MACHINE_HOSTNAME", 108 | "logRotate": { 109 | "sizeThresholdMB": 1000, 110 | "timeThresholdHrs": 24 111 | }, 112 | "name": "blue_2", 113 | "processType": "mongod", 114 | "version": "3.0.1-ent", 115 | "authSchemaVersion": 5 116 | } 117 | ], 118 | "replicaSets": [ 119 | { 120 | "_id": "blue", 121 | "members": [ 122 | { 123 | "_id": 0, 124 | "host": "blue_0" 125 | }, 126 | { 127 | "_id": 1, 128 | "host": "blue_1" 129 | }, 130 | { 131 | "_id": 2, 132 | "arbiterOnly": true, 133 | "host": "blue_2", 134 | "priority": 0 135 | } 136 | ] 137 | } 138 | ], 139 | "roles": [], 140 | "sharding": [] 141 | } 142 | -------------------------------------------------------------------------------- /automation/api_usage_example/automation_api_base.py: -------------------------------------------------------------------------------- 1 | import json 2 | import logging 3 | import time 4 | 5 | from api_base import ApiBase 6 | 7 | class AutomationApiBase(ApiBase): 8 | 9 | def __init__(self, base_url, machine_hostname, group_id, api_user, api_key): 10 | ApiBase.__init__(self, base_url, group_id, api_user, api_key) 11 | self.machine_hostname = machine_hostname 12 | 13 | def clean(self): 14 | self.post_automation_config("configs/api_0_clean.json") 15 | 16 | def wait_for_goal_state(self): 17 | 18 | count = 0 19 | while True: 20 | continue_to_wait = False 21 | status = self.get_automation_status() 22 | goal_version = status['goalVersion'] 23 | 24 | for process in status['processes']: 25 | logging.info("Round: %s GoalVersion: %s Process: %s (%s) LastVersionAchieved: %s Plan: %s " 26 | % (count, goal_version, process['name'], process['hostname'], process['lastGoalVersionAchieved'], process['plan'])) 27 | 28 | if process['lastGoalVersionAchieved'] < goal_version: 29 | continue_to_wait = True 30 | 31 | if continue_to_wait: 32 | time.sleep(5) 33 | else: 34 | logging.info("All processes in Goal State. GoalVersionAchieved: %s" % goal_version) 35 | break 36 | 37 | count += 1 38 | 39 | def get_automation_config(self): 40 | url = "%s/groups/%s/automationConfig" % (self.base_url, self.group_id) 41 | return self.get(url) 42 | 43 | def get_automation_status(self): 44 | url = "%s/groups/%s/automationStatus" % (self.base_url, self.group_id) 45 | return self.get(url) 46 | 47 | def post_monitoring_agent_config(self, file_name): 48 | url = "%s/groups/%s/automationConfig/monitoringAgentConfig" % (self.base_url, self.group_id) 49 | json_body = self.load_config(file_name) 50 | return self.put(url, json_body) 51 | 52 | def post_backup_agent_config(self, file_name): 53 | url = "%s/groups/%s/automationConfig/backupAgentConfig" % (self.base_url, self.group_id) 54 | json_body = self.load_config(file_name) 55 | return self.put(url, json_body) 56 | 57 | def post_automation_config(self, file_name_or_json): 58 | url = "%s/groups/%s/automationConfig" % (self.base_url, self.group_id) 59 | if isinstance(file_name_or_json, basestring): 60 | json_body = self.load_config(file_name_or_json) 61 | else: 62 | json_body = file_name_or_json 63 | 64 | return self.put(url, json_body) 65 | 66 | def load_config(self, file_name): 67 | data = self.load_json(file_name) 68 | 69 | # Replace the Machine_hostname placeholders in the example configs 70 | # with real value. 71 | self.replace_machine_hostnames(data) 72 | self.replace_process_hostnames(data) 73 | self.replace_kerberos_principals(data) 74 | return data 75 | 76 | def load_json(self, file_name): 77 | with open(file_name) as data_file: 78 | data = json.load(data_file) 79 | return data 80 | 81 | def replace_machine_hostnames(self, data): 82 | if 'monitoringVersions' in data: 83 | for index, mv in enumerate(data['monitoringVersions']): 84 | data['monitoringVersions'][index]['hostname'] = self.machine_hostname 85 | 86 | if 'backupVersions' in data: 87 | for index, bv in enumerate(data['backupVersions']): 88 | data['backupVersions'][index]['hostname'] = self.machine_hostname 89 | 90 | def replace_process_hostnames(self, data): 91 | if 'processes' in data: 92 | for index, mv in enumerate(data['processes']): 93 | data['processes'][index]['hostname'] = self.machine_hostname 94 | if data['processes'][index].get('alias'): 95 | data['processes'][index]['alias'] = self.machine_hostname 96 | 97 | def replace_kerberos_principals(self, data): 98 | if 'kerberosPrincipal' in data: 99 | data['kerberosPrincipal'] = data['kerberosPrincipal'].replace('Machine_hostname', self.machine_hostname) 100 | 101 | if 'auth' in data: 102 | if 'autoUser' in data['auth']: 103 | data['auth']['autoUser'] = data['auth']['autoUser'].replace('Machine_hostname', self.machine_hostname) 104 | 105 | for index, user in enumerate(data['auth']['usersWanted']): 106 | data['auth']['usersWanted'][index]['user'] = \ 107 | data['auth']['usersWanted'][index]['user'].replace('Machine_hostname', self.machine_hostname) 108 | -------------------------------------------------------------------------------- /automation/api_usage_example/test_restore_from_backup_sharded_cluster.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | import time 4 | 5 | from automation_api_base import AutomationApiBase 6 | from backup_api_base import BackupApiBase 7 | 8 | logging.basicConfig( 9 | level=logging.DEBUG, format='[%(asctime)s] [%(levelname)s] %(message)s') 10 | 11 | class TestRestoreFromBackupCluster: 12 | 13 | def __init__(self, base_url, machine_hostname, group_id, api_user, api_key): 14 | self.automation_api = AutomationApiBase(base_url, machine_hostname, group_id, api_user, api_key) 15 | self.backup_api = BackupApiBase(base_url, group_id, api_user, api_key) 16 | 17 | def run(self): 18 | config_file = 'configs/mongo32_csrs_cluster.json' 19 | logging.info("Setting up a MongoDB 3.2.5 CSRS cluster") 20 | self.automation_api.post_automation_config(config_file) 21 | logging.info("Waiting for goal state") 22 | self.automation_api.wait_for_goal_state() 23 | 24 | logging.info("Enabling backup and waiting for restore link to be ready") 25 | restore_links = self.get_restore_links() 26 | 27 | automation_config = self.automation_api.load_config(config_file) 28 | # For each non-mongos process, add the appropriate restoreUrl 29 | for process in automation_config['processes']: 30 | if process['processType'] == 'mongod': 31 | process['backupRestoreUrl'] = restore_links[process['args2_6']['replication']['replSetName']] 32 | 33 | logging.info("Initializing restore") 34 | self.automation_api.post_automation_config(automation_config) 35 | logging.info("Waiting for goal state") 36 | self.automation_api.wait_for_goal_state() 37 | 38 | logging.info("Removing restore URLs from config") 39 | self.automation_api.post_automation_config(config_file) 40 | logging.info("Waiting for goal state") 41 | self.automation_api.wait_for_goal_state() 42 | 43 | def get_restore_links(self): 44 | logging.info("Sleeping for 60 seconds to wait for monitoring agent data") 45 | time.sleep(60) 46 | 47 | cluster_name = 'blueCluster' 48 | self.backup_api.start_backup_cluster(cluster_name, "WIRED_TIGER") 49 | 50 | # Wait for a snapshot to be taken for all components in the cluster 51 | snapshot_id = None 52 | while True: 53 | snapshots = self.backup_api.get_snapshots_cluster(cluster_name) 54 | # There will be two parts, one for the shard, and one for the config server replica set 55 | if snapshots and len(snapshots[0].get('parts',[])) == 2: 56 | snapshot_id = snapshots[0]['id'] 57 | break 58 | 59 | logging.info("Snapshot not yet complete. Sleeping for 30 seconds") 60 | time.sleep(30) 61 | 62 | # Request a restore for this snapshot. Be sure to specify an adequate number of 63 | # downloads for each snapshot, and a generous expriation time. The Automation Agents will 64 | # need to download the snapshot _once per mongod process_. In this case, a value of 3 should 65 | # be sufficient (3 processes in each replica set), but we'll use 6, just in case there are any 66 | # network glitches or other transient failures 67 | 68 | max_downloads = 6 69 | expiration_hours = 24 70 | restore_job = self.backup_api.request_restore_http_cluster( 71 | cluster_name, 72 | snapshot_id, 73 | max_downloads, 74 | expiration_hours) 75 | batch_id = restore_job['batchId'] 76 | restore_links = {} 77 | 78 | while True: 79 | restore_job_statuses = self.backup_api.get_restore_job_result_cluster(cluster_name, batch_id) 80 | 81 | # Wait for a restore job to be created for both components of the cluster 82 | if len(restore_links) == 2: 83 | break 84 | 85 | logging.info("Restore files not yet ready. Sleeping for 15 seconds") 86 | time.sleep(15) 87 | 88 | # And then wait for the deliveryUrls to be complete for both components 89 | for restore_job_status in restore_job_statuses['results']: 90 | delivery_status = restore_job_status['delivery']['statusName'] 91 | if delivery_status == 'READY': 92 | restore_link = restore_job_status['delivery']['url'] 93 | cluster_id = restore_job_status['clusterId'] 94 | # Keep a dictionary of the restore links for each component in the cluster 95 | # Identify the component by it's replica set id 96 | restore_links[self.backup_api.get_replica_set_from_cluster_id(cluster_id)] = restore_link 97 | 98 | logging.info("Full set of restore links ready: %s", restore_links) 99 | return restore_links 100 | 101 | 102 | if __name__ == '__main__': 103 | 104 | # USAGE: 105 | # python test_restore_from_backup_cluster.py CLOUD_OR_OPS_MANAGER_URL HOSTNAME GROUP_ID USER_NAME USER_API_KEY 106 | # python test_restore_from_backup_cluster.py http://mms.example.com:8080 mongo-01.example.com 54b5e4df9436322466a89a3e apple@johnandcailin.com 04e2aa59-d410-446f-aba9-70cf37010b7c 107 | parser = argparse.ArgumentParser(description='Automation API Demo') 108 | parser.add_argument('base_url', help="Base URL") 109 | parser.add_argument('machine_hostname', help="Agent Hostname") 110 | parser.add_argument('group_id', help="Group ID") 111 | parser.add_argument('api_user', help="API User") 112 | parser.add_argument('api_key', help="API Key") 113 | args = parser.parse_args() 114 | 115 | test = TestRestoreFromBackupCluster(args.base_url, args.machine_hostname, args.group_id, args.api_user, args.api_key) 116 | test.run() 117 | 118 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_5_replica_set_to_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.0.1"}, 8 | {"name": "3.0.2"} 9 | ], 10 | "backupVersions": [ 11 | { 12 | "hostname": "MACHINE_HOSTNAME" 13 | } 14 | ], 15 | 16 | "monitoringVersions": [ 17 | { 18 | "hostname": "MACHINE_HOSTNAME" 19 | } 20 | ], 21 | "processes": [ 22 | { 23 | "args2_6": { 24 | "net": { 25 | "port": 28000 26 | }, 27 | "replication": { 28 | "replSetName": "blue" 29 | }, 30 | "storage": { 31 | "dbPath": "/data/blue_0" 32 | }, 33 | "systemLog": { 34 | "destination": "file", 35 | "path": "/data/blue_0/mongodb.log" 36 | } 37 | }, 38 | "hostname": "MACHINE_HOSTNAME", 39 | "logRotate": { 40 | "sizeThresholdMB": 1000, 41 | "timeThresholdHrs": 24 42 | }, 43 | "name": "blue_0", 44 | "processType": "mongod", 45 | "version": "3.0.2", 46 | "authSchemaVersion": 3 47 | }, 48 | { 49 | "args2_6": { 50 | "net": { 51 | "port": 28001 52 | }, 53 | "replication": { 54 | "replSetName": "blue" 55 | }, 56 | "storage": { 57 | "dbPath": "/data/blue_1" 58 | }, 59 | "systemLog": { 60 | "destination": "file", 61 | "path": "/data/blue_1/mongodb.log" 62 | } 63 | }, 64 | "hostname": "MACHINE_HOSTNAME", 65 | "logRotate": { 66 | "sizeThresholdMB": 1000, 67 | "timeThresholdHrs": 24 68 | }, 69 | "name": "blue_1", 70 | "processType": "mongod", 71 | "version": "3.0.2", 72 | "authSchemaVersion": 3 73 | }, 74 | { 75 | "args2_6": { 76 | "net": { 77 | "port": 28002 78 | }, 79 | "replication": { 80 | "replSetName": "blue" 81 | }, 82 | "storage": { 83 | "dbPath": "/data/blue_2" 84 | }, 85 | "systemLog": { 86 | "destination": "file", 87 | "path": "/data/blue_2/mongodb.log" 88 | } 89 | }, 90 | "hostname": "MACHINE_HOSTNAME", 91 | "logRotate": { 92 | "sizeThresholdMB": 1000, 93 | "timeThresholdHrs": 24 94 | }, 95 | "name": "blue_2", 96 | "processType": "mongod", 97 | "version": "3.0.2", 98 | "authSchemaVersion": 3 99 | }, 100 | { 101 | "args2_6": { 102 | "net": { 103 | "port": 28017 104 | }, 105 | "systemLog": { 106 | "destination": "file", 107 | "path": "/data/blue_mongos_3/mongodb.log" 108 | } 109 | }, 110 | "cluster": "blueCluster", 111 | "hostname": "MACHINE_HOSTNAME", 112 | "logRotate": { 113 | "sizeThresholdMB": 1000, 114 | "timeThresholdHrs": 24 115 | }, 116 | "name": "blue_mongos_3", 117 | "processType": "mongos", 118 | "version": "3.0.2", 119 | "authSchemaVersion": 3 120 | }, 121 | { 122 | "args2_6": { 123 | "net": { 124 | "port": 28019 125 | }, 126 | "sharding": { 127 | "clusterRole": "configsvr" 128 | }, 129 | "storage": { 130 | "dbPath": "/data/blue_config_4" 131 | }, 132 | "systemLog": { 133 | "destination": "file", 134 | "path": "/data/blue_config_4/mongodb.log" 135 | } 136 | }, 137 | "hostname": "MACHINE_HOSTNAME", 138 | "alias": "MACHINE_HOSTNAME", 139 | "logRotate": { 140 | "sizeThresholdMB": 1000, 141 | "timeThresholdHrs": 24 142 | }, 143 | "name": "blue_config_4", 144 | "processType": "mongod", 145 | "version": "3.0.2", 146 | "authSchemaVersion": 3 147 | } 148 | ], 149 | "replicaSets": [ 150 | { 151 | "_id": "blue", 152 | "members": [ 153 | { 154 | "_id": 0, 155 | "host": "blue_0" 156 | }, 157 | { 158 | "_id": 1, 159 | "host": "blue_1" 160 | }, 161 | { 162 | "_id": 2, 163 | "arbiterOnly": true, 164 | "host": "blue_2", 165 | "priority": 0 166 | } 167 | ] 168 | } 169 | ], 170 | "sharding": [ 171 | { 172 | "configServer": [ 173 | "blue_config_4" 174 | ], 175 | "name": "blueCluster", 176 | "shards": [ 177 | { 178 | "_id": "blue", 179 | "rs": "blue" 180 | } 181 | ] 182 | } 183 | ] 184 | } 185 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_ldap.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "2.6.9-ent" 9 | } 10 | ], 11 | "backupVersions": [ 12 | { 13 | "hostname": "MACHINE_HOSTNAME" 14 | } 15 | ], 16 | 17 | "monitoringVersions": [ 18 | { 19 | "hostname": "MACHINE_HOSTNAME" 20 | } 21 | ], 22 | "processes": [ 23 | { 24 | "args2_6": { 25 | "net": { 26 | "port": 28000 27 | }, 28 | "replication": { 29 | "replSetName": "blue" 30 | }, 31 | "storage": { 32 | "dbPath": "/data/blue_0" 33 | }, 34 | "systemLog": { 35 | "destination": "file", 36 | "path": "/data/blue_0/mongodb.log" 37 | }, 38 | "setParameter": { 39 | "saslauthdPath": "/var/run/saslauthd/mux" 40 | } 41 | }, 42 | "hostname": "MACHINE_HOSTNAME", 43 | "logRotate": { 44 | "sizeThresholdMB": 1000, 45 | "timeThresholdHrs": 24 46 | }, 47 | "name": "blue_0", 48 | "processType": "mongod", 49 | "version": "2.6.9-ent", 50 | "authSchemaVersion": 3 51 | }, 52 | { 53 | "args2_6": { 54 | "net": { 55 | "port": 28001 56 | }, 57 | "replication": { 58 | "replSetName": "blue" 59 | }, 60 | "storage": { 61 | "dbPath": "/data/blue_1" 62 | }, 63 | "systemLog": { 64 | "destination": "file", 65 | "path": "/data/blue_1/mongodb.log" 66 | }, 67 | "setParameter": { 68 | "saslauthdPath": "/var/run/saslauthd/mux" 69 | } 70 | }, 71 | "hostname": "MACHINE_HOSTNAME", 72 | "logRotate": { 73 | "sizeThresholdMB": 1000, 74 | "timeThresholdHrs": 24 75 | }, 76 | "name": "blue_1", 77 | "processType": "mongod", 78 | "version": "2.6.9-ent", 79 | "authSchemaVersion": 3 80 | }, 81 | { 82 | "args2_6": { 83 | "net": { 84 | "port": 28002 85 | }, 86 | "replication": { 87 | "replSetName": "blue" 88 | }, 89 | "storage": { 90 | "dbPath": "/data/blue_2" 91 | }, 92 | "systemLog": { 93 | "destination": "file", 94 | "path": "/data/blue_2/mongodb.log" 95 | }, 96 | "setParameter": { 97 | "saslauthdPath": "/var/run/saslauthd/mux" 98 | } 99 | }, 100 | "hostname": "MACHINE_HOSTNAME", 101 | "logRotate": { 102 | "sizeThresholdMB": 1000, 103 | "timeThresholdHrs": 24 104 | }, 105 | "name": "blue_2", 106 | "processType": "mongod", 107 | "version": "2.6.9-ent", 108 | "authSchemaVersion": 3 109 | } 110 | ], 111 | "replicaSets": [ 112 | { 113 | "_id": "blue", 114 | "members": [ 115 | { 116 | "_id": 0, 117 | "host": "blue_0", 118 | "priority": 5 119 | }, 120 | { 121 | "_id": 1, 122 | "host": "blue_1" 123 | }, 124 | { 125 | "_id": 2, 126 | "arbiterOnly": true, 127 | "host": "blue_2", 128 | "priority": 0 129 | } 130 | ] 131 | } 132 | ], 133 | "roles": [], 134 | "sharding": [], 135 | "auth": { 136 | "disabled": false, 137 | "autoAuthMechanism": "PLAIN", 138 | "autoUser": "automation-agent", 139 | "autoPwd": "apassword", 140 | "deploymentAuthMechanisms": ["PLAIN"], 141 | "key": "someverylongandrandomcontents", 142 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 143 | "usersWanted": [ 144 | { 145 | "db": "$external", 146 | "roles": [ 147 | { 148 | "db": "admin", 149 | "role": "clusterMonitor" 150 | } 151 | ], 152 | "user": "monitoring-agent" 153 | }, 154 | { 155 | "db": "$external", 156 | "roles": [ 157 | { 158 | "db": "admin", 159 | "role": "clusterAdmin" 160 | }, 161 | { 162 | "db": "admin", 163 | "role": "readAnyDatabase" 164 | }, 165 | { 166 | "db": "admin", 167 | "role": "userAdminAnyDatabase" 168 | }, 169 | { 170 | "db": "local", 171 | "role": "readWrite" 172 | }, 173 | { 174 | "db": "admin", 175 | "role": "readWrite" 176 | } 177 | ], 178 | "user": "backup-agent" 179 | } 180 | ], 181 | "usersDeleted": [] 182 | } 183 | } 184 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_x509.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "3.0.1-ent" 9 | } 10 | ], 11 | "backupVersions": [ 12 | { 13 | "hostname": "MACHINE_HOSTNAME" 14 | } 15 | ], 16 | 17 | "monitoringVersions": [ 18 | { 19 | "hostname": "MACHINE_HOSTNAME" 20 | } 21 | ], 22 | "processes": [ 23 | { 24 | "args2_6": { 25 | "net": { 26 | "port": 28000, 27 | "ssl": { 28 | "mode": "requireSSL", 29 | "PEMKeyFile": "/opt/mongodb-1.pem" 30 | } 31 | }, 32 | "replication": { 33 | "replSetName": "blue" 34 | }, 35 | "storage": { 36 | "dbPath": "/data/blue_0" 37 | }, 38 | "systemLog": { 39 | "destination": "file", 40 | "path": "/data/blue_0/mongodb.log" 41 | } 42 | }, 43 | "hostname": "MACHINE_HOSTNAME", 44 | "logRotate": { 45 | "sizeThresholdMB": 1000, 46 | "timeThresholdHrs": 24 47 | }, 48 | "name": "blue_0", 49 | "processType": "mongod", 50 | "version": "3.0.1-ent", 51 | "authSchemaVersion": 5 52 | }, 53 | { 54 | "args2_6": { 55 | "net": { 56 | "port": 28001, 57 | "ssl": { 58 | "mode": "requireSSL", 59 | "PEMKeyFile": "/opt/mongodb-1.pem" 60 | } 61 | }, 62 | "replication": { 63 | "replSetName": "blue" 64 | }, 65 | "storage": { 66 | "dbPath": "/data/blue_1" 67 | }, 68 | "systemLog": { 69 | "destination": "file", 70 | "path": "/data/blue_1/mongodb.log" 71 | } 72 | }, 73 | "hostname": "MACHINE_HOSTNAME", 74 | "logRotate": { 75 | "sizeThresholdMB": 1000, 76 | "timeThresholdHrs": 24 77 | }, 78 | "name": "blue_1", 79 | "processType": "mongod", 80 | "version": "3.0.1-ent", 81 | "authSchemaVersion": 5 82 | }, 83 | { 84 | "args2_6": { 85 | "net": { 86 | "port": 28002, 87 | "ssl": { 88 | "mode": "requireSSL", 89 | "PEMKeyFile": "/opt/mongodb-1.pem" 90 | } 91 | }, 92 | "replication": { 93 | "replSetName": "blue" 94 | }, 95 | "storage": { 96 | "dbPath": "/data/blue_2" 97 | }, 98 | "systemLog": { 99 | "destination": "file", 100 | "path": "/data/blue_2/mongodb.log" 101 | } 102 | }, 103 | "hostname": "MACHINE_HOSTNAME", 104 | "logRotate": { 105 | "sizeThresholdMB": 1000, 106 | "timeThresholdHrs": 24 107 | }, 108 | "name": "blue_2", 109 | "processType": "mongod", 110 | "version": "3.0.1-ent", 111 | "authSchemaVersion": 5 112 | } 113 | ], 114 | "replicaSets": [ 115 | { 116 | "_id": "blue", 117 | "members": [ 118 | { 119 | "_id": 0, 120 | "host": "blue_0", 121 | "priority": 5 122 | }, 123 | { 124 | "_id": 1, 125 | "host": "blue_1" 126 | }, 127 | { 128 | "_id": 2, 129 | "host": "blue_2" 130 | } 131 | ] 132 | } 133 | ], 134 | "roles": [], 135 | "sharding": [], 136 | "auth": { 137 | "disabled": false, 138 | "autoAuthMechanism": "MONGODB-X509", 139 | "autoUser": "CN=automation-agent,O=MMS", 140 | "deploymentAuthMechanisms": ["MONGODB-X509"], 141 | "key": "someverylongandrandomcontents", 142 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 143 | "usersWanted": [ 144 | { 145 | "db": "$external", 146 | "roles": [ 147 | { 148 | "db": "admin", 149 | "role": "clusterMonitor" 150 | } 151 | ], 152 | "user": "CN=monitoring-agent,O=MMS" 153 | }, 154 | { 155 | "db": "$external", 156 | "roles": [ 157 | { 158 | "db": "admin", 159 | "role": "clusterAdmin" 160 | }, 161 | { 162 | "db": "admin", 163 | "role": "readAnyDatabase" 164 | }, 165 | { 166 | "db": "admin", 167 | "role": "userAdminAnyDatabase" 168 | }, 169 | { 170 | "db": "local", 171 | "role": "readWrite" 172 | }, 173 | { 174 | "db": "admin", 175 | "role": "readWrite" 176 | } 177 | ], 178 | "user": "CN=backup-agent,O=MMS" 179 | } 180 | ], 181 | "usersDeleted": [] 182 | }, 183 | "ssl": { 184 | "CAFilePath": "/opt/ca.pem", 185 | "clientCertificateMode": "REQUIRE", 186 | "autoPEMKeyFilePath": "/opt/automation-agent.pem" 187 | 188 | } 189 | } 190 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_kerberos.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "2.6.9-ent" 9 | } 10 | ], 11 | "backupVersions": [ 12 | { 13 | "hostname": "MACHINE_HOSTNAME" 14 | } 15 | ], 16 | 17 | "monitoringVersions": [ 18 | { 19 | "hostname": "MACHINE_HOSTNAME" 20 | } 21 | ], 22 | "processes": [ 23 | { 24 | "args2_6": { 25 | "net": { 26 | "port": 28000 27 | }, 28 | "replication": { 29 | "replSetName": "blue" 30 | }, 31 | "storage": { 32 | "dbPath": "/data/blue_0" 33 | }, 34 | "systemLog": { 35 | "destination": "file", 36 | "path": "/data/blue_0/mongodb.log" 37 | } 38 | }, 39 | "hostname": "MACHINE_HOSTNAME", 40 | "logRotate": { 41 | "sizeThresholdMB": 1000, 42 | "timeThresholdHrs": 24 43 | }, 44 | "name": "blue_0", 45 | "processType": "mongod", 46 | "version": "2.6.9-ent", 47 | "authSchemaVersion": 3, 48 | "kerberos": { 49 | "keytab": "/opt/mongodb-1.keytab" 50 | } 51 | }, 52 | { 53 | "args2_6": { 54 | "net": { 55 | "port": 28001 56 | }, 57 | "replication": { 58 | "replSetName": "blue" 59 | }, 60 | "storage": { 61 | "dbPath": "/data/blue_1" 62 | }, 63 | "systemLog": { 64 | "destination": "file", 65 | "path": "/data/blue_1/mongodb.log" 66 | } 67 | }, 68 | "hostname": "MACHINE_HOSTNAME", 69 | "logRotate": { 70 | "sizeThresholdMB": 1000, 71 | "timeThresholdHrs": 24 72 | }, 73 | "name": "blue_1", 74 | "processType": "mongod", 75 | "version": "2.6.9-ent", 76 | "authSchemaVersion": 3, 77 | "kerberos": { 78 | "keytab": "/opt/mongodb-1.keytab" 79 | } 80 | }, 81 | { 82 | "args2_6": { 83 | "net": { 84 | "port": 28002 85 | }, 86 | "replication": { 87 | "replSetName": "blue" 88 | }, 89 | "storage": { 90 | "dbPath": "/data/blue_2" 91 | }, 92 | "systemLog": { 93 | "destination": "file", 94 | "path": "/data/blue_2/mongodb.log" 95 | } 96 | }, 97 | "hostname": "MACHINE_HOSTNAME", 98 | "logRotate": { 99 | "sizeThresholdMB": 1000, 100 | "timeThresholdHrs": 24 101 | }, 102 | "name": "blue_2", 103 | "processType": "mongod", 104 | "version": "2.6.9-ent", 105 | "authSchemaVersion": 3, 106 | "kerberos": { 107 | "keytab": "/opt/mongodb-1.keytab" 108 | } 109 | } 110 | ], 111 | "replicaSets": [ 112 | { 113 | "_id": "blue", 114 | "members": [ 115 | { 116 | "_id": 0, 117 | "host": "blue_0" 118 | }, 119 | { 120 | "_id": 1, 121 | "host": "blue_1" 122 | }, 123 | { 124 | "_id": 2, 125 | "arbiterOnly": true, 126 | "host": "blue_2", 127 | "priority": 0 128 | } 129 | ] 130 | } 131 | ], 132 | "roles": [], 133 | "sharding": [], 134 | "auth": { 135 | "disabled": false, 136 | "autoAuthMechanism": "GSSAPI", 137 | "autoUser": "automation-agent/MACHINE_HOSTNAME@MMS.COM", 138 | "autoKerberosKeytabPath": "/opt/automation-agent.keytab", 139 | "deploymentAuthMechanisms": ["GSSAPI"], 140 | "key": "GrSLAAsHGXmJOrvElJ2AHTGauvH4O0EFT1r8byvb0G9sTU0viVX21PwUMqBjyXB9WrZP9QvEmCQIF1wOqJofyWmx7wWZqpO69dnc9GUWcpGQLr7eVyKTs99WAPXR3kXpF4MVrHdBMEDfRfhytgomgAso96urN6eC8RaUpjX4Bf9HcAEJwfddZshin97XKJDmqCaqAfORNnf1e8hkfTIwYg1tvIpwemmEF4TkmOgK09N5dINyejyWMU8iWG8FqW5MfQ8A2DrtIdyGSKLH05s7H1dXyADjDECaC77QqLXTx7gWyHca3I0K92PuVFoOs5385vzqTYN3kVgFotSdXgoM8Zt5QIoj2lX4PYqm2TWsVp0s15JELikH8bNVIIMGiSSWJEWGU1PVEXD7V7cYepDb88korMjr3wbh6kZ76Q7F2RtfJqkd4hKw7B5OCX04b5eppkjL598iCpSUUx3X9C6fFavWj2DrHsv9DY86iCWBlcG08DRPKs9EPizCW4jNZtJcm3T7WlcI0MZMKOtsKOCWBZA0C9YnttNrp4eTsQ1U43StiIRPqp2K8rrQAu6etURH0RHedazHeeukTWI7iTG1dZpYk9EyittZ72qKXLNLhi5vJ9TlYw8O91vihB1nJwwA3B1WbiYhkqqRzoL0cQpXJMUsUlsoSP6Q70IMU92vEHbUmna5krESPLeJfQBKGQPNVVE63XYBh2TnvFTdi6koitu209wMFUnHZrzWj3UWGqsyTqqHbPl4RhRLFe24seRwV2SbUuLygBIdptKHnA3kutAbHzsWTT8UxOaiQzFV4auxounrgXj7MoMWEVKKS8AHkELPILGqFVFC8BZsfPC0WacSN5Rg5SaCvfs74hcsCQ3ghq9PyxEb2fbHUiaCjnsBcXqzQw9AjZJG4yX0ubEwicP0bKB6y3w4PUQqdouxH5y16OgkUjrZgodJfRLgP9vqGbHNDpj4yBuswluvCFBh38gBoSIQu11qtQmk43n4G8Dskn0DrJ32l2Gz35q5LaKT", 141 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 142 | "usersWanted": [ 143 | { 144 | "db": "$external", 145 | "roles": [ 146 | { 147 | "db": "admin", 148 | "role": "clusterMonitor" 149 | } 150 | ], 151 | "user": "monitoring-agent/MACHINE_HOSTNAME@MMS.COM" 152 | }, 153 | { 154 | "db": "$external", 155 | "roles": [ 156 | { 157 | "db": "admin", 158 | "role": "clusterAdmin" 159 | }, 160 | { 161 | "db": "admin", 162 | "role": "readAnyDatabase" 163 | }, 164 | { 165 | "db": "admin", 166 | "role": "userAdminAnyDatabase" 167 | }, 168 | { 169 | "db": "local", 170 | "role": "readWrite" 171 | }, 172 | { 173 | "db": "admin", 174 | "role": "readWrite" 175 | } 176 | ], 177 | "user": "backup-agent/MACHINE_HOSTNAME@MMS.COM" 178 | } 179 | ], 180 | "usersDeleted": [] 181 | }, 182 | "kerberos": { 183 | "serviceName": "mongodb-1" 184 | } 185 | } 186 | -------------------------------------------------------------------------------- /automation/api_usage_example/backup_api_base.py: -------------------------------------------------------------------------------- 1 | from api_base import ApiBase 2 | 3 | class BackupApiBase(ApiBase): 4 | 5 | def __init__(self, base_url, group_id, api_user, api_key): 6 | ApiBase.__init__(self, base_url, group_id, api_user, api_key) 7 | 8 | def get_restore_job_result_replica_set(self, replica_set_name, job_id): 9 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 10 | return self._get_restore_job_result(cluster_id, job_id=job_id) 11 | 12 | def get_restore_job_result_cluster(self, cluster_name, batch_id): 13 | cluster_id = self._get_cluster_id_from_sharded_cluster(cluster_name) 14 | return self._get_restore_job_result(cluster_id, batch_id=batch_id) 15 | 16 | def _get_restore_job_result(self, cluster_id, job_id=None, batch_id=None): 17 | url = "%s/groups/%s/clusters/%s/restoreJobs" % (self.base_url, self.group_id, cluster_id) 18 | 19 | if job_id: 20 | url = "%s/%s" % (url, job_id) 21 | 22 | if batch_id: 23 | url = "%s?batchId=%s" % (url, batch_id) 24 | 25 | return self.get(url) 26 | 27 | def request_restore_http_replica_set(self, replica_set_name, snapshot_id, max_downloads, expiration_hours): 28 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 29 | return self._request_restore(cluster_id, snapshot_id, max_downloads, expiration_hours) 30 | 31 | def request_restore_http_cluster(self, cluster_name, snapshot_id, max_downloads, expiration_hours): 32 | cluster_id = self._get_cluster_id_from_sharded_cluster(cluster_name) 33 | return self._request_restore(cluster_id, snapshot_id, max_downloads, expiration_hours) 34 | 35 | def _request_restore(self, cluster_id, snapshot_id, max_downloads, expiration_hours): 36 | url = "%s/groups/%s/clusters/%s/restoreJobs" % (self.base_url, self.group_id, cluster_id) 37 | 38 | json_body = { 39 | 'snapshotId': snapshot_id, 40 | 'delivery': { 41 | 'methodName': 'HTTP', 42 | 'maxDownloads': max_downloads, 43 | 'expirationHours': expiration_hours 44 | } 45 | } 46 | 47 | response = self.post(url, json_body) 48 | return response['results'][0] 49 | 50 | def request_automated_restore(self, cluster_id, snapshot_id, target_group_id, target_cluster_id): 51 | url = "%s/groups/%s/clusters/%s/restoreJobs" % (self.base_url, self.group_id, cluster_id) 52 | 53 | json_body = { 54 | 'snapshotId': snapshot_id, 55 | 'delivery': { 56 | 'methodName': 'AUTOMATED_RESTORE', 57 | 'targetGroupId': target_group_id, 58 | 'targetClusterId': target_cluster_id 59 | } 60 | } 61 | 62 | response = self.post(url, json_body) 63 | return response['results'][0] 64 | 65 | def get_snapshots_replica_set(self, replica_set_name): 66 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 67 | return self._get_snapshots(cluster_id) 68 | 69 | def get_snapshots_cluster(self, cluster_name): 70 | cluster_id = self._get_cluster_id_from_sharded_cluster(cluster_name) 71 | return self._get_snapshots(cluster_id) 72 | 73 | def _get_snapshots(self, cluster_id): 74 | url = "%s/groups/%s/clusters/%s/snapshots" % (self.base_url, self.group_id, cluster_id) 75 | response = self.get(url) 76 | return response['results'] 77 | 78 | def start_backup_replica_set(self, replica_set_name, storage_engine=None): 79 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 80 | return self._start_backup(cluster_id, storage_engine) 81 | 82 | def start_backup_cluster(self, cluster_name, storage_engine=None): 83 | cluster_id = self._get_cluster_id_from_sharded_cluster(cluster_name) 84 | return self._start_backup(cluster_id, storage_engine) 85 | 86 | def _start_backup(self, cluster_id, storage_engine): 87 | 88 | status = self._get_backup_configuration(cluster_id) 89 | if status['statusName'] == 'STARTED': 90 | return 91 | 92 | url = "%s/groups/%s/backupConfigs/%s" % (self.base_url, self.group_id, cluster_id) 93 | json_body = { 94 | 'statusName': 'STARTED', 95 | "syncSource": "SECONDARY" 96 | } 97 | 98 | if storage_engine: 99 | json_body['storageEngineName'] = storage_engine 100 | 101 | self.patch(url, json_body) 102 | 103 | def get_snapshot_schedule_replica_set(self, replica_set_name): 104 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 105 | snapshot_schedule = self._get_snapshot_schedule(cluster_id) 106 | del snapshot_schedule['links'] 107 | del snapshot_schedule['groupId'] 108 | del snapshot_schedule['clusterId'] 109 | return dict(snapshot_schedule) 110 | 111 | def _get_snapshot_schedule(self, cluster_id): 112 | url = "%s/groups/%s/backupConfigs/%s/snapshotSchedule" % (self.base_url, self.group_id, cluster_id) 113 | return self.get(url) 114 | 115 | def update_snapshot_schedule_replica_set(self, replica_set_name, snapshot_schedule): 116 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 117 | return self._update_snapshot_schedule(cluster_id, snapshot_schedule) 118 | 119 | def _update_snapshot_schedule(self, cluster_id, snapshot_schedule): 120 | url = "%s/groups/%s/backupConfigs/%s/snapshotSchedule" % (self.base_url, self.group_id, cluster_id) 121 | return self.patch(url, snapshot_schedule) 122 | 123 | def get_backup_configuration_replica_set(self, replica_set_name): 124 | cluster_id = self._get_cluster_id_from_replica_set(replica_set_name) 125 | return self._get_backup_configuration(cluster_id) 126 | 127 | def get_backup_configuration_cluster(self, cluster_name): 128 | cluster_id = self._get_cluster_id_from_sharded_cluster(cluster_name) 129 | return self._get_backup_configuration(cluster_id) 130 | 131 | def _get_backup_configuration(self, cluster_id): 132 | url = "%s/groups/%s/backupConfigs/%s" % (self.base_url, self.group_id, cluster_id) 133 | return self.get(url) 134 | 135 | def _get_cluster_id_from_replica_set(self, replica_set_name): 136 | url = "%s/groups/%s/clusters" % (self.base_url, self.group_id) 137 | response = self.get(url) 138 | for cluster in response['results']: 139 | if cluster['replicaSetName'] == replica_set_name: 140 | return cluster['id'] 141 | 142 | return None 143 | 144 | def _get_cluster_id_from_sharded_cluster(self, cluster_name): 145 | url = "%s/groups/%s/clusters" % (self.base_url, self.group_id) 146 | response = self.get(url) 147 | for cluster in response['results']: 148 | if cluster['clusterName'] == cluster_name and cluster['typeName'] == 'SHARDED_REPLICA_SET': 149 | return cluster['id'] 150 | 151 | return None 152 | 153 | def get_cluster_id_from_cluster_name(self, cluster_name): 154 | return self._get_cluster_id_from_replica_set(cluster_name) or self._get_cluster_id_from_sharded_cluster(cluster_name) 155 | 156 | def get_replica_set_from_cluster_id(self, cluster_id): 157 | url = "%s/groups/%s/clusters" % (self.base_url, self.group_id) 158 | response = self.get(url) 159 | for cluster in response['results']: 160 | if cluster['id'] == cluster_id and cluster.get('replicaSetName'): 161 | return cluster['replicaSetName'] 162 | 163 | return None 164 | 165 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/mongo30_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.0.6"}, 8 | {"name": "3.1.7"} 9 | ], 10 | "backupVersions": [ 11 | { 12 | "hostname": "MACHINE_HOSTNAME" 13 | } 14 | ], 15 | 16 | "monitoringVersions": [ 17 | { 18 | "hostname": "MACHINE_HOSTNAME" 19 | } 20 | ], 21 | "processes": [ 22 | { 23 | "args2_6": { 24 | "net": { 25 | "port": 28000 26 | }, 27 | "replication": { 28 | "replSetName": "blue" 29 | }, 30 | "storage": { 31 | "dbPath": "/tmp/data/blue_0" 32 | }, 33 | "systemLog": { 34 | "destination": "file", 35 | "path": "/tmp/data/blue_0/mongodb.log" 36 | } 37 | }, 38 | "hostname": "MACHINE_HOSTNAME", 39 | "logRotate": { 40 | "sizeThresholdMB": 1000, 41 | "timeThresholdHrs": 24 42 | }, 43 | "name": "blue_0", 44 | "processType": "mongod", 45 | "version": "3.0.6", 46 | "authSchemaVersion": 3 47 | }, 48 | { 49 | "args2_6": { 50 | "net": { 51 | "port": 28001 52 | }, 53 | "replication": { 54 | "replSetName": "blue" 55 | }, 56 | "storage": { 57 | "dbPath": "/tmp/data/blue_1" 58 | }, 59 | "systemLog": { 60 | "destination": "file", 61 | "path": "/tmp/data/blue_1/mongodb.log" 62 | } 63 | }, 64 | "hostname": "MACHINE_HOSTNAME", 65 | "logRotate": { 66 | "sizeThresholdMB": 1000, 67 | "timeThresholdHrs": 24 68 | }, 69 | "name": "blue_1", 70 | "processType": "mongod", 71 | "version": "3.1.7", 72 | "authSchemaVersion": 3 73 | }, 74 | { 75 | "args2_6": { 76 | "net": { 77 | "port": 28002 78 | }, 79 | "replication": { 80 | "replSetName": "blue" 81 | }, 82 | "storage": { 83 | "dbPath": "/tmp/data/blue_2" 84 | }, 85 | "systemLog": { 86 | "destination": "file", 87 | "path": "/tmp/data/blue_2/mongodb.log" 88 | } 89 | }, 90 | "hostname": "MACHINE_HOSTNAME", 91 | "logRotate": { 92 | "sizeThresholdMB": 1000, 93 | "timeThresholdHrs": 24 94 | }, 95 | "name": "blue_2", 96 | "processType": "mongod", 97 | "version": "3.1.7", 98 | "authSchemaVersion": 3 99 | }, 100 | { 101 | "args2_6": { 102 | "net": { 103 | "port": 28017 104 | }, 105 | "systemLog": { 106 | "destination": "file", 107 | "path": "/tmp/data/blue_mongos_3/mongodb.log" 108 | } 109 | }, 110 | "cluster": "blueCluster", 111 | "hostname": "MACHINE_HOSTNAME", 112 | "logRotate": { 113 | "sizeThresholdMB": 1000, 114 | "timeThresholdHrs": 24 115 | }, 116 | "name": "blue_mongos_3", 117 | "processType": "mongos", 118 | "version": "3.1.7", 119 | "authSchemaVersion": 3 120 | }, 121 | { 122 | "args2_6": { 123 | "net": { 124 | "port": 28019 125 | }, 126 | "sharding": { 127 | "clusterRole": "configsvr" 128 | }, 129 | "storage": { 130 | "dbPath": "/tmp/data/blue_config_4" 131 | }, 132 | "systemLog": { 133 | "destination": "file", 134 | "path": "/tmp/data/blue_config_4/mongodb.log" 135 | } 136 | }, 137 | "hostname": "MACHINE_HOSTNAME", 138 | "alias": "MACHINE_HOSTNAME", 139 | "logRotate": { 140 | "sizeThresholdMB": 1000, 141 | "timeThresholdHrs": 24 142 | }, 143 | "name": "blue_config_4", 144 | "processType": "mongod", 145 | "version": "3.1.7", 146 | "authSchemaVersion": 3 147 | }, 148 | { 149 | "args2_6": { 150 | "net": { 151 | "port": 28020 152 | }, 153 | "sharding": { 154 | "clusterRole": "configsvr" 155 | }, 156 | "storage": { 157 | "dbPath": "/tmp/data/blue_config_5" 158 | }, 159 | "systemLog": { 160 | "destination": "file", 161 | "path": "/tmp/data/blue_config_5/mongodb.log" 162 | } 163 | }, 164 | "hostname": "MACHINE_HOSTNAME", 165 | "alias": "MACHINE_HOSTNAME", 166 | "logRotate": { 167 | "sizeThresholdMB": 1000, 168 | "timeThresholdHrs": 24 169 | }, 170 | "name": "blue_config_5", 171 | "processType": "mongod", 172 | "version": "3.1.7", 173 | "authSchemaVersion": 3 174 | }, 175 | { 176 | "args2_6": { 177 | "net": { 178 | "port": 28021 179 | }, 180 | "sharding": { 181 | "clusterRole": "configsvr" 182 | }, 183 | "storage": { 184 | "dbPath": "/tmp/data/blue_config_6" 185 | }, 186 | "systemLog": { 187 | "destination": "file", 188 | "path": "/tmp/data/blue_config_6/mongodb.log" 189 | } 190 | }, 191 | "hostname": "MACHINE_HOSTNAME", 192 | "alias": "MACHINE_HOSTNAME", 193 | "logRotate": { 194 | "sizeThresholdMB": 1000, 195 | "timeThresholdHrs": 24 196 | }, 197 | "name": "blue_config_6", 198 | "processType": "mongod", 199 | "version": "3.1.7", 200 | "authSchemaVersion": 3 201 | } 202 | ], 203 | "replicaSets": [ 204 | { 205 | "_id": "blue", 206 | "members": [ 207 | { 208 | "_id": 0, 209 | "host": "blue_0" 210 | }, 211 | { 212 | "_id": 1, 213 | "host": "blue_1" 214 | }, 215 | { 216 | "_id": 2, 217 | "arbiterOnly": true, 218 | "host": "blue_2", 219 | "priority": 0 220 | } 221 | ] 222 | } 223 | ], 224 | "sharding": [ 225 | { 226 | "configServer": [ 227 | "blue_config_4", 228 | "blue_config_5", 229 | "blue_config_6" 230 | ], 231 | "name": "blueCluster", 232 | "shards": [ 233 | { 234 | "_id": "blue", 235 | "rs": "blue" 236 | } 237 | ] 238 | } 239 | ] 240 | } 241 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/mongo32_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.0.6"}, 8 | {"name": "3.1.7"} 9 | ], 10 | "backupVersions": [ 11 | { 12 | "hostname": "MACHINE_HOSTNAME" 13 | } 14 | ], 15 | 16 | "monitoringVersions": [ 17 | { 18 | "hostname": "MACHINE_HOSTNAME" 19 | } 20 | ], 21 | "processes": [ 22 | { 23 | "args2_6": { 24 | "net": { 25 | "port": 28000 26 | }, 27 | "replication": { 28 | "replSetName": "blue" 29 | }, 30 | "storage": { 31 | "dbPath": "/tmp/data/blue_0" 32 | }, 33 | "systemLog": { 34 | "destination": "file", 35 | "path": "/tmp/data/blue_0/mongodb.log" 36 | } 37 | }, 38 | "hostname": "MACHINE_HOSTNAME", 39 | "logRotate": { 40 | "sizeThresholdMB": 1000, 41 | "timeThresholdHrs": 24 42 | }, 43 | "name": "blue_0", 44 | "processType": "mongod", 45 | "version": "3.1.7", 46 | "authSchemaVersion": 3 47 | }, 48 | { 49 | "args2_6": { 50 | "net": { 51 | "port": 28001 52 | }, 53 | "replication": { 54 | "replSetName": "blue" 55 | }, 56 | "storage": { 57 | "dbPath": "/tmp/data/blue_1" 58 | }, 59 | "systemLog": { 60 | "destination": "file", 61 | "path": "/tmp/data/blue_1/mongodb.log" 62 | } 63 | }, 64 | "hostname": "MACHINE_HOSTNAME", 65 | "logRotate": { 66 | "sizeThresholdMB": 1000, 67 | "timeThresholdHrs": 24 68 | }, 69 | "name": "blue_1", 70 | "processType": "mongod", 71 | "version": "3.1.7", 72 | "authSchemaVersion": 3 73 | }, 74 | { 75 | "args2_6": { 76 | "net": { 77 | "port": 28002 78 | }, 79 | "replication": { 80 | "replSetName": "blue" 81 | }, 82 | "storage": { 83 | "dbPath": "/tmp/data/blue_2" 84 | }, 85 | "systemLog": { 86 | "destination": "file", 87 | "path": "/tmp/data/blue_2/mongodb.log" 88 | } 89 | }, 90 | "hostname": "MACHINE_HOSTNAME", 91 | "logRotate": { 92 | "sizeThresholdMB": 1000, 93 | "timeThresholdHrs": 24 94 | }, 95 | "name": "blue_2", 96 | "processType": "mongod", 97 | "version": "3.1.7", 98 | "authSchemaVersion": 3 99 | }, 100 | { 101 | "args2_6": { 102 | "net": { 103 | "port": 28017 104 | }, 105 | "systemLog": { 106 | "destination": "file", 107 | "path": "/tmp/data/blue_mongos_3/mongodb.log" 108 | } 109 | }, 110 | "cluster": "blueCluster", 111 | "hostname": "MACHINE_HOSTNAME", 112 | "logRotate": { 113 | "sizeThresholdMB": 1000, 114 | "timeThresholdHrs": 24 115 | }, 116 | "name": "blue_mongos_3", 117 | "processType": "mongos", 118 | "version": "3.1.7", 119 | "authSchemaVersion": 3 120 | }, 121 | { 122 | "args2_6": { 123 | "net": { 124 | "port": 28019 125 | }, 126 | "sharding": { 127 | "clusterRole": "configsvr" 128 | }, 129 | "storage": { 130 | "dbPath": "/tmp/data/blue_config_4" 131 | }, 132 | "systemLog": { 133 | "destination": "file", 134 | "path": "/tmp/data/blue_config_4/mongodb.log" 135 | } 136 | }, 137 | "hostname": "MACHINE_HOSTNAME", 138 | "alias": "MACHINE_HOSTNAME", 139 | "logRotate": { 140 | "sizeThresholdMB": 1000, 141 | "timeThresholdHrs": 24 142 | }, 143 | "name": "blue_config_4", 144 | "processType": "mongod", 145 | "version": "3.1.7", 146 | "authSchemaVersion": 3 147 | }, 148 | { 149 | "args2_6": { 150 | "net": { 151 | "port": 28020 152 | }, 153 | "sharding": { 154 | "clusterRole": "configsvr" 155 | }, 156 | "storage": { 157 | "dbPath": "/tmp/data/blue_config_5" 158 | }, 159 | "systemLog": { 160 | "destination": "file", 161 | "path": "/tmp/data/blue_config_5/mongodb.log" 162 | } 163 | }, 164 | "hostname": "MACHINE_HOSTNAME", 165 | "alias": "MACHINE_HOSTNAME", 166 | "logRotate": { 167 | "sizeThresholdMB": 1000, 168 | "timeThresholdHrs": 24 169 | }, 170 | "name": "blue_config_5", 171 | "processType": "mongod", 172 | "version": "3.1.7", 173 | "authSchemaVersion": 3 174 | }, 175 | { 176 | "args2_6": { 177 | "net": { 178 | "port": 28021 179 | }, 180 | "sharding": { 181 | "clusterRole": "configsvr" 182 | }, 183 | "storage": { 184 | "dbPath": "/tmp/data/blue_config_6" 185 | }, 186 | "systemLog": { 187 | "destination": "file", 188 | "path": "/tmp/data/blue_config_6/mongodb.log" 189 | } 190 | }, 191 | "hostname": "MACHINE_HOSTNAME", 192 | "alias": "MACHINE_HOSTNAME", 193 | "logRotate": { 194 | "sizeThresholdMB": 1000, 195 | "timeThresholdHrs": 24 196 | }, 197 | "name": "blue_config_6", 198 | "processType": "mongod", 199 | "version": "3.1.7", 200 | "authSchemaVersion": 3 201 | } 202 | ], 203 | "replicaSets": [ 204 | { 205 | "_id": "blue", 206 | "members": [ 207 | { 208 | "_id": 0, 209 | "host": "blue_0" 210 | }, 211 | { 212 | "_id": 1, 213 | "host": "blue_1" 214 | }, 215 | { 216 | "_id": 2, 217 | "arbiterOnly": true, 218 | "host": "blue_2", 219 | "priority": 0 220 | } 221 | ] 222 | } 223 | ], 224 | "sharding": [ 225 | { 226 | "configServer": [ 227 | "blue_config_4", 228 | "blue_config_5", 229 | "blue_config_6" 230 | ], 231 | "name": "blueCluster", 232 | "shards": [ 233 | { 234 | "_id": "blue", 235 | "rs": "blue" 236 | } 237 | ] 238 | } 239 | ] 240 | } 241 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_ldap_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "2.6.9-ent" 9 | } 10 | ], 11 | "backupVersions": [ 12 | { 13 | "hostname": "MACHINE_HOSTNAME" 14 | } 15 | ], 16 | 17 | "monitoringVersions": [ 18 | { 19 | "hostname": "MACHINE_HOSTNAME" 20 | } 21 | ], 22 | "processes": [ 23 | { 24 | "args2_6": { 25 | "net": { 26 | "port": 28000 27 | }, 28 | "replication": { 29 | "replSetName": "blue" 30 | }, 31 | "storage": { 32 | "dbPath": "/data/blue_0" 33 | }, 34 | "systemLog": { 35 | "destination": "file", 36 | "path": "/data/blue_0/mongodb.log" 37 | }, 38 | "setParameter": { 39 | "saslauthdPath": "/var/run/saslauthd/mux" 40 | } 41 | }, 42 | "hostname": "MACHINE_HOSTNAME", 43 | "logRotate": { 44 | "sizeThresholdMB": 1000, 45 | "timeThresholdHrs": 24 46 | }, 47 | "name": "blue_0", 48 | "processType": "mongod", 49 | "version": "2.6.9-ent", 50 | "authSchemaVersion": 3 51 | }, 52 | { 53 | "args2_6": { 54 | "net": { 55 | "port": 28001 56 | }, 57 | "replication": { 58 | "replSetName": "blue" 59 | }, 60 | "storage": { 61 | "dbPath": "/data/blue_1" 62 | }, 63 | "systemLog": { 64 | "destination": "file", 65 | "path": "/data/blue_1/mongodb.log" 66 | }, 67 | "setParameter": { 68 | "saslauthdPath": "/var/run/saslauthd/mux" 69 | } 70 | }, 71 | "hostname": "MACHINE_HOSTNAME", 72 | "logRotate": { 73 | "sizeThresholdMB": 1000, 74 | "timeThresholdHrs": 24 75 | }, 76 | "name": "blue_1", 77 | "processType": "mongod", 78 | "version": "2.6.9-ent", 79 | "authSchemaVersion": 3 80 | }, 81 | { 82 | "args2_6": { 83 | "net": { 84 | "port": 28002 85 | }, 86 | "replication": { 87 | "replSetName": "blue" 88 | }, 89 | "storage": { 90 | "dbPath": "/data/blue_2" 91 | }, 92 | "systemLog": { 93 | "destination": "file", 94 | "path": "/data/blue_2/mongodb.log" 95 | }, 96 | "setParameter": { 97 | "saslauthdPath": "/var/run/saslauthd/mux" 98 | } 99 | }, 100 | "hostname": "MACHINE_HOSTNAME", 101 | "logRotate": { 102 | "sizeThresholdMB": 1000, 103 | "timeThresholdHrs": 24 104 | }, 105 | "name": "blue_2", 106 | "processType": "mongod", 107 | "version": "2.6.9-ent", 108 | "authSchemaVersion": 3 109 | }, 110 | { 111 | "args2_6": { 112 | "net": { 113 | "port": 28017 114 | }, 115 | "systemLog": { 116 | "destination": "file", 117 | "path": "/data/blue_mongos_3/mongodb.log" 118 | }, 119 | "setParameter": { 120 | "saslauthdPath": "/var/run/saslauthd/mux" 121 | } 122 | }, 123 | "cluster": "blueCluster", 124 | "hostname": "MACHINE_HOSTNAME", 125 | "logRotate": { 126 | "sizeThresholdMB": 1000, 127 | "timeThresholdHrs": 24 128 | }, 129 | "name": "blue_mongos_3", 130 | "processType": "mongos", 131 | "version": "2.6.9-ent", 132 | "authSchemaVersion": 3 133 | }, 134 | { 135 | "args2_6": { 136 | "net": { 137 | "port": 28019 138 | }, 139 | "sharding": { 140 | "clusterRole": "configsvr" 141 | }, 142 | "storage": { 143 | "dbPath": "/data/blue_config_4" 144 | }, 145 | "systemLog": { 146 | "destination": "file", 147 | "path": "/data/blue_config_4/mongodb.log" 148 | }, 149 | "setParameter": { 150 | "saslauthdPath": "/var/run/saslauthd/mux" 151 | } 152 | }, 153 | "hostname": "MACHINE_HOSTNAME", 154 | "alias": "MACHINE_HOSTNAME", 155 | "logRotate": { 156 | "sizeThresholdMB": 1000, 157 | "timeThresholdHrs": 24 158 | }, 159 | "name": "blue_config_4", 160 | "processType": "mongod", 161 | "version": "2.6.9-ent", 162 | "authSchemaVersion": 3 163 | } 164 | ], 165 | "replicaSets": [ 166 | { 167 | "_id": "blue", 168 | "members": [ 169 | { 170 | "_id": 0, 171 | "host": "blue_0", 172 | "priority": 5 173 | }, 174 | { 175 | "_id": 1, 176 | "host": "blue_1" 177 | }, 178 | { 179 | "_id": 2, 180 | "arbiterOnly": true, 181 | "host": "blue_2", 182 | "priority": 0 183 | } 184 | ] 185 | } 186 | ], 187 | "roles": [], 188 | "sharding": [ 189 | { 190 | "configServer": [ 191 | "blue_config_4" 192 | ], 193 | "name": "blueCluster", 194 | "shards": [ 195 | { 196 | "_id": "blue", 197 | "rs": "blue" 198 | } 199 | ] 200 | } 201 | ], 202 | "auth": { 203 | "disabled": false, 204 | "autoAuthMechanism": "PLAIN", 205 | "autoUser": "automation-agent", 206 | "autoPwd": "apassword", 207 | "deploymentAuthMechanisms": ["PLAIN"], 208 | "key": "someverylongandrandomcontents", 209 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 210 | "usersWanted": [ 211 | { 212 | "db": "$external", 213 | "roles": [ 214 | { 215 | "db": "admin", 216 | "role": "clusterMonitor" 217 | } 218 | ], 219 | "user": "monitoring-agent" 220 | }, 221 | { 222 | "db": "$external", 223 | "roles": [ 224 | { 225 | "db": "admin", 226 | "role": "clusterAdmin" 227 | }, 228 | { 229 | "db": "admin", 230 | "role": "readAnyDatabase" 231 | }, 232 | { 233 | "db": "admin", 234 | "role": "userAdminAnyDatabase" 235 | }, 236 | { 237 | "db": "local", 238 | "role": "readWrite" 239 | }, 240 | { 241 | "db": "admin", 242 | "role": "readWrite" 243 | } 244 | ], 245 | "user": "backup-agent" 246 | } 247 | ], 248 | "usersDeleted": [] 249 | } 250 | } 251 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/mongo32_csrs_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | {"name": "3.2.5"} 8 | ], 9 | "backupVersions": [ 10 | { 11 | "hostname": "MACHINE_HOSTNAME" 12 | } 13 | ], 14 | 15 | "monitoringVersions": [ 16 | { 17 | "hostname": "MACHINE_HOSTNAME" 18 | } 19 | ], 20 | "processes": [ 21 | { 22 | "args2_6": { 23 | "net": { 24 | "port": 28000 25 | }, 26 | "replication": { 27 | "replSetName": "blue" 28 | }, 29 | "storage": { 30 | "dbPath": "/tmp/data/blue_0" 31 | }, 32 | "systemLog": { 33 | "destination": "file", 34 | "path": "/tmp/data/blue_0/mongodb.log" 35 | } 36 | }, 37 | "hostname": "MACHINE_HOSTNAME", 38 | "logRotate": { 39 | "sizeThresholdMB": 1000, 40 | "timeThresholdHrs": 24 41 | }, 42 | "name": "blue_0", 43 | "processType": "mongod", 44 | "version": "3.2.5", 45 | "authSchemaVersion": 3 46 | }, 47 | { 48 | "args2_6": { 49 | "net": { 50 | "port": 28001 51 | }, 52 | "replication": { 53 | "replSetName": "blue" 54 | }, 55 | "storage": { 56 | "dbPath": "/tmp/data/blue_1" 57 | }, 58 | "systemLog": { 59 | "destination": "file", 60 | "path": "/tmp/data/blue_1/mongodb.log" 61 | } 62 | }, 63 | "hostname": "MACHINE_HOSTNAME", 64 | "logRotate": { 65 | "sizeThresholdMB": 1000, 66 | "timeThresholdHrs": 24 67 | }, 68 | "name": "blue_1", 69 | "processType": "mongod", 70 | "version": "3.2.5", 71 | "authSchemaVersion": 3 72 | }, 73 | { 74 | "args2_6": { 75 | "net": { 76 | "port": 28002 77 | }, 78 | "replication": { 79 | "replSetName": "blue" 80 | }, 81 | "storage": { 82 | "dbPath": "/tmp/data/blue_2" 83 | }, 84 | "systemLog": { 85 | "destination": "file", 86 | "path": "/tmp/data/blue_2/mongodb.log" 87 | } 88 | }, 89 | "hostname": "MACHINE_HOSTNAME", 90 | "logRotate": { 91 | "sizeThresholdMB": 1000, 92 | "timeThresholdHrs": 24 93 | }, 94 | "name": "blue_2", 95 | "processType": "mongod", 96 | "version": "3.2.5", 97 | "authSchemaVersion": 3 98 | }, 99 | { 100 | "args2_6": { 101 | "net": { 102 | "port": 28017 103 | }, 104 | "systemLog": { 105 | "destination": "file", 106 | "path": "/tmp/data/blue_mongos_3/mongodb.log" 107 | } 108 | }, 109 | "cluster": "blueCluster", 110 | "hostname": "MACHINE_HOSTNAME", 111 | "logRotate": { 112 | "sizeThresholdMB": 1000, 113 | "timeThresholdHrs": 24 114 | }, 115 | "name": "blue_mongos_3", 116 | "processType": "mongos", 117 | "version": "3.2.5", 118 | "authSchemaVersion": 3 119 | }, 120 | { 121 | "args2_6": { 122 | "net": { 123 | "port": 28019 124 | }, 125 | "replication": { 126 | "replSetName": "blue_config_rs" 127 | }, 128 | "sharding": { 129 | "clusterRole": "configsvr" 130 | }, 131 | "storage": { 132 | "dbPath": "/tmp/data/blue_config_4" 133 | }, 134 | "systemLog": { 135 | "destination": "file", 136 | "path": "/tmp/data/blue_config_4/mongodb.log" 137 | } 138 | }, 139 | "hostname": "MACHINE_HOSTNAME", 140 | "alias": "MACHINE_HOSTNAME", 141 | "logRotate": { 142 | "sizeThresholdMB": 1000, 143 | "timeThresholdHrs": 24 144 | }, 145 | "name": "blue_config_4", 146 | "processType": "mongod", 147 | "version": "3.2.5", 148 | "authSchemaVersion": 3 149 | }, 150 | { 151 | "args2_6": { 152 | "net": { 153 | "port": 28020 154 | }, 155 | "replication": { 156 | "replSetName": "blue_config_rs" 157 | }, 158 | "sharding": { 159 | "clusterRole": "configsvr" 160 | }, 161 | "storage": { 162 | "dbPath": "/tmp/data/blue_config_5" 163 | }, 164 | "systemLog": { 165 | "destination": "file", 166 | "path": "/tmp/data/blue_config_5/mongodb.log" 167 | } 168 | }, 169 | "hostname": "MACHINE_HOSTNAME", 170 | "alias": "MACHINE_HOSTNAME", 171 | "logRotate": { 172 | "sizeThresholdMB": 1000, 173 | "timeThresholdHrs": 24 174 | }, 175 | "name": "blue_config_5", 176 | "processType": "mongod", 177 | "version": "3.2.5", 178 | "authSchemaVersion": 3 179 | }, 180 | { 181 | "args2_6": { 182 | "net": { 183 | "port": 28021 184 | }, 185 | "replication": { 186 | "replSetName": "blue_config_rs" 187 | }, 188 | "sharding": { 189 | "clusterRole": "configsvr" 190 | }, 191 | "storage": { 192 | "dbPath": "/tmp/data/blue_config_6" 193 | }, 194 | "systemLog": { 195 | "destination": "file", 196 | "path": "/tmp/data/blue_config_6/mongodb.log" 197 | } 198 | }, 199 | "hostname": "MACHINE_HOSTNAME", 200 | "alias": "MACHINE_HOSTNAME", 201 | "logRotate": { 202 | "sizeThresholdMB": 1000, 203 | "timeThresholdHrs": 24 204 | }, 205 | "name": "blue_config_6", 206 | "processType": "mongod", 207 | "version": "3.2.5", 208 | "authSchemaVersion": 3 209 | } 210 | ], 211 | "replicaSets": [ 212 | { 213 | "_id": "blue", 214 | "members": [ 215 | { 216 | "_id": 0, 217 | "host": "blue_0" 218 | }, 219 | { 220 | "_id": 1, 221 | "host": "blue_1" 222 | }, 223 | { 224 | "_id": 2, 225 | "arbiterOnly": true, 226 | "host": "blue_2", 227 | "priority": 0 228 | } 229 | ] 230 | }, 231 | { 232 | "_id": "blue_config_rs", 233 | "members": [ 234 | { 235 | "_id": 0, 236 | "host": "blue_config_4" 237 | }, 238 | { 239 | "_id": 1, 240 | "host": "blue_config_5" 241 | }, 242 | { 243 | "_id": 2, 244 | "host": "blue_config_6" 245 | } 246 | ] 247 | } 248 | ], 249 | "sharding": [ 250 | { 251 | "configServerReplica": "blue_config_rs", 252 | "name": "blueCluster", 253 | "shards": [ 254 | { 255 | "_id": "blue", 256 | "rs": "blue" 257 | } 258 | ] 259 | } 260 | ] 261 | } 262 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_x509_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "2.6.9-ent" 9 | } 10 | ], 11 | "backupVersions": [ 12 | { 13 | "hostname": "MACHINE_HOSTNAME" 14 | } 15 | ], 16 | 17 | "monitoringVersions": [ 18 | { 19 | "hostname": "MACHINE_HOSTNAME" 20 | } 21 | ], 22 | "processes": [ 23 | { 24 | "args2_6": { 25 | "net": { 26 | "port": 28000, 27 | "ssl": { 28 | "mode": "requireSSL", 29 | "PEMKeyFile": "/opt/mongodb-1.pem" 30 | } 31 | }, 32 | "replication": { 33 | "replSetName": "blue" 34 | }, 35 | "storage": { 36 | "dbPath": "/data/blue_0" 37 | }, 38 | "systemLog": { 39 | "destination": "file", 40 | "path": "/data/blue_0/mongodb.log" 41 | } 42 | }, 43 | "hostname": "MACHINE_HOSTNAME", 44 | "logRotate": { 45 | "sizeThresholdMB": 1000, 46 | "timeThresholdHrs": 24 47 | }, 48 | "name": "blue_0", 49 | "processType": "mongod", 50 | "version": "2.6.9-ent", 51 | "authSchemaVersion": 3 52 | }, 53 | { 54 | "args2_6": { 55 | "net": { 56 | "port": 28001, 57 | "ssl": { 58 | "mode": "requireSSL", 59 | "PEMKeyFile": "/opt/mongodb-1.pem" 60 | } 61 | }, 62 | "replication": { 63 | "replSetName": "blue" 64 | }, 65 | "storage": { 66 | "dbPath": "/data/blue_1" 67 | }, 68 | "systemLog": { 69 | "destination": "file", 70 | "path": "/data/blue_1/mongodb.log" 71 | } 72 | }, 73 | "hostname": "MACHINE_HOSTNAME", 74 | "logRotate": { 75 | "sizeThresholdMB": 1000, 76 | "timeThresholdHrs": 24 77 | }, 78 | "name": "blue_1", 79 | "processType": "mongod", 80 | "version": "2.6.9-ent", 81 | "authSchemaVersion": 3 82 | }, 83 | { 84 | "args2_6": { 85 | "net": { 86 | "port": 28002, 87 | "ssl": { 88 | "mode": "requireSSL", 89 | "PEMKeyFile": "/opt/mongodb-1.pem" 90 | } 91 | }, 92 | "replication": { 93 | "replSetName": "blue" 94 | }, 95 | "storage": { 96 | "dbPath": "/data/blue_2" 97 | }, 98 | "systemLog": { 99 | "destination": "file", 100 | "path": "/data/blue_2/mongodb.log" 101 | } 102 | }, 103 | "hostname": "MACHINE_HOSTNAME", 104 | "logRotate": { 105 | "sizeThresholdMB": 1000, 106 | "timeThresholdHrs": 24 107 | }, 108 | "name": "blue_2", 109 | "processType": "mongod", 110 | "version": "2.6.9-ent", 111 | "authSchemaVersion": 3 112 | }, 113 | { 114 | "args2_6": { 115 | "net": { 116 | "port": 28017, 117 | "ssl": { 118 | "mode": "requireSSL", 119 | "PEMKeyFile": "/opt/mongodb-1.pem" 120 | } 121 | }, 122 | "systemLog": { 123 | "destination": "file", 124 | "path": "/data/blue_mongos_3/mongodb.log" 125 | } 126 | }, 127 | "cluster": "blueCluster", 128 | "hostname": "MACHINE_HOSTNAME", 129 | "logRotate": { 130 | "sizeThresholdMB": 1000, 131 | "timeThresholdHrs": 24 132 | }, 133 | "name": "blue_mongos_3", 134 | "processType": "mongos", 135 | "version": "2.6.9-ent", 136 | "authSchemaVersion": 3 137 | }, 138 | { 139 | "args2_6": { 140 | "net": { 141 | "port": 28019, 142 | "ssl": { 143 | "mode": "requireSSL", 144 | "PEMKeyFile": "/opt/mongodb-1.pem" 145 | } 146 | }, 147 | "sharding": { 148 | "clusterRole": "configsvr" 149 | }, 150 | "storage": { 151 | "dbPath": "/data/blue_config_4" 152 | }, 153 | "systemLog": { 154 | "destination": "file", 155 | "path": "/data/blue_config_4/mongodb.log" 156 | } 157 | }, 158 | "hostname": "MACHINE_HOSTNAME", 159 | "alias": "MACHINE_HOSTNAME", 160 | "logRotate": { 161 | "sizeThresholdMB": 1000, 162 | "timeThresholdHrs": 24 163 | }, 164 | "name": "blue_config_4", 165 | "processType": "mongod", 166 | "version": "2.6.9-ent", 167 | "authSchemaVersion": 3 168 | } 169 | ], 170 | "replicaSets": [ 171 | { 172 | "_id": "blue", 173 | "members": [ 174 | { 175 | "_id": 0, 176 | "host": "blue_0", 177 | "priority": 5 178 | }, 179 | { 180 | "_id": 1, 181 | "host": "blue_1" 182 | }, 183 | { 184 | "_id": 2, 185 | "arbiterOnly": true, 186 | "host": "blue_2", 187 | "priority": 0 188 | } 189 | ] 190 | } 191 | ], 192 | "roles": [], 193 | "sharding": [ 194 | { 195 | "configServer": [ 196 | "blue_config_4" 197 | ], 198 | "name": "blueCluster", 199 | "shards": [ 200 | { 201 | "_id": "blue", 202 | "rs": "blue" 203 | } 204 | ] 205 | } 206 | ], 207 | "auth": { 208 | "disabled": false, 209 | "autoAuthMechanism": "MONGODB-X509", 210 | "autoUser": "CN=automation-agent,O=MMS", 211 | "deploymentAuthMechanisms": ["MONGODB-X509"], 212 | "key": "someverylongandrandomcontents", 213 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 214 | "usersWanted": [ 215 | { 216 | "db": "$external", 217 | "roles": [ 218 | { 219 | "db": "admin", 220 | "role": "clusterMonitor" 221 | } 222 | ], 223 | "user": "CN=monitoring-agent,O=MMS" 224 | }, 225 | { 226 | "db": "$external", 227 | "roles": [ 228 | { 229 | "db": "admin", 230 | "role": "clusterAdmin" 231 | }, 232 | { 233 | "db": "admin", 234 | "role": "readAnyDatabase" 235 | }, 236 | { 237 | "db": "admin", 238 | "role": "userAdminAnyDatabase" 239 | }, 240 | { 241 | "db": "local", 242 | "role": "readWrite" 243 | }, 244 | { 245 | "db": "admin", 246 | "role": "readWrite" 247 | } 248 | ], 249 | "user": "CN=backup-agent,O=MMS" 250 | } 251 | ], 252 | "usersDeleted": [] 253 | }, 254 | "ssl": { 255 | "CAFilePath": "/opt/ca.pem", 256 | "clientCertificateMode": "REQUIRE", 257 | "autoPEMKeyFilePath": "/opt/automation-agent.pem" 258 | 259 | } 260 | } 261 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/security_kerberos_cluster.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "2.6.9-ent" 9 | } 10 | ], 11 | "backupVersions": [ 12 | { 13 | "hostname": "MACHINE_HOSTNAME" 14 | } 15 | ], 16 | 17 | "monitoringVersions": [ 18 | { 19 | "hostname": "MACHINE_HOSTNAME" 20 | } 21 | ], 22 | "processes": [ 23 | { 24 | "args2_6": { 25 | "net": { 26 | "port": 28000 27 | }, 28 | "replication": { 29 | "replSetName": "blue" 30 | }, 31 | "storage": { 32 | "dbPath": "/data/blue_0" 33 | }, 34 | "systemLog": { 35 | "destination": "file", 36 | "path": "/data/blue_0/mongodb.log" 37 | } 38 | }, 39 | "hostname": "MACHINE_HOSTNAME", 40 | "logRotate": { 41 | "sizeThresholdMB": 1000, 42 | "timeThresholdHrs": 24 43 | }, 44 | "name": "blue_0", 45 | "processType": "mongod", 46 | "version": "2.6.9-ent", 47 | "authSchemaVersion": 3, 48 | "kerberos": { 49 | "keytab": "/opt/mongodb-1.keytab" 50 | } 51 | }, 52 | { 53 | "args2_6": { 54 | "net": { 55 | "port": 28001 56 | }, 57 | "replication": { 58 | "replSetName": "blue" 59 | }, 60 | "storage": { 61 | "dbPath": "/data/blue_1" 62 | }, 63 | "systemLog": { 64 | "destination": "file", 65 | "path": "/data/blue_1/mongodb.log" 66 | } 67 | }, 68 | "hostname": "MACHINE_HOSTNAME", 69 | "logRotate": { 70 | "sizeThresholdMB": 1000, 71 | "timeThresholdHrs": 24 72 | }, 73 | "name": "blue_1", 74 | "processType": "mongod", 75 | "version": "2.6.9-ent", 76 | "authSchemaVersion": 3, 77 | "kerberos": { 78 | "keytab": "/opt/mongodb-1.keytab" 79 | } 80 | }, 81 | { 82 | "args2_6": { 83 | "net": { 84 | "port": 28002 85 | }, 86 | "replication": { 87 | "replSetName": "blue" 88 | }, 89 | "storage": { 90 | "dbPath": "/data/blue_2" 91 | }, 92 | "systemLog": { 93 | "destination": "file", 94 | "path": "/data/blue_2/mongodb.log" 95 | } 96 | }, 97 | "hostname": "MACHINE_HOSTNAME", 98 | "logRotate": { 99 | "sizeThresholdMB": 1000, 100 | "timeThresholdHrs": 24 101 | }, 102 | "name": "blue_2", 103 | "processType": "mongod", 104 | "version": "2.6.9-ent", 105 | "authSchemaVersion": 3, 106 | "kerberos": { 107 | "keytab": "/opt/mongodb-1.keytab" 108 | } 109 | }, 110 | { 111 | "args2_6": { 112 | "net": { 113 | "port": 28017 114 | }, 115 | "systemLog": { 116 | "destination": "file", 117 | "path": "/data/blue_mongos_3/mongodb.log" 118 | } 119 | }, 120 | "cluster": "blueCluster", 121 | "hostname": "MACHINE_HOSTNAME", 122 | "logRotate": { 123 | "sizeThresholdMB": 1000, 124 | "timeThresholdHrs": 24 125 | }, 126 | "name": "blue_mongos_3", 127 | "processType": "mongos", 128 | "version": "2.6.9-ent", 129 | "authSchemaVersion": 3, 130 | "kerberos": { 131 | "keytab": "/opt/mongodb-1.keytab" 132 | } 133 | }, 134 | { 135 | "args2_6": { 136 | "net": { 137 | "port": 28019 138 | }, 139 | "sharding": { 140 | "clusterRole": "configsvr" 141 | }, 142 | "storage": { 143 | "dbPath": "/data/blue_config_4" 144 | }, 145 | "systemLog": { 146 | "destination": "file", 147 | "path": "/data/blue_config_4/mongodb.log" 148 | } 149 | }, 150 | "hostname": "MACHINE_HOSTNAME", 151 | "alias": "MACHINE_HOSTNAME", 152 | "logRotate": { 153 | "sizeThresholdMB": 1000, 154 | "timeThresholdHrs": 24 155 | }, 156 | "name": "blue_config_4", 157 | "processType": "mongod", 158 | "version": "2.6.9-ent", 159 | "authSchemaVersion": 3, 160 | "kerberos": { 161 | "keytab": "/opt/mongodb-1.keytab" 162 | } 163 | } 164 | ], 165 | "replicaSets": [ 166 | { 167 | "_id": "blue", 168 | "members": [ 169 | { 170 | "_id": 0, 171 | "host": "blue_0" 172 | }, 173 | { 174 | "_id": 1, 175 | "host": "blue_1" 176 | }, 177 | { 178 | "_id": 2, 179 | "arbiterOnly": true, 180 | "host": "blue_2", 181 | "priority": 0 182 | } 183 | ] 184 | } 185 | ], 186 | "roles": [], 187 | "sharding": [ 188 | { 189 | "configServer": [ 190 | "blue_config_4" 191 | ], 192 | "name": "blueCluster", 193 | "shards": [ 194 | { 195 | "_id": "blue", 196 | "rs": "blue" 197 | } 198 | ] 199 | } 200 | ], 201 | "auth": { 202 | "disabled": false, 203 | "autoAuthMechanism": "GSSAPI", 204 | "autoUser": "automation-agent/MACHINE_HOSTNAME@MMS.COM", 205 | "autoKerberosKeytabPath": "/opt/automation-agent.keytab", 206 | "deploymentAuthMechanisms": ["GSSAPI"], 207 | "key": "GrSLAAsHGXmJOrvElJ2AHTGauvH4O0EFT1r8byvb0G9sTU0viVX21PwUMqBjyXB9WrZP9QvEmCQIF1wOqJofyWmx7wWZqpO69dnc9GUWcpGQLr7eVyKTs99WAPXR3kXpF4MVrHdBMEDfRfhytgomgAso96urN6eC8RaUpjX4Bf9HcAEJwfddZshin97XKJDmqCaqAfORNnf1e8hkfTIwYg1tvIpwemmEF4TkmOgK09N5dINyejyWMU8iWG8FqW5MfQ8A2DrtIdyGSKLH05s7H1dXyADjDECaC77QqLXTx7gWyHca3I0K92PuVFoOs5385vzqTYN3kVgFotSdXgoM8Zt5QIoj2lX4PYqm2TWsVp0s15JELikH8bNVIIMGiSSWJEWGU1PVEXD7V7cYepDb88korMjr3wbh6kZ76Q7F2RtfJqkd4hKw7B5OCX04b5eppkjL598iCpSUUx3X9C6fFavWj2DrHsv9DY86iCWBlcG08DRPKs9EPizCW4jNZtJcm3T7WlcI0MZMKOtsKOCWBZA0C9YnttNrp4eTsQ1U43StiIRPqp2K8rrQAu6etURH0RHedazHeeukTWI7iTG1dZpYk9EyittZ72qKXLNLhi5vJ9TlYw8O91vihB1nJwwA3B1WbiYhkqqRzoL0cQpXJMUsUlsoSP6Q70IMU92vEHbUmna5krESPLeJfQBKGQPNVVE63XYBh2TnvFTdi6koitu209wMFUnHZrzWj3UWGqsyTqqHbPl4RhRLFe24seRwV2SbUuLygBIdptKHnA3kutAbHzsWTT8UxOaiQzFV4auxounrgXj7MoMWEVKKS8AHkELPILGqFVFC8BZsfPC0WacSN5Rg5SaCvfs74hcsCQ3ghq9PyxEb2fbHUiaCjnsBcXqzQw9AjZJG4yX0ubEwicP0bKB6y3w4PUQqdouxH5y16OgkUjrZgodJfRLgP9vqGbHNDpj4yBuswluvCFBh38gBoSIQu11qtQmk43n4G8Dskn0DrJ32l2Gz35q5LaKT", 208 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 209 | "usersWanted": [ 210 | { 211 | "db": "$external", 212 | "roles": [ 213 | { 214 | "db": "admin", 215 | "role": "clusterMonitor" 216 | } 217 | ], 218 | "user": "monitoring-agent/MACHINE_HOSTNAME@MMS.COM" 219 | }, 220 | { 221 | "db": "$external", 222 | "roles": [ 223 | { 224 | "db": "admin", 225 | "role": "clusterAdmin" 226 | }, 227 | { 228 | "db": "admin", 229 | "role": "readAnyDatabase" 230 | }, 231 | { 232 | "db": "admin", 233 | "role": "userAdminAnyDatabase" 234 | }, 235 | { 236 | "db": "local", 237 | "role": "readWrite" 238 | }, 239 | { 240 | "db": "admin", 241 | "role": "readWrite" 242 | } 243 | ], 244 | "user": "backup-agent/MACHINE_HOSTNAME@MMS.COM" 245 | } 246 | ], 247 | "usersDeleted": [] 248 | }, 249 | "kerberos": { 250 | "serviceName": "mongodb-1" 251 | } 252 | } 253 | -------------------------------------------------------------------------------- /automation/api_usage_example/configs/api_6_enable_auth.json: -------------------------------------------------------------------------------- 1 | { 2 | "options": { 3 | "downloadBase": "/var/lib/mongodb-mms-automation", 4 | "downloadBaseWindows": "C:\\mongodb-mms-automation" 5 | }, 6 | "mongoDbVersions": [ 7 | { 8 | "name": "3.0.1" 9 | }, 10 | { 11 | "name": "3.0.2" 12 | } 13 | ], 14 | "backupVersions": [ 15 | { 16 | "hostname": "MACHINE_HOSTNAME" 17 | } 18 | ], 19 | 20 | "monitoringVersions": [ 21 | { 22 | "hostname": "MACHINE_HOSTNAME" 23 | } 24 | ], 25 | "processes": [ 26 | { 27 | "args2_6": { 28 | "net": { 29 | "port": 28000, 30 | "maxIncomingConnections": 5000 31 | }, 32 | "replication": { 33 | "replSetName": "blue" 34 | }, 35 | "storage": { 36 | "dbPath": "/data/blue_0" 37 | }, 38 | "systemLog": { 39 | "destination": "file", 40 | "path": "/data/blue_0/mongodb.log" 41 | } 42 | }, 43 | "hostname": "MACHINE_HOSTNAME", 44 | "logRotate": { 45 | "sizeThresholdMB": 1000, 46 | "timeThresholdHrs": 24 47 | }, 48 | "name": "blue_0", 49 | "processType": "mongod", 50 | "version": "3.0.2", 51 | "authSchemaVersion": 3 52 | }, 53 | { 54 | "args2_6": { 55 | "net": { 56 | "port": 28001, 57 | "maxIncomingConnections": 5000 58 | }, 59 | "replication": { 60 | "replSetName": "blue" 61 | }, 62 | "storage": { 63 | "dbPath": "/data/blue_1" 64 | }, 65 | "systemLog": { 66 | "destination": "file", 67 | "path": "/data/blue_1/mongodb.log" 68 | } 69 | }, 70 | "hostname": "MACHINE_HOSTNAME", 71 | "logRotate": { 72 | "sizeThresholdMB": 1000, 73 | "timeThresholdHrs": 24 74 | }, 75 | "name": "blue_1", 76 | "processType": "mongod", 77 | "version": "3.0.2", 78 | "authSchemaVersion": 3 79 | }, 80 | { 81 | "args2_6": { 82 | "net": { 83 | "port": 28002, 84 | "maxIncomingConnections": 5000 85 | }, 86 | "replication": { 87 | "replSetName": "blue" 88 | }, 89 | "storage": { 90 | "dbPath": "/data/blue_2" 91 | }, 92 | "systemLog": { 93 | "destination": "file", 94 | "path": "/data/blue_2/mongodb.log" 95 | } 96 | }, 97 | "hostname": "MACHINE_HOSTNAME", 98 | "logRotate": { 99 | "sizeThresholdMB": 1000, 100 | "timeThresholdHrs": 24 101 | }, 102 | "name": "blue_2", 103 | "processType": "mongod", 104 | "version": "3.0.2", 105 | "authSchemaVersion": 3 106 | }, 107 | { 108 | "args2_6": { 109 | "net": { 110 | "port": 28017, 111 | "maxIncomingConnections": 4000 112 | }, 113 | "systemLog": { 114 | "destination": "file", 115 | "path": "/data/blue_mongos_3/mongodb.log" 116 | } 117 | }, 118 | "cluster": "blueCluster", 119 | "hostname": "MACHINE_HOSTNAME", 120 | "logRotate": { 121 | "sizeThresholdMB": 1000, 122 | "timeThresholdHrs": 24 123 | }, 124 | "name": "blue_mongos_3", 125 | "processType": "mongos", 126 | "version": "3.0.2", 127 | "authSchemaVersion": 3 128 | }, 129 | { 130 | "args2_6": { 131 | "net": { 132 | "port": 28019, 133 | "maxIncomingConnections": 3000 134 | }, 135 | "sharding": { 136 | "clusterRole": "configsvr" 137 | }, 138 | "storage": { 139 | "dbPath": "/data/blue_config_4" 140 | }, 141 | "systemLog": { 142 | "destination": "file", 143 | "path": "/data/blue_config_4/mongodb.log" 144 | } 145 | }, 146 | "hostname": "MACHINE_HOSTNAME", 147 | "alias": "MACHINE_HOSTNAME", 148 | "logRotate": { 149 | "sizeThresholdMB": 1000, 150 | "timeThresholdHrs": 24 151 | }, 152 | "name": "blue_config_4", 153 | "processType": "mongod", 154 | "version": "3.0.2", 155 | "authSchemaVersion": 3 156 | } 157 | ], 158 | "replicaSets": [ 159 | { 160 | "_id": "blue", 161 | "members": [ 162 | { 163 | "_id": 0, 164 | "host": "blue_0" 165 | }, 166 | { 167 | "_id": 1, 168 | "host": "blue_1" 169 | }, 170 | { 171 | "_id": 2, 172 | "arbiterOnly": true, 173 | "host": "blue_2", 174 | "priority": 0 175 | } 176 | ] 177 | } 178 | ], 179 | "sharding": [ 180 | { 181 | "configServer": [ 182 | "blue_config_4" 183 | ], 184 | "name": "blueCluster", 185 | "shards": [ 186 | { 187 | "_id": "blue", 188 | "rs": "blue" 189 | } 190 | ] 191 | } 192 | ], 193 | "roles": [ 194 | { 195 | "db": "fruit", 196 | "role": "fruitInsertOnly", 197 | "privileges": [ 198 | { 199 | "resource": { 200 | "db": "fruit", 201 | "collection": "", 202 | "cluster": null 203 | }, 204 | "actions": [ 205 | "insert" 206 | ] 207 | } 208 | ], 209 | "roles": null 210 | } 211 | ], 212 | "auth": { 213 | "disabled": false, 214 | "autoPwd": "Sy4oBX9ei0amvupBAN8lVQhj", 215 | "autoUser": "mms-automation", 216 | "key": "GrSLAAsHGXmJOrvElJ2AHTGauvH4O0EFT1r8byvb0G9sTU0viVX21PwUMqBjyXB9WrZP9QvEmCQIF1wOqJofyWmx7wWZqpO69dnc9GUWcpGQLr7eVyKTs99WAPXR3kXpF4MVrHdBMEDfRfhytgomgAso96urN6eC8RaUpjX4Bf9HcAEJwfddZshin97XKJDmqCaqAfORNnf1e8hkfTIwYg1tvIpwemmEF4TkmOgK09N5dINyejyWMU8iWG8FqW5MfQ8A2DrtIdyGSKLH05s7H1dXyADjDECaC77QqLXTx7gWyHca3I0K92PuVFoOs5385vzqTYN3kVgFotSdXgoM8Zt5QIoj2lX4PYqm2TWsVp0s15JELikH8bNVIIMGiSSWJEWGU1PVEXD7V7cYepDb88korMjr3wbh6kZ76Q7F2RtfJqkd4hKw7B5OCX04b5eppkjL598iCpSUUx3X9C6fFavWj2DrHsv9DY86iCWBlcG08DRPKs9EPizCW4jNZtJcm3T7WlcI0MZMKOtsKOCWBZA0C9YnttNrp4eTsQ1U43StiIRPqp2K8rrQAu6etURH0RHedazHeeukTWI7iTG1dZpYk9EyittZ72qKXLNLhi5vJ9TlYw8O91vihB1nJwwA3B1WbiYhkqqRzoL0cQpXJMUsUlsoSP6Q70IMU92vEHbUmna5krESPLeJfQBKGQPNVVE63XYBh2TnvFTdi6koitu209wMFUnHZrzWj3UWGqsyTqqHbPl4RhRLFe24seRwV2SbUuLygBIdptKHnA3kutAbHzsWTT8UxOaiQzFV4auxounrgXj7MoMWEVKKS8AHkELPILGqFVFC8BZsfPC0WacSN5Rg5SaCvfs74hcsCQ3ghq9PyxEb2fbHUiaCjnsBcXqzQw9AjZJG4yX0ubEwicP0bKB6y3w4PUQqdouxH5y16OgkUjrZgodJfRLgP9vqGbHNDpj4yBuswluvCFBh38gBoSIQu11qtQmk43n4G8Dskn0DrJ32l2Gz35q5LaKT", 217 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 218 | "usersWanted": [ 219 | { 220 | "db": "fruit", 221 | "user": "fruitInsertOnlyUser", 222 | "initPwd": "6782fa9dfefc4da96ddc2e6099f79bbf", 223 | "roles": [ 224 | { 225 | "db": "fruit", 226 | "role": "fruitInsertOnly" 227 | } 228 | ] 229 | }, 230 | { 231 | "db": "admin", 232 | "initPwd": "6782fa9dfefc4da96ddc2e6099f79bbf", 233 | "roles": [ 234 | { 235 | "db": "admin", 236 | "role": "clusterMonitor" 237 | } 238 | ], 239 | "user": "mms-monitoring-agent" 240 | }, 241 | { 242 | "db": "admin", 243 | "initPwd": "d378693eb4888a79d1864f4affb0a739", 244 | "roles": [ 245 | { 246 | "db": "admin", 247 | "role": "clusterAdmin" 248 | }, 249 | { 250 | "db": "admin", 251 | "role": "readAnyDatabase" 252 | }, 253 | { 254 | "db": "admin", 255 | "role": "userAdminAnyDatabase" 256 | }, 257 | { 258 | "db": "local", 259 | "role": "readWrite" 260 | }, 261 | { 262 | "db": "admin", 263 | "role": "readWrite" 264 | } 265 | ], 266 | "user": "mms-backup-agent" 267 | } 268 | ], 269 | "usersDeleted": [] 270 | } 271 | } 272 | -------------------------------------------------------------------------------- /automation/README.md: -------------------------------------------------------------------------------- 1 | # Automation API Endpoints 2 | 3 | ## Automation Config 4 | 5 | ## Critical Concept: Versioning 6 | 7 | The Automation Config for a Group is internally versioned. Each time a new Automation Config is submitted, 8 | the version is incremented. The current version can be seen in the `version` field. 9 | 10 | There is *no protection* in the Automation API to prevent concurrent modifications. Each time a new configuration 11 | is submitted via the API, the current state of the system is adjusted to match the config. If two administrators of 12 | the system both start with a configuration based on version 9, make their own modifications, and then submit their 13 | modifications, the later modification will win. 14 | 15 | The Automation API is very powerful, and should be used with great caution. 16 | 17 | ### Operations 18 | 19 | * GET /api/public/v1.0/groups/GROUP-ID/automationConfig - Get the Automation configuration for a Group 20 | * PUT /api/public/v1.0/groups/GROUP-ID/automationConfig - Put the Automation configuration for a Group 21 | 22 | ### GETting the Automation Config 23 | 24 | The document return follows the format of the [Cluster Configuration](https://docs.mms.mongodb.com/reference/cluster-configuration/) 25 | 26 | ### PUTting the Automation Config 27 | 28 | #### Example Usage 29 | 30 | curl -u "MMS_USER_NAME:MMS_API_KEY" -H "Content-Type: application/json" "https://mms.mongod.com/api/public/v1.0/groups/GROUP_ID/automationConfig" --digest -X PUT -d @/PATH/TO/CONFIG/config.json -i 31 | 32 | The document accepted by the PUT requires a subset of the fields return by the GET and explained in the 33 | [Cluster Configuration](https://docs.mms.mongodb.com/reference/cluster-configuration/). The details of 34 | some areas of the configuration is managed internally by MMS, and therefore less detail is required in the PUT 35 | than is returned in the GET. 36 | 37 | This is best explained by example - building up an Automation Config step by step. 38 | 39 | #### Setting the Download Base 40 | 41 | The central job of MMS Automation is to install, upgrade and maintain MongoDB processes. In order to do so, it will 42 | download specified MongoDB binaries and save these to the local disk. The `options.downloadBase` property defines 43 | the location on disk where the Automation Agent will store these MongoDB binaries. 44 | 45 | Please note that this location must pre-exist on *every* server, and must be writeable by the Automation Agent system 46 | user. If you installed the Automation Agents using our `RPM` or `DEB` packages, the directory `/var/lib/mongodb-mms-automation` 47 | will have been pre-prepared for this purpose. 48 | 49 | An Automation Config with only the download base specified looks like this: 50 | 51 | { 52 | "options": { 53 | "downloadBase": "/var/lib/mongodb-mms-automation" 54 | }, 55 | "mongoDbVersions": [], 56 | "monitoringVersions": [], 57 | "backupVersions": [], 58 | "processes": [], 59 | "replicaSets": [], 60 | "sharding": [] 61 | } 62 | 63 | This example can also be viewed [here](https://github.com/10gen-labs/mms-api-examples/blob/master/automation/api_usage_example/configs/api_0_clean.json) 64 | 65 | #### Defining MongoDB Versions 66 | 67 | Next, specify the MongoDB versions that you wish to use in your deployment. A simple example specifying the use of 68 | MongoDB versions 3.0.1 and 3.0.2 looks like this: 69 | 70 | { 71 | "options": { 72 | "downloadBase": "/var/lib/mongodb-mms-automation" 73 | }, 74 | "mongoDbVersions": [ 75 | {"name": "3.0.1"}, 76 | {"name": "3.0.2"} 77 | ], 78 | "monitoringVersions": [], 79 | "backupVersions": [], 80 | "processes": [], 81 | "replicaSets": [], 82 | "sharding": [] 83 | } 84 | 85 | With this configuration, the Automation Agent will download these MongoDB binaries from http://mongodb.org/downloads. 86 | This example can also be viewed [here](https://github.com/10gen-labs/mms-api-examples/blob/master/automation/api_usage_example/configs/api_1_define_versions.json) 87 | 88 | If you have a license for the Enterprise versions of MongoDB, these may be specified as: 89 | 90 | { 91 | "options": { 92 | "downloadBase": "/var/lib/mongodb-mms-automation" 93 | }, 94 | "mongoDbVersions": [ 95 | {"name": "3.0.1-ent"}, 96 | {"name": "3.0.2-ent"} 97 | ], 98 | "monitoringVersions": [], 99 | "backupVersions": [], 100 | "processes": [], 101 | "replicaSets": [], 102 | "sharding": [] 103 | } 104 | 105 | If you wish to define a custom download location, or use custom versions, please see our Appendix. 106 | 107 | #### (Optional) Installing Monitoring and Backup Agents 108 | 109 | In order to have a fully functional MMS Group, you must also have one Monitoring Agent. If you are using MMS Backup 110 | you also need a Backup Agent. You may optionally configure your Automation Agents to manage your Monitoring and 111 | Backup Agents. 112 | 113 | Suppose we are configuring a single-server MongoDB deployment, on which we want to run a MongoDB replica set, a 114 | Monitoring Agent and Backup Agent. Adding the Monitoring and Backup Agents to our configuration looks like this: 115 | 116 | { 117 | "options": { 118 | "downloadBase": "/var/lib/mongodb-mms-automation" 119 | }, 120 | "mongoDbVersions": [ 121 | {"name": "3.0.1"}, 122 | {"name": "3.0.2"} 123 | ], 124 | "monitoringVersions": [ 125 | { 126 | "hostname": "AGENT_HOSTNAME", 127 | "logPath": "/var/log/mongodb-mms-automation/monitoring-agent.log", 128 | "logRotate": { 129 | "sizeThresholdMB": 1000, 130 | "timeThresholdHrs": 24 131 | } 132 | } 133 | ], 134 | "backupVersions": [ 135 | { 136 | "hostname": "AGENT_HOSTNAME", 137 | "logPath": "/var/log/mongodb-mms-automation/backup-agent.log", 138 | "logRotate": { 139 | "sizeThresholdMB": 1000, 140 | "timeThresholdHrs": 24 141 | } 142 | } 143 | ], 144 | "processes": [], 145 | "replicaSets": [], 146 | "sharding": [] 147 | } 148 | 149 | 150 | where `AGENT_HOSTNAME` is the full-qualified domain name (FQDN) of the server. This can be determined by running 151 | `hostname -f` on the server. It is essential that you specify the FQDN (`hostname -f`) and not the unqualified 152 | hostname (`hostname`). 153 | 154 | #### Building A MongoDB Deployment 155 | 156 | Now that we've set up our environment, we're ready to specify our MongoDB deployment. To create a 3-member replica 157 | set with two data nodes, and one arbiter we start filling in the [processes](https://docs.mms.mongodb.com/reference/cluster-configuration/#processes) 158 | and [replicaSets](https://docs.mms.mongodb.com/reference/cluster-configuration/#replicasets) fields. 159 | 160 | On our simple single-server deployment, these fields might look like: 161 | 162 | "processes": [ 163 | { 164 | "args2_6": { 165 | "net": { 166 | "port": 28000 167 | }, 168 | "replication": { 169 | "replSetName": "blue" 170 | }, 171 | "storage": { 172 | "dbPath": "/data/blue_0" 173 | }, 174 | "systemLog": { 175 | "destination": "file", 176 | "path": "/data/blue_0/mongodb.log" 177 | } 178 | }, 179 | "hostname": "AGENT_HOSTNAME", 180 | "logRotate": { 181 | "sizeThresholdMB": 1000, 182 | "timeThresholdHrs": 24 183 | }, 184 | "name": "blue_0", 185 | "processType": "mongod", 186 | "version": "3.0.1", 187 | "authSchemaVersion": 3 188 | }, 189 | { 190 | "args2_6": { 191 | "net": { 192 | "port": 28001 193 | }, 194 | "replication": { 195 | "replSetName": "blue" 196 | }, 197 | "storage": { 198 | "dbPath": "/data/blue_1" 199 | }, 200 | "systemLog": { 201 | "destination": "file", 202 | "path": "/data/blue_1/mongodb.log" 203 | } 204 | }, 205 | "hostname": "AGENT_HOSTNAME", 206 | "logRotate": { 207 | "sizeThresholdMB": 1000, 208 | "timeThresholdHrs": 24 209 | }, 210 | "name": "blue_1", 211 | "processType": "mongod", 212 | "version": "3.0.1", 213 | "authSchemaVersion": 3 214 | }, 215 | { 216 | "args2_6": { 217 | "net": { 218 | "port": 28002 219 | }, 220 | "replication": { 221 | "replSetName": "blue" 222 | }, 223 | "storage": { 224 | "dbPath": "/data/blue_2" 225 | }, 226 | "systemLog": { 227 | "destination": "file", 228 | "path": "/data/blue_2/mongodb.log" 229 | } 230 | }, 231 | "hostname": "AGENT_HOSTNAME", 232 | "logRotate": { 233 | "sizeThresholdMB": 1000, 234 | "timeThresholdHrs": 24 235 | }, 236 | "name": "blue_2", 237 | "processType": "mongod", 238 | "version": "3.0.1", 239 | "authSchemaVersion": 3 240 | } 241 | ], 242 | "replicaSets": [ 243 | { 244 | "_id": "blue", 245 | "members": [ 246 | { 247 | "_id": 0, 248 | "host": "blue_0" 249 | }, 250 | { 251 | "_id": 1, 252 | "host": "blue_1" 253 | }, 254 | { 255 | "_id": 2, 256 | "arbiterOnly": true, 257 | "host": "blue_2", 258 | "priority": 0 259 | } 260 | ] 261 | } 262 | ], 263 | 264 | Note that every processes must have a unique (but arbitrary `name`) and that the `replicaSets` configuration contains 265 | references to the process `name`s. A full configuration example can be found 266 | [here](https://github.com/10gen-labs/mms-api-examples/blob/master/automation/api_usage_example/configs/api_3_create_replica_set.json). 267 | 268 | #### Removing ("Unmanaging") A Process or Processes from the Automation Configuration 269 | 270 | If a process (or multiple processes) are moved from the Automation Configuration, the Automation Agents will no 271 | longer manage these processes. This means that the Automation Agents will simply ignore the existence of these 272 | processes. Removing a process from the Automation Configuration does *not* shut down the process or delete 273 | its data or log directories. 274 | 275 | It is critical to keep this in mind if you are going to deploy new processes on the same server. If you remove 276 | a process on port 27018 from your Automation Configuration and then add a new process, also on port 27018, the new 277 | process will fail to reach goal state because the port will already be in use by the previously unmanaged process. 278 | 279 | #### Configuring MONGODB-CR Authentication 280 | 281 | ##### Critical Concepts 282 | 283 | 1. All MongoDB processes belonging to a single Automation Config (and therefore a single MMS Group) share a *single* 284 | auth configuration. This means that there is a single, shared keyFile, and a single set of users and roles. 285 | 2. Configuring a username and password for the Automation Agent itself is required. 286 | 3. It is also required that an appropriate user be defined for both the Monitoring and Backup Agents - even if the 287 | Monitoring and Backup Agents themselves are not managed by Automation. The username for the Monitoring Agent must 288 | be `mms-monitoring-agent` and the username for the Backup Agent must be `mms-backup-agent`. 289 | 4. In oder to enable authentication, the `auth.disabled` parameter must be set to `false. 290 | 5. Additional users may be defined manually. MMS Automation will ignore additional users, unless they are included 291 | in the `auth.usersDeleted` field, in which case it will remove them. 292 | 293 | ##### Example and Details 294 | 295 | An example [auth](https://docs.mms.mongodb.com/reference/cluster-configuration/#auth) configuration 296 | might look like this: 297 | 298 | "auth": { 299 | "disabled": false, 300 | "autoPwd": "Sy4oBX9ei0amvupBAN8lVQhj", 301 | "autoUser": "mms-automation", 302 | "key": "GrSLAAsHGXmJOrvElJ2AHTGauvH4O0EFT1r8byvb0G9sTU0viVX21PwUMqBjyXB9WrZP9QvEmCQIF1wOqJofyWmx7wWZqpO69dnc9GUWcpGQLr7eVyKTs99WAPXR3kXpF4MVrHdBMEDfRfhytgomgAso96urN6eC8RaUpjX4Bf9HcAEJwfddZshin97XKJDmqCaqAfORNnf1e8hkfTIwYg1tvIpwemmEF4TkmOgK09N5dINyejyWMU8iWG8FqW5MfQ8A2DrtIdyGSKLH05s7H1dXyADjDECaC77QqLXTx7gWyHca3I0K92PuVFoOs5385vzqTYN3kVgFotSdXgoM8Zt5QIoj2lX4PYqm2TWsVp0s15JELikH8bNVIIMGiSSWJEWGU1PVEXD7V7cYepDb88korMjr3wbh6kZ76Q7F2RtfJqkd4hKw7B5OCX04b5eppkjL598iCpSUUx3X9C6fFavWj2DrHsv9DY86iCWBlcG08DRPKs9EPizCW4jNZtJcm3T7WlcI0MZMKOtsKOCWBZA0C9YnttNrp4eTsQ1U43StiIRPqp2K8rrQAu6etURH0RHedazHeeukTWI7iTG1dZpYk9EyittZ72qKXLNLhi5vJ9TlYw8O91vihB1nJwwA3B1WbiYhkqqRzoL0cQpXJMUsUlsoSP6Q70IMU92vEHbUmna5krESPLeJfQBKGQPNVVE63XYBh2TnvFTdi6koitu209wMFUnHZrzWj3UWGqsyTqqHbPl4RhRLFe24seRwV2SbUuLygBIdptKHnA3kutAbHzsWTT8UxOaiQzFV4auxounrgXj7MoMWEVKKS8AHkELPILGqFVFC8BZsfPC0WacSN5Rg5SaCvfs74hcsCQ3ghq9PyxEb2fbHUiaCjnsBcXqzQw9AjZJG4yX0ubEwicP0bKB6y3w4PUQqdouxH5y16OgkUjrZgodJfRLgP9vqGbHNDpj4yBuswluvCFBh38gBoSIQu11qtQmk43n4G8Dskn0DrJ32l2Gz35q5LaKT", 303 | "keyfile": "/var/lib/mongodb-mms-automation/keyfile", 304 | "usersWanted": [ 305 | { 306 | "db": "fruit", 307 | "user": "fruitReadOnly", 308 | "initPwd": "6782fa9dfefc4da96ddc2e6099f79bbf", 309 | "roles": [ 310 | { 311 | "db": "fruit", 312 | "role": "read" 313 | } 314 | ] 315 | }, 316 | { 317 | "db": "admin", 318 | "initPwd": "6782fa9dfefc4da96ddc2e6099f79bbf", 319 | "roles": [ 320 | { 321 | "db": "admin", 322 | "role": "clusterMonitor" 323 | } 324 | ], 325 | "user": "mms-monitoring-agent" 326 | }, 327 | { 328 | "db": "admin", 329 | "initPwd": "d378693eb4888a79d1864f4affb0a739", 330 | "roles": [ 331 | { 332 | "db": "admin", 333 | "role": "clusterAdmin" 334 | }, 335 | { 336 | "db": "admin", 337 | "role": "readAnyDatabase" 338 | }, 339 | { 340 | "db": "admin", 341 | "role": "userAdminAnyDatabase" 342 | }, 343 | { 344 | "db": "local", 345 | "role": "readWrite" 346 | }, 347 | { 348 | "db": "admin", 349 | "role": "readWrite" 350 | } 351 | ], 352 | "user": "mms-backup-agent" 353 | } 354 | ], 355 | "usersDeleted": [] 356 | 357 | Note that the `initPwd` is the clear-text (unhashed) password for the specified user. 358 | 359 | #### Roles 360 | 361 | MongoDB custom [roles](https://docs.mms.mongodb.com/reference/cluster-configuration/#roles) may also be defined. 362 | 363 | "roles": [ 364 | { 365 | "db": "fruit", 366 | "role": "fruitInsertOnly", 367 | "privileges": [ 368 | { 369 | "resource": { 370 | "db": "fruit", 371 | "collection": "", 372 | "cluster": null 373 | }, 374 | "actions": [ 375 | "insert" 376 | ] 377 | } 378 | ], 379 | "roles": null 380 | } 381 | ], 382 | 383 | A full configuration with authentication is shown [here](https://github.com/10gen-labs/mms-api-examples/blob/master/automation/api_usage_example/configs/api_6_enable_auth.json) 384 | 385 | ## Automation Status 386 | 387 | ### Operations 388 | 389 | * GET /api/public/v1.0/groups/GROUP-ID/automationStatus - Get the Automation status for a Group 390 | 391 | #### GETting the Automation Status 392 | 393 | This endpoint returns the status of every MongoDB process in the deployment, as viewed by the Automation Agent. 394 | 395 | "goalVersion": 29, 396 | "processes": [ 397 | { 398 | "hostname": "AGENT_HOSTNAME", 399 | "name": "blue_0", 400 | "lastGoalAchieved": 28, 401 | "plan": ["Download", "Start", "WaitRsInit"] 402 | }, 403 | { 404 | "hostname": "AGENT_HOSTNAME", 405 | "name": "blue_1", 406 | "lastGoalAchieved": 29, 407 | "plan": [] 408 | } 409 | ] 410 | 411 | The `goalVersion` field corresponds to the `version` of the most recently submitted Automation Config. 412 | 413 | For each process, the `hostname` indicates the full-qualified hostname (`hostname -f`) of the server on 414 | which the Automation Agent and process are hosted. The `name` maps to the process `name` as specified in the 415 | Automation Config. `lastGoalAchieved` indicates the last version of the Automation Config with which this process 416 | was fully in spec. If `lastGoalAchieved` is not equal to `goalVersion` than there will also be a `plan` on how this 417 | process will move from its current state to the new goal state. 418 | 419 | ## Example Code 420 | 421 | A full Python example can be found [here](https://github.com/10gen-labs/mms-api-examples/blob/master/automation/api_usage_example/test_automation_api.py). 422 | This example gradually builds up a MongoDB deployment, going through the following phases: 423 | 424 | 1. Setting up the download base. 425 | 2. Defining MongoDB versions. 426 | 3. Defining a configuration for a Monitoring and Backup Agent. 427 | 4. Deploying a 3-member replica set on MongoDB 3.0.1. 428 | 5. Upgrading this replica set to MongoDB 3.0.2. 429 | 6. Converting this replica set to a single shard cluster. 430 | 7. Enabling authentication 431 | 432 | The series of Automation Configs that are deployed can be found [here](https://github.com/10gen-labs/mms-api-examples/tree/master/automation/api_usage_example/configs). 433 | 434 | This example requires installing the [requests](http://www.python-requests.org/) module. 435 | 436 | To run the code: 437 | 438 | python test_automation_api.py MMS_BASE_URL AGENT_HOSTNAME GROUP_ID MMS_USER_ID MMS_USER_API_KEY 439 | 440 | * MMS_BASE_URL is the base URL for your MMS deployment. For example, for MMS Cloud this would be https://mms.mongodb.com 441 | * AGENT_HOSTNAME is the fully-qualified domain name (`hostname -f`) for the server on which you have installed an Automation Agent. 442 | * GROUP_ID is the MMS Group ID 443 | * MMS_USER_ID and MMS_USER_API_KEY are the credentials for the MMS API. See [here](https://docs.mms.mongodb.com/tutorial/use-mms-public-api/) for how to set these up. 444 | 445 | For example: 446 | 447 | python test_automation_api.py https://mms.mongodb.com mongo-1.example.com 54b5e4df9436322466a89a3e fred@example.com 063fc60e-c5eb-4426-85c1-1e650d6228c6 448 | 449 | --------------------------------------------------------------------------------