├── LICENSE ├── README.md ├── config.json ├── gitlab-ldap-sync.py └── requirements.txt /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Jeff MrBear 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # gitlab-ldap-sync 3 | 4 | Python project to sync LDAP/Active Directory Groups into GitLab. 5 | 6 | The script will create the missing LDAP groups into gitlab and sync membership of all LDAP groups. 7 | 8 | ## Getting Started 9 | 10 | These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system. 11 | 12 | ### Prerequisites 13 | 14 | This project has been tested on CentOS 7.6 with GitLab 11.5.* and OpenLDAP and Active Directory. 15 | 16 | ``` 17 | Python : 3.4.9 18 | pip3 : 8.1.2 19 | python-gitlab : 1.6.0 20 | python-ldap : 3.4.0 21 | ``` 22 | 23 | ### Installing 24 | 25 | You could either install requirements system wide or use virtual environment / conda, choose your poison. 26 | 27 | To get this up and running you just need to do the following : 28 | 29 | * Clone the repo 30 | ```bash 31 | git clone https://github.com/MrBE4R/gitlab-ldap-sync.git 32 | ``` 33 | * Install requirements 34 | ```bash 35 | pip3 install -r ./gitlab-ldap-sync/requirements.txt 36 | ``` 37 | * Edit config.json with you values 38 | ```bash 39 | EDITOR ./gitlab-ldap-sync/config.json 40 | ``` 41 | * Start the script and enjoy your sync users and groups being synced 42 | ```bash 43 | cd ./gitlab-ldap-sync && ./gitlab-ldap-sync.py 44 | ``` 45 | 46 | You should get something like this : 47 | ```bash 48 | Initializing gitlab-ldap-sync. 49 | Done. 50 | Updating logger configuration 51 | Done. 52 | Connecting to GitLab 53 | Done. 54 | Connecting to LDAP 55 | Done. 56 | Getting all groups from GitLab. 57 | Done. 58 | Getting all groups from LDAP. 59 | Done. 60 | Groups currently in GitLab : < G1 >, < G2 >, < G3 >, < G4 >, < G5 >, < P1 >, < P2 >, < P3 > 61 | Groups currently in LDAP : < G1 >, < G2 >, < G3 >, < G4 >, < G5 >, < G6 >, < G7 > 62 | Syncing Groups from LDAP. 63 | Working on group ... 64 | |- Group already exist in GitLab, skiping creation. 65 | |- Working on group's members. 66 | | |- User already in gitlab group, skipping. 67 | | |- User already in gitlab group, skipping. 68 | [...] 69 | |- Done. 70 | [...] 71 | Done 72 | ``` 73 | 74 | You could add the script in a cron to run it periodically. 75 | ## Deployment 76 | 77 | How to configure config.json 78 | ```json5 79 | { 80 | "log": "/tmp/gitlab-ldap-sync.log", // Where to store the log file. If not set, will log to stdout 81 | "log_level": "INFO", // The log level 82 | "gitlab": { 83 | "api": "https://gitlab.example.com", // Url of your GitLab 84 | "ssl_verify": true, // Verify SSL certificate when using HTTPs (true, false, path to own CA bundle) 85 | "private_token": "xxxxxxxxxxxxxxxxxxxx", // Token generated in GitLab for an user with admin access 86 | "oauth_token": "", 87 | "ldap_provider":"", // Name of your LDAP provider in gitlab.yml 88 | "create_user": true, // Should the script create the user in GitLab 89 | "group_visibility": "private", // Set visibility level of new group (private, internal, public) 90 | "add_description": true // Add description from your LDAP as group description 91 | }, 92 | "ldap": { 93 | "url": "ldaps://ldap.loc", // URL to your ldap / active directory 94 | "users_base_dn": "ou=users,dc=example,dc=com", // Where we should look for users 95 | "groups_base_dn": "ou=groupss,dc=example,dc=com", // Where we should look for groups 96 | "user_filter": "(memberOf=CN=GitUsers)", // What filter we should use on user selection 97 | "bind_dn": "login", // User to log with 98 | "password": "password", // Password of the user 99 | "group_attribute": "", // The attribute to search in LDAP. The value must be gitlab_sync 100 | "group_prefix": "" // The prefix of the groups that should be synced 101 | } 102 | } 103 | ``` 104 | You should use ```private_token``` or ```oauth_token``` but not both. Check [the gitlab documentation](https://docs.gitlab.com/ce/user/profile/personal_access_tokens.html#creating-a-personal-access-token) for how to generate the personal access token. 105 | 106 | ```create_user``` If set to true, the script will create the users in gitlab and add them in the corresponding groups. Be aware that gitlab will send a mail to every new users created. 107 | ## TODO 108 | 109 | - [ ] Use async search to avoid errors with large LDAP 110 | - [ ] Maybe implement sync interval directly in the script to avoid using cron or systemd 111 | - [x] Use a true logging solution (no more silly print statements) 112 | - [x] Implement ```group_attribute``` and ```group_prefix``` to allow the selection of the groups to sync (avoid syncing every groups into gitlab) 113 | - [ ] your suggestions 114 | ## Built With 115 | 116 | * [Python](https://www.python.org/) 117 | * [python-ldap](https://www.python-ldap.org/en/latest/) 118 | * [python-gitlab](https://python-gitlab.readthedocs.io/en/stable/) 119 | 120 | ## Contributing 121 | 122 | Please read [CONTRIBUTING.md](https://gist.github.com/PurpleBooth/b24679402957c63ec426) for details on our code of conduct, and the process for submitting pull requests to us. 123 | 124 | ## Authors 125 | 126 | * **Jean-François GUILLAUME (Jeff MrBear)** - *Initial work* - [MrBE4R](https://github.com/MrBE4R) 127 | * **Marcel Pennewiß** - Various improvements - [mape2k](https://github.com/mape2k) 128 | 129 | See also the list of [contributors](https://github.com/MrBE4R/gitlab-ldap-sync/contributors) who participated in this project. 130 | 131 | ## License 132 | 133 | This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details 134 | 135 | ## Acknowledgments 136 | 137 | * Hat tip to anyone whose code was used 138 | -------------------------------------------------------------------------------- /config.json: -------------------------------------------------------------------------------- 1 | { 2 | "log": "/tmp/gitlab-ldap-sync.log", 3 | "log_level": "INFO", 4 | "gitlab": { 5 | "api": "", 6 | "ssl_verify": true, 7 | "private_token": "", 8 | "oauth_token": "", 9 | "ldap_provider":"", 10 | "create_user": true, 11 | "group_visibility": "private", 12 | "add_description": true 13 | }, 14 | "ldap": { 15 | "url": "", 16 | "users_base_dn": "", 17 | "groups_base_dn": "", 18 | "user_filter": "", 19 | "bind_dn": "", 20 | "password": "", 21 | "group_attribute": "", 22 | "group_prefix": "" 23 | } 24 | } 25 | -------------------------------------------------------------------------------- /gitlab-ldap-sync.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | 4 | import gitlab 5 | import sys 6 | import json 7 | import ldap 8 | import ldap.asyncsearch 9 | import logging 10 | 11 | if __name__ == "__main__": 12 | print('Initializing gitlab-ldap-sync.') 13 | config = None 14 | with open('config.json') as f: 15 | config = json.load(f) 16 | if config is not None: 17 | print('Done.') 18 | print('Updating logger configuration') 19 | if not config['gitlab']['group_visibility']: 20 | config['gitlab']['group_visibility'] = 'private' 21 | log_option = { 22 | 'format': '[%(asctime)s] [%(levelname)s] %(message)s' 23 | } 24 | if config['log']: 25 | log_option['filename'] = config['log'] 26 | if config['log_level']: 27 | log_option['level'] = getattr(logging, str(config['log_level']).upper()) 28 | logging.basicConfig(**log_option) 29 | print('Done.') 30 | logging.info('Connecting to GitLab') 31 | if config['gitlab']['api']: 32 | gl = None 33 | if not config['gitlab']['private_token'] and not config['gitlab']['oauth_token']: 34 | logging.error('You should set at least one auth information in config.json, aborting.') 35 | elif config['gitlab']['private_token'] and config['gitlab']['oauth_token']: 36 | logging.error('You should set at most one auth information in config.json, aborting.') 37 | else: 38 | if config['gitlab']['private_token']: 39 | gl = gitlab.Gitlab(url=config['gitlab']['api'], private_token=config['gitlab']['private_token'], ssl_verify=config['gitlab']['ssl_verify']) 40 | elif config['gitlab']['oauth_token']: 41 | gl = gitlab.Gitlab(url=config['gitlab']['api'], oauth_token=config['gitlab']['oauth_token'], ssl_verify=config['gitlab']['ssl_verify']) 42 | else: 43 | gl = None 44 | if gl is None: 45 | logging.error('Cannot create gitlab object, aborting.') 46 | sys.exit(1) 47 | gl.auth() 48 | logging.info('Done.') 49 | 50 | logging.info('Connecting to LDAP') 51 | if not config['ldap']['url']: 52 | logging.error('You should configure LDAP in config.json') 53 | sys.exit(1) 54 | 55 | try: 56 | l = ldap.initialize(uri=config['ldap']['url']) 57 | l.simple_bind_s(config['ldap']['bind_dn'], config['ldap']['password']) 58 | except: 59 | logging.error('Error while connecting') 60 | sys.exit(1) 61 | 62 | logging.info('Done.') 63 | 64 | logging.info('Getting all groups from GitLab.') 65 | gitlab_groups = [] 66 | gitlab_groups_names = [] 67 | for group in gl.groups.list(all=True): 68 | gitlab_groups_names.append(group.full_name) 69 | gitlab_group = {"name": group.full_name, "members": []} 70 | for member in group.members.list(all=True): 71 | user = gl.users.get(member.id) 72 | gitlab_group['members'].append({ 73 | 'username': user.username, 74 | 'name': user.name, 75 | 'identities': user.identities[0]['extern_uid'], 76 | 'email': user.email 77 | }) 78 | gitlab_groups.append(gitlab_group) 79 | 80 | logging.info('Done.') 81 | 82 | logging.info('Getting all groups from LDAP.') 83 | ldap_groups = [] 84 | ldap_groups_names = [] 85 | if not config['ldap']['group_attribute'] and not config['ldap']['group_prefix']: 86 | filterstr = '(objectClass=group)' 87 | else: 88 | if config['ldap']['group_attribute'] and config['ldap']['group_prefix']: 89 | logging.error('You should set "group_attribute" or "group_prefix" but not both in config.json') 90 | sys.exit(1) 91 | else: 92 | if config['ldap']['group_attribute']: 93 | filterstr = '(&(objectClass=group)(%s=gitlab_sync))' % config['ldap']['group_attribute'] 94 | if config['ldap']['group_prefix']: 95 | filterstr = '(&(objectClass=group)(cn=%s*))' % config['ldap']['group_prefix'] 96 | attrlist=['name', 'member'] 97 | if config['gitlab']['add_description']: 98 | attrlist.append('description') 99 | for group_dn, group_data in l.search_s(base=config['ldap']['groups_base_dn'], 100 | scope=ldap.SCOPE_SUBTREE, 101 | filterstr=filterstr, 102 | attrlist=attrlist): 103 | ldap_groups_names.append(group_data['name'][0].decode()) 104 | ldap_group = {"name": group_data['name'][0].decode(), "members": []} 105 | if config['gitlab']['add_description'] and 'description' in group_data: 106 | ldap_group.update({"description": group_data['description'][0].decode()}) 107 | if 'member' in group_data: 108 | for member in group_data['member']: 109 | member = member.decode() 110 | for user_dn, user_data in l.search_s(base=config['ldap']['users_base_dn'], 111 | scope=ldap.SCOPE_SUBTREE, 112 | filterstr='(&(|(distinguishedName=%s)(dn=%s))(objectClass=user)%s)' % ( 113 | member, member, config['ldap']['user_filter']), 114 | attrlist=['uid', 'sAMAccountName', 'mail', 'displayName']): 115 | if 'sAMAccountName' in user_data: 116 | username = user_data['sAMAccountName'][0].decode() 117 | else: 118 | username = user_data['uid'][0].decode() 119 | ldap_group['members'].append({ 120 | 'username': username, 121 | 'name': user_data['displayName'][0].decode(), 122 | 'identities': str(member).lower(), 123 | 'email': user_data['mail'][0].decode() 124 | }) 125 | ldap_groups.append(ldap_group) 126 | logging.info('Done.') 127 | 128 | logging.info('Groups currently in GitLab : %s' % str.join(', ', gitlab_groups_names)) 129 | logging.info('Groups currently in LDAP : %s' % str.join(', ', ldap_groups_names)) 130 | 131 | logging.info('Syncing Groups from LDAP.') 132 | 133 | for l_group in ldap_groups: 134 | logging.info('Working on group %s ...' % l_group['name']) 135 | if l_group['name'] not in gitlab_groups_names: 136 | logging.info('|- Group not existing in GitLab, creating.') 137 | gitlab_group = {'name': l_group['name'], 'path': l_group['name'], 'visibility': config['gitlab']['group_visibility']} 138 | if config['gitlab']['add_description'] and 'description' in l_group: 139 | gitlab_group.update({'description': l_group['description']}) 140 | try: 141 | g = gl.groups.create(gitlab_group) 142 | g.save() 143 | gitlab_groups.append({'members': [], 'name': l_group['name']}) 144 | gitlab_groups_names.append(l_group['name']) 145 | except Exception as e: 146 | logging.error('Creating group %s failed: %s' % (l_group['name'], e)) 147 | # Skip next steps due to group could not be created 148 | continue 149 | else: 150 | logging.info('|- Group already exist in GitLab, skiping creation.') 151 | 152 | logging.info('|- Working on group\'s members.') 153 | for l_member in l_group['members']: 154 | if l_member not in gitlab_groups[gitlab_groups_names.index(l_group['name'])]['members']: 155 | logging.info('| |- User %s is member in LDAP but not in GitLab, updating GitLab.' % l_member['name']) 156 | g = [group for group in gl.groups.list(search=l_group['name']) if group.name == l_group['name']][0] 157 | g.save() 158 | u = gl.users.list(username=l_member['username']) 159 | if len(u) > 0: 160 | u = u[0] 161 | if u not in g.members.list(all=True): 162 | g.members.create({'user_id': u.id, 'access_level': gitlab.DEVELOPER_ACCESS}) 163 | g.save() 164 | else: 165 | if config['gitlab']['create_user']: 166 | logging.info('| |- User %s does not exist in gitlab, creating.' % l_member['name']) 167 | try: 168 | u = gl.users.create({ 169 | 'email': l_member['email'], 170 | 'name': l_member['name'], 171 | 'username': l_member['username'], 172 | 'extern_uid': l_member['identities'], 173 | 'provider': config['gitlab']['ldap_provider'], 174 | 'password': 'pouetpouet' 175 | }) 176 | except gitlab.exceptions as e: 177 | if e.response_code == '409': 178 | u = gl.users.create({ 179 | 'email': l_member['email'].replace('@', '+gl-%s@' % l_member['username']), 180 | 'name': l_member['name'], 181 | 'username': l_member['username'], 182 | 'extern_uid': l_member['identities'], 183 | 'provider': config['gitlab']['ldap_provider'], 184 | 'password': 'pouetpouet' 185 | }) 186 | g.members.create({'user_id': u.id, 'access_level': gitlab.DEVELOPER_ACCESS}) 187 | g.save() 188 | else: 189 | logging.info('| |- User %s does not exist in gitlab, skipping.' % l_member['name']) 190 | else: 191 | logging.info('| |- User %s already in gitlab group, skipping.' % l_member['name']) 192 | logging.info('Done.') 193 | 194 | logging.info('Done.') 195 | 196 | logging.info('Cleaning membership of LDAP Groups') 197 | 198 | for g_group in gitlab_groups: 199 | logging.info('Working on group %s ...' % g_group['name']) 200 | if g_group['name'] in ldap_groups_names: 201 | logging.info('|- Working on group\'s members.') 202 | for g_member in g_group['members']: 203 | if g_member not in ldap_groups[ldap_groups_names.index(g_group['name'])]['members']: 204 | if str(config['ldap']['users_base_dn']).lower() not in g_member['identities']: 205 | logging.info('| |- Not a LDAP user, skipping.') 206 | else: 207 | logging.info('| |- User %s no longer in LDAP Group, removing.' % g_member['name']) 208 | g = [group for group in gl.groups.list(search=g_group['name']) if group.name == g_group['name']][0] 209 | u = gl.users.list(username=g_member['username'])[0] 210 | if u is not None: 211 | g.members.delete(u.id) 212 | g.save() 213 | else: 214 | logging.info('| |- User %s still in LDAP Group, skipping.' % g_member['name']) 215 | logging.info('|- Done.') 216 | else: 217 | logging.info('|- Not a LDAP group, skipping.') 218 | logging.info('Done') 219 | else: 220 | logging.error('GitLab API is empty, aborting.') 221 | sys.exit(1) 222 | else: 223 | print('Could not load config.json, check if the file is present.') 224 | print('Aborting.') 225 | sys.exit(1) 226 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | python-gitlab==1.6.0 2 | python-ldap==3.4.0 3 | --------------------------------------------------------------------------------