├── CHANGELOG-TRELLIS-DATABASE-UPLOADS-MIGRATION.md ├── LICENSE.md ├── README.md ├── bin ├── database.sh └── uploads.sh ├── database-backup.yml ├── database-pull.yml ├── database-push.yml └── uploads.yml /CHANGELOG-TRELLIS-DATABASE-UPLOADS-MIGRATION.md: -------------------------------------------------------------------------------- 1 | ### 1.0.1: August 29th, 2018 2 | 3 | * Improve hosts configuration and other small fixes and typos ([#17](https://github.com/valentinocossar/trellis-database-uploads-migration/pull/17)) 4 | * Improve development host configuration and tested up to Ansible 2.6.1 ([#15](https://github.com/valentinocossar/trellis-database-uploads-migration/issues/15)) 5 | * Update LICENSE.md 6 | * Improve README.md ([#9](https://github.com/valentinocossar/trellis-database-uploads-migration/issues/9)) 7 | * Add ISSUE_TEMPLATE.md 8 | * Add .editorconfig 9 | * Add .gitattributes 10 | 11 | ### 1.0.0: November 7th, 2017 12 | 13 | * Initial release 14 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) Valentino Cossar 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 🎩 trellis-database-uploads-migration 2 | 3 | Ansible playbook for Trellis that manages database and uploads migration. Inspired by [hamedb89/trellis-db-push-and-pull](https://github.com/hamedb89/trellis-db-push-and-pull). 4 | 5 | ## ⚙️ Installation 6 | 7 | 1. [Download latest release](https://github.com/valentinocossar/trellis-database-uploads-migration/releases/latest) 8 | 2. Copy `CHANGELOG-TRELLIS-DATABASE-UPLOADS-MIGRATION.md` file into Trellis root folder (so you can always know the version of the tool you are using) 9 | 3. Copy all `*.yml` files into Trellis root folder 10 | 4. Copy all `bin/*.sh` files into Trellis bin folder 11 | 5. Add `database_backup/*` to the end of the Bedrock `.gitignore` file 12 | 6. Set alias for host files as mentioned below in the hosts configuration section 13 | 14 | ### ‼️ Important 15 | 16 | * Tested up to Ansible 2.6.1 17 | * This tool doesn't work with Ansible 2.4.1.0 due to a bug (see [#9](https://github.com/valentinocossar/trellis-database-uploads-migration/issues/9)) 18 | * The development vagrant VM must be powered on every time you run a command, this because the tool checks if the site folder exists and its name is the same of the `local_path` parameter in `wordpress_sites.yml` 19 | * The `database_backup` folder inside Bedrock will be automatically created if doesn't exist 20 | * I recommend you to not perform `git` operations while running `./bin/database.sh` command, this because the tool uses the Bedrock folder as temp folder to store database dump before importing/exporting it and then delete it 21 | * To support url search and replace for Trellis 0.9.8 and lower, remove `.canonical` from variables `url_from` and `url_to` in the files `database-pull.yml` and `database-push.yml` 22 | * This tool hasn't been tested with a multisite configuration, any help with this implementation would be appreciated 23 | * This tool has only been tested with macOS and Ubuntu 18.04 LTS, any help with other operating systems would be appreciated 24 | 25 | ## 🏄 Usage 26 | 27 | * Run `./bin/uploads.sh ` 28 | * Run `./bin/database.sh ` 29 | 30 | ### 📌 Tips 31 | 32 | * Available `` options for uploads task: `push`, `pull` 33 | * Available `` options for database task: `push`, `pull`, `backup` 34 | * The `push` is for sending to the selected environment and the `pull` for receiving from it 35 | * The `backup` is for backup the database of the selected environment 36 | 37 | ## 🛠 Hosts configuration 38 | 39 | ### Development 40 | 41 | ```ini 42 | [development] 43 | development_host ansible_host=192.168.50.5 ansible_connection=ssh ansible_user=vagrant ansible_ssh_private_key_file=.vagrant/machines/default/virtualbox/private_key ansible_ssh_extra_args="-o StrictHostKeyChecking=no -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ForwardAgent=yes" 44 | 45 | [web] 46 | development_host ansible_host=192.168.50.5 ansible_connection=ssh ansible_user=vagrant ansible_ssh_private_key_file=.vagrant/machines/default/virtualbox/private_key ansible_ssh_extra_args="-o StrictHostKeyChecking=no -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ForwardAgent=yes" 47 | ``` 48 | 49 | ### Staging 50 | 51 | ```ini 52 | [staging] 53 | staging_host ansible_host=your_server_hostname 54 | 55 | [web] 56 | staging_host ansible_host=your_server_hostname 57 | ``` 58 | 59 | ### Production 60 | 61 | ```ini 62 | [production] 63 | production_host ansible_host=your_server_hostname 64 | 65 | [web] 66 | production_host ansible_host=your_server_hostname 67 | ``` 68 | 69 | ## 🤝 Contributing 70 | 71 | 1. [Fork it](https://github.com/valentinocossar/trellis-database-uploads-migration/fork) 72 | 2. Create your feature branch (`git checkout -b my-new-feature`) 73 | 3. Commit your changes (`git commit -am 'Add some feature'`) 74 | 4. Push to the branch (`git push origin my-new-feature`) 75 | 5. Create a new Pull Request 76 | -------------------------------------------------------------------------------- /bin/database.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | shopt -s nullglob 3 | 4 | DATABASE_PULL_CMD="ansible-playbook database-pull.yml -e env=$1 -e site=$2" 5 | DATABASE_PUSH_CMD="ansible-playbook database-push.yml -e env=$1 -e site=$2" 6 | DATABASE_BACKUP_CMD="ansible-playbook database-backup.yml -e env=$1 -e site=$2" 7 | ENVIRONMENTS=( hosts/* ) 8 | ENVIRONMENTS=( "${ENVIRONMENTS[@]##*/}" ) 9 | NUM_ARGS=3 10 | 11 | show_usage() { 12 | echo "Usage: ./database.sh 13 | 14 | is the environment to sync database ("staging", "production", etc) 15 | is the WordPress site to sync database (name defined in "wordpress_sites") 16 | is the sync mode ("push", "pull", "backup") 17 | 18 | Available environments: 19 | `( IFS=$'\n'; echo "${ENVIRONMENTS[*]}" )` 20 | 21 | Examples: 22 | ./bin/database.sh staging example.com push 23 | ./bin/database.sh staging example.com pull 24 | ./bin/database.sh staging example.com backup 25 | ./bin/database.sh production example.com push 26 | ./bin/database.sh production example.com pull 27 | ./bin/database.sh production example.com backup 28 | " 29 | } 30 | 31 | HOSTS_FILE="hosts/$1" 32 | 33 | [[ $# -ne $NUM_ARGS || $1 = -h ]] && { show_usage; exit 0; } 34 | 35 | if [[ ! -e $HOSTS_FILE ]]; then 36 | echo "Error: $1 is not a valid environment ($HOSTS_FILE does not exist)." 37 | echo 38 | echo "Available environments:" 39 | ( IFS=$'\n'; echo "${ENVIRONMENTS[*]}" ) 40 | exit 0 41 | fi 42 | 43 | case $3 in 44 | push) 45 | $DATABASE_PUSH_CMD 46 | ;; 47 | pull) 48 | $DATABASE_PULL_CMD 49 | ;; 50 | backup) 51 | $DATABASE_BACKUP_CMD 52 | ;; 53 | *) 54 | show_usage 55 | ;; 56 | esac 57 | -------------------------------------------------------------------------------- /bin/uploads.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | shopt -s nullglob 3 | 4 | UPLOADS_CMD="ansible-playbook uploads.yml -e env=$1 -e site=$2 -e mode=$3" 5 | ENVIRONMENTS=( hosts/* ) 6 | ENVIRONMENTS=( "${ENVIRONMENTS[@]##*/}" ) 7 | NUM_ARGS=3 8 | 9 | show_usage() { 10 | echo "Usage: ./uploads.sh 11 | 12 | is the environment to sync uploads ("staging", "production", etc) 13 | is the WordPress site to sync uploads (name defined in "wordpress_sites") 14 | is the sync mode ("push", "pull") 15 | 16 | Available environments: 17 | `( IFS=$'\n'; echo "${ENVIRONMENTS[*]}" )` 18 | 19 | Examples: 20 | ./bin/uploads.sh staging example.com push 21 | ./bin/uploads.sh staging example.com pull 22 | ./bin/uploads.sh production example.com push 23 | ./bin/uploads.sh production example.com pull 24 | " 25 | } 26 | 27 | HOSTS_FILE="hosts/$1" 28 | 29 | [[ $# -ne $NUM_ARGS || $1 = -h ]] && { show_usage; exit 0; } 30 | 31 | if [[ ! -e $HOSTS_FILE ]]; then 32 | echo "Error: $1 is not a valid environment ($HOSTS_FILE does not exist)." 33 | echo 34 | echo "Available environments:" 35 | ( IFS=$'\n'; echo "${ENVIRONMENTS[*]}" ) 36 | exit 0 37 | fi 38 | 39 | echo -e -n "Are you sure? (Y/n) " 40 | read -n 1 answer 41 | echo " " 42 | if [ "$answer" == "Y" ]; then 43 | $UPLOADS_CMD 44 | else 45 | echo "Operation aborted." 46 | fi 47 | -------------------------------------------------------------------------------- /database-backup.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Backup {{ site }} {{ env }} database 3 | hosts: web:&{{ env }} 4 | remote_user: "{{ web_user }}" 5 | 6 | vars: 7 | project_root: "{{ www_root }}/{{ site }}" 8 | project_web_dir: "{{ project_root }}/current" 9 | host: "{{ env }}_host" 10 | from_host: "{{ hostvars[host] }}" 11 | local_bedrock_dir: "{{ from_host.wordpress_sites[site].local_path }}" 12 | current_date_and_time: "{{ ansible_date_time.date | regex_replace('\\-+', '_') }}_{{ ansible_date_time.hour }}_{{ ansible_date_time.minute }}_{{ ansible_date_time.second }}" 13 | backup_file: "{{ site | regex_replace('\\.+', '_') }}_{{ env }}_{{ current_date_and_time }}.sql.gz" 14 | 15 | tasks: 16 | - name: Check if {{ site }} folder exists 17 | delegate_to: development_host 18 | stat: 19 | path: "{{ project_root }}" 20 | register: result 21 | 22 | - name: Abort if {{ site }} folder doesn't exist 23 | fail: 24 | msg: "ERROR: {{ site }} is not a valid site name ({{ site }} folder does not exist)." 25 | when: result.stat.exists is defined and result.stat.exists == false or result.stat.isdir is defined and result.stat.isdir == false 26 | 27 | - block: 28 | - name: Create database_backup directory if it doesn't exist 29 | delegate_to: development_host 30 | file: 31 | path: "{{ project_web_dir }}/database_backup" 32 | state: directory 33 | mode: 0755 34 | 35 | - name: Export development database 36 | delegate_to: development_host 37 | shell: wp db export - | gzip > database_backup/{{ backup_file }} 38 | args: 39 | chdir: "{{ project_web_dir }}" 40 | when: env is defined and env == "development" 41 | 42 | - name: Export {{ env }} database 43 | shell: wp db export - | gzip > {{ backup_file }} 44 | args: 45 | chdir: "{{ project_web_dir }}" 46 | when: env is defined and env != "development" 47 | 48 | - name: Pull exported database from {{ env }} to development 49 | fetch: 50 | src: "{{ project_web_dir }}/{{ backup_file }}" 51 | dest: "{{ local_bedrock_dir }}/database_backup/" 52 | flat: yes 53 | when: env is defined and env != "development" 54 | 55 | - name: Delete exported database from {{ env }} 56 | shell: rm -f {{ backup_file }} 57 | args: 58 | chdir: "{{ project_web_dir }}" 59 | warn: false 60 | when: env is defined and env != "development" 61 | when: result.stat.exists is defined and result.stat.exists and result.stat.isdir is defined and result.stat.isdir 62 | -------------------------------------------------------------------------------- /database-pull.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Pull {{ site }} database from {{ env }} to development 3 | hosts: web:&{{ env }} 4 | remote_user: "{{ web_user }}" 5 | 6 | vars: 7 | project_root: "{{ www_root }}/{{ site }}" 8 | project_web_dir: "{{ project_root }}/current" 9 | host: "{{ env }}_host" 10 | from_host: "{{ hostvars[host] }}" 11 | url_from: "{{ from_host.wordpress_sites[site].site_hosts.0.canonical }}" 12 | url_to: "{{ hostvars.development_host.wordpress_sites[site].site_hosts.0.canonical }}" 13 | local_bedrock_dir: "{{ hostvars.development_host.wordpress_sites[site].local_path }}" 14 | dump_file: "{{ site | regex_replace('\\.+', '_') }}_db_dump.sql.gz" 15 | current_date_and_time: "{{ ansible_date_time.date | regex_replace('\\-+', '_') }}_{{ ansible_date_time.hour }}_{{ ansible_date_time.minute }}_{{ ansible_date_time.second }}" 16 | backup_file: "{{ site | regex_replace('\\.+', '_') }}_development_{{ current_date_and_time }}.sql.gz" 17 | 18 | tasks: 19 | - name: Abort if environment variable is equal to development 20 | fail: 21 | msg: "ERROR: development is not a valid environment for this mode (you can't pull from development to development)." 22 | when: env == "development" 23 | 24 | - name: Check if {{ site }} folder exists 25 | delegate_to: development_host 26 | stat: 27 | path: "{{ project_root }}" 28 | register: result 29 | 30 | - name: Abort if {{ site }} folder doesn't exist 31 | fail: 32 | msg: "ERROR: {{ site }} is not a valid site name ({{ site }} folder does not exist)." 33 | when: result.stat.exists is defined and result.stat.exists == false or result.stat.isdir is defined and result.stat.isdir == false 34 | 35 | - block: 36 | - name: Create database_backup directory if it doesn't exist 37 | delegate_to: development_host 38 | file: 39 | path: "{{ project_web_dir }}/database_backup" 40 | state: directory 41 | mode: 0755 42 | 43 | - name: Create database dump on {{ env }} 44 | shell: wp db export --allow-root - | gzip > {{ dump_file }} 45 | args: 46 | chdir: "{{ project_web_dir }}" 47 | 48 | - name: Pull database dump from {{ env }} to development 49 | fetch: 50 | src: "{{ project_web_dir }}/{{ dump_file }}" 51 | dest: "{{ local_bedrock_dir }}/" 52 | flat: yes 53 | 54 | - name: Delete database dump from {{ env }} 55 | shell: rm -f {{ dump_file }} 56 | args: 57 | chdir: "{{ project_web_dir }}" 58 | warn: false 59 | 60 | - name: Export development database before importing dump (backup) 61 | delegate_to: development_host 62 | shell: wp db export - | gzip > database_backup/{{ backup_file }} 63 | args: 64 | chdir: "{{ project_web_dir }}" 65 | 66 | - name: Import database dump on development 67 | delegate_to: development_host 68 | shell: gzip -c -d {{ dump_file }} | wp db import - 69 | args: 70 | chdir: "{{ project_web_dir }}" 71 | 72 | - name: Delete database dump from development 73 | delegate_to: development_host 74 | shell: rm -f {{ dump_file }} 75 | args: 76 | chdir: "{{ project_web_dir }}" 77 | warn: false 78 | 79 | - name: Search for {{ url_from }} and replace with {{ url_to }} on development 80 | delegate_to: development_host 81 | command: wp search-replace '//{{ url_from }}' '//{{ url_to }}' --allow-root --all-tables --precise 82 | args: 83 | chdir: "{{ project_web_dir }}" 84 | tags: ['search-replace'] 85 | when: result.stat.exists is defined and result.stat.exists and result.stat.isdir is defined and result.stat.isdir 86 | -------------------------------------------------------------------------------- /database-push.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Push {{ site }} database from development to {{ env }} 3 | hosts: web:&{{ env }} 4 | remote_user: "{{ web_user }}" 5 | 6 | vars: 7 | project_root: "{{ www_root }}/{{ site }}" 8 | project_web_dir: "{{ project_root }}/current" 9 | host: "{{ env }}_host" 10 | to_host: "{{ hostvars[host] }}" 11 | url_from: "{{ hostvars.development_host.wordpress_sites[site].site_hosts.0.canonical }}" 12 | url_to: "{{ to_host.wordpress_sites[site].site_hosts.0.canonical }}" 13 | local_bedrock_dir: "{{ to_host.wordpress_sites[site].local_path }}" 14 | dump_file: "{{ site | regex_replace('\\.+', '_') }}_db_dump.sql.gz" 15 | current_date_and_time: "{{ ansible_date_time.date | regex_replace('\\-+', '_') }}_{{ ansible_date_time.hour }}_{{ ansible_date_time.minute }}_{{ ansible_date_time.second }}" 16 | backup_file: "{{ site | regex_replace('\\.+', '_') }}_{{ env }}_{{ current_date_and_time }}.sql.gz" 17 | 18 | tasks: 19 | - name: Abort if environment variable is equal to development 20 | fail: 21 | msg: "ERROR: development is not a valid environment for this mode (you can't push from development to development)." 22 | when: env == "development" 23 | 24 | - name: Check if {{ site }} folder exists 25 | delegate_to: development_host 26 | stat: 27 | path: "{{ project_root }}" 28 | register: result 29 | 30 | - name: Abort if {{ site }} folder doesn't exist 31 | fail: 32 | msg: "ERROR: {{ site }} is not a valid site name ({{ site }} folder does not exist)." 33 | when: result.stat.exists is defined and result.stat.exists == false or result.stat.isdir is defined and result.stat.isdir == false 34 | 35 | - block: 36 | - name: Create database_backup directory if it doesn't exist 37 | delegate_to: development_host 38 | file: 39 | path: "{{ project_web_dir }}/database_backup" 40 | state: directory 41 | mode: 0755 42 | 43 | - name: Create database dump on development 44 | delegate_to: development_host 45 | shell: wp db export --allow-root - | gzip > {{ dump_file }} 46 | args: 47 | chdir: "{{ project_web_dir }}" 48 | 49 | - name: Push database dump from development to {{ env }} 50 | copy: 51 | src: "{{ local_bedrock_dir }}/{{ dump_file }}" 52 | dest: "{{ project_web_dir }}/{{ dump_file }}" 53 | 54 | - name: Delete database dump from development 55 | delegate_to: development_host 56 | shell: rm -f {{ dump_file }} 57 | args: 58 | chdir: "{{ project_web_dir }}" 59 | warn: false 60 | 61 | - name: Export {{ env }} database before importing dump (backup) 62 | shell: wp db export - | gzip > {{ backup_file }} 63 | args: 64 | chdir: "{{ project_web_dir }}" 65 | 66 | - name: Pull exported database from {{ env }} to development (backup) 67 | fetch: 68 | src: "{{project_web_dir}}/{{ backup_file }}" 69 | dest: "{{ local_bedrock_dir }}/database_backup/" 70 | flat: yes 71 | 72 | - name: Delete exported database from {{ env }} (backup) 73 | shell: rm -f {{ backup_file }} 74 | args: 75 | chdir: "{{ project_web_dir }}" 76 | warn: false 77 | 78 | - name: Import database dump on {{ env }} 79 | shell: gzip -c -d {{ dump_file }} | wp db import - 80 | args: 81 | chdir: "{{ project_web_dir }}" 82 | 83 | - name: Delete database dump from {{ env }} 84 | shell: rm -f {{ dump_file }} 85 | args: 86 | chdir: "{{ project_web_dir }}" 87 | warn: false 88 | 89 | - name: Search for {{ url_from }} and replace with {{ url_to }} on development 90 | command: wp search-replace '//{{ url_from }}' '//{{ url_to }}' --allow-root --all-tables --precise 91 | args: 92 | chdir: "{{ project_web_dir }}" 93 | tags: ['search-replace'] 94 | when: result.stat.exists is defined and result.stat.exists and result.stat.isdir is defined and result.stat.isdir 95 | -------------------------------------------------------------------------------- /uploads.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Sync uploads between environments 3 | hosts: web:&{{ env }} 4 | remote_user: "{{ web_user }}" 5 | 6 | vars: 7 | project: "{{ wordpress_sites[site] }}" 8 | project_root: "{{ www_root }}/{{ site }}" 9 | 10 | tasks: 11 | - name: Abort if environment variable is equal to development 12 | fail: 13 | msg: "ERROR: development is not a valid environment for this mode (you can't push/pull from development to development)." 14 | when: env == "development" 15 | 16 | - name: Check if {{ site }} folder exists 17 | delegate_to: development_host 18 | stat: 19 | path: "{{ project_root }}" 20 | register: result 21 | 22 | - name: Abort if {{ site }} folder doesn't exist 23 | fail: 24 | msg: "ERROR: {{ site }} is not a valid site name ({{ site }} folder does not exist)." 25 | when: result.stat.exists is defined and result.stat.exists == false or result.stat.isdir is defined and result.stat.isdir == false 26 | 27 | - block: 28 | - name: Push uploads 29 | synchronize: 30 | src: "{{ project.local_path }}/web/app/uploads/" 31 | dest: "{{ project_root }}/current/web/app/uploads/" 32 | mode: push 33 | delete: yes 34 | recursive: yes 35 | rsync_opts: --exclude=.DS_Store --stats 36 | when: mode is defined and mode == "push" 37 | register: output_push 38 | 39 | - name: Push uploads output 40 | debug: 41 | var: output_push.stdout_lines 42 | when: output_push.skipped is not defined 43 | 44 | - name: Pull uploads 45 | synchronize: 46 | src: "{{ project_root }}/current/web/app/uploads/" 47 | dest: "{{ project.local_path }}/web/app/uploads/" 48 | mode: pull 49 | delete: yes 50 | recursive: yes 51 | rsync_opts: --exclude=.DS_Store --stats 52 | when: mode is defined and mode == "pull" 53 | register: output_pull 54 | 55 | - name: Pull uploads output 56 | debug: 57 | var: output_pull.stdout_lines 58 | when: output_pull.skipped is not defined 59 | when: result.stat.exists is defined and result.stat.exists and result.stat.isdir is defined and result.stat.isdir 60 | --------------------------------------------------------------------------------