├── .gitignore ├── 1. Commerce Cloud Fundamentals.md ├── 2. Local Environment.md ├── 3. Cloud Configuration.md ├── 4. Service Configuration.md ├── 5. Deployment Process.md ├── 6. Static Content Deployment.md ├── 7. Development.md ├── 8. Troubleshooting.md ├── 9. Go Live and Maintenance.md └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .idea/ -------------------------------------------------------------------------------- /1. Commerce Cloud Fundamentals.md: -------------------------------------------------------------------------------- 1 | # Commerce Cloud Fundamentals 2 | 3 | #### 1.1 Describe the features and functions of Magento Commerce Cloud 4 | 5 | ###### What is Magento Cloud 6 | ###### Cloud platform overview and features 7 | 8 | > Magento Commerce Cloud is a managed, automated hosting platform for the Magento Commerce software. Magento Commerce Cloud comes with a variety of additional features that sets it apart from the on-premise Magento Commerce and Magento Open Source platforms: 9 | > ![Cloud Benefits](https://devdocs.magento.com/common/images/cloud/CloudBenefits.svg) 10 | > Magento Commerce Cloud provides a pre-provisioned infrastructure that includes PHP, MySQL, Redis, RabbitMQ, and Elasticsearch technologies; a git-based workflow with automatic build and deploy for efficient Rapid development and Continuous deployment every time you push code changes in a Platform as a Service (PaaS) environment; highly Customizable environment configuration files and tools; and AWS hosting that offers a Scalable and Secure environment for online sales and retailing. 11 | (https://devdocs.magento.com/guides/v2.3/cloud/bk-cloud.html) 12 | 13 | Technology stack 14 | ![tech stack](https://devdocs.magento.com/common/images/cloud/CloudStack.png) 15 | > Amazon Web Services (AWS) powers the underlying Infrastructure as a Service (IaaS) for Magento Commerce Cloud. Each Magento Commerce Cloud plan provides a PaaS Integration environment for developing, testing, and integrating services, see Magento Commerce Cloud architecture for a comparison and additional features. Performance tools provide granular visibility into the site performance and PHP profiling. Fastly caches site assets and, as customers access the site and stores, loads the cached assets fast. The Fastly Image Optimization (IO) offloads image processing and resizing load, freeing servers to process orders and conversions efficiently. 16 | 17 | 18 | #### 1.2 Determine how to locate settings with Cloud Admin UI 19 | 20 | ###### Locate project settings, user management, and project variables pages [*](https://devdocs.magento.com/guides/v2.3/cloud/project/projects.html) 21 | 22 | Log into the Project Web Interface at https://accounts.magento.cloud. 23 | 24 | ![Project settings url](https://devdocs.magento.com/common/images/cloud_project_gear.png) 25 | 26 | Project settings url: https://eu.magento.cloud/projects/{project_id}/edit 27 | 28 | - User management 29 | - Certificates 30 | - Deploy Key 31 | > SSH key to access external private code repositories pulled during the build process. 32 | - Project variables 33 | > These variables are made available at build-time and/or runtime in all environments of the project. 34 | 35 | ###### Locate environments, access links, and logs [*](https://devdocs.magento.com/guides/v2.3/cloud/project/project-webint-basic.html) 36 | 37 | ![Access site](https://devdocs.magento.com/common/images/cloud_project-access.png) 38 | 39 | ![git](https://devdocs.magento.com/common/images/cloud_project-clone.png) 40 | 41 | Logs can be filtered: 42 | - branch (branched from ..) 43 | - git (Name pushed to Production) 44 | - merge (Name merged Integration into Staging) 45 | - snapshot (Name created snapshot project_id-integration-hash of Integration - restore) 46 | - sync (synced with the parent) 47 | - system (env variables, user management, routes, ) 48 | 49 | ![history](https://devdocs.magento.com/common/images/cloud_environment-history.png) 50 | 51 | ###### Locate environment settings 52 | 53 | You can manage variables and settings for environments through this interface, or with CLI commands. 54 | Click _Configure environment_ to create and manage environments. 55 | ![Manage environment settings](https://devdocs.magento.com/common/images/cloud_project-env.png) 56 | 57 | In the configuration page: 58 | - Settings 59 | - Change environment status (Enable/Disable) 60 | - Outgoing emails (On/Off) 61 | - Indexing by search engines (On/Off) 62 | - HTTP access control (Basic auth + IP address access control) 63 | - Environment-specific variables 64 | > These variables will be available at runtime in this environment, and will be inherited by child environments. 65 | - Routes [?](https://devdocs.magento.com/guides/v2.3/cloud/project/project-conf-files_routes.html) 66 | > Routes allow you to set redirects or upstream settings for applications for your specific environment 67 | - Users [?](https://devdocs.magento.com/guides/v2.3/cloud/project/user-admin.html) 68 | 69 | 70 | ![Settings](https://devdocs.magento.com/common/images/cloud_project-conf-env.png) 71 | 72 | 73 | #### 1.3 Demonstrate the ability to manage users [*](https://devdocs.magento.com/guides/v2.3/cloud/project/user-admin.html) 74 | 75 | ###### Add SSH key [*](https://devdocs.magento.com/guides/v2.3/cloud/before/before-workspace-ssh.html) 76 | 77 | **Add a key using the CLI** 78 | 79 | `magento-cloud ssh-key:add ~/.ssh/id_rsa.pub` 80 | 81 | ssh-key cli commands: 82 | ``` 83 | ssh-key:add Add a new SSH key 84 | ssh-key:delete Delete an SSH key 85 | ssh-key:list (ssh-keys) Get a list of SSH keys in your account 86 | ``` 87 | 88 | **Add a key using the Project Web Interface** 89 | - In the upper right corner, click {your name} > Account Settings. 90 | - Account Settings 91 | - SSH Keys 92 | - Add a public key (Title, Key) 93 | 94 | 95 | ###### Add users to a project and manage their roles 96 | **Project-level roles**: 97 | - Account owner 98 | - Project administrator (Super user) 99 | > can change settings and execute actions on any environment, including creating and restoring snapshots 100 | - Project reader 101 | > can view all environments in a project but cannot execute any actions on them 102 | 103 | **Environment-level roles**: 104 | - Environment administrator 105 | > can change settings and execute actions on this environment, including merging with the parent environment 106 | - Environment contributor 107 | > can push code to this environment and branch the environment 108 | > by default does not have SSH access to the environment (s. You can change the behavior in .magento.app.yaml by specifying access: ssh: contributor) 109 | - Environment reader 110 | 111 | **Manage users with the CLI** 112 | ``` 113 | user:add Add a user to the project 114 | user:delete Delete a user from the project 115 | user:get View a user's role(s) 116 | user:list (users) List project users 117 | user:update Update user role(s) on a project 118 | ``` 119 | 120 | After a new user is added to an environment, the environment must be rebuilt 121 | 122 | #### 1.4 Determine the difference between Magento Cloud plans [*](https://devdocs.magento.com/guides/v2.2/cloud/architecture/cloud-architecture.html) 123 | 124 | Common features (for both plans): 125 | - Magento 2 CE(EE) core features 126 | - PayPal Onboarding Tool 127 | - B2B module 128 | - Fastly Content Delivery Network (CDN), Image Optimization (IO), and added security with generous bandwidth allowances 129 | - New Relic APM for 3 branches 130 | - Blackfire.io Enterprise 131 | - Platform-as-a-service (PaaS) optimized for Magento Commerce 132 | - 24x7 monitoring and email support for the core application and the cloud infrastructure 133 | - Dedicated technical account management for the initial launch period, starting with your subscription until your initial site launch 134 | 135 | 136 | ###### Starter plan vs. Pro differences 137 | 138 | | Starter | Pro | 139 | |----------|------| 140 | | Business Intelligence Essentials | Business Intelligence Pro | 141 | | 4 total active environments | 8 total active environments) | 142 | | Platform-as-a-service (PaaS) based Production, Staging, and development environments | Infrastructure-as-a-Service (IaaS)—dedicated virtual infrastructure for Production environments and for Staging environments (Dedicated hardware) | 143 | | - | High availability architecture with a three-server setup in the underlying Infrastructure-as-a-Service (IaaS) to provide enterprise grade reliability and availability | 144 | | _master_ branch for Production env | _production_ branch for Production env | 145 | 146 | 147 | #### 1.5 Determine how different environment types operate 148 | 149 | ##### Determine differences in environment types: Integration, staging, production 150 | 151 | >**Starter projects** 152 | > 153 | >_Integration_—The Integration environment provides three testable environments. Each environment includes an active Git branch, database, web server, caching, services, environment variables, and configurations. 154 | > 155 | >_Staging_—As code and extensions pass your tests, you can merge your Integration branch to a Staging environment, which becomes your pre-Production testing environment. It includes the staging active branch, database, web server, caching, services, environment variables, configurations, and services, such as Fastly and New Relic. 156 | > 157 | >_Production_—When code is ready and tested, all code merges to master for deployment to the Production live site. This environment includes your active master branch, database, web server, caching, third-party services, environment variables, and configurations. 158 | > 159 | >_Inactive_—You can have an unlimited number of inactive branches. 160 | > 161 | > 162 | >**Pro projects** 163 | > 164 | >_Integration_—The Integration environment provides a testable environment that includes a database, web server, caching, some services, environment variables, and configurations. You can develop, deploy, and test your code before merging to the Staging environment. 165 | > 166 | >_Inactive_—You can have an unlimited number of inactive branches based on the Integration environment. 167 | > 168 | >_Staging_—The Staging environment is for pre-Production testing and includes a database, web server, caching, services, environment variables, configurations, and services, such as Fastly. 169 | > 170 | >_Production_—The Production environment includes a three-node, high-availability architecture for your data, services, caching, and store. This is your live, public store environment with environment variables, configurations, and third-party services. 171 | (https://devdocs.magento.com/guides/v2.2/cloud/architecture/cloud-architecture.html) 172 | 173 | 174 | 175 | Documentation: 176 | - [Manage your project](https://devdocs.magento.com/guides/v2.3/cloud/project/projects.html) 177 | - [Configure your project](https://devdocs.magento.com/guides/v2.3/cloud/project/project-webint-basic.html) 178 | - [Create and manage users](https://devdocs.magento.com/guides/v2.3/cloud/project/user-admin.html) 179 | - [Magento Commerce Cloud architecture](https://devdocs.magento.com/guides/v2.3/cloud/architecture/cloud-architecture.html) 180 | - [Starter architecture](https://devdocs.magento.com/guides/v2.3/cloud/basic-information/starter-architecture.html) 181 | - [Pro architecture](https://devdocs.magento.com/guides/v2.3/cloud/architecture/pro-architecture.html) 182 | -------------------------------------------------------------------------------- /2. Local Environment.md: -------------------------------------------------------------------------------- 1 | # Local Environment 2 | 3 | #### 2.1 Demonstrate ability to set up local development[*](https://devdocs.magento.com/guides/v2.3/cloud/setup/first-time-setup.html) 4 | 5 | ###### Software you need to have locally for developing a Magento Cloud project 6 | - Operating system: Debian GNU/Linux 8 (jessie) 7 | - Virtual machine or container (host): Vagrant, Docker 8 | - Git 9 | - Composer 10 | - Web server: Apache, Nginx 11 | - PHP: PHP 7.1.3+ or 7.2 12 | - PHP packages: bcmath, curl, ext-dom, fpm, gd, intl, json, mbstring, mcrypt (for PHP 7.1 and earlier only), mysql, xml, zip 13 | - Set up PHP memory limit (memory_limit: 2G) 14 | - Database: MySQL or MariaDB with a Galera Cluster (Magento Commerce environments use it) 15 | > for Pro plans modify the `auto_increment_increment = 3` value 16 | - Magento Cloud CLI 17 | - SSH 18 | - Bash or [Cygwin](https://www.cygwin.com/) or Git Bash (for windows) 19 | 20 | Optional: 21 | - Redis 22 | - RabbitMQ 23 | - Varnish 24 | - ElasticSearch 25 | 26 | Se also [Magento Commerce requirements](https://devdocs.magento.com/guides/v2.3/install-gde/system-requirements.html) 27 | 28 | #### 2.2 Given a scenario, demonstrate ability to use the Magento-cloud CLI tool 29 | 30 | ###### Install Magento-cloud CLI. Retrieve project info 31 | - Log in to your local development machine or switch to the Magento file system owner 32 | - Change to a directory to which the Magento file system owner has write access, such as the home directory 33 | - Install: `curl -sS https://accounts.magento.cloud/cli/installer | php` 34 | - Run: `source $HOME/.bashrc` or `export PATH=$PATH:$HOME/.magento-cloud/bin` 35 | - Verify: `magento-cloud list` 36 | 37 | 38 | ###### Manage project and environments 39 | Clone and branch the project: 40 | - Log in to the Magento Cloud CLI: `magento-cloud login` 41 | - List your projects: `magento-cloud project:list` 42 | - Clone a project: `magento-cloud project:get ` 43 | - Change to the magento2 directory 44 | - List available environments for the project: `magento-cloud environment:list` 45 | - Fetch the remote branches: `git fetch magento` 46 | - Pull updated code: `git pull magento ` 47 | 48 | Change the Magento ADMIN variables: 49 | - View existing variables: `magento-cloud variables` 50 | - Set a variable: `magento-cloud vset ` 51 | - Set next variables: ADMIN_EMAIL, ADMIN_USERNAME, ADMIN_PASSWORD, ADMIN_URL 52 | 53 | Other: 54 | - Redeploy: `git commit --allow-empty -m "redeploy" && git push ` 55 | - Clone a project to a directory: `magento-cloud project:get -e ` 56 | - Create a new branch with a name and an ID: `magento-cloud environment:branch ` 57 | - Check out an existing environment: `magento-cloud environment:checkout ` 58 | - Merge changes in this environment with its parent: `magento-cloud environment:merge -p -e ` 59 | - Synchronize (git pull) code and data from the parent to this environment: `magento-cloud environment:synchronize -p -e {code|data}` 60 | - Delete more than one environment: `magento-cloud environment:delete ` 61 | - Tunnels: `magento-cloud tunnel:list` and `magento-cloud tunnel:open -e --app mymagento` and `magento-cloud tunnel:info -e ` 62 | 63 | 64 | ###### Connect to database and SSH 65 | - SSH: `magento-cloud environment:ssh` 66 | - Get DB creds: `php -r 'print_r(json_decode(base64_decode($_ENV["MAGENTO_CLOUD_RELATIONSHIPS"]))->database);'` or `echo $MAGENTO_CLOUD_RELATIONSHIPS | base64 -d | json_pp` 67 | - Connect to database `mysql --host=127.0.0.1 --user='' --pass='' --database='' --port=''` 68 | 69 | ###### Build the project locally 70 | Build via Magento Cloud CLI: `magento-cloud local:build` 71 | 72 | Via docker: `docker-compose run build cloud-build` [cloud-build](https://github.com/magento/magento-cloud-docker/blob/master/php/7.2-cli/bin/cloud-build) 73 | ``` 74 | #!/bin/bash 75 | 76 | composer --working-dir=$MAGENTO_ROOT install --no-suggest --no-ansi --no-interaction --no-progress --prefer-dist 77 | 78 | echo "Building Magento Cloud..." 79 | ./$MAGENTO_ROOT/vendor/bin/ece-tools build 80 | 81 | echo "Fixing file permissions..." 82 | 83 | find $MAGENTO_ROOT/pub -type f -exec chmod 664 {} \; 84 | find $MAGENTO_ROOT/pub -type d -exec chmod 775 {} \; 85 | 86 | echo "Building complete" 87 | ``` 88 | 89 | _Magento Cloud CLI / local:build_ 90 | ``` 91 | Description: Build the current project locally 92 | 93 | Usage: 94 | magento-cloud build [-a|--abslinks] [-s|--source SOURCE] [-d|--destination DESTINATION] [-c|--copy] [--clone] [--run-deploy-hooks] [--no-clean] [--no-archive] [--no-backup] [--no-cache] [--no-build-hooks] [--no-deps] [--working-copy] [--concurrency CONCURRENCY] [--lock] [--] []... 95 | 96 | Arguments: 97 | app Specify application(s) to build 98 | 99 | Options: 100 | -a, --abslinks Use absolute links 101 | -s, --source=SOURCE The source directory. Defaults to the current project root. 102 | -d, --destination=DESTINATION The destination, to which the web root of each app will be symlinked. Default: _www 103 | -c, --copy Copy to a build directory, instead of symlinking from the source 104 | --clone Use Git to clone the current HEAD to the build directory 105 | --run-deploy-hooks Run deploy and/or post_deploy hooks 106 | --no-clean Do not remove old builds 107 | --no-archive Do not create or use a build archive 108 | --no-backup Do not back up the previous build 109 | --no-cache Disable caching 110 | --no-build-hooks Do not run post-build hooks 111 | --no-deps Do not install build dependencies locally 112 | --working-copy Drush: use git to clone a repository of each Drupal module rather than simply 113 | downloading a version 114 | --concurrency=CONCURRENCY Drush: set the number of concurrent projects that will be processed at the same 115 | time [default: 4] 116 | --lock Drush: create or update a lock file (only available with Drush version 7+) 117 | -h, --help Display this help message 118 | -q, --quiet Do not output any message 119 | -V, --version Display this application version 120 | -y, --yes Answer "yes" to all prompts; disable interaction 121 | -n, --no Answer "no" to all prompts 122 | -v|vv|vvv, --verbose Increase the verbosity of messages 123 | 124 | Examples: 125 | Build the current project: 126 | magento-cloud build 127 | 128 | Build the app "example" without symlinking the source files: 129 | magento-cloud build example --copy 130 | 131 | Rebuild the current project without using an archive: 132 | magento-cloud build --no-archive 133 | ``` 134 | 135 | 136 | **Magento Cloud CLI 1.24.0** 137 | ``` 138 | Global options: 139 | --help -h Display this help message 140 | --quiet -q Do not output any message 141 | --verbose -v|vv|vvv Increase the verbosity of messages 142 | --version -V Display this application version 143 | --yes -y Answer "yes" to all prompts; disable interaction 144 | --no -n Answer "no" to all prompts 145 | 146 | Available commands: 147 | clear-cache (clearcache, cc) Clear the CLI cache 148 | docs Open the online documentation 149 | help Displays help for a command 150 | list Lists commands 151 | multi Execute a command on multiple projects 152 | web Open the Web UI 153 | activity 154 | activity:get View detailed information on a single activity 155 | activity:list (activities, act) Get a list of activities for an environment or project 156 | activity:log Display the log for an activity 157 | app 158 | app:config-get View the configuration of an app 159 | app:list (apps) List apps in the project 160 | auth 161 | auth:browser-login (login) Log in to Magento Cloud via a browser 162 | auth:info Display your account information 163 | auth:logout (logout) Log out of Magento Cloud 164 | auth:password-login Log in to Magento Cloud using a username and password 165 | blackfire 166 | blackfire:setup Setup Blackfire.io integration for the project 167 | certificate 168 | certificate:add Add an SSL certificate to the project 169 | certificate:delete Delete a certificate from the project 170 | certificate:get View a certificate 171 | certificate:list (certificates, certs) List project certificates 172 | commit 173 | commit:get Show commit details 174 | commit:list (commits) List commits 175 | db 176 | db:dump Create a local dump of the remote database 177 | db:size Estimate the disk usage of a database 178 | db:sql (sql) Run SQL on the remote database 179 | domain 180 | domain:add Add a new domain to the project 181 | domain:delete Delete a domain from the project 182 | domain:get Show detailed information for a domain 183 | domain:list (domains) Get a list of all domains 184 | domain:update Update a domain 185 | environment 186 | environment:activate Activate an environment 187 | environment:branch (branch) Branch an environment 188 | environment:checkout (checkout) Check out an environment 189 | environment:delete Delete an environment 190 | environment:http-access (httpaccess) Update HTTP access settings for an environment 191 | environment:info Read or set properties for an environment 192 | environment:list (environments, env) Get a list of environments 193 | environment:logs (log) Read an environment's logs 194 | environment:merge (merge) Merge an environment 195 | environment:push (push) Push code to an environment 196 | environment:redeploy (redeploy) Redeploy an environment 197 | environment:relationships (relationships) Show an environment's relationships 198 | environment:ssh (ssh) SSH to the current environment 199 | environment:synchronize (sync) Synchronize an environment's code and/or data from its parent 200 | environment:url (url) Get the public URLs of an environment 201 | integration 202 | integration:add Add an integration to the project 203 | integration:delete Delete an integration from a project 204 | integration:get View details of an integration 205 | integration:list (integrations) View a list of project integration(s) 206 | integration:update Update an integration 207 | integration:validate Validate an existing integration 208 | local 209 | local:build (build) Build the current project locally 210 | local:dir (dir) Find the local project root 211 | mount 212 | mount:download Download files from a mount, using rsync 213 | mount:list (mounts) Get a list of mounts 214 | mount:size Check the disk usage of mounts 215 | mount:upload Upload files to a mount, using rsync 216 | project 217 | project:clear-build-cache Clear a project's build cache 218 | project:get (get) Clone a project locally 219 | project:info Read or set properties for a project 220 | project:list (projects, pro) Get a list of all active projects 221 | project:set-remote Set the remote project for the current Git repository 222 | repo 223 | repo:cat Read a file in the project repository 224 | repo:ls List files in the project repository 225 | route 226 | route:get View a resolved route 227 | route:list (routes) List all routes for an environment 228 | self 229 | self:install Install or update CLI configuration files 230 | self:update (self-update) Update the CLI to the latest version 231 | service 232 | service:list (services) List services in the project 233 | service:mongo:dump (mongodump) Create a binary archive dump of data from MongoDB 234 | service:mongo:export (mongoexport) Export data from MongoDB 235 | service:mongo:restore (mongorestore) Restore a binary archive dump of data into MongoDB 236 | service:mongo:shell (mongo) Use the MongoDB shell 237 | service:redis-cli (redis) Access the Redis CLI 238 | snapshot 239 | snapshot:create Make a snapshot of an environment 240 | snapshot:list (snapshots) List available snapshots of an environment 241 | snapshot:restore Restore an environment snapshot 242 | ssh-key 243 | ssh-key:add Add a new SSH key 244 | ssh-key:delete Delete an SSH key 245 | ssh-key:list (ssh-keys) Get a list of SSH keys in your account 246 | tunnel 247 | tunnel:close Close SSH tunnels 248 | tunnel:info View relationship info for SSH tunnels 249 | tunnel:list (tunnels) List SSH tunnels 250 | tunnel:open Open SSH tunnels to an app's relationships 251 | tunnel:single Open a single SSH tunnel to an app relationship 252 | user 253 | user:add Add a user to the project 254 | user:delete Delete a user from the project 255 | user:get View a user's role(s) 256 | user:list (users) List project users 257 | user:update Update user role(s) on a project 258 | variable 259 | variable:create Create a variable 260 | variable:delete Delete a variable 261 | variable:get (vget) View a variable 262 | variable:list (variables, var) List variables 263 | variable:update Update a variable 264 | worker 265 | worker:list (workers) Get a list of all deployed workers 266 | ``` 267 | 268 | #### 2.x Docker 269 | [Magento Commerce (Cloud) Docker](https://github.com/magento/magento-cloud-docker) 270 | 271 | Docker images: 272 | - DB: MariaDB versions 10.0, 10.1, 10.2 273 | - FPM: PHP versions 7.0, 7.1, 7.2 274 | - NGINX: 1.9 275 | - Redis: 3.0, 3.2, 4.0, 5.0 276 | - Varnish: 4.x 277 | - RabbitMQ: 3.5, 3.7 278 | - Elasticsearch: 1.7, 2.4, 5.2, 6.5 279 | 280 | Launch Docker configuration: 281 | - Docker configuration generator: `./vendor/bin/ece-tools docker:build` 282 | - Copy and configure env files: 283 | ``` 284 | cp docker/config.php.dist docker/config.php 285 | cp docker/global.php.dist docker/global.php 286 | ``` 287 | - Convert the PHP configuration files to Docker ENV files: `./vendor/bin/ece-tools docker:config:convert` 288 | - Build files to containers and run in the background: `docker-compose up -d` 289 | - Build Magento in the Docker container: `docker-compose run build cloud-build` 290 | - Deploy Magento in the Docker container: `docker-compose run deploy cloud-deploy` 291 | - Stop and remove: `docker-compose down -v` 292 | 293 | ece-tools 294 | ``` 295 | docker 296 | docker:build Build docker configuration 297 | docker:build:integration Build test docker configuration 298 | docker:config:convert Convert raw config to .env files configuration 299 | ``` 300 | 301 | Cli containers: 302 | > build—extends the CLI container to perform operations with writable filesystem, similar to the build phase 303 | > 304 | > deploy—extends the CLI container to use read-only file system, similar to the deploy phase 305 | > 306 | > cron—extends the CLI container to run cron 307 | > 308 | > The setup:cron:run and cron:update commands are not available on Cloud and Docker for Cloud environment 309 | > 310 | > Cron only works with CLI container to run ./bin/magento cron:run command 311 | (https://devdocs.magento.com/guides/v2.3/cloud/docker/docker-development.html) 312 | 313 | Docker compose command: 314 | ``` 315 | Build environment docker-compose run build cloud-build 316 | Deploy environment docker-compose run deploy cloud-deploy 317 | Connect to CLI container docker-compose run deploy bash 318 | Use ece-tools command docker-compose run deploy ece-command 319 | Use Magento command docker-compose run deploy magento 320 | Stop Docker environment without destroying containers docker-compose stop 321 | Resume Docker environment docker-compose start 322 | ``` 323 | 324 | Documentation: 325 | - [Local development](https://devdocs.magento.com/guides/v2.3/cloud/setup/first-time-setup.html) 326 | - [Prepare for local environment setup](https://devdocs.magento.com/guides/v2.3/cloud/before/before-workspace.html) 327 | - [Docker development environment](https://devdocs.magento.com/guides/v2.3/cloud/docker/docker-development.html) 328 | -------------------------------------------------------------------------------- /3. Cloud Configuration.md: -------------------------------------------------------------------------------- 1 | # Cloud Configuration 2 | 3 | #### 3.1 Determine how to configure Cloud 4 | 5 | ###### How to configure different redirects in this file, which types of redirects should not be configured here 6 | A route describes how an incoming HTTP request is going to be processed by Magento Cloud. 7 | Route configuration (from platform.sh) 8 | ![Routes](https://docs.platform.sh/images/config_diagrams/routes2.svg) 9 | 10 | File `.magento/routes.yaml` - defines routes for the Integration environments. 11 | ```yaml 12 | "http://{default}/": 13 | type: upstream 14 | upstream: "mymagento:php" 15 | 16 | # whole-route redirects 17 | "http://www.{default}/": 18 | type: redirect 19 | to: http://{default}/ 20 | 21 | # partial redirects 22 | http://test.{default}/: 23 | type: upstream 24 | redirects: 25 | expires: 1d 26 | paths: 27 | "/from": { "to": "/destination", code: 308, expires: 2w} 28 | "/regexp/(.*)/matching": { "to": "http://example.com/$1", regexp: true } 29 | ``` 30 | 31 | Each rule under `paths` is defined by: 32 | _key_ - expression to match against the request path; 33 | _value_ - object describing both the destination to redirect with detail on how to handle the redirection; 34 | 35 | Value object is defined with the following keys: 36 | - to: (required) partial ("/destination" or "//destination") or full URL ("http://example.com/"). 37 | - regexp: (default: false) specifies whether the path key should be interpreted as a PCRE regular expression 38 | Special arguments in the `to` statement (only if regexp: true) 39 | - `$is_args`: `?` or empty string 40 | - `$args`: full query string if any 41 | - `$arg_foo`: value of the query parameter `foo` 42 | - `$uri`: full URI of the request 43 | - prefix: (default: true, not supported if regexp is true) specifies whether we should redirect both the path and all its children or just the path itself 44 | Example: 45 | if true: /from redirects to /to and /from/another/path will redirect to /to/another/path 46 | if false: /from triggers a redirect, but /from/another/path does not. 47 | - append_suffix: (default: true, not supported if regexp is true or if prefix is false) determines if the suffix is carried over with the redirect 48 | Example: 49 | if true: /from redirects to /to/path/suffix in case /from/path/suffix 50 | if false: /from redirects to /to in case /from/path/suffix 51 | - code: status codes are 301, 302 (default), 307, and 308 52 | - expires: examples of valid values include 3600s, 1d, 2w, 3m 53 | 54 | List the configured routes: `magento-cloud environment:routes` 55 | 56 | Which types of redirects should not be configured here: 57 | - Application-driven redirects 58 | - Fastly 59 | - Long list of storefront domains (not sure) 60 | - As a general rule we recommend keeping the defined routes under 100 61 | - Let's Encrypt also limits an environment to 100 configured domains 62 | - Linux kernel limit on environment variables is 32 pages (each page is 4k on x86, so maximum environment variable length of 128KB). So it's a limits for `routes.yaml` 63 | 64 | ###### How to add these configurations to Staging or Production environments. Magento On-Premises installation migration 65 | 66 | `.magento/routes.yaml` limitations: 67 | > For Pro projects, the changes you make using this YAML file affects your Integration environment only. 68 | > To make these changes in a Staging or Production environment, you must create a Support ticket. 69 | (https://devdocs.magento.com/guides/v2.3/cloud/project/project-conf-files_routes.html) 70 | 71 | Other 72 | > You must submit a support ticket to update and modify the following in the Staging and Production environments: 73 | > - Cron jobs 74 | > - Redirects from routes.yaml file 75 | > - Managing PHP extensions 76 | > - Managing mounts 77 | (https://devdocs.magento.com/guides/v2.3/cloud/trouble/pro-env-management.html) 78 | 79 | Also it's possible to create redirect via Fastly VCL snippets: 80 | 1. Set the custom response code related to redirects 81 | - Name: student_redirect 82 | - Type: recv 83 | - Priority: 100 84 | - VCL snippet content: 85 | ``` 86 | if (req.http.host ~ "student.test.com"") { 87 | error 750 "student.test.com""; 88 | } 89 | ``` 90 | 2. VCL that contains all needed redirects: 91 | - Name: custom_domains_redirect 92 | - Type: error 93 | - Priority: 100 94 | - VCL snippet content: 95 | ``` 96 | if (obj.status == 750) { 97 | if (obj.response ~ "student.test.com") { 98 | set obj.http.Location = "https://test.com/student"; 99 | } 100 | set obj.status = 302; 101 | set obj.response = "Moved Temporarily"; 102 | return (deliver); 103 | } 104 | ``` 105 | 106 | Example [how set up redirects to WordPress using Fastly](https://devdocs.magento.com/guides/v2.3/cloud/cdn/fastly-vcl-wordpress.html) 107 | 108 | VCL snippet for the WordPress redirect 109 | ```json 110 | { 111 | "name": "wordpress_redirect", 112 | "dynamic": "0", 113 | "type": "recv", 114 | "priority": "5", 115 | "content": "if ( req.url.path ~ \"^\\/?([^\\/?]+)\") { if ( table.lookup(wordpress_urls, re.group.1, \"NOTFOUND\") != \"NOTFOUND\" ) { set req.http.X-WP = \"1\"; } }" 116 | } 117 | ``` 118 | 119 | Another way to set up redirects - use [edge dictionary](https://docs.fastly.com/guides/edge-dictionaries/about-edge-dictionaries). 120 | [Working with Edge Dictionary items using the API](https://docs.fastly.com/guides/edge-dictionaries/working-with-dictionary-items-using-the-api) 121 | 122 | 123 | ###### How to migrate an existing Magento installation into Magento Cloud: Code base, database, media migration 124 | 125 | **Code base** 126 | - add required files ([examples from magento-cloud repo](https://github.com/magento/magento-cloud)) 127 | ``` 128 | .magento.app.yaml - manages applications, service relationships, mounts for writable directories, and cron jobs 129 | .magento/services.yaml - for service configurations including MySQL, PHP, Redis, Solr (2.0.X only), ElasticSearch (2.1.X and later) 130 | .magento/routes.yaml - for handling routes including redirections, caching, and server-side includes 131 | magento-vars.php - for multiple websites and stores 132 | ``` 133 | - add Magento authentication keys and custom repositories keys 134 | - using `auth.json` file 135 | ```json 136 | { 137 | "http-basic": { 138 | "repo.magento.com": { 139 | "username": "", 140 | "password": "" 141 | } 142 | } 143 | } 144 | ``` 145 | - using an environment variable (`env:COMPOSER_AUTH`) 146 | - project level variable 147 | - name: env:COMPOSER_AUTH. 148 | - content the same as in `auth.json` file 149 | - visible during build: true 150 | - visible at run: false 151 | - edit composer.json 152 | - replace `magento/product-enterprise-edition` to `"magento/magento-cloud-metapackage": "",` 153 | - update the "files" directive in the autoload section to refer to app/etc/NonComposerComponentRegistration.php (why?) 154 | - run composer update to update composer.lock 155 | - encryption key 156 | - get old encryption key from /app/etc/env.php (crypt/key) 157 | - add variable `CRYPT_KEY` with correct key 158 | - or update crypt/key manually on each env (don't like it) 159 | - see ece-tools [Process/Deploy/SetCryptKey.php](https://github.com/magento/ece-tools/blob/develop/src/Process/Deploy/SetCryptKey.php) 160 | 161 | **Database** 162 | - create a database dump: 163 | `mysqldump -h -P -p -u --single-transaction --no-autocommit --quick | gzip > ~/db.sql.tgz` 164 | - transfer the database dump to the Magent Cloud 165 | `rsync db.sql.tgz :var/db.sql.tgz` 166 | - ssh to needed env 167 | `magento-cloud ssh` 168 | - connect to the database 169 | `mysql -h -P -p -u ` 170 | - drop the database 171 | `drop database ;` 172 | - re-create the database 173 | `create database ;` 174 | - restore dump 175 | `zcat var/db.sql.tgz | sed -e 's/DEFINER[ ]*=[ ]*[^*]*\*/\*/' | mysql -h -P -p -u ` 176 | - update configs (`SELECT * from core_config_data;`) 177 | - base URLs 178 | - `UPDATE core_config_data SET value='' WHERE path='web/unsecure/base_url';` 179 | - `UPDATE core_config_data SET value='' WHERE path='web/secure/base_url';` 180 | 181 | **Media migration** 182 | - back up media files 183 | `php /bin/magento setup:backup --media` 184 | - transfer the media dump 185 | `rsync /var/backups/ :var/media.tgz` 186 | - ssh to needed env 187 | `magento-cloud ssh` 188 | - clear existing media files 189 | `rm -rf pub/media/*` 190 | - extract the media files to the pub/media directory 191 | `tar -xzf var/media.tgz pub/media` 192 | 193 | #### 3.2 Determine how to configure a planned service 194 | 195 | ###### How to configure a service that is planned to be added to the environment 196 | 197 | #### 3.3 Demonstrate ability to add to your environment 198 | 199 | ###### Which configurations you can add to your environment and how to do it 200 | 201 | ###### What to configure in this file, on which environments these configurations are applied, how to add these configurations to environments where this file is not read 202 | 203 | Documentation: 204 | - [Configure Routes / platform.sh](https://docs.platform.sh/configuration/routes.html) 205 | - [Configure routes](https://devdocs.magento.com/guides/v2.3/cloud/project/project-conf-files_routes.html) 206 | - [Redirects / platform.sh](https://docs.platform.sh/configuration/routes/redirects.html) 207 | - [Redirects](https://devdocs.magento.com/guides/v2.3/cloud/project/project-routes-more-redir.html) 208 | - [Custom Fastly VCL snippets](https://devdocs.magento.com/guides/v2.3/cloud/cdn/cloud-vcl-custom-snippets.html) 209 | - [Import existing code into a project](https://devdocs.magento.com/guides/v2.3/cloud/setup/first-time-setup-import-first-steps.html) 210 | -------------------------------------------------------------------------------- /4. Service Configuration.md: -------------------------------------------------------------------------------- 1 | # Service Configuration 2 | 3 | #### 4.1 Demonstrate ability to create service configurations 4 | 5 | ###### How to add system services: MySQL, Redis, Elasticsearch, RabbitMQ 6 | 7 | > For Pro projects, the changes you make using this YAML file affects your Integration environment only. 8 | > To make these changes in a Staging or Production environment, you must create a Support ticket. 9 | 10 | 11 | **Defining services** 12 | 13 | `.magento/services.yaml` file used to configure your services. 14 | 15 | ```yaml 16 | mysql: 17 | type: mysql:10.2 18 | disk: 2048 19 | configuration: 20 | schemas: 21 | - main 22 | - legacy 23 | endpoints: 24 | mysql: 25 | default_schema: main 26 | privileges: 27 | main: admin 28 | mbi: 29 | default_schema: main 30 | privileges: 31 | main: ro 32 | importer: 33 | default_schema: legacy 34 | privileges: 35 | main: rw 36 | properties: 37 | max_allowed_packet: 64 38 | 39 | redis: 40 | type: redis:3.0 41 | 42 | elasticsearch: 43 | type: elasticsearch:6.5 44 | disk: 1024 45 | configuration: 46 | plugins: 47 | - analysis-icu 48 | - lang-python 49 | 50 | rabbitmq: 51 | type: rabbitmq:3.5 52 | disk: 1024 53 | ``` 54 | 55 | **Using services** 56 | 57 | For services to be available to an application in your project, 58 | you must specify [relationships](https://docs.platform.sh/configuration/app/relationships.html) between applications and services in `.magento.app.yaml`. 59 | 60 | The left-hand side is the name of the relationship as it will be exposed to the application in the MAGENTO_CLOUD_RELATIONSHIPS variable. 61 | The right-hand side is in the form `:`. 62 | Where `` comes from `.magento/services.yaml`. 63 | The `` is the exposed functionality of the service to use. For most services that is simply the same as the service type, but on MySQL, for example, could be different if the service is running multiple databases (see `configuration.endpoints`). 64 | 65 | ```yaml 66 | relationships: 67 | database: "mysql:mysql" 68 | mbi: "mysql:mbi" 69 | imports: "mysql:importer" 70 | redis: "redis:redis" 71 | elasticsearch: "elasticsearch:elasticsearch" 72 | rabbitmq: "rabbitmq:rabbitmq" 73 | ``` 74 | 75 | #### 4.2 Demonstrate ability to use Slave connections 76 | 77 | ###### How to leverage slave connections to MySQL, Redis 78 | 79 | 1. Add next options in the `.magento.env.yaml` file 80 | ```yaml 81 | stage: 82 | deploy: 83 | MYSQL_USE_SLAVE_CONNECTION: true 84 | REDIS_USE_SLAVE_CONNECTION: true 85 | ``` 86 | 87 | 2. Create next variables (MYSQL_USE_SLAVE_CONNECTION, REDIS_USE_SLAVE_CONNECTION) 88 | ``` 89 | magento-cloud variable:create --level=environment --value=true --inheritable=false --json=false --prefix=none MYSQL_USE_SLAVE_CONNECTION 90 | magento-cloud variable:create --level=environment --value=true --inheritable=false --json=false --prefix=none REDIS_USE_SLAVE_CONNECTION 91 | ``` 92 | 93 | _REDIS_USE_SLAVE_CONNECTION_ used by ece-tools [Config/Factory/Cache](https://github.com/magento/ece-tools/blob/develop/src/Config/Factory/Cache.php#L65) 94 | > Magento can read multiple Redis instances asynchronously. 95 | > Set to true to automatically use a read-only connection to a Redis instance to receive read-only traffic on a non-master node. This improves performance through load balancing, because only one node needs to handle read-write traffic. Set to false to remove any existing read-only connection array from the env.php file. 96 | 97 | _MYSQL_USE_SLAVE_CONNECTION_ used by ece-tools [Config/Database/MergedConfig](https://github.com/magento/ece-tools/blob/develop/src/Config/Database/MergedConfig.php#L137) 98 | and get credentials by [Config/Database/SlaveConfig](https://github.com/magento/ece-tools/blob/develop/src/Config/Database/SlaveConfig.php#L34) 99 | > Magento can read multiple databases asynchronously. 100 | > Set to true to automatically use a read-only connection to the database to receive read-only traffic on a non-master node. This improves performance through load balancing, because only one node needs to handle read-write traffic. Set to false to remove any existing read-only connection array from the env.php file. 101 | 102 | Documentation: 103 | - [Configure Services / platform.sh](https://docs.platform.sh/configuration/services.html) 104 | - [Services](https://devdocs.magento.com/guides/v2.3/cloud/project/project-conf-files_services.html) 105 | - [Relationships / platform.sh](https://docs.platform.sh/configuration/app/relationships.html) 106 | - [MYSQL_USE_SLAVE_CONNECTION](https://devdocs.magento.com/guides/v2.3/cloud/env/variables-deploy.html#mysql_use_slave_connection) 107 | - [REDIS_USE_SLAVE_CONNECTION](https://devdocs.magento.com/guides/v2.3/cloud/env/variables-deploy.html#redis_use_slave_connection) 108 | -------------------------------------------------------------------------------- /5. Deployment Process.md: -------------------------------------------------------------------------------- 1 | # Deployment Process 2 | 3 | #### 5.1 Determine the processes during deployment 4 | 5 | ###### Describe all processes that are running during deployment: Build, deploy and post-deploy phases. Explain why downtime occurs on your project 6 | 7 | Build and deploy full steps: 8 | 1. Code and configuration validation 9 | - run series of checks and code validation 10 | - modify cluster topology (e.g. elasticsearch was added) 11 | - run `composer install` 12 | - on syntax error in a configuration file, Git server refuses the push, see [Protective block](https://devdocs.magento.com/guides/v2.3/cloud/live/live-prot.html) 13 | - to disable protective block 14 | ```yaml 15 | # .magento.app.yaml 16 | preflight: 17 | enabled: false 18 | ``` 19 | 2. Build 20 | - site is not in maintenance mode 21 | - will not be brought down if errors or issues occur 22 | - cluster has not been created yet (don't try to connect to a database or assume anything was daemonized) 23 | - execute commands before packaging your application 24 | - runs hooks in the `build` section from `.magento.app.yaml` 25 | ```yaml 26 | # We run build hooks before your application has been packaged. 27 | build: | 28 | set -e 29 | php ./vendor/bin/ece-tools build:generate 30 | php ./vendor/bin/ece-tools build:transfer 31 | # old style 32 | #php ./vendor/bin/ece-tools build 33 | ``` 34 | - result of the build phase is a read-only file system referred to as a slug 35 | 3. Prepare the slug 36 | - create an archive and put the slug in permanent storage 37 | - slug includes all files and folders excluding the mounts configured in `magento.app.yaml` 38 | ```yaml 39 | mounts: 40 | "var": "shared:files/var" 41 | "app/etc": "shared:files/etc" 42 | "pub/media": "shared:files/media" 43 | "pub/static": "shared:files/static" 44 | ``` 45 | 4. Deploy slugs and cluster 46 | - file systems are read-only 47 | - mounts each service in a container (web server, Elasticsearch, RabbitMQ) 48 | - mounts the read-write file system (mounted on a highly available distributed storage grid) 49 | - configures the network so services can “see” each other (and only each other) 50 | 5. Deployment hooks 51 | - freeze the incoming traffic at the entry point for 60 seconds 52 | - puts the application in maintenance mode until deployment is complete 53 | - execute commands after packaging and deploying your application 54 | - runs hooks in the `deploy` section from `.magento.app.yaml` 55 | ```yaml 56 | # We run deploy hook after your application has been deployed and started. 57 | deploy: | 58 | php ./vendor/bin/ece-tools deploy 59 | ``` 60 | - deploy script uses the values defined by configuration files in the `.magento` directory, then the script deletes the directory and its contents 61 | 62 | Note: 63 | > During the deployment process, all connections queue for up to 5 minutes preserving any active sessions and pending actions, 64 | > such as adding to cart or checkout. 65 | > After deployment, the queue is released and connections continue without interruption. 66 | > To use this connection hold to your advantage and reduce downtime to zero, you must configure your project to use the most efficient deploy strategy. 67 | 6. Post-deployment: configure routing 68 | - creates backup (BAK) files for the app/etc/env.php and the app/etc/config.php configuration files 69 | - execute commands after deploying your application and after the container begins accepting connections 70 | - runs hooks in the `post_deploy` section from `.magento.app.yaml` 71 | ```yaml 72 | # We run post deploy hook to clean and warm the cache. Available with ECE-Tools 2002.0.10. 73 | post_deploy: | 74 | php ./vendor/bin/ece-tools post-deploy 75 | ``` 76 | 77 | ###### What role every process/phase plays and how to impact every process 78 | ###### How Magento Cloud deploys Magento. What every script does on every deployment phase 79 | 80 | Ece-tools [\Magento\MagentoCloud\App\Container](https://github.com/magento/ece-tools/blob/develop/src/App/Container.php#L50) initialize [\Illuminate\Container\Container](https://laravel.com/api/5.5/Illuminate/Container/Container.html) and configure all commands processes. 81 | 82 | **Build** 83 | 84 | `php ./vendor/bin/ece-tools build` is a proxy for calling build:generate and build:transfer commands. 85 | 86 | ```php 87 | public function execute(InputInterface $input, OutputInterface $output) 88 | { 89 | $this->getApplication()->find(\Magento\MagentoCloud\Command\Build\Generate::NAME)->execute($input, $output); 90 | $this->getApplication()->find(\Magento\MagentoCloud\Command\Build\Transfer::NAME)->execute($input, $output); 91 | } 92 | ``` 93 | 94 | 95 | `php ./vendor/bin/ece-tools build:generate` 96 | 97 | > notice: Starting generate command. (magento/ece-tools version: 2002.0.17, magento/magento2-base version: 2.2.8) 98 | 99 | - [Process\Build\PreBuild](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/PreBuild.php) 100 | - check `VERBOSE_COMMANDS` in `.magento.env.yaml` 101 | - clean `generated/code` folder 102 | - clean `generated/metadata` folder 103 | - delete flag `.static_content_deploy` file 104 | - [Process\ValidateConfiguration](https://github.com/magento/ece-tools/tree/develop/src/Process/ValidateConfiguration.php) 105 | ```php 106 | $this->container->make(\Magento\MagentoCloud\Process\ValidateConfiguration::class, [ 107 | 'validators' => [ 108 | ValidatorInterface::LEVEL_CRITICAL => [ 109 | $this->container->make(ConfigValidator\Build\ComposerFile::class), 110 | $this->container->make(ConfigValidator\Build\StageConfig::class), 111 | $this->container->make(ConfigValidator\Build\BuildOptionsIni::class), 112 | ], 113 | ValidatorInterface::LEVEL_WARNING => [ 114 | $this->container->make(ConfigValidator\Build\ConfigFileExists::class), 115 | $this->container->make(ConfigValidator\Build\DeprecatedBuildOptionsIni::class), 116 | $this->container->make(ConfigValidator\Build\StageConfigDeprecatedVariables::class), 117 | $this->container->make(ConfigValidator\Build\ModulesExists::class), 118 | $this->container->make(ConfigValidator\Build\AppropriateVersion::class), 119 | $this->container->make(ConfigValidator\Build\ScdOptionsIgnorance::class), 120 | $this->container->make(ConfigValidator\IdealState::class), 121 | ], 122 | ], 123 | ]), 124 | ``` 125 | validators list: [Config/Validator/Build/](https://github.com/magento/ece-tools/tree/develop/src/Config/Validator/Build) 126 | - [Process\Build\RefreshModules](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/RefreshModules.php) 127 | - see [\Config\Module::refresh](https://github.com/magento/ece-tools/tree/develop/src/Config/Module.php) 128 | - enable all modules `php ./bin/magento module:enable --all --ansi --no-interaction` if `modules` not found in the `app/etc/config.php` 129 | - do not enable already disabled modules (if `modules` found in the `app/etc/config.php`) 130 | - [Process\Build\ApplyPatches](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/ApplyPatches.php) 131 | - applying patches (`Patch\Manager::applyAll`) 132 | - copyStaticFile: copy `pub/static.php` => `pub/front-static.php` 133 | - applyComposerPatches: from [patches.json](https://github.com/magento/ece-tools/blob/develop/patches.json) file 134 | - applyHotFixes: `git apply` patches from `m2-hotfixes/*.patch` 135 | - [Process\Build\MarshallFiles](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/MarshallFiles.php) 136 | - delete `var/cache/` directory 137 | - copying di.xml files for Magento version < 2.2 138 | - [Process\Build\CopySampleData](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/CopySampleData.php) 139 | - copy (if exists) `vendor/magento/sample-data-media` => `/pub/media` 140 | - [Process\Build\CompileDi](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/CompileDi.php) 141 | - execute `php ./bin/magento setup:di:compile {$verbosityLevel} --ansi --no-interaction` 142 | - [Process\Build\ComposerDumpAutoload](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/ComposerDumpAutoload.php) 143 | - execute `composer dump-autoload -o --ansi --no-interaction` 144 | - [Process\Build\DeployStaticContent](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/DeployStaticContent.php) 145 | - delete `.static_content_deploy` flag file 146 | - validate [Config\Validator\GlobalStage\ScdOnBuild::validate](https://github.com/magento/ece-tools/blob/develop/src/Config/Validator/GlobalStage/ScdOnBuild.php#L72) 147 | - check SCD_ON_DEMAND: false 148 | - check SKIP_SCD: false 149 | - validate [Config\Validator\Build\ConfigFileStructure::validate](https://github.com/magento/ece-tools/blob/develop/src/Config/Validator/Build/ConfigFileStructure.php#L63) 150 | - in the `app/etc/config.php` 151 | - scopes/websites (count($websites) > 0) 152 | - scopes/stores (count($stores) > 0 ) 153 | - if all checks valid, execute [Process\Build\DeployStaticContent\Generate](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/DeployStaticContent/Generate.php) 154 | - build SCD commands [StaticContent\CommandFactory::matrix](https://github.com/magento/ece-tools/blob/develop/src/StaticContent/CommandFactory.php#L98) 155 | - [StaticContent\CommandFactory::build](https://github.com/magento/ece-tools/blob/develop/src/StaticContent/CommandFactory.php#L131) 156 | - command: `php ./bin/magento setup:static-content:deploy --ansi --no-interaction` 157 | - add ` -f` option (for magento > 2.2) 158 | - add ` -s {$stratagy}` from `SCD_STRATEGY` from `.magento.env.yaml` (for magento > 2.2) 159 | - add verbosity level (-v, -vv, -vvv) from `VERBOSE_COMMANDS` from `.magento.env.yaml` 160 | - add ` --jobs {$threadCount}` threads count: `SCD_THREADS` from `.magento.env.yaml` 161 | - add ` --no-html-minify` by `SKIP_HTML_MINIFICATION` (default: true) from `.magento.env.yaml` 162 | - result example: `php ./bin/magento setup:static-content:deploy --ansi --no-interaction -f --jobs 3 --no-html-minify` 163 | - default command (`StaticContent\CommandFactory::create($option, array_keys($matrix))`) based on given options 164 | - run: StaticContent\CommandFactory::build 165 | - get excluded themes: `SCD_EXCLUDE_THEMES` from `.magento.env.yaml` (deprecated) 166 | - exclude themes from `SCD_MATRIX` 167 | - add unique locales [StaticContent\Build\Option::getLocales](https://github.com/magento/ece-tools/blob/develop/src/StaticContent/Build/Option.php#L124) 168 | - `ADMIN_LOCALE` from (MAGENTO_CLOUD_VARIABLES) (default: en_US) 169 | - `general/locale/code` from `app/etc/config.php` 170 | - `admin_user/locale/code` from `app/etc/config.php` 171 | - add `en_US` 172 | - result example: `php ./bin/magento setup:static-content:deploy --ansi --no-interaction -f --jobs 3 --no-html-minify --exclude-theme Magento/blank --exclude-theme Magento/luma en_US nb_NO` 173 | - generate commands based on `SCD_MATRIX` from `.magento.env.yaml` 174 | - run: StaticContent\CommandFactory::build 175 | - add ` --theme {$resolvedTheme}` 176 | - add `language` from `SCD_MATRIX` (en_US, nb_NO) 177 | - result example: `php ./bin/magento setup:static-content:deploy --ansi --no-interaction -f --jobs 3 --no-html-minify --theme Magento/blank en_US nb_NO` 178 | - execute all generated scd commands 179 | - save `.static_content_deploy` flag file 180 | 181 | > notice: Generate command completed. 182 | 183 | `php ./vendor/bin/ece-tools build:transfer` 184 | 185 | > notice: Starting transfer files. 186 | 187 | - [Process\Build\CompressStaticContent](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/CompressStaticContent.php) 188 | - check `.static_content_deploy` flag file 189 | - [Util\StaticContentCompressor::process](https://github.com/magento/ece-tools/tree/develop/src/Util/StaticContentCompressor.php) 190 | - get gzip compression level in `SCD_COMPRESSION_LEVEL` (default: 6 (build stage) or 4 (deploy stage)) from `.magento.env.yaml` 191 | - get seconds maximum time for running static compression command in `SCD_COMPRESSION_TIMEOUT` (default: 600) from `.magento.env.yaml` 192 | - get `VERBOSE_COMMANDS` from `.magento.env.yaml` 193 | - command result: `/usr/bin/timeout -k 30 600 /bin/bash -c 'find '\''/var/magento/pub/static'\'' -type d -name '\''DELETING_*'\'' -prune -o -type f -size +300c '\''('\'' -name '\''*.js'\'' -or -name '\''*.css'\'' -or -name '\''*.svg'\'' -or -name '\''*.html'\'' -or -name '\''*.htm'\'' '\'')'\'' -print0 | xargs -0 -n100 -P16 gzip -q --keep -6'` 194 | - [Process\Build\ClearInitDirectory](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/ClearInitDirectory.php) 195 | - clear temporary directory: `./init/` 196 | - delete `app/etc/env.php` file 197 | - [Process\Build\BackupData](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/BackupData.php) 198 | - [Process\Build\BackupData\StaticContent](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/BackupData/StaticContent.php) 199 | - delete `var/.regenerate` flag file 200 | - check `.static_content_deploy` flag file 201 | - clear `./init/pub/static/` folder 202 | - move `pub/static/` to `init/pub/static` 203 | - [Process\Build\BackupData\WritableDirectories](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/BackupData/WritableDirectories.php) 204 | - copy `app/etc/` to `init/app/etc/` 205 | - copy `pub/media/` to `init/pub/media/` 206 | - copy `var/view_preprocessed/` to `init/var/view_preprocessed/` if `SKIP_HTML_MINIFICATION` = false (Default: true) 207 | - copy `var/log` to `init/var/log` (e.g. cloud.log file) 208 | 209 | > notice: Transfer completed. 210 | 211 | 212 | **Deploy** 213 | 214 | `php ./vendor/bin/ece-tools deploy` 215 | 216 | > notice: Starting deploy. (magento/ece-tools version: 2002.0.17, magento/magento2-base version: 2.2.8) 217 | 218 | - delete `var/.deploy_is_failed` flag file 219 | - [Process\Deploy\PreDeploy](https://github.com/magento/ece-tools/blob/develop/src/Process/Deploy/PreDeploy.php) 220 | - Runs all processes that have to be run before deploy starting. see [Process\Deploy\PreDeploy](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/PreDeploy) folders 221 | - Process\PreDeploy\ConfigUpdate\Cache 222 | - `CACHE_CONFIGURATION` from `.magento.env.yaml` 223 | - Process\Deploy\PreDeploy\CleanStaticContent 224 | - check `.static_content_deploy` flag file exists 225 | - `CLEAN_STATIC_FILES` is true 226 | - clean `pub/static/` folder 227 | - Process\Deploy\PreDeploy\CleanViewPreprocessed 228 | - clean `var/view_preprocessed` directory when the deployment variable `SKIP_HTML_MINIFICATION` is true 229 | - Process\Deploy\PreDeploy\CleanRedisCache 230 | - `redis-cli -h $redisHost -p $redisPort -n $redisCacheDb flushdb` ($redisCacheDb = 1) 231 | - Process\Deploy\PreDeploy\CleanFileCache 232 | - delete `/var/cache` folder 233 | - Process\Deploy\PreDeploy\RestoreWritableDirectories 234 | - see [Filesystem\RecoverableDirectoryList::getList](https://github.com/magento/ece-tools/blob/develop/src/Filesystem/RecoverableDirectoryList.php#L83) 235 | - copy `init/app/etc` to `app/etc` 236 | - copy `init/pub/media` to `pub/media` 237 | - copy (symlink) `init/pub/static` to `pub/static` 238 | - delete `var/.regenerate` flag file 239 | - Process\Deploy\PreDeploy\SetProductionMode 240 | - switching magento to production mode (`app/etc/env.php` => `'MAGE_MODE' => 'production',`) 241 | - Enabling Maintenance mode: `php ./bin/magento maintenance:enable --ansi --no-interaction` 242 | - [Process\Deploy\DisableCron](https://github.com/magento/ece-tools/blob/develop/src/Process/Deploy/DisableCron.php) 243 | - disable cron (`app/etc/env.php` => `['cron' => ['enabled' => 0]]`) 244 | - [Process\ValidateConfiguration](https://github.com/magento/ece-tools/tree/develop/src/Process/ValidateConfiguration.php) 245 | ```php 246 | $this->container->make(\Magento\MagentoCloud\Process\ValidateConfiguration::class, [ 247 | 'validators' => [ 248 | ValidatorInterface::LEVEL_CRITICAL => [ 249 | $this->container->make(ConfigValidator\Deploy\DatabaseConfiguration::class), 250 | $this->container->make(ConfigValidator\Deploy\ResourceConfiguration::class), 251 | $this->container->make(ConfigValidator\Deploy\SessionConfiguration::class), 252 | ], 253 | ValidatorInterface::LEVEL_WARNING => [ 254 | $this->container->make(ConfigValidator\Deploy\AdminData::class), 255 | $this->container->make(ConfigValidator\Deploy\PhpVersion::class), 256 | $this->container->make(ConfigValidator\Deploy\SearchEngine::class), 257 | $this->container->make(ConfigValidator\Deploy\ElasticSearchUsage::class), 258 | $this->container->make(ConfigValidator\Deploy\ElasticSearchVersion::class), 259 | $this->container->make(ConfigValidator\Deploy\AppropriateVersion::class), 260 | $this->container->make(ConfigValidator\Deploy\ScdOptionsIgnorance::class), 261 | $this->container->make(ConfigValidator\Deploy\DeprecatedVariables::class), 262 | $this->container->make(ConfigValidator\Deploy\RawEnvVariable::class), 263 | $this->container->make(ConfigValidator\Deploy\MagentoCloudVariables::class), 264 | $this->container->make(ConfigValidator\Deploy\JsonFormatVariable::class), 265 | ], 266 | ], 267 | ]), 268 | ``` 269 | validators list: [Config/Validator/Deploy/](https://github.com/magento/ece-tools/tree/develop/src/Config/Validator/Deploy) 270 | - [Process\Deploy\UnlockCronJobs](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/UnlockCronJobs.php) 271 | - In magento version 2.2 was implemented locking functionality for cron jobs, new cron jobs can't be started if exist job in status 'running' with same 'job_code'. 272 | - ```UPDATE `cron_schedule` SET `status` = 'error', `messages` = 'The job is terminated due to system upgrade' WHERE `status` = 'running'``` 273 | - [Process\Deploy\SetCryptKey](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/SetCryptKey.php) 274 | - update crypt/key in `app/etc/env.php` with `CRYPT_KEY` variable value (MAGENTO_CLOUD_VARIABLES) 275 | - [Process\Deploy\InstallUpdate](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/InstallUpdate.php) 276 | - magento not installed 277 | - see [Process/Deploy/InstallUpdate/Install](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/InstallUpdate/Install) 278 | - Process\Deploy\InstallUpdate\Install\Setup: `php ./bin/magento setup:install -n --session-save=db --cleanup-database` 279 | - run `php ./bin/magento app:config:import --ansi --no-interaction` 280 | - sends email with link to reset password 281 | - magento already installed 282 | - see [Process/Deploy/InstallUpdate/Update](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/InstallUpdate/Update) 283 | - sets an admin URL from `ADMIN_URL` variable value (MAGENTO_CLOUD_VARIABLES) (Process\Deploy\InstallUpdate\Update\SetAdminUrl) 284 | - Running setup upgrade:`php ./bin/magento setup:upgrade --keep-generated --ansi --no-interaction` (Process\Deploy\InstallUpdate\Update\Setup) 285 | - Updating configuration from environment variables (DeployProcess\InstallUpdate\ConfigUpdate) 286 | - see [Process/Deploy/InstallUpdate/ConfigUpdate](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/InstallUpdate/ConfigUpdate) 287 | ```php 288 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\PrepareConfig::class), 289 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\CronConsumersRunner::class), 290 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\DbConnection::class), 291 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\Amqp::class), 292 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\Session::class), 293 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\SearchEngine::class), 294 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\Urls::class), 295 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\DocumentRoot::class), 296 | 297 | $this->container->when(DeployProcess\InstallUpdate\ConfigUpdate\Urls::class) 298 | ->needs(ProcessInterface::class) 299 | ->give(function () { 300 | return $this->container->makeWith(ProcessComposite::class, [ 301 | 'processes' => [ 302 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\Urls\Database::class), 303 | $this->container->make(DeployProcess\InstallUpdate\ConfigUpdate\Urls\Environment::class), 304 | ], 305 | ]); 306 | }); 307 | ``` 308 | - [Process\Deploy\DeployStaticContent](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/DeployStaticContent.php) 309 | - clean `pub/static` and `var/view_preprocessed` if `SCD_ON_DEMAND` = true 310 | - skip this step if `SKIP_SCD` = true or `.static_content_deploy` file flag exists 311 | - clean `pub/static` and `var/view_preprocessed` if `CLEAN_STATIC_FILES` = true 312 | - Generating fresh static content 313 | - see [Process\Deploy\DeployStaticContent\Generate](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/DeployStaticContent/Generate.php) 314 | - see build SCD commands [StaticContent\CommandFactory::matrix](https://github.com/magento/ece-tools/blob/develop/src/StaticContent/CommandFactory.php#L98) 315 | - [Process\Deploy\CompressStaticContent](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/CompressStaticContent.php) 316 | - skip if `SCD_ON_DEMAND` = true 317 | - skip if exists `.static_content_deploy` flag 318 | - use [Util\StaticContentCompressor::process](https://github.com/magento/ece-tools/tree/develop/src/Util/StaticContentCompressor.php) 319 | - [Process\Deploy\DisableGoogleAnalytics](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/DisableGoogleAnalytics.php) 320 | - if `ENABLE_GOOGLE_ANALYTICS` = false (default: false) in the `.magento.env.yaml` 321 | - if not master branch (`/^(master|production|staging)(?:-[a-z0-9]+)?$/i`) (check `MAGENTO_CLOUD_ENVIRONMENT` env variable) 322 | - execute ```UPDATE `core_config_data` SET `value` = 0 WHERE `path` = 'google/analytics/active'``` 323 | - [Process\Deploy\DeployCompletion](https://github.com/magento/ece-tools/tree/develop/src/Process/Deploy/DeployCompletion.php) 324 | - checks is `post_deploy` hook enabled in `.magento.app.yaml` 325 | - runs processes if only post_deploy hook is not configured 326 | ```php 327 | $this->container->when(DeployProcess\DeployCompletion::class) 328 | ->needs(ProcessInterface::class) 329 | ->give(function () { 330 | return $this->container->makeWith(ProcessComposite::class, [ 331 | 'processes' => [ 332 | $this->container->make(PostDeployProcess\EnableCron::class), 333 | $this->container->make(PostDeployProcess\Backup::class), 334 | $this->container->make(PostDeployProcess\CleanCache::class), 335 | ], 336 | ]); 337 | }); 338 | ``` 339 | - Disable maintenance mode: `php ./bin/magento maintenance:disable --ansi --no-interaction` 340 | 341 | > notice: Deployment completed. 342 | 343 | **Post_deploy** 344 | 345 | `php ./vendor/bin/ece-tools post-deploy` 346 | 347 | > notice: Starting post-deploy. 348 | 349 | - check `var/.deploy_is_failed` flag file and skip post-deploy 350 | - [Config\Validator\Deploy\DebugLogging](https://github.com/magento/ece-tools/blob/develop/src/Config/Validator/Deploy/DebugLogging.php) 351 | - check `dev/debug/debug_logging` is disabled in production environments 352 | - [Process\PostDeploy\EnableCron](https://github.com/magento/ece-tools/blob/develop/src/Process/PostDeploy/EnableCron.php) 353 | - enable cron (`app/etc/env.php` => `unset($config['cron']['enabled']);`) 354 | - [Process\PostDeploy\Backup](https://github.com/magento/ece-tools/blob/develop/src/Process/PostDeploy/Backup.php) 355 | - \Magento\MagentoCloud\Filesystem\BackupList::getList 356 | - copy `app/etc/env.php` to `app/etc/env.php.bak` 357 | - copy `app/etc/config.php` to `app/etc/env.php.bak` 358 | - [Process\PostDeploy\CleanCache](https://github.com/magento/ece-tools/blob/develop/src/Process/PostDeploy/CleanCache.php) 359 | - execute: `php ./bin/magento cache:flush --ansi --no-interaction` 360 | - [Process\PostDeploy\WarmUp](https://github.com/magento/ece-tools/blob/develop/src/Process/PostDeploy/WarmUp.php) 361 | - Process\PostDeploy\WarmUp::getUrlsForWarmUp 362 | - call async urls from `WARM_UP_PAGES` in the `.magento.env.yaml` 363 | - use `web/unsecure/base_url` and `web/secure/base_url` from `core_config_data` table 364 | 365 | > notice: Post-deploy is complete. 366 | 367 | ###### How to extend these scripts and best practices for doing so 368 | 369 | ###### Describe the ways to retrieve logs for phases and its scripts 370 | 371 | **Build logs** 372 | 373 | Logs from the build hook are redirected to the output stream of git push. 374 | 375 | `var/log/cloud.log` file inside the Magento application root directory, compiles build, deploy, post-deploy actions into one file. 376 | 377 | **Deploy logs** 378 | 379 | `var/log/cloud.log` file inside the Magento application root directory, compiles build, deploy, post-deploy actions into one file. 380 | 381 | Logs from the deploy hook are located on the server in the following locations: 382 | - Integration: /var/log/deploy.log 383 | - Staging: /var/log/platform/_stg/deploy.log 384 | - Production: /var/log/platform//deploy.log 385 | 386 | **Post Deploy logs** 387 | 388 | `var/log/cloud.log` file inside the Magento application root directory, compiles build, deploy, post-deploy actions into one file. 389 | 390 | Logs from the post-deploy hook are located on the server in the following locations: 391 | - Integration: /var/log/post-deploy.log 392 | - Staging: /var/log/platform/_stg/post_deploy.log 393 | - Production: /var/log/platform//post_deploy.log 394 | 395 | 396 | > Note1: 397 | > `var/log/cloud.log` logs on the Staging and Production environments are only available on the first node in the cluster. 398 | 399 | > Note2: 400 | > Logs in the /var/log/ directory are not shared between nodes of the enterprise server cluster; 401 | > each server has its own log. We recommend checking logs on every node for complete reporting. 402 | 403 | 404 | #### 5.2 Demonstrate the ability to create Magento Cloud script configurations 405 | 406 | Documentation: 407 | - [Deployment process](https://devdocs.magento.com/guides/v2.3/cloud/reference/discover-deploy.html#cloud-deploy-over-phases) 408 | - [Build and deploy / platform.sh](https://docs.platform.sh/configuration/app/build.html#build-hook) 409 | - [View logs for troubleshooting](https://devdocs.magento.com/guides/v2.3/cloud/trouble/environments-logs.html) 410 | - [Log locations (directories) for Pro plan: Integration, Staging, Production](https://support.magento.com/hc/en-us/articles/360000318834-Log-locations-directories-for-Pro-plan-Integration-Staging-Production) 411 | -------------------------------------------------------------------------------- /6. Static Content Deployment.md: -------------------------------------------------------------------------------- 1 | # Static Content Deployment 2 | 3 | #### 6.1 Demonstrate ability to move SCD to build phase 4 | 5 | ###### Describe the default process of generating SCD and how it impacts downtime 6 | 7 | The default strategy generates static content during the deploy phase when the site is in maintenance mode; 8 | This deployment strategy takes time to write the content directly to the mounted pub/static directory. 9 | 10 | ###### Describe the reason for moving generation static content to the build phase. Consider the way to do this and show the result (timeline) 11 | 12 | > Generating static content during the build phase with minified HTML is the optimal configuration for zero-downtime deployments, also known as the ideal state. Instead of copying files to a mounted drive, it creates a symlink from the ./init/pub/static directory. 13 | > 14 | > By default, the STATIC_CONTENT_SYMLINK environment variable is set to true. After generating the static content during the build phase, it creates a symlink to the content folder. 15 | > 16 | > Generating static content requires access to themes and locales. Magento stores themes in the file system, which is accessible during the build phase; however, Magento stores locales in the database. 17 | > The database is not available during the build phase. In order to generate the static content during the build phase, you must use the config:dump command in the package to move locales to the file system. It reads the locales and saves them in the app/etc/config.php file. 18 | 19 | To configure your project to generate SCD on build: 20 | 21 | 22 | 23 | Verify your settings by running the Smart wizard for the ideal state. 24 | `php ./vendor/bin/ece-tools wizard:ideal-state` 25 | 26 | ###### Determine additional configuration that helps decrease deployment time (SKIP_HTML_MINIFICATION) 27 | 28 | `SKIP_HTML_MINIFICATION` in the `.magento.env.yaml` file (Default: true, for ece-tools 2002.0.13 and later). 29 | 30 | > Enables or disables copying static view files to the /init/ directory at the end of the build stage. 31 | > If set to true files are not copied and HTML minification is available on request. 32 | > Set this value to true to reduce downtime when deploying to Staging and Production environments. 33 | 34 | > You can improve the SCD load time during the deployment process if you skip copying the static view files in the var/view_preprocessed directory and generate minified HTML when requested 35 | 36 | `SCD_THREADS` - Sets the number of threads for static content deployment. Increasing the number of threads speeds up static content deployment; decreasing the number of threads slows it down. 37 | 38 | `SCD_MATRIX` - You can configure multiple locales per theme as long as the theme is not excluded using the SCD_EXCLUDE_THEMES variable during build. This is ideal if you want to speed up the build process by reducing the amount of unnecessary theme files. For example, you can build the magento/backend theme in English and a custom theme in other languages. 39 | 40 | ```yaml 41 | stage: 42 | build: 43 | SCD_MATRIX: 44 | "magento/backend": 45 | language: 46 | - en_US 47 | - nb_NO 48 | # not deploy a theme: 49 | "magento/blank": [ ] 50 | ``` 51 | 52 | `STATIC_CONTENT_SYMLINK` (Default: true) Generates symlinks for static content. This setting is vital in the Pro Production environment for the three-node cluster. 53 | When this variable is set to false, it must copy every file during the deployment, which increases deployment time. Setting SCD_ON_DEMAND to true disables this variable. 54 | 55 | If you generate static content during the build phase, it creates a symlink to the content folder. 56 | If you generate static content during the deploy phase, it writes directly to the content folder. 57 | Generating static content on demand disables this variable. 58 | 59 | ###### Display time measurements 60 | 61 | ![csd load](https://devdocs.magento.com/common/images/cloud/scd-load-times.png) 62 | 63 | #### 6.2 Demonstrate ability to avoid SCD on both phases 64 | 65 | ###### What are causes for avoiding SCD 66 | ###### Describe the way to do this in all cases, and the expected result 67 | 68 | `SKIP_SCD` in the `.magento.env.yaml` - skips static content deployment during the build/deploy phases 69 | 70 | ```yaml 71 | stage: 72 | build: 73 | SKIP_SCD: true 74 | # or 75 | deploy: 76 | SKIP_SCD: true 77 | ``` 78 | 79 | #### 6.3 Describe how to generate static content on demand 80 | 81 | ###### Describe the default Magento behavior in Production mode and why the new "mode" was added 82 | ###### How does Magento behave when it is set to this "mode"? 83 | ###### Know when users can use this configuration and how it works from the Cloud side 84 | ###### Display time measurements 85 | 86 | `SCD_ON_DEMAND` in the `.magento.env.yaml` file (Default: false) enable generation of static content when requested by a user 87 | 88 | > Generating SCD on demand is optimal for a development workflow in the Integration environment. 89 | > It decreases deployment time so that you can quickly review your implementations and run integration tests. 90 | > Enable the SCD_ON_DEMAND environment variable in the global stage of the .magento.env.yaml file. 91 | > The SCD_ON_DEMAND variable overrides all other configurations related to SCD and clears existing content from the ~/pub/static directory. 92 | > 93 | > When using the SCD on-demand strategy, it helps to preload the cache with pages you expect to request, such as the home page. 94 | > Add your list of expected pages in the WARM_UP_PAGES environment variable in the post-deploy stage of the .magento.env.yaml file. 95 | > (https://devdocs.magento.com/guides/v2.3/cloud/deploy/static-content-deployment.html) 96 | 97 | 98 | > The SCD_ON_DEMAND variable skips the SCD and the STATIC_CONTENT_SYMLINK in both phases (build and deploy), 99 | > clears the pub/static and var/view_preprocessed folders, and writes the following to the app/etc/env.php file: 100 | ```php 101 | return array( 102 | ... 103 | 'static_content_on_demand_in_production' => 1, 104 | ... 105 | ); 106 | ``` 107 | 108 | Documentation: 109 | - [Cloud deployment process](https://devdocs.magento.com/guides/v2.3/cloud/deploy/cloud-deployment-process.html) 110 | - [Reduce downtime](https://devdocs.magento.com/guides/v2.3/cloud/deploy/reduce-downtime.html) 111 | -------------------------------------------------------------------------------- /7. Development.md: -------------------------------------------------------------------------------- 1 | # Development 2 | 3 | #### 7.1 Demonstrate ability to change configurations 4 | 5 | ###### What are the sources of Magento configuration, and which priorities have different sources of Magento configuration? 6 | 7 | All system configurations are set during deployment according to the following override scheme: 8 | 9 | 1. If an environment variable exists, use the custom configuration and ignore the default configuration. 10 | 2. If an environment variable does not exist, use the configuration from a `MAGENTO_CLOUD_RELATIONSHIPS` name-value pair in the `.magento.app.yaml` file. Ignore the default configuration. 11 | 3. If an environment variable does not exist and `MAGENTO_CLOUD_RELATIONSHIPS` does not contain a name-value pair, remove all customized configuration and use the values from the default configuration. 12 | 13 | 14 | The general format of system settings variable names follows:`__` 15 | 16 | `` can be either: 17 | - Global scope (that is, the global setting for all scopes) 18 | Global scope variables have the following format: `CONFIG__DEFAULT__` 19 | - A specific scope (that is, the setting affects only a specified store view or website) 20 | Store view scope variables, for example, have the following format: `CONFIG__STORES__ __` 21 | 22 | `` is derived from a configuration setting’s configuration path, which is a / delimited string that uniquely identifies a particular setting. 23 | Replace each / character in the configuration path with two underscore characters to create the system variable. 24 | 25 | Example: 26 | ```yaml 27 | variables: 28 | env: 29 | CONFIG__DEFAULT__PAYPAL_ONBOARDING__MIDDLEMAN_DOMAIN: 'payment-broker.magento.com' 30 | CONFIG__STORES__DEFAULT__PAYMENT__BRAINTREE__CHANNEL: 'Magento_Enterprise_Cloud_BT' 31 | CONFIG__STORES__DEFAULT__PAYPAL__NOTATION_CODE: 'Magento_Enterprise_Cloud' 32 | ``` 33 | 34 | [Use environment variables to override configuration settings](https://devdocs.magento.com/guides/v2.2/config-guide/prod/config-reference-var-name.html) 35 | 36 | #### 7.2 Demonstrate ability to change a locale 37 | 38 | ###### Know how to change a locale on Cloud 39 | 40 | > Any data that exports to the file becomes locked. 41 | > The corresponding field in the Magento Admin becomes read-only. 42 | > This ensures consistent configurations as you push the file across all environments. And every time you run this command, any new configurations are appended to your config.php file. 43 | > If you need to modify or delete an existing configuration, you must edit the file manually. 44 | 45 | update `app/etc/config.php` file, commit and push to Git. 46 | 47 | [Example of managing system-specific settings](https://devdocs.magento.com/guides/v2.3/cloud/live/sens-data-initial.html) 48 | 49 | #### 7.3 Demonstrate ability to add extensions 50 | 51 | ###### Know how to install Magento extensions and themes (limitations, read-only filesystem, etc.) 52 | 53 | - Install module: `composer require :` 54 | > When adding the module to composer.json, the file app/etc/config.php will also be updated. 55 | > This file includes a list of installed modules, themes, and language packages, shared configuration settings. 56 | - Verify the extension: `php bin/magento module:status` 57 | - Commit and push changes: 58 | ```bash 59 | git add -A 60 | git commit -m "" 61 | git push magento 62 | ``` 63 | 64 | [Install, manage, and upgrade extensions](https://devdocs.magento.com/guides/v2.3/cloud/howtos/install-components.html) 65 | 66 | #### 7.4 Demonstrate ability to enable / disable a module 67 | 68 | ###### Know how to enable or disable a module on Cloud 69 | 70 | Enable a module. This command updates the config.php file with the enabled status of the module. 71 | `php bin/magento module:enable ` 72 | 73 | Disable a module. This command updates the config.php file with the disable status of the module. 74 | `php bin/magento module:disable ` 75 | 76 | Commit `app/etc/config.php` file and push to Git. 77 | 78 | #### 7.5 Demonstrate ability to set up a multisite configuration 79 | 80 | ###### Know how to setup multisite configuration: Adding and configuring new websites in Magento; Nginx configuration through the .magento.app.yaml for multisite setup; how to route websites through the magento-vars.php 81 | 82 | - Configure routes in `.magento/routes.yaml` 83 | 84 | > For Pro, you must create a Support ticket to set up routes in the Staging or Production environment. 85 | 86 | ```yaml 87 | "http://{default}/": 88 | type: upstream 89 | upstream: "mymagento:php" 90 | 91 | "https://{default}/": 92 | type: upstream 93 | upstream: "mymagento:php" 94 | 95 | "http://*.{default}/": 96 | type: upstream 97 | upstream: "mymagento:php" 98 | 99 | "https://*.{default}/": 100 | type: upstream 101 | upstream: "mymagento:php" 102 | ``` 103 | 104 | - Set up websites, stores, and store views 105 | [Set up multiple websites, stores, and store views in the Admin](https://devdocs.magento.com/guides/v2.3/config-guide/multi-site/ms_websites.html) 106 | 107 | 108 | - Modify the `magento-vars.php` file 109 | > Instead of configuring an NGINX virtual host, pass the MAGE_RUN_CODE and MAGE_RUN_TYPE variables using the magento-vars.php file located in your project root directory. 110 | 111 | 112 | ```php 113 | . 152 | For example, http://french.master-name-projectID.us.magentosite.cloud/ 153 | 154 | [Set up multiple websites or stores](https://devdocs.magento.com/guides/v2.3/cloud/project/project-multi-sites.html) 155 | 156 | #### 7.6 Demonstrate ability to use variables 157 | 158 | ###### When do you need to use variables; which configurations you can change using variables 159 | 160 | [Cloud variables](https://devdocs.magento.com/guides/v2.3/cloud/env/variables-cloud.html) 161 | [ADMIN variables](https://devdocs.magento.com/guides/v2.3/cloud/env/environment-vars_magento.html) 162 | 163 | example: 164 | - ADMIN_USERNAME 165 | - ADMIN_EMAIL 166 | - ADMIN_PASSWORD 167 | - ADMIN_LOCALE 168 | - ADMIN_URL 169 | - CRON_CONSUMERS_RUNNER 170 | - CRYPT_KEY 171 | - UPDATE_URLS 172 | 173 | Note: it's possible to override values from `.magento.env.yaml` from deploy stage. 174 | see: 175 | - [Config\Stage\Deploy::get](https://github.com/magento/ece-tools/blob/develop/src/Config/Stage/Deploy.php#L47) 176 | - [Config\Stage\Deploy\MergedConfig::get](https://github.com/magento/ece-tools/blob/develop/src/Config/Stage/Deploy/MergedConfig.php#L69) 177 | - [Config\Stage\Deploy\EnvironmentConfig::getAll](https://github.com/magento/ece-tools/blob/develop/src/Config/Stage/Deploy/EnvironmentConfig.php) 178 | 179 | ###### What is the difference between variables and environment variables 180 | 181 | Variables from `MAGENTO_CLOUD_VARIABLES` (base64 encoded json) env variable (common variable in admin panel) 182 | Environment variables have prefix `env:` in admin panel 183 | > When attempting to use environment variables to override configuration settings using the Project Web Interface, you must prepend the variable name with env: as in the following example. 184 | ![env-variable](https://devdocs.magento.com/common/images/cloud_env_var_example.png) 185 | 186 | ###### What is the difference between project and environment level variables 187 | 188 | **Project-specific variables** 189 | To set the same value for all environments in your project, use the magento-cloud project:variable:set command. 190 | These variables are available at build and runtime in all environments. 191 | 192 | **Environment-specific variables** 193 | To set a unique value for a specific environment, use the magento-cloud variable:set command. These variables are available at runtime and are inherited by child environments. 194 | You should specify the environment in your command using the -e option. 195 | 196 | 197 | - [Working with variables](https://devdocs.magento.com/guides/v2.3/cloud/env/working-with-variables.html) 198 | 199 | Documentation: 200 | - [Configuration management for store settings](https://devdocs.magento.com/guides/v2.3/cloud/live/sens-data-over.html) 201 | - [Working with variables](https://devdocs.magento.com/guides/v2.3/cloud/env/working-with-variables.html) 202 | - [Set up multiple websites, stores, and store views in the Admin](https://devdocs.magento.com/guides/v2.3/config-guide/multi-site/ms_websites.html) 203 | - [Install, manage, and upgrade extensions](https://devdocs.magento.com/guides/v2.3/cloud/howtos/install-components.html) 204 | - [Use environment variables to override configuration settings](https://devdocs.magento.com/guides/v2.2/config-guide/prod/config-reference-var-name.html) 205 | -------------------------------------------------------------------------------- /8. Troubleshooting.md: -------------------------------------------------------------------------------- 1 | # Troubleshooting 2 | 3 | #### 8.1 Demonstrate ability to locate and use logs 4 | 5 | ###### Locate the Magento application logs 6 | 7 | `var/log/` 8 | 9 | > Pro projects have a three-node structure and each node contains logs with specific information for that node. 10 | 11 | > Logs in var/log on Staging and Production: separate log for every node 12 | > 13 | > Logs in the var/log/ directory are not shared between nodes of the enterprise server cluster; each server has its own log. 14 | > We recommend checking logs on every node for complete reporting. 15 | 16 | ###### Locate system services logs on integration and Starter environments 17 | 18 | [Log locations (directories) for Starter plan](https://support.magento.com/hc/en-us/articles/360020127552) 19 | 20 | - Build log => var/log/cloud.log 21 | - Debug log => var/log/debug.log 22 | - Exception log => var/log/exception.log 23 | - Report logs (reports) => var/reports 24 | - Support report log => var/log/support_report.log 25 | - Deployment log => /var/log/deploy.log 26 | - Cron log => /var/log/cron.log 27 | - Nginx access log => /var/log/access.log 28 | - Nginx error log => /var/log/error.log 29 | - PHP access log => /var/log/php.access.log 30 | - PHP FPM log => /var/log/app.log 31 | - Redis log => No access 32 | - Elasticseach log => No access 33 | - Mail log => No access 34 | - MySQL error log => No access 35 | - MySQL slow log => No access 36 | - RabbitMQ log => No access 37 | - Solr log => No access 38 | - Syslog => No access 39 | 40 | ###### Locate system services logs on Pro environments 41 | 42 | [Log locations (directories) for Pro plan: Integration, Staging, Production](https://support.magento.com/hc/en-us/articles/360000318834-Log-locations-directories-for-Pro-plan-Integration-Staging-Production) 43 | 44 | > Integration => Staging => Production 45 | - Build log => var/log/cloud.log 46 | - Debug log => var/log/debug.log 47 | - Exception log => var/log/exception.log 48 | - Report logs (reports) => var/reports 49 | - Support report log => var/log/support_report.log 50 | - Deployment log => /var/log/deploy.log => /var/log/platform/{project_id}_stg/deploy.log => /var/log/platform/{project_id}/deploy.log 51 | - Cron log => /var/log/cron.log => /var/log/platform/{project_id}_stg/cron.log => var/log/platform/{project_id}/cron.log 52 | - Nginx access log => /var/log/access.log => /var/log/platform/{project_id}_stg/access.log => /var/log/platform/{project_id}/access.log 53 | - Nginx error log => /var/log/error.log => /var/log/platform/{project_id}_stg/error.log => /var/log/platform/{project_id}/error.log 54 | - PHP access log => /var/log/php.access.log => /var/log/platform/{project_id}_stg/php.access.log => /var/log/platform/{project_id}/php.access.log 55 | - PHP FPM log => /var/log/app.log => /var/log/platform/{project_id}_stg/php5-fpm.log => /var/log/platform/{project_id}/php5-fpm.log 56 | - Redis log => No access => /var/log/platform/{project_id}_stg/redis-server-{project_id}_stg.log => /var/log/platform/{project_id}/redis-server-{project_id}.log 57 | - Elasticseach log => No access => /var/log/elasticsearch/elasticsearch.log 58 | - Mail log => No access => /var/log/mail.log 59 | - MySQL error log => No access => /var/log/mysql/mysql-error.log 60 | - MySQL slow log => No access => /var/log/mysql/mysql-slow.log 61 | - RabbitMQ log => No access => /var/log/rabbitmq/rabbit@host1.log 62 | - Solr log => No access => /var/log/platform/solr.log 63 | - Syslog => No access 64 | 65 | #### 8.2 Demonstrate ability to create snapshots and backups 66 | 67 | #### 8.3 Demonstrate ability to debug 68 | 69 | ###### How to use XDebug on Cloud 70 | 71 | - Add xdebug extension 72 | > For Pro Staging and Production, you need to update this file and enter a Support ticket to have it enabled 73 | 74 | ```yaml 75 | #.magento.app.yaml 76 | runtime: 77 | extensions: 78 | - mcrypt 79 | - redis 80 | - xsl 81 | - json 82 | - xdebug 83 | ``` 84 | 85 | - Optionally, modify the timeout. A default timeout of 300 seconds (5 minutes) 86 | ```yaml 87 | web: 88 | commands: 89 | start: | 90 | cat /etc/php/7.0/fpm/php-fpm.conf | sed -e 's/request_terminate_timeout.*//g' > /tmp/php-fpm.conf 91 | /usr/sbin/php-fpm7.0 -y /tmp/php-fpm.conf 92 | ``` 93 | 94 | - Configure PhpStorm 95 | 96 | - Set up port forwarding 97 | `ssh -R 9000:localhost:9000 ` 98 | 99 | [Configure Xdebug](https://devdocs.magento.com/guides/v2.3/cloud/howtos/debug.html) 100 | 101 | **Pro staging and production** 102 | 103 | Set up the tunnels, this will connect to localhost on port 9000 104 | 105 | Three tunnels are needed because we never know which host the request will hit. The rm commands are necessary to clean up from past sessions. 106 | 107 | ``` 108 | ssh USERNAME@1.CLUSTER.ent.magento.cloud 'rm /run/platform/USERNAME/xdebug.sock' 109 | ssh -R /run/platform/USERNAME/xdebug.sock:localhost:9000 -N USERNAME@1.CLUSTER.ent.magento.cloud& 110 | 111 | ssh USERNAME@2.CLUSTER.ent.magento.cloud 'rm /run/platform/USERNAME/xdebug.sock' 112 | ssh -R /run/platform/USERNAME/xdebug.sock:localhost:9000 -N USERNAME@2.CLUSTER.ent.magento.cloud& 113 | 114 | ssh USERNAME@3.CLUSTER.ent.magento.cloud 'rm /run/platform/USERNAME/xdebug.sock' 115 | ssh -R /run/platform/USERNAME/xdebug.sock:localhost:9000 -N USERNAME@3.CLUSTER.ent.magento.cloud& 116 | ``` 117 | 118 | Next to enable remote debugging, visit the site in the browser, add ?XDEBUG_SESSION_START=KEY to the url, where KEY is the key set in the yaml file. 119 | This will set the cookie causing browser requests to trigger xdebug. ?XDEBUG_SESSION_STOP=KEY can be used to remove the cookie, stopping the session. 120 | 121 | XDEBUG_SESSION_START passed via POST requests is not supported, the cookie must first be set either manually or via a GET request. 122 | 123 | 124 | **CLI** 125 | ssh 1.ent-{project_id}@ssh.eu.magentosite.cloud 126 | 127 | # lean up from past sessions. 128 | rm /run/platform/{project_id}_stg/xdebug.sock 129 | 130 | ssh -R /run/platform/{project_id}_stg/xdebug.sock:localhost:9000 -N 1.ent-{project_id}@ssh.eu.magentosite.cloud 131 | 132 | # configure phpstorm https://devdocs.magento.com/guides/v2.2/cloud/howtos/debug.html#phpstorm 133 | export XDEBUG_CONFIG='PHPSTORM' 134 | export PHP_IDE_CONFIG="serverName=magento-cloud-staging" 135 | 136 | php -c /etc/platform/{project_id}_stg/php.xdebug.ini ./bin/magento 137 | 138 | 139 | #### 8.4 Demonstrate ability to apply Magento fixes in patches 140 | 141 | > Sometimes we provide a custom patch to address a specific issue. 142 | > Also, third-party extension developers can provide a custom patch. 143 | > Copy the custom patch to the `m2-hotfixes/` directory and test it on your local workstation. 144 | > 145 | > [Apply custom patches](https://devdocs.magento.com/guides/v2.3/cloud/project/project-patch.html) 146 | 147 | `php ./vendor/bin/ece-tools build:generate` will apply custom patches during build phase 148 | - [Process\Build\ApplyPatches](https://github.com/magento/ece-tools/tree/develop/src/Process/Build/ApplyPatches.php) 149 | - `git apply` patches from `m2-hotfixes/*.patch` 150 | 151 | #### 8.5 Describe branch synchronization 152 | 153 | #### Describe branch synchronization and merge 154 | 155 | Documentation: 156 | - [View logs for troubleshooting](https://devdocs.magento.com/guides/v2.3/cloud/trouble/environments-logs.html) 157 | - [Log locations (directories) for Pro plan: Integration, Staging, Production](https://support.magento.com/hc/en-us/articles/360000318834-Log-locations-directories-for-Pro-plan-Integration-Staging-Production) 158 | - [Log locations (directories) for Starter plan](https://support.magento.com/hc/en-us/articles/360020127552) 159 | - [Apply custom patches](https://devdocs.magento.com/guides/v2.3/cloud/project/project-patch.html) 160 | - [Configure Xdebug](https://devdocs.magento.com/guides/v2.3/cloud/howtos/debug.html) 161 | - [Project structure / Logs](https://devdocs.magento.com/guides/v2.3/cloud/project/project-start.html) 162 | -------------------------------------------------------------------------------- /9. Go Live and Maintenance.md: -------------------------------------------------------------------------------- 1 | # Go Live and Maintenance 2 | 3 | #### 9.1 Demonstrate ability to configure DNS 4 | 5 | - [Go live checklist](https://devdocs.magento.com/guides/v2.3/cloud/live/go-live-checklist.html) 6 | - [Launch steps](https://devdocs.magento.com/guides/v2.3/cloud/live/launch-steps.html) 7 | 8 | - Update your A and CNAME records for your domains and hostnames. 9 | - Wait for the TTL time to pass and restart your web browser. 10 | 11 | > After checking with your registrar about where to change your DNS settings, add a CNAME record for your website that points to the Fastly service: prod.magentocloud.map.fastly.net. If you use multiple hostnames for your site, you must add a CNAME record for each one. 12 | 13 | #### 9.2 Demonstrate ability to set up and configure Fastly 14 | 15 | [Fastly](https://devdocs.magento.com/guides/v2.3/cloud/cdn/cloud-fastly.html) 16 | 17 | #### 9.3 Demonstrate ability to upgrade to a new version 18 | 19 | ###### Upgrade of Magento and ece-tools to newer versions 20 | `composer update magento/ece-tools` 21 | 22 | #### 9.4 Demonstrate ability to upsize 23 | 24 | Magento Commerce can scale from the smallest Pro12 cluster to the largest Pro120 cluster. 25 | - Pro12 offers a 12-CPU (4 x 3 nodes) and 48GB RAM (16 x 3 nodes) 26 | - Pro120 offers 120 CPU (40 x 3 nodes) up to 480GB RAM (160 x 3 nodes) 27 | 28 | For permanent upsize - contact to account manager. 29 | 30 | For temp upzise - create support ticket or: 31 | > During expected periods of heavy site traffic you can reach out to your Technical Account Manager to request an increase in cluster size. 32 | > Then when traffic re-stabilizes you can request a down-size in cluster size. 33 | 34 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Overview 2 | 3 | Magento 2 Certified Professional Cloud Developer 4 | 5 | # 1. Topics 6 | 7 | 1. 13% 1 - Commerce Cloud Fundamentals (8 questions) 8 | 2. 9% 2 - Local Environment (6 questions) 9 | 3. 13% 3 - Cloud Configuration (8 questions) 10 | 4. 5% 4 - Service Configuration (3 questions) 11 | 5. 10% 5 - Deployment Process (6 questions) 12 | 6. 10% 6 - Static Content Deployment (6 questions) 13 | 7. 20% 7 - Development (12 questions) 14 | 8. 10% 8 - Troubleshooting (6 questions) 15 | 9. 10% 9 - Go Live and Maintenance (6 questions) 16 | 17 | 18 | ### 1.2 Magento 2 Certified Professional Developer 19 | 20 | + 60 multiple choice questions 21 | + 90 minutes to complete the exam 22 | + 62% or higher needed to pass 23 | + for a Magento 2 developer/architect with 6 to 12 months of experience developing for Magento Commerce Cloud 24 | + all questions are based on Magento Commerce Cloud v2.3.x 25 | 26 | # 2. Resources 27 | 28 | ## 2.1. Links 29 | + [Magento 2 Certified Professional Cloud Developer](https://u.magento.com/magento-certified-professional-cloud-developer) 30 | + [Magento 2.3 Cloud Documentation](https://devdocs.magento.com/guides/v2.3/cloud/bk-cloud.html) 31 | 32 | ## 2.1. Other notes for Magento 2 certification 33 | + [Preparation notes for Magento 2 Certified Professional Developer exam](https://github.com/magento-notes/magento2-exam-notes) 34 | 35 | ## 2.3. Magento Cloud links 36 | + [Cloud Project](https://cloud.magento.com/cloud/project/view/) 37 | + [Integration Environment](https://magento.cloud) 38 | + [Help center / Support](https://support.magento.com/hc/en-us/requests) 39 | + [Status](https://status.magento.cloud/) 40 | + [CLA](https://magento.com/legal/terms/cloud-sla) 41 | + [Magento Cloud Repository](https://github.com/magento/magento-cloud) 42 | + [Ece-Tools Repository](https://github.com/magento/ece-tools) 43 | + [Ece-Tools / .magento.env.yaml sample](https://github.com/magento/ece-tools/blob/2002.0.17/dist/.magento.env.yaml) 44 | + [Fastly tester](https://magento-tester.global.ssl.fastly.net/magento-tester/) 45 | + [magento-cloud list](https://gist.github.com/simpleadm/69c3ef764fdd637a109bbaed8420d714) 46 | --------------------------------------------------------------------------------