In this guide, we'll walk you through the process of deploying your Magento 2 project with Capistrano, a powerful tool for automated deployments. We understand that deploying a complex platform like Magento can be daunting, but with Capistrano, we aim to make the process as smooth and efficient as possible.
You'll learn about the process of building a semi-automatic deployment system, which serves as the cornerstone for a fully automated CI/CD pipeline for Magento 2. By the end, you'll be able to implement this workflow in your development projects.
In the world of software development, there has always been a certain friction between the development and operations teams.
You see, developers have this burning desire to constantly release new features and enhancements, while the operations folks are more concerned about maintaining stability and reliability of the system. This clash of priorities often leads to a lack of collaboration.
Slows down the progress.
With the advent of DevOps, a methodology that aims to bridge the gap between these two teams, there is hope for resolving these tensions and creating a more harmonious working environment.
So, let's dive in and discover the benefits of using Capistrano for your Magento 2 deployments!
DevOps is a cultural mindset, not a specific job title.
Overview and limitations of Capistrano for Magento 2
Advantages:
- automation
- rollback capability
- multi-server deployment
- customization
- multi-stage deployment
- atomic deployments (no more var/generation cannot be deleted errors)
Disadvantages:
- Ruby dependency (but it shouldn't be a challenge for you to learn new language when you're coming from PHP background)
- The same user is used as deployer, Nginx and PHP-FPM runner
- app/etc/config.php must be committed to the repository (remember to run
bin/magento setup:upgradebefore commit after adding a new module)
The config.php in repository is the main drawback of using Capistrano for Magento 2. Especially when you have to disable the Two-Factor Authentication module on local environment and enable it upon pushing changes to the upstream
Prepare local machine for the Capistrano deployment
Install required dependencies before launching the first deployment. These are:
- asdf or RVM
- Ruby 3.2.2
- ssh client (should be preinstalled with the OS)
Assumptions about the target server configuration:
- running on Ubuntu 22
- PHP >= 8.1
- Magento >= 2.4
- Magento 2 project is versioned in a Git repository (GitHub, GitLab, Bitbucket, etc).
Ruby Version Manager (RVM) or asdf
It is up to you whether you'll install RVM or asdf. RVM is a popular version manager for Ruby, while asdf is a generic version manager for multiple tools and programs.
Instructions how to install RVM for your platform.
Alternatively, check out the guide for installing asdf.
Install the Ruby as a plugin for asdf
The asdf relies on its plugins as the source for versioned tools packages.
For Linux systems:
sudo apt install -y libyaml-dev
asdf plugin add ruby https://github.com/asdf-vm/asdf-ruby.git
asdf install ruby 3.2.2
For MacOS family:
asdf plugin add ruby https://github.com/asdf-vm/asdf-ruby.git
asdf install ruby 3.2.2
asdf global ruby 3.2.2
Initialize the Capistrano environment
Install capistrano and capistrano-magento2 Gems:
gem install capistrano
gem install capistrano-magento2
Setup Magento 2 project with Capistrano
Do this once per project:
cd <project_root>
$ mkdir -p tools/cap
$ cd ./tools/cap
$ cap install
You should have now the initial structure of the deployment. In next paragraphs you'll explore how to configure and customize deployment steps.
Capistrano is built around the concept of stages.
By default, the command cap install without any parameters creates two stages: staging and production.
You can use your own names, for example:
cap install STAGES=stage,prod
Update the config
Update your Capfile with the following contents:
# Load DSL and set up stages
require 'capistrano/setup'
# Load Magento deployment tasks
require 'capistrano/magento2/deploy'
require 'capistrano/magento2/pending'
# Load Git plugin
require "capistrano/scm/git"
install_plugin Capistrano::SCM::Git
# Load custom tasks from `lib/capistrano/tasks` if you have any defined
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }
Setup config/deploy.rb
lock "~> 3.18.0"
set :application, "magento.lucidmodules.com"
set :repo_url, "[email protected]:path/to/magento2.git"
Setup config/deploy/*.rb files
Sample production.rb
server "magento.lucidmodules.com", user: "deployer", roles: %w{app db web}
set :deploy_to, '/home/deployer/magento2'
set :branch, proc { `git rev-parse --abbrev-ref master`.chomp }
Sample staging.rb
server "staging.magento.lucidmodules.com", user: "deployer", roles: %w{app db web}
set :deploy_to, '/home/deployer/magento2'
set :branch, proc { `git rev-parse --abbrev-ref master`.chomp }
Sample production.rb
server "magento.lucidmodules.com", user: "deployer", roles: %w{app db web}
set :deploy_to, '/home/deployer/magento2'
set :branch, proc { `git rev-parse --abbrev-ref master`.chomp }
Refer to the documentation for the full list of available config parameters.
The
.chompfunction prevents any newline and carriage return characters at the end of the string.
Prepare the target server for deployment with Capistrano
Capistrano uses SSH to connect to the server and deploy the code. Make sure that PHP FPM and server are running on the same user as the one for deployment.
Fast CGI Settings
Capistrano relies on symlinking directories and files. This approach requires an update in the Nginx Fast CGI config.
Head to the /etc/nginx/fastcgi.conf and use the $realpath_root variable instead of $document_root for both SCRIPT_FILENAME and DOCUMENT_ROOT params.
Your file should now look like this:
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $realpath_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REQUEST_SCHEME $scheme;
fastcgi_param HTTPS $https if_not_empty;
** REST OF THE CONFIG **
...
Prepare deployer user
- log in as deployer user or switch account
- generate ssh keys
Create user and group for deployer
You should have the SSH key already if you were able to log in to the machine. For the sake of completeness, you can generate a new ED25519 ssh-key:
ssh-keygen -t ed25519 -C "[email protected]"
Why ED25519? it is more secure and shorter than e.g. 2048-bit SSH-2 RSA. Many cloud providers dropped support for older RSA keys
SSH to your server with sudo user. On ubuntu, it is the ubuntu user. Replace 1.2.3.4 with the server IP:
ssh [email protected]
The following script will create the deployer group and deployer user with home directory.
sudo useradd -m deployer
sudo groupadd deployer
sudo usermod -a -G deployer deployer
Switch the user to newly created deployer and add your local machine ssh keys:
sudo su deployer
vi ~/.ssh/authorized_keys
Update Nginx and PHP-FPM config
Make sure that your Nginx site config points to the current deployment path:
#/etc/sites-available/magento2.lucidmodules.com
server {
listen 8080;
server_name magento2.lucidmodules.com;
set $MAGE_ROOT /home/deployer/magento2/current;
include /home/deployer/magento2/current/nginx.conf;
** REST OF THE CONFIG **
...
}
Change PHP FPM user and group to deployer PHP FPM. The path is different for every PHP version:
#/etc/php/8.1/fpm/pool.d/magento2.conf
[magento2]
user = deployer
group = deployer
listen.owner = deployer
listen.group = deployer
listen = /run/php/php-fpm.sock
pm = ondemand
pm.max_children = 50
pm.process_idle_timeout = 10s
pm.max_requests = 500
chdir = /
Capistrano sets correct permissions for all Magento 2 files during deployment. This is the reason why server must run as the same user as the deployer.
Composer auth credentials in the global composer auth
Being logged in as the deployer user, edit the auth.json composer file:
vi ~/.config/composer/auth.json
Example auth.json, replace contents with your credentials and providers' urls:
{
"http-basic": {
"repo.magento.com": {
"username": "***",
"password": "***"
},
"composer.lucidmodules.com": {
"username": "***",
"password": "***"
}
}
}
Run the first deployment with Capistrano
We have defined two deployment stages: staging and production. Capistrano uses them to distinguish target machines.
Execute cap [stage] deploy to run deployment process for desired server(s).
Deploy staging:
cap staging deploy
Deploy production:
cap production deploy
SSH error on MacOS
On MacOS you might encounter an error:
Required dependencies for ed25519
ed25519 (>= 1.2, < 2.0)
bcrypt_pbkdf (>= 1.0, < 2.0) [not supported on java platform]
In that case, run ssh-add and try again.
Fix the Magento 2 cron path
Magento 2 ships with the CLI command to install the cron runner: bin/magento cron:install --force. However, there are two problems:
- it is using the
releaseTIMESTAMPpath instead of the symlinkedcurrentpath - it does not respect the maintenance mode
Fortunately the fix is simple: test for the existence of .maintenance.flag and manually point to the current directory with Magento installation:
* * * * * test ! -e /home/deployer/magento2/shared/var/.maintenance.flag && /usr/bin/php8.1 /home/deployer/magento2/current/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /home/deployer/magento2/shared/var/log/magento.cron.log
Bonus section
Here are optional steps which you might take depending on the Magento 2 setup.
Nginx reloading permissions
Log in to the server as sudo user, usually it is ssh ubuntu@ip-address.
To reload Nginx config we can rely on sudo service nginx reload command.
It is important to test the config before reloading and this is accomplished with sudo nginx -t.
The which command will output the absolute path to Nginx and Service programs:
which nginx
which service
The command should output respectively:
/usr/sbin/nginx
/usr/sbin/service
Using visudo, create a new config for the deployer user:
sudo visudo /etc/sudoers.d/deployer
Following snippet enables the deployer user to call sudo nginx and sudo service nginx reload commands:
deployer ALL=(ALL) NOPASSWD: /usr/sbin/nginx,/usr/sbin/service nginx reload
The last thing to do is a Capistrano task for handling Nginx config validity test and to reload the Nginx service:
# lib/capistrano/tasks/nginx.rake
namespace :nginx do
desc "Test Nginx server config validity"
task :test do
on release_roles :all do
execute *%w[sudo nginx -t]
end
end
desc "Reload Nginx server config"
task :reload do
on release_roles :all do
execute *%w[sudo service nginx reload]
end
end
end
OPCache
To configure proper OPCache flush after deployment, install CacheTool:
curl -sLO https://github.com/gordalina/cachetool/releases/latest/download/cachetool.phar
sudo mv cachetool.phar /usr/local/bin/cachetool
sudo chmod +x /usr/local/bin/cachetool
and create a file /etc/cachetool.yml with the following contents:
adapter: fastcgi
fastcgi: /var/run/php/php-fpm.sock
extensions: [ opcache ]
Make sure these paths match the system configuration. Especially that /var/run/php/php-fpm.sock points to the active PHP FPM installation.
Update OPCache config in php.ini (for both FPM and CLI):
opcache.enable=1
opcache.enable_cli=1
opcache.memory_consumption=512
opcache.max_accelerated_files=100000
opcache.use_cwd=1
opcache.validate_timestamps=0
Values for opcache.memory_consumption and opcache.max_accelerated_files might vary depending on the amount of modules installed for Magento.
After every deployment you'll get output from the opcache status.
If cache hit ratio is below 99% for a few days old deployment consider increasing the max_accelerated_files param.
The
max_accelerated_filesparameter is based on a set of prime numbers [223, 463, 983, 1979, 3907, 7963, 16229, 32531, 65407, 130987, 262237, 524521, 1048793] and will be rounded to a number greater than or equal to the configured value.
We also disable validate_timestamps because it checks for file changes in intervals every revalidate_freq seconds. PHP scripts are update only after deployment, so it is more efficient to refresh OPCache on demand.
While it is tempting to disable
opcache.save_commentsthis will break the REST API as Magento 2 relies on PHP Docs annotations describing endpoints parameters.
You might need to disable OPCache for CLI when you encounter issues with custom bin/magento commands.
Require extension for the OPCache in the Capfile:
require 'capistrano/magento2/cachetool'
Magepack build
Assuming that Magepack config is split across locales (magepack.config.en_UK.js), create a script to bundle Magepack in tools/bin:
#!/bin/bash
for f in magepack.config.*.js
do
IFS='.' read -r -a locale <<< "$f"
magepack bundle -c ./"$f" -g ./pub/static/frontend/Vendor/theme-name/"${locale[2]}"
done
Next, add Capistrano task to run Magepack bundling after magento:setup:static-content:deploy:
namespace :magepack do
desc "Bundle Magepack"
task :bundle do
on release_roles :all do
within release_path do
execute :bash, "./tools/bin/magepack-build.sh"
end
end
end
end
after "magento:setup:static-content:deploy", "magepack:bundle"
Zero-downtime deployment
To let Magento 2 check whether database schema needs update, configure MySQL or MariaDB param:
explicit_defaults_for_timestamp=on
If Capistrano executes bin/magento setup:upgrade even when there are no changes to the database schema, you should verify whether there are no errors and whitelist has been updated:
bin/magento setup:db-declaration:generate-whitelist
For example, MariaDB has issues with json column type in Magento 2.4.6.
Pushing it further
In Part 2 (coming soon) we're going to setup automated deployment pipeline on GitLab.
