how to upgrade sidekiq to 6.0
I always trying to use the edge versions of softwares. So I upgraded my projects in production to Rails 6.0 in just a few days since the release. Same with Ruby 2.7.
But not Sidekiq.
Sidekiq 6.0 was released in Aug 31, 2019. I have been using version 5.2 for half a year as a new release was available. This is unacceptable, unless Sidekiq 6.0 is a release with breaking change.
Remove the daemonization, logfile and pidfile command line arguments and
sidekiqctl
binary.
Instead, I have to manage sidekiq service with a init system. Well, to be honest, I didn’t understand what the fucking init system is, half a year ago. So the truth is, I did want to upgrade, but I didn’t know how. And Google didn’t give me a handy solution.
I knew I was on my self.
The key was the OS, Linux. I was kind of afraid of it. But the afraid feelings were familiar. It’s about unknown. What I need to do is just to get to know it.
So I did the research. I read a lot of materials about Linux. And a step forward, I bought a Linux Laptop this year, using it for my daily coding. Working in a Linux environment locally is totally different to that with remote server. That gave me a lot confidence to handle the init system stuff.
Untill recently, I upgrade my projects in production to Sidekiq v6.0.5, finally. So, here are the solutions.
Upgrade redis
Redis <4 is no longer supported.
My server is using Ubuntu 18.04. If you install redis using sudo apt install redis-server
, the latest version is 3.0.6, which was really surprising me.
If you want to use the real latest version, 5.0.7, you have to add a repository to apt before installing. Just run these command lines with root.
sudo add-apt-repository ppa:chris-lea/redis-server
sudo apt-get update
sudo apt-get install redis-server
If the terminal complained with Add-Apt-Repository Command Not Found
, just like one of my servers, then run this command overhead.
sudo apt-get update
sudo apt-get install software-properties-common
After a few minutes, a latest version of Redis will be installed. You can check it by
redis-server -v
Create sidekiq.service
I choose systemd as a init system to manage sidekiq service. To do that, we have to create a file named sidekiq.service
. Sidekiq give a example for that. It is not difficult to understand because it contains a lot explanation.
Just follow the command lines:
vim /lib/systemd/system/sidekiq.service
Copy the whole file to it.
Then we need to change a few lines.
WorkingDirectory=/opt/myapp/current
Change /opt/myapp/current
to your project directory, something like /data/www/app/current
.
# If you use rbenv:
ExecStart=/bin/bash -lc 'exec /home/deploy/.rbenv/shims/bundle exec sidekiq -e production'
# If you use the system's ruby:
# ExecStart=/usr/local/bin/bundle exec sidekiq -e production
The line is tell systemd how to start your sidekiq service. Of course we use ruby to run it. I use rbenv, so I uncomment the rbenv line, and comment the system ruby line.
User=deploy
Group=deploy
UMask=0002
The deploy
is the user name in your server, who excute the deploy. Mine is ruby
, so I change it to ruby
.
When all changes is done. We want the service automatically start at boot, so we must enable the service.
sudo systemctl enable sidekiq
Before run the start
command, we have one more thing to do. Considering the sidekiq service is usually managed by the deploy user, not the root, we need to run the systemctl
command as the deploy user. For me, the deploy user names ruby
.
Of course we cannot give ruby
all root privilege, which makes ‘deploy user’ no sense. What we need is to give ruby
the root privilege only when manage the sidekiq service.
We can do that in /etc/sudoers.d
.
Create a file named ruby
(or deploy
if you prefer) in /etc/sudoers.d/
, copy these lines in it.
%ruby ALL= NOPASSWD: /bin/systemctl start sidekiq
%ruby ALL= NOPASSWD: /bin/systemctl stop sidekiq
%ruby ALL= NOPASSWD: /bin/systemctl restart sidekiq
%ruby ALL= NOPASSWD: /bin/systemctl reload sidekiq
This is straightforward. When ruby
runs sudo systemctl start sidekiq
, the system will not ask for password any more.
Another note before you start the service.
When your first start sidekiq service using systemd
, if the system complain about
The name org.freedesktop.PolicyKit1 was not provided by any .service files
Be calm, just install another dependency.
sudo apt install policykit-1
Then the preparations are all setup.
Upgrade Sidekiq to 6.0
Change your Gemfile to
gem 'sidekiq', '< 7'
Run the update command
bundle update sidekiq
Edit deploy task
Usually, we use capistrano or mina to to deploy rails app. I prefer the latter.
Before Sidekiq 6.0, I use mina along with mina-sidekiq to handle the sidekiq deploy. It offers a few really useful tasks, such as
mina sidekiq:quiet
, stop sidekiq before a deploy.mina sidekiq:restart
, restart sidekiq when new deploy is finished.mina sidekiq:log
, to tail the sidekiq log in local machine.
But not in Sidekiq 6.0. An issue has been open since last year. So I have to solve it by my self. And it is not that complicated. All we need it to define a new set of sidekiq tasks.
desc 'Reload Sidekiq'
task :sidekiq_reload do
command %(echo "-----> exec: sudo systemctl reload sidekiq")
command %(sudo systemctl reload sidekiq)
end
desc 'Restart Sidekiq'task :sidekiq_restart do
command %(echo "-----> exec: sudo systemctl restart sidekiq")
command %(sudo systemctl restart sidekiq)
end
desc 'Tail sidekiq log'
task :sidekiq_log do
command %(echo "-----> exec: journalctl -f -u sidekiq")
command %(journalctl -f -u sidekiq)
end
So we define 3 tasks, reload, restart and log. They are just shell scripts passing to remote server by mina.
With them, we can manange sidekiq service in local machine with 3 commands
mina sidekiq_reload
mina sidekiq_restart
mina sidekiq_log
And then, add them to the deploy flow. Here’s mine:
desc 'Deploys the current version to the server.'
task :deploy do
command %(echo "-----> Server: #{fetch(:domain)}")
command %(echo "-----> Path: #{fetch(:deploy_to)}")
command %(echo "-----> Branch: #{fetch(:branch)}")
deploy do
invoke :sidekiq_reload
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
invoke :'deploy:cleanup'
on :launch do
invoke :'rbenv:load'
invoke :'puma:smart_restart'
invoke :sidekiq_restart
invoke :'clockwork:restart'
end
end
end
All done. Just run
mina deploy
Sidekiq 6.0 will be run in your Rails app now.
Good luck.