how to upgrade sidekiq to 6.0

I always trying to use the edge versions of softwares. So I upgraded my projects in production to Rails 6.0 in just a few days since the release. Same with Ruby 2.7.

But not Sidekiq.

Sidekiq 6.0 was released in Aug 31, 2019. I have been using version 5.2 for half a year as a new release was available. This is unacceptable, unless Sidekiq 6.0 is a release with breaking change.

Remove the daemonization, logfile and pidfile command line arguments and sidekiqctl binary.

Instead, I have to manage sidekiq service with a init system. Well, to be honest, I didn’t understand what the fucking init system is, half a year ago. So the truth is, I did want to upgrade, but I didn’t know how. And Google didn’t give me a handy solution.

I knew I was on my self.

The key was the OS, Linux. I was kind of afraid of it. But the afraid feelings were familiar. It’s about unknown. What I need to do is just to get to know it.

So I did the research. I read a lot of materials about Linux. And a step forward, I bought a Linux Laptop this year, using it for my daily coding. Working in a Linux environment locally is totally different to that with remote server. That gave me a lot confidence to handle the init system stuff.

Untill recently, I upgrade my projects in production to Sidekiq v6.0.5, finally. So, here are the solutions.

Upgrade redis

Redis <4 is no longer supported.

My server is using Ubuntu 18.04. If you install redis using sudo apt install redis-server, the latest version is 3.0.6, which was really surprising me.

If you want to use the real latest version, 5.0.7, you have to add a repository to apt before installing. Just run these command lines with root.

sudo add-apt-repository ppa:chris-lea/redis-server
sudo apt-get update
sudo apt-get install redis-server

If the terminal complained with Add-Apt-Repository Command Not Found, just like one of my servers, then run this command overhead.

sudo apt-get update
sudo apt-get install software-properties-common

After a few minutes, a latest version of Redis will be installed. You can check it by

redis-server -v

Create sidekiq.service

I choose systemd as a init system to manage sidekiq service. To do that, we have to create a file named sidekiq.service. Sidekiq give a example for that. It is not difficult to understand because it contains a lot explanation.

Just follow the command lines:

vim /lib/systemd/system/sidekiq.service

Copy the whole file to it.

Then we need to change a few lines.

WorkingDirectory=/opt/myapp/current

Change /opt/myapp/current to your project directory, something like /data/www/app/current.

# If you use rbenv:
ExecStart=/bin/bash -lc 'exec /home/deploy/.rbenv/shims/bundle exec sidekiq -e production'
# If you use the system's ruby:
# ExecStart=/usr/local/bin/bundle exec sidekiq -e production

The line is tell systemd how to start your sidekiq service. Of course we use ruby to run it. I use rbenv, so I uncomment the rbenv line, and comment the system ruby line.

User=deploy
Group=deploy
UMask=0002

The deploy is the user name in your server, who excute the deploy. Mine is ruby, so I change it to ruby.

When all changes is done. We want the service automatically start at boot, so we must enable the service.

sudo systemctl enable sidekiq

Before run the start command, we have one more thing to do. Considering the sidekiq service is usually managed by the deploy user, not the root, we need to run the systemctl command as the deploy user. For me, the deploy user names ruby.

Of course we cannot give ruby all root privilege, which makes ‘deploy user’ no sense. What we need is to give ruby the root privilege only when manage the sidekiq service.

We can do that in /etc/sudoers.d.

Create a file named ruby (or deploy if you prefer) in /etc/sudoers.d/, copy these lines in it.

%ruby ALL= NOPASSWD: /bin/systemctl start sidekiq
%ruby ALL= NOPASSWD: /bin/systemctl stop sidekiq
%ruby ALL= NOPASSWD: /bin/systemctl restart sidekiq
%ruby ALL= NOPASSWD: /bin/systemctl reload sidekiq

This is straightforward. When ruby runs sudo systemctl start sidekiq, the system will not ask for password any more.

Another note before you start the service.

When your first start sidekiq service using systemd, if the system complain about

The name org.freedesktop.PolicyKit1 was not provided by any .service files

Be calm, just install another dependency.

sudo apt install policykit-1

Then the preparations are all setup.

Upgrade Sidekiq to 6.0

Change your Gemfile to

gem 'sidekiq', '< 7'

Run the update command

bundle update sidekiq

Edit deploy task

Usually, we use capistrano or mina to to deploy rails app. I prefer the latter.

Before Sidekiq 6.0, I use mina along with mina-sidekiq to handle the sidekiq deploy. It offers a few really useful tasks, such as

But not in Sidekiq 6.0. An issue has been open since last year. So I have to solve it by my self. And it is not that complicated. All we need it to define a new set of sidekiq tasks.

desc 'Reload Sidekiq'
task :sidekiq_reload do
  command %(echo "-----> exec: sudo systemctl reload sidekiq")
  command %(sudo systemctl reload sidekiq)
end

desc 'Restart Sidekiq'task :sidekiq_restart do
  command %(echo "-----> exec: sudo systemctl restart sidekiq")
  command %(sudo systemctl restart sidekiq)
end

desc 'Tail sidekiq log'
task :sidekiq_log do
  command %(echo "-----> exec: journalctl -f -u sidekiq")
  command %(journalctl -f -u sidekiq)
end

So we define 3 tasks, reload, restart and log. They are just shell scripts passing to remote server by mina.

With them, we can manange sidekiq service in local machine with 3 commands

And then, add them to the deploy flow. Here’s mine:

desc 'Deploys the current version to the server.'
task :deploy do
  command %(echo "-----> Server: #{fetch(:domain)}")
  command %(echo "-----> Path: #{fetch(:deploy_to)}")
  command %(echo "-----> Branch: #{fetch(:branch)}")

  deploy do
    invoke :sidekiq_reload
    invoke :'git:clone'
    invoke :'deploy:link_shared_paths'
    invoke :'bundle:install'
    invoke :'rails:db_migrate'
    invoke :'rails:assets_precompile'
    invoke :'deploy:cleanup'

    on :launch do
      invoke :'rbenv:load'
      invoke :'puma:smart_restart'
      invoke :sidekiq_restart
      invoke :'clockwork:restart'
    end
  end
end

All done. Just run

mina deploy

Sidekiq 6.0 will be run in your Rails app now.

Good luck.

· ruby, rails, sidekiq