Monthly Archives :

March 2021

Ubuntu 18.04 server upgrade to 20.04 Focal Fossa

Ubuntu 18.04 server upgrade to 20.04 Focal Fossa 1280 720 bacsoa

My last two days have been about installations and updates. Yesterday I had to install an Apache, PHP, MySQL combo on my new Macbook Air M1, and today I did the long overdue Ubuntu 20.04 update on my production server. This post is about the Ubuntu upgrade, but maybe I will also describe the M1 Silicon LAMP installation, it was a bit more challenging.

My server has been with Digital Ocean for a year and a half, then it came with Ubuntu 18.04, but it is already 2021, and Focal Fossa came out last year and it was time for the upgrade. There is a detailed English howto on the Digital Ocean website, but if I had only followed it, I would be in trouble now, because I had a small problem during installation (see later).

1. Preparations

I saved three things:

  • webserer files
  • MySQL databases
  • /etc directory

I started by dumping the MySQL databases, a 3-line batch file (let’s call it “backup-sql.sh”) with the following content is suitable for this (“nano backup-sql.sh” command can be used to edit the file, which creates it immediately). You can also use Midnight Commander, but first create it with the “touch backup-sql.sh” command):

#!/bin/sh
for DB in $(mysql -e 'show databases' -s --skip-column-names); do
     mysqldump $DB > "$DB.sql";
done

It also needs execution rights, because in Linux new files do not get such by default:

chmod 500 backup-sql.sh

After running, an SQL file is created from each database in the current directory. I don’t want to deal with what it takes to prevent the MariaDB password from being asked for the logged-in user, but for those who are interested, here is an article about it. So we now have execute rights to the file, we simply run it:

./backup-sql.sh

Once this is done, the /etc directory can be added. This is useful because if, after the Ubuntu 20.04 update, the installer overwrites something and then it won’t work (e.g. postfix/mailing, SSH, openssl, whatever), we can restore it from the old config file, and see if what worked before, why not now, so we can compare the two different version. This command is also run at /var/www/html in the terminal (or from where we will run the rsync):

root@xact:/var/www/html $ tar -zcvf ./backup-etc.tar.gz /etc

The rsync command looks like this under MacOS (user@1.2.3.4 is the address of our server, and /var/www/html is the directory where we have the files to be backed up, obviously the previous command called backup-sql.sh it is also worth putting it here and running it from here so that the SQL dumps are also placed here):

rsync -avr --progress -L user@1.2.3.4:/var/www/html/ /Users/macusername/Documents/backup-directory

The -L switch means that rsync will follow the symlinks and copy them over as well. I don’t have much space on the first partition, so most of the websites are under /mnt (I mounted another volume here). Due to the standards of the Apache configuration, it is symlinked under /var/www/. If the -L switch is omitted, it will copy only the physical files, so only those under /var/www/html.

This takes a while, depending on how many virtual hosts (websites) there are on the webserver and how many files there are.

The backup is done. Before we start the upgrade, it is worth SSH into the server with a user with SUDO rights on a terminal and switch to root, in case a crash occurs during the installation, we have an extra leg. If we do not do this and the install stops, we may not be able to SUDO, i.e. we will lose control over our system. So before do-release, we SSH into the server with a user with SUDO rights (this is important!):

ssh user@1.2.3.4

Then switch to root user:

sudo su

I started the installation in Digital Ocean’s built-in Java console from a browser, but it can also be done from SSH (but even then, start a separate SSH session as described above).

2. Installation

After entering the shell, the first thing you should do is stop essential running services, do not start an operating system update on a live system. I stopped Apache and MySQL, but if you have NGINX with PHP-FPM or anything else, you should also stop them:

sudo systemctl stop apache2

Then stop the MySQL too:

sudo systemctl stop mysql

One more check, to make sure it stopped:

sudo systemctl status apache2
sudo systemctl status mysql

If we see Active: inactive for both, we can start the update with this command:

sudo do-release-upgrade

Then the following screen will appear:

Here, of course, press ‘y’, if you press ‘N’ (case sensitive!), the installer will exit. Then we’ll see something like:

It informs here that the new Ubuntu 20.04 system no longer supports 2 packages compared to the current one, 13 packages will be removed, 208 packages will be uploaded and 665 packages will be updated. It’s funny that if we went back 20 years in time, when we still used a 56k modem (who was born around the 80s, like me), then the 508MB of installation data would be downloaded in 19 hours 🙂 I pressed it here a ‘d’ because I was curious about the details. If you are not interested, press ‘y’ (case sensitive!). So when I pressed ‘d’ I got this:

PHP 7.2 will be removed, which is not a problem, because I wanted to upgrade to 7.4 anyway (in fact, 8.0 is already here), and a few Apache2 modules too. And then here is this question, which makes it worthwhile to SSH in a separate terminal session:

Here it says that a lot of services will be restarted and if something breaks, you won’t be able to log in (not even via SSH).

After that, we will see a lot of things on the screen and the Ubuntu installer will ask many things. Especially ones where the existing config file differs from the one we want to install. I overwrote most of it, mostly the ones I knew I hadn’t touched. However, I left the 18.04 version of what I adjusted a lot (e.g. opendmarc). I haven’t had any problems until now:

I’ve pressed a ‘D’ here to show the differences. Anyway, this is the SSH config, where I setup like so that it is not possible to SSH to the server with a password, only with an SSH key, etc. However, for some reason, after pressing ‘D’, the system threw me back here:

It was also a bit strange that I would get this screen back after a question in the middle of the installation. Since it had been doing the installation for a while, I even thought it was completed, so I checked the server version:

Everything seems OK, right? Unfortunately no, all that happened here was that the basic package was already updated, so the content of the /etc/issue file was updated to the new operating system version number, but everything else remained the same. So the installation was aborted. I checked the PHP version, which was still 7.2 on 18.04, but it should be 7.4 on 20.04, and my suspicions were confirmed:

I thought I would enter the do-release-upgrade command again, but it didn’t help:

The problem is already visible here, the installer tells do-release-upgrade that there is no developer version, i.e. the system is already the latest. However, when dist-upgrade -f (‘-f’, i.e. force switch is required!) command is entered, it appears that the dpkg package manager stopped during the update, so enter this command:

sudo dpkg --configure -a

So it was good, dpkg started to continue the update process. The question came again:

Now, instead of show the differences (the installer stopped here first, and I didn’t care because I had already backup the entire /etc directory), I overwrite the old one with the latest config file. Then came up a lot of similar questions, I pressed Y for almost everything (except e.g. opendmarc). Some errors were encountered:

The dpkg completed, but lxd-client failed to update. Then I re-enter this:

sudo apt dist-upgrade -f

Then started to update again, including this issued LXD client:

I have no idea what it was, so I pressed a ‘latest’ (even though the installer recommended 4.0). The update process continued, and then came this:

What I knew I hadn’t touched after configuring 18.04 (this was my first Ubuntu version) I overwrote with the new version (so I pressed ‘Y’). After it finished updating the packages, it also wanted to update the GRUB:

Since there is no other operating system on my server, only Ubuntu, I chose the ‘package maintainer’s version’ option here. There are also remain config files, for example for Midnight Commander:

I accidentally overwrote this, so on 20.04 I had to setup again so that clicking backspace goes back a directory up (since this setting is stored in mc.keymap file, see here for a howto). Then came the Opendkim config question:

I pressed N, because I spent a lot of time authenticating my email addresses, and I don’t need to setup again. Almost in the end there was another one like this:

I chose Yes here because I want to set this manually in the future. Some questions about databases, passwords, etc.:

After that, the installation was finished, which was indicated by a short ‘done’ message. After restarting, I pressed an update and removed the no longer used packages (do-release-upgrade would do this for us anyway):

sudo apt autoremove

Then I restarted the server:

sudo init 6

After restarting, the usual screen welcome me:

I checked the php version again:

All’s well that ends well. Thanks to Zoltán Szécsényi for the help with the installation!

Switching to a Macbook Air M1

Switching to a Macbook Air M1 2000 1500 bacsoa

This is my 4th Mac. The previous Macbook Pro with Touchbars from 2017 was a nightmare. On the one hand, because of the keyboard, which I had to do twice – fortunately in a factory recall – and on the other hand, because of the touchbar. At first it seemed like a good idea (it useful in Photoshop), but on the Macbook are already less keys compared to a desktop PC, and then even the function keys were taken away…horrible.

The M1 came out last year, which I knew nothing about, but based on the tests, it was convincing. It’s fast and relatively cheap. I mean, compared to an Apple product 🙂 I had an attempt a few months ago to install Linux for a Lenovo notebook to make it easier to try new things (Ubuntu, PHP 8, Redis vs. Memcached, etc.), but it was not good. Instead of that, I put it on my old Macbook Pro on a separate Ubuntu partition, it works perfectly fine. The fact is that the Macbook keyboard and especially the trackpad are simply perfect.

Anyway, back to the M1. I’ve been using it for 2 weeks and I’m mostly satisfied.

Advantages:

  • Fast. Everything opens immediately and 8GB of RAM is more than enough. At least for me, who works with Atom or Sublime Text and Photoshop / Illustrator every day. In addition, about 20 Chrome tabs and 3-4 Firefox tabs are also open. Plus the Slack, Viber and Telegram all the time. I don’t feel that 16GB is needed at all, but the 512GB SSD is.
  • Keyboard. I finally got back the comfortable typing experience. Apple also realized that development with butterfly mechanics was a dead end. So they returned to good old scissor-mechnanism. A thousand times better.
  • Trackpad. There is no need to explain this, it is the best trackpad in the world. This is one of the reasons I work on a notebook and not on a desktop, because the keyboard and trackpad are close to each other, so I don’t have to move my hand to the right of the mouse and therefore lift my hand from the keyboard. It is much more efficient to work this way. I write the code in a text editor all the time, while I look at and test the operation of the website, which requires a mouse. I’d go crazy if I had to do this with a separate mouse.
  • Kedvező ár. I bought this machine without VAT, but it was brand new with Apple warranty of course. If you look at Apple’s price, it’s still good enough. You will not get a good computer that is fast and reliable for less than EUR 500. Neither in desktop version I think.
  • Battery time. It’s not like it matters, because it hasn’t been a problem so far (and it’s constantly plugged into the charger anyway), but it’s good enough that after 2-3 hours of surfing the net and watching YouTube videos in the evening, the battery drains by 10%. This really gives you 14 hours of work time.

Disadvantages:

  • Compatibility. It is a fact that some programs do not run at all or not perfectly on the Big Sur OS. The Atom text editor I’m working on freezes all the time. So now I’m back to good old Sublime Text, it’s perfect. Setting up SSHFS took me about 3 hours because a lot of things were changed in Big Sur, so it was not easy to setup. After a reboot, the installation of Kernel-level add-ons, etc., must to be enable.
  • Brew package manager does not working properly. Brew is instead of the package manager of some Linux distros (e.g. apt, yum, pacman). Many things are missing by default under MacOS (e.g. Midnight Commander), for which Brew offers a solution. It’s pretty much in beta state right now. So much so that I uploaded it and then it disappeared 🙂 Then I installed it again and now it’s not working. I’ll wait with it for now.

Photos