Piousbox

I have the fortune and the misfortune of having to scale my Drupal site immediately. It's getting more traffic than I can currently accommodate - and so the next scaling steps have to be taken.

I think the site was working fine on Drupal 9. But I upgraded to Drupal 10 and it became very, very slow. Now, I'm not going to go back to Drupal 9, but it's another talking point and another reason to not upgrade. As I wrote about previously, we like to keep our stack lean, so when upgrading to a next major version (1) brings no noticeable benefit, and (2) increases resource consumption - we really ask if it's worth it. Which it isn't. A lot of times it is not worth upgrading.

Moving on, let's walk through the steps I'm taking to address performance issues and improve overall site performance.

A first step I'm taking is rebuilding the website with ansible. That means writing and configuring ansible playbooks and roles to install the website on a bare OS. I'm using docker for this internally. You'd say - woa Victor that's a big task - and it is a big task. Still, I feel much more comfortable with infrastructure-as-code than if I don't have this capability. So let's get going!

I will also try to open-source the majority of my code as we go along. The ansible code is available here: https://github.com/wasya-co/wasya_co_ansible.git If you can't find any of the code I mention, or believe it hasn't been open-sources and you would like a copy - reach out to me directly. Note: `master` branch is almost never latest. The best branch to use is the alphabetically latest branch.

Note: I'm using my own docker images, such as piousbox/php83:0.0.2 You can plug in your own image, or pm me if mine doesn't work in our case or you need something specific, like another apache module.

I'll be working on my most popular site, https://piousbox.com Now after a few days, I got the playbook that installs the website working: playbooks/setup_all_drupal_site.yml and you need all the config values to be in vars/<sitename>.yml or something similar. You can run it with:

   ansible-playbook -i inventory/aws.yml -e @vars/piousbox_com.yml  --limit aws4 playbooks/setup_all_drupal_site.yml    -vv

After I wrote it, I discovered that I want to re-write it so that I can troubleshoot the installation locally without ansible. That is, even though I want to drive all of this with ansible, I want to be sure that it works before-hand. The way to do it is to package the app as a docker-compose project. So I split https://github.com/wasya-co/docker_drupal.git out of the ansible repo.

Of course, then I may have to have two templating systems, to plugin configuration values. This is actually already possible and I figured the overhead is not too big. In the project docker_drupal.git I have scripts/update_all , sripts/update_dc (for docker-compose.yml) and scripts/update_settings - these scripts take a template and substitute .env values into the template.

I then rewrote the ansible role to, instead of placing each necessary file onto the server, place the docker_drupal.git repo there, and template-substitute a few files. This lets me drive automation with and without ansible.

The project includes a mysql database. I believe it is the database that's the performance bottleneck, but I also want to be sure, and I want to have the tooling to troubleshoot such issues precisely in the future.

Then, to simplify things, I packaged all my drupal themes and modules as real modules for the composer. Previously, I would git-checkout whatever I needed from my own codebases, however, with increasing automation the easiest method has become to just include everything in the composer.json file, which I have done. the docker_drupal.git/compose.json has the details and is an example of doing that. (I use semantic versioning tags, excelt in my formalism 1.0.0 is always a branch, whereas v1.0.0 is a release tag. The v in the beginning makes it clear that it is not, in fact, a version, since the v is not part of the semver formalism.)

I then actually rewrote my ansible workspace repo - or rather, decided that the only share-able things will the the roles. So each ansible role is open-source in its own repo. Example: wasya-co/wasya_co_ansible_role.git And the playboors and variables and the instructions to run, are elsewhere, probably in the documentation. This way, I don't have to open-source my ansible workspace, and I still get the benefit of open-sourcing all the heavy lifting, which is inside the ansible roles.