New Royal Academy of Dance Website – Tech Deep Dive (Part 1)

The new site had a number of enhancements – some of the biggest ones were as follows:

• A bold new design in line with the Royal Academy of Dance’s new branding, as well as using more imagery & videos.
• Major improvements to the website user experience (UX) & navigation.
• Far more flexibility for the RAD to be able to easily manage the content & layout of pages.
• A simplified approach to managing multi-country content.
• Major improvements to the Find a Dance Teacher functionality on the site.
• A solid base for further enhancements and new website functionality in the future.

We’re delighted with the new site, and have received glowing feedback from both the Royal Academy of Dance themselves, as well as from website visitors! You can access the new website from www.royalacademyofdance.org, and you can read more about the new site features here.

In this article, we delve into some of the underlying tech that was used to build the new website, as well as some of our experiences whilst migrating the previous RAD site to new hosting infrastructure – which involved major upgrades and modernisations to the underlying tech stack. Note that a fair amount of technical terminology is used here – we have attempted to explain this and/or provide links to further information about the topic where necessary.

A Migration to New Website Hosting Infrastructure

Initially, we took on responsibility of supporting & maintaining the previous Royal Academy of Dance website, which was originally built by a different agency. As part of this process, we needed to migrate that site to our hosting infrastructure. This ensured that the existing website would remain functional while we started to work upon the new site.

Migrations of this nature are usually very routine and straightforward, and we have a lot of experience with seamlessly moving existing websites across to new infrastructure. However, in this particular case, it was one of the most technically complex site migrations that we have completed to date, due to:

• An unconventional tech stack used by the previous website.
• The fact that many of the underlying components & code libraries on the existing site were extremely out of date – ultimately we needed to upgrade 100+ code libraries & frameworks to newer versions (including upgrades across multiple major software versions in many cases), in order to complete the migration successfully.
• Major differences between the set-up of the source and target hosting infrastructure.
• The large size of the website (including the associated multi-country sub-sites) – consisting of thousands of webpages, as well as many gigabytes of existing media files.

This migration essentially involved a mini-project in its own right, and was certainly a bit of a ‘baptism of fire’ into the world of the Royal Academy of Dance!

The Previous Website Tech Stack

While the previous website itself was built upon WordPress, it was built in a very different way to the vast majority of other WordPress sites:

• The website was built upon the Lumberjack MVC framework:
  • This framework incorporates Bedrock for the overall folder structure and site configuration, as well as using Composer to manage dependencies (i.e. the WP core, plugins, themes, 3rd party PHP libraries etc).
  • It also incorporates the Timber library, to allow site templates to be set up using the Twig templating engine.
  • The framework provides the ability to set up controllers for the defined URL routes, to build database queries, to log events, and to inject dependencies via the PHP-DI container.
• A relatively complex bespoke components library had been implemented on top of this, to allow elements to be added to website pages:
  • This was designed to connect the Lumberjack framework with the widely used Advanced Custom Fields plugin.
  • It included the concept of both ‘Fixed’ and ‘Flexible’ layout components – with ‘Fixed’ components being the same throughout the site (such as those components appearing in the site header & footer), and ‘Flexible’ components varying from page to page (utilising the ACF plugin’s Flexible Content field type).
  • Each component had frontend Twig template(s), as well as backend driver class(es) responsible for setting up the relevant ACF field(s) (essentially this used the acf_add_local_field_group() function behind the scenes to register the fields via PHP code) and retrieving field data from the database.
  • The components library included some pre-built components, but it was also possible to add or override components from within the site theme.
• For managing frontend assets such as JavaScript code, CSS code, fonts, and theme images, a bespoke build system had been set up:
  • Gulp task runners were implemented to compile & minify the assets, create sprite sheets, etc.
  • CSS code was implemented using the SASS library.
  • JavaScript was implemented using a combination of vanilla JS code, as well as the Vue.js framework for certain frontend components.
  • Webpack was used for compiling & bundling of a number of the assets.
  • NPM was used to manage the various dependencies.
• The website connected to the RAD’s external Microsoft Dynamics CRM (incorporating the Silverbear membership management system) via SOAP, utilising a combination of 3rd-party libraries and bespoke PHP code to connect this mechanism to the Lumberjack framework.
• The site was originally hosted upon AWS infrastructure:
  • Laradock was being used for the management of the PHP environment (this tool is primarily intended for setting up local development environments using Docker, but in this case a tweaked version was also being used to host the production version of the site).
  • Amazon S3 was used to host website media files, with the CloudFront CDN used to serve these to end users. These assets were offloaded from the hosting environment using the WP Offload Media plugin.
• The Phinx PHP library was used for implementing & managing data updates within the MariaDB database.

In summary, the site code was far more structured (and used a much wider range of underlying technologies) than the code upon a typical WordPress website often is – and was much more akin to working with a framework such as Symfony. We therefore quickly realised that our extensive experience of using Symfony whilst working with platforms such as OroCommerce would come in very handy!

Initial Steps

The first step was to try to get a version of the site working within a local development environment.

We initially attempted to get the site to run based on the existing configuration, by building & running the containers for the Laradock dev environment. Unfortunately, this was unsuccessful due to a missing 3rd-party dependency that was no longer available. We therefore created a new development environment from scratch instead, and then gradually configured it to work in the way we needed. Although this approach was more labour intensive, it meant we had much more control over the set-up, making it easier to debug the issues that arose.

This involved quite a range of activities, including installing various packages & dependencies, making obscure server configuration changes, stepping through the website code using a debugger tool, making various database updates, reviewing the generated diagnostic log files, etc, etc, etc… In the process of doing this, we also spotted various website bugs that hadn’t been found by anyone previously, so we were able to fix these as we went along.

Overall, the process was quite interesting, which felt a bit like a treasure hunt in some ways – we saw all kinds of exotic error messages, which provided various clues as to what we needed to do next, until we encountered the next obstacle; and so on!

Eventually, part of the website homepage appeared on the screen! It is hard to put into words just how much joy a developer feels when events like this happen – we knew it would still be a long journey to get the website into a state where it would run fully, but there was light at the end of the tunnel at last!

Potential Hosting Infrastructure Options

Now that we had a partially working copy of the website running upon a local computer, it was important to make a big decision about the underlying hosting infrastructure that would ultimately be used to make the website available to normal visitors moving forward. We had two main options:

• Option 1: Try to replicate the previous hosting set-up as closely as possible, with the website continuing to run on AWS, utilising the Laradock Docker containers, plus Amazon S3 and CloudFront to serve media assets.
• Option 2: Move the site to completely new hosting infrastructure, making any necessary code tweaks, adaptations & software upgrades that would be necessary for the site to work reliably with the new set-up.

This turned out to be quite a tricky decision. On the one hand, option 1 was a ‘proven’ approach, as we knew that the live version of the site already used this kind of set-up – so potentially the overall amount of effort required to set the site up in this way could be lower.

On the other hand, because we only had access to the website code and not the existing hosting infrastructure itself, we knew that to exactly replicate the current set-up, it would require a a fair amount of guess work. We were also concerned that we might encounter similar challenges as the ones we encountered when we attempted to replicate the set-up locally earlier.

It was also very apparent that many of the software libraries that were being used to run the website were extremely out of date, and thus no longer supported by their vendors. In turn, this meant that there were potentially security implications, which could also impact compliance with GDPR legislation.

Because these out-of-date dependencies were the biggest obstacle to moving the site to completely new infrastructure – and because in all likelihood it would ultimately be necessary to apply these updates to resolve the issues mentioned above in due course anyway – we felt that overall, this approach was much better and we could complete these upgrades as part of the site migration process.

This would also provide the opportunity to simplify the hosting infrastructure and decrease the number of ‘moving parts’, reducing the risk of future website issues, as well as the amount of ongoing maintenance that would be needed to keep everything running while the new website was being built.

Applying the Software Upgrades

Having made this decision, it was then necessary to start installing all of the necessary updates. It was more simple to initially perform this task upon the local development copy of the website that we had set up earlier. One of the key aims was to ensure that the site was running on a supported version of PHP, upgrading this from v7.x to v8.x.

In turn, this would require an update of the main Lumberjack framework that powered the website from version 3 to version 6 (!) – and although the authors of this framework provided a handy upgrade guide, it was clear that this would be quite a large undertaking. A lot of other website dependencies would also need to be updated in parallel, to ensure that these major upgrades wouldn’t break anything.

The upgrade did indeed prove to be challenging, and we encountered a number of compatibility issues between the new version of Lumberjack (and its dependencies) and the bespoke PHP library that was used to render components upon the site. For example, we encountered issues where the wrong components were sometimes being loaded on pages; and in other cases, the same component was incorrectly being displayed multiple times upon a webpage.

To get to the bottom of these issues, we needed to wade deeply into the code not only for the website itself, but also the Lumberjack framework, the PHP-DI library, and indeed the core underlying code of WordPress, using the XDebug debugging tool to step through the code line by line until we eventually discovered the underlying causes.

It ultimately became apparent that many of these issues were due to new approaches being used by more recent versions of Lumberjack as well as the PHP-DI dependency – the authors of the Lumberjack framework had decided to move away from using singletons by default, whereas interestingly the authors of the PHP-DI library had decided to adopt the opposite approach…! This effectively caused site issues to manifest themselves in various ways, depending on the way in which the relevant bespoke website component code interacted with these 3rd-party libraries.

Remaining Tasks

Now that we had got to the bottom of what was causing the issues that arose from installing the various software upgrades, and having implemented fixes for these issues, the next step was to set up the site upon the hosting infrastructure itself. This required a few other small website tweaks and bugfixes to be implemented, but ultimately we had a working version of the website available for teams at both our end and at the Royal Academy of Dance to comprehensively review and test.

In the meantime, we had another important task: We needed to migrate the existing website media files (such as images, PDF files, etc) onto the new website hosting infrastructure. Previously these files had been offloaded to CloudFront as described above – and although the new hosting infrastructure also used a Content Delivery Network (CDN), it worked in a different way, so the files needed to be loaded from new locations. One complicating matter was that the website used WP Multisite to manage each sub-site, and that a lot of the media files had been replicated across to separate locations, whilst also being referenced directly from various other sub-sites.

To resolve this challenge, we initially implemented a custom migration script, to automatically move the relevant media files to more appropriate file locations. To ensure that any remaining references to the previous file locations within the site content were updated correctly, we then implemented some new custom website functionality that dynamically ‘rewrote’ the file URLs within the content to point to the correct new locations, using a number of WP action and filter hooks. A similar mechanism was also used to dynamically redirect visits from the old media file URLs to the new ones.

This custom mechanism ultimately needed to become quite elaborate due to a number of identified ‘edge cases’. To identify the causes of these edge cases, we implemented some comprehensive diagnostic logging functionality.

In addition to our extensive testing, we also used a number of tools, such as Screaming Frog, to scan the site to ensure there were no new issues as a result of all these upgrades and changes.

Site Migration Outcomes

While this website migration was challenging, it was very successful and the process was completed smoothly with no major new site issues identified. Our comprehensive approach to this website migration also had a number of side benefits:

• The website ran noticeably faster upon the new infrastructure.
• The upgraded website components the capabilities available within the new hosting infrastructure significantly improved the website security.
• In turn, this resulted in better compliance with GDPR legislation.
• The site was now running newer & actively supported versions of the installed software, reducing risk of future website issues.
• The website now ran on simplified infrastructure with fewer moving parts – helping to improve reliability, and reducing the amount of ongoing management that was needed for the underlying infrastructure to run smoothly.
• We gained a lot of hands-on experience with working with the website’s code right from from the outset, making it easier for us to support and maintain the site; as well as giving us a real insight into the various ‘pain points’ for us to look at during the upcoming website rebuild.

Having successfully migrated the existing website across to our infrastructure, it was now time for us to start looking at how we would rebuild it, to meet the Royal Academy of Dance’s needs more effectively…

Find out more in the second part to our technical series: A Review of Potential Platforms for the New Website »