How do you estimate the effort of a Sitecore upgrade?

We are in the process of, or have recently completed a number of Sitecore upgrades and pooled our knowledge.

In this post I will share things to consider when planning an upgrade and also talk about the process we went through to upgrade and deploy a new version of Sitecore to production.

Planning ahead

Estimating a Sitecore upgrade can be a difficult task. The effort required can vary significantly based on the complexity of your solution. Below are some things to consider when estimating the effort and time an upgrade will take.

What version of Sitecore are you upgrading from and what version are you upgrading to? Have you mapped out what the upgrade process will be? Can you use the Upgrade migration tool? (more on this later)
How much custom Sitecore code is there in the solution?
How many support patches are there in the solution?
How do you perform a full regression test? What is the testing effort?
Do you have adequate test coverage to be confident in the solution post upgrade?
How complex is the solution (i.e more things can break, higher likelihood more bugs / issues will be found)
What is the current state of the Web and Sitecore configs?
How many non production environments will you need to upgrade and deploy the new version to?
What does the path to live look like?
Do you use any external modules? For example XDB, WFFM, EXM - and what is the upgrade path for these modules?
What is the state of package dependencies within the solution? i.e Do different projects have different versions of the same library?
How will you develop new functionality alongside performing an upgrade? - The elapsed time between first creating your upgrade code branch to actually deploying the upgraded solution to production will likely be a number of months.

Our process...

Here are some things we did and learnt whilst upgrading from Sitecore 7.2 to 9.

Reworking the configs

As we were working with a fairly old solution before we did anything upgrade-wise we spent about 2-3 weeks reworking our configs. This involves moving all changes to transforms and Sitecore patch files leaving us with out of the box Sitecore and web configs.

With any new projects I would suggest doing this from the start.

We used file comparison tools to compare the 'compiled' Sitecore config (/sitecore/admin/showconfig.aspx) before and after the changes were made. Even after doing this we missed a few bits so be wary testing effort is still required. There are also Sitecore modules to help you do this e.g.: https://marketplace.sitecore.net/modules/sitecore_configbuilder.aspx

Consider contingency time to fix any issues which might arise.

As a side note to further improve your config setup Sitecore 9 now supports rules based configuration - more about that here: https://jammykam.wordpress.com/2017/10/17/rules-based-configuration/

The express migration tool and how we dealt with configs during the upgrade

In the past the way to upgrade Sitecore was jumping from version to version, with all sorts of fun along the way. Sure the upgrade path allowed you to skip a few versions to get to your target but it was still a painful process.

The express migration tool is much better. It migrates your content from any version since 6.6 to version 9 at the time of writing. Check latest compatibility here: https://kb.sitecore.net/articles/400952

It worked really well for us. However it is not perfect - for example it did not migrate some renderings correctly which resulted in broken presentation details which we only noticed during testing the site. This can be fixed with content packages / unicorn etc (or in our case SINJ scripts which are run after the site has been upgraded.

The tool can also migrate configuration... which we chose not to do. Instead we upgraded our configs manually, which after reworking (above) was not difficult.

As we were not utilizing any additional modules we were able to disable most of the provided patch files in the App_Config folder.

To get a better feel for the effort required one thing we like to do right at the start is a quick and dirty upgrade to get a feel for the sort of problems we might encounter when we do the real thing and to answer some of the bulleted points listed above.

State of the solution and support patches

Luckily we have most of our custom code well documented. You will need to factor in time for fixing any problems that might arise in custom Sitecore code on the newest version of Sitecore. e.g custom pipelines, handlers, helpers etc.

We also document support patches added to the solution. It's easy to forget why Sitecore.Support.12345.dll lives in your bin directory and what problem it is solving! Again validating these problems are still fixed after upgrade needs to happen. We were able to remove nearly all our support patches as the bugs they were implemented to fix were resolved in the newer version of Sitecore.

You might also find after upgrading you have dependency hell in terms of referenced assemblies not playing well with each other. This can take an age to fix.

Upgrading non prod environments

You will need to factor in time for upgrading non production environments. Perhaps you will need to spin up a second QA environment for Sitecore 9 testing as your original QA environment is needed for ongoing development. What about CI and CD - is this a factor? Do you need to run automated tests against 2 versions of Sitecore now until you go live?

Testing

I've mentioned testing quite a bit.. and for good reason. It's so easy to underestimate the testing effort that goes into an upgrade.

We have a whole suite of automated UI tests covering most of the business use cases which we run overnight. This really helps flush out any upgrade issues. We also have a super QA team who manually test the site and specifically the CMS itself which is not covered by our tests.

One example of something which caused us problems was that the way Sitecore stores raw time changed which caused some of the code to become unhappy. Expect small things like this which can crop up and do need identifying and fixing.

Other modules

We were lucky. No XDB, no WFFM and no EXM. This makes the upgrade process much easier.

If you do have other modules to upgrade factor in time for this. It's easy to underestimate the complexity of upgrading these modules and identifying and fixing any issues than might occur.

XDB is notoriously hard to upgrade.

EXM and WFFM - You may have to go through lots of different install steps although the express migration tool does state it can help with WFFM.

This can be painful and time consuming.

Also consider older versions of WFFM are not supported in 9.1.

The live deployment

For zero downtime we utilize blue green deployments. We spun up new Sitecore 9 stacks and then switched traffic via Akamai after we've completed our 'offline' testing against the new stacks.

In a normal deployment we have a short content freeze of say 1 hour during which we backup the databases and use these backups in creating the new environments.

When migrating from Sitecore 7 to 9 we have one major change from our standard release process - the Sitecore 7 database needs updating to Sitecore 9.

We agreed a content freeze from 4pm the day before the release.

We created a new EC2 instance to perform the migration using the express migration tool via a number of custom powershell scripts. It time we needed to migrate a database from 7 to 9 we simply re-run these scripts changing a few params which determine the environment to upgrade.

The process worked like this:

Powershell script is run which creates a backup of the live Sitecore databases (core and master) at 4pm.
Next it restores these databases to our upgrade box.
We then upgrade these databases using the Express migration tool.
We have a blank Sitecore 9 instance installed onto our upgrade box. We login to Sitecore and perform a full site publish.
We push the new migrated databases to S3 where they can be picked up by our deployment process when spinning up the new prod environment.

Menu