Exercise Due Diligence Before Major Upgrades

I recently experienced significant issues after a major software upgrade … and just to be clear, this major upgrade wasn’t just released.

While it was frustrating and required quite a bit of troubleshooting to get to the root cause and arrive at a solution, it fortunately did not reach the extent of damaging or crippling, nor did it materially impact immediate workflow (though it certainly could have easily been worse).

That said, it reminded me about the importance of exercising due diligence (especially for businesses) before major upgrades, implementations, roll outs, and/or deployments in live production environments.

Careful planning and testing (ideally, in non-production environments) are extremely important whenever possible to help mitigate against potential downtime and service disruptions that may be crippling and impactful to the business.

Evaluating and developing backup plans (ex: What’s our business continuity plan if this goes awry? Will we be able to revert or roll back and what’s the potential impact?) can be the difference between getting up and running relatively quickly versus substantial downtime.

Granted, even the most meticulous planning and testing isn’t foolproof!

From experience, we know that even major tech companies with huge budgets + teams, complex and meticulous processes, and who invest heavily into infrastructure to ensure backup, recovery, and failover solutions can and do experience downtime and service disruptions with widespread impacts.

Having started my career in IT, I’m well-versed in what can and does go wrong. I’m also aware that sometimes it may be impractical (especially for small businesses) to be as meticulous in planning and testing prior to a major upgrade, implementation, roll out, and/or deployment for various reasons.

However, that doesn’t mean don’t try at all!

Be proactive, not reactive!