What do you do when your company's systems go down, and you cannot get them back up?

Answered by:

Tom Karakis

Highly recommended
Alpha Geeks
13 Questions answered
Answered on January 29th, 2019
First and foremost, hopefully you have a plan in place BEFORE you need to use it.  When you talk about systems, what really are you talking about?  That word could encompass a lot of different things such as computers (servers and desktops), infrastructure (internet and networking equipment), phones, etc.  I will try to address several of the key components we deal with here and how we handle them.

Internet
If internet is absolutely mission critical, you would have to have a router with ISP fail-over.  If your main source of internet goes out, the secondary source kicks in.  For example, if you have a router that uses either fiber or coax as your main provider and that provider experiences an outage, the secondary ISP configuration would kick in.  For our business clients where internet is mission critical, we use a router that supports 4G fail-over.  If the main internet source goes down, the 4G service becomes active keeping their internet alive.  Speeds may be an issue, but at least they have internet.  All other routing/router functions and rules still apply.  

Phones

In today's modern business landscape, there really is no reason NOT to use a quality VOIP service, whether it is hosted or on premises.  When picking a VOIP provider, make sure that they offer call continuity and system redundancy.  These features automatically forward calls in real time to designated mobile phone numbers in the event your internet service fails (see above), which helps to ensure that no VOIP phone call is ever lost.  At Alpha Geeks we provide a hosted VOIP solution to our clients for $27.95 per month per line, hardware (phones) included, that offers not only call continuity and system redundancy, but over 150 other features as well.

Servers
Servers are probably the biggest mission critical item within an organizations "systems" that have the highest rate of failure, or at least the highest chance of failure as compared to the other "systems" above.  There are many different options when it comes to servers these days, and there is no "one-size fits all" solution.  At their core however, all servers have hard drives, which like us, have a 100% mortality rate.  It is not a matter of if they will die, but when.  

Regardless of if you use a physical, hosted, or virtual server, the ability to restore that server in the event of failure is key.  In business time is money.  The longer your server is down, the more money you lose.  For our business clients we utilize a backup solution that takes snapshots of the server at intervals our clients deem appropriate for their businesses.  These snapshots are stored locally on a device, and also mirrored offsite.  In the event that a server goes down, their entire server can be recreated from these snapshots and launched in a virtual environment in about 15 minutes.  All of the endpoints "check-in" to the virtual server as if it were the real thing and business continues.  When the original servers gets fixed or replaced, any data that has changed on the virtual server gets exported to the primary server.  

Misc

Other "systems" can include physical computers, software programs, and network infrastructure such as routers, switches, or wifi access points to name a few. 

For software issues, if a piece of software that a business we manage has a maintenance contract, we require our customers to keep that contract active.  We will work with the software vendors tech support team to fix the issues, but ultimately it is the software vendors responsibility to ensure that their software is working properly (that is what the maintenance contract is for).  

Physical computers, like individual desktops or laptops are backed up either with the same image based backup system we use for servers, or just data only.  When it comes to determining what backup system to use, the key question to ask is "How long can you go without your data?"  If it needs to be available always, then the imaging software is the solution.  If it can wait a day or two, then data only restoration to a new computer is the way to go.  

All other miscellaneous hardware issues are on a case by case basis.  We keep extra networking equipment at our shop and can have it replaced same day if needed.  Some of our bigger clients keep extra equipment stored at their facilities.  Amazon is also a great source of picking up things really quickly if you live in a market that offers same or next day service.  

The above mentioned practices are for any size business.  We have clients as small as one person, one computer up to multi-state companies with many employees.  Size does not matter, disaster prep does.  Remember, those who fail to plan plan to fail.  

See all 20 answers