I’ve been a DevOps Engineer before anyone called it DevOps. I’ve seen every variation on how to approach troubleshooting an issue. Well, I thought I had, until yesterday, when we were having trouble receiving data from Weibo, The Chinese hybrid of Facebook and Twitter.
We checked the networking to the server on our end which is responsible for ingesting data from Weibo. We had reached out to Weibo, who provided data showing they were functioning normally. We attempted manual versions of emulating the connection process to get the data… and so on.
Finally someone asked the not so obvious question, Have you tried it form Singapore?
This highlighted the completely new world that cloud computing has opened. And that even if you host your own infrastructure, that you need to have a toe in the pool that is the cloud.
Trouble shooting an IT problem is a process of identifying the variables that are in place, and isolating them one by one until you are able to pinpoint the one that is at or near the issue. ”Have you tried it from Singapore?” is the new tool in the DevOps belt now that allows us to not stop at the point where we have to concede that the issue “must” be somewhere in the route between us and them.
It’s now possible to have, in an instant, a server up and available in Cloud Data Centers around the world. This allowed us to test the issue much more local to China. I’m sorry, but that is huge. We were able to open our AWS console, bring up a sever in Singapore and remove all the unknown of transferring data around half the globe, and remove that variable. This is something that not long ago was an inconceivable troubleshooting step. You had to trust your remote partner, and that is no longer the case.
So DevOp, the next time you run into a network issue between you and a provider, remember… Have you tried it from Singapore?