Just last Saturday, 6 days ago, I posted about our recent big successes. Not only that, but I stated “as a general philosophy I believe it’s a good thing to share your successes AND failures“. Well today I’m going to share our big misadventures that happened yesterday. It was a brutal day! Luckily we’ve surmounted everything that was thrown at us and we’re back up and running at 100%.
What happened? Two major things.
The night beforehand we just noticed that our SSL certificate expired. The exact same thing happened to us last year, and I vowed to myself not to let it happen again. I still can’t believe I missed it, but I did. Of course, again like last year, our website was still completely safe it’s just that our website wasn’t a “trusted” site so we got that horrible warning in the web browsers.Obviously the first thing we did is purchase a new SSL certificate. It was installed by early afternoon the next day. But for about 12 hours our purchase page wasn’t officially “trusted” even though it was completely safe (don’t get me started on how useless and a cash grab I think this market is). Although I can’t know for the sure the impact this had on our sales, I can’t imagine it helped.
Secondly, and with a much bigger effect, a small upgrade was performed on our server the night before which didn’t go as planned and eventually caused our webserver to shutdown right around lunchtime. It took about an hour to figure out what was going on and to fix the issue. So for an hour we were completely down.
But it doesn’t stop there. Once we were back up and running, we were getting seg faults from our server. What this means is that the webserver wasn’t entirely stable and would error out. Something was still wrong. Although it wasn’t completely shutting down the server, something wasn’t right and it needed to be resolved right away. Without getting into the technical details, we basically worked with our hosting provider to resolve this issue over an hour or two. The eventually solution that worked was to rebuild our webserver.
The downside of rebuilding our webserver is many of our settings and customizations were lost. So for about 2-3 hours our main page looked horrible. For example the header and menu items weren’t showing up at all. You could still purchase the software safely and securily, but you couldn’t access the menus to navigate the website.
Before I continue let me just say that Servint, our managed solutions hosting provider, was great and worked with us the whole way. They knew what they were doing, and if it wasn’t for them, well I don’t really want to think about it… Yes they do charge a premium, but they’re worth it. When we really need them, they were there. I’d also like to especially thank Tommy from Servint, he was very knowledge and helpful. He was great. He saved the day for us. Thank you Tommy!
Getting back to our adventure, or should I say misadventure, Tommy eventually figured out what happened and resolved the issue. Of course it didn’t end there. My other site FindYourWeddingDress.com was also down due to some other technical issues. Thankfully they were also able to resolve this pretty quickly. We had a few more issues, but everything seems to be completely back to normal now. What a day!
Before I leave, let me share one last thing that really surprised me. During the few hours we were down, or mostly down, we still managed to generated some sales! I’m still shocked by this now. I expected we’d get absolutely no sales during this time. The site was completely down for an hour, then the site was barely operational. I wouldn’t have bought anything from us during this time. Yet we still managed to get some sales. What does it mean? I don’t know. All I can say is that I’m still shocked by this.
And there you go. Today I’ve just shared with you our most recent failures and struggles. This week we failed to renew our SSL certificate on time. Not only that, but we also struggled to keep our website up and fully operational for a few hours. Thankfully we rose to the situation and we’re fully back up and running now. What a day. When it rains it pours!