Been Looping for a While / 2 AM Debugging

It has been a while since I've written here on the blog. The primary reason was that work was just too busy. For two and a half years I was working on new website for our retirement plan participants and sponsors. Almost 20 internal employees and lots of external resources were working on the development of this site. It has lots of new features compared to the old site and everything had to be built from the ground up.

My role in the project, along with two others, was writing OmniScript to pull the needed participant data from our recordkeeping system and send it to the web in the form of XML. As each retirement plan has it's own set of features and options, this involved a lot of variations. I'll write later about some of these various aspects and what I learned through the project, which was a lot.

After a lot of work and some optimization, the site went through extensive UAT testing and was finally launched to participants at the beginning of December. Overall the launch went very well and it was great to see all of that work finally in use in production. Naturally there were some bugs that were found over the first few weeks and that continued to keep us busy. Finally, though, things are slowing down, at least until the next big project, and I plan to get back to blogging about my experiences in learning programming.

2 AM Debugging

I just wanted to share one programming story, going along with the new website. When you become one of the experts on a new website that allows people control over their money, their retirement savings, then when there's a problem, your phone can ring at any hour of the day or night. I discovered this on New Year's Eve. I hadn't been in bed for long when I noticed at 2:00 AM that I had a couple of missed calls and texts. With the turn of the new year, something was preventing anyone from logging into the website.

So I woke myself up and called my boss back and proceeded to get logged in to find out where the error was coming from. I was probably not at my clearest thinking, and as it was coming from a script that I had not worked on, it took a little while to figure out the problem. By 3:00 AM I had found the problem, a logic bug that, with the turn of the year, was causing the low date in a date range to be higher than the high date. This produced a fatal error.

My boss and I talked it over and decided on a temporary fix that would work until we returned to the office. By 3:30 I had made the fix and people were once again able to log in, though who's trying to log in to their retirement savings site at 3:30 AM on New Year's Day, I do not know. 😃

i++