Well, the project I've been QA-ing for the last five months went live two days ago. I'm the "junior" on the project -- the woman I worked with is considered the "Web QA," and she had primary responsibility. My main focus was order processing -- making sure that orders placed on our web site came down to the AS400 properly. She handled stuff like browsing, gift registry, and XML code.

We always launch new software and system upgrades in the middle of the night, to limit interference with business activity. So the site was suspended at like 2 am, and the web team and the QA were brought in to install and test. I was told not to bother coming in for the launch; they'd have me work on any post-launch issues after the QA was gone for the day.

When I got to work yesterday morning, I brought fresh donuts for the launch team and checked in -- everything seemed to be going smoothly. They didn't have anything for me to do. I was then assigned to my new project.

Today, the senior director of development came in and grabbed my boss. Apparently, gift registry transactions were suspended starting around noon today, when someone in our direct marketing center noticed that GR orders were coming through with the receipient's name concatenated into the first name field.

Immediately, my boss wanted answers. Let's call the Web QA Jane for the sake of the situation. GR was Jane's job. As a QA, she should know how to check the back end of sales. However, she never talked much about it, and nobody asked me to check her sales. I had done some GR orders as tests and never saw anything like this come up. So I told my boss I didn't know what had happened, but that I would take responsibility for not knowing to check Jane's tests, and that it probably came down to poor communication on our part.

Frankly, I don't think this is my fault, but hey.

So then I went into our development environments, and polled all my sales. I didn't see this happen, ever. I checked Jane's sales. Nothing. I went into the live environment to see what was going on. When I DBU'd the right file, I could see intermittent sales where the concatenation took place. It didn't affect anything other than the recipient name, on seemingly random GR orders. I let my boss know of my findings, and checked in with Jane.

The person who had written the GR code found something she had done, where she said she had "tried to get too fancy, which is what I get for not just doing it simply." Nobody on the web team seemed too upset about it, but I was concerned about taking the hit.

I went back to my research, and found a grand total of three orders, in one environment, with the same problem. Two were created by an outside partner in test, and one happened during load testing. There was basically no way I would have found these errors. I have no idea how to duplicate them, since they don't appear to have anything different going on than any other orders.

When I left today (late, from working on this), my boss seemed to have my back on it. I had sent her a spreadsheet with the screwy orders. She was going to query all the orders we had done during the QA phase and examine them. Out of 11,344 orders placed in test, a grand total of 40 appear to have the problem. It's completely random.

It's not my fault, but I feel like it's my fault. My boss sent an e-mail to the higher-ups with our collective findings, and basically offered a theory as to what the orders had in common (which is wrong) and then said "Jane and JT, I'm guessing you didn't test to this extent, please confirm."

For someone who dreads getting into any kind of trouble, this is not a fun night for me. Damn it.