Over the years, I’ve noticed that there are two types of coders
- Those who don’t really see testing as something worth their time
- Those who test everything
I fall into the first camp. There’s probably a transtion from 1 to 2, but it seems to be an all or nothing type arrangement to me. You don’t come across many coders who do “some” automated testing.
Now, of course, we all test functionally. You’ve just added a new feature to your untested code, so you run it and test it out. Try and break it. Often, it will break quite happily. You missed something obvious and then you fix the code, retest, and all looks good.
I run quite a lot of code like this. As do many others. I think, secretly, many coders know that test suites are a bit of a con. You may run 5000 tests and everything passes.
So, therefore the code is good?! Bug free? Not necessarily…
I mean, think about it. You are going to test every permutation of permissable inputs? Or outputs?
Nope (unless you’re sqlite).
For a lot of code, it doesn’t really matter.
- You run it
- It works
- Something breaks
- You fix it
- You run it
That seems reasonable to me. And it’s exactly the same process with an automated test suite except there’s a write tests step.
Small time coders can get away with dealing with bugs as they come up. I’ve had a few chewy moments, but if you coded intelligently / defensively in the first place, you’ve built workarounds in to handle minor bugs.
My Workflow
When I start a new project, I initialize a Mercurial repository, bring in all the files I know I’ll need right away, and then start coding. I work alone, by the way, so there’s no corner of the code that I haven’t touched.
I work on one feature at a time. Anything that feature might touch, I functionally test as I go.
Example
Last year, I wrote a web based shopping list for my partner and me to use instead of a paper list. We’ve been using it every week since, and generally it works perfectly.
But, eventually a very annoying bug showed up. This cropped up after it running for about six months. What went wrong was she created a new list, and started to add items. And then she got a 500 error (Internal Server Error).
Now, I’d set up my database so that each shopping list was indexed by date. So if she created a list today (2021-06-11) and then tried to create a new list on the same date, she’d be prompted whether to add to the existing list or whether she wanted to delete the existing list. The reason being; my partner will create a new shopping list every Wednesday, add all the items she knows we need, then on the Thursday, one of us ticks off the items on our phones to remove them as we’re putting items in to the trolley.
But on this day, she created a new list (a week after the old one), started to add items to the list and got the 500 error for nearly every item she tried to add! I was at home at the time, so SSH’d into the server to have a look what was going on. The traceback pointed to an sqlite integrity error.
It turned out that instead of creating a new list, she was adding to the old list. Not her fault as I’d coded it that creating a new list would close the old list, and make a new one. I checked all the relevant code, and everything looked good.
When I designed the software, one of the features I thought she’d like, was being able to look at previous lists, and just click each item to add it to the new list. She generally gets a lot of items each week that are the same as every other week.
Here is a screenshot of the /lists page:
So from here she can click on the first date (or any date) and every item added on that date can be clicked and that will add it to the current list:
If the item has already been added, it won’t be added again.
It was on the /lists page that I realised that a new list hadn’t been created. The solution was to delete the old list, and create a new one.
Conclusion
I never found the actual cause of the bug. I think it was possibly a caching issue on her phone. It has never showed up again. Even if I’d written tests for everything I could think of, I wouldn’t have considered the possibility that today’s date would be anything other than: today’s date!