The Tinkerman
Earlier this year, Claudio Ranieri took unfashionable and unfancied Leicester City to the English Premier League title, one of the biggest upsets in sporting history.
Before this, Ranieri was probably best known for his time as manager of Chelsea, where he acquired the nickname ‘the Tinkerman’. This was bestowed upon him by the media who observed that even when his team were successful Ranieri couldn’t resist changing the personnel and tactics, trying new formations and new players in different positions.
Sometimes his changes were successful but sometimes a winning side was ‘tinkered with’ and lost the next match, which invariably left fans and pundits asking ‘why change things?’ The suspicion was that Ranieri was never happy and just couldn’t resist making adjustments.
I’ve been reminded of this recently.
iQuit
A while back I gave up on using iTunes as my source for purchasing digital music and switched to Amazon. There were a few reasons for this. Firstly, as someone who uses iTunes for Windows, I am hard pressed to think of an application which has consistently infuriated me as much. There have been problems with installation, performance and with duplication of elements of my music library. The interface is confusing, bordering on unusable.
There was an incompatibility of Apple file types with Sonos (my wireless music system). This was frustrating as it required conversion of music downloaded from Apple into an mp3 format. Amazon however appeared as a service for my Sonos player which meant I could stream Amazon purchases directly to speakers in different rooms.
Another source of irritation with Apple came when I moved from the UK to Australia, switched to the Australian Apple store and found that I could no longer download purchases made through the UK store.
All in all I wasn’t hugely pleased with my experiences as an iTunes customer, so the move to Amazon music made sense.
I installed Amazon’s music player software on my PC and was very happy with it. I could order music in mp3 format and download it easily to my network storage where Sonos would pick it up. What is more, any music purchase I had ever made from Amazon (even CDs from many years earlier) was available in digital format. I still had to negotiate iTunes in order to import music and set up playlists for my iPhone but I was very happy with Amazon’s product.
A few weeks back I noticed that Amazon music player had been updated on my PC. I tried to open it to download an album I had bought, but I was unsuccessful. The application seemed to be having problems starting up. I restarted my machine and managed to open the application, at which point it attempted to install my music library. Here it sat for many minutes with no information as to how this process was progressing. I had plenty of opportunity to contemplate the new look and feel of the product, but no opportunity to use it.
In the end I gave up and went to the Amazon website where I could download my music. I have attempted to use the Amazon player several times since and have noticed the look and feel change again, but as yet I haven’t been able to reach my library or download any music.
Checking the version history reveals that there have been a flurry of updates to the app since September. This is in keeping with Amazon’s continuous delivery model, intended to meet the rapidly changing demands from their vast customer base.
Unfortunately for this customer, the result has been that a product which I enjoyed using, which was reliable and performed well is no longer available to me. What is more, the Sonos service which was so useful to me is now only available to Amazon Prime customers. Perhaps I could upgrade to Amazon Prime. Perhaps I could invest more time in investigating the problems I’m experiencing. Or maybe I’ll just find another product which works for me.
Quality in a rapid deployment model
This morning I was again reminded of Mr Ranieri when I read this article by Sally Goble. It is a thought provoking piece for anyone interested in testing or in software quality.
In summary, Sally explains how the team at the Guardian use ‘light touch’ testing before releasing to production and then use techniques to monitor production code, produce alerts, toggle features on and off and to release fixes and updates quickly and regularly. They accept that there will be problems in production but aim to improve their methods of dealing with them in order to reduce the impact on people using their website.
This challenges some conventional thinking about testing and will no doubt upset some testers. I would question what the approach really means in terms of quality and how the experiences and responses of readers are affected. I would also like to understand more about how much consideration is given to quality criteria other than the value of features and the performance of the site.
More pertinent to this blog is the question of ongoing value and consistency. The means to deploy code quickly and frequently has benefits and presents opportunities to improve quality for customers, but it may also have drawbacks.
Updates may require downloads, installation and configuration or they may change the relationship with devices, operating systems and other applications which they interact with. They may be inconsistent with earlier versions in how they look, feel and behave or they may introduce features which need instructions or support.
Each of these threats to quality is worthy of attention. A customer’s experience could be affected or their perception of value altered. In some cases, such as my experience with the Amazon music player, the result may be that the product becomes unusable.
The temptation to tinker with something which is working well and keeping customers happy may be overpowering. Just keep in mind that tinkering doesn’t always lead to better results.
richrtesting 