QA: Clicking on Every Link08.31.11
I recently got a new phone -- the Motorola XPRT [link]. It's a fantastic phone but I went through quite a bit of deliberation between getting it or the iPhone 4 (or wait for the iPhone 5.) A number of years ago, a friend of mine (who was still at Google) was able to get me a free Nexus One so I switched from the iPhone 3GS to that. There were some great things about the Nexus One / Android -- in particular, I'm a heavy Google user and the integration between Google Apps (Gmail, Calendar, etc.) and Android was very seamless. It's quite clunky on the iPhone. Most importantly -- the integration between Google Voice and Android was awesome. I was able to not only receive calls on my Google Voice #, I would be able to dial out and the recipient's caller ID would show my Google Voice #. Amazing.
However, the downsides of Android is I would estimate that everything about it was about 20-30% degraded relative to the iPhone. How the keypad worked, the phone itself, the screen layout, the marketplace -- literally everything. If you compared two features -- the iPhone version would simply be better, more beautiful, and more efficiently and elegantly designed. It was a really tough call to go with the XPRT -- but I did so primarily for the excellent Google integration and the fact that it has tethering (so I could use my phone to get my laptop onto the Internet -- which is also pretty awesome.)
So what does this have to do with QA? Here's an example of the 20-30% degradation that I mentioned. Your phone has a number. However, one of the key points of Google Voice is that it serves as a layer on top of that number -- this is valuable for all sorts of reasons. However, that value is greatly diminished if that number isn't consistently shown. For example, if people can call your Google Voice # and it rings your phone -- that's good. But once you can call people using your phone and it shows up on their phone as your Google Voice #, that's huge. That's what Android is able to do now.
Similar to this -- text messaging. Here's how it works on my phone. There's a base text messaging app. I also have Google Voice. You message my Google Voice #, it buzzes both my Google Voice app and my base text messaging app. That's two notifications and annoying to clear out both. I obviously want this consolidated. I don't want to just use the Google Voice app because what happens if someone texts my base phone #? (Why would this happen? Well, to be blunt, I've had Google Voice be very slow on texts received -- sometimes hours or more delayed -- so if I'm out and about and trying to find someone or what not, I may prefer the reliability of my base text messaging app.) However, if I use my base text messaging app -- it texts out as my base phone #, NOT as my Google Voice # and I don't have a choice. In the case of dialing out with the phone, I can configure it to always use Google Voice, never use Google Voice, or have it ask me. Obviously I need the same option here with texting to have the same consistency.
This may seem like a very specific example -- however, I would argue that this is a feature that's pretty fundamental to Google Voice and to Android considering how important texting is.
The way I like to QA products is to literally click on every link. It's really boring -- but I find a preposterous number of bugs just by making sure every little thing works. It also gives a level of comprehensiveness to your knowledge of how the product functions that's almost impossible to get otherwise -- because you've seen every page on the site.
This problem that I talked about re: Google Voice / Android is a non-trivial problem. It's not the type of problem one would see after a few minutes. It's the type of problem that one would see by fundamentally using it. I didn't realize it right away and in fact, I only realized it when one of my friends wrote back, "Who is this?" -- because I had somehow been pretty fastidious replying back using Google Voice. (On a side note, there's an option inside the base messaging app that says, "Enable Callback Number" and then you can edit that callback number -- which I thought would change this feature and allow my Google Voice # to show when I texted out. It doesn't. I believe it only appends a callback # to the text message the goes out -- which is a pretty crummy workaround.)
I want to say something else about all this. It's possible there's a way to make this work and I haven't been able to figure it out. If there is, that's almost worse. After a lot of research (and testing out all sorts of possible combinations with another phone) -- going through what I imagine was every possible menu and searching on Google for information -- if there is a way to do it, I can't figure it out. On the iPhone -- this stuff just works. At the minimum, if this feature was there on the iPhone, it would be clear it was available. Things like "Enabling Callback Number" -- that wouldn't be not clear what that feature did when you enabled or disabled it.
When it comes to launching good products -- there's a certain obsessiveness that needs to occur. It's literally clicking on every link, going through every screen, thinking through all the scenarios in terms of making it as good as possible. Then it's rallying your team to make what might be minute changes. Some people get the value of this, some people don't. I've gotten the eye rolls and dismissive emails because, say, the font is wrong on a small section of a page. What's the big deal? It's just the font. It's sloppy work. That's not how it should look. Ever use a product where just one thing was off? Maybe it's a car with a great engine, look and feel, but they cheaped out on the interior. Totally diminishes the entire product.
Some people get this and some people don't. Your team may hate you for this level of detail. They might not want to make these changes and may simply want to move on to the next "big" release. Sometimes you have to make compromises because that's the only team you got. Other times, you should get new team members. People who accept sloppy work shouldn't work on your product. Of course, this level of detail needs to be balanced with getting things out the door -- but I think the distinction here is this: you can get out a problematic product to get it out door -- but it's not acceptable to then wash your hands on the product. There then needs to be a focus on working through that QA / bugs list. The iPhone 4 is worlds better than the original iPhone -- why? They kept improving it. They kept rethinking it. They kept clearing through all the litany of bugs.
I don't want to soften this post by saying things like, "I've screwed up XYZ or launched this buggy product or if you watch or use X that I worked on -- here's everything you can find wrong." Because all that is true and I've made tons of errors and mistakes. Sometimes you also make tradeoffs. Speed in exchange for bugs. My point here is around attitude -- it's wanting that last mile and learning from the times you didn't get there.
Finally, this posting isn't meant to be a particular critique on Google Voice or Android even though that's the example I used. After all, even after feeling the way I did about the look and feel of Android vs. iOS -- I still went with an Android-based phone because of all the great things that it does do. It's just that I think there a lot of easy places it could get better -- maybe a lot better. Who knows, maybe they even have this particular problem on their to do list. My larger point here is the level of care and hands-on-ness required is so much higher than I think most people realize. The best way I can describe it is clicking on every link. That level of care leads to (or eventually leads to) high quality releases and products.