Report summaries can damage more than they inform

Disclaimer: I’m a skeptic, sometimes against things I’ve studied or mastered. Nothing is left out of the scrutiny crosshairs.

Jakob Nielsen recently released Kindle Fire Usability Findings, which is a report that speaks to the results of user testing and the summary of the user experience of Amazon’s Kindle Fire mobile device. Summary: I shook my head because it seems to be an example of why so many organizations don’t care for user experience and usability testing at all.

When I saw Nielsen’s summarized findings, I thought ‘If the device sold well, do the findings even matter?’. I’m sure many potential buyers of the Kindle Fire wondered the same thing. Below the fold, in much smaller text is the phrase “Our studies of Kindle Fire weren’t intended to advise consumers on whether to buy a Fire device.” Oh, o….k….. Why was this summary released to the public if the intent was not to influence/affect potential buyers? Why not charge for it, just like so many other NNGroup reports to avoid confusion?

Then, that’s when the questions/speculation seemed to pour down from my cranial computer bank…

• Who is this report for then? Designers? Developers? Managers? Consumers? All? No one?
• I’m sure Amazon did their own testing. Why would Amazon sell it anyway if all of this were true?
• If the report was speaking to the UX viability of 7” tablets, why does the summary seem to be device specific rather than device agnostic?

Furthermore, there are lots of ambiguous phrases like “magazine reading experience could be good but actually is miserable” and “7-inch tablets have either a glorious future or will fail miserably” which make me question the validity of the report (and some might question the discipline) in general.

On the other hand, you have Amazon selling/shipping a projected 3-4 million units of the Kindle Fire within the first few (~8) weeks. And yet, this is a device that has a “disappointingly poor user experience”.

Biggest takeaway from Nielsen’s summary “UX will be great if the money is there and poor if it’s not”. Sounds like a valid conclusion from any other project…This would be consistent with the links to the full 293 report ($298 for single license, $598 organizational rights) and the Usability Week Conference registration links ($769 - $3532 in the US) at the bottom of the page.

Conclusion:

Can we refrain from releasing summarized information like this whose intent is not to influence consumers but winds up eroding the perceived value of UX and usability by the general populace?