Chicago Web Conf 2012

Chicago Web Conf 2012
Chicago Web Conf 2012

I presenting a talk yesterday at Chicago Web Conf 2012 on A/B Testing Your Designs With Real Users, focusing on the role of A/B in digital product development.

I showed at a high level how we use the A/B testing solution Optimizely where I work at Cars.com to test our designs in ways that are quantitatively measurable. Like other A/B test solutions, Optimizely allows you to quickly create and implement tests and start collecting data immediately.

In addition to showing some examples of our tests and results, I also tried to show how A/B testing can fit into a company’s larger user experience testing strategy. My goal was to leave attendees with five key takeaways regarding A/B testing:

  • It’s not a replacement for testing with individuals.
  • Small changes can lead to big improvements.
  • It takes little effort to test those changes once a solution is in place.
  • This kind of testing can take some of the team friction out of the design process.
  • It can help you get better solutions to market faster.

My slides are available (PDF format).

Why Outlook.com is Good For Email

Microsoft launched a preview of its replacement for Hotmail this week (rebranded as Outlook.com) and this could be a good thing for all web-based email users.

While I’m not normally impressed by the user experience of Microsoft products, my first impression of Outlook.com is that it is a significant improvement over Hotmail (which was an ancient and dying beast, to be honest).

Interesting side note: I actually had to create a new Hotmail account and then upgrade to Outlook.com since I hadn’t signed into Hotmail in at least 10 years or so.

Outlook.com is exciting because it could introduce new competition into the web-based email market. And competition is good; it keeps us on our toes and forces us to constantly focus on improving our products. The screen shots in this post clearly show Microsoft is moving to a more simplified user interface that makes better use of layout and content organization. The product appears to be taking cues from the more simplified design of Google’s Gmail.

The annoying ads that plagued Hotmail have been reduced and social media integration with Facebook has been added (although I did not experiment with it).

There are plenty of beta-software moments, like how editing your profile takes you back into the Hotmail user experience, but it was clearly labeled as a preview.

Engadget gives the Outlook.com preview a pretty good review. CNET also gives the preview pretty good marks.

Would I switch over to Outlook.com? Probably not, mostly because I have committed too much effort to living with Gmail and Yahoo Mail. And in the mobile-centric world I live in, most of my email is sent and read from a mobile device anyway. But if Microsoft can keep its smaller Hotmail user base from moving to Gmail or Yahoo, it will probably consider the effort a success. And if Outlook.com causes the other web-based email providers to keep looking at ways to improve their user experience, we all win.

Hotmail
Hotmail
Outlook.com
Outlook.com

Cars.com Agile Transformation Webinar Notes

I participated in my first webinar recently, sharing some learnings from the Cars.com agile transformation. The webinar was sponsored by consulting firm ThrivingOrg and focused on what we learned during a 10-week series of pilot projects undertaken by several product teams. Here are some notes I prepared:

  • Transition from waterfall to agile was difficult at first for user experience team members because most of the techniques used by classically trained user experience people came from the waterfall world. Things like wireframes, a fully designed user experience and flow, and pixel perfect Photoshop comps don’t work well within the context of two-week working sprints.
  • Our profession needs to get better at using Lean UX approaches and shifting our work product to sketches, prototypes that can be used for documentation and user testing, and more lightweight guerrilla usability testing.
  • Interactions between agile product team members are very different than in waterfall project teams. Developers and UX people are working much more closely and more often now (and that’s a good thing).
  • Dedicated team co-location is essential. Having the product manager, UX and visual designers, and developers and testers near each other fosters frequent informal collaboration and reduces the number of structured meetings you need.
  • Daily standup meetings and co-location should drive greater appreciation of the other roles on the team and the constraints they are working under.
  • Failing is good. It shows you your limits and helps teach that an agile product team rises or falls together.
  • When first transitioning to agile, start with easier to implement features or even a backlog of defects. This allows the team to learn the process first without also having to worry about designing and developing complex functionality.

If you are new to agile software development and Lean UX, these resources can get you started:

Cars.com Agile Transformation Webinar Series

Cars.com product development team members recently took part in a series of three webinars sponsored by ThrivingOrg in which we discussed the different phases of our on-going transformation from waterfall to agile software development methodologies. The three webinars focused on three different phases of our journey, which is still under way.

I took part in the Phase II webinar, in which we discussed how we implemented a set of pilot projects to help us define an agile framework that would work for Cars.com. The pilots lasted several months and helped inform how we approached the later transition of all software development to the first iteration of our agile framework. An audio recording of the webinar is available through the ThrivingOrg website.

Marty Cagan and Designing Great Products

Last week I had the good fortune to spend two days learning about designing great software from Marty Cagan of the Silicon Valley Product Group. Cagan’s approach to creating great consumer software products comes from years of working on wildly successful ones, such as Netscape, AOL, and eBay, and is a radical change from the traditional approaches many companies still use.

Cagan’s two-day seminar presented the topics raised in his 2008 book Inspired: How To Create Products Customers Love along with advice tailored directly to the audience, in this case my company, Cars.com.

One of the key aspects of Cagan’s approach is a focus on a core working relationship between the product manager, the lead engineer, and the lead interaction designer supporting a product. His method places heavy emphasis on a great user experience as a key part of making software people really want to use. He also advocates delegating product design decisions down to the product manager and empowering that person to make the right choices, while people managers and other leaders are tasked with doing what needs to be done to develop their product teams so they can be trusted to execute.

The book is a quick read and well worth the time as it opens up a new way of thinking about product management and software design.