Explaining New Technologies: WCF Usability Studies

It’s important for Microsoft that customers considering a technology have an easy time learning about it and trying it out. One goal of this blog is to understand the issues you run into while learning Entity Framework (“EF”). I’m going to blog about own my experiences learning it. I’m very interested in hearing how your own  learning about EF went. I’m on the EF documentation team, so I’ll be able to consider suggestions for changes to the tutorials.

Writing to help people learn new technologies is not easy. It’s hard to identify and explain all the contextual information that you unconsciously know. As a beginner I will not have this problem.

I’ve been interested in how we learn new stuff for a long time (my wife has been a teacher and school principal, which has probably enlightened me somewhat). As a contractor, learning new technologies was a constant part of my job. I was frequently dealing with poor documentation, and came up with the “23 rule”: any given technology has 23 random, unrelated quirks that they don’t tell you about, and until you learn them, you cannot make anything happen.

My first Microsoft employee job was documenting Windows Communication Foundation (“WCF”) . We wanted to evaluate our Getting Started Tutorial, and decided to do a usability study. You may have heard about Microsoft’s Usability Labs, where we watch and record people through a one-way mirror as they try to accomplish a list of tasks.

We created a task list, and chose people who knew the .Net Framework but not WCF. The tasks included: doing the steps in the Getting Started tutorial; making a minor modification to the app created in the tutorial, and making a creative extension to the app. We kept statistics on how long each task took, whether they successfully completed the tasks, how many times they had to ask for help, etc. We also collected subjective data: people’s remarks, where they thought the documentation was bad, suggestions, etc.

We first ran about 8-9 people through the study. It was a humbling experience to see the problems people ran into. You watch someone get lost or confused, and  want to tell them “just read the next sentence” but the moderator running the study won’t allow that. Or you go “gee, I guess we forgot to tell them about that“. Such incidents often indicate that something may be wrong in the documentation or software.  This phase of the  study generated about 30 documentation bugs.

We then fixed the bugs and repeated the study, and the results were much better.

One of the hardest things when writing beginner tutorials is to provide just the right amount of context. If you provide too much detail, the reader may get lost, and not know which parts of it are essential. Often if there are  several alternatives, it’s better to just describe one of them. On the other hand, if you don’t explain why something is done, the reader may not remember that step.

Another major problem is assuming too much reader knowledge. If you were to write “attach the VS debugger to the process”, well, there may be many readers who don’t know how to start the debugger, much less how to attach it to anything.

In my next post, I’ll actually get started learning EF.


About Rick Saling

I work at Microsoft as a Senior Programmer Writer, currently documenting Entity Framework. Prior to that I did database design work for many years. Non-work interests include outdoors (hiking, scrambling, backpacking), political issues, travel, learning languages (currently working on Turkish).
This entry was posted in Entity Framework, Usability Study and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s