I had a meeting with SINTEF today where I showed my prototype of a news feed for Urørt. The look and placement of the feed seemed to be liked and responsiveness was ok.
We had a long discussion about what we wanted to test and how we would go about doing so. There were basically two ways we could test the prototype. If time permitted it could be possible to perform both types of tests:
Real world study of users over time. The users would have to be experienced and frequent users of Urørt and could be recruited directly from the site. Somewhere around 50 users could prove to be optimal. Ask the participants some questions before they start the experiment. Let them use our prototype for about two weeks, then ask the same questions again at the end of the period. This way
Controlled study of about 20 users in a lab with video recording. Inexperienced Urørt users. User scripts and extensions could then be preinstalled. We wont get any feedback on Greasemonkey from end users. Half goes right to an enhanced version of Urørt and the other half goes to a plain vanilla version of Urørt. Switch after 10 minutes. Don’t tell them about the feed, just notice their behavior and measure satisfaction.
The last solution would not require a long running system which would have to be monitored for breakage against possible Urørt updates. It will probably also eliminate a need for detailed installation and uninstallation instructions.
What are we looking for? Primarily interested in navigation, but could also be interested in getting feedback on how well Greasemonkey prototyping is perceived by end users. What kind of behavior signals increased navigation?
How easily can the Greasemonkey installation process be made? Need a low barrier to entry, and the solution would need to deliver so that people don’t UN install the software before the period is over.
It could also be interesting to use a camera to measure users focus on the web page. Compare the news feed to a plain version where a news article is percent. News feed is composed of much more detailed information. The news article is primarily a large picture.
We decided that I should define what navigation is. I’ve already defined this in my thesis as clicking hyper links inside the browsers so that new information appears. I make a distinction between social and asocial navigation. This is also well defined within my thesis. Social navigation is navigation where current users leverage traces left by past users’ actions for finding their way in a web page.
In addition I would have to think about what kind of actions on specifically Urørt which could be defined as being both navigation and social navigation. In addition I would maybe have to think about other features the news feed provides in addition to navigation. It also functions as a means to keep updated on behavior of favorites.