Questionnaire+Questions+...+brainstorm

Hi all, Sorry, I only just reviewed the Survey Monkey so far, so please disregard my suggestions at this point if you like. I think the open-ended q's are right on, but still wonder about our rating system (see below for my thoughts)...JJ Nielsen also reminds us: The simplest solution to survey bloat is to **ask fewer questions**. Ask questions that address only **your core needs** and **skip the subtleties**. Surveys are not great at gauging minor differences anyway — you need direct observation for that." URL: @http://www.useit.com/alertbox/20040202.html
 * "Survey bloat** is a natural consequence of having a diverse group of marketing managers, all of whom want customer feedback on their special issues. Please resist the temptation to collect all the information that anybody could ever want. You will end up with no information (or misleading information) instead.

Hi all,

Thanks for so much brainstorming already! I wonder if we need to be quite so detailed about every little thing in our design? I would suggest that we focus on the main, pertinent issues ( such as, preference of tutorial format, use and clarity of voice over, written instructions, clarity of directions, meanings of icons match the associated text, and the like) As Nielsen states:

"Beware the Query Effect
Whenever you do ask users for their opinions, watch out for the query effect: People can **make up an opinion about anything**, and they'll do so if asked. You can thus get users to comment at great length about something that doesn't matter, and which they wouldn't have given a second thought to if left to their own devices. It's dangerous to make big design changes because "users didn't like this" or "users asked for that." If you ask leading questions or press respondents for answers, they might make up opinions that don't reflect their real preferences in the slightest. For example, if you quiz people about your visual design, you'll inevitably get comments about the colors, even if they're not particularly important to the users. On the other hand, if you hear people mention the colors (unprompted) while they're using the site, then there's probably something to consider. (Say, a comment like "Wow, this neon blue really hurts the eyes," or a more positive statement like "The ultramarine is nice and calming.") at URL: http://www.useit.com/alertbox/interviews.html

I was wondering about the way statements and choice is expressed in the questionnaire format on page **312** of Preece et al. See what you think...can this style be replicated in Survey Monkey? So for, example,
 * Strongly agree, Agree, Neutral, disagree, Strongly Disagree**:
 * 1) The written instructions in the tutorial are clear and easy to follow
 * 2) The recorded instructions in the tutorial are clear and easy to follow
 * 3) The tutorial clearly outlines how to add pictures when creating a new story
 * 4) The visual layout of the tutorial is informative and helpful
 * 5) Each icon has a clear and helpful meaning
 * 6) The animated tutorial is a reasonable length
 * 7) The written tutorial is a reasonable length
 * 8) //etc. if needed?// if you like?

There should then be a section for more open-ended responses: Such as for comparison of the tutorials for preferences, effectiveness, and efficiency (for learning)? Or, add any comments or recommendations for changes to overall design and the like? Here is where Robin's question would be inserted: You have used both a static and a dynamic version of the tutorial. Which version do you prefer and why? JJ

Hi everyone,

Thought I'd get a jump on the questionnaire. We can use this page to record possible questions. As noted in the Production Paper, we were going to ask open-ended questions regarding the effectiveness of the tutorial, ease of use, memorability?, learnability?, pacing?, comparing the two formats, as well as questions about icon clarity, voice over clarity, instructional vocabulary ... anything else? anything less?

I started a survey in Survey Monkey that we can use for the survey - if people like that idea - it would make collating/translating information easier, I think.

Okay, I'll get it started:

You have used both a static and a dynamic version of the tutorial. Which version do you prefer and why?

Okay, I must be avoiding something - I went ahead and created some questions in Survey Monkey - I'll send a link/or link it here - depending on which I can manage! Take a look - it's a starting point - we can add/change, edit, delete and so on.

Thanks!

Beginning survey link:

[] Hello all, I like the questions in the Survey Monkey, I was wondering if some pictures or graphics could also be shown for the users to recollect which tutorial we are referring and especially when they have to remember the aesthtics and ease of use?? Just a thought.. Though I'm not quite sure if survey Monkey would allow this feature? since we are so involved in the project, answers would be very natural to us, however I was thinking from the point of view of a novice user?

1. Icons- a) Did you like the icons in the tutorial?



b) Were the icons clear?



c) Is there any icon you did not like?



d) Any suggestions for improvement?



e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of colour is excellent.



2. Colour- a)Did you like the use of colour in the tutorial?



b) Was the colour too bright ?



c) Was there any colour you did not like?



d) Any suggestions for improvement?



e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of colour is excellent.



3. Voice- A ) Did you like the use of narration in the tutorial?



B) Was the voice clear and instructions easy to follow ?



c) Was there any aspect in the voice narration you did not like or was annoying ?



d) Any suggestions for improvement?



e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of voice is excellent.



4. Animation- a) Did you like the use of static animation in the tutorial? (the one without the voice)



b) Was the animation clear ?



c) Was there any part in the animation you did not like?



d) Any suggestions for improvement?



e) On a scale of 1 to 5, (where 1 represents strongly agree, and 5 represents strongly disagree) please specify if the use of animation is excellent.



5. (Semantic differential Scale)


 * // Instructions: for each pair of adjectives, place a cross at the point between them that reflects the extent to which you believe the adjectives describe the **Static tutorial, (the one without voice).** You // should place //only one cross// between the marks on each line.

Attractive l………..l………….l………..l………..l………….l…………..l………….l Ugly

Clear l………..l………….l………..l………..l………….l…………..l………….l Confusing

Dull l………..l………….l………..l………..l………….l…………..l………….l Colourful

Exciting l………..l………….l………..l………..l………….l…………..l………….l Boring

Annoying l………..l………….l………..l………..l………….l…………..l………….l Pleasing

Helpful l………..l………….l………..l………..l………….l…………..l………….l Unhelpful

Poor l………..l………….l………..l………..l………….l…………..l………….l Well designed ||

6.


 * // Instructions: for each pair of adjectives, place a cross at the point between them that reflects the extent to which you believe the adjectives describe the **Dynamic tutorial, (the one with voice, text, graphics and colour).** You // should place //only one cross// between the marks on each line.

Attractive l………..l………….l………..l………..l………….l…………..l………….l Ugly

Clear l………..l………….l………..l………..l………….l…………..l………….l Confusing

Dull l………..l………….l………..l………..l………….l…………..l………….l Colourful

Exciting l………..l………….l………..l………..l………….l…………..l………….l Boring

Annoying l………..l………….l………..l………..l………….l…………..l………….l Pleasing

Helpful l………..l………….l………..l………..l………….l…………..l………….l Unhelpful

Poor l………..l………….l………..l………..l………….l…………..l………….l Well designed ||

· Ask them to perform the task from the beginning to the end by giving them instructions.( something similar we had done before) · Ro leplying pg. 330

.I don't know why wiki is taking away the boxes I made, so pl. find attached a copy of the same.Thanks, Vg