Stage 2 - Piloting ideas with staff and with students
From Feb 2008 to May 2008, we ran a number of activities to help us learn more about our users. This is a summary of what we have done:
- We ran an experiment in which we invited students to carry out 3 simple search tasks looking for local services using a web page. The page contained both a text entry search box and a tag cloud. Once the students had completed the task, we interviewed them about their feelings and experiences with the searches they had carried out. In all 116 students were interviewed. 89% of them used the tag cloud at least once. Most of them did not realise that the tags were in alphabetical order. Find out more ....
- We ran user centred design sessions with several small groups of students, in which we invited them to participate with us in designing a system to help them visualise their skills. Whilst it was difficult to recruit students into the groups, those students who did participate were extremely engaged with the sessions.
Read the report on the sessions ....
The sessions gave us a really rich insight into what students find difficult about engaging with skills, as well as ideas for ways of addressing the issue. - We followed this up with an open card sorting activity in which we presented students with a number of goal statements and asked them to group them in any way that made sense to them. The statements were based on the previous sessions with students, and on discussions with careers experts at the University. The students were recruited through Facebook using a snowball type approach. 9 students took part.
You can see more images from the open card sorting activity on the skillclouds flickr account.
www.flickr.com 
Go to skillclouds' photostream
The results of all the card sorting were fed into a clustering analysis algorithm producing groupings of goal statements in the form of a dendrogram. Cluster analysis was carried out using thexSort program, to produce a dendrogram showing the patterns of the clusters.
You can view the screencast on card sorting and producing the dendrogram here.
The next stage of the project involved mapping out our open card sorting and user research into a user journey through skills.
-
Skills - Constructing a user journey
From our user research there began to appear a very clear process the users were going through in exploring their skills.
To help us understand the user journey and build the relevant materials to meet the users' needs at each stage, we broke this down into four 'stages'
This user journey is based on the building of knowledge on top of what the user already knows, with stages acting to reinforce or reaffirm knowledge, and offering more in depth information if the user requests it. Users can access parts at any stage, building up the tools to reach their own individual journey goal.
Stage 1 - Understanding 'Skills'
Tags for this section included : Skills, Information, Understanding, General, Nature (of Skills)
Users requested a first building block - a basic generic understanding of what 'skills' are, and why they are useful.
"I'd like to know what a skill is, and do i even have any?"
User testing sessions revealed a general lack of understanding of the term 'skills' and 'transferable skills' and what this implies.
"How will they help me, what can I do with my skills ?"
"How do skills help you at uni?"
"understanding why they (skills) are good to have"
"Understanding what exactly is a skill - it's the start of your development"
Stage 2 - Accessing my skills
Tags for this section included : Skills, Information, Specific, Application
Users at this stage may have an understanding of the Generic information in Stage 1, but it is not essential.
Users requested an entrance point into their own skills.
Primary aim - displaying users' skills
"I'd like to review my degree skills, and see what skills i have"
Secondarily - allowing users to accessing contextual information about the skill and where they have applied a specific skill during study.
Users showed interest in accessing both 'transferable' and 'subject specific' skills.
Stage 3 - Getting to know my skills
Tags for this section included :Skills, Course, Feedback, Specific
Users requested a method to access deeper information on skills they have gained during university.
Definition based - Users requested information specific to a skill they have - descriptions, meanings and alternative words to describe the skill, help understanding a specific skill.
"most useful for students -making skills clearer "
"Inbound information, information I'm gathering about myself specifically"
"Would help with a confidence thing.... just to show people what they are getting in their degree and how it's useful"
Students raised the ideas of knowing your skills, and skills being a part of self-awareness
"Getting into your own skills"
"About self awareness"
"Taking control of your skills"Assessment based - (relating to Stage 2 , where they have applied skills) personal feedback from tutors, how skills were used in a course, related marks and grades and starting and finishing dates for the skill study.
Users in the card sorting activity found it difficult to define a link directly between a skill and a course.
"May have used a skill in lots of courses, but be rubbish at it!"
"Don't see how skills can relate to a grade"
One second year students questioned if assignment grades and feedback are skills related
"Marks for assignment and feedback from tutors are less skills related"
Starting and finishing data for a skill also brought out some interesting questions.
"do you stop learning ? " (In reference to skills and time)
"I don't really get 'start and finishing data' - it's like riding a bike, you don't loose a skill"
Overall the most popular information goal statement within this stage was a request for personal feedback from tutors which related to skills.
"Marks for assignment and feedback from tutors less skills related"
"Never received any feedback that mentioned skills"
Students in their second year saw the outcome of this stage as important in making choices
"Important to know what your skills are before you make a path"
"like to have the option of choosing modules based on skills I have"
Second and first year students motivation centered on self-awareness.
"All about the person looking at their own skills - not from job perspective, just cos they want to know what skills they have"
Final year students saw this stage as a way of reviewing and summarising their achievements and skills in preparation for a job, further study or their next steps in life.
Stage 4 - Applying my skills
Tags for this section included :Skills, Applying, Jobs, CV, Employment, Goal
Users requested a way of using their skills to construct a CV, or application - how to demonstrate and use a skill on a CV, jobs which a skill applies to and how to arrange your skills on a CV.
Students in their final year often found it difficult to recall skills they had learned during university
"When you get application forms and they say 'Give us an example of how you demonstrated this skill' - and you kind of think 'I have NO idea! I'm sure I can do this job but ..'"
Users associated the building of a CV as the point at which they would currently add skills gained externally and skills of their own - it was not generally seen as an important goal or as part of the previous stages.
Students had difficulties identifying if their degrees related to a job
"Much easier if given starting point - What are your skills in relation to working"
And how to apply degree skills in the real world "Using the things you have learned"
Listing jobs your skill set is applicable to gave mixed reviews
"Not sure like the idea of jobs based on your skills - possibly too idealistic"
"I'd like a job where I can use my skills - how?"
Users in the card sorting activity were found the idea of 'One line to copy and paste into a CV' either unrealistic or something a graduate should not need to be given
"Spoon feeding - if you're a graduate you shouldn't need 'One line to copy and paste into a CV' "
What does the user journey show us?
From an institutional perspective, this has been a useful piece of work on how students learn about skills. It's providing a valuable focus for our project and the evidence suggests the tool will form a 'bridge' for students to access their skills information.
Download the skillclouds user journey as a word document >>
- We met with Paul Cecil, Head of the Academic Office, to discuss how feasible it would be to use an interface such as the delicious interface to support staff in entering skill data for the University's various courses (aka modules). The thumbnail screenshot below shows the "My del.icio.us" page for the Sussex Skills user. As an example, a few pages from the Sussex Course Directory have been tagged with some example skill tags (the skills are purely for illustrative purposes).
We have identified a number of potential barriers which we need to work through before running any further development sessions. Find out more ....
Research consent form for students participating in SkillClouds user centred design activities.
Research consent form for students participating in the tag cloud search experiment.