Seminar 2 - Summary

Recollection of discussed topics during seminar 2

• We have learned the importance of evaluating, but sometimes it is hard to user-test something beforehand (scale, patents, secrecy). We also discussed ways to circumvent this.
• For testing our own design, we could use testing in the wild: adding markers and prototypes to the SL stations, and have people try to accomplish tasks (getting from A to B while using our prototype app). This would be testing in a natural setting involving users.
• We could use people from Copenhagen (which has a similar system in place) to heursitically evaluate our prototype and tell us what they feel are the good parts and bad parts of our system when it is adjusted to Stockholm.
• We could also take Stockholmers to Copenhagen and let them see what they like and don’t like about that system, and see how they compare it to stockholm.
• This would be be a kind of triangulation: we see the problem both from the perspective of users who have used it for a long time, and users who are using it for a first time. Their combined heuristics would give us a better view of the actual good and bad sides of our idea.
• Also, Opportunistic Evaluation is a thing that we can actually, practically do (not assuming infinite resources etc). We could even ask the group we’re presenting to what they think, since they are almost surely users of the public transportation. We can also interview people in the subway and present our design to the, ask what they think.
• We discuss how we should evaluate the different parts of our designs, since it is quite broad. It might be good to evaluate each part on its own, ut we also need to evaluate the whole together since we need to know how well the prats cooperate.

Design step 2 - Parallel design

For the second part of the design step we split into two groups, we were given the same pain point with the goal to create a design centered around it. The chosen pain point was: Being unsure on what exit to take. We gave it about 40 minutes before sitting down to compare the designs. To our surprise, both managed to create very similar designs, both featuring an application with some modification to the station.

Low-fi prototypes of both designs

Both groups had used the existing SL-app as a base for the design and then expanded on it. With the pain point being what exit to take, we had the idea of colouring or assigning a letter to the exit so that when looking at your planned route, you could easily see what exit to take. Using colours to distinguish between the exits allowed us to use the already existing "T" that people are used to seeing. As it is something familiar to the user and as such, takes little getting used to.

When it came to making changes to the actual station we were basically on the same page yet again, changing the signs to reflect what exit they point at (pink, grey, blue, A, B). One group also had the addition of arrows on the ground that you could follow to the exit. The visibility and if it's easy to find your arrow when there's a lot of exits or on a crowded platform is something to consider.

Detailed exposition of the design progress in group 1

First we considered the situation when a traveller arrives at a platform, and tries to figure out where to go.

We noted that there are signs at the ends and in reasonable intervals along the platforms. These signs describe the exits by street name and occasionally also by important locations in the direction of the exit. Unfortunately these are often insufficient e.g. if the traveller is heading to a parallel street or further away.

The most straightforward improvement would be to add a longer list of streets and locations for each exit. It occurred to us that sometimes travellers don't know any of the nearby street names but only which general direction they want to exit. It therefore seemed appropriate to have maps rather than signs naming streets.

Each of these maps would show all the exits and the nearby surrounding area. Nearby streets and important locations could also be named in the map.

The exits marked in the maps should be colour-coded, perhaps using textures or multiple colours each to make them unique and distinctive. These distinctive colours could then be used to mark or point to exits throughout the station.

Distinctive arrows could be put at each intersection, or lines could be drawn all along the floor. It was noted that floor lines could be obscured during rush hour. It was also noted that coloured arrows could be confusing if seen by themselves without a map, but this did not seem like a major issue since maps would be at all entrances and at intervals on the platforms. Furthermore, in places where there's no place for a map, where today there's only a sign that names the street the exit opens up to, in those places the same street could be named in addition to the colour coding of the sign.

Next we considered the situation before a traveller arrives at a platform — ideally they want to exit the train onto the platform nearby to the exit they want off the platform, and for that they need to know where they should position themselves in the train well before arriving at the platform. The best time to make use of this information is before entering the train, while waiting for the train to arrive.

It seemed unreasonable to have all this information displayed statically at each station, i.e. all stations having the map described above for their exits, as well as those maps for every other station on the line (and connecting lines). It did however seem possible to have this information in a dynamic application.

To investigate this we looked at an existing SL schedule app. The app, much like the SL webpage, lets you input start and target destinations, along with planned starting time for the journey, and then it shows you a chronology of evens: Walking to the nearest relevant station, what line to take, when the train will arrive, how long the journey will be, which station to exit on.

In the existing layout of the app it seemed fitting to add a suggested wagon to board (i.e. which end of the train, or sometimes middle). Another good addition would be on the exit station to say which colour-coded exit to head for.

The app lets the user see more detailed information about some items in the chronology by tapping an expansion arrow on such items. Continuing this interface the indication for which wagon to take could be an iconified train with three wagons (as seen from the platform), with the recommended wagon to take marked. For new users who don't understand this icon an expansion arrow could be tapped to show an explanation.

If there was a way to measure real-time seat-availability and/or general crowding in the wagons, this information could be used to improve the wagon recommendations.

We realised that navigating by colour-codes rather than Swedish street names would be especially useful for tourists. Unfortunately tourists would be especially unlikely to have an SL route app. This could be solved by having tourist information at station entrances together with WiFi enabled downloading of the app, or by placing touch-screen interfaces to the app on stations.

Individual notes - Seminar 2

When designing a product it is very easy to get stuck in coming up with ideas, thinking about new things to add to the product or just improving the design as you see fit. It is easy to forget that designing should be an iterative process where we should evaluate the product every so often. An obvious reason to why evaluation is needed is because there might be bugs or other problems with the product that are hindering the usability of the product. Finding such problems early in the development cycle is important as the cost and time to fix them increase over time. Another reason to evaluate is to deal with problems that impact the users experience in a negative way. While designing you as a designer might think that your system is intuitive and easy to understand, but a new user might experience it as nonintuitive and difficult to use. I think many of us have experienced this with webpages where it feels like it's impossible to find what you are looking for, only to find out that you have been searching in the wrong place. 

On a lecture, Jan Gulliksen mentioned two types of evaluation: Empirical and Analytical evaluation. These could both be used during a formative evaluation during the design process to test prototypes. I feel that analytical evaluation is a lower budget evaluation if you can't afford setting up a test lab, if the test can't be performed with test subjects (for example: too dangerous) or if you are standing between solutions that are similar, costly to develop, and can be predicted. The empirical evaluation, I feel, will yield more interesting, qualitative data. It might be costly to use, as you need test subjects, a test environment and a way to record the data, but the information you get will be from the users, the people that will be the ones to use the product. Personally I feel that involving the users is the preferred method.  

An example of an analytical method is the KLM method where you break down a task into individual keystrokes. An important thing to have in mind is that there can be different ways to perform the task depending on if the user is experienced and knows shortcuts or if the user is inexperienced and takes the long route. When the "route" is known you can calculate the time it takes to press the keys and even move the mouse using a formula. An empirical method is the Thinking-out-loud, where the actions on the user and the voice of the user will be recorded. While testing the product the user will think out loud, hence the name, talking about how and why it does certain things. This test is very dependant on the user, as doing two things at the same time can be difficult, especially if you get stuck. Getting some biased results will also be the case, as you'll want an observer in the same room in case the user gets stuck. 

My question is: What evaluation method should we use on our design this early in the process, and why?

Individual notes - seminar 2

Today, tech industry is obsessed with metrics - every decision must be evaluated, every change is A/B tested, and hard numbers are king. I've often wondered how useful this data-driven approach is in practice, since they often only collect quantitative data, and does not focus as much on the reasons behind the numbers

Therefore, I found it interesting that the book takes up other approaches than using technology to evaluate designs on your target group quantitatively. Measuring usability using e.g. interviews and think-alouds can give additional context to the feedback from the evaluation that helps understanding

On the other hand, collecting quantitative data from the use of the product when it is actually has the upside that it more accurately reflects how the product is used in the intended setting. Collecting qualitative data could interfere with the user in its testing of the design, which may bias the results.

I found that heuristic evaluation was very interesting. My view of usability was that it can often be counter-intuitive, since elements of a design may interact with each other in ways that are not obvious. However, using heuristics such as Fitts' law seems like it could provide useful information despite this. So while it is very hard to evaluate usability without getting feedback from users, heuristics could have great use with when testing certain properties.

My question for the seminar is: in what ways can we collect qualitative without interfering with the user, to avoid introducing bias.

Individual notes, Seminar 2 - Niklas Lindqvist

Chapter 13 – Introducing Evaluation

This chapter presents an introduction on how to evaluate a design. It focuses on both the usability of the design and the user’s experience/satisfaction.

The problem with a design often occur when the producers find the product useful but don’t understand that other users don’t understand how to use it. This I find true in numerous of cases, for example webpages where it’s almost impossible to find what you are looking for. Evaluation of a product enables them to check that their design fit others as well, and should probably be used more often.

There are 3 main types of evaluation:

• Controlled settings involving users (labs, everything is measured)
• Natural setting involving users (field work)
• Any setting not involving users (models and analytics)

Doing a combination of the evaluations is something I would like to aim for in our project following something similarly to:

Chapter 14 - Evaluation studies: From controlled to natural settings

Chapter 14 describes how to evaluate in different types of environments. For example controlled lbs and natural settings. Usability testing is the focus of the chapter.

An interesting concept is the in the wild studies where the researchers are far away and monitor the user with different methods let the users act more “natural”. This is a study that probably have become much easier to implement with new technology and might be improved even further in the future.

Chapter 15 - Evaluation: Inspections, Analytics, and Models

Inspection evaluation methods are in focus of this chapter but I find the Predictive model to be the most interesting. In Fitts' Law where predicting the time it takes to complete a small task with a pointing device such as the mouse on a computer can be used to estimate the total time of a complex task in the same program. This could be develop further using time it takes to press certain keys on a keyboard or other bottoms on a special dashboard when the final design in to expensive to produce iterate to its final stat.

Question for seminar:

What sort of evaluation will be able for us to do on our design? 

Third Design Concept

Third Design Concept

In the third design concept we focus on a new pain point: Crowding. The design is based around two simple ideas, the first being a new typ of method for entering the train, and the second one being a clear information board regarding the crowding information on incoming trains. 

The new doors are integrated in a glass wall along the platform making sure that you can not fall down on the railway. This would hopefully create new space for the passengers waiting for the train since you now can stand closer to the edge of the platform without being afraid of falling down on the rail. The platforms is around 150-200 meter and today around half a meter to the edge is not in use. This means that there is 75-100 m^2 of space wasted on the platform today but with the new doors this might be used more efficiently.

Guiding arrows on the ground to help people locate their wanted exit and marked areas on the ground close to the entrance to make sure that passengers entering the train are not in the way when people are exiting the train.

The goal of the new board is to give travelers on the station information on the upcoming trains crowding level to help the travelers position them self depending on their preference. If a passenger wants a free seat they should know where to be standing to increase their chances of getting it.

Self-reflection before seminar 2

It is interesting how much different methods of prototyping can actually help with the process. When reading about things such as using interative collaboration, parallell design, and brainstorming, it often feels like it's going to be what I call "productivity porn": getting lost in finding ways to be more productive instead of actually doing the work. Nevertheless, at the exercise when we used these different methods, we did get a lot done, and now have several designs going. I would however say that the "word association" method was completely useless to us, but maybe it's good it you have no ideas what to do whatsoever.


I was also surprised at the bits of math here and there, Interaction Design being such a "human" field. Fitt's law, for example, gives you a mathematical formula for how long pressing a button is going to take, while the "double-diamond" speaks more broadly about how a design process should look (and indeed, when prototyping, we experienced this "spreading" and then "collecting" very well).

Having "spread out", we need tools to evaluate our different ideas and know which ones are worth pursuing. This is a complex task, but at the same time important, because of the increasing cost of correcting a problem. For software products and smaller user interactions, the method of user evaluation and think-aloud was the one that stuck most with me, because it is very hard to evaluate an interface if you're an expert and know exactly how the system works. It's hard for you to know how a user will see it. Having recordings of user sessions can also then be used to convince developers of what they need to fix, but one has to make sure to prioritize his problems with the application, so that that developers don't only fix what's easiest for them (which might not always be the things that matter the most).

My question for the seminar is whether we should user-test with the users in different mental states (e.g. drunk, or tired) even if a majority of users aren't is those states.

Individual notes, seminar 2

We are about to start our first iterative cycle of design-evaluate redesign, at least I assume it will be the first of several. It is important to evaluate so we can be sure that our software is usable and is what users want, and it has shown to be less expensive to continuously evaluate the design than to fix problems that are discovered after the system has been launched (of course, expenses will not be a problem in our case). In the book, they mention how software and web designers are prone to assume everyone else can use their product just because they can. I believe this could be especially true for our group since we all can identify with our relatively broad target group and therefor on some level designs a product for ourselves.This can be prevented by methods such as user evaluation and think-alouds. Studies have shown that people nowadays expect more from a product than just for it to meet the exact needs of the customer, for instance simplicity and elegance that makes products a joy to own and to use. The latter is especially hard without user involvement in the design process and evaluation. Usability testing involves measuring the performance of typical users on typical tasks. Satisfaction can be evaluated by continuing with the interviews and questionnaires. There has been an increasing trend towards observing users while they interact with the product in the intended settings, which could vary a lot from if they were observed in a laboratory. These kinds of tests could be hard for us to perform depending of which design we choose and what kind of prototype we can make (we can’t for instance make a prototype of a glass wall between the platform and tracks that people can interact with during their usual commute). In our case it might be easier to use heuristic evaluation which aren’t dependent on the users’ involvement to the same extent.

Is there a way to test our 3rd design on the users?

Seminar 2 - Individual notes

Evaluation

I was intrigued by how many different factors and methods that comes in play when evaluating. Surprised that it’s such a big area in interaction design. I thought on forehand that the most common way of evaluating a product today must be analytics using the data of natural setting involving users because most tech made today is connected and thus can feed user data to the designer. It became clear that’s not the case. I believe that think-a-louds, which is a controlled setting involving users, provides a lot of useful information to a designer. Unfortunately, the results can be somewhat non representative because the part of the brain that’s responsible for decision making is also used for speech, high level of cognitive awareness, meaning that when the most interesting decisions are made the user is most prone to be quiet. 

As seen in the chapter of data gathering usability testing also might provide both quantitative and qualitative data. The first might be given by keystrokes, mouse movement or the time to complete a task. Qualitative might collected semi structured interviews for example. It’s important though that when interpreting and presenting the data one should be aware of that results might be influenced, a test conducted in a lab says little about how users will interact with the design “in the wild”.

When lacking real user’s heuristic evaluation is the way to go. It seems that the set of heuristics is great guidelines to see whether the design is meeting demands of good interaction design. Doing this before releasing might save lots of time. Fitts’ law describes, in a mathematical manner, the time it takes to select objects on a screen and thus need no users at all and can still be motivating major design decisions which I does seem really useful.

Question: What evaluation method would be the best for our design nr.2?

Design step 1 - Collaborative Iteration

Choosen painpoint: Worried when train stops and no reason is given

We choose one painpoint and wrote it on a piece of paper. One member of the group started with an idea that could solve the issue at hand, summarized it on a post it note and stuck it on the paper with the pain point. The next person expanded the idea and in the same manner summarized the essence of it and stuck it on the paper. When everyone had done so we discussed how this chain of solutions and designs might come togeather and visualize what such design would look like. We also tried to keep in mind the earlier interviews and what reqierments the conceptual design would meet.

The white paper is the painpoint in focus in this blogpost

The first idea was to give commuters better information on the cause of delay and which alternative itineraries throgh the existing SL-application. After a few steps it took a different turn to screens in the subway. We made this sketch and low-fidelity prototype on paper.

Quick sketch

Jan Gulliksen mentioned on a lecture of his that he’d been involved on a project involving railways in Sweden and informed us that in most cases the driver of the train doesn’t have the information about the delay but just a signal to stop. By reducing the chain of information and let the available information at HQ, from technology already in place, be presented imideatly to the user in a familiar format to them. The red dot indicates delays and the numer is estimated time before solved. The screen also lets user easily orientate themselves where they’re on the line and which stations are coming up. This way of visualizing the railway, including delays, amplifies human cognition because it's enabling users to see patterns and easily detect anomalities.

Field Study - Interview transcript

How often do you commute?
5 days every week.

What route do you take?
Train to T-Centralen then subway to Slussen

Where do you stand when you're waiting for the subway?
If possible I try to stand close to the front of the train as it's less to walk.

Even if it's crowded?
No, if it's crowded I'll walk towards the middle

Do you use the SL-app?
Yes

What function do you like with it and what do you dislike?
I use the routeplanner a lot and like it. The traffic information doesn't really show all the information and sometimes it doesn't even work properly

Why are you traveling with the subway?
It's pretty smooth, it's fast and you don't have to wait for long if you miss a train. Also, I get a free SL-card from the school

What do you do to pass time while commuting?
I listen to music, play games on the phone or practice the drum lesson

What's the worst that could happen on the subway?
If there's a power failure and the train gets stuck between to stations

Personas

Primary persona:

Karl Karlsson

Age: 22

Occupation: Student

Hobbies: Likes to party, engage in political manifestations and play video games.

Commutes: Daily, different times though

Tech: Uses his iPhone way too much otherwise not so handy with technology.

Karl grew up in Luleå but moved to Stockholm when he were to start upper secondary school because he wanted a niche to economics which wasn’t available in Luleå. After high school he started to study industrial engineering and management at KTH. He is also ambitious and does some courses at Södertörn in positive psychology.

Karl enjoys socializing and being a student at KTH there’s a lot of parties to do so. As he’s not raised in Stockholm he still manages to get lost sometimes, especially when trying to find his way to a party somewhere he’s never been before. Karl is also the type of personality that always procrastinates everything as much as possible, eating, sleeping, studying and going to school. This makes him depend a lot on frequent departures using public transport.

At this moment...

Karl failed some courses last semester and have been declined pay outs from CSN for a while. The tight budget forces Karl to skip tickets and cheat on paying for the subway a few times. He tries to ride his bike as often as he can though. He have a sublet at Södermalm for a year now so distance to school is doable but not every day. His girlfriend is living at Kungsholmen and the bike ride there is a leg killer.

Scenario 1

At 7:40 Karl walks briskly towards Skanstull subway station.  He is not a morning person and usually skips the morning classes but today he has an exam and he is already late. He zigzags across the crowded station and gets on the train just before the doors close. He gets off at Slussen to change to the red line, the sign on the platform tells him there’s a train coming in 3 minutes. Annoyed by all the people, he backs up against the wall and picks up his cellphone to pass the time. After scrolling through Facebook a bit he glances at the time, 7 minutes has passed and the train hasn’t arrived! There’s no information on the delay either on the platform or the app, Karl hasn’t got the time for this. He checks the app for an alternative route and it tells him to take bus 2 towards Norrtull. He pushes through the crowd and up the escalators and rushes out on Götgatan. Where to now? Where are all the busses? He checks the app but the map is poor on details and he doesn’t know where he is himself. He grudgingly starts asking people on the street and after being rejected a few times by people in a hurry, a little old lady points him in the right direction. When he finally gets to the bus stop he has already missed his intended buss and he is going to be very, very late.

Scenario 2

Karl’s girlfriend has a younger sister whose birthday happened to be today. To celebrate she’s having a laserdome birthday party, which he and his girlfriend are invited to. The evening single variable calculus lecture ran over the allotted time, so not only is Karl tired this Friday evening after a long day in school, but he is also late to the party.

Being a bit stressed, he rushes to the subway and manages to get on the red line heading south that was just leaving. He knows the laserdome arena was somewhere on the green line, at a location he’s never been before. The thought of 20 elementary school students complaining to his girlfriend about not starting the laserdome games due to Karl being late stress him out even more.

Nearing T-centralen, he double-checks which of T-centralen and Slussen the southbound green line leaves from - something he never remember. He will only have a 1 minute margin to the next train, so it’s important he picks the right station to switch lines.

When he arrives at Globen, he’s unsure what exit he should choose. They’re far apart, and he don’t want to delay the party more than necessary. He doesn’t recognize the street names on the signage, so he picks an exit at random. It turns out he picked the wrong exit, delaying him another 5 minutes trying to find his way around Skanstull. When he arrives, Karl’s girlfriend is very annoyed that Karl kept her little sister waiting.

Gun-Britt Olsson

Age: 55

Occupation: Member of parliament

Hobbies: Playing boule, painting.

Commutes: Travels to work every day on regular times.

Tech: Apart from doing work related stuff on her computer she doesn't use it too much. Her son bought her a smartphone and she is slowly getting used to it, she does enjoy being able to read email on her way to work with it.

When she was younger, Gun-Britt used to live in Lund, however, her husband had to move to Enköping because of his work. She had always been politically engaged and decided to finally take the chance and work in the parliament. The main reason she is in Stockholm is because of her work, as such she's not familiar with places outside her usual way to work. While traveling she likes to use the time efficiently by working on her laptop on the longer train journey and then reading email on her phone while on the subway. On the weekends she enjoys playing boule with the senior team she joined, usually it's in the local area, but sometimes they play against a team in Stockholm and have to travel there. The first times she got lost for a while, before calling a friend and eventually finding the right place, but nowadays she knows where to go.

Today:

Since her son showed her the SL-app she feels more confident traveling new routes. When finding the time to do so, Gun-Britt and her husband enjoys exploring museums, theaters and other cultural events. However, as good as the SL-app is it is very vague on where to go once you're off the station, so getting lost when walking out of the station is a common occurrence. Now that winter has passed she’s hoping that the trains won’t be late as often, since she doesn’t like being late to work.

Scenario 1

Gun-Britt is an early riser, so getting on her bike to catch the train at 7:08 in the morning does not phase her one bit — especially now that spring has arrived. Using her priority pass she's able to find a seat with a table, and starts working on her laptop before the train has started rolling off the station.

Just under 45 minutes later she instinctively knows that the train is about to arrive at Stockholm Central so she packs away her stuff, puts on her jacket and heads for the door. Bustling onto the platform and into the rush hour crowd, she barely notices the throng of people since their presence is nothing out of the ordinary. As she approaches the metro junction she habitually checks the SL-app — no delays, next train in 2 minutes, next after that in 3 — her feet doesn't miss a step, trudging on toward the red/green platform. She's heading to Gamla Stan.

To get to Riksdagen she could take the blue line to Strömkajen and walk from there, or walk directly from the Central station, but Gun-Britt prefers walking through the beautiful Gamla Stan. The Gamla Stan route is also faster with both of the busy red and green lines passing through.

She positions herself on the platform ready to enter at the back of the train, which is where her exit is. The overhead display says that next metro train arrives in 1 minute. She begins checking her emails on her phone — this takes much more of her attention than the arrival of the train, or than herself stepping on the train and finding a handrail. Her only momentary distraction is another passenger bumping into her phone-hand, but before she knows it she has reached her station and is walking out of it.

Scenario 2

One day, things are different than usual. Gunbritt exits the pendeltåg at T-Centralen and knows if she walks at a normal pace she will be able to take a green-line train and arrive at work right on time. Gun-britt isn’t a morning person, so she doesn’t schedule more time than necessary for the morning commute. Today is an important day, as they will be voting on an important matter as the parliament.

Gun-britt is stopped on her usual way through the central station by some scaffoldings and a sign saying “Ombyggnation: genomgång tillfälligt avstängd. Var vänlig följ markeringarna. For information in English please visit sl.se”. This is terrible news for Gun-britt! She picks up the pace to follow the route the orange arrows are showing her, but anticipates that it will be much longer than her everyday walk. She risks being late for work. “I wish I had known about this earlier, I would have taken an earlier train to have time to figure this out. They really should inform people”, she thinks in her head. Her frustration increases when she once lose track of the orange arrows, and has to ask someone for help to get back on track. She does finally find her way and arrive to work only five minutes late. Nobody notices, but she was very stressed during the trip.

Pain points:

Why do people use the metro? What pain does it alleviate?

• avoid above-ground traffic
• is inexpensive
• possible to read/work on the metro
• runs at most travelled hours of the day
• runs frequently enough

What are the pain points during normal operation of the metro?

• able to avoid delays
• not having to worry about there being delays
• know about resegarantier when there are delays
• know wait time/stress for transfers
• trust that the route is correct
• know about alternative routes
• know where to stand/enter/exit trains depending on station exit
• know if the trains are short

What are the pain points in the environment of the metro?

• bothered by crowding in stations
• bothered by crowding on the train
• bothered by not having a place to sit
• bothered by dirt/grime
• safety at stations and in trains
• scared about getting stuck in the train
• worried when trains stop and no reason given
• worried about catastrophic events in the metro
• scared of rowdy/drunk people on late nights
• scared of the ideo of such people
• women worrying about people late at night in general
• fear of turnstiles?
• bothered by old/non-modern trains