Seminar 2 - Summary

Recollection of discussed topics during seminar 2

• We have learned the importance of evaluating, but sometimes it is hard to user-test something beforehand (scale, patents, secrecy). We also discussed ways to circumvent this.
• For testing our own design, we could use testing in the wild: adding markers and prototypes to the SL stations, and have people try to accomplish tasks (getting from A to B while using our prototype app). This would be testing in a natural setting involving users.
• We could use people from Copenhagen (which has a similar system in place) to heursitically evaluate our prototype and tell us what they feel are the good parts and bad parts of our system when it is adjusted to Stockholm.
• We could also take Stockholmers to Copenhagen and let them see what they like and don’t like about that system, and see how they compare it to stockholm.
• This would be be a kind of triangulation: we see the problem both from the perspective of users who have used it for a long time, and users who are using it for a first time. Their combined heuristics would give us a better view of the actual good and bad sides of our idea.
• Also, Opportunistic Evaluation is a thing that we can actually, practically do (not assuming infinite resources etc). We could even ask the group we’re presenting to what they think, since they are almost surely users of the public transportation. We can also interview people in the subway and present our design to the, ask what they think.
• We discuss how we should evaluate the different parts of our designs, since it is quite broad. It might be good to evaluate each part on its own, ut we also need to evaluate the whole together since we need to know how well the prats cooperate.

Design step 2 - Parallel design

For the second part of the design step we split into two groups, we were given the same pain point with the goal to create a design centered around it. The chosen pain point was: Being unsure on what exit to take. We gave it about 40 minutes before sitting down to compare the designs. To our surprise, both managed to create very similar designs, both featuring an application with some modification to the station.

Low-fi prototypes of both designs

Both groups had used the existing SL-app as a base for the design and then expanded on it. With the pain point being what exit to take, we had the idea of colouring or assigning a letter to the exit so that when looking at your planned route, you could easily see what exit to take. Using colours to distinguish between the exits allowed us to use the already existing "T" that people are used to seeing. As it is something familiar to the user and as such, takes little getting used to.

When it came to making changes to the actual station we were basically on the same page yet again, changing the signs to reflect what exit they point at (pink, grey, blue, A, B). One group also had the addition of arrows on the ground that you could follow to the exit. The visibility and if it's easy to find your arrow when there's a lot of exits or on a crowded platform is something to consider.

Detailed exposition of the design progress in group 1

First we considered the situation when a traveller arrives at a platform, and tries to figure out where to go.

We noted that there are signs at the ends and in reasonable intervals along the platforms. These signs describe the exits by street name and occasionally also by important locations in the direction of the exit. Unfortunately these are often insufficient e.g. if the traveller is heading to a parallel street or further away.

The most straightforward improvement would be to add a longer list of streets and locations for each exit. It occurred to us that sometimes travellers don't know any of the nearby street names but only which general direction they want to exit. It therefore seemed appropriate to have maps rather than signs naming streets.

Each of these maps would show all the exits and the nearby surrounding area. Nearby streets and important locations could also be named in the map.

The exits marked in the maps should be colour-coded, perhaps using textures or multiple colours each to make them unique and distinctive. These distinctive colours could then be used to mark or point to exits throughout the station.

Distinctive arrows could be put at each intersection, or lines could be drawn all along the floor. It was noted that floor lines could be obscured during rush hour. It was also noted that coloured arrows could be confusing if seen by themselves without a map, but this did not seem like a major issue since maps would be at all entrances and at intervals on the platforms. Furthermore, in places where there's no place for a map, where today there's only a sign that names the street the exit opens up to, in those places the same street could be named in addition to the colour coding of the sign.

Next we considered the situation before a traveller arrives at a platform — ideally they want to exit the train onto the platform nearby to the exit they want off the platform, and for that they need to know where they should position themselves in the train well before arriving at the platform. The best time to make use of this information is before entering the train, while waiting for the train to arrive.

It seemed unreasonable to have all this information displayed statically at each station, i.e. all stations having the map described above for their exits, as well as those maps for every other station on the line (and connecting lines). It did however seem possible to have this information in a dynamic application.

To investigate this we looked at an existing SL schedule app. The app, much like the SL webpage, lets you input start and target destinations, along with planned starting time for the journey, and then it shows you a chronology of evens: Walking to the nearest relevant station, what line to take, when the train will arrive, how long the journey will be, which station to exit on.

In the existing layout of the app it seemed fitting to add a suggested wagon to board (i.e. which end of the train, or sometimes middle). Another good addition would be on the exit station to say which colour-coded exit to head for.

The app lets the user see more detailed information about some items in the chronology by tapping an expansion arrow on such items. Continuing this interface the indication for which wagon to take could be an iconified train with three wagons (as seen from the platform), with the recommended wagon to take marked. For new users who don't understand this icon an expansion arrow could be tapped to show an explanation.

If there was a way to measure real-time seat-availability and/or general crowding in the wagons, this information could be used to improve the wagon recommendations.

We realised that navigating by colour-codes rather than Swedish street names would be especially useful for tourists. Unfortunately tourists would be especially unlikely to have an SL route app. This could be solved by having tourist information at station entrances together with WiFi enabled downloading of the app, or by placing touch-screen interfaces to the app on stations.

Individual notes - Seminar 2

When designing a product it is very easy to get stuck in coming up with ideas, thinking about new things to add to the product or just improving the design as you see fit. It is easy to forget that designing should be an iterative process where we should evaluate the product every so often. An obvious reason to why evaluation is needed is because there might be bugs or other problems with the product that are hindering the usability of the product. Finding such problems early in the development cycle is important as the cost and time to fix them increase over time. Another reason to evaluate is to deal with problems that impact the users experience in a negative way. While designing you as a designer might think that your system is intuitive and easy to understand, but a new user might experience it as nonintuitive and difficult to use. I think many of us have experienced this with webpages where it feels like it's impossible to find what you are looking for, only to find out that you have been searching in the wrong place. 

On a lecture, Jan Gulliksen mentioned two types of evaluation: Empirical and Analytical evaluation. These could both be used during a formative evaluation during the design process to test prototypes. I feel that analytical evaluation is a lower budget evaluation if you can't afford setting up a test lab, if the test can't be performed with test subjects (for example: too dangerous) or if you are standing between solutions that are similar, costly to develop, and can be predicted. The empirical evaluation, I feel, will yield more interesting, qualitative data. It might be costly to use, as you need test subjects, a test environment and a way to record the data, but the information you get will be from the users, the people that will be the ones to use the product. Personally I feel that involving the users is the preferred method.  

An example of an analytical method is the KLM method where you break down a task into individual keystrokes. An important thing to have in mind is that there can be different ways to perform the task depending on if the user is experienced and knows shortcuts or if the user is inexperienced and takes the long route. When the "route" is known you can calculate the time it takes to press the keys and even move the mouse using a formula. An empirical method is the Thinking-out-loud, where the actions on the user and the voice of the user will be recorded. While testing the product the user will think out loud, hence the name, talking about how and why it does certain things. This test is very dependant on the user, as doing two things at the same time can be difficult, especially if you get stuck. Getting some biased results will also be the case, as you'll want an observer in the same room in case the user gets stuck. 

My question is: What evaluation method should we use on our design this early in the process, and why?

Individual notes - seminar 2

Today, tech industry is obsessed with metrics - every decision must be evaluated, every change is A/B tested, and hard numbers are king. I've often wondered how useful this data-driven approach is in practice, since they often only collect quantitative data, and does not focus as much on the reasons behind the numbers

Therefore, I found it interesting that the book takes up other approaches than using technology to evaluate designs on your target group quantitatively. Measuring usability using e.g. interviews and think-alouds can give additional context to the feedback from the evaluation that helps understanding

On the other hand, collecting quantitative data from the use of the product when it is actually has the upside that it more accurately reflects how the product is used in the intended setting. Collecting qualitative data could interfere with the user in its testing of the design, which may bias the results.

I found that heuristic evaluation was very interesting. My view of usability was that it can often be counter-intuitive, since elements of a design may interact with each other in ways that are not obvious. However, using heuristics such as Fitts' law seems like it could provide useful information despite this. So while it is very hard to evaluate usability without getting feedback from users, heuristics could have great use with when testing certain properties.

My question for the seminar is: in what ways can we collect qualitative without interfering with the user, to avoid introducing bias.

Individual notes, Seminar 2 - Niklas Lindqvist

Chapter 13 – Introducing Evaluation

This chapter presents an introduction on how to evaluate a design. It focuses on both the usability of the design and the user’s experience/satisfaction.

The problem with a design often occur when the producers find the product useful but don’t understand that other users don’t understand how to use it. This I find true in numerous of cases, for example webpages where it’s almost impossible to find what you are looking for. Evaluation of a product enables them to check that their design fit others as well, and should probably be used more often.

There are 3 main types of evaluation:

• Controlled settings involving users (labs, everything is measured)
• Natural setting involving users (field work)
• Any setting not involving users (models and analytics)

Doing a combination of the evaluations is something I would like to aim for in our project following something similarly to:

Chapter 14 - Evaluation studies: From controlled to natural settings

Chapter 14 describes how to evaluate in different types of environments. For example controlled lbs and natural settings. Usability testing is the focus of the chapter.

An interesting concept is the in the wild studies where the researchers are far away and monitor the user with different methods let the users act more “natural”. This is a study that probably have become much easier to implement with new technology and might be improved even further in the future.

Chapter 15 - Evaluation: Inspections, Analytics, and Models

Inspection evaluation methods are in focus of this chapter but I find the Predictive model to be the most interesting. In Fitts' Law where predicting the time it takes to complete a small task with a pointing device such as the mouse on a computer can be used to estimate the total time of a complex task in the same program. This could be develop further using time it takes to press certain keys on a keyboard or other bottoms on a special dashboard when the final design in to expensive to produce iterate to its final stat.

Question for seminar:

What sort of evaluation will be able for us to do on our design? 

Third Design Concept

Third Design Concept

In the third design concept we focus on a new pain point: Crowding. The design is based around two simple ideas, the first being a new typ of method for entering the train, and the second one being a clear information board regarding the crowding information on incoming trains. 

The new doors are integrated in a glass wall along the platform making sure that you can not fall down on the railway. This would hopefully create new space for the passengers waiting for the train since you now can stand closer to the edge of the platform without being afraid of falling down on the rail. The platforms is around 150-200 meter and today around half a meter to the edge is not in use. This means that there is 75-100 m^2 of space wasted on the platform today but with the new doors this might be used more efficiently.

Guiding arrows on the ground to help people locate their wanted exit and marked areas on the ground close to the entrance to make sure that passengers entering the train are not in the way when people are exiting the train.

The goal of the new board is to give travelers on the station information on the upcoming trains crowding level to help the travelers position them self depending on their preference. If a passenger wants a free seat they should know where to be standing to increase their chances of getting it.

Self-reflection before seminar 2

It is interesting how much different methods of prototyping can actually help with the process. When reading about things such as using interative collaboration, parallell design, and brainstorming, it often feels like it's going to be what I call "productivity porn": getting lost in finding ways to be more productive instead of actually doing the work. Nevertheless, at the exercise when we used these different methods, we did get a lot done, and now have several designs going. I would however say that the "word association" method was completely useless to us, but maybe it's good it you have no ideas what to do whatsoever.


I was also surprised at the bits of math here and there, Interaction Design being such a "human" field. Fitt's law, for example, gives you a mathematical formula for how long pressing a button is going to take, while the "double-diamond" speaks more broadly about how a design process should look (and indeed, when prototyping, we experienced this "spreading" and then "collecting" very well).

Having "spread out", we need tools to evaluate our different ideas and know which ones are worth pursuing. This is a complex task, but at the same time important, because of the increasing cost of correcting a problem. For software products and smaller user interactions, the method of user evaluation and think-aloud was the one that stuck most with me, because it is very hard to evaluate an interface if you're an expert and know exactly how the system works. It's hard for you to know how a user will see it. Having recordings of user sessions can also then be used to convince developers of what they need to fix, but one has to make sure to prioritize his problems with the application, so that that developers don't only fix what's easiest for them (which might not always be the things that matter the most).

My question for the seminar is whether we should user-test with the users in different mental states (e.g. drunk, or tired) even if a majority of users aren't is those states.