Testing ReGame

Last week we (lexlibratorMillysophiedoesmediaGong Liang) had the chance to present the alpha version of our prototype ReGame. Already on the spot we received a lot of feedback on our project. Most important were comments on the choices we made according to the formatting of app elements and their functionality. In the end of our process, we didn’t focus on design, but instead on the fact that the most important parts should work for real. Through this decision, the final outcome became more limited than the paper prototype suggested.

regame_testing

As a final step of this group work we conducted a small testing round, in order to gain feedback from some potential users. This is actually the second time we test our system; much earlier in the process, we tested our idea with users already, to see if the idea is genuinely of interest. Sander and Dandavate [1] describe that in order to understand the user we have to listen to what they say, look what they do and see how they express themselves to know what they think and dream of. We therefore created a questionnaire, which is presented in the methods part. The most interesting piece is of course the presentation and comparison of the results we got. Here we will see if we can detect any similarities or patterns. Additionally we will discuss which parts have to be reconsidered or altered to finalize the application.

Method

In order to know how the app was received, and how it was perceived from a usable perspective, we conducted a small usability test. We decided to do structured interviews and analyse them in a qualitative method. In that way we got very detailed feedback on our specific design yet maintained the possibility of comparing our testers to each other [2]. We created short scenarios that each of us would follow in their respective test.  Since our target was very broad and not limited by age we decided to choose our subjects by accessibility. Four persons participated in the testing round. We wanted to know how they felt about the app and also if the main function, that’s the possibility to share images on moments, worked as we expected.

To start, we tried to gather user’s first impressions. We gave them short instructions to follow.

1. Login and have a look, don’t do anything more yet; describe what you see and think this is about (People that don’t have any previous knowledge of the app.)

– What is your first impression of this application?

Second step, was to lead the user into the app and to the functionality where the user sees the moment for the first time.

2. Go to Group and choose the first one, then try to choose the first moment.

– What do you think you can do here?

Then as to test the core functionality we let the user try out the share function and upload an image to a moment.

3. Go to Share and share an image that you already have.

– Did the user understand that the picture was added to the moment/find it?

The user was encouraged to think aloud and notes were taken during the session. We also posed a series of questions that would give us even further insight into our users experience.

4. Questionnaire

– Who do you see as a target group?
– What do you like about this application?
– For what would you use it?
– In which situation would you use this application?
– Which functions would you additionally like to have?
– Would you use this application when it’s free?
– If the application costs money, would you consider to pay for it? How much?

Results

1. First impression

The participants mostly understood the application as a game forum. It’s easy for them to navigate and generally they like the colour and layout. However, our alpha version, didn’t have so many functions for the participants to try out. As a result, they didn’t find anything special to keep them using the application.

A participant reacted on the logo itself and stated that: “It’s obviously about games, and it refers to doing something again, because of the “re” in the name”. The sneak peek  function got some attention as well, and it confirm our try-out option (at least in theory).

2. Interaction results

During the scenario part all four participants struggled understanding the functionalities. They liked the navigation in the footer and understood where to press and what would to expect behind it. Especially as we used commonly known titles as “profile” and “share” the participants recognized those from other social media. But problems occurred when opening the “group” and “moments” pages. A participant stated that the moments page has “too many uploads, this might work for small groups, but how does it stay up? A service that requires constant bumping of your own content, isn’t something you want to share”. So the sorting option after the newest moment will not work as good in a real application. Instead it needs some rating or counting option, so the most discussed moments would show up first. Another participant didn’t feel that “the picture itself is a moment with a separate space”. Some suggested that it would be better, if the picture you clicked on in the moments, should show up in the chat as well. Users complained about some general problems to locate where they were, or why things happened. A big problem seemed to be the missing of the picture in the chat. This confused users, how the chat was related to the chosen discussion. The tested participants understood how to add a picture and liked the sharing possibility, but argued that it could have been made clearer.

3. Questionnaire results

liked vs. problems

Most of the participants liked the clean style and layout. They thought it was easily readable, easy to go through and relatively responsive. Most elements were recognized as what they were for. The most problematic part was considered to be the group function. The limited version made it hard for the users to extinguish how the main idea behind it was meant to be: “Right now they [the groups] are like search filters, it’s not just based on pictures”.

target group

It is clear to all the participants that gamers are the targeted group for this application. They stated that it doesn’t has to be some special gamers, but instead gamers in general. It is nice that they noticed that the application should serve as a way to connect people based on a special interest. One participant said that the target group , might be similar to those who already use reddit, 9gag or similar services.

Usage

We wanted to know if the application would be used in the way  we were creating it for and in what situation. Here the participants had different opinions: Sharing pictures and comments was one answer, others said that it was as a forum, to chat and discuss. In general they said that they would use it, whenever they would discuss something. So in that case it wouldn’t be bound to a location. Others would like to use it in the home. Just by looking at those few answers it becomes clear, that the usage really depends on the person, as well as it is bound to the functions the application provides.

Costs

None of the test users considered to buy the application, and they didn’t think was providing any new purpose for them. A participant stated: “Not like it looks now. Maybe when it’s finalized. Right now it doesn’t fill any gap, which my forums don’t fill in. I am active in forums, so it takes a lot to get me to use something else”. Even if the application would be for free, the asked testers weren’t convinced of the service. But they would consider using it, if it would provide more functions.

Additional functions

The test participants gave some ideas for additional functions, in order to raise the value of the presented application. They were considering audio features (audio recording or audible discussions) or adding sub-groups.

Discussion

What became quite clear after our short study is the fact that, as it is now, our app does not have the appeal required to succeed in the market. It needs more visual cues about it’s own functionality and possibly a broader area of offered activities.  We definitely have to work on clarity of functions and perhaps redesign the way we called certain tabs. Since the name of moments seems quite confusing and our usage of groups is non standard we will have to make important decisions whether we aim to educate our users in our terms or conform and rename what we do. We learned that despite our initial feeling that our app might offer something new to the gamer community, it seems that to our audience it is comparable to already existing services such as for example 9gag. That said, it seems our design and easy interface properly transmitted our message of gamer focus and could be worth hanging onto. Before we can continue developing our app, we will have to go all the way back to the drawing board and focus more on the very needed distinction between our app and online forums. We need to discover a convincing reason that could attract our target audience and make the effort of moving to a new platform worth their time.

_____

References

[1] Sanders, E. B. N., Dandavate, U. (1999). Design for experiencing: New tools. In Proceedings of the First International Conference on Design and Emotion, Overbeeke, CJ, Hekkert, P.(Eds.), Delft University of Technology, Delft, The Netherlands. 87-91.

[2] Lindlof, T. R., B. C. Taylor. (2011) Qualitative Communication Research Methods. SAGE.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s