Tvori, Animation in VR| Usability Testing
About the app
Tvori is a VR animation tool that allows to create, animate and record scenes solely in VR.
The tool is currently in early access and has limited functionality. The team are working on developing new features and making Tvori stable and reliable. It is available to get in Oculus Store, Steam and Viveport.
In this project, I was tasked to conduct usability testing to understand how clear the on-boarding process is, and to assess if there are any flaws in usability. I also conducted review analysis to assess if there are any patterns in given feedback by reviewers who have been using the tool.
Tasks and Approach
Time: March 2019
Role: User interviews, Usability testing
Tools: Tvori, Google Spreadsheets
Hardware: Oculus Rif
To run this usability test, I recruited 5 people, who have had no experience in using Tvori, have had some experience in using VR applications, and have interest in VR creation tools.
With this exercise we wanted to test the following hypotheses:
Players would find on-boarding tutorial helpful in understanding how to use tools covered in the tutorial;
Players would not be able to easily create, animate and record the scene before going through the on-boarding tutorial;
Players would be able to use Tvori in VR for over 30 minutes straight without any physical discomfort.
After creating a script with tasks for players to complete, I ran moderated face-to-face usability testing sessions. I used Oculus Rift; to record each session, I used the Windows 10 Screen Recorder and an Android phone video camera. Having had 5 users to test on, I spent 20–60 minutes for each session. For the hygiene purposes, I used the one time use eye masks for HMD.
The script consisted of the warm up questions to learn about players’ experience with VR, animation and modelling tools; series of tasks; and general feedback questions after the test. Before and after the test I also ran the Simulator Sickness Questionnaire to measure the level of comfort in players, and whether the VR experience like Tvori can cause any physical discomfort.
To document observations, I created a Matrix, with the list of tasks as rows and players as columns, so that the performance of each task can be easily viewed in a row, and the completion rate ranked.
After filling in the observations and players’ quotes into the matrix, I measured the tasks completion rate and looked for patterns in user behaviours. I then prioritised findings from major problems to minor problems, general observations, positive findings, and ideas.
Examples of findings
To showcase usability findings, I created short clips with the issues observed. Here are some of the examples:
Out of 5 participants, 3 were tested and recorded in the in-house lab, and 2 were just observed. Having had 5 participants altogether allowed to reach saturation with findings, and collect valuable observations and feedback.
The results of the Simulation Sickness Questionnaire showed that for the new user, the experience didn’t cause any major discomfort. However, some participants did have mild tiredness in the arms, and discomfort in their eyes after the task.
The observations highlighted over 60 findings, all of which were coded and grouped under 10 focus areas, prioritised, and given UX recommendations. With this exercise, we were able to validate the two of the three hypotheses:
Players found the on-boarding tutorial helpful in understanding how to use tools covered in the tutorial;
Players were not able to easily create, animate and record the scene before going through the on-boarding tutorial.
Hypothesis 3 “Players were able to use Tvori in VR for over 30 minutes straight without any physical discomfort” was not validated, as there still were signs of slight physical discomfort in the participants who were in a headset for over 30 minutes.
After sessions, I also ran interviews and asked participants for their general feedback.
A few players went further with the feedback and suggested a number of ideas that they expected to see while using Tvori.
With this usability testing, Tvori commented that they validated some of their old assumptions, as well as learned about new findings. All usability findings are planned to be addressed, with some issues already solved, and some put into their backlog.
They liked the reporting format and documentation of findings, as well as commented on the usefulness of the study for their product. :)
I found this project extremely interesting and useful in trying out new usability testing techniques, applicable specifically for PC VR.