A few weeks ago, we shared our thoughts on evidence-based product development. Not only do we believe you should test your product as soon as you have something viable, but those tests should be backed by the rigor of the scientific method. We take this approach at Presence and wanted to share an example of how we’re using this decision-making process to consider changes internally. In this short experiment, we wanted to know if VR would improve our internal project meetings. Is VR ready for work?
Designing the Experiment
Our hypothesis was that using VR headsets during meetings would improve our staff’s level of engagement in meetings, their ability to connect and communicate with team members, and their energy levels after meetings. We chose a team that had already been meeting for several months to exclude the challenges that come with forming a new team. The team also met regarding a specific project to exclude the variables of discussing multiple projects and to avoid context shifting.
We asked staff to report back weekly on the experiment. We asked structured questions with quantitative replies (e.g., how successful was this feature on a scale of 1-5) so feedback could be measured and averaged. We also asked for open-ended feedback. We allowed two weeks for employees to adjust to the VR headset and software, and followed this with two weeks of immersion with all internal meetings taking place in VR (i.e., standups, grooming, retros, etc.).
Is The Tech Ready?
For this experiment, eight participants were outfitted with an Oculus Quest 2, 256GB model. We used Facebook Workrooms for the meetings. While our main focus of the experiment was how VR contributed to the team, we were curious about the ease of using VR for our meetings. We also wondered how best to include non-VR participants in a VR meeting.
It was good we planned two weeks for onboarding, because even for our team of tech-savvy engineers, adjusting to how the Oculus and Workrooms work took about three weeks, cutting into our scheduled immersion time. Users who took to the technology quickly still needed that extra time to explore the VR space before having the comfort to use it regularly for meetings.
There were persistent challenges with the software and some ergonomic issues with the headset itself. Facebook Workrooms was often unreliable, with desktop screen sharing only working about half the time and users often finding themselves unable to unmute themselves in a meeting. One user with vision challenges reported headaches and blurred vision when using the Oculus and others found reading text to be difficult. The Oculus itself was cumbersome at times; some users reported it was too heavy or that they had difficulty adjusting the straps. While it seems likely that many of the hardware related issues could be resolved with the use of ergonomic accessories (i.e., upgraded straps, integrated lenses, etc.) the biggest impediment to reliable VR meetings appears to be the stability of the software. The software for meetings in VR is still far from mature. Workrooms has found a great balance in their UX that lends itself well toward productivity, however, it is still unreliable and buggy.
Survey Says. . .
The team reported increased energy after meetings with the team. In terms of the team’s level of engagement, ability to connect and communicate, and team morale, results were mixed. Week 3 of the experiment saw an increase in engagement, connection/communication, and morale, but then took a big dip in Week 4, which we believe was due to a change in the overall project timeline and unrelated to the use of VR.
Comments from staff described their experiences as “incredible,” “amazing,” and a “GOOD EXPERIENCE.” Several reported feeling “excited.” Representative comments included
“We figured out how VR works and I believe it works incredibly for interactions with the team, [we’ve] been able to share information faster and even work on whiteboards. I really enjoyed the experience.”
“When it’s at its best, I actually feel like we’re in a work room. The dream is there, but the reality is buggy.”
“When all things align and work, I love it. I absolutely love it. My “ah ha” is I love being able to look to my right and see the person speaking in my ear. I really, really, really like that.”
In our exit survey, respondents answered the question, “Overall, how would you rate your experience using VR with your team” at 4.4 out of 5. 60% indicated they would like to continue using VR as a technology for meeting with their team. When asked what they enjoyed least about the experience, staff unanimously pointed to the technical issues. When asked what they enjoyed most, responses included:
“It adds an interesting dynamic to team meetings, one that’s difficult to achieve in a remote work environment.”
“Seeing team members in avatars and gestures is fun.”
“The feeling that someone else is right next to you is so good. The tools for interactions and teamwork are nice as well, even to work with people outside of VR.”
Will We Use VR Going Forward?
There is a lot of potential in VR for team connection and collaboration. When the hardware and software issues are mitigated, the experience is immersive, personal, and highly interactive. So, will Presence use VR for meetings in the future? Probably so!
We’re pleased with these initial results and we intend to run a longer experiment, allowing more time for our staff to work together within the VR environment and to more fully test the features of Workrooms or other apps. Every experiment marks the path for a better follow-up experiment and clarifies where to focus our energy. While running internal meetings in VR isn’t top of our list of goals for 2022, it’s certainly something we’ll continue to investigate.
Many thanks to Kevin Rohling for designing and running this experiment and to Natalie Hansen, Dane Summers, Duffy Gillman, Mikayel Amirkhanyan, Sravanthi Somalaraju, Jorge Garcia, and Joshua Kozlowski for their spirit of experimentation and collaboration!
Do you have an idea you’d like to bring to life? Drop us a message at contact@presencepg.com to start a conversation!