Codepunk

Codepunk 063 Proximity in Virtual Reality

Your browser does not support the audio element.This is a written extension of the Codepunk podcast [/podcast/codepunk/]. You can also listen to the audio version or watch the video version at the end of this article. When discussing virtual reality (VR), there is a specific aspect of design involved—whether world design, production design, or application design. These environments are not exempt from the same design principles available in 2D design environments. In fact, in many ways, some of the core principles of user experience design are even more relevant and important in virtual reality because much of the work we've done in 2D or computer application design has always attempted to take into account a spatial feel. As fate would dictate, multiple conversations around the topic of virtual reality, remote work, and faux conference rooms collided on the Internet in a matter of weeks. Between Ben Evans tweeting low-blow commentary on VR to Azeem Azhar asking about virtual conference rooms to the figurative explosion of augmented and virtual reality announcements and acquisitions from Snap and Facebook, virtual reality continues to have its moment. It's a moment of confusion, uncertainty, enthusiasm, and high use of the word "metaverse," but it's still a moment. Around this time, I left the following on a Friday members-only post from Azhar's Exponential View [https://exponentialview.co] web site: > I'll start with something Ben Evans tweeted where he said that working in VR seems like the opposite of "everything everyone has built in the last decade. It's not about a bigger screen." This is more a statement on the attempted solution and not an acknowledgment of the problem. Although it's true that progress in remote work (and work in general) has led us to mobility, increasing freedom of movement, productivity, etc., the trade-off is a loss in spatial interaction. Every conferencing software has attempted to solve the problems with spatial interaction and spatial awareness through hand-raising emojis, table and grid views, shared whiteboards, etc., and despite progress, we're still using a predominantly 2D interface and organizational system while working. We may have "layers" to our software, but movement and interaction within those layers still only incorporates 2 dimensions. VR/AR and the XR group in general (not to mention several other non-XR software applications) have been working on ways to solve this spatial deficit and eliminate some of the final barriers to a seamless remote work transition. WorkRooms isn't it, but is a good experiment for other initiatives to learn from. Evans has a habit of poking at VR because he sees it as little more than a subset of gaming. But Mark Zuckerberg wants to re-brand Facebook a metaverse company and sees the future through an augmented and virtual lens. The truth is somewhere in the middle for now, but technological progress dictates that it probably leans closer to Zuckerberg eventually. But what is this "spatial software?" What does that phrase actually mean? Think about this statement from John Palmer [https://darkblueheaven.com/spatialinterfaces/]: > Think about this: there is no app that replicates a deck of cards. Go ahead and look on the app store, and you'll find a bunch of apps with bad reviews and extremely limited functionality. If you start designing a deck of cards app as a traditional 2D app the way we think about them today, with 2D panes and menus and buttons, it becomes painfully difficult to think about. Palmer posits that although in several cases reducing something to a 2D interface might make it easier to understand, there are just as many cases where leaving something as a 3D interface makes it more intuitive since humans are inherently spatial thinkers. More from Palmer: > In particular, some applications use spatial relationships to suggest how they should be used. Incarnations of this thinking are called spatial affordances. It was once Palmer started talking about spatial affordances that I was reminded of another concept in user interface design: proximity. And this idea of proximity essentially makes his case for spatial needs in software. Proximity in design tells us that the closer elements are together the more they are functionally related, and as things become functionally similar in design they become closer together until they are right on top of each other. A good example is an elevator panel. There are three elements: the buttons, the numbers, and the lights. The button selects the floor. The number tells you which floor. The light confirms your choice. All three of these are highly integrated and thus the most pervasive elevator panel design has the button, light, and number integrated into one piece. It would make little sense to have a legend of number and button correspondences on the wall that you need to consult in order to know which button you need to select. To say that this would be bad design is an understatement. In 2D design proximity is determined on an x-axis and y-axis, so the expectation should be that related ideas and concepts should be closer together on the screen, and how we represent that closeness becomes the spatial aspect of a 2D design. It's important to note that spatial representation doesn't have to mean virtual reality or 3D software. A third dimension just makes it easier. However, you can represent spatial concepts in 2D and the success of this representation often determines the usability or intuitiveness of your software. Palmer gives a great example with the chat application Sococo [https://www.sococo.com], but you can also look to the popular WorkAdventure [https://workadventu.re]. The chat applications are designed like old school video games, but interestingly enough, if used properly you can immediately tell at a glance what people are working on, who is chatting with whom, who is on break, and many other "presence" indicators with much greater accuracy and speed than a simple chat window. > [Sococo is] just a 2D floor plan of an office, with a bird's eye view. Every employee has an avatar. You can simply place your avatar in one of the rooms with other people. The room provides text and video chat for the people inside. Now try to answer the same question with this software. "Who is everyone at my company meeting with right now?" All of a sudden, it's extremely easy. You just look at the rooms. At a glance, it's immediately obvious which people are working together. The floor plan is a skeuomorphic spatial affordance, one of the few I can recall seeing. It displays rich information and makes that information easier to understand by analogizing a real-world spatial environment. With 3D design, you now have a spatial environment to work with, but more importantly, when discussing conversational or social applications of 3D software such as virtual reality, spatial software takes on a dynamic much closer to real life. Bill Ahern in Facebook Horizon [/images/proximity-in-vr-01.png] Proximity and presence are much more powerful in virtual environments. Augmented reality might add an extra layer to the physical universe (a meta layer), but virtual reality is about world-building and represents a different experience. When we use text messages, phone calls, and even video chats, there is a recognition of spatial barriers and distance between participants. But videos games started to break down this barrier as multi-player games, open worlds, and chat parties created an environment that could be internalized and create an imagined closeness. You feel like you're with your gamer friends in the same room/universe. Virtual reality takes that a step further. When we chat; When we Zoom—there is this perceived barrier—a screen between us. But when we're in virtual reality, we feel like we're with the other person. We perceive a closeness in proximity, and the best virtual software will fold this spatial dimension succinctly into their design.

Codepunk 062 Virtually Reality is Virtually Ready

Your browser does not support the audio element.This is a written extension of the Codepunk podcast [/podcast/codepunk/]. You can also listen to the audio version or watch the video version at the end of this article. With the launch of season 6 of the Codepunk podcast, we'll be coalescing around a single theme since we'll be producing less episodes at a higher quality with accompanying video inside of virtual reality. This builds on the last [https://youtu.be/-BKySvgYaY4] two [https://youtu.be/Kv-PTZmXt-U] episodes of the previous season where we recorded inside of AltSpaceVR exploring the ideas, themes, and feel of a digital space. With the new season, we're committed to exploring podcasting within virtual reality, and we'll be building out a space inside of Facebook Horizon [https://www.oculus.com/facebook-horizon/] as we progress through the episodes during the year. Essentially, Bill [https://twitter.com/neurothustra] and I are intent on understanding the new frontier and world-building of virtual reality and we're bringing you along for the ride. I'm sure it seems like we're beating the drum ad nauseam over virtual reality—and we've certainly been keen to defend the technology [/are-we-in-a-vr-winter] against economists and financial analyst naysayers—but the progress made thus far is too far not to see the promise beyond the horizon. Prior to the Oculus Quest, my primary experience with VR had been a few low-end Samsung devices that mostly stick a screen close to your eyeballs—neither enjoyable, nor comfortable—and it's important to realize that the Oculus Quest doesn't stray far from that, but the advances in that area alone have resulted in burgeoning experimentation for work, play, and every crevice between. Oculus Quest at Wikimedia Commons [/images/vr3-oculus.jpg] As we turned the corner towards 2020 and slammed directly into a pandemic, I found myself spending way too much time backing almost anything on Kickstarter that had to do with cyberpunk as a genre. Why? I'm not sure. Maybe it's nostalgia [/apotheosis/cyberia-old-and-new], but a couple of those campaigns were solid looking virtual reality games, so I jumped at the opportunity. I backed them even though I didn't have any VR equipment at the time. Once those campaigns started to succeed, I realized that it might be time to cross that chasm. At the same time that I was backing these Kickstarter campaigns, Facebook started promoting it's VR world Horizon. Facebook certainly hasn't been on my list of most ethical companies, but Horizon looked like a keen experiment, and ultimately brought me to the conclusion that although I probably won't spend too much time with games, the social experimentation of virtual reality worlds is a worthy pathway of investigation—presenting an evolving future of communication and commerce [https://about.fb.com/news/2020/08/facebook-horizon-welcomes-first-virtual-explorers/]. > Horizon invites you to explore an ever-expanding universe of virtual experiences designed and built by the entire community. Everything you see in Horizon, including the Plaza and worlds created by our teams at Facebook, has been built with the Horizon creation tools. In Horizon, you can build the things you want to see and places you want to visit. Our mission is to create meaningful connections between people and foster a strong sense of community for everyone who joins Horizon. That’s why we’re excited to hear your feedback and work with early adopters to evolve Horizon as we begin to grow its community. Horizon is as much about meaningful connections as it is about creating VR worlds and expressing yourself. Eventually, we envision large spaces where many people can gather in Horizon, but for now, up to eight people can share a space. Although Horizon has seen significant delays and currently sits squarely on the shoulders of beta testers, the user experience is far superior than any other world-building application. It's clearly designed to push world-building into the virtual hands of casual users. Games, obviously, will not be left out in the cold—despite my personal ambivalence. One need only to look as far as massive multiplayer online (MMO) games to see the future of virtual warfare. In fact, there was an XBox game called Defiance [https://en.wikipedia.org/wiki/Defiance_%28video_game%29] (tied to a SyFy channel show) that Bill and I played for a period of time when it launched. The open world wasn't really one that lent itself to extensive hangout sessions, but that didn't stop people from standing their avatars next to each other in bars and safe havens, or having fun driving their quads off high-rise structures to see how many times they can turn in the air… all within a post-apocalyptic world. It was good fun that let the imagination run wild, and you see much of the same going on in other, more popular open worlds such as Fortnite, as well as niche genre games like Star Trek Online, where you can hang out on a planet without much to do other than pretend. In all of this we've said very little about augmented reality despite the recent Facebook investment in wearable devices to enhance AR experiences [https://newatlas.com/vr/facebook-wrist-ar-control-neuromotor/]. > In an online "Road to AR glasses" roundtable for global media, the Facebook Reality Labs (FRL) team laid out some of the eye-popping next-gen AR technology it's got up and running on the test bench. […] Camera-enabled hand tracking like what the Quest headsets are already doing will always be a little clunky, and for AR to take off as the next great leap in human-machine relations, it needs to operate with total precision, at the speed of thought, so smoothly and effortlessly that you can interact with it without breaking your stride or interrupting your conversations. […] To make that happen, the FRL team has been working on a next-level wrist-mounted controller that quietly sits there reading neural impulses as they go down your arm. And while Facebook continues to innovate and Apple attempts to break into the high end consumer market, Microsoft continues to pursue military contracts with cloud, artificial intelligence, and yes, virtual reality offerings [https://www.cnbc.com/2021/03/31/microsoft-wins-contract-to-make-modified-hololens-for-us-army.html]. Although virtual reality has been used for years to treat soldiers with PTSD, Microsoft and the U.S. military are taking things a step further with a large-scale HoloLens contract to blend augmented reality with warfare training. > The U.S. Army said Wednesday that Microsoft has won a contract to build more than custom HoloLens augmented reality headsets. The contract for over 120,000 headsets could be worth up to $21.88 billion over 10 years, a Microsoft spokesperson told CNBC. […] An IVAS prototype that a CNBC reporter tried out in 2019 displayed a map and a compass and had thermal imaging to reveal people in the dark. The system could also show the aim for a weapon. Microsoft HoloLens at Wikimedia Commons [/images/vr3-hololens.jpg] VR has always been a great technology to follow as we move through science fiction, science, and cinema. It's evolved as much on screen, as it has in real life, and in the imaginations of those interested in carving out a place in the new cyberia. Although people raved about Battlestar Galactica (the remix), most forgot about the prequel: Caprica [https://en.wikipedia.org/wiki/Caprica]. Caprica was notable for a few things; Among them, the concept of the Holoband. This headband took wearers into a virtual world created for many things, but ultimately it had degraded into a virtual reality overrun by games and teenage clubs. The Holoband was easily hacked by would-be programmers, opening up new worlds for users to explore. It was essentially an evolving virtual hackerspace. This concept isn't new. The idea of a virtual world was explored heavily in other series and films such as Tron and Tron Legacy. Caprica also shared another concept with Tron: The idea of sentient avatars or programs. Tron Legacy even referred to isomorphic algorithms as spontaneously emergent programs and Kevin Flynn thoroughly believed that the discovery of these emergent entities was the key to changing society, medicine, and even religion. Whereas Tron was sci-fi on the "grid," what made Caprica different was its grounding in a near future coupled with the "realness" of the virtual worlds. Recently, entertainment has come a step forward in this concept with the way augmented reality is treated in The Feed on Amazon Prime [https://en.wikipedia.org/wiki/The_Feed_(British_TV_series)]. The Feed focuses on an Internet-enabled/cloud-enabled augmented reality feed directly embedded in people's brains. With the feed "on" not only do you have unlimited access to playback of your memories, news feeds, etc., but it also allows your brain to process QR codes that influence the world that the feed (and you) see, taking augmented reality to a new level, and furthering concepts of a programmable reality. And all of this innovation is without us even scratching the surface of initiatives like Elon Musk's NeuralLink—something that most dismissed when it was initially announced, but recent news stories [https://www.businessinsider.com/elon-musk-neuralink-video-monkey-games-pong-brain-chip-2021-4] suggest that progress is being made: > Neuralink, a company founded by Musk that is developing artificial-intelligence-powered microchips to go in people's brains, released a video Thursday appearing to show a macaque using the tech to play video games, including "Pong." […] In August 2020 the company did a live demonstration of the technology in a pig named Gertrude. To be sure, this progress is not exclusive to NeuralLink, as past researches have been highly successful in similar feats, but as the article states, it does show that the technology Musk is pursuing is proving out in tried-and-true areas, bringing the company closer to its science fiction ambitions. Although all of this is on the edge of science fiction… augmented reality and virtual reality are not, and we've finally reached the point where headsets and other devices are economically feasible, as well as entertaining, educational, and full of potential. A complete brain interface might be currently out of reach, but we don't need that to get closer to the world immersion that takes social networking and MMOs to a level of science previously only seen as science fiction.

Codepunk 061 Experiments in AltSpaceVR Part 2

Part 2 of our experiments in virtual reality! Join us as we take the podcast into virtual reality. In this episode (part 2 of 2), we continue our experiments recording inside of virtual reality (VR) using the Oculus Quest. For this episode, we we stay inside of AltSpaceVR, but update our avatars, talk shop, and play some Bots + Beer on the integrate YouTube screen.

Codepunk 061: Experiments in AltSpaceVR Part 2

Join us as we take the podcast into virtual reality. In this episode (part 2 of 2), we continue our experiments recording inside of virtual reality (VR)...

Codepunk 060: Experiments in AltSpaceVR

Join us as we take the podcast into virtual reality. In this episode, we begin our experiments recording inside of virtual reality (VR) using the Oculus...