VR: The Future and Best Development Practices in Unity

I recently acquired a VIVE and after a day of oohing and ahing about how cool it was, began to create some simulations for it in Unity. The first of such is located on my main site, along with a demo video in case you don’t have a VR headset. You should definitely check it out.

I discovered a couple of things. First, I’m totally sold on VR being the future. I don’t get motion sick (at least while not moving from a fixed point in VR, more on this in a bit) , so I’m fine to whip my head around in Virtual Reality all I want. The experience is really cool, and tricked my brain into thinking I was somewhere else much more than expected. I first picked up a jetpack and immediately got butterflies in my stomach, because I felt like I was actually flying upwards! Over the next couple of years VR tech will improve drastically, just like all new devices. A few areas that could improve are portability, resolution of the eye pieces, and performance on lower end devices. We will also see advancements in handling sound. Currently you need to provide your own headphones and it’s a bit of a clunky setup.

VR Development Best Practices

So, what’s actually different about VR development? Beyond the obvious need for different gameplay design, there are some key details that devs might overlook. I refer to Unity with these points but they can be adapted to other engines, as conceptually they are the same.

Performance

Performance is much more important in VR than typical game design. This is because if the display lags, it can induce physical discomfort and nausea in some users.

Rendering

Rendering is one of the most recurring bottlenecks in VR projects. Optimizing rendering is essential to building a comfortable and enjoyable experience in VR. In Unity, “setting Stereo Rendering Method to Single Pass Instanced or Single Pass in the XR Settings section of Player Settings will allow for performance gains on the CPU and GPU.

Lighting

Every lighting strategy has its pros, cons, and implications. Don’t use full realtime lighting and realtime global illumination in your VR project. This impacts rendering performance. For most projects, favor the use of non-directional lightmaps for static objects and the use of light probes for dynamic objects instead.

Post-Processing

In VR, image effects are expensive as they are rendering the scene twice – once for each eye. Many post-processes require full screen draws, so reducing the number of post-processing passes helps overall rendering performance. Full-frame post process effects are very expensive and should be used sparingly.

Anti-aliasing is a must in VR as it helps to smooth the image, reduce jagged edges, and improve the “look” for the user. The performance hit is worth the increase in quality.

Cameras

  • Orientation and position (for platforms supporting 6 degrees of freedom) should always respond to the user’s motion, no matter which of camera viewpoint is used.
  • Actions that affect camera movement without user interaction can lead to simulation sickness. Avoid using camera effects similar to “Walking Bob” commonly found in first-person shooter games, camera zoom effects, camera shake events, and cinematic cameras. Raw input from the user should always be respected.
  • Unity obtains the stereo projection matrices from the VR SDKs directly. Overriding the field of view manually is not allowed.
  • Depth of field or motion blur post-process effects affect a user’s sight and often lead to simulation sickness. These effects are often used to simulate what your eyes do naturally, and attempting to replicate them in a VR environment is disorienting.
  • Moving or rotating the horizon line or other large components of the environment can affect the user’s sense of stability and should be avoided.
  • Set the near clip plane of the first-person camera(s) to the minimal acceptable value for correct rendering of objects. Test how it feels to put an object into your face in VR. Set your far clip plane to a value that optimizes frustum culling.
  • When using a Canvas, favor World Space render mode over Screen Space render modes, as it very difficult for a user to focus on Screen Space UI.

UI

More on that last bullet point above.

Something very interesting about VR is the need for a Diegetic UI. A Diegetic UI means a user interface that exists in the universe (in this case, a game) that we are experiencing. So, a non-Diegetic UI would be your health floating at the bottom left of your screen on a normal computer game.

Now here’s the problem. In VR: your eyes can’t focus on something that close. Putting something on screen close to the face of the viewer works really well for normal games where you can focus on the screen at a specific part. However, VR goggles work by projecting two separate images on each lens, and your brain combines it to achieve depth perception. Putting something that statically close to the screen makes the user’s eye attempt to focus on it, which makes the viewer go cross-eyed and the whole immersion is broken. The solution? Use diegetic UI elements. What this means is attaching the UI to objects IN the game world. This looks really cool, and accomplishes the goal of not breaking immersion and looking terrible.

Notice the time left is stuck to the gun, so the user can look at the UI themselves vs it being stuck on the screen

This type of UI hasn’t been limited to VR either, it just works really well in it. We’ve seen examples of this kind of user interface all over.


VR will hit mainstream within 20 years, and we will see long term usage within 50.


It helps me if you share this post

Published 2018-11-08 15:53:28

A word on Electron.

ElectronJS, also referred to as just Electron, is a really good idea.

Someone, at some point, thought to themselves, “Hey, web developers build websites all the time with HTML, CSS, and JavaScript. Why can’t we just render the same web pages people use as desktop applications?” Which is a brilliant idea. Put into practice though, I’m not sure that Electron should be your first choice when developing a Desktop application.

The implementation of it is what I have problems with. Now, there are a lot of better and smarter coders than me out there, but I feel as if I bring up some valid complaints. Here they are, in no particular order.

The size. A C# or C++ “Hello World” application (click for definition) is under a megabyte, and if memory serves, the C# one is less than 10KB. An Electron app published for the Mac is 115 MB in its “Hello World” application.  That’s insane. RuneTale *wink wink nudge nudge* is 36 MB. A full-on videogame is more than 3 times smaller than a basic program to display a string. This makes making small programs like the Searchifier, for instance, a no-go. If every program on your disk took up 115MB+, you wouldn’t be able to have very many applications. Plus, every time an Electron app starts up, it has to load its libraries into the RAM, which also weigh a ton. If you had a few Electron apps open, you could waste gigabytes of volatile memory.

The redundancy. Speaking of having to load all those libraries into RAM, let’s talk about that. Every single electron app comes with Node.js and Chromium. This is what is making the app so big in the first place. As stated above, bundling a complete browser rendering engine that has almost the same number of lines of code as the Windows OS doesn’t make for efficiency. This slows down newer and older computers alike. On terrible machines (Like this piece of garbage I had the displeasure of owning for a couple of years), running even ONE Electron app with a web browser like Firefox or even Edge slows the computer to a crawl, and literally increases loading times to above 20 seconds per click. Those were hard days.

The cluttered working environment. What’s Yarn? A year ago I would have told you it’s for knitting. Now? It’s a dependency manager for JavaScript. How do you get it? Through NPM. What’s NPM? A dependency manager for JavaScript. The first thing you do when you want to work on a project using React.js or Electron is “npm install” through the command line. Then, you wait as NPM installs the entire internet onto your local project. It gets ridiculous after a while when you have a couple projects on your hard drive all with their own “node_modules” folder that takes AGES to transfer all the tiny little files in it. Committing in git takes forever.

If I can add on to the other paragraph before you grind your teeth into dust, the fact that everything breaks at the slightest touch is a good one and goes hand in hand with the cluttered working environment. Every time you install the packages a project needs, you have to use the same version of those packages because one little change can break the whole thing. There are a couple of ways to denote what version you want, and it’s mostly taken care of for you. If you lock down your dependencies and specify that you only want specific versions of them, the dependencies themselves may not do the same. For example:

  • You want version 0.9.3 of dependency “a”.
  • “a” depends on package “b”, but allows versions 1.3.1 – 1.9.9, denoted by ^ (^1.3.1).
  • Package “b” depends on any path version of 3.4.1, denoted by ~ (~3.4.1) of packages “c” and “d”, both of which depend on any minor version of “e”, which is currently on 1.4.1.

So even if you lock down your dependency versions, package variations can exist across identical installs of the same project. This can lead to small bugs popping up out of seemingly nowhere as bugs are introduced in packages you don’t have any control over. What’s the fix? Often, it’s “turn it off and back on”, by way of

rm -rf node_modules && npm cache clean && npm install

What does this do? In plain English, “Remove the node_modules folder, clean the cache, and reinstall”.

Luckily, at some point someone realized this was an issue and dependency locking was added in Node 5.x+, but a mountain made up of tiny obstacles is still a mountain.

Electron produces some great looking results, however. Since anyone can use the power of CSS and JavaScript, applications can look very fancy and feel nice. Listed on Electron’s showcase page are a few applications that you might not have thought were built with this, including Discord, Skype, and Slack, and I can’t say I hate all Electron apps because I use Visual Studio Code almost religiously in my work and I love it.

Can’t help but wonder how much smaller and better performing Visual Studio Code would be as a native desktop application written in C++ though.


It helps me if you share this post

Published 2018-10-10 13:55:20

The Death of Flash… and the End of an Era.

Every millennial or Gen Z remembers playing Flash games in class on a school computer, or playing Super Smash Flash on your home computer. Those were magical, simpler times. But all of that is coming to an end, very soon. Flash is dead.

But that’s okay. Newer technologies are almost where they need to be to be a full-fledged Flash replacement. WebGL, WebAssembly, and HTML5 canvases have shown great promise and can fill Flash’s absence. There is one question though, what happens to all the existing flash content on the web?

Good question. And there’s no good answer… right now. Some are attempting to immortalize it as best they can, as is the case with Flashpoint, which is focused on saving flash games and making sure you can play them in the future when flash dies. It seems to be a worthy project and I encourage you to support them as best you can.

So, why is Flash being killed? Numerous reasons, chief among them being that Flash player is proprietary and Adobe controls it. Another is that Flash is a gaping security hole.

In terms of websites still using Flash… those websites will simply cease to function in 2020. For most browsers, including Edge and IE: “Users will no longer have any ability to enable or run Flash,” said John Hazen, a program manager on the Edge team. Google has a similar timeline, and by the end of 2020, you will not be able to run Flash on any major browser.

My thoughts on the matter are pretty neutral. I was a flash developer myself, and so it does make me slightly nostalgic and sad to think that all of that is going away. On the other hand though, technology is always evolving, and being a software engineer unfortunately means going with the flow sometimes. I look forward to seeing what advancements WebGL brings.

Again, if you’re feeling nostalgic about all those Flash games you used to play, I recommend checking out one of the numerous services wanting to immortalize them, like Flashpoint mentioned above. Go check out their Discord server and say hello!

Concussion, my last flash game I made, is luckily available online still. 😉


It helps me if you share this post

Published 2018-10-06 14:51:46

Be careful with app signing keys

Recently, I received an email from Google Play services. “Your app has been removed from the Google Play Store for a policy violation”, or something like that. How odd, I thought. I don’t remember doing anything against their terms of service. The email revealed that I didn’t have a valid privacy policy inside the app or on the store listing.

Oh. Right. The whole GDPR thing. It was time to write some privacy policies. After doing so, I began the process of digging up old files to old apps to make the necessary changes to the code. After about 2 hours of reinstalling Android Studio (my hard drive was wiped as some readers may remember), I began the process of exporting the app from Unity to an .APK.

Eventually, I was able to upload the finished .APK to Google’s servers. However, the Play Console threw an error at me; “The signatures do not match”. Wait, what? It’d been too long since I’d actually done this process. I googled the error to remind myself and broke out into a cold sweat.

Apparently, you generate a .keystore file upon first creating an Android app to sign the application with. It prevents people from uploading versions that aren’t originally from you, in the event that a developer’s account got hacked or something. There was no way to recover said .keystore file if you didn’t have it anymore, meaning there was no way to update my app. Ever. A full, in-depth system scan revealed no .keystore files. Luckily, with the two brain cells that were still functioning, I managed to remember that the other day I had deleted the app-which-I-was-updating’s Android version off my hard drive, because there was no real difference between the iOS and Android version, and I thought it was redundant. Perhaps it was in there?

I checked my Recycle Bin and breathed a sigh of relief. I hadn’t emptied it. It was still there. Opening the folder, the first thing I saw was a “user.keystore” file at the very bottom of the file list. A quick test confirmed that was the one. Phew.

Apparently those things are important. Don’t lose ’em, kids.


HEY, LISTEN! It’d be really cool if you checked out the app here on the play store, since it just got updated. 😉


It helps me if you share this post

Published 2018-09-19 15:02:16

What is Ray Tracing?

With the advent of Nvidia’s new GPUs coming out, the inevitable question was asked of me by friends: What is ray tracing?

Over the past decade or so, graphics in video-games have made enormous leaps and bounds, and it’s now trivial to simulate hundreds of thousands of physics objects all at once with a very standard graphics card. However, the huge leaps in processing power have slowed somewhat, and now small tweaks and improvements to what we already have are being made. This includes hair, water, and light.

Light is one of the hardest things to simulate correctly. There are many ways to fake it, and game developers have gotten very good at certain techniques to make you believe that the sun is over there and casting shadows here, and so on. However, ray tracing is a method that attempts to actually simulate the behavior of the photons of light. Sounds cool, right? You can imagine how intense of a calculation that would be for many light rays. Because of this, it’s not possible for a real-time rendered game or simulation to process it quickly enough to display on a normal computer. Ray tracing is currently only used in pre-rendered films or animations.

Without going into the actual physics of how light and photons work, interesting read here, ray tracing is an expensive operation (takes a lot of computational power). Nvidia is taking steps to make it more feasible and possible in the future by providing “ray tracing upgrades” to their graphics cards. We will have to wait and see how much of an actual improvement they’ve made.


It helps me if you share this post

Published 2018-09-14 14:24:12

What is blockchain?

I’ve seen a lot of people who don’t understand completely what blockchain technology is, they just know it has to do with Bitcoin. Here’s a very brief and understandable explanation.

Let’s say that you have a group of 100 people at your workplace who all have copies of a particular spreadsheet. This is a blockchain. Blockchain is secure against hackers in the idea that even if one person’s spreadsheet got hacked and changed or even lost, there are still 99 others who have the exact same spreadsheet, and they will each know that because their spreadsheet matches with 98 other people, they have the correct version. Now, let’s say that your boss sends out an email to add another row to the spreadsheet. Each one of you will add that particular row, and then check with your colleagues to make sure you’ve done it correctly. Anyone who claims that the row is wrong has to compare with 99 others.

This is a very basic idea of how blockchain works, and why it is basically impenetrable to hacking. To corrupt the spreadsheet, someone would have to hack in to 51 people’s computers and change the spreadsheet. Now imagine there are a million employees all with the same spreadsheet, and the task becomes a little harder to accomplish.

If you’re interested in learning more of the technical side of blockchain, I recommend taking a look here.


It helps me if you share this post

Published 2018-08-31 17:17:56

How to ask good Stack Overflow questions.

Many new programmers have low opinions of the community on Stack Overflow. For those of you who don’t know, Stack Overflow is a Q&A site meant specifically for coders. However, it has a reputation for… how shall I say it… being a bit of an elitist gatekeeper-esque, unfriendly place. On the flip side, Stack Overflow is the largest repository of developer answers and questions on the internet, and it’s one of the most visited websites for coding questions by developers, if not the most.

Continue reading How to ask good Stack Overflow questions.


It helps me if you share this post

Published 2018-07-28 04:26:07