The primary problem with realistic rendering, and how mirage aims to solve it

The primary problem with realistic rendering, and how mirage aims to solve it

So, I have now given a brief introduction to what mirage is - but I haven't touched on what it aims to actually solve at all, which is something I can hopefully clear up a bit in this entry. 

First and foremost I shall give you a brief lesson on how lighting works in the real world 

(note, this is an oversimplification) 

As many of you probably already know, in the real world lighting is in its most simple form; a bunch of photons bouncing around in a scene. When a light ray hits a surface, some of it gets absorbed, some of it refracted and some of it is reflected into the rest of the scene. A tiny amount of that reflected light will eventually end up in our eye, where our brain then translates that into what we then see. 

Now, we can actually split this up into two different categories - direct and indirect light. Direct light is light that hits only one surface, and then hits our eye. (for example lighting coming from a directional light source) whereas indirect light is light that then bounces around the scene, hitting other surfaces before ending up in our eye. 

Direct lighting isn't a very big deal - even in real time; this is mainly because we know where it originates from, both origin wise and direction wise. There are many good ways to handle direct lighting in real-time with very nice results, and we've been using those techniques for many years. 

Indirect lighting is much harder, because we don't really have any idea where it originates from or what direction it has. Another key factor with indirect lighting is that to have very high quality results you effectively need a way to store your scene information and access any point of it (there are a few ways of doing this; for instance you can store everything as voxels in a texture or an octree, or you can go all out and store the triangles in your scene - something I'll get into later) 

Mirage direct, notice how everything not directly lit up is pitch black

Mirage indirect - notice now how clearly you make can make it out every shape in the room - but how there are no longer any direct shadows

Mirage direct+indirect - now you can clearly make out everything in the room, and there are some very pleasant direct shadows.

So, how have we gone about handling indirect light in the past? 

constant-ambient-factor

Or what is often called just "ambient lighting", constant ambient factor is the least intresting method as it just adds a constant light for everything not directly lit, leading to very flat looking results. 

ambient - left most side, notice the very flat appearance - it's very hard to make out the exact shape of the object. 

global-illumination-approximations 

(VXGI, point-light based GI, reflective shadow maps, and more fall under this category) 

approximations to global illumination have spawned over the years, but almost all of them have many cases where they fail (most of them limit the scale of your world, and doesn't work well with smaller geometry, and basically all of them have problems with light leakage) 

So, when starting work on mirage - i quickly realized that none of these solutions would work well in my case - so I looked away from games and on to films. 

The difference between films and games (rendering wise)

The primary difference between games and films is that films have time - in many cases one frame takes hours to render which is fine for films as they aren't interactive, but for games its completely unacceptable (~33ms per frame/30 fps is around the sweet spot for non-action filled games) 

This does however mean that films can use much more advanced lighting than games - and I felt this was a good starting point for mirage. 

So how do films do it? 

Films have for a few years now used a technique known as path tracing - which is a specific method of ray tracing. The reason that path tracing is so powerful is because it's very physically accurate - as in it does things very similar to how light actually behaves in the real world. The catch? It's a monte carlo method - meaning that originally it doesn't look very good (noise is a major issue) but it gets better over time. Path tracing quality can be measured in "samples", where most scenes start to look noise free around the 1000 sample mark. The only issue with this is that path tracing is very expensive, so actually rendering 1000 samples per frame is out of the picture. Indeed, mirage only renders ½ a sample per frame! The good thing about path tracing is that there are other ways to get rid of the noise - and I'll get into many of these methods later on in the blog. 

Path tracing with increasing amount of samples - each frame doubles the amount. 

Hopefully you should now at least have a very basic understanding of rendering, path tracing and what it aims to accomplish. If you do want to know more - I have left a few helpful resources at the bottom of this page. 

[1] - Wikipedia article about global illumination

[2] - Wikipedia article about path-tracing 

[3] - Wikipedia article about monte-carlo methods 

[4] - Disney's practical guide to path tracing; a very helpful and intuitive video explaining path tracing further