We finished 2019 with a bang, and now, after a challenging four months of work, we are finally ready to announce Bright Engine v0.1.7a! This update brings about a complete overhaul of the rendering architecture, a brand new material system and a node-based editor, along with a handful of other new tools! But most importantly, we fixed a lot of the performance limitations that had been present in earlier versions, with fps almost doubling across the board with no reduction in quality! With over 40% of the entire code rewritten, there is a lot to talk about!
New Features
Material Editor
Let’s start things off with something which has been the most requested feature since Bright Engine went into alpha: the Material Editor!
The Material Editor is the primary tool for creating the visual style for your project as well as many exciting effects. It allows you to connect a series of easy-to-use nodes together to create complex material without needing to write a single line of shader code!
Materials can be edited in real-time, and the results can be seen both in the 3D viewport of the Material Editor and Main Editor.
There are a total of 25 nodes at your disposal, with plenty more planned for future updates. This plethora of nodes opens the doors to a wide range of possibilities which are too plentiful to talk about here, but below is a simple method of blending two textures together based on their heightmap!
The preview scene has a range of different lighting environments in which you can preview your material, so you can see exactly how it looks in-engine and, more importantly, in-game.
Until now, creating custom material could be achieved via custom shaders linked via the Shader Manager. However, this approach to doing things was rather clunky and pretty much made it impossible without knowing how to program shader files. As such, the Shader Manager has been completely removed, as it is now redundant.
Go To & Bookmarking
We also introduced a few other new minor tools to increase development efficiency, namely a Go-To system and a Bookmarking System.
A basic version of this previously existed with the Camera To Origin option from the View drop-down menu. The replacement Go To system pretty much does the same thing, except now you can specify to which coordinates you want to travel too.
The Bookmarking system works as an extension of the Go-To system. When editing large open worlds, there are a lot of locations to work on and having to remember all those coordinates is a real nuisance. The Bookmark Manager allows you to save locations under a set name which you can later travel to via its interface.
New Demo Scene
A final addition to this update is the release of a brand-new Demonstration Project!
The last demo project, while containing a few exciting scenes, became increasingly messy and didn’t assist users in understanding how the features of the engine work. This new demo project places each element on display so users can see exactly how everything works and can follow along with the documentation.
Changes & Improvements
Rendering Architecture
Let’s talk about the biggest star of this update: The Rendering Architecture. What became increasingly more apparent over the past few updates is that with the ever-increasing number of additional features, the overall performance of the engine simply became worse and worse. Despite our attempts to improve things (and we did), we were unable to truly resolve the problem to a satisfactory level for lower-end machines.
The primary issue was how PBR was calculated and rasterised. While SSAO and Shadow maps were taking full advantage of geometric instancing, the main PBR stage was not. It remained on a naïve per-object drawing system. Let’s break down a basic outline of the old architecture of the PBR pass:
If we take the desert scene as an example, it had approximately 100 models in that small space. Each of those models had a single material which consisted of three-channel packed un-optimised 2k textures.
On an NVIDIA GTX 1070 (a higher-end spec GPU), it struggled to reach 46 frames per second. Ignoring the incorrect use of 2k textures on small assets like gravel, why would such a GPU have difficulty? The answer is: it doesn’t. The cause of the performance problems is due to the CPU-to-GPU bottleneck. Whenever drawing a model, the CPU has to transfer all the information from the RAM to the V-RAM, and this process is slow due to physical limitations that cannot be avoided. This is why we saw a massive boost in performance in v0.1.3b when instanced rendering was first introduced. Instancing allows you to send data in groups from RAM to V-RAM, thus reducing the total number of send operations the CPU has to perform (we call these draw calls).
With the PBR pass not utilising instanced rendering, this creates a total of 100 draw calls (one for each model) even if multiple models are geometrically the same.
But this alone does not explain the poor performance, after all, most games have up to 1500 draw calls per frame and don’t suffer from performance issues as drastic as this. Thus enters the second half of this problem: textures. Much like sending vertex data to the V-RAM, the same process occurs when sending texture information (we call this texture binding). As the PBR pass was drawing object-by-object, it meant that the textures for each object would have to be bound each and every time, even if they all used the same textures. Very Bad!
So what does that mean? Well, the Desert scene only had 6 different materials (5 for rocks, 1 for flora), which consisted of 3 different textures, so that’s a total of 18 different textures being used. But each one of those textures is being bound per object, so that’s 3 texture binds for every 100 objects resulting in a total of 300 texture binding operations! No wonder the engine was struggling!
Time for a new approach:
The new architecture addresses both the draw call and texturing binding problems. Models are now drawn on a far more complex but efficient material-based system. This allows for the instancing of geometric data (as done with the SSAO and Shadow maps), as well as only binding the textures of materials once per frame as opposed to once per object.
And here are the results:
I think we can all agree that this was a massive improvement! However, this is only the first part of this architectural upgrade. One additional system that deserves far more attention is how shader uniforms are handled, which we will be turning our attention towards in the next update.
We have grossly oversimplified the process of CPU-to-GPU communication to make it easier for you to understand. However, if you are interested to know more, here is a great article written by Keith O’Conor which delves into far more detail than we have.
CPU Improvements
Let’s take a walk down a familiar lane and talk about lighting (as usual). By shifting the rendering architecture to a material-based system, it meant taking a closer look at the CPU portion of the lighting calculation. Much like textures, the method which determined which lights should be sent to the GPU based on the distance to the current model being drawn was having to repeat itself for every object. This slow calculation meant that only a small number of lights could be active at one time.
The new method selects and prepares light information based on which lights are closest to the camera, removing a lot of unnecessary work for practically the same result. This was so much more efficient that we were able to increase the maximum number of lights to be rendered at once from 12 to 48 and still saw an improvement in performance!
Lighting data was also packed into more tightly packed data structures which allowed for fewer uniform shader calls (passing variables into shaders), resulting in the reduction of more unnecessary CPU cycles.
One final area of improvement in regard to rendering was how room data was being handled. The effects, such as transitioning skybox textures, fog properties, and sun properties, were all being done on the GPU. Interestingly enough, this is one instance where this is actually slower than just doing it on the CPU. Why? Well, the transition factors and properties are all the same per frame, so doing the calculations on the GPU means the operation was repeated per fragment of everything being drawn. Since the values are never changing during a single frame, it is just far more sensible to calculate them once on the CPU and send the results to the GPU.
Bug Fixes
World Editor
- Fixed a bug where changing a Spot light to an Area Light while having baked shadows caused a crash
- Fixed a bug where transitioning skybox effects were not being applied to Dynamic foliage
- Fixed a bug where the UI for Linear Fog settings would remain disabled when selecting a fog layer which was using the linear fog equation
- Fixed a bug where no checker was in place to ensure that a valid zone name was selected by the user, which would otherwise cause a crash
- Fixed a bug where skybox data was being compressed to 8bit instead of remaining its native 32bit when baked to textures
- Fixed a bug where reflection probes data was being compressed to 8bit instead of remaining its native 32bit when baked to textures
- Fixed a bug where some of the settings in the Render Panel were being clipped by the scroll bar
- Fixed a bug where the Particle Editor would remain open even when no project is loaded
- Fixed a bug where changing a value in the Post Processing panel caused the cursor to jump to the start of each component making typing values impossible
- Fixed a bug where using timeline mode on a newly placed Area Light caused a crash
- Fixed a bug where removing a light source directly after changing its type would cause a crash
- Fixed a bug where objects which had just been added to the scene did have a visibility checkbox in the hierarchy until after the zone was reloaded
- Fixed a bug where removing objects could sometimes cause crashes
- Fixed a bug where objects using Mask alpha that were duplicated were not being ordered correctly before drawing, resulting in transparency artefacts
- Fixed a bug where changing a sound source from 2D to 3D while the camera is inside the audio area caused a crash
- Fixed a bug where changing brush size, strength or gradient cut off using shortcut keys while painting terrain caused a crash
- Fixed a bug where vertex colour painting terrain at maximum strength caused a crash
- Fixed a bug where vertex colour painting terrain at maximum strength caused a crash
- Fixed a bug where the Audio areas would hide camera, targets and emitter objects
- Fixed a bug when baking with a reflection probe that did not update the clipping planes, causing incorrect results
- Fixed a bug where adding, renaming or deleting a new skybox with a Room placement selected instead of the Room itself caused a crash
- Fixed a bug where adding, renaming or deleting a new fog layer with a Room placement selected instead of the Room itself caused a crash
- Fixed a bug where adding, renaming or deleting a new cloud layer with a Room placement selected instead of the Room itself caused a crash
- Fixed a bug where adding, renaming or deleting a new weather layer with a Room placement selected instead of the Room itself caused a crash
- Fixed a bug where selecting a room and any other object and then attempting to use the 3D cursor caused a crash
- Fixed a bug where if two audio zones were using the same sound file, it would cause a crash if the player entered one of the sound zones
- Fixed a bug where selecting an object after locking a previously selected object caused some settings to remain disabled when they should be enabled
- Fixed a bug where assigning a camera to a waypoint path caused a crash
- Fixed a bug where assigning a target to a waypoint path caused a crash
- Fixed a bug where reflection probes did not correctly display their baked results
What’s Next?
This update was the most extensive update in terms of work done to date. Almost 40% of the entire engine’s code was rewritten, and a massive code change like this means lots of bugs! Specific effects, such as Sub-surface scattering and alpha blending, as well as Editor side issues, have been created as a result of this grandiose update. Furthermore, as the number of features and settings increased, the user interface of the editor itself has become quite clunky and slow. We intend to fix this by redesigning the UI to be far more fluid as well as requiring fewer resources, fixing slow selection, scroll tearing, and awful tab-switching delays.
Therefore, version 0.1.7b is all about finishing improvements to the rendering architecture, refining features and tools that are already in place, and of course, fixing as many bugs as possible!
