To kick start the first of many updates in 2019 we begin with the 0.1.5 series. As previously mentioned in the 0.1.4c patch notes, this series of updates is all about improving what already exists in the engine. We want to introduce better visuals, more stability and higher performance than ever before and for this update we begin with the most fundamental part of the engine. It’s core. We’ve added more compatibility with additional file formats, performed some usability prototyping and wiped out a lot of bugs!
Sorry nothing here this time, but wait till you see what’s next! :D
Changes & Improvements
We first drew our attention to the asset importers. Until now only .fbx and .obj file formats were supported. While these pretty much cover all grounds in terms of model file formats it felt quite restrictive, so we have expanded the capabilities to now also support Collada .dae. This brings in a second file format which supports animations along with a suite of other future potential options (such as camera animations and way-point paths).
Loading times is something that has particularly been steadily increasing with each update, which makes sense since there is a lot more going on. We identified that the longest parts of the loading system was models & terrain, although as it turned out these weren’t the problem, textures were. We started exploring exactly what was going on and discovered that the existing texture loading system was just plain awful. So we changed it.
We re-wrote the entire texture loading system and introduced drastic improvements, in both compatibility and speed. The supported formats haven’t changed (.bmp, .png, .jpg, .hdr, .dds, .psd) and upon loading are automatically converted to the DXT dds format which for those who don’t know, this is the most optimized texture format for modern GPU’s. This also fixed the problem where non-dds textures weren’t correctly generating mipmaps. If you are already loading a .dds texture (which we strongly recommend) this conversion process and mipmap generation are skipped which results in a much faster loading time.
While these figures are far from perfect we are happy with the results for now!
The dds texture loading system also got an upgrade and now supports a larger array of formats:
- DXT1 (4bit RGB)
- DXT1a (4bit RGBA)
- DXT3 (8bit RGBA)
- DXT5 (8bit RGBA)
- 5.5.5 (16bit RGBA)
- 5.6.5 (16bit RGBA)
- 188.8.131.52 (16bit RGBA)
- 184.108.40.206 (16bit RGB)
- 8.8.8 (24bit RGB)
- 220.127.116.11 (32bit RGBA)
- 16f.16f.16f.16f (64bit RGBA)
- 32f.32f.32f.32f (128bit RGBA)
After some feedback from the community we wanted drastically improve the object editing tools. Last update we revamped the entire placement system which works like a charm but forgot about fast object movement and rotation. This update we addressed that by once again revamping the 3D cursor, introducing 2 new tools and adding a collection of new short cuts!
The first of the new tools is the Grab Tool! As the name would suggest, it grabs hold of the selected object and moves it along the surface of the terrain with the mouse.
We then also adopted the previously created align to ground algorithm so that as the object is being moved around it also rotates it self to be constantly in line with the terrain (this can be toggled on and off)
The second new addition is essentially an alternative means of accessing an existing one. When in scale mode, you can now perform uniform scaling along the xyz axis using the 3D cursor.
Lastly one annoyance that kept coming up is when adjusting the rotation of an object the editor would not allow you to go beyond the bounds of -180° to 180°. While these bounds are still in place (stops the math from breaking), the editor will simply jump from one end to the other if going beyond this range. Visually the object keeps rotating as normal instead of getting blocked like before, causing a few headaches.
Let's talk terrain. In the 014 update series we drastically upgraded the tools available to you in terms of world building, and yet we found more room for improvement! The blending algorithm used for each of the texture layers was creating some strange visual artefact errors. We actually traced this back to an error in our PBR shader which has been fixed, but before getting to that conclusion we rewrote the blending system within the Terrain shader to be significantly more optimized which helped remove 40 lines of code. Sadly the performance boost was negligible but better code is still better!
Terrain albedo textures can now support alpha information which the blending algorithm takes into account. This opens up a whole new avenue of detailing which is hard to explain in words but here is an example that shows what can be done:
Moving on, a few occasions when working on a large project, files get moved around or just simply go missing. One of our testers had this issue and kept wondering why the engine kept crashing. While we can do nothing to stop the disappearance of files, we can help you solve the problem. Thus enters the Dummy System. The Dummy System is a countermeasure against this problem, which simply inserts a dummy asset into the place of a missing one. In the engine, if a model goes missing, it will be replaced by a cube, if a texture goes missing, it will be replaced by a bright pink checkerboard pattern.
Aside from the visual aid of missing assets, the console will also print warnings, informing exactly which file has gone missing and the path it is looking for so you can investigate quickly. This approach not only helps fix the problem, but also prevents the engine from crashing allowing you to continuing editing other stuff and fix it later whenever you want.
The user interface got a face lift in 0.1.4b and with it came a lot of problems which we have been fixing as fast as possible. While all the nasty engine crashing issues have been resolved, a few components of the UI didn’t get upgraded while these issues were present. Now that everything is working as it should, all components of the main Editor have now been upgraded to the new UI system.
The only remaining UI components that still needs an upgrade is the Animation Manger and the Particle Editor, both of which will be completed in this update series!
On top of all that, we’ve thrown in a lot of under the hood improvements for general usage. Searching for a model in your project data files now remains in the last directory used, and the default max render range is 250m instead of the way too small 80m. (In future we’ll be introducing an options menu where you’ll be able to control all the default values for every aspect of the engine, as well as changing shortcuts etc.) The project updating system has also been completely re-written removing around 400 lines of redundant code that was dragging things down.
- Fixed bug where some computers were having compatibility problems.
- Fixed bug where terrain had strange artifacts forming around the edges of texture layers due to improper blending.
- Fixed bug where using the flatten interpolated tool with the circular brush along one of the edges of the terrain caused a crash.
- Fixed bug where a placed model would sometimes cause a crash due to incorrect data being saved on placement.
- Fixed bug where creating a new set of sky textures from a HDRi when no other cubemaps exist in the project caused a crash.
- Fixed bug where the cubemap manager isn't recognizing whether the name of the cubemap had been changed causing an endless loop of error message until it was changed to something else.
- Fixed bug where specular IBL was not working correctly and giving incorrect results.
- Fixed bug where uniform scaling was being applied when the checkbox was not.
- Fixed bug importing models with multiple meshes which weren't lod marked caused a crash.
- Fixed bug where models with multiple materials caused multiple bounding boxes to appear when the object was selected.
- Fixed bug where thumbnail textures generated by the engine were being stored in memory even when they aren't being used (memory leak).
- Fixed bug where Instancing Cores were being regenerated when any object was being moved rather than just 3D models.
- Fixed bug where pressing the ESC key with a model selected to deselect, didn't correctly clear out the memory causing lingering and eventual crash (memory leak).
- Fixed bug where the Scale and Rotate 3D cursors sometimes had the wrong rotation.
- Fixed bug where some of the data from DDS textures was not being correctly removed from the memory (memory leak).
- Fixed bug where a blank cubemap was being loaded each time a zone was loaded for no reason resulting in a harmless but annoying error in the output window.
- Fixed bug where removing World Room Layers didn't correctly clear the data from the memory (memory leak).
- Fixed bug where removing all the Fog layers caused the old fog data to linger in the GPU (memory leak).
- Fixed bug where Sculpting or Painting near the edge of a terrain Object didn't fill correctly leaving behind a cross shape of edited terrain.
- Fixed bug where changing the Face culling mode caused SSAO to break and created massive visual artifacts.
- Fixed bug where selecting a custom shader for a model caused a crash.
- Fixed bug where changing a material texture didn't save unless the object was also moved in the Scene.
- Fixed bug where objects which were duplicated weren't loading the right textures if they had different materials.
- Fixed bug where changing a 3D model Object's texture in its material sometimes wouldn't update correctly.
- Fixed bug where when the Editor created a thumbnail texture for the UI (if non standard format) didn't correctly remove the data from memory (memory leak).
- Fixed bug where switching to a different zone while having an object selected, did not clear that objects data from the memory and would lead to a crash if any edit attempts were made.
- Fixed bug where selecting a terrain size didn't actually apply and would result in a crash if the terrain was attempted to be made.
This has been our biggest Performance and Optimization Patch to date, and yet it’s nothing in comparison to what is coming. With general usage now much less clunky and loading times back to a reasonable level, it’s time to bring visuals on par. Lighting is being upgraded, and quite drastically. Over the next few weeks, we’ll be revamping shadows to make them more memory efficient, introducing Subsurface scattering for realistic skin and foliage, Area lighting, volumetrics, and a whole new suite of post processing options!