Scenario
Previous engines notorious for undergoing incremental improvements over a long period of time and becoming redundant in technology and design.
While the learning experience is good, it is not a good idea for these experiences to reach commercial production as they have done in past iterations.
The problems that Gen 4 face need to be analysed and detailed so steps can be made to improve the engine and create a more robust development environment that meets the new requirements that the engine is now facing.
The Gen 4 engine has now gone beyond the scope of just Battle World and is being used in commercial products on a variety of platforms, this was not the original intention and problems have already arisen around the engine's first projected development plan and the current use it has.
History
- Gen 1 – High level software graphics, single threaded, Lua script, no modularity
- Gen 2 – High level hardware graphics, single threaded, Lua script, no modularity
- Gen 3 – High level hardware graphics, some threading support, Angel script, high modularity, networked
- Gen 4 – Hardware graphics, some threading support, Ruby script, medium modularity
It is a bad idea to create an engine when you are learning especially now that engine technology has evolved so much.
I wrote an analysis paper for creating a game engine, which won the award for project of the year for my university, the focus of the paper was the mistakes made during engine Gen 3's production and the lessons learnt from that contributed to Gen 4, the current working engine technology that is now powering high-performance 3D mobile applications.
The biggest problem identified with the previous engine developments is the dependency chain which hampered portability, code-health, maintainability and legibility, this is a problem frequently found with high level engines, removing the dependencies for Gen 4 has greatly helped it in most aspects.
Generation 4 Problems
Memory allocation was identified as a bottleneck during Gen 3's analysis with a recommendation to use custom memory allocators with fixed limits to improve performance at the cost of safety, however this was not adopted by Gen 4 as additional research into allocator implementation is required.
At this moment, C is the most appropriate language for custom memory allocation and research in this language is mostly complete, the inevitable C++ implementation has not been started and has had no research.
Too much work is placed on the Gen 4 main thread; threading paradigms were analysed with a conclusive paradigm given in the Gen 3 report, however this was not implemented as part of the Gen 4 graphics library (
Hardened Edge) until performance problems were becoming apparent on mobile devices.
There are also problems with header clashing where different operating system's internal headers are being included with engine API headers, causing reservations with common function names (AddJob on the Win32 API is a frequent headache), in addition to this the mobile/desktop graphics APIs are frequently split with pre-processor definitions, there is no layer between the graphics API and the graphics engine.
Scripting environment is not implemented due to current project plans. A scripting environment is pretty much needed in modern development for rapid iteration and quick content creation.
Texture generation is the slowest area of graphics API interaction, the Gen 4 texture loaders are designed to create textures on the GPU as soon as the texture is read, this is a mistake as it means there is a GPU lock and the use of the graphics API restricts this to the main thread, causing a massive slowdown for this area.
Future projects
Generation 4 showcased stereo 3D rendering with VR development in mind and the high portability led to mobile applications being the main focus, both VR and mobile applications require high frame-rates for the best end-user experience and particularly on mobile application load-time is expected to be minimal.
This engine is now too developed for a clean re-factoring and at this point it is now a better idea to reimplement with VR and low-power devices in mind.
The current outlook for devices is more threads, more cores, more physical processors, more memory, but not faster clocks, it is now even more crucial to think about clever methods of keeping a high frame-rate rather than optimising functions and expecting them to run on a future generation of faster processors.
The mobile API in particular need to be more independent and less reliant on the mobile operating systems which hinder performance and invade the application run-time more often than necessary (Android comes to mind here).
Shaders are per-material and not per-object and are difficult to interact with from application.
Improvements
The graphics API must run on the thread of the spawning context with the scene objects managed in dedicated memory space and pushing calls for the graphics API safely from their managed thread.
Texture streaming is now worth it, the conventional method of generating GPU textures presumes that the texture is already loaded and ready to go onto the GPU, to reduce load times it is now better to load textures as they are needed during run-time and load them directly into memory that is planned to be mapped onto GPU memory.
This also allows for streaming video data or other external texture data as the frame is readied, ideal for projects that require this.
2D rendering should be explored. At the moment this is largely unexplored but is increasingly more often required. Gen 2 utilised a dedicated 2D buffer for UI elements, something similar could be useful, but ultimately a new UI API will be needed.
A lot of potential for a bespoke, independent UI API is around the corner, so this could be implemented as a new module entirely.
Font rendering is important for 2D rendering.
Rendering to any surface is very useful, allows in-game UI controls (Doom 3 style interactive monitors)
Shader binding class, called when specific shaders are bound with a pointer to read-only memory. The memory is written to by any point that wishes to modify shader programs through the binding class.
Stepping away from fixed-named shader variable is important, however the GLSL version differences are a problem between mobile and desktop, it could be worth introducing a second language that generates the required GLSL code, perhaps the scripting language can be used for this?
Shaders written in Ruby would be fantastic.
An engine console and networked interface aligns well with future project plans, particularly with Battle World's unique use of networking systems.
The protocol should be compatible with both thread communications and networked communications depending on the context; additional research is required here as well as IPv6 testing for IP networking.
Conclusion
The growth of Gen 4 should be locked down with immediate effect until the first set of products have been released.
By locking down the engine, modules that were originally designed within Battle World's scope can be jettisoned for development as individual components with the intention of running them on Gen 4 and future engine builds.
Unfortunately this looks like another failure for Battle World but a great success in testing the new development practises and designs suggested by my Gen 3 report. The differences between Gen 3 and Gen 4 are worlds apart, so any future changes will now appear minimal in contrast, which indicates that things are going right.
There is also an opportunity to start developing engine tools that are in-engine as this technology is now at hand with the new resources accessible to Gen 4, this is an opportune moment to expand licensing of the graphics engine to cover clientèle usage and no longer keep the engine as an internal-tool (The demand for this is also high at this moment).