The Oxford Dictionary defines “lightmapping” as “A process with takes an absolutely ludicrous amount of time“. And if you thought lightmapping in Unity 4 took long, then you’re in for a shock when you try 5.
If you’ve not used Unity before, lightmapping is basically a way to pre-render lighting and shadow data at design-time instead of wasting processing power at run-time. In games, you often get lights (and therefore shadows) that don’t ever move, which leads to the poor CPU having to constantly work out the lighting data every frame, even though it doesn’t need to. The answer is to bake the lighting into a lightmap. That lightmap is just a big image that essentially gets overlaid on your objects and makes it seem like there is lighting in the scene, even though there isn’t.
The problem is that this baking process can take anywhere from a few minutes (for a very small scene) to hours, depending on your scene/level and PC. So I decided to test out if I could do the processing on Azure instead. To do this I used a single reference level and baked it 8 times on each VM/PC I setup. All the settings were the same, and I created a new instance for each one (didn’t just reuse an older installation). Along with just baking, I ran some benchmarks with PassMark PerformanceTest.
I did all the tests on the following machines:
Azure A10: 8 Core Xeon E5-2670, 56 GB RAM.
Azure A11: 16 Core Xeon E5-2670, 112 GB RAM.
Azure G3: 8 Core Xeon E5-2698B v3, 112 GB RAM.
Azure G5: 32 Core Xeon E5-2698B v3, 448 GB RAM.
Surface Pro 3: i7 4650U, 8 GB RAM (256 GB storage model).
Other PC: i7 4770K, 16 GB RAM, 256 GB OCZ Vertex 4 SSD (this is my office machine).
All tests were done with Windows 8.1, Unity 5.0.1f1 and PassMark V8 build 1047.
Before showing some charts, let me summarize my observations (summaries go at the beginning right?). These Azure machines can be really, really powerful, but also really, really expensive. If you’re paying for this sort of hardware, you want to use it all. For the particular use of lightmapping it simply isn’t worth the cost. You’ve got tons of RAM and storage which Unity never touches, plus top-grade hardware at every point in the system.
If all you want to do is bake some lights, and time isn’t a major concern, just build a fast PC out of a bunch of dodgy parts – you simply don’t need the fancy stuff that Azure provides.
What I’d really like is for Azure to add a custom config where I can specify that I want lots of CPU power, but hardly any RAM or HDD space.
Don’t read too much into these values. Although I ran every test multiple times, ensured I was getting consistent results, and did everything I could to test properly, something might have gone wrong without me realizing it.
All of the standard benchmark scores (CPU, Memory, Disk) are higher is better.
Something that interested me with the CPU tests was that the G3 was “slower” than the A10. I did re-run the test a few times to make sure.
Once again, the G3 is slightly below even the A machines here. I’m not sure why, and I would have assumed that all the Azure results would roughly align (which they sort of do besides the G3).
For some reason, even though all these Azure machines have SSD’s, they seem to be pretty throttled. The G3 consistently scored WAY higher than anything else. Maybe it was just a glitch in the matrix, and Azure forgot to throttle my G3 instance. Or maybe they were throttling all my others too much. Either way, the slow HDD speed was noticeable. It took a long time to install and unzip stuff, and the CPUs definitely weren’t the issue.
The time difference between my PC and the G5 is roughly 15 minutes, which makes the G5 double the speed. 15 (or 30) minutes might not seem like a lot, but consider that this was a small reference level. Then also take into account that a game could easily have 30 levels. On my machine that is 15 hours, and doesn’t take into account that you’ll need to bake each level lots of times. And while it is baking the PC is often not useable at all.
This may or may not actually be a useful chart. Either way, it interested me because it portrays the scenario of buying/renting a dedicated machine that does nothing but bake lighting all day.
Fun fact: If you were buying all this hardware, you could buy about 18 of the CPUs in my machine for the same price as the ones in the G5, even though it only bakes double the speed. Obviously, this is just one very specific application which almost definitely doesn’t totally use the power of the G5.
Finally, cost. I couldn’t include my PC or SP3 because I have no “per hour” dollar value for them.
The A10 is clearly the best value per bake but assumes that time doesn’t matter.