Levels of Detail for Large Top-View Terrain

[najak]

Thank you for providing the world with this awesome Gaia Pro 2021 product!
==

Our game is a Civilization-like top view of a huge terrain, but allows the player to zoom into ground view, to enjoy the scene up close.  This means that as you zoom out quickly - we'll need LOD's for each terrain (e.g. each 2x2 LOD0 tile set would combine into 1 LOD1 tile, thus cutting resolution in half along both XZ-axes).   We'd need this all the way out to LOD X (e.g. 4).

We are experienced hobbyists with big ideas and a decent budget, and have a veteran/expert C# programmer on our team, so we can augment most of the run-time streaming/loading logic ourselves (as well as the FP fixes for when you zoom to LOD0 at locations that are far from the 0,0,0 origin - for all other scene content).   And so we're not expecting Gaia to help us with the run-time streaming, as our needs are likely unique/specialized.   We need Gaia to "produce the LOD0 terrain + terrain-content".

What we need from Gaia (or some system) is a way to programmatically create X LOD's from our original LOD0 (highest detail) terrain.

Our Game's default full-world view is LOD4 - Mouse-wheel-forward will zoom you from Top-God-View down to LOD 0 - ground view.   And vice-versa.  The transition needs to be without hiccups, and minimum "popping" of terrain resolution.    Each LOD must look "near equivalent" to the next LOD level, so that the transitions look reasonably smooth.

For higher powered player PC's, we'll push these LOD transitions further out (to reduce the visual popping effects of LOD transitions).

Additionally, for transitions, we're like to execute a 0.15 second Fade animation, as we fade out the old LOD and fade in the new LOD.  This further reduces the popping effects.

So our plan (hopefully with Gaia) is this:
1. Build out the HUGE terrain using 512x512 tiles, so that the huge terrain is a 64x64 grid of these tiles.  (1 Gig height values - compressed)
2. Run this through a custom/scripted build process that creates 4 more LOD's.  (1-4)

     LOD0 would be 64x64 grid of tiles. - what you see at Ground Level, looking horizontally, only showing 4x4 tiles at once, approx.
     LOD1 would be  32x32 grid of tiles. -- only showing 4x4 tiles at once, approx.
     LOD2 is 16x16 of grid tiles.
     LOD3 8x8
     LOD4 4x4 -- so when fully zoomed out -- this would be the full map view.
     LOD6 1x1 of 512x512 -- if it looks good, would be used for our Mini-Map.

===
Our burning, most vital questions are:

1. How do we create our LOD's of the Gaia terrain outputs such that it achieves the required goal of "each LOD looks like the lower-resolution equivalent of the next LOD"???

2.  Can we reasonably expect that Gaia will be able to produce this 64x64 (mostly procedural) grid of terrains?  Or is this workload too much for Gaia/Unity?   (if so, we can find our own way to break it down, and enforce transitions (same elevations along the seams between larger segments).

3. Which version of Unity should we be using to avoid those horrific "flickering shadows" (Unity defect) that we see in many of your recent videos? (they even occurred in Play Mode in your videos)


NOTES:
If we choose to go the Gaia route -- our plan is to sign up for the "Professional subscription" here, after dropping $300/seat onto the Gaia Pro 2021 package.

For our game, the Terrain/content in Unity is "purely visual" - our game runs mostly headless just the same, without Unity at all.   Unity is our plugin for UI and beauty/visualization - and since we *can*, our goal is to treat our users to a very rich/beautiful view of their game world, thanks to Gaia and Unity.

After completing this LOD-based terrain system, our plan would be to also market it on Unity Asset store, alongside your Gaia product as an "add-on", to better enable Gaia to be used for Top-View to Ground-View world maps,


And here's the pure Visual3D.NET- pure C# (top to bottom) game engine that we developed back in 2009, but we were put out of business by Unity3D (damn them).   Terrains was our strong-suit (that's me narrating the video):
 

 

[najak]

Our Visual3D terrains were built-specifically for transitioning from near-full-earth-sized terrains all the way down to 0.5 meter resolution -- via a highly efficient database of heights/landcover.   We stopped development of it in 2010 -- and so is grossly outdated, and only runs on DX/XNA!....  So it's not an option for us now.      Thus -- we are now looking to Unity/Gaia! 🙂

[najak]

I'm a Pro-subscriber now.   In some capacity I will be using PW tools/assets, so I dove in.

The main thing I'm hoping to use Gaia for is the terrain geometry/splitting.    In the Gaia PkgMgr, "Create World" tab, when I select "Tile Size 8192 SqMeters" - it gives me an encouraging warning :

"Please consider making multiple terrains...  The terrain mesh exporter in Gaia Pro can also create IMPOSTOR TERRAINS for you..."

YES, this is exactly what I need.   I want to take each terrain tile and create "Impostors" (lower resolution mesh terrains that simply appear as a lower-resolution version of the real terrain).    Does Gaia really create IMPOSTOR TERRAINS?   If so, please direct me to the documentation, or instruct me on how this can be done (hopefully in a scripted/programmatic pipeline).

I'm trying to generate a POC of our pipeline now, and do some performance/quality testing to help determine our limits and set our expectations for poly/object counts, etc.

 

[najak]

Note, I'm starting to see from usage/experiments, that perhaps Gaia might not be equipped to handle our needs, directly.

Please let me know what you think the Maximum size of terrain I can hope to generate with Gaia?   For sake of faster loading (without hiccups), we'll likely break things down into either 256 size tiles, or 512.     We were hoping for a maximum size of 32Kx32K total height samples (Split into a 64x64 grid of 512x512 tiles).   Clearly this is way over the limit for what Gaia can provide (I'm assuming).   So it would be nice if someone could help establish the "realistic" max size that Gaia can handle at once.   And from this estimate, we can do our planning.

Creating very-large world terrain seems to be a common theme we see in forums, and appears to have no current good solutions on the market to solve this yet.   It appears to me that Gaia has a good foundation, and might be evolvable into a solution that works better for much larger terrains, which should also include the ability to generate LODs, so that the whole lower-rez terrain can be viewed from a distance, all at once.

[najak]

I'd sure like to hear something from the Gaia folk.  I'm trying to get your input, so that we can put things into perspective, so that we can do some top-level planning and set our expectations upfront.

From what we can tell so far, is that Gaia has the best toolset for generating artistic high-quality terrain meshes.   We opt to use other tools for splatting/painting, biomes, and vegetation - depending upon how Gaia stacks up per our specific needs.

One specific specialized need that we have is that we need "run-time" (on the client devices) generation/assembly of terrains, of some sort  (we can have predefined tiles, but then those tiles need to be stitched cleanly without noticeable seams).

[Peter]

Hi @najak, please excuse the delayed response, we were stretched very thin across the weekend due to the current unity sale.  Regarding your most vital questions:
 

Quote

1. How do we create our LOD's of the Gaia terrain outputs such that it achieves the required goal of "each LOD looks like the lower-resolution equivalent of the next LOD"???

As mentioned Gaia has a terrain mesh exporter that can convert an unity terrain object into a mesh, but also can create impostor terrains with multiple LODs. These are however always at the same size as the original terrain they are derived from, this kind of "collapsing LODs" that you describe where multiple impostors are grouped together into a single object in the next LOD level would not work. This is something that you would need to custom build then.

For a quick evaluation of how those impostors are being generated, how they would look like etc. I would suggest the following approach: Open the Gaia Manager in a new scene and generate a world with the following settings:



This would ensure you have a small scene with terrain loading generated by the world designer. When you click "Create World Designer, the world designer opens. 

Check the box for "Always full preview", then click the "Randomize all" button a few times until you see a terrain shape you like and click "Generate World"



This will create a 4x4 sized world based on the shape you saw in the preview. (You can increase the loading range for terrains in the scene view from the top panel)



Now from the advanced tab in the Gaia Manager you can open the terrain mesh exporter:



Select the "04 Create impostors" preset for the mesh export (if you are curious what settings are available you can dive deeper with the small "+" button, but for a first evaluation the preset should do. Click on "Start Export" to generate the impostors.




This will take a couple of minutes as Gaia creates 3 mesh terrains for each of the original terrains for the impostors. After the process is finished, you can create the runtime from the Gaia Manager with the following settings:



Then you can run the scene and fly away from / towards the terrain to see how the real / impostor terrains behave visually. As you fly further away you should see the switch from unity terrain to impostor if you look closely.



It is possible to make out a difference in color between the impostor and the real terrain - for the more common use cases where the player is on the terrain this is neglectible especially if the terrain is full of trees etc., for your use case it might not be sufficient enough.

 

On 4/8/2022 at 9:26 PM, najak said:

2.  Can we reasonably expect that Gaia will be able to produce this 64x64 (mostly procedural) grid of terrains?  Or is this workload too much for Gaia/Unity? 

Yes, this should be possible, you could test this relatively easily by extending the world size from the example above to 64 by 64. Generation will take longer, but it should work in general. You will need to find a balance for the visibility of the real terrains & impostors, every  unity terrain object in the scene creates a noticeable performance overhead.
We did get reports of users that run into issues with the terrain mesh exporter and large worlds where the impostor generation crashed due to out of memory errors - we do free up memory where we can, but from past experience unity does not free up memory well from a single, uninterrupted process, we need to move the impostor generation into a coroutine to make it more robust in that regard.
 

On 4/8/2022 at 9:26 PM, najak said:

3. Which version of Unity should we be using to avoid those horrific "flickering shadows" (Unity defect) that we see in many of your recent videos? (they even occurred in Play Mode in your videos)

I do know that this happens in the early Unity 2020.3 versions in built-in rendering, if you avoid those you should be fine.

 

4 hours ago, najak said:

One specific specialized need that we have is that we need "run-time" (on the client devices) generation/assembly of terrains, of some sort  (we can have predefined tiles, but then those tiles need to be stitched cleanly without noticeable seams).

This might be a problem: The terrain generation from Gaia does only work in editor. Gaia uses a lot of "Editor only" code that is not available in a standalone build. The predefined tiles might work.

[najak]

Peter, thank you for the comprehensive responses to our concerns.  I'm glad to see Gaia producing Impostor terrains that very similar to the real terrain (including the paint).

What we may do is use Gaia to create a selection of 100's of "tiles" that we package with our game (such as 20 versions of "mountains" tile to choose from, and then 20 versions of each other tile type as well); and then we'll have our own on-client stitching logic to eliminate visible seams when they assemble these onto the Hex Map.

Then for those who want to "reskin the map", they can use Unity + Gaia (buy it from you) to create their own versions of tiles that they want to see.  Our game's actual logic doesn't care what the terrain is showing - although preferably it resembles the named "tile type" (e.g. Mountains, vs. Hills vs. Plains vs. Swamp vs. Ocean).