Vector Data Access Optimization

I developed a dynamic data structure to optimize access to mass datasets of polygons and polylines. This structure accelerated terrain rendering by an average of +80%.

In military simulations, mass GIS (Geographic Information System) datasets are a common resource used to represent features like forests, lakes, roads, and rivers through polygons and polylines. These datasets play a vital role in defining the materials in large terrains (exceeding 100x100km) where traditional material baking is storage-intensive and can be impractical. This leads to the need for real-time texture rendering. Despite the use of acceleration data structures (e.g., hash, BVH, and quadtree) to reduce data access costs in terrain rendering, there may still be limitations to their effectiveness.

To address the issue of costly access to bulk GIS datasets for terrain rendering, I developed an intermediate data structure between the terrain shaders and acceleration data structures. This structure significantly streamlines material selection and minimizes access to the acceleration data structures, thereby enhancing performance.

The structure in summary:

• On-demand, we select visible areas and set up the structure using compute shaders;
• Each visible area is represented by a 2D texture.
• Each pixel covers 10x10cm, requires 1 byte, and encodes until 8 features (e.g., forests and rivers);
• The encoded data not only defines the feature(s) but also determines whether access to the acceleration data structure is necessary for more precise data;
• It supports multilayers, allowing pixels from distant areas to cover larger areas, thereby saving memory;
• The structure is computed and released on-demand, optimizing memory consumption;
• Memory is reused to prevent memory allocation spikes and fragmentation;
• No data transfer between RAM and VRAM after the game setup, enhancing efficiency.

Check the demonstration in a terrain with 80x35km.

• Red fragments indicate access to data structures (costly);
• Blue fragments represent my data structure (much cheaper) in use to define the fragment material;
• Observe the toggles and the performance changes.

• Memory Management
• Compute shaders
• Computer graphics
• Shaders
• Real-time
• GPU-Based
• Large-Scale Scenario