Disclaimer: The code in this post is written using version 9.1.36100 of Simplygon. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

Using the remeshing processor is a great way of optimizing a complex asset into something that can be used on a mobile platform or on the web. The remeshing processor will combine all input meshes into a new mesh and you can transfer all input materials into a combined material using material casters.

But what if you instead want to apply your original materials to the remeshed asset and even be able to change the materials in runtime?

In this post we’ll demonstrate exactly that where we apply original materials to the remeshed asset in runtime using geometry data casters.

Prerequisites

This example will use the Simplygon integration in Unity, but the same concepts can be applied to all game engines using the Simplygon API.

Problem to solve

We’ll use the following chair as input asset, and we want to be able to change seat and leg materials in runtime.
To cast every material combination to new textures is not an option due to memory constraints so we want to use the original materials instead and change the materials using a custom shader in runtime.

Remeshing

Let us start optimizing the input asset using the remeshing processor. This will reduce the number of triangles and create a single mesh we can render with just one draw call.
Here you can see the difference in complexity between the original and remeshed object.

But if we try to apply the original materials to the remeshed object it will look like garbage. This is because the remeshed object has a completely new set of texcoords and all material ids are lost.

Here you can see the difference in texcoords between the original and the remeshed object.

Geometry data casting

This is where the geometry data caster comes in. Instead of casting regular colors or normals it casts other types of geometry data like texcoords and material ids.
A complete list of supported data types can be found in the geometry data caster documentation.

NOTE

Texture compression like DXT will create artifacts when you sample these textures in a shader so please use an uncompressed texture format.
And to preserve as much detail as possible of the original texcoords we’ll cast the texcoords to a 16bit per channel texture format.

Let us create two geometry data casters, one for texcoords and one for material ids with the following settings.

We can now use these textures in a custom shader to map geometry data from the remeshed asset back to the input asset.

Here you can see a comparison between the original texcoords and the remeshed texcoords using the texcoords texture created by the geometry data caster to map the texcoords back to the original.

Here you can see the original material ids applied to the remeshed object using the material id texture created by the geometry data caster and represented by color values.

Result

With the newly created textures by the geometry data caster, we can combine everything into a custom shader that maps both texcoords and material ids back to the input asset and applies original materials in runtime.

Complete Unity shader

Shader "GeometryDataCaster/RunTimeShader"
{
    Properties
    {
        //Settings to change material variants
        _SeatVariant("Seat variant", Int) = 0
        _LegsVariant("Legs variant", Int) = 0

        //Textures created by the geometry data caster
        _GeometryDataTexCoordTex("TexCoord", 2D) = "white" {}
        _GeometryDataMaterialIdTex("MaterialId", 2D) = "white" {}

        //Original seat textures
        _SeatTex1 ("Seat variant 1", 2D) = "white" {}
        _SeatTex2 ("Seat variant 2", 2D) = "white" {}

        //Original legs textures
        _LegsTex1("Legs variant 1", 2D) = "white" {}
        _LegsTex2("Legs variant 2", 2D) = "white" {}
    }
    SubShader
    {
        Tags { "RenderType"="Opaque" }
        LOD 200

        CGPROGRAM
        // Physically based Standard lighting model, and enable shadows on all light types
        #pragma surface surf Standard fullforwardshadows

        // Use shader model 3.0 target, to get nicer looking lighting
        #pragma target 3.0

        half _SeatVariant;
        half _LegsVariant;

        sampler2D _GeometryDataTexCoordTex;
        sampler2D _GeometryDataMaterialIdTex;

        sampler2D _SeatTex1;
        sampler2D _SeatTex2;
        sampler2D _LegsTex1;
        sampler2D _LegsTex2;

        struct Input
        {
            float2 uv_GeometryDataTexCoordTex;
            float2 uv_GeometryDataMaterialIdTex;
        };

        // Add instancing support for this shader. You need to check 'Enable Instancing' on materials that use the shader.
        // See https://docs.unity3d.com/Manual/GPUInstancing.html for more information about instancing.
        // #pragma instancing_options assumeuniformscaling
        UNITY_INSTANCING_BUFFER_START(Props)
            // put more per-instance properties here
        UNITY_INSTANCING_BUFFER_END(Props)

        void surf (Input IN, inout SurfaceOutputStandard o)
        {
            //Map texcoords back to original texcoords using geometry data caster texture
            float4 texCoord = tex2D(_GeometryDataTexCoordTex, IN.uv_GeometryDataTexCoordTex);
            //Map material ids back to original material ids using geometry data caster texture
            float materialId = tex2D(_GeometryDataMaterialIdTex, IN.uv_GeometryDataMaterialIdTex).r * 255.0f;

            if (materialId == 0) // Material 0 is seat
            {
                if (_SeatVariant == 0)
                {
                    o.Albedo = tex2D(_SeatTex1, texCoord.xy);
                }
                else
                {
                    o.Albedo = tex2D(_SeatTex2, texCoord.xy);
                }
            }
            else if (materialId > 0) // Material 1 is legs
            {
                if (_LegsVariant == 0)
                {
                    o.Albedo = tex2D(_LegsTex1, texCoord.xy);
                }
                else
                {
                    o.Albedo = tex2D(_LegsTex2, texCoord.xy);
                }
                
            }
            o.Alpha = 1.0f;
        }
        ENDCG
    }
    FallBack "Diffuse"
}