Simplygon Blog

Bake textures into vertex colors

2022-05-16

Written by Jesper Tingvall, Product Expert, Simplygon

Disclaimer: The code in this post is written using version 9.2.7000.0 of Simplygon and Unity 2021.3.0f1. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

In this post we are going to bake textures into vertex colors. This is useful for LODs which are really far away, where we just want some color. In order to do this, we need to disable sRBG on all texture nodes in the scene. So we'll cover how to traverse shader network as well.

We are going to use this brightly colored container asset for this post.

Stack of 4 differently colored containers.

Prerequisites

This example will use the Simplygon integration in Unity, but the same concepts can be applied to all other integrations of the Simplygon API.

Problem to solve

We want a LOD that uses vertex colors for coloring. Reason for this can be that we want to avoid loading any texture into memory and can skip texture sampling in our shader.

To bake data from textures into vertex colors we can use a Vertex color caster. We can do a first try via the user interface and use the settings below.

LOD of game character with preserved grey and white vertex colors.

After processing we discover a problem however. The colors are a wrong. Reason is that we have sampled the color texture with sRGB flag on.

LOD of game character with preserved grey and white vertex colors.

Solution

The solution is to not read the textures as sRBG. To do this we need to manipulate the shading network on our exported scene. To do this we need to perform the LOD generation through scripting.

Find all texture nodes in a shading network

First we are going to create a helper function for finding all texture nodes.

/// <summary>
/// Recursively get all texture nodes in a shading network.
/// </summary>
private static void GetTextureNodes(spShadingNode node, List<spShadingTextureNode> textureNodes)

We can detect if the node is a spShadingTextureNode by doing a SafeCast to it. After casting we can see if it was successfully casted by checking IsNull. It will be false if it was casted correctly.

var nodeAsTexture = spShadingTextureNode.SafeCast(node);
if (!nodeAsTexture.IsNull()) // We need to use IsNull instead of  != null to check if it was a valid cast.
{
    textureNodes.Add(nodeAsTexture);
}

If we are not in a texture node we check if we are in a spShadingFilterNode. This is the parent class to all nodes which can have other nodes as input. We can use GetParameterCount to find how many inputs it has and GetParameterIsInputable to detect if a specific input can be other nodes. Then we can get the next nodes to traverse via GetInput.

var nodeAsFilter = spShadingFilterNode.SafeCast(node); // Filter node class is the parent to all nodes with other nodes under it
if (!nodeAsFilter.IsNull())
{
    for (int j = 0; j < nodeAsFilter.GetParameterCount(); j++)
    {
        if (nodeAsFilter.GetParameterIsInputable(j)) // Parameter can be other nodes
        {
            GetTextureNodes(nodeAsFilter.GetInput(j), textureNodes);
        }
    }
}

Disable sRGB flags

When we have found every texture node in our material we can disable sRGB using SetUseSRGB.

/// <summary>
/// Set sRGB flag on all texture nodes on material for specific channel.
/// </summary>
private static void SetUseSRGB(spMaterial material, string channel, bool sRGB)
{
    var shadingNetwork = material.GetShadingNetwork(channel);
    List<spShadingTextureNode> textureNodes = new List<spShadingTextureNode>();
    GetTextureNodes(shadingNetwork, textureNodes);
    foreach (var node in textureNodes)
    {
        node.SetUseSRGB(sRGB);
    }
}

Remeshing pipeline

To process our LOD we are going to use a RemeshingPipeline. To this we add a VertexColorCaster with Unity specific casting settings.

Since the corners of our asset is sharp we set SetHardEdgeAngles to 45 to ensure those are kept. This means any angle above 45 will be counted as a hard edge, below it will be a smooth edge.

// Unity specific settings
const string UNITY_COLOR_CHANNEL_NAME = "diffuseColor";
const string UNITY_VERTEX_COLOR_NAME = "color";

using (var remeshingPipeline = simplygon.CreateRemeshingPipeline())
using (var remeshingSettings = remeshingPipeline.GetRemeshingSettings())
using (var vertexColorCaster = simplygon.CreateVertexColorCaster())
using (var vertexColorCasterSettings = vertexColorCaster.GetVertexColorCasterSettings())
using (var materialTable = sgScene.GetMaterialTable())
using (var mappingImageSettings = remeshingPipeline.GetMappingImageSettings())
{
    // Set up remeshing pipeline settings
    remeshingSettings.SetOnScreenSize(SCREEN_SIZE);

    // Ensure we get sharp angles on our containers.
    remeshingSettings.SetHardEdgeAngle(45);
		
		// Need to create a mapping image for us to transfer colors.
    mappingImageSettings.SetGenerateMappingImage(true);

    // Add a vertex color caster with Unity specific settings.
    vertexColorCasterSettings.SetMaterialChannel(UNITY_COLOR_CHANNEL_NAME); // Input albedo color channel specific to Unity
    vertexColorCasterSettings.SetOutputColorName(UNITY_VERTEX_COLOR_NAME); // Output vertex color channel
    remeshingPipeline.AddMaterialCaster(vertexColorCaster, 0);

Before running the scene we disable sRGB on all materials using the helper functions we created above.

// For every material in scene disable sRGB on all albedo channels.
for (int i = 0; i < materialTable.GetMaterialsCount(); i++)
{
    using (var material = materialTable.GetMaterial(i))
    {
        SetUseSRGB(material, UNITY_COLOR_CHANNEL_NAME, false);
    }
}

// Run processing
remeshingPipeline.RunScene(sgScene, EPipelineRunMode.RunInThisProcess);

After processing and importing the model looks like this. It is optimized enough, but lacks color.

Four white lowpoly containers.

Vertex color shader

Unity's standard shader does not use display vertex colors. To achieve this we are going to write a simple shader that showcase them.

Shader "Simplygon/VertexColored"
{
	SubShader
	{
		Tags { "RenderType" = "Opaque" }
		LOD 200

		CGPROGRAM
		#pragma surface surf Standard fullforwardshadows
		#pragma target 3.0

		struct Input
		{
			float4 color : Color; // Vertex color
		};

		void surf(Input IN, inout SurfaceOutputStandard o)
		{
			o.Albedo = IN.color.rgb; // User vertex color as color
		}
		ENDCG
	}
	FallBack "Diffuse"
}

With the shader in place the optimized mesh looks like this. The colors are displayed correctly.

Original container asset and processed lowpoly container asset with correct colors.

Fixing smeared vertex colors

While the colors are displayed correctly we get lots of smearing between them. Using SetColorSpaceEdgeThreshold we can control when we should get a hard or smooth color edge. This is similar to how SetHardEdgeAngle which we covered above works. A color difference above treshold will give us a hard color edge, below will give us smooth colors.

Different values for Color space edge treshold gives different vertex color blending.

The lowest value of 0.0 gives noticable hard color edges at places we do not want. A value of 0.025 is suitable for our purpose. Our script is thus updated with following setting.

// Fix vertex color smearing
vertexColorCasterSettings.SetColorSpaceEdgeThreshold(0.025f);

Result

Up close it is easy to see the difference between the optimized model and original. However from far away it is hard. Can you see any difference in image below?

6 containers viewed from far away.

Complete scripts

Remeshing script

// Copyright (c) Microsoft Corporation. 
// Licensed under the MIT license. 

using Simplygon.Unity.EditorPlugin;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

namespace Simplygon.Examples.UnityVertexColorLOD.Editor
{
    /// <summary>
    /// Creates a remeshed object colored using vertex colors.
    /// </summary>
    public class VertexColoredLODCreator
    {
        const int SCREEN_SIZE = 60;
        const string OUTPUT_FOLDER = "LOD";

        // Unity specific settings
        const string UNITY_COLOR_CHANNEL_NAME = "diffuseColor";
        const string UNITY_VERTEX_COLOR_NAME = "color";

        /// <summary>
        /// Entry point from menu.
        /// </summary>
        [MenuItem("Simplygon/Create vertex colored LOD")]
        static void CreateVertexColoredLODEntryPoint()
        {
            if (Selection.objects.Length > 0)
            {
                using (ISimplygon simplygon = Loader.InitSimplygon(out EErrorCodes simplygonErrorCode, out string simplygonErrorMessage))
                {
                    if (simplygonErrorCode == EErrorCodes.NoError)
                    {
                        foreach (var o in Selection.objects)
                        {
                            CreateVertexColoredLOD(o as GameObject, simplygon);
                        }
                    }
                    else
                    {
                        Debug.Log("Initializing Simplygon failed: " + simplygonErrorCode);
                    }
                }
            }
        }

        /// <summary>
        /// Create a vertex colored LOD and import it into scene.
        /// </summary>
        public static void CreateVertexColoredLOD(GameObject gameObject, ISimplygon simplygon)
        {
            var objectsToProcess = new List<GameObject>() { gameObject };

            var exportTempDirectory = SimplygonUtils.GetNewTempDirectory();

            using (var sgScene = SimplygonExporter.Export(simplygon, exportTempDirectory, objectsToProcess))
            {
                using (var remeshingPipeline = simplygon.CreateRemeshingPipeline())
                using (var remeshingSettings = remeshingPipeline.GetRemeshingSettings())
                using (var vertexColorCaster = simplygon.CreateVertexColorCaster())
                using (var vertexColorCasterSettings = vertexColorCaster.GetVertexColorCasterSettings())
                using (var materialTable = sgScene.GetMaterialTable())
                using (var mappingImageSettings = remeshingPipeline.GetMappingImageSettings())
                {
                    // Set up remeshing pipeline settings
                    remeshingSettings.SetOnScreenSize(SCREEN_SIZE);

                    // Ensure we get sharp angles on our containers.
                    remeshingSettings.SetHardEdgeAngle(45);

                    // Need to create a mapping image for us to transfer colors.
                    mappingImageSettings.SetGenerateMappingImage(true);

                    // Add a vertex color caster with Unity specific settings.
                    vertexColorCasterSettings.SetMaterialChannel(UNITY_COLOR_CHANNEL_NAME); // Input albedo color channel specific to Unity
                    vertexColorCasterSettings.SetOutputColorName(UNITY_VERTEX_COLOR_NAME); // Output vertex color channel

                    // Fix vertex color smearing
                    vertexColorCasterSettings.SetColorSpaceEdgeThreshold(0.025f);

                    remeshingPipeline.AddMaterialCaster(vertexColorCaster, 0);

                    // For every material in scene disable sRGB on all albedo channels.
                    for (int i = 0; i < materialTable.GetMaterialsCount(); i++)
                    {
                        using (var material = materialTable.GetMaterial(i))
                        {
                            SetUseSRGB(material, UNITY_COLOR_CHANNEL_NAME, false);
                        }
                    }

                    // Run processing
                    remeshingPipeline.RunScene(sgScene, EPipelineRunMode.RunInThisProcess);

                    // Handle output folder
                    var baseFolder = "Assets/" + OUTPUT_FOLDER;
                    if (!AssetDatabase.IsValidFolder(baseFolder))
                    {
                        AssetDatabase.CreateFolder("Assets", OUTPUT_FOLDER);
                    }

                    var assetFolderGuid = AssetDatabase.CreateFolder(baseFolder, gameObject.name);
                    var assetFolderPath = AssetDatabase.GUIDToAssetPath(assetFolderGuid);

                    int startingLodIndex = 0;
                    SimplygonImporter.Import(simplygon, remeshingPipeline, ref startingLodIndex, assetFolderPath, gameObject.name);
                }
            }
        }

        /// <summary>
        /// Set sRGB flag on all texture nodes on material for specific channel.
        /// </summary>
        private static void SetUseSRGB(spMaterial material, string channel, bool sRGB)
        {
            var shadingNetwork = material.GetShadingNetwork(channel);
            List<spShadingTextureNode> textureNodes = new List<spShadingTextureNode>();
            GetTextureNodes(shadingNetwork, textureNodes);
            foreach (var node in textureNodes)
            {
                node.SetUseSRGB(sRGB);
            }
        }

        /// <summary>
        /// Recursively get all texture nodes in a shading network.
        /// </summary>
        private static void GetTextureNodes(spShadingNode node, List<spShadingTextureNode> textureNodes)
        {
            var nodeAsTexture = spShadingTextureNode.SafeCast(node);
            if (!nodeAsTexture.IsNull()) // We need to use IsNull instead of  != null to check if it was a valid cast.
            {
                textureNodes.Add(nodeAsTexture);
            }
            else
            {
                var nodeAsFilter = spShadingFilterNode.SafeCast(node); // Filter node class is the parent to all nodes with other nodes under it
                if (!nodeAsFilter.IsNull())
                {
                    for (int j = 0; j < nodeAsFilter.GetParameterCount(); j++)
                    {
                        if (nodeAsFilter.GetParameterIsInputable(j)) // Parameter can be other nodes
                        {
                            GetTextureNodes(nodeAsFilter.GetInput(j), textureNodes);
                        }
                    }
                }
            }
        }
    }
}

Vertex color Unity shader

Shader "Simplygon/VertexColored"
{
	SubShader
	{
		Tags { "RenderType" = "Opaque" }
		LOD 200

		CGPROGRAM
		#pragma surface surf Standard fullforwardshadows
		#pragma target 3.0

		struct Input
		{
			float4 color : Color; // Vertex color
		};

		void surf(Input IN, inout SurfaceOutputStandard o)
		{
			o.Albedo = IN.color.rgb; // User vertex color as color
		}
		ENDCG
	}
	FallBack "Diffuse"
}

⇐ Back to all posts

Request 30-days free evaluation license

Your name *

Your email address (company domain required) *

Company name *

Link to your company website *

Industry

PC, console, mobile games development

VR, AR experiences

CAD engineering

Other

Describe your use case *

Book meeting with technical expert

I accept evaluation license terms

Submit request