Bake textures into vertex colors
Written by Jesper Tingvall, Product Expert, Simplygon
Disclaimer: The code in this post is written using version 9.2.7000.0 of Simplygon and Unity 2021.3.0f1. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.
Introduction
In this post we are going to bake textures into vertex colors. This is useful for LODs which are really far away, where we just want some color. In order to do this, we need to disable sRBG on all texture nodes in the scene. So we'll cover how to traverse shader network as well.
We are going to use this brightly colored container asset for this post.
Prerequisites
This example will use the Simplygon integration in Unity, but the same concepts can be applied to all other integrations of the Simplygon API.
Problem to solve
We want a LOD that uses vertex colors for coloring. Reason for this can be that we want to avoid loading any texture into memory and can skip texture sampling in our shader.
To bake data from textures into vertex colors we can use a Vertex color caster. We can do a first try via the user interface and use the settings below.
After processing we discover a problem however. The colors are a wrong. Reason is that we have sampled the color texture with sRGB flag on.
Solution
The solution is to not read the textures as sRBG. To do this we need to manipulate the shading network on our exported scene. To do this we need to perform the LOD generation through scripting.
Find all texture nodes in a shading network
First we are going to create a helper function for finding all texture nodes.
/// <summary>
/// Recursively get all texture nodes in a shading network.
/// </summary>
private static void GetTextureNodes(spShadingNode node, List<spShadingTextureNode> textureNodes)
We can detect if the node is a spShadingTextureNode by doing a SafeCast to it. After casting we can see if it was successfully casted by checking IsNull. It will be false if it was casted correctly.
var nodeAsTexture = spShadingTextureNode.SafeCast(node);
if (!nodeAsTexture.IsNull()) // We need to use IsNull instead of != null to check if it was a valid cast.
{
textureNodes.Add(nodeAsTexture);
}
If we are not in a texture node we check if we are in a spShadingFilterNode. This is the parent class to all nodes which can have other nodes as input. We can use GetParameterCount to find how many inputs it has and GetParameterIsInputable to detect if a specific input can be other nodes. Then we can get the next nodes to traverse via GetInput.
var nodeAsFilter = spShadingFilterNode.SafeCast(node); // Filter node class is the parent to all nodes with other nodes under it
if (!nodeAsFilter.IsNull())
{
for (int j = 0; j < nodeAsFilter.GetParameterCount(); j++)
{
if (nodeAsFilter.GetParameterIsInputable(j)) // Parameter can be other nodes
{
GetTextureNodes(nodeAsFilter.GetInput(j), textureNodes);
}
}
}
Disable sRGB flags
When we have found every texture node in our material we can disable sRGB using SetUseSRGB.
/// <summary>
/// Set sRGB flag on all texture nodes on material for specific channel.
/// </summary>
private static void SetUseSRGB(spMaterial material, string channel, bool sRGB)
{
var shadingNetwork = material.GetShadingNetwork(channel);
List<spShadingTextureNode> textureNodes = new List<spShadingTextureNode>();
GetTextureNodes(shadingNetwork, textureNodes);
foreach (var node in textureNodes)
{
node.SetUseSRGB(sRGB);
}
}
Remeshing pipeline
To process our LOD we are going to use a RemeshingPipeline. To this we add a VertexColorCaster with Unity specific casting settings.
Since the corners of our asset is sharp we set SetHardEdgeAngles to 45 to ensure those are kept. This means any angle above 45 will be counted as a hard edge, below it will be a smooth edge.
// Unity specific settings
const string UNITY_COLOR_CHANNEL_NAME = "diffuseColor";
const string UNITY_VERTEX_COLOR_NAME = "color";
using (var remeshingPipeline = simplygon.CreateRemeshingPipeline())
using (var remeshingSettings = remeshingPipeline.GetRemeshingSettings())
using (var vertexColorCaster = simplygon.CreateVertexColorCaster())
using (var vertexColorCasterSettings = vertexColorCaster.GetVertexColorCasterSettings())
using (var materialTable = sgScene.GetMaterialTable())
using (var mappingImageSettings = remeshingPipeline.GetMappingImageSettings())
{
// Set up remeshing pipeline settings
remeshingSettings.SetOnScreenSize(SCREEN_SIZE);
// Ensure we get sharp angles on our containers.
remeshingSettings.SetHardEdgeAngle(45);
// Need to create a mapping image for us to transfer colors.
mappingImageSettings.SetGenerateMappingImage(true);
// Add a vertex color caster with Unity specific settings.
vertexColorCasterSettings.SetMaterialChannel(UNITY_COLOR_CHANNEL_NAME); // Input albedo color channel specific to Unity
vertexColorCasterSettings.SetOutputColorName(UNITY_VERTEX_COLOR_NAME); // Output vertex color channel
remeshingPipeline.AddMaterialCaster(vertexColorCaster, 0);
Before running the scene we disable sRGB on all materials using the helper functions we created above.
// For every material in scene disable sRGB on all albedo channels.
for (int i = 0; i < materialTable.GetMaterialsCount(); i++)
{
using (var material = materialTable.GetMaterial(i))
{
SetUseSRGB(material, UNITY_COLOR_CHANNEL_NAME, false);
}
}
// Run processing
remeshingPipeline.RunScene(sgScene, EPipelineRunMode.RunInThisProcess);
After processing and importing the model looks like this. It is optimized enough, but lacks color.
Vertex color shader
Unity's standard shader does not use display vertex colors. To achieve this we are going to write a simple shader that showcase them.
Shader "Simplygon/VertexColored"
{
SubShader
{
Tags { "RenderType" = "Opaque" }
LOD 200
CGPROGRAM
#pragma surface surf Standard fullforwardshadows
#pragma target 3.0
struct Input
{
float4 color : Color; // Vertex color
};
void surf(Input IN, inout SurfaceOutputStandard o)
{
o.Albedo = IN.color.rgb; // User vertex color as color
}
ENDCG
}
FallBack "Diffuse"
}
With the shader in place the optimized mesh looks like this. The colors are displayed correctly.
Fixing smeared vertex colors
While the colors are displayed correctly we get lots of smearing between them. Using SetColorSpaceEdgeThreshold we can control when we should get a hard or smooth color edge. This is similar to how SetHardEdgeAngle which we covered above works. A color difference above treshold will give us a hard color edge, below will give us smooth colors.
The lowest value of 0.0 gives noticable hard color edges at places we do not want. A value of 0.025 is suitable for our purpose. Our script is thus updated with following setting.
// Fix vertex color smearing
vertexColorCasterSettings.SetColorSpaceEdgeThreshold(0.025f);
Result
Up close it is easy to see the difference between the optimized model and original. However from far away it is hard. Can you see any difference in image below?
Complete scripts
Remeshing script
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
using Simplygon.Unity.EditorPlugin;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;
namespace Simplygon.Examples.UnityVertexColorLOD.Editor
{
/// <summary>
/// Creates a remeshed object colored using vertex colors.
/// </summary>
public class VertexColoredLODCreator
{
const int SCREEN_SIZE = 60;
const string OUTPUT_FOLDER = "LOD";
// Unity specific settings
const string UNITY_COLOR_CHANNEL_NAME = "diffuseColor";
const string UNITY_VERTEX_COLOR_NAME = "color";
/// <summary>
/// Entry point from menu.
/// </summary>
[MenuItem("Simplygon/Create vertex colored LOD")]
static void CreateVertexColoredLODEntryPoint()
{
if (Selection.objects.Length > 0)
{
using (ISimplygon simplygon = Loader.InitSimplygon(out EErrorCodes simplygonErrorCode, out string simplygonErrorMessage))
{
if (simplygonErrorCode == EErrorCodes.NoError)
{
foreach (var o in Selection.objects)
{
CreateVertexColoredLOD(o as GameObject, simplygon);
}
}
else
{
Debug.Log("Initializing Simplygon failed: " + simplygonErrorCode);
}
}
}
}
/// <summary>
/// Create a vertex colored LOD and import it into scene.
/// </summary>
public static void CreateVertexColoredLOD(GameObject gameObject, ISimplygon simplygon)
{
var objectsToProcess = new List<GameObject>() { gameObject };
var exportTempDirectory = SimplygonUtils.GetNewTempDirectory();
using (var sgScene = SimplygonExporter.Export(simplygon, exportTempDirectory, objectsToProcess))
{
using (var remeshingPipeline = simplygon.CreateRemeshingPipeline())
using (var remeshingSettings = remeshingPipeline.GetRemeshingSettings())
using (var vertexColorCaster = simplygon.CreateVertexColorCaster())
using (var vertexColorCasterSettings = vertexColorCaster.GetVertexColorCasterSettings())
using (var materialTable = sgScene.GetMaterialTable())
using (var mappingImageSettings = remeshingPipeline.GetMappingImageSettings())
{
// Set up remeshing pipeline settings
remeshingSettings.SetOnScreenSize(SCREEN_SIZE);
// Ensure we get sharp angles on our containers.
remeshingSettings.SetHardEdgeAngle(45);
// Need to create a mapping image for us to transfer colors.
mappingImageSettings.SetGenerateMappingImage(true);
// Add a vertex color caster with Unity specific settings.
vertexColorCasterSettings.SetMaterialChannel(UNITY_COLOR_CHANNEL_NAME); // Input albedo color channel specific to Unity
vertexColorCasterSettings.SetOutputColorName(UNITY_VERTEX_COLOR_NAME); // Output vertex color channel
// Fix vertex color smearing
vertexColorCasterSettings.SetColorSpaceEdgeThreshold(0.025f);
remeshingPipeline.AddMaterialCaster(vertexColorCaster, 0);
// For every material in scene disable sRGB on all albedo channels.
for (int i = 0; i < materialTable.GetMaterialsCount(); i++)
{
using (var material = materialTable.GetMaterial(i))
{
SetUseSRGB(material, UNITY_COLOR_CHANNEL_NAME, false);
}
}
// Run processing
remeshingPipeline.RunScene(sgScene, EPipelineRunMode.RunInThisProcess);
// Handle output folder
var baseFolder = "Assets/" + OUTPUT_FOLDER;
if (!AssetDatabase.IsValidFolder(baseFolder))
{
AssetDatabase.CreateFolder("Assets", OUTPUT_FOLDER);
}
var assetFolderGuid = AssetDatabase.CreateFolder(baseFolder, gameObject.name);
var assetFolderPath = AssetDatabase.GUIDToAssetPath(assetFolderGuid);
int startingLodIndex = 0;
SimplygonImporter.Import(simplygon, remeshingPipeline, ref startingLodIndex, assetFolderPath, gameObject.name);
}
}
}
/// <summary>
/// Set sRGB flag on all texture nodes on material for specific channel.
/// </summary>
private static void SetUseSRGB(spMaterial material, string channel, bool sRGB)
{
var shadingNetwork = material.GetShadingNetwork(channel);
List<spShadingTextureNode> textureNodes = new List<spShadingTextureNode>();
GetTextureNodes(shadingNetwork, textureNodes);
foreach (var node in textureNodes)
{
node.SetUseSRGB(sRGB);
}
}
/// <summary>
/// Recursively get all texture nodes in a shading network.
/// </summary>
private static void GetTextureNodes(spShadingNode node, List<spShadingTextureNode> textureNodes)
{
var nodeAsTexture = spShadingTextureNode.SafeCast(node);
if (!nodeAsTexture.IsNull()) // We need to use IsNull instead of != null to check if it was a valid cast.
{
textureNodes.Add(nodeAsTexture);
}
else
{
var nodeAsFilter = spShadingFilterNode.SafeCast(node); // Filter node class is the parent to all nodes with other nodes under it
if (!nodeAsFilter.IsNull())
{
for (int j = 0; j < nodeAsFilter.GetParameterCount(); j++)
{
if (nodeAsFilter.GetParameterIsInputable(j)) // Parameter can be other nodes
{
GetTextureNodes(nodeAsFilter.GetInput(j), textureNodes);
}
}
}
}
}
}
}
Vertex color Unity shader
Shader "Simplygon/VertexColored"
{
SubShader
{
Tags { "RenderType" = "Opaque" }
LOD 200
CGPROGRAM
#pragma surface surf Standard fullforwardshadows
#pragma target 3.0
struct Input
{
float4 color : Color; // Vertex color
};
void surf(Input IN, inout SurfaceOutputStandard o)
{
o.Albedo = IN.color.rgb; // User vertex color as color
}
ENDCG
}
FallBack "Diffuse"
}