Demystifying vertex colors in 3ds Max
Written by Jesper Tingvall, Product Expert, Simplygon
Disclaimer: The code in this post is written using version 10.2.10100.0 of Simplygon and 3ds Max 2021. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.
Introduction
In this blog post we are going to optimize a game character in 3ds Max. We are going to use two sets of vertex colors; one which the game utilized and one which control the reduction.
Prerequisites
This example will use the Simplygon integration in 3ds Max, but the same concepts can be applied to all other integrations of the Simplygon API. However you will probably not encounter the same 3ds Max specific problems.
Problem to solve
We have a game character which we want to optimize. In our game we use vertex colors to differentiate clothes from the character's skin. The optimized mesh must keep these colors.
If we do a 25% reduction via the user interface in 3ds Max we get this LOD. It is quite a heavy reduction so not intended to be viewed up this close. However, just to be safe we want to allocate some more vertices to the face.
Solution
The solution is to add a secondary vertex color set to control the reduction.
Vertex paint face
Humans pay close attention to faces, so we want to keep more vertices in that area. To achieve this we are going to use map channel three to weight paint the face. First every vertex is colored grey, then we paint a white T-shape on the face. This is the area where humans are looking most at.
After painting these are all map channels in our mesh. Zero is first vertex color set which we use to differentiate between skin and clothes. Three is vertex color set we will use to control reduction.
Preserve more then one vertex color set in 3ds Max
If we process the character with a standard 25% reduction pipeline in UI after painting we get the following output. Something is definitely wrong.
What has happened is that in 3ds Max every channel >0 is counted as an UV channel. Our color has thus lost its blue value since only first 2 values are kept; U and V coordinates, corresponding to red and green values.
We need to use sgsdk_SetIsVertexColorChannel
to indicate that map channel 3 should be used as vertex color channel and not another UV channel. To use this function we need to perform the optimization via scripting. sgsdk_SetIsVertexColorChannel
should be called before exporting from 3ds Max into Simplygon.
def main():
# Initialize Simplygon
sg = simplygon_loader.init_simplygon()
# Set map channel 3 to vertex color type
rt.sgsdk_SetIsVertexColorChannel(3, True)
# Export scene to file
bResult = rt.sgsdk_ExportToFile(export_path, False)
# Reduce exported file
run_pipeline(sg)
# Importing
bResult = rt.sgsdk_ImportFromFile(processed_path, True, True, True)
print('Import result: ', bResult)
# De-initialize Simplygon
sg = None
gc.collect()
print('Finished')
With sgsdk_SetIsVertexColorChannel
in place we can see that the third vertex color set is preserved.
Use vertex color to control reduction
To tell Simplygon to use our newly preserved vertex color set to guide optimization we need to access the reduction pipeline's VertexWeightSettings
. First we are going to enable using vertex colors to guide optimization by setting SetUserVertexWeightsInReducer
to True
. We also need to specify what vertex color set we are going to use.
In 3ds Max we do not have names for color sets like in Maya. Instead we refer to them via their mapping channel ID. In our case we use map channel three as our vertex color set, thus we set SetWeightsFromColorName
to 3.
def run_pipeline(sg):
pipeline = sg.CreateReductionPipeline()
# Set reduction ratio
reduction_settings = pipeline.GetReductionSettings()
reduction_settings.SetReductionTargetTriangleRatio(0.25)
# Use vertex color 3 to control optimization
weight_settings = pipeline.GetVertexWeightSettings()
weight_settings.SetUseVertexWeightsInReducer(True)
weight_settings.SetWeightsFromColorName("3")
pipeline.RunSceneFromFile(export_path, processed_path, Simplygon.EPipelineRunMode_RunInThisProcess)
After using vertex color set three as weights to prioritize the face we get a much more nice looking LOD.
The observant documentation reader might notice that WeightsFromColorComponent
's default value is red, and think that we should be able to use vertex color set 3 to guide optimization without specifying it is a color channel. This is however not the case as in Simplygon UVs and vertex color sets are stored separately. We can not use an UV set to control reduction.
Result
After weight painting the face, marking that map channel as a vertex color channel and using it to control reduction we get a much more nice looking LOD for 25% reduction.
If we look at our original vertex color set we can see that it is kept as well.
Complete script
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
from pymxs import runtime as rt
from simplygon10 import simplygon_loader
from simplygon10 import Simplygon
import gc
export_path = 'C:/Temp/ExportedScene.sb'
processed_path = 'C:/Temp/ProcessedScene.sb'
def run_pipeline(sg):
pipeline = sg.CreateReductionPipeline()
# Set reduction ratio
reduction_settings = pipeline.GetReductionSettings()
reduction_settings.SetReductionTargetTriangleRatio(0.25)
# Use vertex color 3 to control optimization
weight_settings = pipeline.GetVertexWeightSettings()
weight_settings.SetUseVertexWeightsInReducer(True)
weight_settings.SetWeightsFromColorName("3")
pipeline.RunSceneFromFile(export_path, processed_path, Simplygon.EPipelineRunMode_RunInThisProcess)
def main():
# Initialize Simplygon
sg = simplygon_loader.init_simplygon()
# Set map channel 3 to vertex color type
rt.sgsdk_SetIsVertexColorChannel(3, True)
# Export scene to file
bResult = rt.sgsdk_ExportToFile(export_path, False)
# Reduce exported file
run_pipeline(sg)
# Importing
bResult = rt.sgsdk_ImportFromFile(processed_path, True, True, True)
print('Import result: ', bResult)
# De-initialize Simplygon
sg = None
gc.collect()
print('Finished')
if __name__== "__main__":
main()