Written by Jesper Tingvall, Product Expert, Simplygon

Disclaimer: The code in this post is written using version 10.3.6400.0 of Simplygon. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

In this blog we'll showcase how the surface mapper enables reusing the same model as LOD levels for multiple models with different topology. It also showcase how you can bake materials to an self supplied proxy model.

Prerequisites

This example will use the Simplygon Python API, but the same concepts can be applied to all other integrations of the Simplygon API.

Problem to solve

We have a number of objects with similar geometry and want to create LOD levels for them. Since the objects are very similar in shape we want these to share their lower LOD model to save memory. The original models do not share topology at all, so if we would use the reducer or remesher on them we would get different LOD models. Hence we need to do something else.

Solution

We will use a surface mapper to create a mapping image between our proxy model and the original model. With this we can cast materials to the proxy model.

Another use case for this solution is if Simplygon does not generate a suitable proxy model, and we want to provide our own. In essence we can transfer materials from one geometry to another.

Proxy model

First we'll create a proxy model. This model should be as close as possible to the original geometries. We just create a low poly cylinder in Blender with suitable scale. It has a simple UV chart that we will cast all materials to.

One important thing to point out is that we require normals and tangents to be present in the model. If this is not the case then the baked normal map will be empty. Hence it is important that we check Geometry → Tangent Space during export from Blender to fbx. It is also possible to use Simplygon's Normal Repairer to recalculate new normals.

Create mapping image using surface mapper

To transfer data and do material casting between models Simplygon uses a mapping image. This texture like structure maps every point on the optimized model to the original model. In most cases a mapping image is created behind the scenes, like when we use pipelines, and we do not have to create one manually. Here we will use a surface mapper to create a mapping image.

We first create a surface mapper and specify which Simplygon scenes that is SourceScene and DestinationScene. We also set SearchOffset and SearchDistance. How large these values needs to be depends on how much the proxy model differs from the original model. Be careful though, too large values can make material mapping behave strangely.

We also specify output texture width and height. Lastly we generate a mapping image by calling RunSurfaceMapping.

def create_mapping_image_between_scenes(sg, original_scene, optimized_scene, texture_size):
    """Use surface mapper to generate mapping image from original_scene to optimized_scene."""

    surface_mapper = sg.CreateSurfaceMapper()
    surface_mapper.SetSourceScene(original_scene)
    surface_mapper.SetDestinationScene(optimized_scene)
    
    # Adapt these values if needed.
    surface_mapper.SetSearchOffset(2)
    surface_mapper.SetSearchDistance(4)
    
    # Set texture size
    mapping_image_settings = surface_mapper.GetMappingImageSettings()
    output_material_settings = mapping_image_settings.GetOutputMaterialSettings(0)
    output_material_settings.SetTextureWidth( texture_size )
    output_material_settings.SetTextureHeight( texture_size )
    
    # Generate mapping image from original_scene to optimized_scene
    surface_mapper.RunSurfaceMapping()
    return surface_mapper.GetMappingImage()

Cast diffuse texture

Using the mapping image we just created we can cast diffuse textures suitable for our proxy model's UV map. We create a color caster and specify that it should use the mapping image we just created. Since we are not using Simplygon pipelines we need to do some additional setup for the material casting.

In addition we needs to set the original scene's material table and texture table. We specify output file path and color space for output texture. Then we perform material casting by calling RunProcessing.

def perform_color_casting(sg, scene, path, mapping_image, material_channel):
    """Setup and run a color caster material casting."""

    caster = sg.CreateColorCaster()
    caster.SetMappingImage( mapping_image )
    caster.SetSourceMaterials( scene.GetMaterialTable() )
    caster.SetSourceTextures( scene.GetTextureTable() )
    caster.SetOutputFilePath( path + material_channel + ".webp")

    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel( material_channel )
    caster_settings.SetOutputSRGB(True)

    caster.RunProcessing()
    return caster.GetOutputFilePath()

Here is the output texture. It matches the proxy model's UV coordinates.

Cast normal map

Normal map casting is almost identical to color casting. The only difference is that we use a normal caster instead of color caster, and we need to specify NormalCasterSettings that corresponds to how our engine uses normal maps.

def perform_normal_casting(sg, scene, path, mapping_image, normal_channel):
    """Setup and run a normal caster."""

    caster = sg.CreateNormalCaster()
    caster.SetMappingImage( mapping_image )
    caster.SetSourceMaterials( scene.GetMaterialTable() )
    caster.SetSourceTextures( scene.GetTextureTable() )
    caster.SetOutputFilePath( path + normal_channel + ".webp")

    caster_settings = caster.GetNormalCasterSettings()
    caster_settings.SetMaterialChannel( normal_channel )

    # Change these settings to suit your normal map
    caster_settings.SetGenerateTangentSpaceNormals(True)
    caster_settings.SetFlipGreen(False)
    caster_settings.SetCalculateBitangentPerFragment(True)
    caster_settings.SetNormalizeInterpolatedTangentSpace(False)

    caster.RunProcessing()
    return caster.GetOutputFilePath()

In the output we can see that it captures both geometry details from the model and small details from the original normal map.

Optimize

Now let's put it all together and generate some textures suitable for our proxy model. We start by initializing Simplygon and import the original scene and proxy scene. After that we create a mapping image between the original scene and proxy scene. Using this mapping image we bake diffuse and normal textures.

def process_selection(input_file, proxy_file, texture_size, output_file):
    """Generate a LOD for input_file using proxy_file and save to output_file."""

    # Init Simplygon
    sg = simplygon_loader.init_simplygon()

    # Import files
    original_scene = import_file(sg, input_file)
    proxy_scene = import_file(sg, proxy_file)

    # Create mapping image
    mapping_image = create_mapping_image_between_scenes(sg, original_scene, proxy_scene, texture_size)
    
    # Bake color texture
    color_texture = perform_color_casting(sg, original_scene, output_path + input_file, mapping_image, color_channel)
    normal_texture = perform_normal_casting(sg, original_scene, output_path + input_file, mapping_image, normal_channel)

If we just want to reuse the same model we are done now. We can switch between the texture sets on our proxy model to use it as LOD level for different models. Here is the output diffuse and normal textures, notice how all uses the proxy model's UV layout.

Add casted texture to proxy model

We start by adding a helper function which adds a texture as a channel to a specified material. First we import the texture and add it to our scene's texture table. After that we create a simple shading network which only consists of a single ShadingTextureNode. On this node we specify which UV field to use with TexCoordLevel and texture with TextureName. We can then add our newly created shading network as a MaterialChannel on the material.

def add_channel_to_material(sg, scene, material, material_channel, texture_path):
    """Add a new material with specified texture to scene"""

    # Create texture from texture_path and add it to scene
    texture = sg.CreateTexture()
    texture.SetName( material_channel )
    texture.SetFilePath( texture_path)
    scene.GetTextureTable().AddTexture( texture )

    # Create shading node using texture we created above
    shading_node = sg.CreateShadingTextureNode()
    shading_node.SetTexCoordLevel( 0 )
    shading_node.SetTextureName( material_channel )
    
    # Create material using shading node we created above and add it to scene
    material.AddMaterialChannel( material_channel )
    material.SetShadingNetwork( material_channel, shading_node )

With our helper function in place. All we have to do to output models with casted texture is first a little cleanup of existing materials and textures in our proxy model. We then create a new material and add it to our proxy's material table. To that material we add our baked diffuse and normal texture. Lastly we export the scene.

def process_selection(input_file, proxy_file, texture_size, output_file):
    """Generate a LOD for input_file using proxy_file and save to output_file."""

    ...

    normal_texture = perform_normal_casting(sg, original_scene, output_path + input_file, mapping_image, normal_channel)
    
    # If we are reusing the proxy mesh we probably do not steps below as we will use use the output textures directly.

    # Clear textures and material from optimized_scene
    clear_scene_material_and_texture_tables(proxy_scene)
    
    # Create material
    material = sg.CreateMaterial()
    proxy_scene.GetMaterialTable().AddMaterial( material )

    # Add textures to material channels
    add_channel_to_material(sg, proxy_scene, material, color_channel, color_texture)
    add_channel_to_material(sg, proxy_scene, material, normal_channel, normal_texture)

    # Export scene
    export_file(sg, proxy_scene, output_file)

Result

Here is the resulting models. As we have specified a very low poly proxy model, which does not align completely with the geometry of our input model, it is quite easy to spot a difference up close. But from a distant it would be hard to spot a difference.

Complete script

# Copyright (c) Microsoft Corporation. 
# Licensed under the MIT license. 

import gc

from simplygon10 import simplygon_loader
from simplygon10 import Simplygon

output_path = "c:/tmp/"

# fbx specific channel names
color_channel = "Diffuse"
normal_channel = "Normals"

def import_file(sg, file_path):
    """Import file to Simplygon scene."""

    sceneImporter = sg.CreateSceneImporter()
    sceneImporter.SetImportFilePath(file_path)
    sceneImporter.Run()
    scene = sceneImporter.GetScene()
    return scene


def export_file(sg, scene, file_path):
    """Export Simplygon scene to file."""

    scene_exporter = sg.CreateSceneExporter()
    scene_exporter.SetExportFilePath(file_path)
    scene_exporter.SetScene(scene)
    scene_exporter.Run()


def create_mapping_image_between_scenes(sg, original_scene, optimized_scene, texture_size):
    """Use surface mapper to generate mapping image from original_scene to optimized_scene."""

    surface_mapper = sg.CreateSurfaceMapper()
    surface_mapper.SetSourceScene(original_scene)
    surface_mapper.SetDestinationScene(optimized_scene)
    
    # Adapt these values if needed.
    surface_mapper.SetSearchOffset(2)
    surface_mapper.SetSearchDistance(4)
    
    # Set texture size
    mapping_image_settings = surface_mapper.GetMappingImageSettings()
    output_material_settings = mapping_image_settings.GetOutputMaterialSettings(0)
    output_material_settings.SetTextureWidth( texture_size )
    output_material_settings.SetTextureHeight( texture_size )
    
    # Generate mapping image from original_scene to optimized_scene
    surface_mapper.RunSurfaceMapping()
    return surface_mapper.GetMappingImage()


def clear_scene_material_and_texture_tables(scene):
    """Clean scene's texture and material table."""

    scene.GetTextureTable().Clear()
    scene.GetMaterialTable().Clear()


def perform_color_casting(sg, scene, path, mapping_image, material_channel):
    """Setup and run a color caster material casting."""

    caster = sg.CreateColorCaster()
    caster.SetMappingImage( mapping_image )
    caster.SetSourceMaterials( scene.GetMaterialTable() )
    caster.SetSourceTextures( scene.GetTextureTable() )
    caster.SetOutputFilePath( path + material_channel + ".webp")

    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel( material_channel )
    caster_settings.SetOutputSRGB(True)

    caster.RunProcessing()
    return caster.GetOutputFilePath()

def perform_normal_casting(sg, scene, path, mapping_image, normal_channel):
    """Setup and run a normal caster."""

    caster = sg.CreateNormalCaster()
    caster.SetMappingImage( mapping_image )
    caster.SetSourceMaterials( scene.GetMaterialTable() )
    caster.SetSourceTextures( scene.GetTextureTable() )
    caster.SetOutputFilePath( path + normal_channel + ".webp")

    caster_settings = caster.GetNormalCasterSettings()
    caster_settings.SetMaterialChannel( normal_channel )

    # Change these settings to suit your normal map
    caster_settings.SetGenerateTangentSpaceNormals(True)
    caster_settings.SetFlipGreen(False)
    caster_settings.SetCalculateBitangentPerFragment(True)
    caster_settings.SetNormalizeInterpolatedTangentSpace(False)

    caster.RunProcessing()
    return caster.GetOutputFilePath()

def add_channel_to_material(sg, scene, material, material_channel, texture_path):
    """Add a new material with specified texture to scene"""

    # Create texture from texture_path and add it to scene
    texture = sg.CreateTexture()
    texture.SetName( material_channel )
    texture.SetFilePath( texture_path)
    scene.GetTextureTable().AddTexture( texture )

    # Create shading node using texture we created above
    shading_node = sg.CreateShadingTextureNode()
    shading_node.SetTexCoordLevel( 0 )
    shading_node.SetTextureName( material_channel )
    
    # Create material using shading node we created above and add it to scene
    material.AddMaterialChannel( material_channel )
    material.SetShadingNetwork( material_channel, shading_node )


def process_selection(input_file, proxy_file, texture_size, output_file):
    """Generate a LOD for input_file using proxy_file and save to output_file."""

    # Init Simplygon
    sg = simplygon_loader.init_simplygon()

    # Import files
    original_scene = import_file(sg, input_file)
    proxy_scene = import_file(sg, proxy_file)

    # Create mapping image
    mapping_image = create_mapping_image_between_scenes(sg, original_scene, proxy_scene, texture_size)
    
    # Bake color texture
    color_texture = perform_color_casting(sg, original_scene, output_path + input_file, mapping_image, color_channel)
    normal_texture = perform_normal_casting(sg, original_scene, output_path + input_file, mapping_image, normal_channel)
    
    # Clear textures and material from optimized_scene
    clear_scene_material_and_texture_tables(proxy_scene)
    
    # Create material
    # If we are reusing the proxy mesh we probably do not need this and steps below as we will use use the output textures directly.
    material = sg.CreateMaterial()
    proxy_scene.GetMaterialTable().AddMaterial( material )

    # Add textures to material channels
    add_channel_to_material(sg, proxy_scene, material, color_channel, color_texture)
    add_channel_to_material(sg, proxy_scene, material, normal_channel, normal_texture)

    # Export scene
    export_file(sg, proxy_scene, output_file)

    # Cleanup Simplygon
    sg = None
    gc.collect()


def main():
    process_selection("leather_cleaner.fbx", "proxy_lowpoly.fbx", 1024, "leather_output.fbx")
    process_selection("can_rusted_2k.fbx", "proxy_lowpoly.fbx", 1024, "can_rusted_output.fbx")
    process_selection("cleaner_tin.fbx", "proxy_lowpoly.fbx", 1024, "cleaner_tin_output.fbx")

if __name__== "__main__":
    main()