How to remesh your house with Simplygon

Written by Jesper Tingvall, Product Expert, Simplygon

Disclaimer: The code in this post is written using version 9.2.1400.0 of Simplygon and Blender 2.93.5. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

There are many assets which have different types of materials; both transparent and opaque. Remeshing these with Simplygon requires extra care if one wants to preserve the transparency.

In this post we are going to optimize a house with glass windows, a quite common asset in many games.

House with transparent glass and it's wireframe

Prerequisites

This example will use the Simplygon integration in Blender, but the same concepts can be applied to all other integrations of the Simplygon API.

Problem to solve

We want to create a proxy from an asset with some transparent materials. However we do not want the entire new proxy to be rendered with a transparent shader, that would be wasteful. A transparent shader should just be used for the transparent parts..

Solution

The solution has two parts; first seperate the objects into different sets then process them individually.

Split into different sets

Scenes can be split into different SelectionSets. We are going to create and add two to the scene; one for transparent objects and one for opaque objects.

transparent_set = sg.CreateSelectionSet()
opaque_set = sg.CreateSelectionSet()
transparent_set_id = scene.GetSelectionSetTable().AddSelectionSet(transparent_set)
opaque_set_id = scene.GetSelectionSetTable().AddSelectionSet(opaque_set)

We will then split the scene recursively into the two sets depending on if the object is considered transparent or not.

def split_opaque_transparent(scene, scene_node, transparent_set, opaque_set):
    for i in range(0, scene_node.GetChildCount()):
        child = scene_node.GetChild(i)
        if child.IsA("ISceneMesh"):
            scene_mesh = Simplygon.spSceneMesh.SafeCast(child)
            if is_mesh_transparent(scene, scene_mesh):
                transparent_set.AddItem(child.GetNodeGUID())    
            else:
                opaque_set.AddItem(child.GetNodeGUID())  
                
        split_opaque_transparent(scene, child,  transparent_set, opaque_set)

Detect transparency

In order to detect that a material is transparent we check the IsTransparent flag.

def is_material_transparent(scene, material_id):
    material = scene.GetMaterialTable().GetMaterial(material_id)
    return material.IsTransparent()

A mesh is considered transparent if any of the materials it uses is transparent.

def is_mesh_transparent(scene, scene_mesh):
    materials =  scene_mesh.GetGeometry().GetMaterialIds()
    for j in range(0, materials.GetItemCount()):
        if is_material_transparent(scene, materials.GetItem(j)):
            return True
    return False

Remeshing of opaque mesh

To generate a proxy we are going to use the Remeshing Pipeline. To only apply the pipeline to certain parts of the scene we can specify that it should only run on a SelectionSet via SetProcessSelectionSetID. We also need to set SetKeepUnprocessedSceneMeshes to true, otherwise everything that is not processed will be thrown away.

We set the target quality via OnScreenSize. On this specific house asset we also need to disable HoleFilling, as otherwise some holes for windows will be removed.

Remeshed opaque house

Lastly we need to add Material casters for the texture maps on our object, in our case we have albedo, metal, roughness and a normal map.

def create_remeshing_pipeline(sg, set_id, resolution):
    pipeline = sg.CreateRemeshingPipeline()
    settings = pipeline.GetRemeshingSettings()
    settings.SetProcessSelectionSetID(set_id)
    settings.SetKeepUnprocessedSceneMeshes(True)
    settings.SetHoleFilling( Simplygon.EHoleFilling_Disabled )

    settings.SetOnScreenSize(resolution)
    
    mapping_image_settings = pipeline.GetMappingImageSettings()
    material_output_settings = mapping_image_settings.GetOutputMaterialSettings(0)
    material_output_settings.SetTextureHeight(2048)
    material_output_settings.SetTextureWidth(2048)
    
    # Color caster
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel("Basecolor") 
    pipeline.AddMaterialCaster( caster, 0 )
    
    # Metal caster
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel("Metalness") 
    pipeline.AddMaterialCaster( caster, 0 )
    
    # Roughness caster
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel("Roughness") 
    pipeline.AddMaterialCaster( caster, 0 )
    
    # Normal caster
    caster = sg.CreateNormalCaster()
    caster_settings = caster.GetNormalCasterSettings()
    caster_settings.SetGenerateTangentSpaceNormals(True)
    caster_settings.SetMaterialChannel("Normals") 
    
    # Change these settings to correspond to your engine
    # caster_settings.SetFlipGreen(False)
    # caster_settings.SetCalculateBitangentPerFragment(True)
    # caster_settings.SetNormalizeInterpolatedTangentSpace(False)
    
    pipeline.AddMaterialCaster( caster, 0 )
    return pipeline

Reduction of transparent parts

For the transparent windows we are going to use a reduction pipeline. Reason for this is that windows in many cases is a simple quad. Using a remeshing pipeline would create an watertight mesh resulting in each window getting two sides. In our asset the glass and window frame stored in the same mesh, thus both of them is in the reduction set. A future improvement would be to split this mesh into one transparent and one opaque part.

Reduced transparent models

As with the remeshing pipeline we can use SetProcessSelectionSetID to specify what parts to process. We also need to set SetKeepUnprocessedSceneMeshes to true as otherwise all other meshes would be thrown away.

Target quality is set via SetReductionTargetOnScreenSize so we can use the same metric as for the remeshing pipeline. This will however only work if both sets have roughly the same screen size.

def create_reduction_pipeline(sg, set_id, resolution):
    pipeline = sg.CreateReductionPipeline()
    pipeline_settings = pipeline.GetReductionSettings()
    pipeline_settings.SetProcessSelectionSetID(set_id)
    pipeline_settings.SetMergeGeometries(True)
    pipeline_settings.SetKeepUnprocessedSceneMeshes(True)
    
    pipeline_settings.SetReductionTargetTriangleRatioEnabled(False)
    pipeline_settings.SetReductionTargetTriangleCountEnabled(False)
    pipeline_settings.SetReductionTargetOnScreenSizeEnabled(True)
    pipeline_settings.SetReductionTargetOnScreenSize(resolution)
    
    return pipeline

Result

The result is a proxy split into two parts; one with all opaque materials and one with all transparent materials.

Asset Objects Triangles
Unoptimized 173 206123
Optimized 2 33312

Remeshed house proxy with transparent windows

Complete scripts

The resulting scripts can be found below.

# Copyright (c) Microsoft Corporation. 
# Licensed under the MIT license. 

import os
import bpy
from simplygon import simplygon_loader
from simplygon import Simplygon

def split_opaque_transparent(scene, scene_node, transparent_set, opaque_set):
    for i in range(0, scene_node.GetChildCount()):
        child = scene_node.GetChild(i)
        if child.IsA("ISceneMesh"):
            scene_mesh = Simplygon.spSceneMesh.SafeCast(child)
            if is_mesh_transparent(scene, scene_mesh):
                transparent_set.AddItem(child.GetNodeGUID())    
            else:
                opaque_set.AddItem(child.GetNodeGUID())  
                
        split_opaque_transparent(scene, child,  transparent_set, opaque_set)

def is_mesh_transparent(scene, scene_mesh):
    materials =  scene_mesh.GetGeometry().GetMaterialIds()
    for j in range(0, materials.GetItemCount()):
        if is_material_transparent(scene, materials.GetItem(j)):
            return True
    return False
        
def is_material_transparent(scene, material_id):
    material = scene.GetMaterialTable().GetMaterial(material_id)
    return material.IsTransparent()

def create_reduction_pipeline(sg, set_id, resolution):
    pipeline = sg.CreateReductionPipeline()
    pipeline_settings = pipeline.GetReductionSettings()
    pipeline_settings.SetProcessSelectionSetID(set_id)
    pipeline_settings.SetMergeGeometries(True)
    pipeline_settings.SetKeepUnprocessedSceneMeshes(True)
    
    pipeline_settings.SetReductionTargetTriangleRatioEnabled(False)
    pipeline_settings.SetReductionTargetTriangleCountEnabled(False)
    pipeline_settings.SetReductionTargetOnScreenSizeEnabled(True)
    pipeline_settings.SetReductionTargetOnScreenSize(resolution)
    
    return pipeline
    
def create_remeshing_pipeline(sg, set_id, resolution):
    pipeline = sg.CreateRemeshingPipeline()
    settings = pipeline.GetRemeshingSettings()
    settings.SetProcessSelectionSetID(set_id)
    settings.SetKeepUnprocessedSceneMeshes(True)
    settings.SetHoleFilling( Simplygon.EHoleFilling_Disabled )

    settings.SetOnScreenSize(resolution)
    
    mapping_image_settings = pipeline.GetMappingImageSettings()
    material_output_settings = mapping_image_settings.GetOutputMaterialSettings(0)
    material_output_settings.SetTextureHeight(2048)
    material_output_settings.SetTextureWidth(2048)
    
    # Color caster
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel("Basecolor") 
    pipeline.AddMaterialCaster( caster, 0 )
    
    # Metal caster
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel("Metalness") 
    pipeline.AddMaterialCaster( caster, 0 )
    
    # Roughness caster
    caster = sg.CreateColorCaster()
    caster_settings = caster.GetColorCasterSettings()
    caster_settings.SetMaterialChannel("Roughness") 
    pipeline.AddMaterialCaster( caster, 0 )
    
    # Normal caster
    caster = sg.CreateNormalCaster()
    caster_settings = caster.GetNormalCasterSettings()
    caster_settings.SetGenerateTangentSpaceNormals(True)
    caster_settings.SetMaterialChannel("Normals") 
    
    # Change these settings to correspond to your engine
    # caster_settings.SetFlipGreen(False)
    # caster_settings.SetCalculateBitangentPerFragment(True)
    # caster_settings.SetNormalizeInterpolatedTangentSpace(False)
    
    pipeline.AddMaterialCaster( caster, 0 )
    return pipeline
    
def process_file(tmp_file):
    # Change these settings to correspond to your engine
    # sg.SetGlobalDefaultTangentCalculatorTypeSetting(Simplygon.ETangentSpaceMethod_MikkTSpace)
    
    sceneImporter = sg.CreateSceneImporter()
    sceneImporter.SetImportFilePath(tmp_file)
    sceneImporter.RunImport()
    scene = sceneImporter.GetScene()
    
    transparent_set = sg.CreateSelectionSet()
    opaque_set = sg.CreateSelectionSet()
    transparent_set_id = scene.GetSelectionSetTable().AddSelectionSet(transparent_set)
    opaque_set_id = scene.GetSelectionSetTable().AddSelectionSet(opaque_set)
    split_opaque_transparent(scene, scene.GetRootNode(), transparent_set, opaque_set)
    
    pipeline_transparent = create_reduction_pipeline(sg, transparent_set_id, 900)
    pipeline_opaque = create_remeshing_pipeline(sg, opaque_set_id, 900)
    
    pipeline_opaque.RunScene(scene, Simplygon.EPipelineRunMode_RunInThisProcess)
    pipeline_transparent.RunScene(scene, Simplygon.EPipelineRunMode_RunInThisProcess)

    scene_exporter = sg.CreateSceneExporter()
    scene_exporter.SetExportFilePath(tmp_file)
    scene_exporter.SetScene(scene)
    scene_exporter.RunExport()


        
sg = simplygon_loader.init_simplygon()
print(sg.GetVersion())
file_path = 'c:/Temp/_intermediate.glb'
bpy.ops.export_scene.gltf(filepath = file_path, use_selection=True)
process_file(file_path)
bpy.ops.import_scene.gltf(filepath=file_path)
sg = None
gc.collect()
⇐ Back to all posts

Request 30-days free evaluation license

*
*
*
*
Industry
*

Request 30-days free evaluation license

*
*
*
*
Industry
*