How to remesh your house with Simplygon
Written by Jesper Tingvall, Product Expert, Simplygon
Disclaimer: The code in this post is written using version 9.2.1400.0 of Simplygon and Blender 2.93.5. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.
Introduction
There are many assets which have different types of materials; both transparent and opaque. Remeshing these with Simplygon requires extra care if one wants to preserve the transparency.
In this post we are going to optimize a house with glass windows, a quite common asset in many games.
Prerequisites
This example will use the Simplygon integration in Blender, but the same concepts can be applied to all other integrations of the Simplygon API.
Problem to solve
We want to create a proxy from an asset with some transparent materials. However we do not want the entire new proxy to be rendered with a transparent shader, that would be wasteful. A transparent shader should just be used for the transparent parts..
Solution
The solution has two parts; first seperate the objects into different sets then process them individually.
Split into different sets
Scenes can be split into different SelectionSets. We are going to create and add two to the scene; one for transparent objects and one for opaque objects.
transparent_set = sg.CreateSelectionSet()
opaque_set = sg.CreateSelectionSet()
transparent_set_id = scene.GetSelectionSetTable().AddSelectionSet(transparent_set)
opaque_set_id = scene.GetSelectionSetTable().AddSelectionSet(opaque_set)
We will then split the scene recursively into the two sets depending on if the object is considered transparent or not.
def split_opaque_transparent(scene, scene_node, transparent_set, opaque_set):
for i in range(0, scene_node.GetChildCount()):
child = scene_node.GetChild(i)
if child.IsA("ISceneMesh"):
scene_mesh = Simplygon.spSceneMesh.SafeCast(child)
if is_mesh_transparent(scene, scene_mesh):
transparent_set.AddItem(child.GetNodeGUID())
else:
opaque_set.AddItem(child.GetNodeGUID())
split_opaque_transparent(scene, child, transparent_set, opaque_set)
Detect transparency
In order to detect that a material is transparent we check the IsTransparent flag.
def is_material_transparent(scene, material_id):
material = scene.GetMaterialTable().GetMaterial(material_id)
return material.IsTransparent()
A mesh is considered transparent if any of the materials it uses is transparent.
def is_mesh_transparent(scene, scene_mesh):
materials = scene_mesh.GetGeometry().GetMaterialIds()
for j in range(0, materials.GetItemCount()):
if is_material_transparent(scene, materials.GetItem(j)):
return True
return False
Remeshing of opaque mesh
To generate a proxy we are going to use the Remeshing Pipeline. To only apply the pipeline to certain parts of the scene we can specify that it should only run on a SelectionSet via SetProcessSelectionSetID. We also need to set SetKeepUnprocessedSceneMeshes to true, otherwise everything that is not processed will be thrown away.
We set the target quality via OnScreenSize. On this specific house asset we also need to disable HoleFilling, as otherwise some holes for windows will be removed.
Lastly we need to add Material casters for the texture maps on our object, in our case we have albedo, metal, roughness and a normal map.
def create_remeshing_pipeline(sg, set_id, resolution):
pipeline = sg.CreateRemeshingPipeline()
settings = pipeline.GetRemeshingSettings()
settings.SetProcessSelectionSetID(set_id)
settings.SetKeepUnprocessedSceneMeshes(True)
settings.SetHoleFilling( Simplygon.EHoleFilling_Disabled )
settings.SetOnScreenSize(resolution)
mapping_image_settings = pipeline.GetMappingImageSettings()
material_output_settings = mapping_image_settings.GetOutputMaterialSettings(0)
material_output_settings.SetTextureHeight(2048)
material_output_settings.SetTextureWidth(2048)
# Color caster
caster = sg.CreateColorCaster()
caster_settings = caster.GetColorCasterSettings()
caster_settings.SetMaterialChannel("Basecolor")
pipeline.AddMaterialCaster( caster, 0 )
# Metal caster
caster = sg.CreateColorCaster()
caster_settings = caster.GetColorCasterSettings()
caster_settings.SetMaterialChannel("Metalness")
pipeline.AddMaterialCaster( caster, 0 )
# Roughness caster
caster = sg.CreateColorCaster()
caster_settings = caster.GetColorCasterSettings()
caster_settings.SetMaterialChannel("Roughness")
pipeline.AddMaterialCaster( caster, 0 )
# Normal caster
caster = sg.CreateNormalCaster()
caster_settings = caster.GetNormalCasterSettings()
caster_settings.SetGenerateTangentSpaceNormals(True)
caster_settings.SetMaterialChannel("Normals")
# Change these settings to correspond to your engine
# caster_settings.SetFlipGreen(False)
# caster_settings.SetCalculateBitangentPerFragment(True)
# caster_settings.SetNormalizeInterpolatedTangentSpace(False)
pipeline.AddMaterialCaster( caster, 0 )
return pipeline
Reduction of transparent parts
For the transparent windows we are going to use a reduction pipeline. Reason for this is that windows in many cases is a simple quad. Using a remeshing pipeline would create an watertight mesh resulting in each window getting two sides. In our asset the glass and window frame stored in the same mesh, thus both of them is in the reduction set. A future improvement would be to split this mesh into one transparent and one opaque part.
As with the remeshing pipeline we can use SetProcessSelectionSetID to specify what parts to process. We also need to set SetKeepUnprocessedSceneMeshes to true as otherwise all other meshes would be thrown away.
Target quality is set via SetReductionTargetOnScreenSize so we can use the same metric as for the remeshing pipeline. This will however only work if both sets have roughly the same screen size.
def create_reduction_pipeline(sg, set_id, resolution):
pipeline = sg.CreateReductionPipeline()
pipeline_settings = pipeline.GetReductionSettings()
pipeline_settings.SetProcessSelectionSetID(set_id)
pipeline_settings.SetMergeGeometries(True)
pipeline_settings.SetKeepUnprocessedSceneMeshes(True)
pipeline_settings.SetReductionTargetTriangleRatioEnabled(False)
pipeline_settings.SetReductionTargetTriangleCountEnabled(False)
pipeline_settings.SetReductionTargetOnScreenSizeEnabled(True)
pipeline_settings.SetReductionTargetOnScreenSize(resolution)
return pipeline
Result
The result is a proxy split into two parts; one with all opaque materials and one with all transparent materials.
| Asset | Objects | Triangles |
|---|---|---|
| Unoptimized | 173 | 206123 |
| Optimized | 2 | 33312 |
Complete scripts
The resulting scripts can be found below.
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
import os
import bpy
from simplygon import simplygon_loader
from simplygon import Simplygon
def split_opaque_transparent(scene, scene_node, transparent_set, opaque_set):
for i in range(0, scene_node.GetChildCount()):
child = scene_node.GetChild(i)
if child.IsA("ISceneMesh"):
scene_mesh = Simplygon.spSceneMesh.SafeCast(child)
if is_mesh_transparent(scene, scene_mesh):
transparent_set.AddItem(child.GetNodeGUID())
else:
opaque_set.AddItem(child.GetNodeGUID())
split_opaque_transparent(scene, child, transparent_set, opaque_set)
def is_mesh_transparent(scene, scene_mesh):
materials = scene_mesh.GetGeometry().GetMaterialIds()
for j in range(0, materials.GetItemCount()):
if is_material_transparent(scene, materials.GetItem(j)):
return True
return False
def is_material_transparent(scene, material_id):
material = scene.GetMaterialTable().GetMaterial(material_id)
return material.IsTransparent()
def create_reduction_pipeline(sg, set_id, resolution):
pipeline = sg.CreateReductionPipeline()
pipeline_settings = pipeline.GetReductionSettings()
pipeline_settings.SetProcessSelectionSetID(set_id)
pipeline_settings.SetMergeGeometries(True)
pipeline_settings.SetKeepUnprocessedSceneMeshes(True)
pipeline_settings.SetReductionTargetTriangleRatioEnabled(False)
pipeline_settings.SetReductionTargetTriangleCountEnabled(False)
pipeline_settings.SetReductionTargetOnScreenSizeEnabled(True)
pipeline_settings.SetReductionTargetOnScreenSize(resolution)
return pipeline
def create_remeshing_pipeline(sg, set_id, resolution):
pipeline = sg.CreateRemeshingPipeline()
settings = pipeline.GetRemeshingSettings()
settings.SetProcessSelectionSetID(set_id)
settings.SetKeepUnprocessedSceneMeshes(True)
settings.SetHoleFilling( Simplygon.EHoleFilling_Disabled )
settings.SetOnScreenSize(resolution)
mapping_image_settings = pipeline.GetMappingImageSettings()
material_output_settings = mapping_image_settings.GetOutputMaterialSettings(0)
material_output_settings.SetTextureHeight(2048)
material_output_settings.SetTextureWidth(2048)
# Color caster
caster = sg.CreateColorCaster()
caster_settings = caster.GetColorCasterSettings()
caster_settings.SetMaterialChannel("Basecolor")
pipeline.AddMaterialCaster( caster, 0 )
# Metal caster
caster = sg.CreateColorCaster()
caster_settings = caster.GetColorCasterSettings()
caster_settings.SetMaterialChannel("Metalness")
pipeline.AddMaterialCaster( caster, 0 )
# Roughness caster
caster = sg.CreateColorCaster()
caster_settings = caster.GetColorCasterSettings()
caster_settings.SetMaterialChannel("Roughness")
pipeline.AddMaterialCaster( caster, 0 )
# Normal caster
caster = sg.CreateNormalCaster()
caster_settings = caster.GetNormalCasterSettings()
caster_settings.SetGenerateTangentSpaceNormals(True)
caster_settings.SetMaterialChannel("Normals")
# Change these settings to correspond to your engine
# caster_settings.SetFlipGreen(False)
# caster_settings.SetCalculateBitangentPerFragment(True)
# caster_settings.SetNormalizeInterpolatedTangentSpace(False)
pipeline.AddMaterialCaster( caster, 0 )
return pipeline
def process_file(tmp_file):
# Change these settings to correspond to your engine
# sg.SetGlobalDefaultTangentCalculatorTypeSetting(Simplygon.ETangentSpaceMethod_MikkTSpace)
sceneImporter = sg.CreateSceneImporter()
sceneImporter.SetImportFilePath(tmp_file)
sceneImporter.RunImport()
scene = sceneImporter.GetScene()
transparent_set = sg.CreateSelectionSet()
opaque_set = sg.CreateSelectionSet()
transparent_set_id = scene.GetSelectionSetTable().AddSelectionSet(transparent_set)
opaque_set_id = scene.GetSelectionSetTable().AddSelectionSet(opaque_set)
split_opaque_transparent(scene, scene.GetRootNode(), transparent_set, opaque_set)
pipeline_transparent = create_reduction_pipeline(sg, transparent_set_id, 900)
pipeline_opaque = create_remeshing_pipeline(sg, opaque_set_id, 900)
pipeline_opaque.RunScene(scene, Simplygon.EPipelineRunMode_RunInThisProcess)
pipeline_transparent.RunScene(scene, Simplygon.EPipelineRunMode_RunInThisProcess)
scene_exporter = sg.CreateSceneExporter()
scene_exporter.SetExportFilePath(tmp_file)
scene_exporter.SetScene(scene)
scene_exporter.RunExport()
sg = simplygon_loader.init_simplygon()
print(sg.GetVersion())
file_path = 'c:/Temp/_intermediate.glb'
bpy.ops.export_scene.gltf(filepath = file_path, use_selection=True)
process_file(file_path)
bpy.ops.import_scene.gltf(filepath=file_path)
sg = None
gc.collect()