Written by Jesper Tingvall, Product Expert, Simplygon

Disclaimer: The code in this post is written using version 9.1.42900.0 of Simplygon and Unity 2020.3.23f1. If you encounter this post at a later stage, some of the API calls might have changed. However, the core concepts should still remain valid.

Introduction

Physics meshes are used in almost every 3d game. It is also very common that we want to know the different materials of the physics object for spawning certain impact effects; leather, metal stone and so on. In this post will showcase how to create a physics mesh and keep material data in Unity as a texture.

We are going to be using this chair asset. It has two materials; metal and leather (highlighted).

Prerequisites

This example will use the Simplygon integration in Unity, but the same concepts can be applied in all other integrations of the Simplygon API.

Problem to solve

We want to have a well optimized physics mesh and data about what material certain parts has. We do not want to split the mesh up into several tiny meshes for each material nor do we want to have a very detailed physics mesh to include tiny material details like screws.

Solution

The solution is to bake material data into a texture which we can use along with the optimized collision mesh.

Helper functions for mapping materials to colors

Since we are going to use a 2d texture to store material data we need to map materials to colors. First we need to define what different materials we have.

public enum ImpactMaterial
{
    Wood,
    Metal,
    Stone,
    Leather,
    Unknown
}

Each ImpactMaterial is mapped into a different color. It is wise to choose as different colors as possible to make it easier to detect the different materials.

private static Dictionary<ImpactMaterial, Color> MaterialMap = new Dictionary<ImpactMaterial, Color> {
    { ImpactMaterial.Metal, Color.blue },
    { ImpactMaterial.Stone, Color.red },
    { ImpactMaterial.Wood, Color.green },
    { ImpactMaterial.Leather, Color.magenta }
};

We are now going to introduce a number of helper functions. The first one maps a material's name into ImpactMaterial. This one relies upon a material naming scheme where names start prefixed with the kind. For example materials named metal_parts and leather_parts.

public static ImpactMaterial GetMaterialFromName(string materialName)
{
    foreach (var material in MaterialMap)
    {
        if (materialName.StartsWith(material.Key.ToString(), System.StringComparison.InvariantCultureIgnoreCase))
        {
            return material.Key;
        }
    }
    return ImpactMaterial.Unknown;
}

We are introducing a measurement of distance between two colors. Since we are only interested in using this to find the closest color and not euclidean distance we can avoid using a costly square root operation.

private static float ColorDistanceSquared(Color color1, Color color2)
{
    var r = color1.r - color2.r;
    var b = color1.b - color2.b;
    var g = color1.g - color2.g;
    return r * r + b * b + g * g;
}

Using distance function above we can get the ImpactMaterial that best matches a color. This will be used each time we are going to determine the ImpactMaterial from texture.

public static ImpactMaterial GetMaterialFromColor(Color color)
{
    float distance = float.PositiveInfinity;
    ImpactMaterial closestMaterial = ImpactMaterial.Unknown;
    foreach (var material in MaterialMap)
    {
        var colorDistance = ColorDistanceSquared(material.Value, color);
        if (colorDistance < distance)
        {
            distance = colorDistance;
            closestMaterial = material.Key;
        }
    }
    return closestMaterial;
}

We are also going to add a function for mapping ImpactMaterial to color. This will be used for generating the texture.

public static Color GetColorFromMaterial(ImpactMaterial material)
{
    if (MaterialMap.ContainsKey(material))
        return MaterialMap[material];
    return Color.black;
}

Material color shading network

Using the helper functions, we are going to create a Simplygon shading network that outputs the corresponding ImpactMaterial color depending on what name the material begins with. The shading network is very simple and consists only of a single ShadingColorNode.

private static void CreateImpactMaterialShadingNetwork(ISimplygon simplygon, spMaterial material)
{
    var materialType = MaterialsHelper.GetMaterialFromName(material.GetName());
    var materialColor = MaterialsHelper.GetColorFromMaterial(materialType);
    material.AddMaterialChannel(COLOR_CHANNEL_NAME);
    var shading_node = simplygon.CreateShadingColorNode();

    shading_node.SetColor(materialColor.r, materialColor.g, materialColor.b, materialColor.a);
    material.SetShadingNetwork(COLOR_CHANNEL_NAME, shading_node);
}

Max deviation to screen size

Since we are going to use Simplygon to create a physics mesh screen size is not a good measurement of quality. We would like to decide on how many distance units the mesh is allowed to deviate from the source mesh. With the screen size metric the deviation would depend on the size of the object. This formula shows how deviation is connected to screen size. Further reading can be found here.

deviation(length) = scene_bounding_sphere_diameter(length) / on_screen_size(pixels)

This function converts from max deviation into what the remeshing processor understands; screen size.

private static uint MaxDeviationToScreenSize(float maxDeviation, spScene sgScene)
{
    var sceneDiameter = sgScene.GetRadius() * 2;
    var screenSize = (uint)Mathf.Max(MIN_SCREEN_SIZE, Mathf.RoundToInt(sceneDiameter / maxDeviation));
    return screenSize;
}

Create physics mesh

First we need to reset any transformation done to the object so it is placed in origo. Otherwise the optimized collision mesh will have the transform applied twice.

public static void CreatePhysicMeshForGameObject(GameObject gameObject, ISimplygon simplygon)
{
    // Reset position temporary
    var originalPosition = gameObject.transform.localPosition;
    var originalScale = gameObject.transform.localScale;
    var originalRotation = gameObject.transform.localRotation;
    gameObject.transform.localPosition = Vector3.zero;
    gameObject.transform.localScale = Vector3.one;
    gameObject.transform.localRotation = Quaternion.identity;

    var objectsToProcess = new List<GameObject>() { gameObject };
    var processedObjects = new List<GameObject>();

    var exportTempDirectory = SimplygonUtils.GetNewTempDirectory();

We are then creating a remeshing pipeline. It uses our MaxDeviationToScreenSize function to set OnScreenSize depending on our specified max deviation from original mesh.

    using (var sgScene = SimplygonExporter.Export(simplygon, exportTempDirectory, objectsToProcess))
    {
        using (var remeshingPipeline = simplygon.CreateRemeshingPipeline())
        using (var remeshingSettings = remeshingPipeline.GetRemeshingSettings())
        using (var mappingImageSettings = remeshingPipeline.GetMappingImageSettings())
        using (var outputMaterialSettings = mappingImageSettings.GetOutputMaterialSettings(0))
        using (var materialColorCaster = simplygon.CreateColorCaster())
        using (var colorCasterSettings = materialColorCaster.GetColorCasterSettings())
        using (var materialTable = sgScene.GetMaterialTable())
        {
            uint screenSize = MaxDeviationToScreenSize(MAX_DEVIATION, sgScene);
            Debug.Log("Using screen size " + screenSize);
            remeshingSettings.SetOnScreenSize(screenSize);

For each material in the scene we are using our CreateImpactMaterialShadingNetwork function create a shading network. We have also set SetUseMultisampling to false to get clearer boundaries between the materials. Setting SetUseAutomaticTextureSize to true allow us to automatically calculate the best texture size.

            mappingImageSettings.SetGenerateMappingImage(true);
            mappingImageSettings.SetUseAutomaticTextureSize(true);
            colorCasterSettings.SetMaterialChannel(COLOR_CHANNEL_NAME);
            colorCasterSettings.SetUseMultisampling(false);
            remeshingPipeline.AddMaterialCaster(materialColorCaster, 0);

            for (int i = 0; i < materialTable.GetMaterialsCount(); i++)
            {
                using (var material = materialTable.GetMaterial(i))
                {
                    CreateImpactMaterialShadingNetwork(simplygon, material);
                }
            }

We run the pipeline and import the result from Simplygon into Unity. Corresponding folders are created as well.

            remeshingPipeline.RunScene(sgScene, EPipelineRunMode.RunInThisProcess);

            var baseFolder = "Assets/PhysicsMeshes";
            if (!AssetDatabase.IsValidFolder(baseFolder))
            {
                AssetDatabase.CreateFolder("Assets", "PhysicsMeshes");
            }

            var assetFolderGuid = AssetDatabase.CreateFolder(baseFolder, gameObject.name);
            var assetFolderPath = AssetDatabase.GUIDToAssetPath(assetFolderGuid);

            int startingLodIndex = 0;
            SimplygonImporter.Import(simplygon, remeshingPipeline, ref startingLodIndex, assetFolderPath, gameObject.name, processedObjects);
        }
    }

After importing we are cleaning up the original asset from any colliders. We are then adding and setting up a MeshCollider component and a ImpactMaterialComponent using the created mesh and texture.

    var processedObject = processedObjects.First();
    foreach(var childCollider in gameObject.GetComponentsInChildren<Collider>())
    {
        GameObject.DestroyImmediate(childCollider);
    }

    var collider = gameObject.GetComponent<MeshCollider>();
    if (collider == null)
        collider = gameObject.AddComponent<MeshCollider>();

    var materialComponent = gameObject.GetComponent<ImpactMaterialComponent>();
    if (materialComponent == null)
        materialComponent = gameObject.AddComponent<ImpactMaterialComponent>();

    collider.sharedMesh = processedObject.GetComponent<MeshFilter>().sharedMesh;
    materialComponent.MaterialTexture = (Texture2D)processedObject.GetComponent<MeshRenderer>().sharedMaterial.mainTexture;

After processing we are moving object back to original transform. We are also deleting the created GameObject which been imported into the scene.

    gameObject.transform.localPosition = originalPosition;
    gameObject.transform.localScale = originalScale;
    gameObject.transform.localRotation = originalRotation;

    GameObject.DestroyImmediate(processedObject);
}

Menu

We are adding a menu item to Unity that processes all selected meshes and create physics meshes for them using function above.

[MenuItem("Simplygon/Create Physic Mesh")]
static void CreatePhysicMeshEntryPoint()
{
    if (Selection.objects.Length > 0)
    {
        using (ISimplygon simplygon = global::Simplygon.Loader.InitSimplygon
            (out EErrorCodes simplygonErrorCode, out string simplygonErrorMessage))
        {
            if (simplygonErrorCode == EErrorCodes.NoError)
            {
                foreach (var o in Selection.objects)
                {
                    CreatePhysicMeshForGameObject(o as GameObject, simplygon);
                }
            }
            else
            {
                Debug.Log("Initializing failed: " + simplygonErrorCode);
            }
        }
    }
}

Impact component

Once the mesh has a collider we need a way to determine what ImpactMaterial is at a certain hit location. This can be done by getting the pixel value of our generated texture and convert it into ImpactMaterial. This can be done via the helper functions we created. Getting pixels from a texture can be costly so we might want to use this function with care.

public ImpactMaterial GetMaterialAtUV(Vector2 pixelUV)
{
    pixelUV.x *= MaterialTexture.width;
    pixelUV.y *= MaterialTexture.height;

    var color = MaterialTexture.GetPixel(
        Mathf.RoundToInt(pixelUV.x),
        Mathf.RoundToInt(pixelUV.y));
    return MaterialsHelper.GetMaterialFromColor(color);
}

public ImpactMaterial GetMaterialAt(RaycastHit hitInfo)
{
    return GetMaterialAtUV(hitInfo.textureCoord);
}

Spawn impact effects

We can now use the ImpactMaterialComponent to detect the ImpactMaterial where we hit and spawn different impact effects.

private void Fire()
{
    var ray = Camera.main.ScreenPointToRay(Input.mousePosition);
    if (Physics.Raycast(ray, out var hitInfo))
    {
        var impactMaterial = hitInfo.collider.GetComponent<ImpactMaterialComponent>();
        if (impactMaterial != null)
        {
            var material = impactMaterial.GetMaterialAt(hitInfo);
            Debug.Log("Hit " + material);
            SpawnImpact(hitInfo, material);
        }
    }
}

The different effects prefabs are stored in a dictionary.

private Dictionary<ImpactMaterial, GameObject> ImpactEffectPrefabs = new Dictionary<ImpactMaterial, GameObject>();

We can spawn different effects depending on the ImpactMaterial at hit location.

private void SpawnImpact(RaycastHit hitInfo, ImpactMaterial material)
{
    if (ImpactEffectPrefabs.ContainsKey(material))
    {
        Instantiate(ImpactEffectPrefabs[material], hitInfo.point, Quaternion.LookRotation(hitInfo.normal, Vector3.up));
    }
}

Result

We have now a new menu in Unity for creating physics meshes. Select the asset we want to create a physics mesh for then go to Simplygon->Create Physics Mesh.

After processing we need to mark the resulting texture Read/Write Enabled to true. This allows us to access it through scripts. A future improvement of the script would be to mark this as true after import.

The resulting texture where each material has a different color.

The chair has a collision mesh and we can see how the texture color maps to different materials on the original asset.

Attaching DebugGun.cs to a GameObject in the scene and assigning impact prefabs allows us to shoot at the asset with different impact effects depending on where we hit.

One thing to observe is that sampling the texture upon impact could take some performance, so it might not be suitable in all use cases. Another way of baking material data into collision meshes is to put it in vertex color.

Resulting scripts

ImpactMaterial.cs

// Copyright (c) Microsoft Corporation. 
// Licensed under the MIT license. 

namespace Simplygon.Examples.UnityPhysicMesh
{
    public enum ImpactMaterial
    {
        Wood,
        Metal,
        Stone,
        Leather,
        Unknown
    }
}

MaterialsHelper.cs

// Copyright (c) Microsoft Corporation. 
// Licensed under the MIT license. 

using System.Collections.Generic;
using UnityEngine;

namespace Simplygon.Examples.UnityPhysicMesh
{
    public class MaterialsHelper
    {
        private static Dictionary<ImpactMaterial, Color> MaterialMap = new Dictionary<ImpactMaterial, Color> {
            { ImpactMaterial.Metal, Color.blue },
            { ImpactMaterial.Stone, Color.red },
            { ImpactMaterial.Wood, Color.green },
            { ImpactMaterial.Leather, Color.magenta }
        };

        public static ImpactMaterial GetMaterialFromName(string materialName)
        {
            foreach (var material in MaterialMap)
            {
                if (materialName.StartsWith(material.Key.ToString(), System.StringComparison.InvariantCultureIgnoreCase))
                {
                    return material.Key;
                }
            }
            return ImpactMaterial.Unknown;
        }

        public static ImpactMaterial GetMaterialFromColor(Color color)
        {
            float distance = float.PositiveInfinity;
            ImpactMaterial closestMaterial = ImpactMaterial.Unknown;
            foreach (var material in MaterialMap)
            {
                var colorDistance = ColorDistanceSquared(material.Value, color);
                if (colorDistance < distance)
                {
                    distance = colorDistance;
                    closestMaterial = material.Key;
                }
            }
            return closestMaterial;
        }

        private static float ColorDistanceSquared(Color color1, Color color2)
        {
            var r = color1.r - color2.r;
            var b = color1.b - color2.b;
            var g = color1.g - color2.g;
            return r * r + b * b + g * g;
        }

        public static Color GetColorFromMaterial(ImpactMaterial material)
        {
            if (MaterialMap.ContainsKey(material))
                return MaterialMap[material];
            return Color.black;
        }
    }
}

CreatePhysicMesh.cs

This script is depending on editor only functions and should be placed inside a folder named Editor or an editor only Assembly Definition.

// Copyright (c) Microsoft Corporation. 
// Licensed under the MIT license. 

using Simplygon.Unity.EditorPlugin;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;
using System.Linq;

namespace Simplygon.Examples.UnityPhysicMesh.Editor
{
    public class CreatePhysicMesh
    {
        const int MIN_SCREEN_SIZE = 20;
        const float MAX_DEVIATION = 0.05f;
        const string COLOR_CHANNEL_NAME = "diffuseColor";

        [MenuItem("Simplygon/Create Physic Mesh")]
        static void CreatePhysicMeshEntryPoint()
        {
            if (Selection.objects.Length > 0)
            {
                using (ISimplygon simplygon = global::Simplygon.Loader.InitSimplygon
                    (out EErrorCodes simplygonErrorCode, out string simplygonErrorMessage))
                {
                    if (simplygonErrorCode == EErrorCodes.NoError)
                    {
                        foreach (var o in Selection.objects)
                        {
                            CreatePhysicMeshForGameObject(o as GameObject, simplygon);
                        }
                    }
                    else
                    {
                        Debug.Log("Initializing failed: " + simplygonErrorCode);
                    }
                }
            }
        }

        public static void CreatePhysicMeshForGameObject(GameObject gameObject, ISimplygon simplygon)
        {
            // Reset position temporary
            var originalPosition = gameObject.transform.localPosition;
            var originalScale = gameObject.transform.localScale;
            var originalRotation = gameObject.transform.localRotation;
            gameObject.transform.localPosition = Vector3.zero;
            gameObject.transform.localScale = Vector3.one;
            gameObject.transform.localRotation = Quaternion.identity;

            var objectsToProcess = new List<GameObject>() { gameObject };
            var processedObjects = new List<GameObject>();

            var exportTempDirectory = SimplygonUtils.GetNewTempDirectory();

            using (var sgScene = SimplygonExporter.Export(simplygon, exportTempDirectory, objectsToProcess))
            {
                using (var remeshingPipeline = simplygon.CreateRemeshingPipeline())
                using (var remeshingSettings = remeshingPipeline.GetRemeshingSettings())
                using (var mappingImageSettings = remeshingPipeline.GetMappingImageSettings())
                using (var outputMaterialSettings = mappingImageSettings.GetOutputMaterialSettings(0))
                using (var materialColorCaster = simplygon.CreateColorCaster())
                using (var colorCasterSettings = materialColorCaster.GetColorCasterSettings())
                using (var materialTable = sgScene.GetMaterialTable())
                {
                    uint screenSize = MaxDeviationToScreenSize(MAX_DEVIATION, sgScene);
                    Debug.Log("Using screen size " + screenSize);
                    remeshingSettings.SetOnScreenSize(screenSize);

                    mappingImageSettings.SetGenerateMappingImage(true);
                    mappingImageSettings.SetUseAutomaticTextureSize(true);

                    colorCasterSettings.SetMaterialChannel(COLOR_CHANNEL_NAME);
                    colorCasterSettings.SetUseMultisampling(false);
                    remeshingPipeline.AddMaterialCaster(materialColorCaster, 0);

                    for (int i = 0; i < materialTable.GetMaterialsCount(); i++)
                    {
                        using (var material = materialTable.GetMaterial(i))
                        {
                            CreateImpactMaterialShadingNetwork(simplygon, material);
                        }
                    }

                    remeshingPipeline.RunScene(sgScene, EPipelineRunMode.RunInThisProcess);

                    var baseFolder = "Assets/PhysicsMeshes";
                    if (!AssetDatabase.IsValidFolder(baseFolder))
                    {
                        AssetDatabase.CreateFolder("Assets", "PhysicsMeshes");
                    }

                    var assetFolderGuid = AssetDatabase.CreateFolder(baseFolder, gameObject.name);
                    var assetFolderPath = AssetDatabase.GUIDToAssetPath(assetFolderGuid);

                    int startingLodIndex = 0;
                    SimplygonImporter.Import(simplygon, remeshingPipeline, ref startingLodIndex, assetFolderPath, gameObject.name, processedObjects);
                }
            }
            var processedObject = processedObjects.First();

            foreach(var childCollider in gameObject.GetComponentsInChildren<Collider>())
            {
                GameObject.DestroyImmediate(childCollider);
            }

            var collider = gameObject.GetComponent<MeshCollider>();
            if (collider == null)
                collider = gameObject.AddComponent<MeshCollider>();

            var materialComponent = gameObject.GetComponent<ImpactMaterialComponent>();
            if (materialComponent == null)
                materialComponent = gameObject.AddComponent<ImpactMaterialComponent>();

            collider.sharedMesh = processedObject.GetComponent<MeshFilter>().sharedMesh;
            materialComponent.MaterialTexture = (Texture2D)processedObject.GetComponent<MeshRenderer>().sharedMaterial.mainTexture;

            gameObject.transform.localPosition = originalPosition;
            gameObject.transform.localScale = originalScale;
            gameObject.transform.localRotation = originalRotation;

            GameObject.DestroyImmediate(processedObject);
        }

        private static void CreateImpactMaterialShadingNetwork(ISimplygon simplygon, spMaterial material)
        {
            var materialType = MaterialsHelper.GetMaterialFromName(material.GetName());
            var materialColor = MaterialsHelper.GetColorFromMaterial(materialType);
            material.AddMaterialChannel(COLOR_CHANNEL_NAME);
            var shading_node = simplygon.CreateShadingColorNode();

            shading_node.SetColor(materialColor.r, materialColor.g, materialColor.b, materialColor.a);
            material.SetShadingNetwork(COLOR_CHANNEL_NAME, shading_node);
        }

        private static uint MaxDeviationToScreenSize(float maxDeviation, spScene sgScene)
        {
            var sceneDiameter = sgScene.GetRadius() * 2;
            // From https://documentation.simplygon.com/SimplygonSDK_9.1.42900.0/concepts/deviationscreensize.html
            var screenSize = (uint)Mathf.Max(MIN_SCREEN_SIZE, Mathf.RoundToInt(sceneDiameter / maxDeviation));
            return screenSize;
        }
    }
}

ImpactMaterialComponent.cs

// Copyright (c) Microsoft Corporation. 
// Licensed under the MIT license. 

using UnityEngine;

namespace Simplygon.Examples.UnityPhysicMesh
{
    public class ImpactMaterialComponent : MonoBehaviour
    {
        public Texture2D MaterialTexture;

        public ImpactMaterial GetMaterialAtUV(Vector2 pixelUV)
        {
            pixelUV.x *= MaterialTexture.width;
            pixelUV.y *= MaterialTexture.height;

            var color = MaterialTexture.GetPixel(
                Mathf.RoundToInt(pixelUV.x),
                Mathf.RoundToInt(pixelUV.y));
            return MaterialsHelper.GetMaterialFromColor(color);
        }

        public ImpactMaterial GetMaterialAt(RaycastHit hitInfo)
        {
            return GetMaterialAtUV(hitInfo.textureCoord);
        }
    }
}

DebugGun.cs

// Copyright (c) Microsoft Corporation. 
// Licensed under the MIT license. 

using System.Collections.Generic;
using UnityEngine;

namespace Simplygon.Examples.UnityPhysicMesh
{
    public class DebugGun : MonoBehaviour
    {
        private Dictionary<ImpactMaterial, GameObject> ImpactEffectPrefabs = new Dictionary<ImpactMaterial, GameObject>();

        public GameObject MetalImpactPrefab;
        public GameObject LeatherImpactPrefab;

        private void Start()
        {
            ImpactEffectPrefabs.Add(ImpactMaterial.Metal, MetalImpactPrefab);
            ImpactEffectPrefabs.Add(ImpactMaterial.Leather, LeatherImpactPrefab);
        }

        private void Update()
        {
            if (Input.GetMouseButtonDown(0))
            {
                Fire();
            }
        }

        private void Fire()
        {
            var ray = Camera.main.ScreenPointToRay(Input.mousePosition);
            if (Physics.Raycast(ray, out var hitInfo))
            {
                var impactMaterial = hitInfo.collider.GetComponent<ImpactMaterialComponent>();
                if (impactMaterial != null)
                {
                    var material = impactMaterial.GetMaterialAt(hitInfo);
                    Debug.Log("Hit " + material);
                    SpawnImpact(hitInfo, material);
                }
            }
        }

        private void SpawnImpact(RaycastHit hitInfo, ImpactMaterial material)
        {
            if (ImpactEffectPrefabs.ContainsKey(material))
            {
                Instantiate(ImpactEffectPrefabs[material], hitInfo.point, Quaternion.LookRotation(hitInfo.normal, Vector3.up));
            }
        }
    }
}