When creating physics meshes, it's typically a good idea to optimize the original heavily, to make calculations cheaper. You would also want to keep material information in the optimized object, in order to spawn appropriate effects, play correct sounds etc. Combining the vertex color caster and remeshing is a great starting point to achieve super-simple geometry that checks the boxes for good physics meshes.