One powerful tool we can use to create 3d assets is photogrammetry. In this discipline, we supply a photogrammetry engine with a data set, which can be either a collection of standard 2d photographs or a collection of LiDAR scans, and the engine will process these to create a 3d model. This has been used as a tool to create photorealistic assets in both the games industry and in AR and VR.
Apple Object Capture
With Apple’s inclusion of a LiDAR scanner in it’s iPhone12/13 Pro phones, and in its new iPad Pro, these devices are set up to provide the perfect tool for creating point cloud data for photogrammetry. And Apple have lept onto this with their Object Capture API
https://developer.apple.com/augmented-reality/object-capture/
Capturing Reality
This subsidiary company of Epic Games provide great options for photogrammetry which I had the chance to try out. These consist of the desktop application, RealityCapture which provides a complete set of tools for creating assets from input data. And RealityScan, currently in beta, which builds off of the technology of Apple’s LiDAR technology to provide a mobile solution to photogrammetry.
RealityCapture
As I currently do not own a device with Apple’s LiDAR scanning technology, my solution to experimenting with photogrammetry came from using a humble set of smartphone camera images. The RealityCapture software gives us the option of using LiDAR scans or plain photos, so that was perfect for me. Images are processed by an engine, most probably guided by a computer vision/machine learning solution, and an output 3d model can be swiftly generated by feeding the software a large collection of photographs.
To say this experiment was done from a small collection of around 50 photographs, what the software does is impressive. I can see that getting a high-quality, production ready asset will take a lot of trial an error in order to discover the correct techniques to capture an object. Certainly, a LiDAR solution, as opposed to photographs will probably yield much better results as a 3 dimensional point-field is a much better starting point than a mere set of images! RealityCapture is capable of processing hundreds of input photographs. I suspect that increasing the information in a photograph data set could also be key to great results.
robodojo 