Image processing pipeline will empower software companies to build advanced building maintenance and inspection solutions
Structure Reports combines the rich sensor data available from Apple’s ARKit with the 3d reconstruction capabilities of the industry leading mobile phone 3d reconstruction pipeline from URC Ventures. StucturePro enables software companies to build applications that can be used by construction workers, building inspectors, or insurance claims adjusters to successfully model buildings from iPhone imagery.
Historical attempts to build similar solutions have not made it to market because of failures in 3d reconstruction algorithms to handle extreme camera rotations, textureless surfaces, and repetitive structures breaking traditional algorithms and approaches. “After successfully launching Stockpile Reports to manage inventory using iPhones in 2012 we invested significantly in a potential solution to model residential homes and commercial buildings from 2013 to 2015”, says David Boardman CEO of URC Ventures. We went to field trials with more than one company and even went under contract to model 4,000 buildings to help plan the allocation of maintenance budget with a large government agency. At the end of the day we decided to pull back from the market due to usability issues training field users to successfully collection sufficient imagery to ensure complete 3d models every time”.
By integrating the advanced sensor data from ARKit, the URC Ventures image processing pipeline is now able to successfully handle the extreme rotations introduced by average end users, textureless surfaces such as large solid color walls, and repetitive structures such as ceiling tiles. “An untrained user taking video of an indoor scene will typically collect video with many blurry frames caused by fast camera rotations and the long image exposure times needed to handle weak indoor lighting. These blurry frames will cause breaks in vision only 3d reconstruction pipelines. By fusing inertial navigation based pose estimates from ARKit into our solution we create complete 3D models even in these challenging circumstances” says Dr. Jared Heinly, Chief Scientist of URC Ventures. “Another source of failure in vision-only reconstruction systems is repetitive scene structures like windows, doors or acoustic ceiling tiles. Using ARKit relative pose data we can eliminate many poor matches in the image pair graph and make our 3D reconstruction engine robust to repetitive features” says Heinly. “Textureless surfaces like blank white walls are also a major source of failure for vision only reconstruction systems. Using ARKit pose constraints we can tie together a reconstruction that would otherwise break when the camera views a textureless wall.”
There are other benefits beyond addressing the historical barriers to realizing solutions. Solutions now have absolute scale enabling measurements without the introduction of control points. Performance is significantly improved by leveraging scaled motions from the accelerometer in bundle adjustment.”Leveraging sensor data provides valuable reference measurements. These can significantly decrease drift in the results when fused with the vision data in the error mitigation stage of the reconstruction. This allows to estimate the volumes of objects even more accurately.” says Prof. Jan-Michael Frahm, Scientific Advisor to URC Ventures.
It is now possible to have an average person with limited training walk in and around building structures to successfully model it’s structure. “It is like magic”, says Boardman. “Solutions that would fail or break using imagery alone are now being processed into successful building models using additional sensor data from ARKit”. We look forward to re-investing in this use case and partnering with industry leading companies to bring solutions to market.
URC got its start with 2d and 3d real-time mapping for military use cases, back in the early 2000’s. In 2010 they started a new company to explore commercial use cases for their technology—URC Ventures. Subsequently, they have created an immensely powerful 3d reconstruction platform utilized by companies all around the world. The company Stockpile Reports licenses the platform to power its software, measuring billions of tons of construction materials, processing millions of images each year. They’ve done 3d reconstruction projects for the real estate, telecommunications, video game, movie, archeology, and architecture industries.
To explore licensing URC Ventures image processing pipeline or partnering on the development of solutions please contact Tony Jacobson, VP of Product, (425) 285-4303, firstname.lastname@example.org.