EnterpriseIS - Machine Dreams creating Computer Vision

Machine Dreams: The application of machine learning within Asset Management

Following on from our introduction to Machine Dreams at Mainstream 2019, we caught up with Toby Tremayne, Chief Technology Officer, to learn more about the technology and how it could help companies to improve the way they manage their assets.

What is Machine Dreams and what does it do?

Toby: Machine Dreams is all about delivering on the promises of machine learning and computer vision, only faster and cheaper. The name comes from the fact that in computer vision / object recognition, the pre-requisite data (photos, images, illustrations) isn’t always available, so we synthesize it by getting the machine to “dream it up”.

The process starts with our simulator, Morpheus, which generates super high-resolution synthetic images of objects that we want to train the computer to detect, by remixing scenes, assets and defects, lighting, and angles based on parameters set by the user or asset owner. Essentially this allows us to build 3D models with realistic textures and lighting, to create a library of images that we otherwise wouldn’t be able to find, or may not even exist. As it’s simulated, we have complete control over every aspect of the image, meaning we can produce pixel perfect annotations and segmentation, with all of the labelling and training data you need to train an object recognition model.

This means there is no requirement to gather super expensive asset image libraries and we can skip the equally expensive, laborious and error prone step of labelling of the imagery and we can do it as many times as we need to, with as many images as we need to perfect our detection models.

When the data has been simulated and the training completed (on detection), the resulting model will then be able detect defects in real photos and footage, which can be used at the core of an automated remote inspection system, for predictive asset maintenance or condition monitoring.

We are also using Morpheus to help us generate larger models, through trialling different methods to develop the kind of recognition models we need for ubiquitous mixed reality, so we can recognise hundreds of thousands of objects, not just a few.

Essentially, we want to help people get jump started in their machine learning projects, to enable them to succeed when data is expensive or doesn’t exist, and development budgets are tight.

What got you started in thinking about developing Machine Dreams initially?

Toby: I’ve been using tools like the Unity game development engine to build enterprise applications for a while, and some of the object recognition tasks I wanted to experiment with couldn’t find adequate data sources for. I came across a number of projects that had been limited by the same issue, so I started to look into the idea of synthetic data, wondering if it would work. After speaking with a number of businesses an opportunity came up to give it a try and it worked surprisingly well. So I retooled some old ideas and built Morpheus. In my mind it was a tool designed to help me build something else, but we’ve discovered a lot of need for it in various industries and a surprising amount of interest in the platform.

What are the next steps in building Machine Dreams and what sort of help could people in industry provide?

Toby: We want to provide a platform anyone can use to iterate and re-iterate on machine learning models, to make these future looking projects more achievable. This includes trying to build components for the Spatial Web, Mixed Reality and the emerging technological landscape, by taking advantage of the massively reduced cost of experimenting with synthetic data. Morpheus is branching out into various other forms of simulation, like biological scans, ultrasounds, thermal imaging and alternative spectroscopy, and we’re looking at also incorporating Unity’s own machine learning agents with Morpheus to create self-optimizing systems that can be used for economic modelling/design.

As we grow the platform, we really want to understand the pain points that can be solved across various industries by machine learning and where we can help. Synthesizing data for far-flung or difficult to reach assets, generating defects and variations you that can’t get enough images for is only one aspect. As we’re learning from each person we talk to, there are problems everywhere that can be solved by teaching the computer to spot or predict things for you and so we’re eager to build a detection tool that can help solve them.

Things we’d like to know are, what are the issues you have that are related to; inspections, defect detection, asset management, or safety, and what you think could be automated to reduce costs, improve safety or create efficiencies. Or just tell us your ideas or dreams for automation or problem solving so we can continue to learn, and then help you teach the machines.

What are you working on currently?

Toby: We’re working to set-up a pilot project with a large asset owner, where we are looking at simulating wear and tear and damage on various surfaces for building inspections, thermo-graphic imagery and traffic light defect detection. Our main focus is engaging with industry to better understand real world problems being faced in sectors like mining, energy, defence and logistics. These projects will help us round out our initial feature set and continue to develop the platform to suit industry users.

The application of new technologies, like machine learning, will continue to improve the way we manage assets in the future. Over time, these technologies and applications will shape the way asset owners operate and maintain their equipment and systems. As asset management professionals, an important part of our role is to remain up to date with technology developments and to understand the manner in which these tools can be applied to assist clients to manage their assets.To this end, we will be staying in touch with the Machine Dreams team to keep track on how this technology continues to develop and applied to different industries. Thanks again for your time Toby.

If you’d like to know more or can help the Machine Dreams out with the provision of industry knowledge, get in touch with Toby via the Machine Dreams website @ https://machinedreams.com.au/contact/ .

Get in touch with us at EnterpriseIS by calling our main office on (02 -4271-7774) to speak with Dave or through our website ( www.enterpriseis.com.au/contact-us/ )