A new approach to training videos for manufacturers and fabricators


Someone records a machine expert performing a task. After this, the video is automatically uploaded to an AI engine that indexes and edits the content into digestible segments.

Metal fabrication’s acute labor shortage isn’t new, and neither is the finger-pointing. Kids grew up fiddling not with carbureted engines but with smartphones and computers. Vocational education has been on the decline for years. Manufacturing has an image problem.

Fab shops that can’t find experience might hire people based on soft skills, including peoples’ eagerness to learn. But how are these people taught, exactly? Quite often the learning process is distinctly old-school. Someone shadows an experienced press brake or laser cutting machine operator to slowly learn the ropes over time. Problem is, what if a newbie has a question, and the veteran he’s shadowing becomes impatient? He has parts to produce, after all, and the new guy’s just slowing him down.

Many at this point talk about company culture, leadership, and other vague concepts, but then stop short at a concrete solution. Sam Zheng, CEO of DeepHow, says his company has found one.

Launched in 2018 by Zheng, Wei-Liang Kao, and Patrik Matos, DeepHow offers a kind of automated video indexing and editing. By opening a mobile app, a person can record an expert on the floor performing certain tasks. The recorded content is automatically uploaded to a secure cloud, where an artificial intelligence (AI) engine identifies the different steps of the task and then cuts the video into digestible segments. New employees then can view (and review multiple times) the video on a website or mobile app. Viewing behavior in turn feeds the AI engine, so it can edit content more effectively over time.

Think of it like a kind of AI-driven Khan Academy for the fab shop floor. Unlike Khan, though, DeepHow is built not on general knowledge but on task-specific know-how. The idea is that people might learn better not just by shadowing but by viewing and reviewing instructive video content over and over. That content isn’t just made and posted like a random internet video (it’s always on a secure server). According to the company, content is instead segmented and edited in a unique way, with AI drawing from datapoints that reveal how the viewer learns best. As this highly customizable content library builds, so does a fabricator’s ability to develop shop floor talent.

Human-Machine Mismatch

As a researcher in academia and at Siemens Corporate Research Innovation Lab in Princeton, N.J., Zheng has a background in engineering psychology, studying how people and machines interact.

“When I led my digital innovation projects, I basically saw how we were introducing all these complex automated systems to the shop floor. There was a thinking that automation could solve all problems, instead of dealing with actual people. That wasn’t working.”

Fab shops employ machines and people both young and old. A fabricator might have a new press brake complete with a touchscreen control showing 3D bending simulation. But just a few feet away, someone else might be operating a dinosaur of a press brake. It still works just fine for certain parts, and besides, a few of the shop’s veterans are comfortable operating it. Newer employees won’t touch the old machine, though, and the shop veterans aren’t comfortable with the newer machine either. This person-machine mismatch, Zheng explained, leads to mountains of operational headaches, and it’s a problem that can’t be solved with automated machines alone.

Even worse, training regimens vary widely from shop to shop. The FABRICATOR columnist Steve Benson, president of ASMA LLC, a Salem, Ore.-based press brake training firm, has seen it all. Some companies offer solid training in blueprint reading and in the use of precision measurement tools. They’re still newbies when they start making parts. But they’re no stranger to the machine manual, either, and they know how to read and calibrate a caliper and other measurement tools.

Still other fabricators follow the trial-by-fire training philosophy. As Benson put it in this month’s Bending Basics column, some rookies start their day after some all-too-brief orientation in which the lead person says, “Push this button and step on this pedal 300 times.”


Sam Zheng, CEO and co-founder of DeepHow, has a background in engineering psychology, studying how people interact with technology.

Every machine and process in metal fabrication comes with its own learning curve, and every person experiences that curve differently. As fabricators everywhere undergo their digital transformations, they begin to analyze production patterns and hidden inefficiencies.

Such tracking is becoming increasingly granular. Production software can reveal actual machine cycle times (versus just job clock-in and clock-out times in enterprise resource planning software) and correlate that information to the people running the machines. This way managers and supervisors can dig deep through the data and start to recognize patterns—not just in the flow of products but in the performance of people: These people process certain jobs faster than anyone else; why?

As training software platforms emerge, managers can start asking questions. How did these people outperform everyone else? How did they learn the process, and what road did they take to become so proficient so quickly? Today, managers can base their answers not just on a hunch, but on real data.

About Humans and Machines

Zheng recalled a visit to an Anheuser-Busch plant when he saw a large bulletin board that showed the faces of everyone in the plant, arranged by their experience. “There was a huge cluster of people with two to three years of experience, and a big cluster of people with 20 to 30 years of experience, many of them clearly approaching retirement. And then there was no one in the middle.”

That age gap has become all too common at many manufacturers, Zheng said, and it presents yet another challenge to the labor shortage. It means fabricators must ensure that know-how transfers to the next generation as soon as possible.

Still, why and how was that age gap created in the first place? Many point to globalization and automation, but as Zheng pointed out, the nature of the automation played a role too. “The skill requirements have changed over the past 20 years,” he said. “In the past, we all talked about having trade skills. Now, you need to know about HMIs [human-machine interfaces].”

Some of the latest metal fabrication machines are starting to have HMIs that at least borrow graphical elements from consumer electronics, especially smartphones—but it wasn’t always that way, and many clunky HMIs remain.

“We know these HMIs are horrible,” Zheng said. “They’re so far away from typical user experiences people have in their everyday life.”

Analytics also has raised hurdles in recruitment. “Every operation today is so data-driven. Everyone says, ‘We need to measure productivity.’ People need to develop skills around that.”

In manufacturing, people tend to think in terms of human or machine; an operation is either manual or automated. Of course, manufacturing really is about human and machine, and beyond a few user-friendly HMIs, that human-machine connection hasn’t been scrutinized or improved upon. The machine and human work together as a system.

Video training using AI technology in a manufacturing setting

An expert wearing a headset brings viewers through a machine setup procedure.

A machine that fabricates extraordinarily quickly might look impressive, but it isn’t very practical if few people can learn to operate the thing. If an expert sets up a straightforward job on a press brake with a modern control, a relative novice probably can run the job. But how does that novice eventually become an expert?

“Younger workers might spend weeks or months shadowing the expert around the shop, but can we capture the know-how that expert has? And can we digitize it and make it so that expert is available all the time? We want to digitize and map the know-how so that it is very easy to share it,” Zheng said.

Meet Stephanie

DeepHow uses an AI engine, which the company calls Stephanie, to read audio and video content that users upload. “Stephanie takes the audio and video as input,” Zheng said, “and the first thing she does is try to understand the complexity and model the workflow. She then indexes [the information] and segments that complexity into digestible chunks, providing step-by-step guidance. Stephanie can understand experts speaking different languages and with strong accents. She also can translate the audio into different languages for later distribution.

“On the learning side, we are using AI in a good way. Instead of AI optimizing recommendations on social media, to grab your attention and get you addicted, all just to sell more ads, we’re using AI to improve how we present information to learners, considering where they are in their skills training. This helps create a personalized training.”

The AI engine uses research based on deep learning, an area of AI that (oversimplified) mimics the neural network of the human brain. The technology adapts what Zheng called a “pre-training model” to the industrial environment. Think of pre-training like high school. “That’s where you learn all these generic topics,” Zheng said. “Afterward, you become more focused, and you need to learn a specific area.” That’s effectively what Stephanie does as she analyzes a specific manufacturing task.

A Know-how Map

Zheng differentiates knowledge from know-how. Knowledge is general, while know-how is that knowledge applied to achieve a specific task. When people have both, they have a better chance of building successful, fulfilling careers in the fab shop.

This is where what Zheng calls the “know-how map” comes into play. He compared it to Google Maps. Years of data have made the maps app smarter and smarter. “You just type your destination, and it will show you how to get there, step by step, based on current conditions.”

The same concept applies to the know-how map. Say a press brake operator wants to know how to operate another press brake with a different kind of controller. Stephanie knows his current status—that is, the know-how the operator already has, and how he learned it (what videos he watched, what parts he repeated, how often, and how long). Drawing from a video library full of those “digestible chunks” of know-how, Stephanie gives the operator the best “route” to learn the other control.

The nature of that route—how long, how many twists and turns—depends on how similar the technology is. To learn panel bending might require a different route that’s a little longer; to learn laser cutting might require a route that’s longer still.

What About the People?

Today DeepHow technology is being used at Stanley Black & Decker, Anheuser-Busch, and elsewhere in plants in the U.S. and abroad. In most cases, users record experts with their smartphone, though some utilize more advanced cameras. Using wearable video recording devices can be especially valuable when having the operator’s point of view helps convey instructions. The technology also can incorporate illustrative graphics.

Moreover, it can connect to companywide performance and productivity dashboards. Eventually, Zheng sees such AI becoming the “missing piece” of the smart factory, one that helps align ever-changing technology with the people who need to get the most out of that technology.

“When it comes to Industry 4.0, everyone talks about technology and making machines smarter,” Zheng said. “But what about the people?”

Leave a Reply

Your email address will not be published. Required fields are marked *