September 30, 2023


The Internet Generation

Fujitsu’s AI-Video Recognition Technology to Promote Correct Hand Washing

Fujitsu Laboratories Ltd. and Fujitsu Analysis and Advancement Heart Co., Ltd. (FRDC) right now announced the progress of “Actlyzer” hand wash motion recognition technology, which leverages AI and machine learning techniques to recognize elaborate hand washing movements from video clip data captured by camera.

Graphic credit: Pixabay (Free Pixabay license)

Amidst the ongoing global COVID-19 pandemic, the great importance of hand washing as a measure to shield people’s overall health from microorganisms, influenza, and other infectious conditions is getting renewed attention around the globe. Less than new rules(one) planned to occur into effect in June 2020 in Japan, foods business operators will also be needed to employ more robust measures to ensure hygiene in accordance with intercontinental HACCP(two) foods security specifications, building an urgent will need for a non-invasive strategy to quickly and correctly validate that handwashing is carried out in a proper way.

Anticipating this, Fujitsu has expanded the recognition operate of its present “Actlyzer behavioral evaluation technology”, which can understand a assortment of subtle and elaborate human movements with out relying on large amounts of schooling data. Exclusively, Fujitsu has refined recognition capabilities for hand movements to create technology to mechanically understand sophisticated hand movements carried out during hand washing.

This technology makes it attainable to conveniently identify no matter if someone is pursuing just about every of the 6 techniques for handwashing proposed by the Japanese Ministry of Health, Labour and Welfare, lessening the variety of man-several hours needed for intrusive visual checks by inspectors for on-web page sanitation administration.

Shifting ahead, Fujitsu envisions this technology staying employed in a assortment of other contexts, which includes in healthcare facilities, universities, resorts, and venues for large gatherings, and strategies to carry out discipline trials and further investigate and progress into it as a likely remedy for its AI portfolio in the long run.

Developmental Track record

The Japanese Ministry of Health, Labour and Welfare suggests that people adhere to 6 techniques to ensure their arms are appropriately washed and avert foods poisoning as properly as the spread of infectious conditions (Determine one).

Determine one: 6 Measures for Proper Hand Washing. Graphic credit: Fujitsu

At present, foods company business workers are needed to accomplish these 6 techniques when washing their arms and validate that during just about every phase the diverse elements of their arms have been rubbed far more than a selected variety of times. To this conclude, foods company vendors fill out a test sheet and make a self-report, and workers on-web page endure visual checks by supervisors.

Mainly because verification is continue to executed manually, on the other hand, prices associated with human error, securing methods for inspectors, and many others. remain a persistent challenge. As lots of makers, which includes in the foods business, continue on the march to approach automation for inspections by making use of machine learning systems, hand washing checks characterize a key prospect for approach optimization as a result of automation.


Gesture recognition making use of deep learning is a frequent method for determining hand and finger movements. This typical method can detect numerous element points, these as joints and fingertips, from an picture of the hand, and identify the hand gesture primarily based on the positional romance of the element points (Determine two Still left).

On the other hand, a person challenge with the present technology was that when people wash their arms properly, both equally arms overlap and are lathered with soap, which obscures the detection points on the fingers and prevents exact gesture recognition (Determine two Appropriate).

Determine two: Typical hand tracking tech (Still left: Gesture recognition benefits Appropriate: Hand wash habits). Graphic credit: Fujitsu

Freshly Created Technological know-how

To take care of this challenge, Fujitsu Laboratories Ltd. and FRDC have produced a new AI technology that mechanically and correctly recognizes hand movements beneath the ailments explained earlier mentioned, expanding the recognition operate of the primary Actlyzer behavioral evaluation technology.

With the new technology, the elaborate hand movements of handwashing are captured as a mixture of hand form and repetitive rubbing motions, detected by two deep learning engines: Hand Shape Recognition and Movement Recognition (Determine 3).

Determine 3: Recognition of elaborate two-hand finger movements as a mixture of the overall form and motion styles of both equally arms. Graphic credit: Fujitsu

The two-hand form recognition engine uses a figured out design of a essential form of two arms, which is a regular form of hand motion in which arms are put on top rated of just about every other, to identify the handshape for just about every body of the picture. Focusing on the overall form solves the trouble of when fingertips and joint element points cannot be properly detected thanks to hand overlap or foam.

In addition, Fujitsu’s one of a kind AI technology “High Durability Mastering (Higher Durability Mastering)”, which can keep track of data variations, is applied to ensure that the essential form of both equally arms is recognized with higher accuracy even when the camera position or lighting variations during procedure.

The motion recognition engine uses a figured out design that detects periodically altering motion from successive frames and counts the variety of iterations as the variety of rubs from the iteration sample and its period. In addition, the benefits of these two recognition engines are fed back again to just about every other to enhance recognition accuracy.

The motion recognition engine sets a threshold benefit for the magnitude of the motion to be judged in accordance with the techniques recognized by the two-hand form recognition engine to avert detection of erroneous durations, these as hand tremors not linked to foam motion or rubbing. The two-hand form recognition engine enhances detection accuracy by filtering the judgment result making use of the repetition sample period detected by the motion recognition engine.


A hand-wash video clip data set with around 2000 variants which includes people, camera positions, and soap forms, was independently filmed and gathered for learning and evaluation. It was confirmed that the accuracy of 6 techniques of proper hand washing is typical of 95% or far more, and that the accuracy of the variety of arms rubbing movements is 90% or far more.

When functioning the technique on-web page, the omission of action could be prevented for the reason that the man or woman washing their arms always is aware how prolonged to scrub their arms and which of the 6 phases they’ve concluded for the reason that the technique determines it’s shown on the display screen till it’s concluded. In addition to this, the technique mechanically documents data with the commencing timestamp and duration of action of finishing just about every 6 techniques, and the variety of times of rubbing. (Determine four).

Determine four: Illustration of recognition screen. Graphic credit: Fujitsu

This technology automates on-web page handwashing checks for workplaces that have to have demanding hygiene command, doing away with the variety of man-several hours necessary for visual confirmation and guide recording. On top of that, for the reason that the technique doesn’t understand incorrect or incomplete hand washing, Fujitsu expects the remedy will support educate buyers and ensure standardization for the proper way of handwashing.

(one) New rules Act for Partial Revision of the Food Sanitation Act

(two) HACCP Hazard Evaluation and Vital Manage Place. A sanitation administration process that guarantees security by controlling procedures that are especially crucial for doing away with or lessening hazardous elements in the full approach from the receipt of uncooked elements to the shipment of products and solutions, primarily based on the business operator’s understanding of dangers these as contamination by foods-borne microorganisms. This is turning out to be a necessary process primarily in produced nations.

Source: ACN Newswire