Scientists Create Innovative Algorithm for Single-Step Fixing of Digital Image Flaws

Digital images, such as lifesaving medical scans or snapshots from a phone camera, have a significant role in the manner in which people communicate information. However, digital images are prone to different kinds of imperfections such as missing pixels, blurriness, color corruption and grainy noise.

A research team headed by a computer scientist from the University of Maryland has developed an innovative algorithm that integrates artificial neural networks to simultaneously apply a broad array of corrections to distorted digital images. Due to the fact that the algorithm can be “trained” to identify how a typical, uncorrupted image will look like, it has the potential to correct multiple imperfections in a single image.

The team, including researchers from the University of Bern, Switzerland, investigated their algorithm by capturing undistorted images of higher quality, deliberately instigating acute degradations, and then adopting the algorithm to fix the imperfections. In various aspects, the algorithm performed better than other competitive methods, almost transforming the images to their original state.

The team presented its results at the 31^st Conference on Neural Information Processing Systems in Long Beach, California, on December 5^th, 2017.

Traditionally, there have been tools that address each problem with an image separately. Each of these uses intuitive assumptions of what a good image looks like, but these assumptions have to be hand-coded into the algorithms. Recently, artificial neural networks have been applied to address problems one by one. But our algorithm goes a step further—it can address a wide variety of problems at the same time.

Matthias Zwicker, the Reginald Allan Hahne Endowed E-Nnovate Professor in Computer Science,UMD and Senior Author

Artificial neural networks are a form of artificial intelligence algorithm based on the human brain’s structure. They can gather together sequences of functions depending upon input data, through a procedure similar to the manner in which a human brain grasps new information. For instance, human brains have the ability to grasp a new language by means of repeated exposure to words and sentences in particular contexts.

Zwicker and his team could “train” their algorithm by introducing it to a huge database of uncorrupted images of higher quality largely adopted for studies with artificial neural networks. Due to the fact that the algorithm has the potential to absorb a huge amount of data and extrapolate the complicated parameters defining images (e.g. variations in texture, color, light, shadows and edges), it has the ability to visualize how an undistorted, typical image would look. Subsequently, it can identify and correct deviations from these typical parameters in a new image.

“This is the key element. The algorithm needs to be able to recognize a good image without degradations. But for an image that is already degraded, we can’t know what this would look like,” stated Zwicker, also from the University of Maryland Institute for Advanced Computer Studies (UMIACS). “So instead, we first train the algorithm on a database of high-quality images. Then we can give it any image and the algorithm will modify the imperfections.”

According to Zwicker, many other research teams are striving to achieve the same results and have developed algorithms that accomplish similar outcomes. Many teams observed that when their algorithms were made to only remove noise, or graininess, from an image, the algorithm would also automatically correct various other flaws. However, Zwicker’s team put forth an innovative theoretical interpretation for this impact that results in an extremely uncomplicated but efficient algorithm.

When you have a noisy image, it is randomly shifted or jittered away from a high-quality image in all possible dimensions. Other degradations, such as blurring for example, diverge from the ideal only in a subset of dimensions. Our work revealed how fixing noise will bring all dimensions back in line, allowing us to address several types of other degradations, like blurring, at the same time.

Matthias Zwicker

Zwicker also stated that although the innovative algorithm is powerful, it still has scope for further advancement. At present, the algorithm functions well in terms of correcting easily identifiable “low-level” structures in images (e.g. sharp edges). The researchers hope to push the algorithm to recognize and repair “high-level” features, including complex textures such as water and hair.

To recognize high-level features, the algorithm needs context to understand what is in the image. For example, if there is a face in an image, it’s likely that the pixels near the top are probably hair. It’s like assembling a jigsaw puzzle. If you’re only looking at one piece, it’s hard to place that part of the image in context. But once you find where the piece belongs, it’s much easier to recognize what the pixels represent. It’s quite clear that this approach can be pushed much further still.

Matthias Zwicker