According to leading scientists from the University of Surrey, teaching machine learning tools to detect particular objects in a specific image and discount others is a “game-changer” that could result in advancements in cancer detection.
Image Credit: University of Surrey
An artist’s sketch is full of individual cues that words cannot convey concisely, reiterating the phrase ‘a picture paints a thousand words’. For newer AI systems, simple descriptive words help to generate images, but none can express the individualism of the user or the exact match the user is looking for.
Yi-Zhe Song, Study Lead and Professor, Institute for People-Centred AI, University of Surrey
Song added, “This is where our sketch-based tool comes into play. AI is instructed by the artist via sketches to find an exact object and discount others. Which can be amazingly helpful in medicine, by finding more aggressive tumors, or helping to protect wildlife conservation by detecting rare animals.”
An example that scientists utilize in their paper to the conference is of the tool assisting in searching a picture full of zebras—with just a sketch of a single zebra eating to direct its search. The AI tool considers visual cues like pose and structure but relies the decisions on the exact needs provided by the amateur artist.
The ability for AI to detect objects based on individual amateur sketches introduces a significant leap in harnessing human creativity in Computer Vision. It allows humans to interact with AI from a whole different perspective, no longer letting AI dictate the decisions, but asking it to behave exactly as instructed, keeping necessary human intervention.
Yi-Zhe Song, Study Lead and Professor, Institute for People-Centred AI, University of Surrey