Facebook wants to use AI to understand image content and transform the way we search

Facebook is applying computer vision and artificial intelligence (AI) technology that it developed for visually impaired people to its image search function.

Tech2 News Staff|Feb 03, 2017, 12:05:27 IST

Facebook is applying computer vision and artificial intelligence (AI) technology that it developed for visually impaired people to its image search function.Using its Lumos computer vision platform, Facebook will now enhance your image search results by actually “looking” at images and understanding the content within, reports TechCrunch.As Facebook explains on its developer blog, search today is based on tags and text. Images need to have the right tags and captions or search will simply not work.Lumos is a deep neural network, one implementation of AI that learns from experience. This system looks at images and then at the tags that people have given them. Considering that Facebook has over 1.7 billion users, there must be billions upon billions of images to learn from.

Facebook engineers were working on a project that developed ‘automatic alt text’ (AAT) for images. This text is a computer-generated description of an image and was designed to help visually impaired people. Early models would describe an image solely based on its content, without context. It could, for example, say that an image had a cat that was sitting or a light that was on, but that’s it.Today, AAT can describe things like “people walking”, “people dancing”, “people riding horses”, etc., says Facebook.[caption id="attachment_360593" align="aligncenter" width="640"]

Image: Facebook[/caption]In an example, Facebook asks us to assume that we’re looking for a person riding a horse. Lumos will first look for images of people and horses. It will then attempt to understand the relation between the horses and the people and “semantically cluster” images together. With sufficient training, Lumos will be able to determine the clusters that you’re looking for and in fact, give you images of people riding horses.The system is also smart enough, apparently, to not give you dozens of images of the same thing taken from slightly different angles. All this time, Lumos will be adding ever more enhanced AAT to images.

Why AI notetakers are raising serious privacy and security concerns

AI notetakers promise effortless meeting summaries, but experts warn they could expose confidential conversations, corporate secrets and personal voiceprints. As businesses increasingly adopt AI-powered meeting assistants, questions over data storage, privacy, consent and legal risks are becoming impossible to ignore

5 min read

China's low-cost AI models are changing the global AI race. Here's why Silicon Valley is worried

As Chinese firms continue to improve performance while keeping prices low, the AI race is no longer just about building the smartest model—it is increasingly becoming a battle over who can deliver the best value

2 min read

China's Kimi K3 challenges US AI leaders with frontier-level performance at lower cost

Chinese artificial intelligence startup Moonshot AI has unveiled its latest open-weight AI model, Kimi K3, with early results suggesting it could compete with some of the world's most advanced AI systems developed by leading US companies

2 min read

How did Instagram run ads promoting child abuse in India?

India has issued a notice to Meta after an investigation alleged that Instagram displayed paid advertisements promoting child sexual abuse material. MeitY ordered Meta to remove such Instagram ads and explain within seven days how they were approved

8 min read

Why has India halted WhatsApp’s username feature before launch?

India has halted WhatsApp’s planned username feature, citing concerns about cybercrime, impersonation, and law enforcement challenges. As Meta races to address security concerns, here’s why MeitY has paused the rollout, what the feature does, and how it could influence privacy and online safety in India

3 min read

Up Next