MIT researchers develop artificial intelligence that can suggest recipes based on photos of food

Researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) lab have teamed up with the Qatar Computing Research Institute to train an artificial intelligence system called Pic2Recipe. The AI can analyse photos of food, figure out the ingredients used in making the dish, and suggest a recipe. The researchers combed through food recipe websites to create a database of over one million recipes.The research team will present their findings at the Computer Vision and Pattern Recognition conference in Honolulu, to be held this month. Yusuf Aytar, co-author of the research paper says, "In computer vision, food is mostly neglected because we don’t have the large-scale datasets needed to make predictions. But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences."When an image is fed into the Pic2Recipe system, the AI identifies the ingredients used for the dish, and then suggests the recipes to dishes that are similar in appearance to the provided image. As one of the main themes in the database were desserts, Pic2Recipe performs particularly well when it comes to identifying dishes such as cookies or muffins. However, the AI struggles when the food items are more ambiguous, such as sushi rolls or smoothies.