Multi-Task Learning for Food Identification and Analysis with Deep Convolutional Neural Networks

• 作 者: Xi-Jin Zhang Yi-Fan Lu Song-Hai Zhang
• 机 构: Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
• 出 处: 计算机科学技术学报（英文版）
• 发表日期: 2016年3期
• 关键词: multi-task learning convolutional neural network food recognition machine learni

In this paper, we proposed a multi-task system that can identify dish types, food ingredients, and cooking methods from food images with deep convolutional neural networks. We built up a dataset of 360 classes of different foods with at least 500 images for each class. To reduce the noises of the data, which was collected from the Intet, outlier images were detected and eliminated through a one-class SVM trained with deep convolutional features. We simultaneously trained a dish identifier, a cooking method recognizer, and a multi-label ingredient detector. They share a few low-level layers in the deep network architecture. The proposed framework shows higher accuracy than traditional method with handcrafted features, and the cooking method recognizer and ingredient detector can be applied to dishes which are not included in the training dataset to provide reference information for users.