A Novel RGB-Depth Imaging Technique for Food Volume Estimation

Abstract
Evaluating nutrient intake among patients in a hospital is crucial, as it can accelerate their recovery process. Estimating calorie intake can be achieved by monitoring the quantity of food consumed by patients both before and after their meals. This approach involves various methods, including the use of digital scales, the Comstock level, and digital imaging techniques. Nonetheless, these techniques have their own limitations, particularly the risk of subjective assessments. To reduce errors arising from human factors, an objective evaluation is proposed. This paper introduces a new technique for measuring volume using RGB-Depth images. The method incorporates image segmentation and edge detection in RGB images to correspond with the depth image. Subsequently, the segmented regions and boundaries from the depth images are converted into point clouds. The volume of interest is calculated by fitting the point cloud to an ellipsoid. The lowest Mean Average Percentage Error (MAPE) is 2.73. It indicates that the proposed method is sufficient to measure the food volume.
Downloads
References
J. M. Fontana, M. Farooq, and E. Sazonov, “Detection and characterization of food intake by wearable sensors,” in Wearable Sensors, Elsevier, 2021, pp. 541–574.
Y. A. Sari, J. M. Maligan, and A. F. Prakoso, “Improving the Elementary Leftover Food Estimation Algorithm by Using Clustering Image Segmentation in Nutrition Intake Problem,” in 2020 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM), IEEE, 2020, pp. 435–439.
B. Nagajayanthi, M. Neebha, M. Sagayam, A. A. Elngar, and others, “New calculation of calorie content and determining nutritional level from day-to-day intake of food using Image Processing,” 2022.
F. M. Perna et al., “Muscular grip strength estimates of the US population from the national health and nutrition examination survey 2011–2012,” Journal of strength and conditioning research, vol. 30, no. 3, p. 867, 2016.
Y.-C. Liu, D. D. Onthoni, S. Mohapatra, D. Irianti, and P. K. Sahoo, “Deep-Learning-Assisted Multi-Dish Food Recognition Application for Dietary Intake Reporting,” Electronics, vol. 11, no. 10, p. 1626, 2022.
H. Li and G. Yang, “Dietary nutritional information autonomous perception method based on machine vision in smart homes,” Entropy, vol. 24, no. 7, p. 868, 2022.
J. Sak and M. Suchodolska, “Artificial intelligence in nutrients science research: a review,” Nutrients, vol. 13, no. 2, p. 322, 2021.
A. Graikos, V. Charisis, D. Iakovakis, S. Hadjidimitriou, and L. Hadjileontiadis, “Single image-based food volume estimation using monocular depth-prediction networks,” in International Conference on Human-Computer Interaction, Springer, 2020, pp. 532–543.
H. Yu, K. Lee, and G. Morota, “242 Development of image analysis pipeline to predict body weight in pigs,” Journal of Animal Science, vol. 98, no. Supplement_4, pp. 177–178, 2020.
R. G. Bland, D. Goldfarb, and M. J. Todd, “The ellipsoid method: A survey,” Operations research, vol. 29, no. 6, pp. 1039–1091, 1981.
“Nima Moshtagh (2022). Minimum Volume Enclosing Ellipsoid (https://www.mathworks.com/matlabcentral/fileexchange/9542-minimum-volume-enclosing-ellipsoid), MATLAB Central File Exchange. 取得済み November 4、2022.”.
Intel® RealSenseTM Product Family D400 Series, “Intel Realsense Datasheet.” Apr. 13, 2022.
X. Wang, H. Mao, X. Han, and J. Yin, “Vision-based judgment of tomato maturity under growth conditions,” African Journal of Biotechnology, vol. 10, no. 18, pp. 3616–3623, 2011.
Y. A. Sari and S. Adinugroho, “Tomato ripeness clustering using 6-means algorithm based on v-channel otsu segmentation,” in 2017 5th International Symposium on Computational and Business Intelligence (ISCBI), IEEE, 2017, pp. 32–36.
L. G. Khachiyan and M. J. Todd, “On the complexity of approximating the maximal inscribed ellipsoid for a polytope,” Cornell University Operations Research and Industrial Engineering, 1990.

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under an Attribution 4.0 International (CC BY 4.0) that allows others to share — copy and redistribute the material in any medium or format and adapt — remix, transform, and build upon the material for any purpose, even commercially with an acknowledgment of the work's authorship and initial publication in this journal.