A Computer Vision System for the Assessment of Ice Cream Melting Behavior

Summary

Here, we present a protocol based on a computer vision system (CVS) to determine the melting behavior of multi-phase food systems.

Abstract

Melting behavior is one of the most important quality indices of ice cream. It is usually evaluated by gravimetric methods and expressed in terms of starting time and rate of melting. However, the aspect of ice cream during melting is also important because shape retention is linked to a good structure of the product. The protocol proposed here illustrates a computer vision system (CVS) that can be used to support the already existing gravitational methodology to calculate two new melting indices related to shape retention and melting rate. Pictures of ice cream during melting are taken every 15 min for a total of 90 min. Afterward, digital images are elaborated using a purposely developed image processing method to calculate ice cream area, height, and width. The ratio between height and width at each melting time, referred to the ratio at time 0 (R_t/R₀), is an index of the shape retention of ice cream, while the area at the different melting times referred to the area at time 0 (A_t/A₀) is related to the melting rate. This computer vision system allows the obtaining of highly sensitive and reliable results, and it can be applied not only to ice cream but also to different food matrices, such as whipped milk cream or egg albumen.

Introduction

Ice cream is a multi-phase system in which liquid, solid, and gas phases are strictly connected. The continuous liquid phase envelops air bubbles and ice crystals, and it contains partially crystallized fats, colloidal proteins, salts, sugars (eventually crystallized), and stabilizers. The ice cream composition varies depending on the local market requests and possible regulations. Although the processing technology affects the characteristics of the final ice cream, each constituent plays an important role in defining the product quality¹. Melting behavior is one of the most important quality indices of ice cream, considering phenomena occurring both during consumption and in the mouth. With heat penetration into the ice cream, ice crystals melt, and water diffuses and mixes with the serum phase, which can drain through the remaining structure². A fast-melting product is undesirable for comfortable eating but also for guaranteeing a higher heat-shock resistance. However, slow-melting products also indicate some defects in the formulation¹. It is known that ice cream microstructure is responsible for melting properties³, but contrasting results have been published so far, showing that knowledge about the influence of microstructural factors on melting is still limited⁴. Thus, more studies are necessary to elucidate the meltdown mechanism, which is crucial also in the design of new formulations³.

Melting behavior is usually evaluated by gravimetric methods and expressed in terms of starting time and rate of melting⁵. A given portion of ice cream is placed on a wire mesh in a controlled temperature cabinet, and the weight of the melted product is registered. From the weight-time curve, three phases can be highlighted: the lag phase during which heat penetration occurs, the fast-melting phase with the dilute serum phase flowing through the ice cream structure at the maximum rate, and the stationary phase, where most of the product has dripped out².

With the gravimetric method, slow- and fast-melting products can be recognized; however, the aspect of ice cream during meltdown is important too, because shape retention is linked to a good composition and structure of the product⁶. Thus, a procedure based on a computer vision system (CVS) can support the already existing gravitational methodology by allowing the study of the product's appearance during melting. CVSs can acquire numerous food attributes³ (e.g., size, weight, shape, texture, and color) with accurate details that cannot be observed by the human eye. Such systems are usually made of digital cameras and image processing software⁷. Indeed, a protocol based on CVSs includes two main steps: 1) image acquisition and 2) image processing. Various levels of image processing can be applied⁷, from the simplest to the more complex, such as deep-learning methods for Artificial Intelligence development^8,9. Great attention has been recently paid to CVSs in the food sector, and a high number of applications have been developed for food safety inspection, food processing monitoring, foreign object detection, and other fields. They are fast, efficient, and non-destructive, thus representing valid tools to provide consumers with safe foods of high quality¹⁰.

In the field of ice cream, an image analysis method was suggested to study ice recrystallization by optical microscopy¹¹. More recently, X-ray computed tomography images were processed to analyze the 3D microstructure of soft-porous matters, including ice cream³. However, the elaboration of simple digital Charge Coupled Device (CCD) images can present several advantages in terms of easiness of acquisition and rendering of the ice cream aspect as perceived by consumers. Some Authors show images of ice cream during melting¹², but, to the best of our knowledge, the extraction of numerical indices from the images was reported for the first time by Moriano and Alamprese¹³.

Therefore, the protocol here proposed, based on the work by Moriano and Alamprese¹³, illustrates a simple CVS that can be applied to support the already existing gravitational methodology for the study of ice cream melting behavior. A block diagram of the proposed method is illustrated in Figure 1. The use of such a system allows the calculation of two melting indices related to shape retention and the melting rate. In particular, the paper describes for the first time the detailed experimental setup and procedure for digital image acquisition during ice cream meltdown and the image processing steps. Besides, the results obtained from ice creams produced with different sweeteners (i.e., sucrose, sucromalt, and erythritol) are reported to show the potential of the method.

Figure 1: Block diagram of the proposed methodologies. Summary of the general steps for the proposed Computer Vision System and the gravimetric method for studying ice cream melting behavior. Please click here to view a larger version of this figure.

Protocol

​1. Experimental setup and procedures for melting trials

1. Ice cream sample preparation
   1. Choose transparent cups of fixed volume and shape, with lids. Cut two long stripes of baking paper about 2 cm in width, and, with the help of paper tape, fix them to the inside walls of the cup to form a cross at the bottom. Start to fill the cup with the ice cream sample by using a spatula.
      NOTE: Before starting, be sure that the ice cream has a temperature in the range of -7 °C to -12 °C so that it can be spread inside the cup. Pay attention not to create empty spaces during the cup filling by regularly checking through the cup transparency.
   2. When the cup is full, gently pull away the excess of ice cream with the spatula, creating a smooth and flat surface. Close the cup with its lid and store the sample at -30 °C for at least 24 h.
   3. Before the analysis, condition the sample at -16 °C for 24 h.
2. Setup of the melting trial and image acquisition
   1. Set a thermostatic cabinet at 20 ± 1 °C. Insert a digital scale in the cabinet and connect it to a computer with software for the registration of weight as a function of time.
   2. Place a graduated cylinder on the digital scale and reset the weight. Over the cylinder, place a hanging funnel to help collect the melted ice cream. The setup of the melting trial is shown in Figure 2.
      NOTE: The funnel should be suspended over the cylinder to avoid exceeding the full scale of the digital scale. To avoid instability of the funnel during the sample positioning at the beginning of the analysis, it should be fixed to the cabinet shelf over the cylinder.
   3. Set a camera with a tripod in front of the cabinet door at a defined height and distance to have the best reliable framing of the sample.
      NOTE: Be sure that the camera is well aligned with the ice cream sample to avoid parallax errors when processing the image.
3. Melting trial
   1. Prepare a metal wire-mesh screen equipped with a size reference. Take the ice cream cup from the freezer, remove the lid, and start gently to put a spatula between the ice cream and the cup walls to detach the sample from the container.
      NOTE: The ice cream cup must be removed from the freezer only when the previous steps (until step 1.2.3) are completed to avoid meltdown before data registration.
   2. While detaching the ice cream sample from the cup walls, keep the baking paper stuck on the surface of the ice cream.
   3. When the whole ice cream sample has been detached from the cup walls, gently pull the baking paper ends to extract the ice cream and lay it down on the metal wire-mesh screen along with the baking paper stripes. Then, carefully remove the baking paper stripes from the ice cream surface.
      NOTE: Pay attention not to alter the ice cream shape when performing steps 1.3.1-1.3.3.
   4. Place the metal wire mesh screen with the ice cream sample on the funnel in the cabinet. Using the digital camera on the tripod, take the first picture (t0) of the ice cream sample with the cabinet door open, paying attention to also shoot the size reference. Close the cabinet door and start recording the gravimetric data every minute with the software connected to the scale.
      NOTE: Do not use flash to avoid shadows in the image, and use a very contrasting background to improve ice cream segmentation. Check the focus of the acquired picture; if the image is out of focus, take immediately a new one.
   5. Record the gravimetric data for 90 min (one registration per minute) and take a picture of the ice cream sample (see step 1.3.4) every 15 min (for a total of 7 pictures).

Figure 2: Melting test setup. The figure shows how to set up the melting trial in the thermostatic cabinet: Put a graduated glass cylinder on a digital scale to collect and weigh the melted ice cream. The ice cream sample is laid on a metal wire mesh screen on a funnel suspended over the cylinder. Please click here to view a larger version of this figure.

2. Image processing for melting indexes' calculation

1. Image processing
   1. Download the digital images from the camera memory card and save them as .tiff or .jpg files without compression.
   2. Use the File > Open commands of the image analysis software to open the ice cream images. Start processing the first image (t0), correcting the rotation if needed (commands: Edit > Rotate).
   3. Use the size reference acquired in each picture (see steps 1.3.1. and 1.3.4.) to spatially calibrate the image (commands: Measure > Calibration > Spatial > Image), converting pixels in millimeters (Supplementary Figure 1).
   4. Select the rectangular area of interest (AOI), including the ice cream sample and avoiding the edge of the metal wire mesh screen (commands: Edit > New AOI > Rectangular). Crop the AOI and convert it into a grey scale (color depth, 8) using the commands Edit > Covert to > Grey Scale 8 (Supplementary Figure 2).
   5. Apply the Best Fit filter that automatically adjusts brightness and contrast (commands: Enhance > Equalize > Best Fit).
   6. Go to the Measure > Count/Size window of the software and open the Measure window to select the following parameters: Area, Box Height, and Box Width. Then click OK to close this window (Supplementary Figure 3).
   7. Select the manual measurement (flag Manual in the Count/Size window) and click on Select Ranges to set the histogram values to segment the bright shape of the ice cream exactly. Close the Segmentation window (Supplementary Figure 4).
   8. Click on Count in the Count/Size window of the software to measure the three parameters selected in step 2.1.6. If bright objects other than the ice cream sample are counted, use an area threshold to filter the objects (commands: Measure > Select measurements > Area; adjust start and end ranges). View the results of measurement by clicking View > Measurement Data (Supplementary Figure 5). Use File > Data to Clipboard to copy the measurement results and paste them into a spreadsheet of data management software.
   9. Repeat steps from 2.1.2 to 2.1.8 for each image collected during the ice cream meltdown.
2. Melting indexes evaluation
   NOTE: From now on, the size parameters measured from the ice cream images will be indicated as follows: A, area; H, box height; W, box width (Supplementary Figure 6).
   1. By using the H and W data calculated in the previous step 2.1 at the different times t during melting (0 min, 15 min, 30 min, 45 min, 75 min, and 90 min), calculate the shape retention index (R_t) according to equation 1:
           (1)
   2. According to equation 2 and equation 3, refer R and A data calculated at each time to the corresponding index at time 0 (R₀ and A₀) and plot the obtained results as a function of time as shown in Figure 3A, B. Trend of A_t as a function of time is related to the melting rate of ice cream.
           (2)
           (3)

Supplementary Figure 1: Image spatial calibration. (A) Go to the window Measure > Calibration > Spatial of the image analysis software. Select New, then flag Image to open the Scaling window. The reference length in the unit to convert pixels (e.g., millimeters) is indicated. (B) Carefully overlap the green bar with the reference portion corresponding to the indicated length and click OK. Please click here to download this File.

Supplementary Figure 2: Cropping AOI and converting it into a grayscale. (A) Conversion of the Area of Interest (AOI) into the grayscale and (B) the resulting image. Please click here to download this File.

Supplementary Figure 3: Selection of the parameters to be measured. In the Select Measurement window, the parameters to be measured can be selected; for the ice cream meltdown evaluation, area, box width, and box height must be selected. Please click here to download this File.

Supplementary Figure 4: Segmentation of the ice cream sample. In the “Segmentation” window it is possible to select the histogram ranges to be considered to cover exactly the area of the ice cream shape. Please click here to download this File.

Supplementary Figure 5: Filtering objects and the Count function. The red lines highlight the recognized bright objects. By applying the count function and opening the “View, Measurement data” window, the results of the selected parameters will be shown (A). To filter only the ice cream shape, it is possible to select a minimum and a maximum area range in the “Select Measurement” window, thus counting only the parameters of one object (B). Please click here to download this File.

Supplementary Figure 6: Shape retentionindex (R). Box height (H_t, red dotted line) and box width (W_t, black solid line) used for the shape retention index (R) calculation are shown. Please click here to download this File.

Figure 3: Shape and area retention curves. Example of an ice cream (A) shape and (B) area retention curves, in which R_t/R₀ and A_t/A₀ average values are plotted over time; error bars correspond to the standard deviation values obtained by the analysis replicates. Please click here to view a larger version of this figure.

3. Elaboration of gravimetric data

1. At the end of the melting trial (90 min), save the spreadsheet with the weight of the melted sample (grams) per each minute of the analysis obtained by the software connected to the digital scale.
2. Open the weight spreadsheet in a data management software to create a plot of the melted weight (grams) as a function of time (minutes), thus obtaining the melting curve of the ice cream sample.
3. Select data in the linear portion of the melting curve and calculate the least squares regression line, registering equation 4 and the regression coefficient (R²).
        (4)
4. The slope value of the regression line (m) is the ice cream melting rate (grams/minute). Calculate the starting time of melting (t_s; minute) as the x-intercept (when y = 0) as follows:
        (5)

Figure 4: Gravimetric curve. Example of an ice cream melting curve obtained by the gravimetric method. The original curve is shown in red; the selected series of data in the linear portion are shown in green; the calculated regression line is shown in black dots. The equation and the coefficient of determination (R²) of the regression line are also shown. Please click here to view a larger version of this figure.

NOTE: To have reliable results to be statistically analyzed, replicate the whole procedure of melting trial and image processing at least three times for each sample.

Results

As an example of the proposed CVS outputs, results of meltdown analyses for three different ice cream formulations are shown, compared with data obtained from the gravimetric method. In particular, the melting behavior of ice creams made with different sweeteners (i.e., sucrose, sucromalt, and erythritol) was studied.

Table 1 and Figure 5A show the results of the shape retention index (R_t/R₀) for the three ice cream samples d...

Discussion

The proposed CVS allows the calculation of the shape and area retention indexes of ice cream samples during melting, besides visualizing the melting process. It can be coupled with the traditional gravimetric method applied to assess the melting behavior of ice cream⁵, to obtain results related to the aspect of the ice cream. This is very important because consumers evaluate its quality also based on the visual appearance of the product, and the ability to keep the shape during melting is linked t...

Materials

Name	Company	Catalog Number	Comments
Cabinet	Cavallo s.r.l.	FTX700	Location for the melting test
Digital camera	Sony Group Corp	DSC-S650
Digital scale	Gibertini Elettronica	EU-C 4002 LCD
ImagePro Plus 7.0	Media Cybernetics, Inc	N/A	Image analysis elaboration software
Microsoft Excel	Microsoft	N/A	Data and graphical elaboration
Scalecom	Gibertini Elettronica	N/A	Digital scale software acquisition
Tripod	Manfratto	#055