Supplementary material for ICCV submission 667:

Understanding deep features with computer generated imagery

Experiment on synthetic bicolor patches

This experiment is similar to the color experiments presented in sections 4.1 and 4.2 (we felt the latter were more important to the paper). We studied the embedding of uniformly-colored central squares on a uniformly-colored background. We chose the size of the central square to be half of the size of the image. The results are presented in the table below. The resulting global embedding is much higher dimensional (not reported in the table, to be compared to the 20 dimensions for a single color reported for the single color experiment in the paper) than for a single color, with dimensions varying between 80 and 174 depending on the networks.

A first observation is that the features do not separate very well the foreground and background color representation. We observe that the residual is significant, explaining more than 30% of the feature variance in all cases, and even more than 40% for VGG. This may be related to a relative interpretation of the colors. Another interesting fact is that for all the networks, the variance associated to the background color is higher than the variance associated to the foreground. The difference is more striking for the Places network fc7 layer (3.8x versus 2x for AlexNet fc7, 1.8 for VGG fc7 and 2.4x for Places pool5). It would be an interesting subject for further study to determine if this is related to the fact that the color of the background of an image is especially important for scene classification, while the color of the foreground is less relevant.

The tables report the relative variance (first line) and dimension (second line) of our foreground, background, and residual feature. The dimensions reported corresponds to the PCA dimensions necessary to explain more than 95% of the variance.