Local Binary Patterns as Texture Descriptors

Textures

In image processing, textures refer to a set of calculated metrics designed to quantify the features of area or a specific region. Often, these contain information describing the spatial arrangement and variation of color or intensities in an image (or in a selected part). They can be artificially created or are observed in captured images of natural scenes. They can be random or stochastic or periodic, especially in the case of artificially created textures. Because it is a feature of an area or a spatial variation, a pixel does not have any texture. 

Texture analysis is used in many image segmentation and classification research. It has many industrial applications as well, such as surface inspection, remote-sensing, and bio-medical image analysis.

There are two approaches when analysing textures: structured and statistical. In the structured approach, textures are considered as containing repeated occurrences of primitive texture elements called texels. The statistical approach treats textures as a quantitative measure describing the arrangement of intensities in a region. In general the statistical approach is more widely used.

Local Binary Pattern

There are lots of ways to statistically analyze and extract textures. This is a challenging task because often, real world textures are often not uniform due to variations in scale, orientation, or other visual features. At the minimum, texture extraction methods must be gray-scale invariant such that uneven illumination does not cause great variability in the classification.

The local binary pattern (LBP) operator was developed in order to allow two-dimensional surface textures to be described by two complementary measures: local spatial patterns and gray scale contrast.

In the original implementation by Ojala et al (1996), the  LBP  encodes the spatial variation around the 3x3 neighborhood surrounding the pixel. Each surrounding pixel also has a weight, corresponding to an exponent of 2. A threshold is applied to all the  neighboring pixels. Neighbors with grey values greater than that of the center are set to 1, 0 otherwise. The LBP value of the pixel is then the sum of the 2^P values of the pixels that are equal to 1 after the threshold was applied. Contrast can also be computed as a ratio between pixels having a grey value larger than that of the center pixel. One advantage of the LBP is that it is not computationally expensive and very simple to implement.

The LBP values then map to 256 values (0-255) and can be used for comparison or as part of the classification process. This operator was also extended by Ojala et. al in 2002 to handle different neighborhood sizes.

Textures (MIT Visual Textures database)

We selected 5 samples from 3 categories of textures in the analysis. These texture data sets are available at MIT Vision Texture Database. We've computed the LBP and the corresponding thresholds of each of the 15 images. Here are the results

Bark

Barks 0001 and 0002 have very similar histogram plots, as well as their LBP patterns. They are almost indistinguishable. There are some gaps in the histogram may indicate that they are reducible. In the next section we will perform the histogram reduction based on a enhanced LBP operator.

Brick

The same thing is observed here in Brick 000 and 0001 having very similar histogram profiles as well as the near indistinguishable LBP patterns. Brick 0003 is just Brick 0002 enlarged at a specific part. The peaks in the histogram of 0002 and 0003 are almost correlated except that changes in magnitude are very apparent especially the LBP values between 112 and 148.

Fabric

Fabric 0000 and 0001, as well as 0002 and 0003 are almost identical except 0001 and 0003 have different orientations compared to 0000 and 0002 (respectively). Histograms of 0000 and 0001 are nearly identical and indistinguishable. While in 0002 and 0003 the changes in the frequencies are much more pronounced (between LBP values of 0 and 52). Meanwhile Fabric 0004 is completely different from the other four.

Histogram reduction via LBP_P,R^rui2 operator

There are rotational symmetries LBP values (or LBP neighborhood set) . These are related to the number of transitions int the binary digits of the LBP, i.e. 0 to 1 and vice-versa. Thus patterns of 0000000 have 0 transitions, 0000001 and 11000000 have 2 transitions. A pattern is uniform if it contains at most 2 transitions. 

Extending the LBP operator further, we obtain

where U(LBP_P,R) computes the number of binary transitions. riu2 in  LBP_P,R^riu2

implies only uniform patterns are uniquely labeled. The rest of non-uniform patterns are labeled P+1. Thus the histogram contains at most P+2 values (including 0), where uniform patterns are labelled 0 to P, while non-uniform patterns are labelled P+1.

Reduced histograms

Overall, the reduced histograms are easier to compare. Bark 0001 and 0002 are clearly shown to have very similar if not identical textures. Bark 0000 and 0003 are also comparable to some extent.

Bricks 0000 and 0001 have nearly identical histograms.  0002 and 0003 have somewhat similar histograms, i.e. some features are found in both histograms. However there is a pronounced dip in the LBP_P,R^riu2  = 6 peak. Is this the effect of scale? Possibly. When you zoom in or magnify a region within a rough texture, it becomes smoother.

Finally, for the reduced histograms of the fabric textures, the slight differences in the orientation of each texture did not contribute to very significant changes in the reduced histogram.

Summary

We have investigated the use of local binary patterns as texture descriptors. The LBP operator is computationally inexpensive and we were able to demonstrate their use in classifying different classes of textures.

References

• Matti Pietikäinen (2010) Local Binary Patterns. Scholarpedia, 5(3):9775.

• Wikipedia contributors, Image Texture, Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/wiki/Image_texture

• Wikipedia contributors, Local binary patterns, Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/wiki/Local_binary_patterns

• Timo Ojala, Matti PietikaÈinen, Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, NO. 7, July 2002

• Soriano, M (2015), Texture Features, lecture notes distributed in Physics 301 - Special Topics in Experimental Physics (Advanced Signal and Image Processing) at National Institute of Physics, University of Philippines Diliman, Quezon City on 3 September 2015.

Useful R tips #2

Bit manipulation

Using &, |, and !, >>, <<,  ^ on values and expressions in R do not produce the expected binary operation result. These are all used for logical expressions (except for ^, which is used to raise a value or an expression to an exponent) and not bit-wise operations. Instead, the following functions are available:

NEG (negation, transform bit 0 to 1, and vice versa)

bitwNot(a) 

AND ( a & b )

bitwAnd(a, b) 

OR ( a | b )

bitwOr(a, b)

XOR ( a ^ b )

bitwXor(a, b)

SHL, Shift-Left ( a << b )

bitwShiftL(a, n)

SHR, Shift-Right ( a >> b )

bitwShiftR(a, n)

IN, NOT IN operations on vectors and arrays

A quick and easy way to check if elements of one array are present in another array is the use of the %in% operator. For example

a = c(1,2,3,4,5)

b = c(2,4,6)

Elements of a that are present in b

> a[a %in% b]

[1] 2 4

Elements of a that are not present in b

> a[!(a %in% b)]

[1] 1 3 5

Similary, for b, the elements are present in a

> b[b %in% a]

[1] 2 4

Elements of b that are not present in a

> b[!(b %in% a)]

[1] 6