Patent Grant
10248876
The present application claims priority to and the benefit of Indian Provisional Patent Application Serial No. 201641022034, filed Jun. 27, 2016 and entitled CYCLE OPTIMAL DENSE HAAR LIKE FEATURE CALCULATION FOR EMBEDDED PROCESSORS, the entirety of which is incorporated herein by reference.
This disclosure relates generally to Haar-like feature calculations, especially dense Haar-like feature calculations, associated with object detection for advanced driver assistance system (ADAS) applications. More specifically, this disclosure relates to an aspect of Haar-like feature calculations that implements a processor-based method for determining a summation of pixel characteristics for a rectangular region of a digital image. In particular, the disclosed method permits multiple words in the rectangular region of an integral image to be loaded into the processor in an aligned manner even if the base storage location for the multiple words is not aligned for single instruction multiple data (SIMD) operations by the processor. It is understood that the processor-based method for determining the summation of pixel characteristics for the rectangular region can be used for other purposes beyond those described herein regarding Haar-like feature calculations and ADAS applications. For example, the method can be used for weighted summation, where weight can be constant for rectangular window or it can vary within the rectangle based on the pixel location inside the rectangle. Where there is only one rectangle is there in processing and weights are decided based on pixel location in the rectangle then the process becomes image level 2D convolution. Thus, this technique can also be used in 2D image convolution.
Haar-like features along with AdaBoost (i.e., adaptive boosting) techniques are popular methods for object detection and classification in computer vision applications. For example, these techniques can be used in pedestrian detection, traffic sign recognition or any other kinds of object detection in ADAS applications. Haar-like features are calculated at a particular location of a digital image using weighted sum of image pixels for rectangular regions of the image. Weights are applied to the summed pixels for the particular regions of the Haar-like feature based on learned aspects from object detection and classification for similar features and similar images. Weights and feature are learned from training (e.g., AdaBoost training).
Dense Haar-like feature calculations require the feature to be calculated at each pixel location in an image or areas of interest within the image by sliding the Haar-like feature by one pixel in raster order through the area. Typically, raster order refers to a left-to-right sequence along each row of the image or area and sequentially moving down from an uppermost row to a lowest row of the image or area.
A popular method to calculate pixel characteristics under a rectangular region is to calculate an integral image representation of the raw image and utilize pixel summations in the integral image instead of pixels of the raw image. Each element of an integral image may be 32 bit (one word) for normal use of application. In general, the performance and efficiency of Haar-like feature calculations are heavily influenced by processor data loads. For example, a Haar-like feature calculation may require data loads from non-aligned locations of a buffer storing the integral image because Haar feature size can be any suitable number of pixels (e.g., odd or even pixel dimensions). Align or non-align words are associated with an alignment boundary. In the current context, the alignment boundary is 64 bits (i.e., two words). Anything that is not at the boundary of 64 bits is not aligned.
A problem is that the data loads required in a Haar-like feature calculation may not utilize the maximum capability of the processor (e.g., C6000 Digital Signal Processor (DSP)), where non-aligned data reads are costlier than aligned data reads. This problem is amplified for dense Haar feature calculations that are performed on each pixel location of the image. This problem also exists when rotated Haar-like features are calculated.
In one aspect, a method of determining a summation of pixel characteristics for a rectangular region of a digital image is provided. In one embodiment, the method includes: determining if a first base address for a first data element in an integral image buffer is aligned for a first single instruction multiple data (SIMD) operation by a processor embedded in an electronic assembly configured to perform Haar-like feature calculations, wherein the first data element represents a first corner of a pre-selected rectangular region of an integral image, wherein the integral image is a representation of a pre-selected digital image, wherein the integral image is formed by data elements stored in the integral image buffer on a storage device accessible to the processor; loading the first data element from the integral image buffer to the processor when the first base address is aligned for the first SIMD operation; and loading an offset first data element of an offset integral image from an offset integral buffer on the storage device to the processor when the first base address is non-aligned for the first SIMD operation, wherein the offset first data element represents the first corner of the pre-selected rectangular region; wherein the offset data elements of the offset integral image are defined by the corresponding data elements of the integral image and stored at address locations in the offset integral buffer that are offset right or left by one address location such that a select data element in the integral image buffer at a select address that is non-aligned for a select SIMD operation by the processor is located at a select offset address in the offset integral buffer that is aligned for the select SIMD operation.
In another embodiment, the method of determining a summation of pixel characteristics for a rectangular region of a digital image includes: determining if first, second, third, and fourth base addresses for respective first, second, third, and fourth data elements in an integral image buffer are aligned for respective first, second, third, and fourth SIMD operations by a processor embedded in an electronic assembly configured to perform Haar-like feature calculations, wherein the first, second, third, and fourth data elements respectively represent upper left, upper right, lower left, and lower right corners of a pre-selected rectangular region of an integral image, wherein the integral image is a representation of a pre-selected digital image, wherein the integral image is formed by data elements stored in the integral image buffer on a storage device accessible to the processor; loading the first, second, third, and fourth data elements from the integral image buffer to the processor when the respective first, second, third, and fourth base address is aligned for the respective first, second, third, and fourth SIMD operation; and loading offset first, offset second, offset third, and offset fourth data elements of an offset integral image from an offset integral buffer on the storage device to the processor when the respective first, second, third, and fourth base address is non-aligned for the respective first, second, third, and fourth SIMD operation, wherein the offset first, offset second, offset third, and offset fourth data elements respectively represent the upper left, upper right, lower left, and lower right corners of the pre-selected rectangular region; wherein the offset data elements of the offset integral image are defined by the corresponding data elements of the integral image and stored at address locations in the offset integral buffer that are offset right or left by one address location such that a select data element in the integral image buffer at a select address that is non-aligned for a select SIMD operation by the processor is located at a select offset address in the offset integral buffer that is aligned for the select SIMD operation.
In another aspect, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium storing program instructions that, when executed by a processor embedded in a processor-controlled electronic assembly configured to perform Haar-like feature calculations, cause the processor-controlled electronic assembly to perform a method of determining a summation of pixel characteristics for a rectangular region of a digital image. In one embodiment, the method includes: determining if first, second, third, and fourth base addresses for respective first, second, third, and fourth data elements in an integral image buffer are aligned for respective first, second, third, and fourth SIMD operations by a processor embedded in an electronic assembly configured to perform Haar-like feature calculations, wherein the first, second, third, and fourth data elements respectively represent upper left, upper right, lower left, and lower right corners of a pre-selected rectangular region of an integral image, wherein the integral image is a representation of a pre-selected digital image, wherein the integral image is formed by data elements stored in the integral image buffer on a storage device accessible to the processor; loading the first, second, third, and fourth data elements from the integral image buffer to the processor when the respective first, second, third, and fourth base address is aligned for the respective first, second, third, and fourth SIMD operation; and loading offset first, offset second, offset third, and offset fourth data elements of an offset integral image from an offset integral buffer on the storage device to the processor when the respective first, second, third, and fourth base address is non-aligned for the respective first, second, third, and fourth SIMD operation, wherein the offset first, offset second, offset third, and offset fourth data elements respectively represent the upper left, upper right, lower left, and lower right corners of the pre-selected rectangular region; wherein the offset data elements of the offset integral image are defined by the corresponding data elements of the integral image and stored at address locations in the offset integral buffer that are offset right or left by one word address location such that a select data element in the integral image buffer at a select address that is non-aligned for a select SIMD operation by the processor is located at a select offset address in the offset integral buffer that is aligned for the select SIMD operation, the method further comprising: calculating a summation of pixel characteristics for the pre-selected rectangular region using a regional area algorithm accessible to the processor by subtracting the corresponding data elements for the upper right corner and the lower left corner from a sum of the corresponding data elements for the upper left corner and the lower right corner.
In the drawings, like reference numerals refer to like elements throughout, and the various features are not necessarily drawn to scale.
This disclosure describes various embodiments of a process, electronic assembly, and system for determining a summation of pixel characteristics for a rectangular region of a digital image. The various embodiment can be implemented to perform Haar-like feature calculations, including dense Haar-like feature calculations. The Haar-like feature calculations can be used in conjunction with detection of objects of interest within the digital image. The objects of interest may include edges, lines, and center-surround objects. Various combinations of Haar-like feature calculations can be used to detect faces, pedestrians, traffic signs, and other objects of interest.
With reference to
For example, the value for the data elements in Intimg [ ][ ] of
Data element value (DEV) (0,0)=pixel characteristic value (PCV) (0,0);
DEV (0,1)=DEV (0,0)+PCV (0,1);
DEV (0,2)=DEV (0,1)+PCV (0,2);
DEV (0,3)=DEV (0,2)+PCV (0,3);
DEV (0,4)=DEV (0,3)+PCV (0,4);
DEV (1,0)=DEV (0,0)+PCV (1,0);
DEV (1,1)=DEV (1,0)+PCV (0,1)+PCV (1,1);
DEV (1,2)=DEV (1,1)+PCV (0,2)+PCV (1,2);
DEV (1,3)=DEV (1,2)+PCV (0,3)+PCV (1,3);
DEV (1,4)=DEV (1,3)+PCV (0,4)+PCV (1,4);
DEV (2,0)=DEV (1,0)+PCV (2,0);
DEV (2,1)=DEV (2,0)+PCV (0,1)+PCV (1,1)+PCV (2,1);
DEV (2,2)=DEV (2,1)+PCV (0,2)+PCV (1,2)+PCV (2,2);
DEV (2,3)=DEV (2,2)+PCV (0,3)+PCV (1,3)+PCV (2,3);
DEV (2,4)=DEV (2,3)+PCV (0,4)+PCV (1,4)+PCV (2,4);
DEV (3,0)=DEV (2,0)+PCV (3,0);
DEV (3,1)=DEV (3,0)+PCV (0,1)+PCV (1,1)+PCV (2,1)+PCV (3,1);
DEV (3,2)=DEV (3,1)+PCV (0,2)+PCV (1,2)+PCV (2,2)+PCV (3,2);
DEV (3,3)=DEV (3,2)+PCV (0,3)+PCV (1,3)+PCV (2,3)+PCV (3,3);
DEV (3,4)=DEV (3,3)+PCV (0,4)+PCV (1,4)+PCV (2,4)+PCV (3,4);
DEV (4,0)=DEV (3,0)+PCV (4,0);
DEV (4,1)=DEV (4,0)+PCV (0,1)+PCV (1,1)+PCV (2,1)+PCV (3,1)+PCV (4,1);
DEV (4,2)=DEV (4,1)+PCV (0,2)+PCV (1,2)+PCV (2,2)+PCV (3,2)+PCV (4,2);
DEV (4,3)=DEV (4,2)+PCV (0,3)+PCV (1,3)+PCV (2,3)+PCV (3,3)+PCV (4,3);
and
DEV (4,4)=DEV (4,3)+PCV (0,4)+PCV (1,4)+PCV (2,4)+PCV (3,4)+PCV (4,4).
Using the integral image to calculate a feature value for a shaded rectangular region, a value for the 5×5 shaded rectangular area of the Intimg [ ][ ] of
Summed Rectangular Region (SRR)=DEV (0,0)+DEV (4,4)−DEV (0,4)−DEV (4,0).
As shown in the rectangular area algorithm, only four data element values must be read when the feature value is calculated using the integral image of
With reference to
Feature Value (FV)=w0*((DEV (2,2)+DEV (4,7))−(DEV (2,7)−DEV (4,2)))+w1*((DEV (2,4)+DEV (4,5))−(DEV (2,5)−DEV (4,4))).
It is noted that the exemplary Haar-like line feature calculation above requires eight reads of DEVs from the integral image to determine the sum of pixel characteristics for two rectangular regions (A, B) and applies predetermined weights (w 0, w 1 ) to the two summed rectangular regions. The four corners of the first rectangular region (A) are represented by data elements (2,2), (2,7), (4,2), and (4,7). The four corners of the second rectangular region (B) are represented by data elements (2,4), (2,5), (4,4), and (4,5). For example, the first weight (w0) can be selected to emphasize darker pixel characteristics where the feature to be detected is expected to be darker. Similarly, the second weight (w1 ) can be selected to emphasize lighter pixel characteristics where the feature to be detected is expected to be lighter.
With reference again to
A consideration in managing performance and efficiency of Haar-like feature calculations is the bottlenecking that may occur when loading data elements of an integral image into a processor. One of the goals of this disclosure is to exploit effective utilization of data load capabilities of processors. For example, the TMS320C66x Digital Signal Processor (DSP) by Texas Instruments uses a 32-bit data bus for word reads and 32-bit register pairs (i.e., 64 bits) to accommodate doubleword reads (i.e., two 32-bit word reads). For one word loads to the C66x DSP, the load operation is represented as one cycle (.D)+one cycle (.T), where .D and .T are functional units on the processor that is involved in data read operation. For doubleword loads to the C66x DSP, the load operation is represented as one cycle (.D)+one cycle (.T) if the base address for the doubleword is doubleword-aligned with the processor. However, for doubleword loads to the C66x DSP, the load operation is represented as one cycle (.D)+two cycles (.T) if the base address for the doubleword is not doubleword-aligned with the processor. Thus, the best capability of the C66x DSP is 0.25 cycle per word load for doubleword loads when the base address for the doublewords are doubleword-aligned with the processor.
A conventional method for Haar-like feature calculations can work on two locations at a time when the processor can load two words (e.g., doublewords) in a single instruction. For example, the Texas Instruments' C6000 series processors can work on two locations at a time because they can load two words in a single instruction (i.e., single instruction multiple data (SIMD) operations). For example, with reference again to
With reference again to
Next FV=w0*((DEV (2,3)+DEV (4,8))−(DEV (2,8)−DEV (4,3)))+w1*((DEV (2,5)+DEV (4,6))−(DEV (2,6)−DEV (4,5))).
For the next feature value, the two rectangular regions are shifted two data element to the right. The four corners of the next first rectangular region are data elements (2,4), (2,9), (4,4), and (4,9). The four corners of the next second rectangular region are data elements (2,6), (2,7), (4,6), and (4,7). For doubleword reads, all pointers jump by two in order to perform a next doubleword read operation for the next computation. The use of doubleword reads facilitates dense Haar-like feature calculations because two parallel computations are set up for consecutive Haar-like features.
The parallel Haar-like feature calculations for data elements (2,2) and (2,3) can be determined as follows:
FV (2,2)|Next FV (2,3)=(w0|w0)*(((Intimg (2,2)|Intimg (2,3))+(Intimg (4,7)|Intimg (4,8)))−((Intimg (2,7)|Intimg (2,8))−(Intimg (4,2)|Intimg (4,3))))+(w1|w1)*((Intimg (2,4)|Intimg (2,5))+(Intimg (4,5)|Intimg (4,6)))−((Intimg (2,5)|Intimg (2,6))−(Intimg (4,4)|Intimg (4,5))).
Another way of presenting the doubleword reads and the feature value and next feature value computations is presented below in separate calculations:
FV (2,2)=w0*((Intimg (2,2)+Intimg (4,7))−(Intimg (2,7)−Intimg (4,2)))+w1*((Intimg (2,4)+Intimg (4,5))−(Intimg (2,5)−Intimg (4,4))); and
Next FV (2,3)=w0*((Intimg (2,3)+Intimg (4,8))−(Intimg (2,8)−Intimg (4,3))+w1*((Intimg (2,5))+Intimg (4,6))−(Intimg (2,6)−Intimg (4,5)).
With continued reference to
The further feature value for (2,4) can be calculated using the same algorithm as feature values (2,2) and (2,3):
Further FV=w0*((DEV (2,4)+DEV (4,9))−(DEV (2,9)−DEV (4,4)))+w1*((DEV (2,6)+DEV (4,7))−(DEV (2,7)−DEV (4,6))).
Likewise, the even further feature value for (2,5) can be calculated using the same algorithm as feature values (2,2), (2,3), and (2,4):
Even Further FV=w0*((DEV (2,5)+DEV (4,10))−(DEV (2,10)−DEV (4,5)))+w1*((DEV (2,7)+DEV (4,8))−(DEV (2,8)−DEV (4,7))).
For the further feature value, the two rectangular regions are shifted one data element to the right of the rectangular regions for the next feature value. The four corners of the further first rectangular region are data elements (2,4), (2,9), (4,4), and (4,9). The four corners of the further second rectangular region are data elements (2,6), (2,7), (4,6), and (4,7). For the even further feature value, the two rectangular regions are shifted one data element to the right of the rectangular regions for the further feature value. The four corners of the even further first rectangular region are data elements (2,5), (2,10), (4,5), and (4,10). The four corners of the even further second rectangular region are data elements (2,7), (2,8), (4,7), and (4,8). For quadword reads, all pointers jump by four in order to perform a next quadword read operation for the next computation. The use of quadword reads facilitates dense Haar-like feature calculations because four parallel computations are set up for consecutive Haar-like features.
For example, the integral image buffer can be arranged such that data elements with an even “X” coordinate can have an address that is aligned for multiple data word read operations by the processor and data elements with an odd “X” coordinate can be considered non-aligned. Thus, even if the integral image pitch is even and the base address for the initial data element is doubleword-aligned, it is possible that some of the doubleword reads are non-aligned. For example, vector Intimg (4,7)|Intimg (4,8); vector Intimg (2,7]|Intimg (2,8); vector Intimg (4,5)|Intimg (4,6); vector Intimg (2,5)|Intimg (2,6)) are non-aligned when data elements at odd “X” coordinates are non-aligned for the integral image buffer.
Two doubleword data can be loaded in a single cycle if a base address is doubleword-aligned. For Texas Instruments' C66X processor, this results in 0.25 cycles per word read. If the data is not doubleword-aligned, the C66X processor will take 0.5 cycles per word load. Use of the non-aligned integral image buffer exploits the more efficient data load capability of the C66x processor. When N pixels are worked together, to read N words (32 bit) in a single cycle. N will depend on how many words can be read in a single instruction for a specific processor. For the C66x processor, N=2. For the aligned offset, a pointer is selected from the doubleword-aligned integral image buffer. For the non-aligned offset, the pointer is selected from the word-aligned integral image buffer. This makes the load from the word-aligned integral image buffer also doubleword-aligned.
With reference to
With reference to
The C6000 series processor can perform doubleword reads from address locations that are aligned for SIMD operations by the processor as well as from address locations that are non-aligned for SIMD operations. However, doubleword reads from address locations that are aligned for SIMD operations are more efficient (i.e., faster) than doubleword reads from address locations that are non-aligned for SIMD operations.
With reference to
A modified algorithm for doubleword read operations and Haar-like feature calculations can be implemented using the A integral image buffer (
The parallel Haar-like feature calculations for data elements (2,2) and (2,3) can be determined as follows:
FV (2,2)|Next FV (2,3)=(w0|w0)*(((IntimgA (2,2)|IntimgA (2,3))+(IntimgNA (4,7)|IntimgNA (4,8)))−((IntimgNA (2,7)|IntimgNA (2,8))−(IntimgA (4,2)|I IntimgA (4,3))))+(w1|w1)*((IntimgA (2,4)|IntimgA (2,5))+(IntimgNA (4,5)|IntimgNA (4,6)))−((IntimgNA (2,5)|IntimgNA (2,6))−(IntimgA (4,4)|IntimgA (4,5))).
Here, a|b represent a vector with two 32-bit data elements associated with a doubleword read operation. The entire rectangular region is processed in the same manner. If the algorithm is implemented for dense Haar-like feature calculations, this same processing is repeated as the rectangular region is shifted through an entire area of interest, which may include the entire integral image or select portions thereof. After each output computation, all pointers jump by two for the next calculation. Thus, the rectangular region is shift two pixels in the horizontal direction instead of pixel-by-pixel by using the doubleword read operation.
Another way of presenting the doubleword reads and the feature value and next feature value computations that make use of the aligned (A) integral image buffer (
FV (2,2)=w0*((IntimgA (2,2)+IntimgNA (4,7))−(IntimgNA (2,7)−IntimgA (4,2)))+w1*((IntimgA (2,4)+IntimgNA (4,5))−(IntimgNA (2,5)−IntimgA (4,4))); and
Next FV (2,3)=w0*((IntimgA (2,3)+IntimgNA (4,8))−(IntimgNA (2,8)−IntimgA (4,3))+w1*((IntimgA (2,5))+IntimgNA (4,6))−(IntimgNA (2,6)−IntimgA (4,5)).
The aligned (A) integral image buffer is stored such that the initial data element for the rectangular area of interest is placed at a base address that is aligned for doubleword operations by the processor. Similarly, the non-aligned (NA) integral image buffer is stored such that the initial data element for the rectangular area of interest is placed at base address that is non-aligned for doubleword operations by the processor. Hence, data elements in the aligned (A) integral image buffer at odd locations that are non-aligned for doubleword operations become aligned when read from the non-aligned (NA) integral image buffer. This results in maximum read utilization from processor. Pointers for feature value calculations are decided at the start of core processing loop and remain at the same pitch value during the whole image processing operation. The extra cost of the image processing disclosed herein is the larger memory requirement associated with storing the integral image in multiple storage buffers. However, the buffer requirement does not change once a desired capacity is established even if a processor can handle other than two word read in a single instruction. The buffer requirement will increase by two times if integral image is of 16 bits. Thus, three duplicate copies of integral image will be needed along with original integral image. This method of determining a summation of pixel characteristics for a rectangular region of a digital image can be implemented for dense operation of Haar-like feature computations (i.e., feature values being evaluated at each pixel of a desired area or the entire image).
With reference to
The system may also be configured to determine a summation of pixel characteristics for a rectangular region that is rotated in a positive direction in relation to the digital image. For example, the pixels of the original digital image may be processed using a rotation algorithm to produce an original image that is rotated +45 degrees about a central axis. Pixels of the +45 degree digital image are stored in in a +45 degree digital image buffer. The pixels of the +45 degree digital image are processed using the integral image algorithm to produce a +45 degree integral image with data elements that correspond to the pixels of the +45 degree digital image. Data elements of the +45 degree integral image are stored in a +45 degree integral image buffer. The data elements of the +45 degree integral image buffer may be stored in address locations for which a reference data element of a rectangular region of interest that is at a −45 degree orientation in relation to the +45 degree integral image is aligned for doubleword operations by the processor. The data elements of the +45 degree integral image buffer are copied using an EDMA process to produce a +45 degree offset integral image buffer. Data elements of the +45 degree offset integral image buffer are stored and shifted (i.e., offset) one address location to the left or right in relation to address locations for the data elements of +45 degree integral image buffer. The shifting results in the reference data elements in the +45 degree integral image buffer being stored in an address location that is non-aligned for doubleword operations by the processor. The shifting also results in data elements in the +45 degree integral image buffer that are non-aligned for doubleword operations by the processor being aligned for doubleword operations by the processor when they are read from the +45 degree offset integral image buffer.
The system may also be configured to determine a summation of pixel characteristics for a rectangular region that is rotated in a negative direction in relation to the digital image. For example, the pixels of the original digital image may be processed using a rotation algorithm to produce an original image that is rotated −45 degrees about a central axis. Pixels of the −45 degree digital image are stored in in a −45 degree digital image buffer. The pixels of the −45 degree digital image are processed using the integral image algorithm to produce a −45 degree integral image with data elements that correspond to the pixels of the −45 degree digital image. Data elements of the −45 degree integral image are stored in a −45 degree integral image buffer. The data elements of the −45 degree integral image buffer may be stored in address locations for which a reference data element of a rectangular region of interest that is at a +45 degree orientation in relation to the −45 degree integral image is aligned for doubleword operations by the processor. The data elements of the −45 degree integral image buffer are copied using an EDMA process to produce a −45 degree offset integral image buffer. Data elements of the −45 degree offset integral image buffer are stored and shifted (i.e., offset) one address location to the left or right in relation to address locations for the data elements of −45 degree integral image buffer. The shifting results in the reference data elements in the −45 degree integral image buffer being stored in an address location that is non-aligned for doubleword operations by the processor. The shifting also results in data elements in the −45 degree integral image buffer that are non-aligned for doubleword operations by the processor being aligned for doubleword operations by the processor when they are read from the −45 degree offset integral image buffer.
Current methods of dense Haar-like feature calculations only use one integral image. Additionally, current method of dense rotated Haar-like feature calculations use a single captured image and an on-the-fly pixel sum of the area is calculated. Using two integral images where the data elements of one integral image are stored in a buffer at address locations that are aligned for multiple data word read operations by a processor and a second integral image in which the data elements are stored in another buffer at address locations that are non-aligned was not contemplated by current methods. Additionally, creating two rotated image for calculation of rotated Haar-like features as well as their respective integral images at aligned address locations and at non-aligned address locations was not contemplated by current methods. One rotated image for +45 degree clockwise rotation and another rotated image for −45 degree anti-clockwise rotation was not contemplated by current methods.
The various embodiments of a system and method for determining a summation of pixel characteristics for a rectangular region of a digital image eliminate the overhead of non-aligned memory reads from a data buffer (e.g., integral image in Haar-like feature computations) by maintaining aliased/duplicate copies of the data buffer. The various embodiments also reduce on-the-fly rotated image pixel sum calculations by maintaining another integral image which was generated from a rotated image version of an original captured image. The maximum utilization of the read capability of the processor having increased efficiency for non-aligned memory reads. For example, the CPU cycle computation may drop by half.
With reference to
With reference to
With reference to
With reference to
With reference to
With reference to
With reference to
With reference to
With reference to
With reference to
With reference to
The first, second, third, and fourth corners represent a set of four corners for the preselected rectangular region. The set of four corners includes an upper left corner, an upper right corner, a lower left corner, and a lower right corner of the preselected rectangular region in any combination. At 2320, a summation of pixel characteristics for the pre-selected rectangular region is calculated using a regional area algorithm accessible to the processor by subtracting the corresponding data elements for the upper right corner and the lower left corner from a sum of the corresponding data elements for the upper left corner and the lower right corner.
With reference again to
In a further embodiment of the process 2100, the summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for an upright orientation of the pre-selected digital image. In this embodiment, orientation of the target rectangular region follows the upright orientation of the pre-selected digital image.
In another further embodiment of the process 2100, the summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for a −45 degree orientation in relation to an upright orientation of the pre-selected digital image. In this embodiment, orientation of the target rectangular region is +45 degrees in relation to orientation of the pre-selected digital image.
In yet another further embodiment of the process 2100, the summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for a +45 degree orientation in relation to an upright orientation of the pre-selected digital image. In this embodiment, orientation of the target rectangular region is −45 degrees in relation to orientation of the pre-selected digital image.
In yet another embodiment, the process 2100 also includes processing the pre-selected digital image using a rotation algorithm to form a clockwise digital image. The pre-selected digital image is formed by pixels stored in a digital image buffer reflecting an upright orientation, wherein the clockwise digital image presents the pixels of the pre-selected digital image in a clockwise image buffer reflecting a −45 degree orientation. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the clockwise digital image. The clockwise digital image is processed using a summation algorithm to form the integral image in the −45 degree orientation. The summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element. A target rectangular region of the integral image is selected for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region. Orientation of the target rectangular region is +45 degrees in relation to orientation of the clockwise digital image.
In still another embodiment, the process 2100 also includes processing the pre-selected digital image using a rotation algorithm to form a counterclockwise digital image. The pre-selected digital image is formed by pixels stored in a digital image buffer reflecting an upright orientation. The counterclockwise digital image presents the pixels of the pre-selected digital image in a counterclockwise image buffer reflecting a +45 degree orientation. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the counterclockwise digital image. The counterclockwise digital image is processed using a summation algorithm to form the integral image in the +45 degree orientation. The summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element. A target rectangular region of the integral image is selected for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region. Orientation of the target rectangular region is −45 degrees in relation to orientation of the counterclockwise digital image.
In still yet another embodiment, the process 2100 also includes generating the offset integral image in the offset integral buffer by copying the integral image from the integral image buffer such that addresses for the data elements of the offset integral image in the offset integral buffer are incremented or decremented by one address location in relation to the addresses for the corresponding data elements of the integral image in the integral image buffer.
In another embodiment of the process 2100, the first SIMD operation is a doubleword operation involving two consecutive data words associated with the first base address and a “plus one” address adjacent to the first base address. The “plus one” address is for a next data element in the integral image buffer that represents a first corner of a next rectangular region for a next Haar-like feature calculation. The next data element is loaded from the integral image buffer to the processor along with the first data element when the first base address is aligned for the first SIMD operation. The next data element is loaded from the offset integral buffer to the processor along with the first data element when the first base address is non-aligned for the first SIMD operation.
In yet another embodiment of the process 2100, the first SIMD operation is a quadword operation involving four consecutive data words associated with the first base address, a “plus one” address adjacent to the first base address, a “plus two” address adjacent to the “plus one” address, and a “plus three” address adjacent to the “plus two” address. The “plus one” address is for a next data element in the integral image buffer that represents a first corner of a next rectangular region for a next Haar-like feature calculation. The “plus two” address is for a further data element in the integral image buffer that represents a first corner of a further rectangular region for a further Haar-like feature calculation. The “plus three” address is for an even further data element in the integral image buffer that represents a first corner of an even further rectangular region for an even further Haar-like feature calculation. The next, further, and even further data elements are loaded from the integral image buffer to the processor along with the first data element when the first base address is aligned for the first SIMD operation. The next, further, and even further data elements are loaded from the offset integral buffer to the processor along with the first data element when the first base address is non-aligned for the first SIMD operation.
With reference to
In another embodiment, the process 2400 also includes calculating a summation of pixel characteristics for the pre-selected rectangular region using a regional area algorithm accessible to the processor by subtracting the corresponding data elements for the upper right corner and the lower left corner from a sum of the corresponding data elements for the upper left corner and the lower right corner (2406).
In yet another embodiment, the process 2400 also includes processing the pre-selected digital image using a summation algorithm to form the integral image. The pre-selected digital image is formed by pixels stored in a digital image buffer. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the pre-selected digital image. A target rectangular region of the integral image is selected for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region.
In a further embodiment of the process 2400, the summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for an upright orientation of the pre-selected digital image. In this embodiment, orientation of the target rectangular region follows the upright orientation of the pre-selected digital image.
In another further embodiment of the process 2400, the summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for a −45 degree orientation in relation to an upright orientation of the pre-selected digital image. In this embodiment, orientation of the target rectangular region is +45 degrees in relation to orientation of the pre-selected digital image.
In yet another further embodiment of the process 2400, the summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for a +45 degree orientation in relation to an upright orientation of the pre-selected digital image. In this embodiment, orientation of the target rectangular region is −45 degrees in relation to orientation of the pre-selected digital image.
In still another embodiment, the process 2400 also includes processing the pre-selected digital image using a rotation algorithm to form a clockwise digital image. The pre-selected digital image is formed by pixels stored in a digital image buffer reflecting an upright orientation. The clockwise digital image presents the pixels of the pre-selected digital image in a clockwise image buffer reflecting a −45 degree orientation. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the clockwise digital image. The clockwise digital image is processed using a summation algorithm to form the integral image in the −45 degree orientation. The summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element. A target rectangular region of the integral image is selected for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region. Orientation of the target rectangular region is +45 degrees in relation to orientation of the clockwise digital image.
In still yet another embodiment, the process 2400 also includes processing the pre-selected digital image using a rotation algorithm to form a counterclockwise digital image. The pre-selected digital image is formed by pixels stored in a digital image buffer reflecting an upright orientation. The counterclockwise digital image presents the pixels of the pre-selected digital image in a counterclockwise image buffer reflecting a +45 degree orientation. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the counterclockwise digital image. The counterclockwise digital image is processed using a summation algorithm to form the integral image in the +45 degree orientation. The summation algorithm sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element. A target rectangular region of the integral image is selected for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region. Orientation of the target rectangular region is −45 degrees in relation to orientation of the counterclockwise digital image.
In another embodiment, the process 2400 also includes generating the offset integral image in the offset integral buffer by copying the integral image from the integral image buffer such that addresses for the data elements of the offset integral image in the offset integral buffer are incremented or decremented by one address location in relation to the addresses for the corresponding data elements of the integral image in the integral image buffer.
In yet another embodiment of the process 2400, the first SIMD operation is a doubleword operation involving two consecutive data words associated with the first base address and a first “plus one” address adjacent to the first base address. The first “plus one” address is for a first next data element in the integral image buffer that represents an upper left corner of a next rectangular region for a next Haar-like feature calculation. The first next data element is loaded from the integral image buffer to the processor along with the first data element when the first base address is aligned for the first SIMD operation. The first next data element is loaded from the offset integral buffer to the processor along with the first data element when the first base address is non-aligned for the first SIMD operation. The second SIMD operation is a doubleword operation involving two consecutive data words associated with the second base address and a second “plus one” address adjacent to the second base address. The second “plus one” address is for a second next data element in the integral image buffer that represents an upper right corner of the next rectangular region for the next Haar-like feature calculation. The second next data element is loaded from the integral image buffer to the processor along with the second data element when the second base address is aligned for the second SIMD operation. The second next data element is loaded from the offset integral buffer to the processor along with the second data element when the second base address is non-aligned for the second SIMD operation. The third SIMD operation is a doubleword operation involving two consecutive data words associated with the third base address and a third “plus one” address adjacent to the third base address. The third “plus one” address is for a third next data element in the integral image buffer that represents a lower left corner of the next rectangular region for the next Haar-like feature calculation. The third next data element is loaded from the integral image buffer to the processor along with the third data element when the third base address is aligned for the third SIMD operation. The third next data element is loaded from the offset integral buffer to the processor along with the third data element when the third base address is non-aligned for the third SIMD operation. The fourth SIMD operation is a doubleword operation involving two consecutive data words associated with the fourth base address and a fourth “plus one” address adjacent to the fourth base address. The fourth “plus one” address is for a fourth next data element in the integral image buffer that represents a lower right corner of the next rectangular region for the next Haar-like feature calculation. The fourth next data element is loaded from the integral image buffer to the processor along with the fourth data element when the fourth base address is aligned for the fourth SIMD operation. The fourth next data element is loaded from the offset integral buffer to the processor along with the fourth data element when the fourth base address is non-aligned for the fourth SIMD operation.
In still another embodiment of the process 2400, the first, second, third, and fourth SIMD operations are quadword operations involving four consecutive data words for each SIMD operation and associated with the respective first, second, third, and fourth base address, a corresponding “plus one” address adjacent to each base address, a corresponding “plus two” address adjacent to each “plus one” address, and a corresponding “plus three” address adjacent to each “plus two” address. Each “plus one” address is for a corresponding next data element in the integral image buffer that represents a corresponding upper left, upper right, lower left, or lower right corner of a next rectangular region for a next Haar-like feature calculation. Each “plus two” address is for a corresponding further data element in the integral image buffer that represents a corresponding upper left, upper right, lower left, or lower right corner of a further rectangular region for a further Haar-like feature calculation. Each “plus three” address is for a corresponding even further data element in the integral image buffer that represents a corresponding upper left, upper right, lower left, or lower right corner of an even further rectangular region for an even further Haar-like feature calculation. The corresponding next, further, and even further data elements are loaded from the integral image buffer to the processor along with the respective first, second, third, and fourth data element when the respective first, second, third, and fourth base address is aligned for the respective first, second, third, and fourth SIMD operation. The corresponding next, further, and even further data elements are loaded from the integral image buffer to the processor along with the respective first, second, third, and fourth data element when the respective first, second, third, and fourth base address is non-aligned for the respective first, second, third, and fourth SIMD operation.
With reference to
The application program 2512 may include a rotation algorithm 2518, a summation algorithm 2520, a regional area algorithm 2522, and other algorithms associated with determining the summation of pixel characteristics for the rectangular region in any suitable combination. The application data 2514 may include a digital image buffer 2524, an integral image buffer 2526, an offset integral buffer 2528, and other buffers associated with determining the summation of pixel characteristics for the rectangular region in any suitable combination. The results data 2516 may include one or more summed rectangular regions 2530, one or more Haar-like features 2532, and other results associated with determining the summation of pixel characteristics for the rectangular region and/or Haar-like feature calculations in any suitable combination.
In one embodiment of the electronic assembly 2500, the at least one processor 2502 is configured to determine if first, second, third, and fourth base addresses for respective first, second, third, and fourth data elements in an integral image buffer are aligned for respective first, second, third, and fourth SIMD operations by the at least one processor 2502. The first second, third, and fourth data elements respectively represent upper left, upper right, lower left, and lower right corners of a pre-selected rectangular region of an integral image. The integral image is a representation of a pre-selected digital image. The integral image is formed by data elements stored in the integral image buffer 2526 in the data memory 2506 on the external storage device 2508 or any suitable storage device accessible to the at least one processor 2502. The at least one processor 2502 is configured to load first, second, third, and fourth data elements from the integral image buffer 2526 when the respective first, second, third, and fourth base address is aligned for the respective first, second, third, and fourth SIMD operation. The at least one processor 2502 is configured to load offset first, offset second, offset third, and offset fourth data elements of an offset integral image from the offset integral buffer 2528 in the data memory 2506 on the external storage device 2508 or any suitable storage device when the respective first, second, third, and fourth base address is non-aligned for the respective first, second, third, and fourth SIMD operation. The offset first, offset second, offset third, and offset fourth data elements respectively represent the upper left, upper right, lower left, and lower right corners of the pre-selected rectangular region. The offset data elements of the offset integral image are defined by the corresponding data elements of the integral image and stored at address locations in the offset integral buffer 2528 that are offset right or left by one address location such that a select data element in the integral image buffer 2526 at a select address that is non-aligned for a select SIMD operation by the at least one processor 2502 is located at a select offset address in the offset integral buffer 2528 that is aligned for the select SIMD operation.
In another embodiment of the electronic assembly 2500, the at least one processor 2502 is configured to calculate a summation of pixel characteristics for the pre-selected rectangular region using the regional area algorithm 2522 by subtracting the corresponding data elements for the upper right corner and the lower left corner from a sum of the corresponding data elements for the upper left corner and the lower right corner.
In yet another embodiment of the electronic assembly 2500, the at least one processor 2502 is configured to process the pre-selected digital image using the summation algorithm 2520 to form the integral image. The pre-selected digital image is formed by pixels stored in the digital image buffer 2524. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the pre-selected digital image. The at least one processor 2502 is configured to select a target rectangular region of the integral image for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region.
In a further embodiment of the electronic assembly 2500, the summation algorithm 2520 sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for an upright orientation of the pre-selected digital image. Orientation of the target rectangular region follows the upright orientation of the pre-selected digital image.
In another further embodiment of the electronic assembly 2500, the summation algorithm 2520 sums pixel characteristics of the pre-selected digital image such each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for a −45 degree orientation in relation to an upright orientation of the pre-selected digital image. Orientation of the target rectangular region is +45 degrees in relation to orientation of the pre-selected digital image.
In yet another embodiment of the electronic assembly 2500, the summation algorithm 2520 sums pixel characteristics of the pre-selected digital image such each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element for a +45 degree orientation in relation to an upright orientation of the pre-selected digital image. Orientation of the target rectangular region is −45 degrees in relation to orientation of the pre-selected digital image.
In still another embodiment of the electronic assembly 2500, the at least one processor 2502 is configured to process the pre-selected digital image using the rotation algorithm 2518 to form a clockwise digital image. The pre-selected digital image is formed by pixels stored in the digital image buffer 2524 reflecting an upright orientation. The clockwise digital image presents the pixels of the pre-selected digital image in a clockwise image buffer reflecting a −45 degree orientation. Except for the different orientation, the digital image in the clockwise image buffer is the same as the digital image in the digital image buffer 2524. Each pixel is defined by pixel characteristics. Each element of the integral image represents a corresponding pixel in the clockwise digital image. The at least one processor 2502 is configured to process the clockwise digital image using the summation algorithm 2520 to form the integral image in the −45 degree orientation. The summation algorithm 2520 sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element. The at least one processor 2502 is configured to select a target rectangular region of the integral image for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region. Orientation of the target rectangular region is +45 degrees in relation to orientation of the clockwise digital image.
In still yet another embodiment of the electronic assembly 2500, the at least one processor 2502 is configured to process the pre-selected digital image using the rotation algorithm 2518 to form a counterclockwise digital image. The pre-selected digital image is formed by pixels stored in the digital image buffer 2524 reflecting an upright orientation. The counterclockwise digital image presents the pixels of the pre-selected digital image in a counterclockwise image buffer reflecting a +45 degree orientation. Except for the different orientation, the digital image in the counterclockwise image buffer is the same as the digital image in the digital image buffer 2524. Each pixel is defined by pixel characteristics. Each data element of the integral image represents a corresponding pixel in the counterclockwise digital image. The at least one processor 2502 is configured to process the counterclockwise digital image using the summation algorithm 2520 to form the integral image in the +45 degree orientation. The summation algorithm 2520 sums pixel characteristics of the pre-selected digital image such that each data element of the integral image corresponds to a sum of pixel characteristics for pixels above and left of the corresponding data element. The at least one processor 2502 is configured to select a target rectangular region of the integral image for which summation of pixel characteristics is desired in conjunction with performing Haar-like feature calculations. The target rectangular region defines the pre-selected rectangular region. Orientation of the target rectangular region is −45 degrees in relation to orientation of the counterclockwise digital image.
In still yet another embodiment of the electronic assembly 2500, the at least one processor 2502 is configured to generate the offset integral image in the offset integral buffer 2528 by copying the integral image from the integral image buffer 2526 such that addresses for the data elements of the offset integral image in the offset integral buffer 2528 are incremented or decremented by one address location in relation to the addresses for the corresponding data elements of the integral image in the integral image buffer 2526.
In yet another embodiment of the electronic assembly 2400, the first SIMD operation is a doubleword operation involving two consecutive data words associated with the first base address and a first “plus one” address adjacent to the first base address. The first “plus one” address is for a first next data element in the integral image buffer 2526 that represents an upper left corner of a next rectangular region for a next Haar-like feature calculation. The first next data element is loaded from the integral image buffer 2526 to the at least one processor 2502 along with the first data element when the first base address is aligned for the first SIMD operation. The first next data element is loaded from the offset integral buffer 2528 to the at least one processor 2502 along with the first data element when the first base address is non-aligned for the first SIMD operation. The second SIMD operation is a doubleword operation involving two consecutive data words associated with the second base address and a second “plus one” address adjacent to the second base address. The second “plus one” address is for a second next data element in the integral image buffer 2526 that represents an upper right corner of the next rectangular region for the next Haar-like feature calculation. The second next data element is loaded from the integral image buffer 2526 to the at least one processor 2502 along with the second data element when the second base address is aligned for the second SIMD operation. The second next data element is loaded from the offset integral buffer 2528 to the at least one processor 2502 along with the second data element when the second base address is non-aligned for the second SIMD operation. The third SIMD operation is a doubleword operation involving two consecutive data words associated with the third base address and a third “plus one” address adjacent to the third base address. The third “plus one” address is for a third next data element in the integral image buffer 2526 that represents a lower left corner of the next rectangular region for the next Haar-like feature calculation. The third next data element is loaded from the integral image buffer 2526 to the at least one processor 2502 along with the third data element when the third base address is aligned for the third SIMD operation. The third next data element is loaded from the offset integral buffer 2528 to the at least one processor 2502 along with the third data element when the third base address is non-aligned for the third SIMD operation. The fourth SIMD operation is a doubleword operation involving two consecutive data words associated with the fourth base address and a fourth “plus one” address adjacent to the fourth base address. The fourth “plus one” address is for a fourth next data element in the integral image buffer 2526 that represents a lower right corner of the next rectangular region for the next Haar-like feature calculation. The fourth next data element is loaded from the integral image buffer 2526 to the at least one processor 2502 along with the fourth data element when the fourth base address is aligned for the fourth SIMD operation. The fourth next data element is loaded from the offset integral buffer 2528 to the at least one processor 2502 along with the fourth data element when the fourth base address is non-aligned for the fourth SIMD operation.
In still another embodiment of the electronic assembly 2500, the first, second, third, and fourth SIMD operations are quadword operations involving four consecutive data words for each SIMD operation and associated with the respective first, second, third, and fourth base address, a corresponding “plus one” address adjacent to each base address, a corresponding “plus two” address adjacent to each “plus one” address, and a corresponding “plus three” address adjacent to each “plus two” address. Each “plus one” address is for a corresponding next data element in the integral image buffer 2526 that represents a corresponding upper left, upper right, lower left, or lower right corner of a next rectangular region for a next Haar-like feature calculation. Each “plus two” address is for a corresponding further data element in the integral image buffer 2526 that represents a corresponding upper left, upper right, lower left, or lower right corner of a further rectangular region for a further Haar-like feature calculation. Each “plus three” address is for a corresponding even further data element in the integral image buffer 2526 that represents a corresponding upper left, upper right, lower left, or lower right corner of an even further rectangular region for an even further Haar-like feature calculation. The corresponding next, further, and even further data elements are loaded from the integral image buffer 2526 to the at least one processor 2502 along with the respective first, second, third, and fourth data element when the respective first, second, third, and fourth base address is aligned for the respective first, second, third, and fourth SIMD operation. The corresponding next, further, and even further data elements are loaded from the integral image buffer 2526 to the at least one processor 2502 along with the respective first, second, third, and fourth data element when the respective first, second, third, and fourth base address is non-aligned for the respective first, second, third, and fourth SIMD operation.
Various exemplary embodiments of non-transitory computer-readable medium storing program instructions that, when executed by a processor embedded in a processor-controlled electronic assembly configured to perform Haar-like feature calculations, cause the processor-controlled electronic assembly to perform a method of determining a summation of pixel characteristics for a rectangular region of a digital image. For example, various embodiments of the electronic assembly 2500 are described above with reference to FIG. 25. Various embodiments of the method of determining a summation of pixel characteristics for a rectangular region of a digital image 2100, 2200, 2300, 2400, for example, are described above with reference to
The various embodiments of non-transitory computer-readable medium store program instructions that, when executed by a processor, may cause a corresponding processor-controlled electronic assembly to perform various combinations of functions associated with the various embodiments of the processes 2100, 2200, 2300, 2400 for determining a summation of pixel characteristics for a rectangular region of a digital image described above with reference to
The program instructions of the various exemplary embodiments of non-transitory computer-readable medium are defined by any suitable combination of the processes 2100, 2200, 2300, 2400 described above with reference to
The above examples are merely illustrative of several possible embodiments of various aspects of the present disclosure, wherein equivalent alterations and/or modifications will occur to others skilled in the art upon reading and understanding this specification and the associated drawings. For example, the rectangular sum is one example for illustrating the use of multiple buffers to solve the problem of non-aligned addresses read. However, the same technique can be extended and used with any dense image based operation, such as convolution. In addition, although a particular feature of the disclosure may have been disclosed with respect to only one of multiple implementations, such feature may be combined with one or more other features of other embodiments as may be desired and advantageous for any given or particular application. Also, to the extent that the terms “including”, “includes”, “having”, “has”, “with”, or variants thereof are used in the detailed description and/or in the claims, such terms are intended to be inclusive in a manner similar to the term “comprising”.
| Number | Date | Country | Kind |
|---|---|---|---|
| 201641022034 | Jun 2016 | IN | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7099510 | Jones | Aug 2006 | B2 |
| 7274832 | Nicponski | Sep 2007 | B2 |
| 7903870 | Budagavi | Mar 2011 | B1 |
| 8165401 | Funayama | Apr 2012 | B2 |
| 8781234 | Zhang | Jul 2014 | B2 |
| 8805081 | Chotard | Aug 2014 | B2 |
| 9036937 | Tsai | May 2015 | B2 |
| 9092695 | Ogale | Jul 2015 | B1 |
| 9135521 | Lim | Sep 2015 | B2 |
| 9237350 | Cheong | Jan 2016 | B2 |
| 9251569 | Hsieh | Feb 2016 | B2 |
| 9398297 | Gupta | Jul 2016 | B2 |
| 9442723 | Yang | Sep 2016 | B2 |
| 9589175 | Arnold | Mar 2017 | B1 |
| 9715717 | Liou | Jul 2017 | B2 |
| 9898681 | Lee | Feb 2018 | B2 |
| 20110211233 | Yokono | Sep 2011 | A1 |
| 20120134586 | Pajaniradja | May 2012 | A1 |
| 20120328160 | Sa | Dec 2012 | A1 |
| 20130163861 | Mariatos | Jun 2013 | A1 |
| 20140023232 | Kim | Jan 2014 | A1 |
| 20160042246 | Herrmann | Feb 2016 | A1 |
| 20170038405 | Zhou | Feb 2017 | A1 |
| 20170213099 | Matsui | Jul 2017 | A1 |
| Entry |
|---|
| Ehsan S., et al., “Integral Images: Efficient Algorithms for Their Computation and Storage in Resource-Constrained Embedded Vision Systems”, Sensors 2015, 15, 16804-16830. |
| Kisacanin B., “Integral Image Optimizations for Embedded Vision Applications,” IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), pp. 181-184, Mar. 2008. |
| R. Lienhart and J. Maydt, “An Extended Set of Haar-like Features for Rapid Object Detection,” Proceedings. International Conference on Image Processing, 2002, pp. I-900-I-903 vol. 1. |
| J. Whitehill and C. W. Omlin, “Haar Features for FACS AU Recognition,” 7th International Conference on Automatic Face and Gesture Recognition (FGR06), Southampton, 2006, pp. 5-101. |
| Yoav Freund and Robert E. Schapire, “A Short Introduction to Boosting,” Journal of Japanese Society for Artificial Intelligence, vol. 14, No. 5 (1999), pp. 771-780. |
| Paul Viola and Michael Jones, “Robust Real-time Object Detection,” Second International Workshop on Statistical and Computational Theories of Vision—Modeling, Learning Computing, and Sampling, Jul. 13, 2001, pp. 1-25. |
| Franklin C. Crow, “Summed-Area Tables for Texture Mapping,” ACM SIGGRAPH Computer Graphics, vol. 18 Issue 3, Jul. 1984, pp. 207-212. |
| Number | Date | Country | |
|---|---|---|---|
| 20170372154 A1 | Dec 2017 | US |
