Patent Application
20160267111
Some operations performed by computing devices are time consuming and/or resource intensive. Known methods of multi-frame processing (MFP), for example, utilize complex calculations that take more than 1.8 seconds per frame and/or utilize dedicated hardware that consumes substantial power. Moreover, at least some known methods retrieve and, in at least some implementations, re-retrieve data from a memory area, which consumes bandwidth each time the data is re-retrieved.
Examples of the disclosure process a data set to produce an enhanced data set. In some examples, a system includes a plurality of first processor elements that processes a first data set and a second data set using a first function to generate a third data set, and processes the third data set using a second function to generate an output element. The first processor elements are arranged in a two-dimensional systolic array such that one or more first processor elements of the first plurality of processor elements receive input from one or more first adjacent first processor elements and transmit output to one or more second adjacent first processor elements (e.g., using systolic computation). The system includes a plurality of second processor elements that aggregate the output elements to at least partially generate a fourth data set. The plurality of second processor elements are arranged in a one-dimensional array.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
Corresponding reference characters indicate corresponding parts throughout the drawings.
The disclosed system includes an architecture configured to perform systolic processing of a data set. For example, a raw image is processed by the architecture using a kernel data set to generate a processed image. The architecture includes a two-dimensional systolic array and a one-dimensional systolic array. Examples of the disclosure processing a first data set using the two-dimensional systolic array and a second data set to generate a third data set. The third data set is processed using a second function to generate an output element. The one-dimensional systolic array is configured to aggregate the output element to at least partially generate a fourth data set.
Aspects of the disclosure facilitate increasing speed, conserving memory, reducing processor load or an amount of energy consumed, and/or reducing network bandwidth usage by calculating one or more values, storing the one or more values in a local buffer, and reusing the one or more values. Local buffering is utilized at various stages of processing to leverage the architectural elements described herein. In some examples, buffering data locally decreases or eliminates the need to re-fetch data from external memory, lowering memory bandwidth and/or local storage used. Additionally or alternatively, fine-grained parallel implementations are used within various processing elements of the accelerator. For example, many blocks involve a series of two-level vector reduction operations. The disclosed system employs arrays of specialized processing elements that are interconnected to exploit this computation pattern.
A user 101 may operate the computing device 100. In some examples, the computing device 100 may be always on, or the computing device 100 may turn on and/or off in response to stimuli such as a change in light conditions, movement in the visual field, change in weather conditions, etc. In other examples, the computing device 100 may turn on and/or off in accordance with a policy. For example, the computing device 100 may be on during predetermined hours of the day, when a vehicle is on, etc.
The computing device 100, in some examples, includes a user interface device or interface module 102 for exchanging data between the computing device 100 and the user 101, computer-readable media, and/or another computing device. In at least some examples, the interface module 102 is coupled to or includes a presentation device configured to present information, such as text, images, audio, video, graphics, alerts, and the like, to the user 101. The presentation device may include, without limitation, a display, speaker, and/or vibrating component. Additionally or alternatively, the interface module 102 is coupled to or includes an input device configured to receive information, such as user commands, from the user 101. The input device may include, without limitation, a game controller, camera, microphone, and/or accelerometer. In at least some examples, the presentation device and the input device may be integrated in a common user-interface device configured to present information to the user 101 and receive information from the user 101. For example, the user-interface device may include, without limitation, a capacitive touch screen display and/or a controller including a vibrating component.
The computing device 100 includes one or more computer-readable media, such as a memory area 104 storing computer-executable instructions, video or image data, and/or other data, and one or more processors 106 programmed to execute the computer-executable instructions for implementing aspects of the disclosure. The memory area 104 includes any quantity of media associated with or accessible by the computing device 100. The memory area 104 may be internal to the computing device 100 (as shown in
In some examples, the memory area 104 stores, among other data, one or more applications. The applications, when executed by the processor 106, operate to perform functionality on the computing device 100. Example applications include mail application programs, web browsers, calendar application programs, address book application programs, messaging programs, media applications, location-based services, search programs, and the like. The applications may communicate with counterpart applications or services such as web services accessible via a network. For example, the applications may represent downloaded client-side applications that correspond to server-side services executing in a cloud.
The processor 106 includes any quantity of processing units, and the instructions may be performed by the processor 106 or by multiple processors within the computing device 100 or performed by a processor external to the computing device 100. The processor 106 is programmed to execute instructions such as those illustrated in the figures (e.g.,
The processor 106 is transformed into a special purpose microprocessor by executing computer-executable instructions or by otherwise being programmed. For example, the processor 106 may execute the computer-executable instructions to identify one or more interest points in a plurality of images, extract one or more features from the one or more interest points, align the plurality of images, and/or combining the plurality of images. Although the processor 106 is shown separate from the memory area 104, examples of the disclosure contemplate that the memory area 104 may be onboard the processor 106 such as in some embedded systems.
In this example, the memory area 104 stores one or more computer-executable components for multi-frame processing of images. A network communication interface 108, in some examples, exchanges data between the computing device 100 and a computer-readable media or another computing device. The network communication interface 108 may transmit the image to a remote device and/or receive requests from the remote device. Communication between the computing device 100 and a computer-readable media or another computing device may occur using any protocol or mechanism over any wired or wireless connection.
The block diagram of
An image signal processor (ISP) 208 is configured to retrieve or pull down one or more raw images 228 from the first frame bus 224 and clean up or otherwise process the raw images 228. The ISP 208 may place one or more processed images onto the first frame bus 224 (raw images 228 and processed images are represented as F0, F1 . . . FN in
An accelerator 210 is configured to retrieve or pull down one or more images 228 from the first frame bus 224 and align the images 228. The accelerator 210 may place one or more aligned images 230 onto a second frame bus (e.g., aligned frame bus) 226. In some examples, the accelerator 210 includes an interest point-detection (IPD) module 212, a feature-extraction (FE) module 214, a homography estimation (HE) module 216, and/or an image warping (IWP) or warp module 218. Alternatively, the accelerator 210 may include any combination of modules that enables the computing device 200 to function as described herein.
The IPD module 212 may retrieve or take one or more images 228 from the first frame bus 224 and detect, identify, or search for one or more relevant interest points on the images 228. Interest-point detection helps identify pixel locations associated with relevant information. Examples of pixel locations include closed-boundary regions, edges, contours, line intersections, corners, etc. In one example, corners are used as interest points because corners form relatively robust control points and/or detecting corners has a relatively low computational complexity. The FE module 214 may extract one or more features from the interest points using, for example, a daisy feature-extraction algorithm. The HE module 216 may align or shift one or more images 228 such that the images utilize the same or a common coordinate system. The IWP module 218 warps, modifies, or adjusts one or more images 228 such that the images 228 are aligned. One or more aligned images 230 are placed on the aligned frame bus 226.
A processor module 219 includes a central processing unit (CPU) 220 and/or a graphics processing unit (GPU) 222 configured to retrieve or pull down one or more aligned images 230 from the aligned frame bus 226 and combine or composite the images and place the composite images 232 onto the first frame bus 224. In at least some examples, the CPU 220 and/or GPU 222 are interchangeable.
Images 228 are consumed by the accelerator 210 and are replaced on the first frame bus 224 by the processor module 219 with composite images 232. In at least some examples, raw images 228 are consumed by the ISP 208 and are replaced on the first frame bus 224 by the ISP 208 with processed images. This consumption and/or replacement process enables the first frame bus 224 to run at or below capacity. In some examples, the computing device 200 includes a third bus onto which the processor module 219 places the composite images 232. In some examples, one or more frame buses 224 and 226 are alternating, non-colliding, or isolated. This reduces an opportunity for an element of the architecture from being starved and/or from acting as a bottleneck to another element of the architecture. In this example, one or more frame buses 224 and 226 are connected to an application or another output, for instance, on a mobile device. In some examples, the frame buses 224 and 226 are connected to an output using a multiplexer.
Typical classification algorithms use histogram-based feature-extraction methods, such as scale-invariant feature transform (SIFT), histogram oriented gradient (HoG), gradient location and orientation histogram (GLOH), etc. The FE module 214 enables a computation engine using a modular framework to represent or mimic many other feature-extraction methods depending on tunable algorithmic parameters that may be set at run-time. As shown in
In some examples, different candidate blocks are swapped in and out to produce new overall descriptors. In addition, parameters that are internal to the candidate features may be tuned in order to increase the performance of the descriptor as a whole. In this example, the FE module 214 is pipelined to perform stream processing of pixels. The feature-extraction algorithm includes a plurality of processing steps that are heavily interleaved at the pixel, patch, and frame levels.
In a first block or filter module, the FE module 214 includes a pre-smoothing or G-Block 302 that is configured to smooth a P×P image patch of pixels 310 around each interest point by convolving it with a two-dimensional Gaussian filter of standard deviation (σs). In one example, it is convolved with a kernel having dimensions A×A 312. This results in a smoothened P×P image patch of pixels 314. The number of rows and/or columns in the G-Block 302 may be adjusted to achieve a desired energy and throughput scalability.
In a second block or gradient module, the FE module 214 includes a transformation or T-Block 304 that is configured to map the P×P smoothened patch of pixels 314 onto a length k vector with non-negative elements to create k×P×P feature maps 318. At a high level, the T-Block is a single processing element that generates the T-Block features sequentially. There are four sub-blocks defined for the transformation, namely, T1, T2, T3, and T4 (collectively illustrated as “Gradient and Bin” 316).
In sub-block T1, at each pixel location (x, y), the disclosure computes gradients along both horizontal (Δx) and vertical (Δy) directions. The magnitude of the gradient vector is then apportioned into k bins (where k equals 4 in T1a and 8 in T1b mode), split equally along the radial direction—resulting in an output array of k feature maps, each of size P×P.
In sub-block T2, the gradient vector is quantized in a sine-weighted fashion into 4 (T2a) or 8 (T2b) bins. For T2a, the quantization is done as follows: |Δx|−Δx; |Δx|+Δx; |Δy|−Δy; |Δy|+Δy. For T2b, the quantization is done by concatenating an additional length 4 vector using Δ45 D45, which is the gradient vector rotated through 45 degrees.
In sub-block T3, at each pixel location (x, y), steerable filters are applied using n orientations, and the response is computed from quadrature pairs. Next, the result is quantized in a manner similar to T2a to produce a vector of length k=4n (T3a), and in a manner similar to T2b to produce a vector of length k=8n (T3b). In some examples, filters of second or higher-order derivatives and/or broader scales and orientations are used in combination with the different quantization functions.
In sub-block T4, two isotropic difference of Gaussian (DoG) responses are computed with different centers and scales (effectively reusing the G-block 302). These two responses are used to generate a length k=4 vector by rectifying the positive and negative parts into separate bins as described in T2.
In one example, only the T1 and T2 blocks are utilized. For example, the data path for the T-block includes gradient-computation and quantization engines for the T1 (a), T1 (b), T2 (a), and T2 (b) modes of operation. In another example, T3 and T4 are also utilized. In some examples, various combinations of T1, T2, T3, and T4 are used to achieve different results. The T-block outputs are buffered in a local memory of size 3×(R+2)×24b and the pooling region boundaries are stored in a local static random-access memory (SRAM) of size Np×3×8b.
In a third block or pooler module, the FE module 214 includes a spatial pooling or S-Block 306 configured to accumulate the weighted vectors, the k×P×P feature maps 318, from the T-Block 304 to give N linearly summed vectors of length k 320. These N vectors are concatenated to produce a descriptor of length kN. In the S-Block 306, there are a configurable number of parallel lanes for the spatial-pooling process. These lanes include comparators that read out Np pooling region boundaries from a local memory and compare with the current pixel locations. The power consumption and performance of the S-Block 306 may be adjusted by varying a number of lanes in S-Block 306.
In the final block or normalizer module, the FE module 214 includes a post normalization or N-Block 308 that is configured to remove descriptor dependency on image contrast. The output from the S-block 306 is processed by the N-block 308, which includes an efficient square-rooting algorithm and division module (based on CORDIC). In a non-iterative process, the S-Block 306 features are normalized to a unit vector (e.g., dividing by the Euclidean norm) and all elements above a threshold are clipped. The threshold is defined, in some examples, depending on the type of ambient-aware application operating on the mobile device or, in other examples, the threshold is defined by policies set by a user (e.g., user 101), the cloud, and/or an administrator. In some examples, a system with higher bandwidth, or more cost effective transmission, may set the threshold lower than other systems. In an iterative process, these steps repeat until a predetermined number of iterations has been reached.
Data precisions are tuned to increase an output signal-to-noise-ratio (SNR) for most images. The levels of parallelism in the system, the output precisions, memory sizes etc. may all be parameterized in the code. Assuming no local data buffering between the IPD module 212 and FE modules 214, the feature-extraction block (for nominal ranges) consumes (assuming 64×64 patch size and 100 interest points) approximately 1.2 kB (4×4 two-dimensional array and 25 pooling regions) for a frame resolution of VGA (128×128 patch size and 100 interest points) and approximately 3.5 kB (8×8 two-dimensional array and 25 pooling regions) for a frame resolution of 720p HD. Local buffering between the IPD module 212 and FE module 214 enable those elements to work in a pipelined manner and, thus, mask the external data access bandwidth. Estimated storage capacities for the IPD module 212 and FE modules 214 are approximately 207.38 kB for VGA, 257.32 kB for 1080p, and approximately 331.11 kB for 4k image resolutions.
In some examples, vector data may be processed in two stages utilizing two-dimensional-processing elements in a systolic array alongside an array of one-dimensional-processing elements. For example, the G-Block 302 may process images utilizing this two stage approach. The processing elements of the array iteratively process data, passing the results of any computations to the nearest neighbors of each processing element. In this example, an image is processed by a kernel, or type of filter, using this hardware architecture, resulting in a more efficient, faster processing of images on a device. A processing element or a computational unit may be any device or unit that takes an input and produces an output. Examples of processing elements may be implemented in hardware using gates and realized using field-programmable gate arrays or application-specific integrated circuits.
At least some of the modules described herein may utilize or incorporate a two-level vector reduction. In some examples, vector data, such as images, may be processed in two stages utilizing two-dimensional-processing elements in a systolic array alongside an array of one-dimensional-processing elements. The processing elements of the array iteratively process data, passing the results of any computations to the nearest neighbors of each processing element. In this example, an image is processed by a kernel, or type of filter, using this hardware architecture, resulting in a more efficient, faster processing of images on a device.
Utilizing a systolic array enables parallel processing, in two levels of reduction, of the data set U 406. Although the illustrated examples relate to processing images and/or image patches, any data sets may be processed in a systolic array in this manner. In the first level of reduction (e.g., L1), data sets U 406 and V 402 are processed element-wise using a first reduction function F 404. To achieve this, inter-vector data parallelism is utilized, which enables allowing the data set V 402 to be reused across all L1 lanes. The systolic array is utilized to perform the operations and/or to reduce resource costs.
As an example, in a first level of reduction, the first element of data set V 402 is applied to the first element of data set U 406 using function F 404, which yields the first element of data set W 408. In one example, the function F 404 is multiplication and, thus, the vector W 408 is generated by multiplying each element of vector V 402 (for instance, [v1, v2, . . . vN]) by the corresponding element of vector U 406 (for instance, [u1, u2, . . . uN]). Specifically, in this example, v1×u1=w1, v2×u2=w2, and so on until all elements of data set V 402 have been multiplied by all elements of data set U 406 resulting in a complete data set W 408 ([w1, w2, . . . wN]), which has the same number of elements as data sets V 402 and U 406.
In the second level of reduction (e.g., L2), each element wj of the resultant data set W 408 is processed by a second reduction function G 410 to generate an element hj 412. In one example, the function G 410 is an accumulator and/or addition and, thus, the element hj is a scalar product. In this example, the element hj is equal to the sum of w1+w2+ . . . +wN. The element hj is generated for each image patch of an image including a plurality of image patches to generate a resultant data set H=[h1, h2, hj . . . hM] 414.
When processing overlapping image patches, elements of the data set H 414 and/or and operations associated with generating the elements of the data set H 414 may be interleaved or reused to facilitate decreasing or eliminating the need to recalculate and/or re-fetch data repeatedly from external memory, lowering both memory bandwidth and local storage used.
Various combinations of functions are contemplated for the operations described above. In one example, function F 404 is multiplication and, thus, data set W 408 is the element-wise product of data sets U 406 and V 402. In that example, function G 410 may be addition or accumulation, in which case element hj is the scalar product. In another example of clustering, function F 404 is a distance and, thus, data set W 408 is a distance map of data sets U 406 and V 402 from a centroid. In that example, function G 410 is a comparator, in which case element hj is the nearest neighbor. In another example of image processing, function F 404 is an average and, thus, data set W 408 includes the mean filtered (by data set V 402) pixels of an image patch associated with data set U 406. In that example, function G 410 is a threshold, in which case element hj is an edge location of pixels. In another example of image processing, function F 404 is a gradient and, thus, data set W 408 includes the smoothed filtered (by data set V 402) pixels of an image patch associated with data set U 406. In that example, function G 410 is an addition, in which case element is a dominant optical flow of objects in the image. Although the disclosure is drawn to images, it is understood that the disclosure is not limited to images, but it may also be utilized to process other information such as tags, points in space, generic vectors, etc.
The disclosed systolic array architecture 500 provides the benefits discussed herein, feeding inputs a limited number of times, reusing data, and/or reducing bandwidth consumed as a result of accessing external memory 502. Further, the vector reduction process allows the system to perform two-dimensional convolution along any direction, with varying stride lengths, and kernel sizes. For example, the systolic array architecture 500 may retrieve or receive data from the external memory 502 a limited number of times (e.g., once), and process or reduce the data locally at the systolic array architecture 500 without transmitting data to or retrieving additional data from the external memory 502.
In at least some examples, a control 508 manages an operation and/or a schedule (e.g., clock cycle) of the systolic array architecture 500. On a first clock cycle, element u1 associated with the first row is transmitted to a 2d-PE 506 positioned on the first row, first column, and element v1 associated with the first column is transmitted to the 2d-PE 506 positioned on the first row, first column. The F 404 and G 410 functions are implemented at the 2d-PE 506 positioned on the first row, first column (e.g., 2d-PE11) to generate element w11 (e.g., w11=v1×u1, and h1=w11). On each clock cycle, the elements are transmitted to adjacent 2d-PEs 506. For example, on a second clock cycle, one or more relevant elements (e.g., element u1) are transmitted to an adjacent 2d-PE 506 positioned on the first row, second column (e.g., 2d-PE12), and one or more relevant elements (e.g., element v1) are transmitted to an adjacent 2d-PE 506 positioned on the second row, first column (e.g., 2d-PE21), where they are processed with an element u2. For example, at 2d-PE12, element u1 is processed with element v2 (e.g., w12=v2×u1, and h1=v1×u1+v2×u1), and at 2d-PE21, element u2 is processed with element v1 (e.g., w21=v1×u2, and h2=w21). After N-clock cycles, 2d-PE1N generates element h1 (e.g., h1=v1×u1+v2×u1+ . . . vN×u1), and 2d-PE2(N-1) generates element h2 (e.g., h2=v1×u2+v2×u2+ . . . v(N-1)×u2), and so on. Accordingly, at any given point in time, the systolic array includes some combination of fully- and partially-convolved outputs. As shown in
At least a part of some of the outputs are reused, as at least some elements are re-fed into the engine by passing them from one processing element to its neighbors. In order to accommodate the partially-convolved outputs, a set of one-dimensional processing elements (1d-PEs) 510 is used along the edge of the 2d-PEs 506. The set of 1d-PEs 510 is, in some examples, arranged in a column, as illustrated in
The functions performed by the systolic array architecture 500 may be any operation that enables the system to function as described herein. The advantage of passing relevant elements to adjacent or near neighbor 2d-PEs 506 is that the computations are localized and sequential, thereby increasing an opportunity to reuse at least some elements and/or reducing a latency. This system is configurable to any image or kernel size, stride, type, etc.
In some examples, the systolic array architecture 500 may be modified to include any quantity of 2d-PEs 506 and/or 1d-PEs 510 in any quantity of lanes (e.g., increase or decrease a quantity of rows, increase or decrease a quantity of columns). In this manner, the systolic array architecture 500 may be tailored to scale up or scale down a throughput of the systolic array architecture 500. For example, a rate at which the output element and/or the fourth data set are generated may be modified. In at least some examples, modifying the systolic array architecture 500 enables an amount of power consumed by the systolic array architecture 500 to be managed or controlled. This may be implemented using power gating transistors, clock gating, distributed power supplies etc.
As shown in
The output is stored in local memory to further reduce the latency of the processing. As shown in
Memory consumption associated with the block are RCd×8b for input/output FIFOs of depth d (e.g., 16) and PC×24b to store partially convolved outputs. If pixels are re-fetched from external memory, the hardware consumes an external memory bandwidth of TP2×8b. However, in this example, local buffers are added between the IPD module and the feature-extraction blocks to reduce an opportunity for re-fetching.
A clock cycle may be initiated by, for example, increasing a clock cycle at 708. Alternatively, the clock cycle may be increased after the data set(s) have been processed (e.g., at the end of the clock cycle). At 710, a subject data element (e.g., element u1) is transmitted or passed from a FIFO row 504 towards a first processor element (e.g., a 2d-PE 506), and, at 712, a kernel data element (e.g., element v1) is transmitted or passed from a FIFO column 505 towards the first processor element. At 714, the subject data element is processed using a first function (e.g., function F) and the kernel data element to generate a product data element (e.g., third data set element, w11), and, at 716, the product data element is processed using a second function (e.g., function G) to generate an output element (e.g., h). Because each 2d-PE 506 accepts one product data element and one kernel data element at a time (e.g., per clock cycle), this results in an element by element processing of the subject data set by the kernel data set. In at least some examples, the 2d-PE 506 may generate the output element based at least in part on an output element received from a previous, adjacent 2d-PE 506 (e.g., from another 2d-PE to the left of the 2d-PE).
In at least some examples, one or more 2d-PEs 506 in the last column (e.g., 2d-PExN) may transmit or pass an output element to an adjacent 1d-PE at 718. Additionally or alternatively, a 1d-PE may transmit or pass an output element to a subsequent, adjacent 1d-PE (e.g., to another 1d-PE above of the 1d-PE) and/or a FIFO stack, which feeds an output element into the 1d-PE in the last row. At 720, the output elements are aggregated (e.g., accumulated) at the 1d-PE array to generate an output data set (e.g., fourth data set, data set H).
At 722, it is determined whether the process is complete. For example, the control 508 may determine whether all elements of the subject data set have been passed through the systolic array and/or all elements of the output data set have been aggregated. When the process is determined to be complete, the process ends at 724. As shown in
When the process is determined to be not complete, the process continues by increasing a clock cycle at 708. During this new clock cycle, subject data elements and/or output elements are transmitted or passed down the row towards a subsequent, adjacent 2d-PE 506 at 710, and kernel data elements are transmitted or passed from the column towards a subsequent, adjacent 2d-PE 506 at 712 such that another output element may be generated at one or more 2d-PEs 506. In this way, each 2d-PE 506 may sequentially process a plurality of subject data elements using one kernel data element or process one subject data element sequentially using a plurality of kernel data elements.
The DPU 804 and/or the KFU 806 realize a distance computation. Support vectors 802, which represent the trained model, or in some examples the kernel 602, kernel matrix, filter matrix, or kernel data set, are stored in a streaming memory bank along the borders of the DPU 804 array. During on-line classification, the DPUs 804 perform L1 vector reduction (illustrated in more detail in
After this, the dot products are streamed out to the KFU 806, where the kernel function (representing the L2 reduction, illustrated in more detail in
Example computer readable media include flash memory drives, digital versatile discs (DVDs), compact discs (CDs), floppy disks, and tape cassettes. By way of example and not limitation, computer readable media comprise computer storage media and communication media. Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media are tangible and mutually exclusive to communication media. Computer storage media are implemented in hardware and exclude carrier waves and propagated signals. Computer storage media for purposes of this disclosure are not signals per se. Example computer storage media include hard disks, flash drives, and other solid-state memory. In contrast, communication media typically embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media.
Although described in connection with an example computing system environment, examples of the disclosure are capable of implementation with numerous other general purpose or special purpose computing system environments, configurations, or devices.
Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with aspects of the disclosure include, but are not limited to, mobile computing devices, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, gaming consoles, microprocessor-based systems, set top boxes, programmable consumer electronics, mobile telephones, mobile computing and/or communication devices in wearable or accessory form factors (e.g., watches, glasses, headsets, or earphones), network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. Such systems or devices may accept input from a user in any way, including from input devices such as a keyboard or pointing device, via gesture input, proximity input (such as by hovering), and/or via voice input.
Examples of the disclosure may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices in software, firmware, hardware, or a combination thereof. The computer-executable instructions may be organized into one or more computer-executable components or modules. Generally, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform particular tasks or implement particular abstract data types. Aspects of the disclosure may be implemented with any number and organization of such components or modules. For example, aspects of the disclosure are not limited to the specific computer-executable instructions or the specific components or modules illustrated in the figures and described herein. Other examples of the disclosure may include different computer-executable instructions or components having more or less functionality than illustrated and described herein.
Aspects of the disclosure transform a general-purpose computer into a special-purpose computing device when configured to execute the instructions described herein.
The examples illustrated and described herein as well as examples not specifically described herein but within the scope of aspects of the disclosure constitute example means for processing a data set. For example, the elements described herein constitute at least an example means for generating an image, an example mans for applying a first function to a first data set using a second data set to generate a third data set, and an example means for applying a second function to a third data set to generate an output element, and/or an example means for aggregating an output element to at least partially generate a fourth data set.
The order of execution or performance of the operations in examples of the disclosure illustrated and described herein is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and examples of the disclosure may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the disclosure.
When introducing elements of aspects of the disclosure or the examples thereof, the articles “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements. The phrase “one or more of the following: A, B, and C” means “at least one of A and/or at least one of B and/or at least one of C.” Having described aspects of the disclosure in detail, it will be apparent that modifications and variations are possible without departing from the scope of aspects of the disclosure as defined in the appended claims. As various changes could be made in the above constructions, products, and methods without departing from the scope of aspects of the disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.
Alternatively or in addition to the other examples described herein, examples include any combination of the following:
In some examples, the operations illustrated may be implemented as software instructions encoded on a computer readable medium, in hardware programmed or designed to perform the operations, or both. For example, aspects of the disclosure may be implemented as a system on a chip or other circuitry including a plurality of interconnected, electrically conductive elements.
While the aspects of the disclosure have been described in terms of various examples with their associated operations, a person skilled in the art would appreciate that a combination of operations from any number of different examples is also within scope of the aspects of the disclosure.
This application claims the benefit of U.S. Provisional Application No. 62/131,814, filed Mar. 11, 2015, and of U.S. Provisional Application No. 62/131,815, filed Mar. 11, 2015. This application is related to Context-Awareness Through Biased On-Device Image Classifiers, filed concurrently herewith and incorporated by reference herein. This application is related to Methods and Systems for Low-Energy Image Classification, filed concurrently herewith and incorporated by reference herein. This application is related to Methods and Systems for Generating Enhanced Images Using Multi-Frame Processing, filed concurrently herewith and incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 62131814 | Mar 2015 | US | |
| 62131815 | Mar 2015 | US |
