This application claims priority under 35 USC §119 to Korean Patent Application No. 10-2013-0026036 filed on Mar. 12, 2013, the subject matter of which is hereby incorporated by reference.
The inventive concept relates generally to image processing technologies. More particularly, certain embodiments of the inventive concept relate to image processing devices and computing systems comprising the same.
Modern imaging devices, such as cameras, often include both image capture components and image processing components. The image capture components typically capture an image by sensing light on an array of pixels, and the image processing components perform operations on the captured image, such as adjusting captured pixel values for quality purposes or extracting information from the captured pixel values.
High quality imaging devices often include multi-layer image sensors capable of generating multiple color signals from each unit pixel. The use of multi-layer image sensors tends to increase the amount of data produced by an imaging device, which in turn increases the processing load of the image processing components.
Unfortunately, when the processing load of the image processing components is increased, power consumption tends to increase accordingly. As a result, there is a general need for image processing devices and related processing techniques that can perform large amounts of processing without excessive power consumption.
In one embodiment of the inventive concept, an image processing device comprises a pixel array comprising multiple unit pixels each configured to generate multiple color signals in response to incident light, and a data processing unit configured to generate output image data by processing the color signals in parallel in a first operating mode, and further configured to generate two image signals for each unit pixel based on the color signals and to generate the output image data by processing the two image signals in parallel in a second operating mode.
In another embodiment of the inventive concept, a computing system comprises an image processing device configured to generate output image data corresponding to incident light, a storage device configured to store the output image data, and a processor configured to control the image processing device and the storage device. The image processing device comprises a pixel array comprising multiple unit pixels each configured to generate multiple color signals in response to the incident light, and a data processing unit configured to generate the output image data by processing the color signals in parallel in a first operating mode, and further configured to generate two image signals for each unit pixel based on the color signals and to generate the output image data by processing the two image signals in parallel in a second operating mode.
In yet another embodiment of the inventive concept, an image processing device comprises a pixel array comprising multiple unit pixels each configured to generate “n” color signals in response to incident light, where n is greater than one, and a data processing unit configured to generate output image data by processing the “n” signals in parallel in a first operating mode, and further configured to generate “m” image signals for each unit pixel based on the “n” signals and to generate the output image data by processing the m image signals in parallel in a second operating mode, where m is a positive number less than n.
These and other embodiments of the inventive concept can potentially improve the power efficiency of imaging systems and devices by allowing certain operations to be performed at lower power.
The drawings illustrate selected embodiments of the inventive concept. In the drawings, like reference numbers indicate like features.
Embodiments of the inventive concept are described below with reference to the accompanying drawings. These embodiments are presented as teaching examples and should not be construed to limit the scope of the inventive concept.
In the description that follows, the terms first, second, etc. may be used to describe various features, but these features should not be limited by these terms. Rather, these terms are used merely to distinguish between different features. For example, a first feature could be termed a second feature and vice versa without departing from the scope of the inventive concept. As used herein, the term “and/or” indicates any and all combinations of one or more of the associated listed items.
Where a feature is referred to as being “connected” or “coupled” to another feature, it can be directly connected or coupled to the other feature or intervening features may be present. In contrast, where a feature is referred to as being “directly connected” or “directly coupled” to another feature, there are no intervening features present. Other words used to describe the relationship between features should be interpreted in a similar fashion (e.g., “between” versus “directly between,” “adjacent” versus “directly adjacent,” etc.).
The terminology used herein is for the purpose of describing particular embodiments and is not intended to limit the inventive concept. As used herein, the singular forms “a,” “an” and “the” are intended to include the multiple forms as well, unless the context clearly indicates otherwise. Terms such as “comprises,” “comprising,” “includes” and/or “including,” where used herein, indicate the presence of stated features but do not preclude the presence of other features.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Referring to
Unit pixel 110 typically comprises multiple light receiving devices stacked on each other. The light receiving device may be implemented with either a photo diode or a photo gate. Because transmission factors of optical signals are different from each other according to wavelengths, the receiving devices stacked in unit pixel 110 absorb optical signals having different wavelengths to generate color signals CS1, CS2 and CS3 having different colors.
In some embodiments, unit pixel 110 in pixel array 100 comprises organic materials reacting to an optical signal of a specific wavelength. Thus, in response to optical signals having different wavelength, the organic materials in unit pixel 110 may generate color signals CS1, CS2 and CS3 corresponding to different colors.
Control unit 300 controls operations of the pixel array in units of row through a first control signal CON1. Thus, unit pixels 110, which are in the same row of pixel array 100, simultaneously generate color signals CS1, CS2 and CS3, respectively and provide color signals CS1, CS2 and CS3 to data processing unit 200.
Control unit 300 controls operations of data processing unit 200 through a control signal CON2 and a mode signal MD. In some embodiments, control unit 300 generates mode signal MD based on a selection by a user.
Data processing unit 200 serializes color signals CS 1, CS2 and CS3 provided in units of rows from pixel array 100 based on second control signal CON2, and generates output image data O_DATA based on color signals CS1, CS2 and CS3 generated from unit pixel 110.
Data processing unit 200 operates in a first operating mode or a second operating mode based on mode signal MD. For example, where mode signal MD has a first logic level, data processing unit 200 operates in the first operating mode. Where mode signal MD has a second logic level, data processing unit 200 operates in the second operating mode.
Data processing unit 200 processes color signals CS1, CS2 and CS3 in parallel in the first operating mode to generate output image data O_DATA. Data processing unit 200 generates two image signals every unit pixels 110 based on color signals CS1, CS2 and CS3 in the second operating mode and then, processes the two image signals in parallel to generate output image data O_DATA.
In the first operating mode, because data processing unit 200 uses all of color signals CS1, CS2 and CS3 provided from each unit pixel 110 to perform a data processing so that output image data O_DATA is generated, data processing unit 200 may provide a high-quality image. Thus, the first operating mode may be a high-quality imaging mode.
In the second operating mode, data processing unit 200 generates two image signals based on color signals CS1, CS2 and CS3 provided from each unit pixel 110 so that an amount of data is reduced. Then, data processing unit 200 performs a data processing using the two image signals to generate output image data O_DATA, so that a consumed electric power may be reduced. Thus, the second operating mode may be a power saving mode.
In some embodiments, data processing unit 200 converts color signals CS1, CS2 and CS3, which are provided from each unit pixel 110, into two image signals in YUV format in the second operating mode. For example, each of unit pixels 110 generates green, red and blue signals GS, RS and BS corresponding to green, red and blue colors of light incident thereupon. In the second operating mode, data processing unit 200 converts the green, red and blue signals GS, RS and BS into signal in YUV format, such that data processing unit 200 generates a first image signal IS1 corresponding to a Y component (i.e., luminance component) and a second image signal IS2 corresponding to one of an U component (i.e., a first chrominance component) and a V component (i.e., a second chrominance component). Then, data processing unit 200 processes first and second image signals IS1 and IS2 in parallel to generate output image data O_DATA. In this case, data processing unit 200 generates second image signal IS2 by alternately selecting the U and V components for each of unit pixels 110 arrayed in pixel array 100 in a matrix form. Thus, data processing unit 200 may generates first and second image signals IS1 and IS2 in YUV422 format in the second operating mode.
Various examples of data processing unit 200 are described below with reference to
Referring to
The 3-channel analog-digital conversion units 211-1, . . . and 211-n are connected to the columns of pixel array 100, respectively, to receive the green, red and blue signals GS, RS and BS. Each of the 3-channel analog-digital conversion units 211-1 and 211-n comprises first to third analog-digital converters ADC1, ADC2 and ADC3. First to third analog-digital converters ADC1, ADC2 and ADC3 perform an analog-to-digital conversion in parallel to convert the green, red and blue signals GS, RS and BS into first to third digital signals DS1, DS2 and DS3.
Column decoder 212 sequentially outputs first to third digital signals DS1, DS2 and DS3 received from the 3-channel analog-digital conversion units 211-1, and 211-n based on second control signal CON2 provided from control unit 300.
In the first operating mode where mode signal MD has the first logic level, 3-channel processing unit 213 performs the 3-channel data processing for first to third digital signals DS1, DS2 and DS3, so that 3-channel processing unit 213 may generate 3-channel data 3CH_DATA comprising three processing signals. The 3-channel data processing signifies the generation of three output signals by performing a data processing for three input signals independently from each other and/or dependently on each other. For example, 3-channel processing unit 213 may perform a noise removing operation, a sharpening operation and a gamma correction for first to third digital signals DS1, DS2 and DS3.
Meanwhile, 3-channel processing unit 213 may be turned off in the second operating mode where mode signal MD has the second logic level.
Data reduction unit 214 converts the format of first to third digital signals DS1, DS2 and DS3 into the YUV format to generate the Y signal corresponding to the Y component as first image signal IS1 and generate one of the U and V signals corresponding to the U and V components as the second image signals IS2 in the second operating mode in which mode signal MD has the second logic level. For example, data reduction unit 214 may generate second image signal IS2 by alternately selecting the U and V signals whenever data reduction unit 214 receives first to third digital signals DS1, DS2 and DS3 from column decoder 212. Thus, first and second image signals IS1 and IS2 generated by data reduction unit 214 in the second operating mode may have the YUV422 format. In some embodiments, first to third digital signals DS1, DS2 and DS3 and first and second image signals IS1 and IS2 are 8-bit signals. Therefore, data reduction unit 214 may reduce the amount of data from 24-bit data to 16-bit data. Meanwhile, data reduction unit 214 may be turned off in the first operating mode in which mode signal MD has the first logic level.
Referring to
Y=0.299R+0.587G+0.114B (1)
U=(B−Y)*0.565=(−0.299R−0.587G+0.886B)*0.565 (2)
V=(R−Y)*0.713=(0.701R−0.587G−0.114B)*0.713 (3)
Because first to third digital signals DS1, DS2 and DS3 provided from column decoder 212 correspond to the green, red and blue signals G, R and B, respectively, data reduction unit 214a may convert first to third digital signals DS1, DS2 and DS3 into Y, U and V signals in YUV format through Equations (1)-(3) to output the Y signal as first image signal IS1 and may output one signal selected from the U and V signals based on selection signal SS as second image signal IS2.
Where selection signal SS has a logic low level, first multiplexer 289 may output ‘0’ (zero), second multiplexer 290 may output the third digital signal DS3, and third multiplexer 291 may output 0.565. Where selection signal SS has a logic high level, first multiplexer 289 outputs second digital signal DS2, second multiplexer 290 outputs ‘0’, and third multiplexer 291 outputs 0.713. Thus, where selection signal SS has a logic low level, data reduction unit 214a outputs the Y signal as first image signal IS1 and the U signal as second image signal IS2. Where selection signal SS has a logic high level, data reduction unit 214a outputs the Y signal as first image signal IS1 and the V signal as second image signal IS2. The logic level of selection signal SS may be switched whenever data reduction unit 214a receives first to third digital signals DS1, DS2 and DS3 from column decoder 212. In some embodiments, selection signal SS may be provided from control unit 330. In some other embodiments, selection signal SS is generated from inside data reduction unit 214a.
Meanwhile, first to fourth multipliers 281 to 284, first to fourth adders 285 to 288 and first to third multiplexers 289, 290 and 291 may be turned off in the first operating mode in which mode signal MD has the first logic level.
Referring again to
Meanwhile, 2-channel processing unit 215 may be turned off in the first operating mode where mode signal MD has the first logic level.
Image format unit 216 generates output image data O_DATA by converting format of the 3-channel data 3CH_DATA from 3-channel processing unit 213 or the 2-channel data 2CH_DATA from 2-channel processing unit 215 into a format corresponding to a standard protocol. For example, image format unit 216 may generate output image data O_DATA by converting the format of either the 3-channel data 3CH_DATA or the 2-channel data 2CH_DATA into a format corresponding to Joint Photographic Expert Group (JPEG) standard protocol. In some embodiments, image format unit 216 generates multiple output image data O_DATA by converting the format of the 3-channel data 3CH_DATA or the 2-channel data 2CH_DATA into formats corresponding to multiple standard protocols.
Data processing unit 200a may be integrated on one chip together with pixel array 100. 3-channel processing unit 213, data reduction unit 214, 2-channel processing unit 215 and image format unit 216 in data processing unit 200a may be integrated on separated chips to form an image signal processor (ISP).
As described above, in a first operating mode, image processing device 10 comprising data processing unit 200a of
Referring to
Second data reduction unit 217 converts the format of the 3-channel data 3CH_DATA into YUV format to generate a Y signal corresponding to the Y component as a third image signal IS3 and generate one of U and V signals corresponding to the U and V components as the fourth image signals IS4 in the first operating mode in which mode signal MD has the first logic level. For example, second data reduction unit 217 may generate the fourth image signal IS4 by alternately selecting the U and V signals whenever second data reduction unit 217 receives the 3-channel data 3CH_DATA from 3-channel processing unit 213.
Meanwhile, second data reduction unit 217 may be turned off in the second operating mode in which mode signal MD has the second logic level.
Second data reduction unit 217 may be implemented with data reduction unit 214a of
In the first operating mode in which mode signal MD has the first logic level, 2-channel processing unit 218 performs the 2-channel data processing for the third and fourth image signals IS3 and IS4 provided from second data reduction unit 217 to generate 2-channel data 2CH_DATA comprising two processing signals. In the second operating mode in which mode signal MD has the second logic level, the 2-channel processing unit 218 performs the 2-channel data processing for first and second image signals IS1 and IS2 provided from the first data reduction unit 214 to generate the 2-channel data 2CH_DATA comprising two processing signals. The 2-channel data processing signifies the generation of two output signals by performing a data processing for two input signals independently from each other and/or dependently on each other. For example, the 2-channel processing unit 218 perform a sharpening operation and a color correcting operation for first and second image signals IS1 and IS2 or third and fourth image signals IS3 and IS4 in YUV422 format.
Image format unit 219 generates output image data O_DATA by converting the format of 3-channel data 3CH_DATA from 3-channel processing unit 213 and/or 2-channel data 2CH_DATA from the 2-channel processing unit 218 into a format corresponding to a standard protocol. For example, image format unit 219 may generate output image data O_DATA by converting the format of the 3-channel data 3CH_DATA and/or the 2-channel data 2CH_DATA into a format corresponding to JPEG standard protocol. In an embodiment, image format unit 219 generates multiple output image data O_DATA by converting the format of the 3-channel data 3CH_DATA and/or the 2-channel data 2CH_DATA into formats corresponding to multiple standard protocols.
Data processing unit 200b may be integrated on one chip together with pixel array 100. 3-channel processing unit 213, the first and second data reduction unit 214 and 217, the 2-channel processing unit 218 and image format unit 219 in data processing unit 200b may be integrated on separated chips to form an ISP.
As described above, in a first operating mode, image processing device 10 comprising data processing unit 200b of
Referring to
3-channel analog-digital conversion units 211-1, . . . and 211-n and column decoder 212 in data processing unit 200c of
Comparing data processing unit 200c of
Regardless of the operating mode, first 3-channel processing unit 221 may perform the 3-channel data processing for first to third digital signals DS1, DS2 and DS3 to generate first 3-channel data 3CH_DATA1 comprising three processing signals. first 3-channel processing unit 221 may partially perform the 3-channel data processing performed by 3-channel processing unit 213 in data processing unit 200a of
In the first operating mode where mode signal MD has the first logic level, second 3-channel processing unit 222 may perform the 3-channel data processing for first 3-channel data 3CH_DATA, so that first 3-channel processing unit 222 may generate second 3-channel data 3CH_DATA2 comprising three processing signals. second 3-channel processing unit 222 may perform the 3-channel data processing performed by 3-channel processing unit 213 in data processing unit 200a of
In the second operating mode in which mode signal MD has the second logic level, data reduction unit 223 converts the format of first 3-channel data 3CH_DATA1 into the YUV format to generate the Y signal corresponding to the Y component as first image signal IS1 and generate one of the U and V signals corresponding to the U and V components as the second image signals IS2. For example, data reduction unit 223 may generate second image signal IS2 by alternately selecting the U and V signals whenever data reduction unit 223 receives first 3-channel data 3CH_DATA1 from first 3-channel processing unit 221. Thus, first and second image signals IS1 and IS2 generated by data reduction unit 223 in the second operating mode may have the YUV422 format. Meanwhile, data reduction unit 223 may be turned off in the first operating mode in which mode signal MD has the first logic level.
Data reduction unit 223 may be implemented with data reduction unit 214a of
In the second operating mode in which mode signal MD has the second logic level, the 2-channel processing unit 224 performs the 2-channel data processing for first and second image signals IS1 and IS2 to generate the 2-channel data 2CH_DATA comprising two processing signals. The 2-channel data processing signifies the generation of two output signals by performing the data processing for two input signals independently form each other and/or dependently on each other. For example, the 2-channel processing unit 224 may perform a sharpening operation and a color correcting operation for first and second image signals IS1 and IS2 in YUV422 format. Meanwhile, the second data reduction unit 224 may be turned off in the first operating mode in which mode signal MD has the first logic level.
Image format unit 225 generates output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 received from second 3-channel processing unit 222 or the 2-channel data 2CH_DATA received from the 2-channel processing unit 224 into a format corresponding to a standard protocol. For example, image format unit 225 may generate output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 or the 2-channel data 2CH_DATA into a format corresponding to JPEG standard protocol. In some embodiments, image format unit 225 generates multiple output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 or the 2-channel data 2CH_DATA into formats corresponding to multiple standard protocols.
Data processing unit 200c may be integrated on one chip together with pixel array 100. First and second 3-channel processing units 221 and 222, data reduction unit 223, 2-channel processing unit 224 and image format unit 225 in data processing unit 200c may be integrated on separated chips to form an ISP.
As described above, in a first operating mode, image processing device 10 comprising data processing unit 200c of
Referring to
Second data reduction unit 226 may convert the format of second 3-channel data 3CH_DATA2 provided from second 3-channel processing unit 222 into YUV format to generate a Y signal corresponding to the Y component as a third image signal IS3 and generate one of U and V signals corresponding to the U and V components as the fourth image signals IS4 in the first operating mode in which mode signal MD has the first logic level. For example, the second data reduction unit 226 may generate the fourth image signal IS4_by alternately selecting the U and V signals whenever the second data reduction unit 226 receives second 3-channel data 3CH_DATA2 from second 3-channel processing unit 222. Meanwhile, the second data reduction unit 226 may be turned off in the second operating mode in which mode signal MD has the second logic level.
Second data reduction unit 226 may be implemented with data reduction unit 214a of
In the first operating mode in which mode signal MD has the first logic level, 2-channel processing unit 227 performs the 2-channel data processing for third and fourth image signals IS3 and IS4 provided from second data reduction unit 226 to generate 2-channel data 2CH_DATA comprising two processing signals. In the second operating mode in which mode signal MD has the second logic level, 2-channel processing unit 227 performs the 2-channel data processing for first and second image signals IS1 and IS2 provided from first data reduction unit 223 to generate the 2-channel data 2CH_DATA comprising two processing signals. The 2-channel data processing signifies the generation of two output signals by performing a data processing for two input signals independently from each other and/or dependently on each other. For example, the 2-channel processing unit 227 may perform a sharpening operation and a color correcting operation for first and second image signals IS1 and IS2 or the third and fourth image signals IS3 and IS4 in YUV422 format.
Image format unit 228 generates output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 from second 3-channel processing unit 222 and/or 2-channel data 2CH_DATA from the 2-channel processing unit 227 into a format corresponding to a standard protocol. For example, image format unit 228 may generate output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 and/or the 2-channel data 2CH_DATA into a format corresponding to JPEG standard protocol. In some embodiments, image format unit 228 generates multiple output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 and/or 2-channel data 2CH_DATA into formats corresponding to multiple standard protocols.
Data processing unit 200d may be integrated on one chip together with pixel array 100. First and second 3-channel processing units 221 and 222, the first and second data reduction unit 223 and 226, the 2-channel processing unit 227 and image format unit 228 in data processing unit 200d may be integrated on separated chips to form an ISP.
As described above, in a first operating mode, image processing device 10 comprising data processing units 200d of
Referring to
In the first operating mode where mode signal MD has the first logic level and the second operating mode where mode signal MD has the second logic level, first 3-channel processing unit 231 may be turned on where power control signal PCS has the first logic level, and may perform the 3-channel data processing for first to third digital signals DS1, DS2 and DS3 to generate first 3-channel data 3CH_DATA1 comprising three processing signals. First 3-channel processing unit 231 may partially perform the 3-channel data processing performed by 3-channel processing unit 213 in data processing unit 200a of
Meanwhile, first 3-channel processing unit 231 may be turned off where power control signal PCS has the second logic level in the second operating mode where mode signal MD has the second logic level.
In the first operating mode where mode signal MD has the first logic level, second 3-channel processing unit 232 performs the 3-channel data processing for first 3-channel data 3CH_DATA, so that first 3-channel processing unit 232 generates second 3-channel data 3CH_DATA2 comprising three processing signals. Second 3-channel processing unit 232 performs the 3-channel data processing performed by 3-channel processing unit 213 in data processing unit 200a of
Meanwhile, second 3-channel processing unit 232 may be turned off in the second operating mode where mode signal MD has the second logic level.
Multiplexer 233 outputs second 3-channel data 3CH_DATA2 in the first operating mode where mode signal MD has the first logic level. Multiplexer 233 outputs first 3-channel data 3CH_DATA2 where power control signal PCS has the first logic level in the second operating mode where mode signal MD has the second logic level. Multiplexer 233 outputs first to third digital signals DS1, DS2 and DS3 where power control signal PCS has the second logic level in the second operating mode where mode signal MD has the second logic level.
Regardless of the operating mode, data reduction unit 234 may convert the format of the digital provided from multiplexer 233 into the YUV format to generate the Y signal corresponding to the Y component as first image signal IS1 and to generate one of the U and V signals corresponding to the U and V components as the second image signals IS2. For example, data reduction unit 234 may generate second image signal IS2 by alternately selecting the U and V signals whenever data reduction unit 234 receives data from multiplexer 233. Thus, first and second image signals IS1 and IS2 generated by data reduction unit 234 may have the YUV422 format.
Data reduction unit 234 may be implemented with data reduction unit 214a of
Regardless of the operating mode, the 2-channel processing unit 235 may perform the 2-channel data processing for first and second image signals IS1 and IS2 to generate the 2-channel data 2CH_DATA comprising two processing signals. The 2-channel data processing signifies the generation of two output signals by performing the data processing for two input signals independently from each other and/or dependently on each other. For example, the 2-channel processing unit 235 may perform a sharpening operation and a color correcting operation for first and second image signals IS1 and IS2 in YUV422 format.
Image format unit 236 generates output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 received from second 3-channel processing unit 232 or the 2-channel data 2CH_DATA received from the 2-channel processing unit 235 into a format corresponding to a standard protocol. For example, image format unit 236 may generate output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 or the 2-channel data 2CH_DATA into a format corresponding to JPEG standard protocol. In some embodiments, image format unit 236 generates multiple output image data O_DATA by converting the format of second 3-channel data 3CH_DATA2 or the 2-channel data 2CH_DATA into formats corresponding to multiple standard protocols.
Data processing unit 200e may be integrated on one chip together with pixel array 100. first and second 3-channel processing units 231 and 232, multiplexer 233, data reduction unit 234, the 2-channel processing unit 235 and image format unit 236 in data processing unit 200e may be integrated on separated chips to form an ISP.
As described above, in a first operating mode, image processing device 10 comprising data processing unit 200e of
Referring again to
Various additional examples of data processing unit 200 will be described below with reference to
Referring to
Multiplexers 241-1, . . . and 241-n are connected to the columns of pixel array 100 to receive the red and blue signals RS and BS, respectively. Each of multiplexers 241-1, . . . and 241-n receives corresponding selection signals SS1, . . . and SSn and the control signal MD. Each of multiplexers 241-1, . . . and 241-n outputs red signal RS in the first operating mode where mode signal MD has the first logic level, and outputs one selected from the red and blue signals RS and BS based on the corresponding selection signals SS1, . . . and SSn in the second operating mode as second image signal IS2. For example, in the second operating mode, each of multiplexers 241-1, . . . and 241-n may output red signal RS as second image signal IS2 where the corresponding selection signal SS1, . . . and SSn has the first logic level and the blue signal BS as second image signal IS2 where the corresponding selection signal SS1, . . . and SSn has the second logic level.
The green and blue signals GS and BS provided from each of the columns of pixel array 100 may be directly input to variable processing unit 240.
Thus, variable processing unit 240 may receive the green, red and blue signals GS, RS and BS from the columns of pixel array 100 in the first operating mode. In the second operating mode, variable processing unit 240 receives green signals GS from the columns of pixel array 100 as first image signal IS1, and it receives output signals from multiplexers 241-1, . . . and 241-n as the second image operating mode. As described below, in the second operating mode, blue signals BS directly provided from pixel array 100 to variable processing unit 240 may be ignored.
In some embodiments, in the second operating mode, selection signals SS1, . . . and SSn, which are provided to multiplexers 241-1, . . . and 241-n connected to the columns of pixel array 100 adjacent to each other, may have different logic levels, and the logic levels of selection signals SS1, . . . and SSn may be inverted whenever the green, red and blue signals GS, RS and BS may be output from pixel array 100 in units of row. In this case, as shown of
In certain other embodiments, in the second operating mode, selection signals SS1, . . . and SSn, which are provided to multiplexers 241-1, . . . and 241-n connected to the odd columns of pixel array 100, may have the first logic level and selection signals SS1, . . . and SSn, which are provided to multiplexers 241-1, . . . and 241-n connected to the even columns of pixel array 100, may have the second logic level. In this case, as shown of
Variable processing unit 240 comprises multiple 3-channel analog-digital conversion units 242-1, . . . and 242-n, a column decoder 243 and a 3-channel processing unit 244. Each of the 3-channel analog-digital conversion units 242-1, . . . and 242-n comprises first to third analog-digital converters ADC1, ADC2 and ADC3. In the first operating mode where mode signal MD has the first logic level, first to third analog-digital converters ADC1, ADC2 and ADC3 perform analog-to-digital conversion in parallel with respect to the green, red and blue signals GS, RS and BS to convert the green, red and blue signals GS, RS and BS into first to third digital signals DS1, DS2 and DS3. In the second operating mode, where mode signal MD has the second logic level, the third analog-digital converter ADC3 may be turned off and the first and second analog-digital converters ADC1 and ADC2 may perform an analog-to-digital conversion in parallel with respect to first and second image signals IS1 and IS2 to generate the first and second digital signals DS1 and DS2.
Column decoder 243 may sequentially output first to third digital signals DS1, DS2 and DS3 or the first and second digital signals DS1 and DS2 received from the 3-channel analog-digital conversion units 242-1, . . . and 242-n based on second control signal CON2 provided from control unit 300. 3-channel processing unit 244 comprises first to third processing units 245-1, 245-2 and 245-3.
In the first operating mode, where mode signal MD has the first logic level, 3-channel processing unit 244 may perform the 3-channel data processing for first to third digital signals DS1, DS2 and DS3 by using first to third processing units 245-1, 245-2 and 245-3, so that 3-channel processing unit 244 generates 3-channel data 3CH_DATA comprising three processing signals. The 3-channel data processing signifies the generation of three output signals by performing a data processing for three input signals independently from each other and/or dependently on each other. For example, the 3-channel processing unit 244 may perform a noise removing operation, a sharpening operation and a gamma correction for first to third digital signals DS1, DS2 and DS3.
In the second operating mode, where mode signal MD has the second logic level, 3-channel processing unit 244 may allow third processing unit 245-3 to be turned off and may perform the 2-channel data processing for the first and second digital signals DS1 and DS2 by using only the first and second processing units 245-1 and 245-2 to generate the 2-channel data 2CH_DATA comprising two processing signals. The 2-channel data processing signifies the generation of two output signals by performing the data processing for two input signals independently from each other and/or dependently on each other. For example, the 3-channel processing unit 244 may perform a noise removing operation, a sharpening operation and a gamma correction for the first and second digital signals DS1 and DS2.
Image format unit 246 generates output image data O_DATA by converting the format of the 3-channel data 3CH_DATA2 or the 2-channel data 2CH_DATA received from the 3-channel processing unit 244 into a format corresponding to a standard protocol. For example, image format unit 246 may generate output image data O_DATA by converting the format of the 3-channel data 3CH_DATA or the 2-channel data 2CH_DATA into a format corresponding to JPEG standard protocol. In some embodiments, image format unit 246 generates multiple output image data O_DATA by converting the format of the 3-channel data 3CH_DATA or the 2-channel data 2CH_DATA into formats corresponding to multiple standard protocols.
In some embodiments, data processing unit 200f further comprises a YUV processing unit 247. YUV processing unit 247 converts the format of the 3-channel data 3CH_DATA or the 2-channel data 2CH_DATA received from the 3-channel processing unit 244 into YUV format. YUV processing unit 247 converts the format of the 3-channel data 3CH_DATA into YUV444 format or YUV422 format where receiving the 3-channel data 3CH_DATA from the 3-channel processing unit 244, and it converts the format of the 2-channel data 2CH_DATA into YUV422 format where receiving the 2-channel data 2CH_DATA from the 3-channel processing unit 244. Further, YUV processing unit 247 may perform a data processing for the data converted into the YUV format to generate YUV data YUV_DATA. For example, YUV processing unit 247 may perform a sharpening operation and a color correcting operation. In this case, image format unit 246 may generate output image data O_DATA by converting the format of the 3-channel data 3CH_DATA or the 2-channel data 2CH_DATA received from the 3-channel processing unit 244 into a format corresponding to a standard protocol.
Data processing unit 200f may be integrated on one chip together with pixel array 100. 3-channel processing unit 244, YUV processing unit 247 and image format unit 246 in data processing unit 200f may be integrated on separated chips to form an ISP.
First analog-digital converter ADC1 and first processing unit 245-1 in variable processing unit 240 may constitute a first channel processing circuit. Second analog-digital converter ADC2 and second processing unit 245-2 in variable processing unit 240 may constitute a second channel processing circuit. Third analog-digital converter ADC3 and third processing unit 245-3 in variable processing unit 240 may constitute a third channel processing circuit. As described above, variable processing unit 240 may generate the 3-channel data 3CH_DATA comprising three processing signals by performing the 3-channel data processing for the green, red and blue signals GS, RS and BS by using the first to third channel processing circuits in the first operating mode. In the second operating mode, variable processing unit 240 may allow the third channel processing circuit to be turned off and may generate the 2-channel data 2CH_DATA comprising two processing signals by performing the 2-channel data processing for first and second image signals IS1 and IS2 by using only the first and second channel processing circuits. Thus, image processing device 10 comprising data processing unit 200f of
Referring to
Multiplexer 253 receives second and third digital signals DS2 and DS3 from column decoder 252, and it receives selection and mode signals SS and MD as the control signal. Multiplexer 253 outputs second 3-channel data 3CH_DATA2 in the first operating mode where mode signal MD has the first logic level. In the second operating mode where mode signal MD has the second logic level, multiplexer 253 selects one of second and third digital signals DS2 and DS3 based on the logic level of selection signal SS and outputs the selected signal as second image signal IS2. For example, in the second operating mode, multiplexer 253 may output the second digital signal DS2 as second image signal IS2 where selection signal SS has the first logic level and may output the third digital signal DS3 as the third image signal IS3 where selection signal SS has the second logic level. In some embodiments, selection signal SS is provided from control unit 300. In some other embodiments, selection signal SS is generated within data processing unit 200g.
First and third digital signals DS1 and DS3 provided from column decoder 252 may be directly input to the 3-channel processing unit 254. Therefore, the 3-channel processing unit 254 may receive first to third digital signals DS1, DS2 and DS3 from column decoder 252 in the first operating mode. In the second operating mode, the 3-channel processing unit 254 may receive the first digital signal DS1 from column decoder 252 as first image signal IS1 and the output signal of multiplexer 253 as second image signal IS2. In the second operating mode, the third digital signal DS3 directly provided from column decoder 252 to the 3-channel processing unit 254 may be ignored.
The 3-channel processing unit 254 comprises the first to third processing units 255-1, 255-2 and 255-3. 3-channel processing units 254, YUV processing unit 257 and image format unit 256 in data processing unit 200g of
Comparing data processing unit 200g of
Referring to
First 3-channel processing unit 261 comprises first to third processing units 262-1, 262-2 and 262-3. Regardless of the operating mode, first 3-channel processing unit 261 performs the 3-channel data processing for first to third digital signals DS1, DS2 and DS3 by using the first to third processing units 262-1, 262-2 and 262-3 to generate first 3-channel data 3CH_DATA1 comprising first to third processing signals PS1, PS2 and PS3. First 3-channel processing unit 261 may partially perform the 3-channel data processing performed by the 3-channel processing unit 244 in data processing unit 200f of
Multiplexer 265 receives second and third processing signals PS2 and PS3 from first 3-channel processing unit 261, and it receives the selection and mode signals SS and MD as control signal. multiplexer 265 outputs second processing signal PS2 in the first operating mode where mode signal MD has the first logic level. In the second operating mode, where mode signal MD has the second logic level, multiplexer 265 selects one signal from second and third processing signals PS2 and PS3 based on the logic level of selection signal SS and outputs the selected signal as second image signal IS2. For example, in the second operating mode, multiplexer 265 may output the second processing signal PS2 as second image signal IS2 where selection signal SS has the first logic level and may output third processing signal PS3 as second image signal IS2 where selection signal SS has the second logic level. In some embodiments, selection signal SS is provided from control unit 300. In some other embodiments, selection signal SS is generated within data processing unit 200h.
First and third processing signals PS1 and PS3 provided from first 3-channel processing unit 261 may be directly input to second 3-channel processing unit 263. Therefore, second 3-channel processing unit 263 may receive the first to third processing signals PS1, PS2 and PS3 from first 3-channel processing unit 261 in the first operating mode. In the second operating mode, second 3-channel processing unit 263 may receive the first processing signal PS1 from first 3-channel processing unit 261 as first image signal IS1 and may receive the output signal of multiplexer 265 as second image signal IS2. In the second operating mode, the third processing signal PS3 directly provided from first 3-channel processing unit 261 to second 3-channel processing unit 263 may be ignored.
Second 3-channel processing unit 263 comprises the first to third processing units 264-1, 264-2 and 264-3.
In the first operating mode, where mode signal MD has the first logic level, second 3-channel processing unit 263 may perform the 3-channel data processing for first 3-channel data 3CH_DATA1 comprising the first to third processing signals PS1, PS2 and PS3 by using the first to third processing units 264-1, 264-2 and 264-3. The 3-channel data processing signifies the generation of three output signals by performing a data processing for three input signals independently from each other and/or dependently on each other. Second—3-channel processing unit 263 performs the 3-channel data processing performed by the 3-channel processing unit 244 in data processing unit 200f of
In the second operating mode, where mode signal MD has the second logic level, second 3-channel processing unit 263 may allow third processing unit 264-3 to be turned off and may perform the 2-channel data processing for first and second image signals IS1 and IS2 by using only the first and second processing units 264-1 and 264-2 to generate the 2-channel data 2CH_DATA comprising two processing signals. The 2-channel data processing signifies the generation of two output signals by performing the data processing for two input signals independently from each other and/or dependently on each other. For example, second 3-channel processing unit 263 may perform a noise removing operation, a sharpening operation and a gamma correction for first and second image signals IS1 and IS2.
YUV processing unit 257 and image format unit 256 in data processing unit 200h of
As described above, in the first operating mode, image processing device 10 comprising data processing unit 200h performs 3-channel data processing for the green, red and blue signals GS, RS and BS generated from each unit pixel 110 of pixel array 100 to generate output image data O_DATA, so that an high-quality image may be provided. In addition, in the second operating mode if output image data O_DATA are generated through the 2-channel data processing while third processing unit 264-3 of second 3-channel processing unit 263 is turned off after the 3-channel data processing for the green, red and blue signals GS, RS and BS generated from each unit pixel 110 of pixel array 100 is partially performed to reduce the amount of data, the power consumption may be slightly increased as compared with that of the second operating mode of image processing device 10 comprising data processing unit 200a of
Referring to
Computing system 900 further comprises a memory device 940, an input/output device 950, and a power supply 960. Although not illustrated in
Processor 920 performs various calculations or tasks. In some embodiments, processor 920 may be a microprocessor or a CPU. Processor 920 typically communicates with storage device 930, memory device 940 and the input/output device 950 via an address bus, a control bus, and/or a data bus. In some embodiments, processor 920 is coupled to an extended bus, such as a peripheral component interconnection (PCI) bus.
Storage device 930 typically comprises a non-volatile memory device such as a flash memory device, a solid state drive (SSD), a hard disk drive (HDD), a compact disk read-only memory (CD-ROM) drive, etc.
Memory device 940 may store data required for an operation of electronic device 900. Memory device 940 may be, e.g., a dynamic random access memory (DRAM), a static random access memory (SRAM), or a non-volatile memory, such as an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, etc. Input/output device 950 typically comprises a keyboard, a mouse, a printer, a display device, etc. Power supply 960 supplies operational power.
Image processing device 910 may be connected to processor 920 through one or more of the above buses or other communication links to communicate with processor 920.
Image processing device 910 comprises a pixel array and a data processing unit. The pixel array comprises multiple unit pixels, and each of the unit pixels may generate multiple color signals in response to an incident light. The data processing unit may generate output image data by processing the color signals in parallel in a first operating mode, and generate two image signals for each unit pixel based on the color signals and generate the output image data by processing the two image signals in parallel in a second operating mode.
Image processing device 910 may be embodied with image processing device 10 of
Image processing device 910 may be packaged in various forms, such as package on package (PoP), ball grid arrays (BGAs), chip scale packages (CSPs), plastic leaded chip carrier (PLCC), plastic dual in-line package (PDIP), die in waffle pack, die in wafer form, chip on board (COB), ceramic dual in-line package (CERDIP), plastic metric quad flat pack (MQFP), thin quad flat pack (TQFP), small outline IC (SOIC), shrink small outline package (SSOP), thin small outline package (TSOP), system in package (SIP), multi chip package (MCP), wafer-level fabricated package (WFP), or wafer-level processed stack package (WSP).
In some embodiments, image processing device 910 is integrated with processor 920 in one chip, or image processing device 910 and processor 920 may be implemented as separate chips.
Computing system 900 may be any type of computing system using an image sensor. For example, computing system 900 may comprise a digital camera, a mobile phone, a smart phone, a portable multimedia player (PMP), a personal digital assistant (PDA), etc.
Referring to
A CSI host 1112 of application processor 1110 may perform a serial communication with a CSI device 1141 of image sensor 1140 via a camera serial interface (CSI). In some embodiments, CSI host 1112 comprises a deserializer (DES), and CSI device 1141 comprises a serializer (SER). A display serial interface (DSI) host 1111 of application processor 1110 may perform a serial communication with a DSI device 1151 of display device 1150 via a DSI. In some embodiments, DSI host 1111 comprises a serializer (SER), and DSI device 1151 comprises a deserializer (DES).
Computing system 1000 may further comprise a radio frequency (RF) chip 1160 performing a communication with application processor 1110. A physical layer (PHY) 1113 of computing system 1000 and a physical layer (PHY) 1161 of RF chip 1160 may perform data communications based on a MIPI DigRF. Application processor 1110 may further comprise a DigRF MASTER 1114 that controls the data communications according to the MIPI DigRF of PHY 1161, and RF chip 1160 may further comprise a DigRF SLAVE 1162 controlled by DigRF MASTER 1114.
Computing system 1000 may further comprise a global positioning system (GPS) 1120, a storage 1170, a MIC 1180, a DRAM device 1185, and a speaker 1190. In addition, computing system 1000 may perform communications using an ultra wideband (UWB) 1210, a wireless local area network (WLAN) 1220, a worldwide interoperability for microwave access (WIMAX) 1230, etc. However, the structure and the interface of computing system 1000 are not limited thereto.
The foregoing is illustrative of embodiments and is not to be construed as limiting thereof. Although a few embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible in the embodiments without departing from the scope of the inventive concept as defined in the claims.
Number | Date | Country | Kind |
---|---|---|---|
10-2013-0026036 | Mar 2013 | KR | national |