This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2013-058765, filed on Mar. 21, 2013; the entire contents of which are incorporated herein by reference.
Embodiments described herein relate generally to a multi-view imaging apparatus and a method of sending image data.
In conventional image sending systems, in order to send pieces of image data taken by a plurality of imaging units to an interface unit, the plurality of imaging units and the interface unit are daisy-chain connected to each other in some cases. In such cases, the number of the imaging units, the size (resolution) of the image data taken by each imaging unit, the transmission timing of the image data, and the like need to be set in advance to the interface unit and each imaging unit.
For this reason, in the case where information set in advance is incorrect, a problem that the image data is not correctly sent to the interface unit arises.
A multi-view imaging apparatus of an embodiment includes: a plurality of imaging units each including an image sensor and a memory configured to store therein image data taken by the image sensor, the imaging units being daisy-chain connected to each other in order to send the image data; and an interface unit connected to a lowermost imaging unit, the interface unit being configured to output pieces of image data taken by the plurality of imaging units to an outside. The imaging units each add own-stage data to data outputted from an upper-stage imaging unit, and output the resultant data to a lower-stage imaging unit.
Embodiments will now be explained with reference to the accompanying drawings. It should be noted that, in each drawing, components having equivalent functions are denoted by the same reference signs, and detailed description of such components denoted by the same reference signs will not be repeated.
A multi-view imaging apparatus 1 according to the first embodiment includes: imaging units 10[1] to 10[X] (X: an integer equal to or more than 2); and an interface unit (I/F unit) 50 that is connected to the imaging unit 10[1] and outputs X pieces of image data taken by the imaging units 10[1] to 10[X] to the outside. The imaging units 10[1] to 10[X] are daisy-chain connected to each other. Further, the first imaging unit 10[1] and the last imaging unit 10[X] are connected to each other in order to transmit/receive a control signal.
In the multi-view imaging apparatus 1, the pieces of image data taken by the imaging units 10[1] to 10[X] are sent from the imaging unit 10[X] to the imaging unit 10[1] on a line basis, and are outputted to the interface unit 50. Because such line-based data is sequentially sent, X pieces of image data taken by the imaging units 10[1] to 10[X] are finally outputted to the interface unit 50.
The imaging unit 10[X] further includes a control signal receiver 14, and the imaging unit 10[1] further includes a control signal transmitter 19.
The reception data determiner 16[n] receives (X−n)th data outputted from the upper-stage imaging unit 10[n+1]. Further, the reception data determiner 16[n] performs a process of determining whether or not the reception data is valid. If determining that the reception data is valid, the reception data determiner 16[n] outputs a data reception report to the controller 18[n].
It should be noted that the reception data determiners 16[1] to 16[X−1] may each include a buffer (for example, a line buffer) that stores therein data outputted from the upper-stage imaging unit 10. With this configuration, for example, even in the case where data sending is retarded, a loss of received data can be prevented.
Upon reception of the data reception report from the reception data determiner 16[n], the controller 18[n] instructs the transmission data generator 17[n] to generate (X−n+1)th data. Further, the controller 18[n] instructs the transmission data generator 17[n] to read out line data to be next transmitted (line data to which a desired number is given).
The transmission data generator 17[n] reads the line data to which a desired number is given, out of the line memory 12. The transmission data generator 17[n] adds the read-out line data to the (X−n)th data, and thus generates the (X−n+1)th data. Then, the transmission data generator 17[n] outputs the (X−n+1)th data to the subsequent-stage imaging unit 10[n−1]. It should be noted that the transmission data generator 17[1] outputs Xth data to the interface unit 50. Further, upon completion of transmission of the (X−n+1)th data, the transmission data generator 17[n] reports transmission completion to the controller 18[n].
It should be noted that there is no data reception of the transmission data generator 17[X] from the upper stage. The transmission data generator 17[X] reads the line data to which a desired number is given, out of the line memory 12, adds a header thereto, and thus generates first data. Then, the transmission data generator 17[X] outputs the first data to the subsequent-stage imaging unit 10[X−1]. The header contains information concerning the size of the image data, for example, a line data length (W) and a total number of lines (H).
Upon reception of the transmission completion report from the transmission data generator 17[1], the controller 18[1] reports the control signal transmitter 19 of the imaging unit 10[1] to that effect. Upon transmission of the Xth data to the interface unit 50, the control signal transmitter 19 transmits a control signal to the imaging unit 10[X]. The control signal receiver 14 of the imaging unit 10[X] receives, from the imaging unit 10[1], the control signal indicating that the imaging unit 10[1] is in a receivable state (in other words, the imaging unit 10[X] can transmit data). On the basis of this control signal, the controller 18[X] instructs the transmission data generator 17[X] to start transmission of the next line data.
Next, an operation of the multi-view imaging apparatus according to the first embodiment will be described. First, upon imaging by each of the image sensors 11[1] to 11[X] of the imaging units 10[1] to 10[X], the obtained image data is stored in each of the line memories 12[1] to 12[X]. The last imaging unit 10[X] adds a header containing information (W, H) concerning the size of the image data to line data with a line number 0 in the image data, thus generates first data, and outputs the first data to the imaging unit 10[X−1].
Next, the imaging unit 10[X−1] receives the first data, and the reception data determiner 16[X−1] determines whether or not this data is valid. If determining that this data is valid, the reception data determiner 16[X−1] outputs a data reception report to the controller 18[X−1]. Upon reception of the data reception report, the controller 18[X−1] instructs the transmission data generator 17[X−1] to generate second data. The transmission data generator 17[X−1] adds its own-stage line data with a line number 0, to the received first data, and thus generates the second data. The transmission data generator 17[X−1] outputs the second data to the imaging unit 10[X−2].
In this way, each imaging unit 10 sequentially transfers data obtained by adding its own-stage image data (line data) to received data. Finally, the imaging unit 10[1] transmits the Xth data containing pieces of line data with a line number 0 of all the imaging units, to the interface unit 50.
Upon transmission of the Xth data, the imaging unit 10[1] transmits a control signal reporting completion of data transmission corresponding to one line from each imaging unit 10, to the imaging unit 10[X]. Upon reception of this control signal, the imaging unit 10[X] outputs line data with a line number 1 to the imaging unit 10[X−1].
Such data transfer as described above is repeated until the final line, whereby all pieces of image data taken by each imaging unit 10 are transmitted to the interface unit 50.
In the multi-view imaging apparatus 1 of the present invention, image data transfer is performed while image data taken by the imaging unit at each stage is sequentially added to image data taken by the imaging unit 10[X], and hence the number of imaging units does not need to be set in advance. Further, the size of image data is contained in a header, and hence the size of image data also does not need to be set in advance. In addition, a transmission process at each stage is started upon data reception as the trigger, and hence transmission timing also does not need to be set in advance.
Accordingly, the first embodiment can provide a multi-view imaging apparatus that can autonomously send image data without making settings in advance. Further, because the need to make settings in advance is eliminated, even the case where the number of imaging units or resolution is changed can be easily dealt with.
Next, a generation example of transmission data in the transmission data generator will be described.
In the case as illustrated in
In a determination process, the reception data determiner 16[i] (i: an integer from 1 to X−1) checks whether or not a set last flag is at the tail end of (X−1)th data. If the set last flag is thereat, the reception data determiner 16[i] determines that the data is valid.
Then, when generating (X−i+1)th data, the transmission data generator 17[i] clears the last flag in the (X−i)th data, and adds line data having a tail end to which a last flag is set, to after the (X−i)th data.
That is, the imaging unit 10[i] detects the last flag at the tail end of the data received from the imaging unit 10[i+1], and changes the last flag from “1” to “0”. Moreover, the imaging unit 10[i] sets “1” to the last flag at the tail end of its own-stage line data, and transmits the resultant data.
In a scheme illustrated in
In the case of
Then, when generating the (X−i+1)th data, the transmission data generator 17[i] adds line data with a desired line number to after the header of the (X−i)th data.
In the case of a scheme illustrated in
In the meantime, in the case where line data with a line number to be transmitted is missing due to a trouble in the image sensor, a difference in image sensor resolution between the imaging units 10, and the like, a position in the image, of line data added at its own stage are displaced from those of other imaging units in some cases. In order to smoothly send image data even in such cases, transmission data may contain a line number (N) and a final line flag (L). That is, the transmission data generator 17[i] (i: an integer from 1 to X) may add, to the line data added to the (X−i)th data, the line number (N) of this line data and the final line flag (L) indicating whether or not this line data is in the final line.
In the case where line data to be transmitted is missing, the transmission data generator 17[i] uses, for example, dummy data or line data with the next line number. Consequently, image data can be smoothly sent.
In the case where the sizes of pieces of image data taken by the imaging units 10 are different from one another, an imaging unit having low resolution may receive data from the upper-stage imaging unit even after the end of sending of all pieces of its own image data. The transmission data generator 17 of the imaging unit that has already sent all pieces of line data adds dummy data to which the final line flag (L) is set, and transmits the resultant data. Consequently, image data can be smoothly sent. It should be noted that the interface unit 50 performs a process of ignoring or deleting the dummy data, that is, the line data of second and subsequent times to which the final line flag (L) is set.
Further, in the case where the sizes of pieces of image data taken by the imaging units 10 are the same as one another, the last imaging unit 10[X] adds the line number (N), and the other imaging units may not add the line number (N). In this case, the transmission data generator 17[X] stores the line number of line data transmitted to the imaging unit 10[X−1], in the header of the first data. Meanwhile, the transmission data generator 17[i] (i: an integer from 1 to X−1) adds line data corresponding to the line number stored in the header of the first data, to the (X−i)th data.
Further, the present embodiment can deal with even the case where the sizes of pieces of image data taken by the imaging units 10 are different from one another. In this case, similarly to the imaging unit 10[X], the imaging unit 10[i] (i: an integer from 1 to X−1) adds a header containing information concerning the size of its own-stage image data, to its own-stage line data. Consequently, for example, even in the case where horizontal image resolution is different for each imaging unit 10, each imaging unit 10 sends line data having a different length, whereby image sending can be achieved without making settings in advance. That is, the horizontal resolution and/or the vertical resolution of an image taken by the imaging unit can be freely set. The present scheme is advantageous in, for example, the case where, in a multi-view imaging apparatus including imaging units arranged in a grid-like pattern, high-resolution image sensors are placed in a central region thereof, and low-resolution image sensors are placed in a peripheral region thereof.
Hereinafter, detailed description of components described in the first embodiment will be omitted, and only differences from the first embodiment will be described.
In the multi-view imaging apparatus 2, pieces of image data taken by the image sensors 11[1] to 11[X] are sent from the imaging unit 20[X] to the imaging unit 20[1] on a line basis.
The reception data determiner 16[n] includes a line buffer 16a, and stores, in the line buffer 16a, (X−n)th data received from the upper-stage imaging unit 20[n+1]. It should be noted that, not limited to a line buffer, the line buffer 16a may be a different buffer.
Upon completion of transmission of (X−n+1)th data, the transmission data generator 17[n] reports transmission completion to the controller 22[n].
The transmission possibility/impossibility determiners 23[2] to 23[X] each determine a transmittable state on the basis of a control signal received from the imaging unit 20[n−1] and a data reception report from the reception data determiner 16[n]. Further, upon transmission of the (X−n+1)th data to the imaging unit 20[n−1], the transmission possibility/impossibility determiner 23[n] transmits a control signal indicating that its own stage is in a receivable state, to the imaging unit 20[n+1]. Upon transmission of Xth data to the interface unit 50, the transmission possibility/impossibility determiner 23[1] transmits a control signal indicating that its own stage is in a receivable state, to the imaging unit 20[2].
If the transmission possibility/impossibility determiner 23[n] determines a transmittable state, the controller 22[n] instructs the transmission data generator 17[n] to generate the (X−n+1)th data. Further, the controller 22[n] instructs the transmission data generator 17[n] to read out line data to be next transmitted (line data to which a desired number is given).
Next, an operation of the multi-view imaging apparatus according to the second embodiment will be described. First, upon imaging by each of the image sensors 11[1] to 11[X] of the imaging units 20[1] to 20[X], the obtained image data is stored in each of the line memories 12[1] to 12[X]. The last imaging unit 20[X] adds a header containing information (W, H) concerning the size of the image data to line data with a line number 0 in the image data, thus generates first data, and outputs the first data to the imaging unit 20[X−1].
Next, the imaging unit 20[X−1] receives the first data, and the reception data determiner 16[X−1] determines whether or not this data is valid. If determining that this data is valid, the imaging unit 20[X−1] adds line data with a line number 0 in the image data to the first data, thus generates second data, and outputs the second data to the imaging unit 20[X−2], on the basis of a control signal received from the imaging unit 20[X−2]. Upon completion of the data transmission, the imaging unit 20[X−1] transmits a control signal indicating that the imaging unit 20[X−1] is in a receivable state, to the imaging unit 20[X].
In this way, each imaging unit sequentially transfers data obtained by adding its own-stage image data (line data) to received data. Finally, the imaging unit 20[1] transmits the Xth data containing pieces of line data with a line number 0 of all the imaging units, to the interface unit 50.
It should be noted that the same operation is performed in second and subsequent rounds (sending of pieces of line data with line numbers equal to and more than 1). That is, upon reception of a control signal indicating that the imaging unit 20[n−1] (n is an integer that is equal to or more than 2 and equal to or less than X) is in a receivable state, the imaging unit 20[n] adds line data with the corresponding line number, thus generates the (X−n+1)th data, and outputs the generated data to the imaging unit 20[n−1]. Upon completion of the data transmission, the imaging unit 20[n] transmits a control signal to the imaging unit 20[n+1].
Such data transfer as described above is repeated until the final line, whereby all pieces of image data taken by each imaging unit 20 are transmitted to the interface unit 50.
As has been described above, in the multi-view imaging apparatus according to the second embodiment, similarly to the first embodiment, the number of imaging units, the size of image data, the transmission timing of image data, and the like do not need to be set in advance. Accordingly, the second embodiment can provide a multi-view imaging apparatus that can autonomously send image data without making settings in advance.
Moreover, in the second embodiment, transmission is performed at each stage on the basis of a control signal received from the lower-stage imaging unit, without waiting for the imaging unit 20[1] to transmit data to the interface unit 50. Hence, data sending speed can be increased.
It should be noted that, on the basis of states of all the imaging units from its own stage to the imaging unit 20[1], the transmission possibility/impossibility determiners 23[2] to 23[X−1] may determine whether or not data can be transmitted. This makes transmission possibility/impossibility determination more accurate, so that image data can be more smoothly sent.
The transmission possibility/impossibility determiner 23[n] receives a control signal wReadyIn[n] (=wReadyOut[n−1]) from the lower-stage imaging unit 20[n−1], and transmits a control signal wReadyOut[n] (=wReadyIn[n+1]) to the upper-stage imaging unit 20[n+1].
Upon reception, from the reception data determiner 16[n], of a signal wLast[n] indicating whether or not a determination process (for example, a process of checking the last flag C) is completed, the transmission possibility/impossibility determiner 23[n] transmits a signal rReady[n] and a signal rLast[n] to the controller 22[n].
The circuit illustrated in
wReady[n]=wReady[n−1] & !(rLast[n−1] & !rReady[n−1]) & !(wLast[n] & rReady[n]) (1)
Here, part of the terms in the right-hand side of Expression (1) corresponds to an output of the transmission possibility/impossibility determiner 23[n−1], and hence Expression (1) can be transformed in the following manner.
wReadyIn[n] in Expression (2) becomes active while a control signal is received from the lower-stage imaging unit. !(wLast[n] & rReady[n]) in Expression (2) becomes active while a determination process is not performed by the lower-stage imaging unit.
It should be noted that wReady[n−1] in Expression (1) is a signal that propagates over a plurality of imaging units, and thus may be a critical path. Meanwhile, it is sufficient for a wReady signal to propagate to the imaging unit 20[X] until transfer of data corresponding to one transaction is completed, and hence the wReady signal can be a path for a plurality of cycles. Accordingly, an embodiment in which a flip-flop is disposed in the course of the wReady signal path can be assumed.
In
It should be noted that, in
Further, in
Hereinabove, the first and second embodiments have been described. In the above description, the imaging units included in the multi-view imaging apparatus are categorized into three types of the first imaging unit, the intermediary imaging unit(s), and the last imaging unit, but a multi-view imaging apparatus including imaging units all having the same configuration can also be assumed. In this case, each imaging unit determines its own position by means of connection of a control line for control signal exchange, and performs an operation suited to its own position. In the case of the first embodiment, for example, if a given imaging unit is connected to the interface unit, the given imaging unit is recognized as the first imaging unit. On the other hand, if a given imaging unit is not connected to the interface unit but connected to a control line, the given imaging unit is recognized as the last imaging unit. If neither applies, the given imaging unit is recognized as the intermediary imaging unit. In the case of the second embodiment, for example, if a given imaging unit is connected to the interface unit, the given imaging unit is recognized as the first imaging unit. On the other hand, if only the receiving side of a control line is connected to a given imaging unit, the given imaging unit is recognized as the last imaging unit. If neither applies, the given imaging unit is recognized as the intermediary imaging unit. If all the imaging units are thus configured in the same manner, the manufacturability and expandability of the multi-view imaging apparatus can be improved.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Number | Date | Country | Kind |
---|---|---|---|
2013-058765 | Mar 2013 | JP | national |