Patent Application
20090226144
The present invention relates to generation of a digest scene, and more particularly to generation of a digest scene by calculating a feature amount of a video or audio transmitted through a television broadcast and determining a specific important scene based on the calculated feature amount.
Conventionally, there is a digest (summary) generation device for calculating a feature amount of a video and audio transmitted through a television broadcast so as to determine an important scene by means of the calculated feature amount. In such a digest generation device, the following scheme is generally used for generating a digest. Firstly, a feature amount of the video and audio is calculated for one program based on an AV signal which has been recorded on a recording medium, and a Commercial Messages (CM) segment is detected based on the calculated feature amount, thereby calculating time information, for example, on a playlist for reproducing the digest based on segments excluding the CM segment.
A configuration of a digest generation device adopting the above-mentioned scheme will be described with reference to
Furthermore, there is another scheme in which a feature amount is calculated simultaneously when recording a program so as to be previously stored on a recording medium.
[patent document 1] Japanese Laid-Open Patent Publication No. 2004-128550
[patent document 2] Japanese Laid-Open Patent Publication No. 10-039890
[patent document 3] Japanese Laid-Open Patent Publication No. 2001-119649
The aforementioned digest generation devices have the following problems. Firstly, in a first scheme, when a digest reproduction start instruction is received from the user, for example, after finishing recoding the program, processes such as a feature amount calculation, CM segment detection, digest scene detection and digest information creation are executed. Therefore, there is a problem in that after receiving the digest reproduction start instruction, awaiting time period is generated until the program actually starts to be reproduced. Also, in a second scheme, while recording the program, the feature amount is calculated and the information on the scene which is to be determined as the digest candidate is detected. Thus, a time period required for a process of calculating the feature amount can be reduced as compared to the first scheme in which the process of calculating the feature amount is executed at the time of the reproduction start instruction. However, for detecting the CM segment, the beginning and terminating ends of the CM segment cannot be determined in real time. Thus, the CM segment has to be detected after finishing recording the program (at the time of the reproduction start instruction, for example). Therefore, even in this scheme, the waiting time period required for a process of creating the digest information is generated. In general, a general consumer product such as a DVD recorder, in particular, typically stores a CPU having approximately one-tenth of performance of that stored in a personal computer. Therefore, the aforementioned waiting time period is prolonged, thereby providing the user with unfavorable impressions such as uncomfortable feelings and poor usability due to the aforementioned waiting time period.
Therefore, an object of the present invention is to provide a digest generation device for creating no waiting time required for the process of generating, after finishing recording a program, the digest information of the program.
To achieve the above object, the present invention has the following aspects.
A first aspect is a digest generation device for generating, when receiving broadcast signals of a program to be broadcast and recording the broadcast signals on a recording medium, digest scene information concerning the program, comprising a feature amount calculating section, a specific segment end detecting section, and a digest scene information creating section. The feature amount calculating section calculates, each time the broadcast signals corresponding to a unit time period are received, at least one type of a feature amount indicating a characteristic of at least one of video and audio included in the broadcast signals based on the received broadcast signals corresponding to the unit time period. The specific segment end detecting section detects time points of a beginning end and a terminating end of a specific segment by determining, each time the feature amount is calculated, whether or not a predetermined time point included in a portion of signals, for which the feature amount is already calculated, among the received broadcast signals, is either the beginning end or the terminating end. The digest scene information creating section determines, each time the feature amount is calculated, whether or not the broadcast signals included in segments, other than the specific segment, of an entire segment of the program are included in a digest scene based on the feature amount so as to generate digest scene information.
In a second aspect based on the first aspect, the digest scene information creating section includes a digest segment detecting section for detecting digest candidate segments from the received broadcast signals by determining, each time the feature amount included in the broadcast signals corresponding to the unit time period is calculated, whether or not a content included in the broadcast signals corresponding to the unit time period is the digest scene based on the feature amount. Furthermore, the digest scene information creating section determines, each time the specific segment end detecting section detects a pair of the beginning end and the terminating end of the specific segment, whether or not the specific segment from the beginning end to the terminating end overlaps one of the digest candidate segments, and generates information indicating at least one segment, other than the one of the digest candidate segments which overlaps the specific segment, included in the digest candidate segments detected by the digest segment detecting section as the digest scene information.
In a third aspect based on the first aspect, the digest scene information creating section includes a temporary storage section for storing the calculated feature amount for a predetermined time period from a most recent calculation time point. Furthermore, the digest scene information creating section determines, each time the feature amount is calculated, whether or not the most recent calculation time point of the feature amount stored in the temporary storage section is included in the specific segment from the beginning end to the terminating end which are detected by the specific segment end detecting section, and detects, when it is determined that the most recent calculation time point is not included in the specific segment, at least one content of the digest scene from contents included in the broadcast signals corresponding to the unit time period so as to generate the digest scene information.
In a fourth aspect based on the second aspect, the feature amount calculating section calculates a first feature amount and a second feature amount, the specific segment end detecting section determines the beginning end or the terminating end of the specific segment based on the first feature amount, and the digest segment detecting section detects any of the digest candidate segments based on the second feature amount.
In a fifth aspect based on the first aspect, the specific segment end detecting section includes: a specific segment candidate detecting section for detecting, when the feature amount satisfies a predetermined condition, a segment including only the feature amount satisfying the condition as a specific segment candidate; and a specific segment determining section for detecting a candidate of the beginning end or the terminating end of the specific segment based on a time difference between the specific segment candidate and another specific segment candidate, both of which are included in the program.
In a sixth aspect based on the fifth aspect, each time the specific segment candidate is detected, the specific segment determining section determines, if a time point which is a predetermined time period prior to the detected specific segment candidate is included in an already-detected specific segment candidate, the time point which is the predetermined time period prior to the detected specific segment candidate as the beginning end of the specific segment and the detected specific segment candidate as the terminating end of the specific segment.
In a seventh aspect based on the fifth aspect, the specific segment detecting section includes: a determination section for determining, each time the specific segment candidate is detected, whether or not an already-detected specific segment candidate exists at a time point which is a predetermined first time period prior to a most recently detected specific segment candidate or at a time point which is a predetermined second time period prior to the most recently detected specific segment candidate; an addition section for adding, when the determination section determines that the already-detected specific segment candidate exists at either of the time points, a point to each of the already-detected specific segment candidate and the most recently detected specific segment candidate; a beginning end determining section for determining, each time a predetermined third time period is elapsed since a target candidate having the point greater than or equal to a predetermined value is detected, whether or not the specific segment candidate having the point greater than or equal to the predetermined value exists at a time point which is the predetermined third time period prior to the target candidate, and determining, if the specific segment candidate having the point greater than or equal to the predetermined value does not exist at the time point which is the predetermined third time period prior to the target candidate, the target candidate as the beginning end of the specific segment; and a terminating end determining section for determining, each time the predetermined third time period is elapsed since the target candidate having the point greater than or equal to the predetermined value is detected, whether or not the specific segment candidate having the point greater than or equal to the predetermined value exists at a time point at which the predetermined third time period is elapsed, and determining, if the specific segment candidate having the point greater than or equal to the predetermined value does not exist at the time point at which the predetermined third time period is elapsed, the target candidate as the terminating end of the specific segment.
In an eighth aspect based on the fifth aspect, the feature amount calculating section calculates an audio power level of an audio signal as the feature amount, and the specific segment candidate detecting section detects a silent segment having a power level smaller than or equal to a predetermined value as the specific segment candidate.
In a ninth aspect based on the fifth aspect, the feature amount calculating section calculates brightness level information obtained based on a video signal as the feature amount, and the specific segment candidate detecting section detects a scene change point having a change amount, of the brightness level information, greater than or equal to a predetermined value as the specific segment candidate.
A tenth aspect is a digest generation method of generating, when receiving broadcast signals of a program to be broadcast and recording the broadcast signals on a recording medium, digest scene information concerning the program, comprising a feature amount calculation step, a specific segment end detecting step, and a digest scene information creating step. The feature amount calculating step calculates, each time the broadcast signals corresponding to a unit time period are received, at least one type of a feature amount indicating a characteristic of at least one of video and audio included in the broadcast signals based on the received broadcast signals corresponding to the unit time period. The specific segment end detecting step detects time points of a beginning end and a terminating end of a specific segment by determining, each time the feature amount is calculated, whether or not a predetermined time point included in a portion of signals, for which the feature amount is already calculated, among the received broadcast signals, is either the beginning end or the terminating end. The digest scene information creating step determines, each time the feature amount is calculated, whether or not the broadcast signals included in segments, other than the specific segment, of an entire segment of the program are included in a digest scene based on the feature amount so as to generate digest scene information.
In an eleventh aspect based on the tenth aspect, the digest scene information creating step includes a digest segment detecting step of detecting digest candidate segments from the received broadcast signals by determining, each time the feature amount included in the broadcast signals corresponding to the unit time period is calculated, whether or not a content included in the broadcast signals corresponding to the unit time period is the digest scene based on the feature amount. Furthermore, the digest scene information creating step determines, each time the specific segment end detecting step detects a pair of the beginning end and the terminating end of the specific segment, whether or not the specific segment from the beginning end to the terminating end overlaps one of the digest candidate segments so as to generate information indicating at least one segment, other than the one of the digest candidate segments which overlaps the specific segment, included in the digest candidate segments detected by the digest segment detecting step as the digest scene information.
In a twelfth aspect based on the tenth aspect, the digest scene information creating step includes a temporary storage step of storing the calculated feature amount for a predetermined time period from a most recent calculation time point. Furthermore, the digest scene information creating step determines, each time the feature amount is calculated, whether or not the most recent calculation time point of the feature amount stored in the temporary storage step is included in the specific segment from the beginning end to the terminating end which are detected by the specific segment end detecting step, and detects, when it is determined that the most recent calculation time point is not included in the specific segment, at least one content of the digest scene from contents included in the broadcast signals corresponding to the unit time period so as to generate the digest scene information.
A thirteenth aspect is a recoding medium storing a digest generation program executed by a computer of a digest generation device for generating, when receiving broadcast signals of a program to be broadcast and recording the broadcast signals on a recording medium, digest scene information concerning the program, the digest generation program storing a feature amount calculation step, a specific segment end detecting step, and a digest scene information creating step. The feature amount calculating step calculates, each time the broadcast signals corresponding to a unit time period are received, at least one type of a feature amount indicating a characteristic of at least one of video and audio included in the broadcast signals based on the received broadcast signals corresponding to the unit time period. The specific segment end detecting step detects time points of a beginning end and a terminating end of a specific segment by determining, each time the feature amount is calculated, whether or not a predetermined time point included in a portion of signals, for which the feature amount is already calculated, among the received broadcast signals, is either the beginning end or the terminating end. The digest scene information creating step determines, each time the feature amount is calculated, whether or not the broadcast signals included in segments, other than the specific segment, of an entire segment of the program are included in a digest scene based on the feature amount so as to generate digest scene information.
In a fourteenth aspect based on the thirteenth aspect, the digest scene information creating step includes a digest segment detecting step of detecting digest candidate segments from the received broadcast signals by determining, each time the feature amount included in the broadcast signals corresponding to the unit time period is calculated, whether or not a content included in the broadcast signals corresponding to the unit time period is the digest scene based on the feature amount. Furthermore, the digest scene information creating step determines, each time the specific segment end detecting step detects a pair of the beginning end and the terminating end of the specific segment, whether or not the specific segment from the beginning end to the terminating end overlaps one of the digest candidate segments, and generates information indicating at least one segment, other than the one of the digest candidate segments which overlaps the specific segment, included in the digest candidate segments detected by the digest segment detecting step as the digest scene information.
In a fifteenth aspect based on the thirteenth aspect, the digest scene information creating step includes a temporary storage step of storing the calculated feature amount for a predetermined time period from a most recent calculation time point. Furthermore, the digest scene information creating step determines, each time the feature amount is calculated, whether or not the most recent calculation time point of the feature amount stored in the temporary storage step is included in the specific segment from the beginning end to the terminating end which are detected by the specific segment end detecting step, and detects, when it is determined that the most recent calculation time point is not included in the specific segment, at least one content of the digest scene from contents included in the broadcast signals corresponding to the unit time period so as to generate the digest scene information.
A sixteenth aspect is an integrated circuit used for a digest generation device for generating, when receiving broadcast signals of a program to be broadcast and recording the broadcast signals on a recording medium, digest scene information concerning the program, comprising a feature amount calculating section, a specific segment end detecting section, and a digest scene information creating section. The feature amount calculating section calculates, each time the broadcast signals corresponding to a unit time period are received, at least one type of a feature amount indicating a characteristic of at least one of video and audio included in the broadcast signals based on the received broadcast signals corresponding to the unit time period. The specific segment end detecting section detects time points of a beginning end and a terminating end of a specific segment by determining, each time the feature amount is calculated, whether or not a predetermined time point included in a portion of signals, for which the feature amount is already calculated, among the received broadcast signals, is either the beginning end or the terminating end. The digest scene information creating section determines, each time the feature amount is calculated, whether or not the broadcast signals included in segments, other than the specific segment, of an entire segment of the program are included in a digest scene based on the feature amount so as to generate digest scene information.
In a seventeenth aspect based on the sixteenth aspect, the digest scene information creating section includes a digest segment detecting section for detecting digest candidate segments from the received broadcast signals by determining, each time the feature amount included in the broadcast signals corresponding to the unit time period is calculated, whether or not a content included in the broadcast signals corresponding to the unit time period is the digest scene based on the feature amount. Furthermore, the digest scene information creating section determines, each time the specific segment end detecting section detects a pair of the beginning end and the terminating end of the specific segment, whether or not the specific segment from the beginning end to the terminating end overlaps one of the digest candidate segments, and generates information indicating at least one segment, other than the one of the digest candidate segments which overlaps the specific segment, included in the digest candidate segments detected by the digest segment detecting section as the digest scene information.
In an eighteenth aspect based on the sixteenth aspect, the digest scene information creating section includes a temporary storage section for storing the calculated feature amount for a predetermined time period from a most recent calculation time point. Furthermore, the digest scene information creating section determines, each time the feature amount is calculated, whether or not the most recent calculation time point of the feature amount stored in the temporary storage section is included in the specific segment from the beginning end to the terminating end which are detected by the specific segment end detecting section, and detects, when it is determined that the most recent calculation time point is not included in the specific segment, at least one content of the digest scene from contents included in the broadcast signals corresponding to the unit time period so as to generate the digest scene information.
According to the first invention, the specific segment (a CM segment, for example) can be detected while recording a program. Therefore, the digest scene information in which the specific segment is removed can be simultaneously generated while recording the program. Thus, a waiting time period required for executing, after finishing recording the program, a process of generating the digest scene information can be eliminated, thereby making it possible to provide the user with a conformable digest reproduction operation. Furthermore, in the case where a follow-up reproduction is performed while recording the program, a digest reproduction also can be reproduced up to a time period close to that at which the program is currently being recorded thereby making it possible to provide the user with a reproduction environment with better usability.
According to the second and third inventions, an effect similar to that of the first invention can be obtained.
According to the fourth invention, the two types of feature amounts are used. Therefore, either of the feature amounts which is appropriate for detecting each of the specific segment and the digest segment is used, thereby making it possible to more accurately detect each of the specific segment and the digest segment.
According to the fifth and sixth inventions, the specific segment is determined based on the time interval between time points of the specific segment candidate and the said another specific segment candidate. Thus, it becomes possible to more accurately determine the specific segment.
According to the seventh invention, the point is added to each of the specific segment candidates based on the predetermined time intervals. Therefore, it becomes possible to assess how likely each of the specific segment candidates is to be located at the beginning end or the terminating end of the specific segment. Furthermore, the specific segment candidate having a higher point is determined as the beginning end or the terminating end of the specific segment, thereby making it possible to prevent a specific segment candidate accidentally existing in a program from being mistakenly determined as the beginning end or the terminating end of the specific segment. As a result, it becomes possible to create the digest scene information in which the specific segment is more accurately removed.
According to the eighth invention, the silent segment is the specific segment candidate. Therefore, the specific segment such as the CM segment can be more accurately detected, utilizing properties that the silent segments are located at the both beginning and end of the CM segment.
According to the ninth invention, the scene change point at which the brightness level information is substantially changed is the specific segment candidate. Therefore, a scene change portion, from a program to the specific segment, in which the brightness level information is substantially changed can be determined as the specific segment candidate. As a result, it becomes possible to more accurately determine the specific segment.
According to the tenth to eighteenth inventions, an effect similar to that of the first embodiment can be obtained.
According to the present invention, a digest scene list indicating positions of digest scenes is simultaneously created while recording a program. In embodiments of the present invention to be described below, a scene locally having a large value of an audio power level, i.e., a scene of excitement, is adopted as a digest scene. Therefore, a scene having an audio power level greater than or equal to a predetermined value is extracted as a digest candidate segment. In accordance with this, a segment having the audio power level smaller than or equal to the predetermined value is also extracted as a silent segment, and a segment between the silent segments appearing at times having a predetermined interval (15 seconds, for example) therebetween is extracted as a Commercial Messages (CM) segment. This is because, since the CM segment has properties that the silent segments exit at the beginning and end of the CM segment and the CM segment has a constant length, a portion between the silent segments appearing at times having a constant time interval therebetween may be considered as the CM segment. Each time the CM segment is extracted, information corresponding to the CM segment is removed from information on the digest candidate segment, thereby creating the digest scene list indicating the digest scenes included in a program segment. Note that in the present embodiments, a maximum length of one CM segment is 60 seconds.
The receiving section 11 receives a broadcast signal and demodulates the signal into a video and audio signal (hereinafter an AV signal). Also, the receiving section 11 outputs the demodulated AV signal to the feature amount calculating section 12, the mass recording medium 17, and the reproduction controlling section 18.
The feature amount calculating section 12 analyzes the AV signal so as to calculate a feature amount, and outputs the feature amount to the silent segment detecting section 13 and the candidate segment detecting section 14. Note that the feature amount is used for determining the CM segment or digest scene included in the program. As the feature amount used for determining the CM segment, an audio feature amount such as a power level or power spectrum of an audio signal may be used, for example, since the CM segment is determined based on a time interval between time points at which the silent segments appear as described above. On the other hand, as the feature amount used for determining the digest scene, a video feature amount such as brightness level information and a motion vector of the video signal, or the audio feature amount such as the power level or power spectrum of the audio signal may be used, for example. In the present embodiment, the power level of the audio signal is used, as the feature amount, for determining both the CM segment and the digest scene.
The silent segment detecting section 13 detects the silent segment included in the program based on the aforementioned feature amount, and generates silent segment information 24. Also, the silent segment detecting section 13 outputs the silent segment information 24 to the CM segment determining section 15. The candidate segment detecting section 14 detects a segment which is to be determined as a digest scene candidate (hereinafter a candidate segment) included in the program based on the aforementioned feature amount, and generates candidate segment information 25. Also, the candidate segment detecting section 14 outputs the candidate segment information 25 to the digest list creating section 16.
Based on the silent segment information 24, the CM segment determining section 15 determines the CM segment by checking the time interval between the time points at which the silent segments appear. Then, the CM segment determining section 15 outputs the determined CM segment to the digest list creating section 16 as the CM segment information 27.
Based on the candidate segment information 25 and the CM segment information 27, the digest list creating section 16 creates a digest scene list 28 which is information indicating the positions of the digest scenes. Then, the digest list creating section 16 outputs the digest scene list 28 to the mass recording medium 17 and the reproduction controlling section 18.
The mass recording medium 17 is a medium for recording the AV signal or the digest scene list 28 thereon, and is a DVD, an HDD or the like.
The reproduction controlling section 18 performs a reproduction control such as reproducing the received AV signal or the AV signal recorded on the mass recording medium and outputting the aforementioned signals to a monitor.
Note that the feature amount calculating section 12, the silent segment detecting section 13, the candidate segment detecting section 14, the CM segment determining section 15 and the digest list creating section 16, all of which are shown in
Next, various data used in the present embodiment will be described with reference to
Silent beginning end information 22 (
Candidate beginning end information 23 (
The silent segment information 24 (
The candidate segment information 25 (
Temporary CM beginning end information 26 (
CM segment information 27 (
Digest scene list 28 (
Hereinafter, a detailed operation of a digest scene list creating process executed by the digest generation device 10 will be described with reference to
In
Next, the silent segment detecting section 13 executes a silent segment detecting process, thereby detecting the silent segments (step S4).
On the other hand, as a result of step S11, when it is determined that the power level of the audio signal extracted in step S3 is not smaller than or equal to the predetermined threshold value (NO in step S11), the silent segment detecting section 13 reads the immediately preceding feature amount 212, thereby determining whether or not a power level stored therein is smaller than or equal to the predetermined threshold value (step S14). As a result, when it is determined that the power level is smaller than or equal to the predetermined threshold value (YES in step S14), a continued silent segment ends after the frame immediately preceding the current frame. Thus, the silent segment detecting section 13 outputs, to the silent segment information 24, a segment from the silent beginning end time of the silent beginning time information 22 to the time information 211 on the frame immediately preceding the current frame as one silent segment (step S15). Next, the silent segment detecting section 13 executes a point assessment process (step S16) on the silent segment outputted in step S15, as will be described hereinafter.
As a result of step S14, when it is determined that the power level of the immediately preceding feature amount 212 is not smaller than or equal to the predetermined threshold value (NO in step S14), a segment other than the silent segment is continuing, and thus the silent segment detecting section 13 finishes the process. Note that immediately after the process is started, any information is not yet stored in the immediately preceding feature amount 212. Therefore, also in this case, the process proceeds assuming that the power level is not smaller than or equal to the predetermined threshold value. As such, the silent segment detecting process is finished.
Next, the point assessment process in step S16 will be described in detail with reference to
In
Referring back to
On the other hand, as a result of step S31, when it is determined that the power level, of the audio signal, calculated in step S3 is not greater than or equal to the predetermined threshold value (NO in step S31), the candidate segment detecting section 14 reads the immediately preceding feature amount 212, thereby determining whether or not a power level stored therein is greater than or equal to the predetermined threshold value (step S34). As a result, when it is determined that the power level is greater than or equal to the predetermined threshold value (NO in step S34), a continued candidate segment ends after the frame immediately preceding the current frame. Therefore, the candidate segment detecting section 14 outputs, to the candidate segment information 25, a segment from the candidate beginning end time stored in candidate beginning end information 23 to the time information 211 indicating a time of the frame immediately preceding the current frame as one candidate segment (step S35).
On the other hand, as a result of step S34, when it is determined that the value of the immediately preceding feature amount 212 is not greater than or equal to the predetermined threshold value (NO in step S34), a segment other than the candidate segment is continuing. Thus, the candidate segment detecting section 14 advances the process to step S36. Note that immediately after the process is started, any information is not yet stored in the immediately preceding feature amount 212. Therefore, in this case, the process proceeds assuming that the value is not greater than or equal to the predetermined threshold value. In step S36, the candidate segment detecting section 14 stores the power level of the audio signal acquired in step S3 in the immediately preceding feature amount 212 (step S36). As such, the candidate segment detecting process is finished.
Referring back to
On the other hand, as a result of step S41, when it is determined that the silent segment exists at the time point 60 seconds prior to the current frame (YES in step S41), the CM segment determining section 15 determines whether or not any data exists in the temporary CM beginning end information 26 (step S42). As a result, when it is determined that no data exists in the temporary CM beginning end information 26 (NO in step S42), the CM segment determining section 15 outputs time information on the searched silent segment to the temporary CM beginning end information 26 (step S49). On the other hand, when it is determined that any data already exists (YES in step S42), the CM segment determining section 15 retrieves a temporary beginning end time from the temporary CM beginning end information 26, and outputs, to the CM segment information 27, the retrieved temporary beginning end time associated with the CM number 271 as the CM beginning end time 272. In accordance with this, a terminating end time of the silent segment searched in step S41 (i.e., the silent segment existing at the time point 60 seconds prior to the current frame) is outputted to the CM segment information 27 as the CM terminating end time 273 (step S43).
Next, the CM segment determining section 15 sets a D list creating flag on (step S44). The D list creating flag is a flag for creating the digest scene list to be described later. Then, the CM segment determining section 15 outputs information on a terminating end time of the silent segment existing at the time point 60 seconds prior to the current frame as the beginning end time of the temporary CM beginning end information 26 (step S45).
Then, the CM segment determining section 15 determines whether or not 120 seconds or more have been elapsed since the beginning end time of the temporary CM beginning end information 26 (step S46). In other words, during 120 seconds after a silent segment which may be a beginning end of a CM is detected, if any other silent segment having the point 242 greater than or equal to the predetermined value does not exist, the silent segment is not determined as the beginning end of the CM. Note that a reference time period required for the determination is 120 seconds since the present embodiment assumes that the maximum length of one CM segment is 60 seconds. In other words, even if a beginning end candidate of a CM segment is once detected and then another silent segment is detected 60 seconds thereafter, another 60 seconds are still required to determine that the said another silent segment is a terminating end of the CM segment.
As a result of step S46, when it is determined that 120 seconds or more have been elapsed (YES in step S46), the CM segment determining section 15 clears the temporary CM beginning end information 26 (step S47). Then, the CM segment determining section 15 sets the D list creating flag on (step S48). On the other hand, when it is determined that 120 seconds or more have not been elapsed (NO in step S46), the process is to be finished. As such, the CM segment determining process is finished.
A supplemental description of the CM segment determining process will be provided with reference to
Referring back to
Next, the digest list creating section 16 determines whether or not a process of the sorting has been already performed on each of the at least one candidate segment which has been newly added (step S56). As a result, when it is determined that any of the at least one candidate segment which has been newly added still remains unprocessed (NO in step S56), the digest list creating section 16 returns to step S53 and repeats the process. On the other hand, when it is determined that the process of the sorting has been already performed on each of the at least one candidate segment which has been newly added, the digest list creating section 16 sets the D list creating flag off (step S57), and finishes the digest scene list outputting process. As such, the digest scene list creating process according to the first embodiment is finished.
As described above, in the first embodiment, digest candidate segments each simply having an audio power level greater than or equal to a predetermined value are simultaneously extracted while recording a program, and a segment corresponding to the CM segment is deducted from the digest candidate segments, thereby making it possible to simultaneously create a digest scene list obtained by extracting only digest scenes included in a program segment while recording the program. Therefore, it is unnecessary to separately execute a process of creating the digest scene list after finishing recording the program. Thus, it becomes possible to provide the user with a comfortable viewing environment with no process waiting time required for executing the process of creating the digest scene list.
In the above embodiment, the silent segment detecting section 13 executes the silent segment detecting process. However, the present invention is not limited thereto. The CM segment determining section 15 may detect a silent segment prior to the CM segment determining process.
Furthermore, as a scheme of detecting the digest scene, the audio power level is not always necessarily used. For example, sports is selected as a specific program genre, and a scene showing a slow motion (a repeated slow motion scene) may be specified based on a motion vector of an image and several scenes immediately preceding the scene showing the slow motion may be detected as scenes of excitement. Or a combination of text information assigned to a program and a feature amount included in an audio/video signal may be used to detect an important scene. As a matter of course, the present invention is not limited to the above-mentioned digest scene detecting schemes. Only if a digest scene is detected, any scheme may be used. Similarly, as a scheme of detecting the CM segment, the audio power level is not always necessarily used. For example, scene change points included in an image may be detected based on brightness level information of the image, thereby determining a CM segment based on an interval between time points at which the scene change points appear. In this case, the brightness level information of the image may be used as the feature amount.
Still furthermore, while recording a program, a follow-up reproduction of the program may be performed by using the digest list. In this case, the user issues an instruction to perform the follow-up reproduction. In response to the instruction, the reproduction controlling section 18 determines whether or not two minutes or more have been elapsed since recording is started. When it is determined that two minutes or more have been elapsed, by means of a digest list currently being generated by executing the aforementioned processes, only the digest scene is reproduced. On the other hand, when it is determined that only less than two minutes have been elapsed, the reproduction controlling section 18 performs a speed-up reproduction (reproduction at a speed 1.5 times as fast as a normal speed, for example). Thereafter, when the speed-up reproduction catches up with actual broadcast, the speed-up reproduction may be stopped and switched to an output of the actual time broadcast. Also, after finishing reproducing the digest scene, the user may decide on a subsequent reproduction. For example, a normal reproduction of the digest scene may be performed, or the digest scene may be thinned out to be reproduced. For example, in the case of a program of 60 minutes, it is assumed that when 30 minutes have been elapsed since the program starts, the user issues an instruction to perform the follow-up reproduction indicating “10-minute reproduction of a digest scene is requested”. In this case, based on a digest scene list which is currently being created, the reproduction controlling section 18 reproduces the digest scene so as to be finished in 10 minutes. Thereafter, the user will decide on what to view after finishing the reproduction of the digest scene, and the reproduction controlling section 18 will stand by to receive an instruction from the user. In other words, after the reproduction of the digest scene finishes, 40 minutes have been elapsed since the program starts. Therefore, in response to the instruction from the user, a 10-minute portion of the program broadcast during the reproduction of the digest scene may be thinned out to be reproduced or the speed-up reproduction may be performed on the 10-minute portion. Of course, the user may view the actual broadcast without reproducing the 10-minute portion of the program. In this case, the reproduction controlling section 18 finishes a reproduction process in response to the instruction from the user. As described above, according to the present embodiment, the digest scene list is being simultaneously created while recording the program. Thus, it becomes possible to perform digest reproduction even at any timing during recording of the program. Still furthermore, in the above embodiment, the CM segment is deducted from the digest candidate segments so as to create the digest scene information. However, a segment which should be deducted from the digest candidate segments is not limited to the CM segment. A segment displaying a static image, for example, may be detected so as to be deducted. For example, when a program is rebroadcast, there may be a case where a scene which cannot be broadcast within the program is generated due to licensing or portrait rights. In such a case, the program is broadcast after an editing is performed prior to being broadcast such that a static image (to which an indication “a display is not permitted” is attached) is displayed instead of the scene which cannot be broadcast. Therefore, a feature amount (0 of a motion vector of the image, for example) included in the static image is detected, thereby detecting a static image segment in which the static image continues to be displayed. Thereafter, the static image segment (i.e., broadcast-prohibited segment) may be deducted from the digest candidate segments so as to create the digest scene information. Such CM segment and segment having a predetermined characteristic such as the static image or the like are detected as a specific segment, and the detected specific segment is deducted from the digest candidate segments, thereby making it possible to create the digest list obtained by extracting only the digest scenes in an appropriate manner.
Next, a second embodiment of the present invention will be described with reference to
Next, data used in the second embodiment will be described with reference to
Hereinafter, the digest scene list creating process according to the second embodiment of the present invention will be described with reference to
Subsequent to step S64, the CM segment determining section 15 executes the CM segment determining process, thereby creating the CM segment information (step S65). An operation in step S65 is the same as that in step S6 described in the first embodiment with reference to
After the process in step S65, the digest list creating section 32 executes the digest list outputting process (step S66).
Thereafter, the digest list creating section 32 determines whether or not a time indicated by the time information 361 retrieved in step S72 exists in a CM segment by reading the CM segment information (step S73). As a result, when it is determined that the time exists in the CM segment (YES in step S73), the digest list creating section 32 finishes the digest list creating process. On the other hand, when it is determined that the time does not exist in the CM segment (NO in step S73), the digest list creating section 32 determines whether or not a value of the feature amount 362 is greater than or equal to a predetermined value (step S74). As a result, when it is determined that the value is greater than or equal to the predetermined value (YES in step S74), the digest list creating section 32 determines whether or not the immediately preceding digest feature amount 372 is greater than or equal to the predetermined value (step S75). That is, a change in the audio power level between a frame retrieved in step S72 and a frame immediately preceding the frame is determined. As a result, when it is determined that the immediately preceding digest feature amount 372 is not greater than or equal to the predetermined value (NO in step S75), the time information on the frame is saved in the digest beginning end information 38 (step S76). Note that when the digest list outputting process is initially executed, any information is not yet stored in the immediately preceding feature amount 212. Therefore, in this case, the process proceeds assuming that the value is not greater than or equal to the predetermined value. On the other hand, as a result of step S75, when it is determined that the immediately preceding digest feature amount 372 is greater than or equal to the predetermined value (YES in step S75), the digest list creating section 16 skips a process in step S76 and advances the process to step S77.
On the other hand, as a result of step S74, when it is determined that the value of the feature amount 362 is not greater than or equal to the predetermined value (NO in step S74), the digest list creating section 32 further determines whether or not the immediately preceding digest feature amount 372 is greater than or equal to the predetermined value (step S78). As a result, when it is determined that the immediately preceding digest feature amount 372 is not greater than or equal to the predetermined value (NO is step S78), the digest list creating section 16 finishes the digest list creating process. On the other hand, when it is determined that the immediately preceding digest feature amount 372 is greater than or equal to the predetermined value (YES in step S78), a continued silent segment ends after the frame immediately preceding the frame. Thus, the digest list creating section 32 outputs, to the digest scene list 28, a segment from the digest beginning end time indicated by the digest beginning end information 38 to a time indicated by the immediately preceding digest time information 371 as one digest segment (step S79). Next, the digest list creating section 16 saves the audio power level of the frame in the immediately preceding digest feature amount 372 (step S77). As such, the digest scene list creating process according to the second embodiment is finished.
As described above, in the second embodiment, a CM segment is simultaneously detected while recording a program, thereby making it possible to detect a digest scene from a program segment other than the CM segment. Therefore, it is unnecessary to separately execute a process of creating the digest scene list after finishing recording the program. Thus, it becomes possible to provide the user with a comfortable viewing environment with no process waiting time required for executing the process of creating the digest scene list.
Note that each of the above embodiments may be provided by a recording medium storing a program executed by a computer. In this case, the digest generation device (more precisely, a not shown control section thereof) may read a digest generation program stored on the recording medium, and execute the processes as shown in
A digest generation device, a digest generation method, a recording medium storing a digest generation program thereon, and an integrated circuit used in the digest generation device according to the present invention are capable of generating digest scene information while recording a program, and are applicable as an HDD recorder, a DVD recorder and the like.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2005-217724 | Jul 2005 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2006/314589 | 7/24/2006 | WO | 00 | 1/4/2008 |
