Patent Grant
11930227
The invention relates to processing recorded video and audio, and more specifically, to systems and methods for determining where in a movie to insert an ad.
Linear media editing systems used for analog audio, video tape and photographic film are manual, time consuming and cumbersome to reduce content into a final form and distribute. In more recent times computer systems allow for time efficient non-linear video editing systems. Current non-linear editing on computer oriented systems involves capturing media content permitting rapid access to the media content at any point in the linear sequence for moving portions into any order and storing the media content on a storage device, such as a magnetic disk drive or digital versatile disk (DVD).
The average person currently has small set of alternatives for editing content from media capture devices such as camcorders, camera phones, audio recorders, and other media capture devices without having to incur the costs of a computer system and software for editing. In addition, non-linear editing systems are complex and very difficult to use.
People capture various random and personally interesting events, such as work, travel and entertainment event using their camcorders or camera phones. To edit this content, people require easy to use non-linear editing systems that facilitate editing without a high degree of computer or editing skill.
Media content storage technologies provide for storing great amounts of interactive multimedia, for example, the DVD format. Unfortunately, the DVD Specification for authoring is very complex, as are the computer systems that attempt to embody it. A further disadvantage of conventional DVD authoring systems is that they provide a DVD author with only minimal control and flexibility.
The process of authoring a DVD includes a number of complex steps and equipment. Accordingly, there is a need for authoring systems and methods that reduces the time, cost and complexity of the authoring and distributing DVD.
Separate and distinct systems for computer based non-linear editing, DVD authoring, and distributions are known. However, no system exists that enables an ad to be flexibly placed in different areas of a movie according to an automatic procedure.
In accordance with the invention, there is provided a method for ensuring that an ad in a movie is viewed by a viewer. When the viewer is at a selected point in playback of the movie, a skip command is received from the viewer to omit playback of a portion of the movie that includes the ad. The location of the start of the ad in the movie is determined. Playback of the portion of the movie between the selected point and the start of the ad is omitted, and playback of the movie is resumed beginning at the start of the ad.
In accordance with another aspect of the invention, there is provided a method for ensuring that an ad in a movie is viewed by a viewer. When the viewer is at a selected point in playback of the movie, a skip command is received from the viewer to omit playback of a portion of the movie that includes the ad. The ad is moved from the omitted portion of the movie to a portion subsequent to the omitted portion, and playback is resumed after the omitted portion and prior to the position of the moved ad. In accordance with a further aspect of the invention, there is provided a method for ensuring that at least two ads in a movie are viewed by a viewer. When the viewer is at a selected point in playback of the movie, a skip command is received from the viewer to omit playback of a portion of the movie that includes the ads. It is determined that one of the ads is a static ad that cannot be moved, and that one of the ads is a proxy ad that can be moved. The location of the start of the static ad in the movie is determined, and playback of the portion of the movie between the selected point and the start of the static ad is omitted. The proxy ad is moved from the omitted portion of the movie to a portion subsequent to the omitted portion, and playback of the movie is resumed beginning at the start of the static ad and prior to the position of the moved proxy ad.
It is not intended that the invention be summarized here in its entirety. Rather, further features, aspects and advantages of the invention are set forth in or are apparent from the following description and drawings.
As used herein and in the claims, the term “movie” refers to video and/or audio data intended for display to a human. In some cases, the video and audio data are separate but associated data, while in other cases, the video and audio are combined. In still other cases, video exists without audio, and vice-versa. Video encompasses still image data and moving image data.
As used herein and in the claims, the term “editing” includes but is not limited to removing portions of a movie, combining at least two movies to create a new movie, and/or generation of special effects that are composited into a movie. The term “compositing” refers to combining by inserting or overlaying additional content into movie content, for example, adding foreign language subtitles.
As used herein, the term “editor” sometimes means a human who is editing a movie, and sometimes means a computer software program used by the human who is editing a movie to perform the editing.
The disclosures of the following patents are hereby incorporated by reference in their entireties:
There are several environments in which a user might want to use a movie. Turning to
Movie system 10 includes videobase 20, database 30, transcoder 40, server 50, DVD burner 60 and internal network 70. In some embodiments, videobase 20 and database are combined. Elements 20, 30, 40, 50 are general purpose computers programmed according to the invention, and include suitable processors, memory, storage and communication interfaces; and each element may be embodied in one or many physical units depending on factors such as expected processing volume, redundant hardware design and so on.
Videobase 20 serves to store movies uploaded by users, in their uploaded format, and to store transcoded versions of these movies. Videobase 20 also stores advertisement movies, referred to herein as “ads,” intended for inclusion in the transcoded movies. Database 30 serves to store data for movie system 10, including data relating to users of movie system 10, movies processed by movie system 10, and suitable administrative data such as usage, throughput and audit trail information. In some embodiments, users use movie system 10 for free and the suppliers of ads pay upon ad viewing. In other embodiments, users pay based on usage or a flat rate.
Transcoder 40 serves to receive uploaded movies and process them to generate transcoded movies, as described in detail below with regard to
Server 50 receives requests from users via network 100 and responds thereto. In cases where responding to a request requires the services of transcoder 40, server 50 passes appropriate messages between transcoder 40 and network 100. Server 50 also functions as a firewall to protect network 70 from improper usage.
Server 50 executes upload manager 55, a software program that works with uploader 112, described below, to upload a movie to server 50.
DVD burner 60 is responsive to commands from transcoder 40 to create a digital video disk, which is then shipped via conventional physical shipping services. In other embodiments, instead of a digital video disk, a read-only or read-write storage medium is created and then the medium is physically delivered to a recipient. Suitable formats include CD (compact digital), DVD (digital video disk), Blu-ray Disc, HD (high definition) DVD, flash drives (transistor memory), jump drives (removable disk drives) and the like.
Billing program 65 examines usage data created by transcoder 40 and server 50. The usage data is part of the administrative data in database 30. Billing program 65 then generates invoices and applies authorized payments. For example, some users may have preauthorized charges to their credit cards, telephone bills, or bank accounts for their usage of movie system 10.
As another example, transcoded movies created by users may include advertising, for which the advertisers agree to pay based on number of views, and if so, billing system 65 arranges payments to users based on usage of the transcoded movies with advertising.
There is a cost to store and distribute movies. To offset this cost, and to reward users, movie system 10 enables movie creators to include ads, either manually or automatically, in their movies. Movie system 10 enables flexible ad placement, including at the start or end of a movie, within selected frames of a movie, and at a selected location and size within the selected frames. Advertisers generally pay for placement of their ads based on number of times their ad is viewed, and possibly in accordance with the popularity of the place in the movie where the ad is inserted, how much time the ad is inserted for, and the size of the ad relative to a movie frame.
Internal network 70 serves to carry communication traffic between the elements of movie system 10. Internal network 70 may be a local area network at a single premise, or may span multiple premises.
Server 80 is a general purpose computer coupled to storage 90. Server 80 responds to requests from communication network 100 by providing movies stored in storage 90. By providing the address of server 80 to movie system 10, one of the movies stored in storage 90 can be used as an input for transcoder 40.
Distributed server 82 is a general purpose computer coupled to storage 92. Storage 92 holds copies of some or all of the transcoded movies stored in storage 20 of movie system 10. In some embodiments, there are many instances of server 82 and storage 92. Distributed server 82 responds to requests from viewers for the transcoded movies stored therein. Use of distributed server 82 reduces response time to requests, reduces the amount of work that movie system 10 needs to do, and reduces bandwidth utilization since usually the instance of server 82 that is closest to the requester responds thereto. Products and services according to distributed server 82 are available from Kontiki, a VeriSign company, and from Akamai.
Proxy ad server 84 is a general purpose computer coupled to storage 94. Storage 94 stores ads. Operation of proxy ad server 84 is discussed below with regard to
Media center 105 generally represents a television that is able to communicate via network 100. In some embodiments, a viewer provides instructions to media center 105 via a remote keypad, while in other embodiments, a viewer provides instructions to media center 105 using a full keyboard with function keys and so on. In still other embodiments, media center 105 includes a voice recognition unit so that the viewer provides instructions through speech. In some embodiments, such as a home, media center 105 is able to communicate directly with PC 110 and/or phone 130.
PC 110 is a general purpose personal computer coupled to camcorder 120. Camcorder 120 enables a user to record a movie and transfer the movie to PC 110.
PC 110 executes uploader 112 and player 114. Uploader 112 is a software program that enables a movie to be uploaded from PC 110 to server 50. Player 114 is a software program that enables PC 110 to view and edit movies, in conjunction with transcoder 40. When PC 110 registers with server 50, server 50 downloads uploader 112 and player 114 to PC 110.
Uploader 112 functions to locate movie files stored in PC 110, and to manage transmission of the movie files to upload manager 55 of server 50 using a suitable protocol such as the secure file transfer protocol (S FTP). In embodiments having a peer-to-peer (P2P) network for downloading, such as networks using the BitTorrent protocol, the peer-to-peer network is also used for uploading. Since movie files are large, the file uploading may be interrupted; uploader 112 enables the uploading to resume at its interruption point. In some embodiments, uploader 112 converts a very large file, such as a 36 Mb file in DV format, to a smaller file of comparable visual quality, such as a 3 Mb file in MPEG format. Uploader 112 enables the user of PC 110 to select a file for uploading; to monitor the status of the upload, such as percent completed and speed of uploading; to pause and resume the uploading; and to cancel the uploading.
Player 114 is a client application for a PC browser. In some embodiments, player 114 resides on a portable device such as a mobile phone or network-attached digital video camera without a browser, and in these embodiments, player 114 is a network enabled client application.
Player 114 enables a user to view a movie, including forward seek and rewind functions; to seek the compressed video on a remote server using random seek points; to request a movie download from the random seek point, for example, in accordance with U.S. Pat. No. 6,157,771, the disclosure of which is hereby incorporated by reference in its entirety; and to use the functions described below with regard to
Phone 130 is a wireless communication device executing versions of uploader 112 and player 114 adapted for the device capabilities of phone 130. Phone 130 is coupled to camera 135, which serves to capture images and provide the captured images to phone 130 as a movie signal. In some embodiments, phone 130 uses the multimedia messaging service (MMS) protocol to transmit and/or receive movies. In other embodiments (not shown), phone 130 communicates with a local network using a protocol such as Wi-Fi, and the Wi-Fi network in turn communicates with communication network 100. PC 140 is a general purpose personal computer that is able to view transcoded movies by obtaining an address for the movie, or a segment thereof, and providing the address to server 50. As an example, a user of PC 110 or a user of phone 130 may upload a movie to movie system 10, edit the uploaded movie, and provide to the user of PC 140, via email, an address of an edited segment that the user of PC 140 is permitted to view.
Camcorder 145 is a network enabled movie capture device configured to upload its recordings to movie system 10. In some embodiments, there is at least one predefined user group able to immediately edit information uploaded from camcorder 145. This configuration is useful in a security application, for example.
The user of PC 110 or phone 130 serves as an editor. PC 110, phone 130, PC 140, and camcorder 145 are each at respective locations that are remote from the location of movie system 10.
The capabilities of movie system 10 are available via each of the user devices shown in
Other embodiments of the invention wherein the transcoder is local to the movie file are described below. Embodiments in which the movie file is not uploaded via a public communication network are sometimes referred to as a transcoder appliance model.
At step 200, transcoder 144B divides the job into tasks. At step 202, transcoder 144B sends one task to transcoder 144A. At step 206, transcoder 144B sends another task to transcoder 144C. At step 210, transcoder 144E receives the completed task from transcoder 144A. At step 211, transcoder 144B receives the completed task from transcoder 144C. At step 212, transcoder 144B assembles the completed tasks into a completed job.
At step 203, transcoder 144A receives a task. At step 204, transcoder 144A executes its task. At step 205, transcoder 144A sends the executed task back to transcoder 144B.
At step 207, transcoder 144C receives a task. At step 208, transcoder 144C executes its task. At step 209, transcoder 144C sends the executed task back to transcoder 144B. Turning to the transcoder appliance model,
At step 220, PC 140 requests the movie from movie system 10. At step 221, movie system 10 receives the request, and at step 222 responds thereto by sending the movie including the proxy ad to PC 140. The proxy ad includes an ad request that PC 140 is expected to send. The ad request typically comprises the network address of proxy ad server 84, and optional or required parameters to be used by proxy ad server 84 such as movie characteristics and/or viewer characteristics, e.g., keywords relating to the movie, display characteristics of PC 140, vertical and horizontal picture resolution of an ad image that can be accommodated in the movie, and so on.
At step 223, PC 140 receives the movie and ad request. At step 224, PC 140 sends the ad request and parameters, if any, to proxy ad server 84. At step 225, proxy ad server 84 receives the ad request and parameters, if any. At step 226, proxy ad server 84 selects an ad from storage 94 in accordance with the parameters, and prepares the ad, such as by sizing, in accordance with the parameters. At step 227, proxy ad server 84 sends the prepared ad to PC 140.
At step 228, PC 140 receives the prepared ad, and plays the movie while inserting the received ad at the location indicated by the proxy ad in the movie.
Turning to
At step 320, transcoder 40 stores the transcoded movie in videobase 20. As described below, user1 can grant permission to other users to view the whole of the transcoded movie, or to view segments of the transcoded movie, by providing suitable addresses to the authorized users.
Turning to
At step 340, server 50 requests the selected movie from videobase 20, indicating the format for the movie.
At step 350, if the requested format happens to match the stored format, then the movie is provided directly to server 50. Otherwise, videobase 20 is operative to convert the format from the stored format to the requested format. The movie is provided in the requested format, also referred to as a third format, a downloaded format, or an output format. Server 50 then sends the provided movie to user2 via suitable distribution method such as streamed video or podcast, or presentation on a web page, blog, wiki, really simple syndication (RSS) or other technique. In some embodiments, videobase 20 sends the stored movie to transcoder 40, for conversion to the requested format, and then transcoder 40 provides the movie in the requested format to server 50.
At step 2005, transcoder 40 analyzes the movie characteristics. As used herein and in the claims, “inherent movie characteristics” means information about the movie itself, without reference to usage or viewing. Examples of inherent movie characteristics include motion, scene changes, face presence, audio track loudness, keywords relating to the movie's subject, and so on. In this document, inherent movie characteristics are sometimes referred to as movie characteristics. At step 2010, transcoder 40 determines whether the movie has been previously viewed. If not, processing proceeds to step 2020. If the movie has been previously viewed, then at step 2015, transcoder 40 analyzes the viewed characteristics of the movie, also referred to herein as the popularity of the movie. Viewed characteristics include number of times that the movie has been requested, number of times that a link or deep tag to the movie has been sent from one viewer to another potential viewer, number of deep tags in the movie, number of times that a particular movie segment was replayed, number of times that viewers paused at a particular movie position, and so on. In some embodiments, the viewed characteristics are associated with the demographics of the viewer, such as gender, age, location, income, interests and so on.
At step 2020, transcoder 40 determines if the ad placement is occurring in real time. As explained below, movie system 10 can operate in one or more of the following modes:
Non-real time ad placement mode enables movie system 10 to operate in essentially a broadcast fashion, for ads. Real-time ad placement mode enables movie system 10 to operate in a narrowcast fashion, wherein the characteristics of the viewer determine what ads are sent to the viewer. If movie system 10 is operating in non-real time ad placement mode, then processing proceeds to step 2030. If movie system 10 is operating in real-time ad placement mode, or hybrid ad placement mode, then at step 2025, transcoder 40 analyzes the viewer's characteristics. In this context, characteristics of the viewer include demographic information as well as activity of the viewer, such as entering a keyword, the material previously viewed by the viewer and so on.
At step 2030, transcoder 40 retrieves the advertiser's preferences for ad placement. Advertiser preferences can specify movie characteristics, movie popularity characteristics, viewer characteristics, and how much the advertiser is willing to pay depending on the combination of features delivered, which may include whether the ad is viewed or whether the viewer takes an action relating to the ad. For example, some ads include hyperlinks that a viewer can click on, and the advertiser may be willing to pay a first rate if the ad is merely presented to the viewer, and pay a second rate if the viewer clicks on the hyperlink.
At step 2035, transcoder 40 determines prices for positioning ads at various places in the movie. The price determination procedure is a function of the movie characteristics, the viewed characteristics, the viewer characteristics, advertiser demand, and the price setting mechanism.
At step 2040, transcoder 40 places the at least one ad in the movie. It will be appreciated that an advertiser's rules may specify manual placement, in which case transcoder 40 notifies a person associated with the advertiser of the prices for placing the ad at various positions, and a human manually selects ad placement.
In some cases, an ad is composed into the scene, usually manually. For example, the movie scene might be a living room with a television, and the ad is composed to appear to be on the television in the movie scene. As another example, the movie scene may be a wall with a picture frame, and the ad is composed to appear in the picture frame of the movie scene.
At step 2045, the movie is delivered to the viewer that requested the movie. When movie system 10 operates in non-real time ad placement mode, there is a step (not shown) of storing the movie plus ads in videobase 30 prior to delivering the movie to the requesting viewer.
At step 2050, statistics are collected about movie viewing, such as by server 50 receiving play trails (discussed below), notices of deep tag transmissions and so on, and storing them in database 30.
At step 2055, appropriate viewing reports are generated by billing program 65, server 50 and transcoder 40.
A prior art search engine enables advertisers to bid on keywords. During a set-up phase, the advertiser selects the desired keyword(s), and their maximum bid price. In operation, when a searcher enters the desired keyword, advertiser ads are presented in an order corresponding to their bid amounts, that is, the ad with the highest bid is listed first, followed by ads with sequentially lower bid amounts. When the searcher clicks on an ad, the search engine bills the advertiser, that is, if the ad is merely presented with no response from the searcher, the advertiser does not pay.
Advertisers can then view reports on the “click-through” activities for their advertisements.
In an embodiment where the price of placing an ad in a movie varies, advertisers can determine the price at least in part by bidding. For example, movie system 10 can determine a starting price based for ad placement in portions of a movie based on popularity, level of motion, length of ad, whether the ad is full-motion video or a static image, and so on. Advertisers can then (i) submit specific bids for specific movies, ex: $100 for placement in movie xyz for two weeks, (ii) can define bidding rules, ex: placement in any travel movies viewed at least 100 times per day, ex: placement in any movie segment having a deep tag with the text label “airplane” and where the movie is one of the most popular 1,000 movies for the day, or (iii) can define results and how much they are willing to pay, ex: at least 100 views per hour by girls of age 10-14 located in California at a cost of up to $20 per hour.
The value placed upon media inventory across a timeline of a movie can vary dramatically and dynamically based on continuously updated statistics on both behavioral and contextual analysis of the video stream; for example, the most prized video segments may be those that are most popular and contain the deep tags for “cars”, “hotels”, “swimming”, etc.
As an overview, transcoder 40 receives an uploaded movie, creates a representation for easy editing, and adds user-supplied editing data (steps 400-440) to create a transcoded movie. Then, at the user's option, some, all or none of the following functions can be performed, in any desired sequence, and in as many editing sessions as desired:
Including an advertising movie in a transcoded movie ensures that even if the viewer of the transcoded movie has a filter for blocking advertising pop-ups and the like, the included advertising movie is viewed, since the filter considers the advertising movie to be part of requested content.
At step 400 of
At step 410, transcoder 40 builds a texture strip representing the movie. Specifically, transcoder 40 applies a function to each frame to generate texture data (see
At step 420, transcoder 40 creates a source proxy for the uploaded movie. Generally, a source proxy is a representation of the frames of the movie in a particular format that is easy to convert to other formats and to distribute via public communication network 100. For example, the Flash video format, according to the H.263 standard, can be used for the source proxy.
Using a source proxy reduces the format conversion issue. Specifically, if there are n movie formats, a general transcoder should be able to convert from any input to any output format, which, by brute force, would require n2 different format converters. However, using a source proxy means that only 2n format converters are needed (n converters to the source proxy format, and another n converters from the source proxy format to the output format). Additionally, as new movie formats become available, supporting them requires creating only 2 converters per format (one to the source proxy format, and one from the source proxy format), rather than 2n with the brute force approach. It is recognized that, sometimes, the source proxy format may be the desired output format.
Editing of the proxy format, also referred to as proxy editing, may occur in several ways.
In one embodiment of proxy editing, the edits are applied directly to the proxy frames.
In another embodiment of proxy editing, the proxy frames are maintained as generated, and an edit list is created, comprising edits to be sequentially applied to the proxy frames. Each time the edited movie is provided, the edits are applied anew to the proxy frames. This embodiment is particularly useful when edits need to be undone, or when many users are editing one movie to create separate edited movies.
In a further embodiment of proxy editing, a hybrid approach is used, wherein during an edit session, an edit list is created, and only at the termination of the edit session are the edits applied directly to the proxy frames.
At step 430, transcoder 40 generates a thumbnail, also referred to herein as a snapshot, as a visual representation of the entire movie. Typically, the user selects a frame, and transcoder 40 reduces it to a thumbnail size, such as 177×144 pixels. A user having many stored movies can conveniently view their thumbnails, rather than or in addition to text descriptions and/or filename descriptions.
At step 440, transcoder 40 accepts metadata from the user. Movie metadata may include a filename for the transcoded movie, subject matter keywords associated with the movie, a short text description to be associated with the thumbnail, any deep tags the user cares to define, address information such as a hyperlink of information to be associated with the transcoded movie, and an associated movie such as an audio file describing the contents of the movie.
A deep tag is a video bookmark, indicating a sequential group of frames that are to be treated as a separately addressable segment; the deep tag metadata includes the movie filename, the user filename, date of creation of the deep tag, date of most recent modification of the deep tag, a deep tag filename, the start frame, the end frame, the duration of the segment, and a short text description of the segment. A deep tag is understood to be a convenient way of identifying a segment. A deep tag can be thought of as a movie segment bookmark.
Video display 710 shows the current frame of video. When the editor's device, such as PC 110 or phone 130 permits, the video frame is displayed in its proxy format. However, if the editor's device cannot support the proxy format, the transcoder 40 converts edited frames to an output format suitable for the editor's device prior to sending the edited frames to the editor for display.
Thumbnail 715 is a small image representing the entire movie.
Texture strip 720 comprises sequential frame representations 725 and subsequent information; each frame representation 725 is the result of the function used to create the texture strip, such as a vertical column of 64 pixels, and represents a single frame. Subsequent information indicates special effects applied to the frames and any advertising inserted in the frames. Texture strip 720 is not limited to video transform visualizations, and can be created by any of the following techniques, or other suitable techniques:
Positioner 730 indicates where the frame display in video display 710 is located relative to the entirety of the movie. Positioner 730 enables the editor to use texture strip 720 to seek frames in the movie in a random access manner.
Deep tag marker 735 has a left edge that can be adjusted by a user, and also has a right edge that can be adjusted by the user; after the user has adjusted the left and right edges of deep tag marker 735, the user indicates that these settings should be saved as a deep tag, such as by clicking deep tag button 740, and providing a text description corresponding to the movie segment indicated by the deep tag. Deep tag marker 735 enables the editor to use texture strip 720 to select a segment of the movie.
Deep tag bar 750 is a visual representation of deep tags that have already been created for the movie. In the example of
Function buttons 760, 771, 772, 773, 775, 780 and 790 enable the user to edit the movie. Data button 760 enables the user to view and edit metadata associated with the movie. Playback product buttons 771, 772 and 773 take the user to specialized editing interfaces, discussed below. Effects/scripts button 775 enables the user to add and edit special effects and scripts. Ad button 780 enables the user to include advertising in the movie. Sharing button 790 enables the user to grant permission to other users or user groups to view selected segments of the movie.
Returning to
Examples of special effects include: watermark, mosaic, barndoor, noise, dissolve, spiral, fade in, fade out, increase contrast, decrease contrast, soften perimeter, cut frame, overlay, translucency, wipes, cuts, zooms, and so on.
A movie watermark for a movie including video is one of (a) a permanent graphic such as a logo inserted in each frame of the movie, (b) an overlay graphic such as a logo inserted in each frame of the movie, or (c) hidden information such as a logo, also referred to as steganography. A movie watermark for an audio only movie is (i) a sound file inserted at a predetermined point in the movie, such as its start or end, or (ii) hidden information such as text, also referred to as steganography.
Examples of scripts include: when the movie is fully loaded, then play a sound file; when the cursor clicks on an area, then create a pop-up window with a particular web page; when the cursor clicks on an object in a video scene, then launch a form—such as for product purchase or survey; and so on.
An effect can also include an executable command, as described above with respect to the tree playback product editor.
Texture strip 1030 indicates the frame representations of the movie selected for effects, such as frame representation 1031. Positioner 1032 enables rapid selection of a frame of the movie. Slider 1035 has a left and right edge that can be adjusted by the editor, to indicate frames to receive an effect. After the editor selects an effect in effects area 1010, and adjust slider 1035, the editor clicks apply button 1033 to apply the effect. After the editor has finished applying effects, he or she clicks preview button 896 to preview. If an editor wishes to cancel an effect, she positions slider 1035 on the appropriate frame representations and clicks cancel button 1034. Publish button 897, burn button 898, and back button 899 function as described above.
Returning to
If the user wants to include an ad movie in the transcoded movie, at step 510, transcoder 40 determines whether the user wishes to select the ad movie, or to accept an ad movie selected by movie system 10. If the user wishes to select the ad movie, at step 520, transcoder 40 provides the user with a menu of ad movies that are consistent with the characteristics of the user's transcoded movie, and the user selects one or more ad movies for inclusion in their transcoded movie. If the user does not wish to select the ad movies, at step 530, transcoder 40 selects one or more ad movies based on an ad movie selection procedure and its own determination of how many ad movies are appropriate for the transcoded movie. In some embodiments, the ad movie selection procedure is based on maximizing revenue for the user, and following a least-recently-used ad movie selection procedure. In some embodiments, the determination of how many ad movies are appropriate is based on at least one of: the length of the transcoded movie, keywords in the metadata, how many segments are deep tagged, and the length of the deep tagged segments, and so on.
An ad used with movie system 10 can be static or dynamic. A static ad is inserted in the transcoded movie prior to its storage in videobase 20. For a dynamic ad, the transcoded movie is stored in videobase 20 with a placeholder, and when the transcoded movie is presented to a user, the actual ad is inserted, the ad being chosen based on a characteristic of the user such as the user's location, referred to as a “geo-aware” ad, the characteristics of the display device, referred to as a “device aware” ad, or other suitable characteristic.
At step 540, transcoder 40 determines whether the user wishes to control the position of the ad movie(s) within a frame of the transcoded movie, or to accept positioning determined by movie system 10. If the user wishes to control the positioning of the ad movie, at step 550, transcoder 40 provides the user with a graphical interface for controlling ad movie positioning, and a menu of how the ad-movie should be inserted, such as a picture-in-picture at the top, left, right or bottom of the transcoded movie, the top being referred to as a “banner” ad, or as a standalone segment in the transcoded movie. In some embodiments, transcoder 40 also provides the user with popularity statistics for portions of the movie, discussed in detail below, so that the user can position the ad in accordance with the parts of the movie that viewers like. In some embodiments, the user selects the size of the space in each frame that the ad may occupy. If the user does not wish to control the positioning of the ad movie, at step 560, transcoder 40 decides where the movie ad should be placed, typically by looking at the default position in the metadata associated with the ad movie.
At step 570, transcoder 40 determines whether the user wishes to control the frames of the transcoded movie where the ad will be placed, or to accept positioning determined by movie system 10. If the user wishes to control the positioning of the ad movie, at step 580, transcoder 40 provides the user with a graphical interface for controlling ad movie positioning, such as a texture strip and slider, and in some cases, popularity statistics (discussed below) for portions of the movie. If the user does not wish to control the positioning of the ad movie, at step 590, transcoder 40 estimates where the movie ad should be placed. Any suitable estimation procedure may be employed. As an example, if the transcoded movie is a completely new movie with no deep tags, the estimation procedure specifies that the first scene having a length of at least the length of the ad movie is the selected position, and that the ad movie should be inserted as a picture-in-picture in the lower right of the transcoded movie. However, if the transcoded movie is associated with at least one other user, then a popularity procedure is used to select the position for the ad movie.
Collection and processing of user statistics will now be discussed. These statistics are employed in the popularity procedure used to automatically position the ad movie.
At step 1300, a viewer, such as a user of PC 140, who is understood to also or alternatively be a listener, requests a movie, such as by clicking on a link to the movie, the link being embedded in an e-mail previously sent to the viewer. At step 1325, server 50 receives the movie request, and retrieves the requested movie from videobase 20. At step 1330, server 50 sends the movie to the requesting viewer, and at step 1335, server 50 updates its download statistics for the movie.
At step 1305, the viewer receives the movie, and at step 1310 plays the movie. Playing the movie includes viewing the movie in its intended temporal sequence, pausing, replaying a section of the movie, fast-forwarding, adding one or more deep tags and so on. Meanwhile, at step 1315, a play trail is being recorded. As used herein and in the claims, a “play trail” means a time stamped sequence of actions that a viewer takes when playing a movie. The timestamp is typically measured in tenths of a second since the start of the movie, but may be measured in clock time or by frame number of the movie, or any other convenient metric. Each action in the sequence carries appropriate information to reconstruct what the viewer did, for example, “pause” indicates how long the movie was paused. When the viewer finishes viewing the movie, at step 1320, the play trail is sent to server 50.
In
At step 1340 of
As described above, a play trail is created at the user's location, and then sent to server 50. In other embodiments, the play trail is stored in a so-called cookie, a locally stored data file that tracks an authenticated user's actions on a PC, on the user's PC so that the play trail data can persist over time in the cookie data even when a user's browser session has completed or has closed prematurely. The cookie can be extensive in terms of data structures and may include demographic data as well.
Alternatives to cookies include the use of HTTP authentication, client side persistence, local stored Flash objects, JavaScript window, name variables, and so on; each of these methods provides mechanisms to store and track user play trail data. The advantage of a cookie relative to a local file is that a cookie uses a standardized Internet mechanism to track user data in a well-defined location on a user's hard drive. The data is tracked across user browser sessions and is persistent across those sessions. Cookies protect user privacy by tracking user behavior without tracking identity.
Play trails are a form of user behavior tracking. Other user behavior that is tracked to determine popularity of a movie or movie segment includes creating a deep tag and/or sending a deep tag, such as via email, instant message (1M), message service such as SMS or MMS, or podcasting. Deep tags indicate what users consider interesting.
An email or IM version of a deep tag typically comprises a URL providing the address of the deep tag, such as www.server.coraldeeptag/movieID/StartTime/EndTime, or www.server.cornImovieIDIdeeptag/deeptagID. Additionally, the email may include text associated with the deep tag, a thumbnail image for the deep tag, and an attachment comprising the portion of the movie indicated by the deep tag. SMS, MMS and podcast versions of a transmitted deep tag are similar.
Replay frequency measures how many times a listener/viewer replays a particular portion, as opposed to its initial playing.
Pause frequency measures how many seconds that a listener/viewer pauses at a particular point in a movie.
Community score is based on ratings provided by listeners/viewers, explained below in connection with
While
Histogram 960, also referred to as popularity density function 960, indicates the popularity metric, herein the number of times that a portion of the movie has appeared in a deep tag, which serves as an indication of the popularity of the various parts of the movie, plotted against the sequence in which the frames of the movie are displayed.
Color graph 970 uses different colors to represent different levels of popularity. In one scheme, the most popular portions of a movie are indicated by red areas, while the least popular portions of a movie are indicated by blue areas. In another scheme, different shades of gray are used to indicate popularity.
Color graph 970 can be generated as follows. Let the metric being plotted be the Total Interest in a specific part of the movie. The Total Interest metric is an indicator of the level of popularity of a section of a movie. The Total Interest is a weighted combination of metrics, such as the amount of time paused, the number of playbacks, or the number of deep tags created in a specific section of a movie. First, the metric is normalized to be a value between 0-100. Next, the normalized metric is mapped to the 0-255 code for each of the display colors of blue, green, and red.
As a specific example, let the metric plotted be the number of playback events per unit time for a movie, and assume the maximum number of playback events for any segment of a video is 25, so that 25 playback events are normalized to a value of 100. The color code is calculated using the following formula:
Red=(MAX(((Normalized_Metric−50)*255 50)),O))
Blue=(MAX(((50−Normalized_Metric)*255 50),0))
For: Normalized_Metric<=50:Green=(Normalized_Metric*255150)else: Green=((100−Normalized Metric)*255150)
Sample usage statistics for a particular movie, and their conversion to RGB values are shown in the following table.
The Red Green Blue color values determine the color for a vertical column of pixels in color graph 970 corresponding to the time interval of the movie segment. As is well-known, each pixel of a display is actually a group of three light emitting elements: red, green and blue, and the intensity values for each of the elements determine the color perceived by the viewer. In some embodiments, the columns of color graph 970 are placed next to each other, like bar graphs having the same height, whereas in other embodiments, the colors are interpolated from column to column, to provide a more aesthetically pleasing appearance.
Cloud graph 980 has clouds or circles whose size corresponds to the popularity of a movie portion.
Other types of graphical representations of popularity may be employed, such as three dimensional strips with colored segments, three dimensional bar graphs, surface contour maps in three dimensions, and so on. As used herein and in the claims, popularity statistics are based on one or more popularity metrics such as total number of requests for a movie, total number of times that the movie has been selected as a favorite (see
A further aspect of the movie may be provided by demographic or other selected data about viewers, such as age bracket, gender, location, income level and so on, instead of or in addition to the popularity metrics discussed above. Of course, this can be accomplished only when the viewer provides such demographic data, such as during establishing a new user account.
Popularity graphs can be reported to the owner of the movie.
As shown in
Popularity graphs can also be used internally by transcoder 40. Movie ads are placed in the popular parts, if they are long enough, or with respect to the popular parts, if the popular parts are shorter than the ad movie. For example, the movie ad could be placed so that its start is co-located with the start of a popular segment having a duration of at least 70% of the duration of the ad movie.
Returning to
At step 1380, an owner of a movie requests custom statistics about the movie, such as a number of deep tags created by a particular demographic group during a particular time period. At step 1355, server 50 receives the custom statistics request. At step 1360, server 50 retrieves appropriate play trails, and at step 1365, server 50 retrieves appropriate user data. At step 1370, server 50 generates the requested custom statistics report. At step 1375, server 50 sends the custom statistics report to the requester. At step 1385, the requester receives the custom statistics report.
Returning to
Ad bounceback is a technique for prohibiting skipping of a static ad. The static ad can have a pre-ad zone defined before its start and/or a post-ad zone defined after its end. The pre-ad zone forces a bounceback to a few frames before the start of the ad; this is useful when the context of the movie is helpful to appreciating the ad. The post-ad zone is relevant to certain types of ads, and prohibits a user's skip from beginning until after the end of the post-ad zone.
One strategy is to bounceback to the earliest ad that would otherwise be skipped. Segment 366 is a bounceback to the start of ad 93E, and segment 367 indicates resumption of conventional frame playback.
Another strategy is to bounceback to any ads that the user has indicated that he or she wishes to view. For example, prior to viewing a movie, the user may have indicated in preferences provided to movie system 10 that he or she is interested in ads relating to washing machines and/or ads that pay the user to watch them, via an input mechanism such as a user preferences web page. Assume ad 93F is a washing machine ad. Segment 368 is a bounceback to the start of ad 93F, and segment 369 indicates resumption of conventional frame playback.
Another strategy is to bounceback to any ads for which the advertiser has required, such as by paying a premium, that the ad be unskippable. For example, the advertiser associated with ad 93G may have paid to make ad 93G unskippable. Segment 379 is a bounceback to the start of ad 93G, and segment 380 indicates resumption of conventional frame playback. Another strategy is to bounceback to the ad closest to the target skip destination, ad 93H in this case. Segment 381 is a bounceback to the start of ad 93H, and segment 382 indicates resumption of conventional frame playback.
Slip-ad is a technique for prohibiting skipping of a proxy ad. Movies can have no-ad zones defined anywhere therein. Slip-ad causes a proxy ad to move its position from the skipped material to after the resumption of playback. In operation, the proxy ad slips to where the user wants to watch.
At step 602, transcoder 40 of movie system 10 determines whether a static ad or a proxy ad is being inserted.
If a static ad is being inserted, then at step 603, the ad is inserted into the appropriate frames of the movie, at the appropriate place within the frames. At step 604, it is determined whether this ad should be protected by bounceback, such as by querying the editor. If no bounceback is needed, then processing is complete. If bounceback protection is desired, then at step 605, it is determined whether there should be a pre-ad zone, and if so, at step 606, how many frames should be in the pre-ad zone. At step 607, it is determined whether there should be a post-ad zone, and if so, at step 608, how many frames should be in the post-ad zone. At step 609, the start of the no-seek zone is defined as the start of the ad minus the pre-ad zone, if any. At step 610, the end of the no-seek zone is defined as the end of the ad plus the post-ad zone, if any. Ad insertion processing is complete.
If a proxy ad is being inserted, then at step 611, the proxy ad is inserted into the appropriate frames of the movie, at the appropriate place within the frames. At step 612, it is determined whether this ad should be protected by slip-ad, such as by querying the editor. If no slip-ad is needed, then processing is complete. If slip-ad protection is desired, then at step 613, it is determined whether there should be at least one no-ad zone in the movie, and if so, at step 614, the frames ineligible to receive a slipped proxy ad are indicated. At step 615, it is determined whether the size of the slipped proxy ad should be adjusted relative to the size of the original proxy ad. If so, at step 616, the number of frames in the slipped ad proxy are defined, and an insertion processing is complete. An increased size basically provides a buffer zone relative to any other ads in the movie. A decreased size is meaningful when the proxy ad is a graphic image, and increases the chance that the slipped ad proxy will be small enough to fit into the unskipped part of the movie. If the slipped ad proxy does not require size adjustment, then at step 617, the size of the slipped ad proxy is set equal to the size of the original ad proxy, and ad insertion processing is complete.
If the movie has been previously viewed, at step 1515, transcoder 40 obtains movie characteristics and popularity statistics for the movie, such as popularity density graph 960, color graph 970, cloud graph 980, or a computed metric such as Total Interest, discussed above as a weighted sum of various play events. Transcoder 40 also obtains the placement rules for an ad. Typically, an ad provider specifies default placement rules when their account is set up, and can modify, or override for a particular ad or movie, these default placement rules as they wish. Default ad provider placement rules specify whether an ad must be placed manually or automatically, by transcoder 40. If automatically, the ad provider placement rules are able to further specify whether the ad should be aligned with the start of a popular portion, be centered on a popular portion, be aligned with the end of a popular portion, be placed in the popular portion of closest duration to the ad, or other placement procedure. The duration of the ad and the duration of the movie may or may not be identical. The ad provider placement rules are also able to specify whether the ad provider requires the most popular portion of the movie, or will accept another portion that is still popular, such as by indicating how much of a premium the ad provider is willing to pay. In some embodiments, for aesthetic reasons, transcoder 40 generally ensures that there is only one movie ad in any portion of a movie.
On the other hand, if the movie has never been viewed, which in some embodiments corresponds to the number of views being below a threshold, then transcoder 40 uses a predefined procedure based on movie characteristics to place ads. Instances of a predefined procedure for ad placement include: (i) to place ads so they are uniformly distributed in time throughout the video, (ii) clustering ads into groups between movie portions of two to three minutes duration, permitting four to six ads to be inserted for every eight to ten minutes of movie content, as is done in broadcast television, (iii) to place ads at scene changes in the movie, (iv) to place ads at mid-scene in the movie, and so on. In some embodiments, for aesthetic reasons, transcoder 40 generally ensures that there is only one movie ad in any portion of a movie.
At step 1525, transcoder 40 places an ad in the movie, with the placement determined either by popularity statistics or the predefined procedure. At step 1530, transcoder 40 stores the movie with ad(s) in videobase 20.
Other procedures may be used to automatically insert an ad into a movie, instead of or in combination with the techniques described above.
Motion analysis of a movie may be used to determine where to place an ad, for example, in a scene having the least amount of motion, or alternatively in a scene having a lot of motion but in a relatively static area of the high motion scene. Motion analysis ensures that ad placement is complementary to activity in the movie, rather than obliterating it. For example, some advertisers may specify that their ad should be placed so that it does not interfere or intersect with dramatic scene changes or scenes of high interest with primary characters visible in the scene.
Face analysis of a movie may be used to ensure that an ad is not placed over a face and/or to ensure that an ad is placed in a scene with at least one human face. Advertiser business rules may indicate a preference for ad placement near faces (i.e., just above or below) so that they do not obscure any characters but are assured that their advertisement will be in an important scene.
Image analysis of a movie may be used to ensure that text or logos are not obscured by an ad. Alternatively, it may be desirable to obscure a logo in the movie with an ad.
Examples of products that perform object recognition and face detection on a video stream are:
Ad area 1110 includes ad thumbnail windows 1111-1120 and navigation buttons 1101, 1102 for altering which thumbnails of the ad thumbnails in videobase 20 are displayed in area 1110. Filename entry window 1103 enables a user to type in the name of an ad file, or select an ad filename from a directory, which puts the file's thumbnail in ad thumbnail window 1111. Ad area 1110 also includes automatic button 1104, for indicating that movie system 10 should select an ad.
An adpack is a pairing of an ad movie and an adpack description. An adpack description controls how the ad movie will be displayed in the transcoded movie. Examples of adpack descriptions are:
The International Advertising Bureau has defined standard ad display areas, and these may be used as adpack descriptions; see, for example, http://www.iab.netIstandards/popuplindex.asp.
An adpack can also include one or more executable commands, as described above with respect to the tree playback product editor. The provider of an ad movie typically specifies certain ad packs as selectable for its ad movie, and configures the commands to be associated with its ad movie. Examples of commands are: (i) a hyperlink to the ad provider's web site, (ii) sending the email address of the viewer of the transcoded movie to the ad provider, and (iii) requesting a file download from the ad provider to the viewer of the transcoded movie; other commands are also contemplated.
If the ad is a proxy ad, the adpack includes the address of the proxy ad server where the ad resides.
An adpack can also include special effects and scripts. For example, an adpack may be comprised of a special effect to fade in a graphical banner, then to fade out the graphical banner, wherein the graphical banner is overlaid in a video stream that permits the user to execute a script when clicking on the banner ad, the script causing the user's browser to launch a web site for an advertiser with analytics and tracking for the user interactions. Adpack area 1121 includes adpack description windows 1125-1128 and navigation buttons 1122, 1123 for altering which adpack descriptions are displayed in area 1121. Filename entry window 1124 enables a user to type in the name of an adpack, or select an adpack from a directory, which puts the adpack in adpack description window 1125. Adpack area 1121 includes script parameters button 1129, for enabling a user to enter parameters for a script. Adpack area 1121 also includes automatic button 1105, for indicating that movie system 10 should select the placement of the ad in the frame.
Texture strip 1130 includes frame representations of the movie being edited, such as frame representation 1131. Positioner 1132 enables rapid selection of a frame of the movie. Slider 1135 indicates frame representations in the texture strip; the editor can adjust the left and right edges of slider 1135. Automatic button 1106 is used when the editor wishes to indicate that movie system 10 should select the frames in which the ad is placed.
To manually insert an ad, the editor selects an ad, such as by clicking ad thumbnail 1117, then selects an adpack description to control where in the frame the ad is placed, such as by clicking adpack description 1127, then adjusts slider 1135 to indicate which frames the ad is placed in, then clicks insert button 1133.
To instruct movie system 10 to select an ad and put it in the movie being edited, the editor clicks automatic buttons 1104, 1105 and 1106.
Rules button 1107 enables the editor to specify the advertiser's ad placement preferences. Advertiser preferences can specify movie characteristics, movie popularity characteristics, viewer characteristics, and how much the advertiser is willing to pay depending on the combination of features delivered, which may include whether the ad is viewed or whether the viewer takes an action relating to the ad. For example, some ads include hyperlinks that a viewer can click on, and the advertiser may be willing to pay a first rate if the ad is merely presented to the viewer, and a second rate if the viewer clicks on the hyperlink. Clicking rules button 1107 makes another window (not shown) appear, that guides the editor through specifying the advertiser preferences. In some embodiments, when an advertiser first registers with movie system 10, the advertiser specifies their default preferences; in these embodiments, the window (not shown) has the defaults filled in, and enables the editor to override the defaults.
To remove an ad, the editor adjusts slider 1135 then clicks cut button 1134.
Buttons 896, 897, 898, 899 function as follows.
Preview button 896 being actuated causes transcoder 40 creates preview window 802, shown in
Publish button 897 being actuated causes transcoder 40 to pop-up a window (not shown) that enables the user to select a thumbnail and destination.
Burn button 898 enables the user to indicate to transcoder 40 that burning (recording) onto a medium is desired. Clicking burn button 898 causes transcoder 40 to pop-up a window (not shown) that enables the user to select a media for burning and provide delivery directions for the burned media.
Back button 899 enables the user to return to edit window 700 in
Returning to
User2 sees segment 904 as movie 910. User2 can define a deep tag indicating segment 914 of movie 910, and permit an outside user, user3, to access segment 914. Now, in movie 910, segments 912 and 916 are private to user 2, while segment 914 is shared.
User3 sees segment 914 as movie 920, and can similarly enable access to all or parts of movie 920.
At step 631 of
Turning to
A mash-up is a sequential display of selected segments. A viewer of a mash-up playback product can only navigate forward or backward in the product.
A tree is a set of segments and a hierarchical, linear control structure for displaying the segments. Generally, a viewer of a tree playback product clicks on selections to navigate the product, in addition to forward and backward.
A link-up is a set of segments and a non-linear control structure for displaying the segments. Generally, a viewer of a link-up playback product navigates via one or more of: forward and back movement, clicking on selections, and/or providing alphanumeric input. A tree and a mash-up are constrained forms of a link-up.
If a mash-up is selected, at step 655, the user selects the sequence of segments to be included via a mash-up editor. Then, at step 685, the user selects whether the mash-up is to be published or burned.
Publishing means transferring the mash-up to videobase 20 or to user PC 110. If publishing is selected, at step 690, the user selects a thumbnail to represent the mash-up, and provides metadata if desired such as a mash-up filename. At step 695, the user indicates a destination for the mash-up file, such as videobase 20, or PC 110. Transcoder 40 responds by transferring the mash-up in accordance with the user's selections.
Burning means writing the mash-up to a removable storage medium, such as a DVD or memory chip, and sending the removable storage medium to the user. If burning is selected, at step 698, transcoder 40 transfers the mash-up file to the removable storage medium type designated by the user. In the case of a DVD, transcoder 40 sends the mash-up file to DVD burner 60, which creates a DVD having the mash-up.
Texture strip 830 represents the mash-up being created. Phantom start and end frames 831, 832 enable the user to add thumbnails before or after the selected thumbnails. Frame selector 835 has start and end portions that can be adjusted by the user. After the user is satisfied with the thumbnails dragged into slots 837-840, the user clicks insert button 833 to insert these thumbnails into the mash-up. In response, transcoder 40 creates a frame representation of each thumbnail, puts the thumbnail frame representation in the appropriate frame of texture strip 830, and clears slots 837-840. To insert subsequent files into the mash-up, the user moves frame selector 835 to a position after the inserted thumbnails. To insert preceding files into the mash-up, the user moves frame selector 835 to include phantom frame 831. To delete files from the mash-up, the user positions frame selector 835 on the frame representations of the thumbnails of the files to be deleted, and clicks cut button 834.
At any time, the user can click preview button 896 to see what the mash-up will look like. In response to preview button 896, transcoder 40 creates preview window 802, shown in
Publish button 897 enables the user to indicate to transcoder 40 that publication of the mash-up is desired. Clicking publish button 897 causes transcoder 40 to pop-up a window (not shown) that enables the user to select a thumbnail and destination.
Burn button 898 enables the user to indicate to transcoder 40 that burning of the mash-up is desired. Clicking bum button 898 causes transcoder 40 to pop-up a window (not shown) that enables the user to select a media for burning and provide delivery directions for the burned media.
Back button 899 enables the user to return to edit window 700 in
If a tree playback product is selected, at step 660, the user selects the sequence of segments to be included via a tree editor, and at step 665, defines the tree structure. Then, at step 685, the user selects whether the tree playback product is to be published or burned.
Tree structure window 855 shows a graph of the control structure of the tree playback product being created. Initially, the window is blank, since nothing has been specified.
Each segment of a tree playback product comprises a background, a foreground and a link. In one embodiment, a background is a still image, a foreground is a video or audio segment, and the link is indicated by a graphic image. When the viewer of the playback product clicks on the link, the viewer is taken to the next segment, that is, a link indicates one segment. Each segment can have 0, 1 or multiple links.
A tree segment can also include 0, 1 or multiple commands. Typically, a command is indicated by a graphic image. When the viewer of the playback product clicks on the command, the command is sent to the source of the playback product, such as server 50, for execution.
Returning to
If a tree playback product segment is created with at least one link, transcoder 40 creates an empty segment as the destination of each link, and displays the empty segment in tree structure window 855. The editor clicks on the empty segment in tree structure window 855 and inserts thumbnails into at least one of the background, foreground and link slots.
If the editor wishes to delete a segment, the editor selects the segment in tree structure window 855, then clicks cut button 856 to remove the segment. Removing a segment automatically removes the link leading to the segment from the preceding segment.
To create a command in a segment, the editor clicks add command button 865. Transcoder 40 provides a pop-up window with a command editor (not shown) that enables the editor to drag and drop a thumbnail indicating the command, select a command from a menu of commands or type the command directly into a command line window (if the editor knows how to write commands in the command language, such as JavaScript), and, when appropriate, provide parameters for the command. Examples of commands are: (i) a hyperlink to a webpage, (ii) provide the email address of the viewer of the playback product to the owner of the playback product, (iii) provide the email address of the viewer to a third party, (iv) download a program and execute it, and so on.
It will be understood that a thumbnail can be dropped into multiple slots during creation of a tree playback product.
Clicking preview button 896 causes transcoder 40 to create a window similar to that shown in
If a link-up playback product is selected, at step 670, the user selects the sequence of segments to be included via a link-up editor. At step 675, the user defines the link-up structure. At step 680, the user defines the link-up navigation questions and answers. Then, at step 685, the user selects whether the link-up is to be published or burned.
Link-up structure window 875 shows a graph of the control structure of the link-up playback product being created. Initially, the window is blank, since nothing has been specified.
Returning to
A Q&A dialog (not shown), consists of an optional header and a set of choices associated with the respective links. One instance of a Q&A is,
Click on the word to indicate what you want to view next:
Another instance of a Q&A is,
Another instance of a Q&A is,
If the editor wishes to delete a segment, the editor selects the segment in link-up structure window 875, then clicks cut button 876 to remove the segment. Removing a segment automatically removes the portion of the Q&A leading to the segment from the preceding segments.
Clicking preview button 896 causes transcoder 40 to create a window similar to that shown in
The transcoded movie is then stored in videobase 20, and database 30 is suitably updated.
A use case will now be discussed.
Let it be assumed that a user uploads three files (
Movie system 10 converts each of these three files to proxy files in H.263 Flash format (
Now the user creates a mash-up (
Next, the user adds special effects (
Finally, the user inserts two ads, allowing the system to automatically choose the ads and their placement (
It will be appreciated that some editing functions can be accomplished in several ways. For example, if the user wishes to delete a frame from the movie, this can be accomplished via (1) creating a mash-up comprising the frames before the deleted frame, followed by the frames after the deleted frame, and not including the deleted frame; (2) adding a special effect to cause the deleted frame to be entirely dark or entirely light, as part of a scene transition; (3) directly selecting “delete frame” from the special effects menu; (3) enabling sharing of the entirety of the movie except for the to-be-deleted frame; and so on.
Elements 720, 730, 740 and 804-807 function as described above; the description will not be repeated here for brevity.
Deep tag thumbnails 738A, 738B, 738C appear in a deep tag windowpane. Each of the deep tags associated with the movie being displayed has an associated thumbnail. Each thumbnail is a small image and associated text including deep tag creator name, creation date and time, and any keywords provided by the creator. The small image is configured by the deep tag creator, and can be a portion of an image in a scene, a shrunken image of the scene, an image of the creator, or other suitable visual cue. The deep tag windowpane enables the viewer to scroll among the deep tag thumbnails for the movie. Typically, the thumbnails appear in sequential order corresponding to their location in the movie, but other orders can be used, such as popularity order.
Send button 1392 is used to email a hyperlink of the movie to an email account. Clicking on send button 1392 causes a window to pop-up for entry of an email address, similar to the popup window shown in
Favorites button 1393 is used to add the movie to the viewer's list of favorite movies maintained by the player for the user.
Popularity selector 1395 enables the user to control display of a popularity graph, as explained above with regard to
Rating selector 1396 enables the user to rate the movie. Rating selector 1396 comprises a set of so-called radio buttons, from which one button can be selected. The default is “unrated.” If a viewer wishes to rate the movie, she or he clicks on the appropriate number of stars, from one star to five stars. In other embodiments, rating selector 1396 uses a different format, such as a slider bar instead of radio buttons.
Server 80 serves to provide a website to visitors that access the website using network 100. Typically, server 80 provides software from movie system 10 that enables display of movies stored in movie storage 90 using the screen interface shown in
PC 140 is a general purpose computer executing software in accordance with the invention. In one embodiment, PC 140 has a web browser that is capable of interacting with information received from server 50 or 80.
Registered content owner host 141 is a general purpose computer programmed to receive and send messages using communication network 100, and may be a personal computer or a web server coupled to other computers.
It will be recalled that a deep tag can be thought of as a movie segment bookmark.
A movie-id-print is like a fingerprint for a movie, that is, a short way to identify a movie.
It is recognized that the ability to deep tag movies, and to send the deep tags to others opens the door to copyright concerns. The configuration shown in
At step 2070, the user of PC 140 views a movie, either from movie storage 20 that is part of movie system 10 (see
At step 2082, the user of phone 131 receives the deep tag, such as in the form of a hyperlink in an email, an icon in an instant messaging system, or other suitable method. At step 2084, the user of phone 131 activates the deep tag, which causes a movie player to begin executing on phone 131 and to play the movie segment indicated by the deep tag.
It will be recalled that a snapshot, also referred to as a thumbnail, is a frame chosen to represent the object, for example, a frame at the start, middle or end of the deep tag.
Deep tags for which the content owner is known are stored in known content owner deep tags database 32. Deep tags for which the content owner is unknown are stored in unknown content owner deep tags database 34.
Movie-id-prints for which the content owner is known are stored in known content owner movie-id-prints database 31. Movie-id-prints for which the content owner is unknown are stored in unknown content owner movie-id-prints database 33.
Deep tag imprint 2320 is superimposed on texture strip 720, and indicates frames that have a deep tag associated therewith. There may be many instances of deep tag imprint 2320 for each texture strip 720, each instance corresponding to a separate deep tag, the deep tag being stored in known content owner deep tag database 32 or unknown content owner deep tag database 34.
Deep tag descriptor 2330 is associated with deep tag imprint 2320, and provides the text descriptor, if any, that is part of the deep tag. For example, if a cursor is positioned on deep tag descriptor 2330, a small area opens on screen 2300 and displays the text descriptor. Alternately, a pop-up window displays the text descriptor.
Deep tag thumbnail 2335 is similar to deep tag thumbnail 738A of
Deep tag send button 2340 is associated with deep tag imprint 2320, and provides a quick way to send the deep tag to another party. For example, if the cursor clicks on deep tag send button 2340, a small pop-up window opens and enables the user to specify the recipient's address.
A streamlined way for creating and sending a deep tag is also provided, sometimes referred to as a “one-click deep tag.”
Assume that movie 20J is being viewed in screen display 2300 on PC 140 by a user. Movie 20J is typically provided from movie storage 20 (
Generally, the owner of the website from which movie 20J is provided specifies how a fast deep tag is created, selecting from one or more of the following:
User preferences—follow the preference of a registered (formal) viewer for how the fast deep tag button operates, and if the viewer is not a registered viewer, then use one of the following techniques as a default;
Access permission refers to whether and how the recipient can alter the deep tag. At one level, no alterations are permitted. At the standard level, the recipient can add comments but cannot otherwise alter the deep tag. At another level, the recipient can create their own version of the deep tag, indicating more or less of the original movie and possibly indicating another movie, and inheriting the text associated with the first deep tag.
Instant deep tag sending is discussed below with regard to
In one embodiment of movie system 10, a deep tag creator is either an anonymous user, a casual user or a formal user. Movie system 10 can consider deep tags created by some or all of anonymous, casual and formal users when determining the popularity or other statistic of a movie.
An anonymous user is not known to movie system 10. Deep tags created by an anonymous user are generally not stored for subsequent retrieval by the anonymous user, that is, they exist only while the anonymous user is interacting with the movie associated with the deep tag. Deep tags created by anonymous users have analytic value to movie system 10. An anonymous user can comment on and rate only their own deep tags. Movie system 10 tracks deep tags from anonymous users via a suitable temporary mechanism, such as a cookie.
A casual user is a user for which movie system 10 knows only his or her address, such as email address or instant message ID or telephone number. Typically, an anonymous user converts to a casual user by entering an address in the sender portion of pop-up window 2350. A casual user can comment on and rate only their own deep tags. Deep tags created by a casual user can be retrieved by the casual user, generally because a so-called cookie is stored on the casual user's access device, or because the casual user provides their address to movie system 10.
A formal user is a user that has registered with movie system 10 via a sign-up process involving creating a profile with a registered i.d. and demographic information relating to the user, and perhaps validating other of their information such as that their email address is actually controlled by them. Deep tags created by a formal user are stored in known content owner deep tag database 32 or unknown content owner deep tag database 34.
A formal user can comment on, and rate movies and deep tags created by other formal users, can upload and deep tag movies, can make deep tags and movies public, and can search and browse their deep tags and public deep tags created by other formal users. A formal user can be a content owner. Movie system 10 may periodically invite casual users to convert to formal users. The demographic information associated with formal users makes their activities more valuable to movie system 10 than the activities of casual or anonymous users. In various embodiments of movie system 10, the capabilities and privileges of a formal user may differ.
Let it be assumed that movie system 10 has received a request to create a deep tag, such as when a user clicks on deep tag button 740 or fast deep tag button 741.
At step 2400, movie system 10 determines whether the content owner of the movie is known, such as by checking known content owner movie-id-print database 31 or by recognizing that the movie is stored in movie storage 20. If the content owner is known, then at step 2405, movie system 10 follows the content owner's rules for deep tag creation relative to the movie. For example, the content owner may permit deep tags only to be made from a specified version of the movie, not from other copies thereof. As another example, the content owner may require that at the conclusion of play of a deep tagged segment, the player loads a particular webpage. As a further example, the content owner may require that the deep tag include one or more advertisements included in the content. As a yet further example, the content owner may require a share of revenue realized from viewing of the content or any deep tagged portion of the content. Other rules are contemplated such as controlling the ways in which deep tags can be combined, where some deep tags might only be allowed for combining with deep tags from the same content owner. At step 2410, the deep tag is added to known content owner deep tag database 32, and deep tag creation processing is complete.
However, if the content owner is unknown, then at step 2415, movie system 10 notifies the would-be creator that the content owner is unknown and so movie system 10 will be collecting data on who views the to-be-created deep tag, and when, and will be notifying recipients of the deep tag that if they click it, data about them will be collected. The notification is provided by a pop-up window, playing an audio file, or other suitable means. In some embodiments, the would-be creator notifies movie system 10 of who the likely content owner is, such as by selecting from a menu of registered content owners and/or by providing contact information and/or a name of the content owner; this serves as a request for permission to use the content, which is generally forwarded by movie system 10 to the likely content owner. At step 2420, it is determined whether the would-be creator consents to the data collection. If not, processing is complete. If consent is provided, then at step 2425, movie system 10 creates the deep tag and adds it to unknown content owner deep tag database 34. Then, at step 2430, movie system 10 checks whether the movie has a movie-id-print in unknown content owner movie-id-print database 33. If so, this means that registered content owners have already been notified, and so processing is complete. If not, at step 2433, movie system 10 places the movie-id-print to unknown content owner database 33, and at step 2435, movie system 10 sends an alert to all registered content owners, or to a subset of all registered content owners, and to the owner of the site where the deep tagged movie resides, if the site owner is known. The subset may be determined by the text associated with the movie, by the similarity of the movie to movies with known content owners, or by another suitable method.
Registered content owner 141 is a party who has previously registered with movie system 10 to receive alerts when a movie with an unknown content owner is deep tagged. At step 2480, registered content owner 141 receives an alert from movie system 10. At step 2485, owner 141 determines whether they own the content, and if so, establishes ownership by a suitable procedure. In some cases, content ownership is established in response to other than alerts from movie system 10; that is, a content owner may otherwise become aware that its content is being used through movie system 10. At step 2490, owner 141 specifies rules for deep tagging the movie, and sends the rules to movie system 10.
At step 2450, movie system 10 receives the newly created content owner's rules for deep tagging. If not already done during step 2485, movie system 10 transfers the movie-id-print from database 33 to database 31. At step 2460, movie system 10 ensures that deep tags associated with the movie comply with the rules, and if they do not, either (a) the deep tags are suitably adjusted, or (b) the deep tag is rendered inactive, and the deep tag creator is invited to adjust the deep tag to make it active again. At step 2465, the deep tags associated with the movie are moved from database 34 to database 32. At step 2470, movie system 10 notifies the deep tag creator and all parties who have viewed the deep tag that the content owner is known, and any unauthorized copies should be destroyed or brought into compliance with the content owner's rules.
Dynamically updated viewer displays will now be discussed. First, dynamically updated deep tags will be discussed. Then, dynamically updated movie selections will be discussed.
Dynamically updated deep tags are relevant when a viewer is watching a movie in a manner in which the viewer can see the deep tags associated with the movie, such as:
Movie system 10 receives, in real time, information about how a viewer is interacting with a movie, such as clicking the play button, the pause button and so on. Accordingly, movie system 10 knows all current viewers of a movie.
When one viewer of a movie acts on a deep tag, such as creating, editing or deleting the deep tag, then the deep tag displays, if any, for all other viewers of the movie are updated to reflect the deep tag act. This is dynamically updated deep tag display.
It will be recalled that if the viewer is an anonymous user, the deep tags created by that viewer persist only for the duration of the viewer's session. Specifically, when movie system 10 determines that the anonymous user is no longer watching the movie, movie system 10 dynamically updates the displays of all other viewers of the movie to delete any deep tags created by the anonymous user.
In some embodiments, formal users and/or the owner of the movie and/or the owner of the website that provides the movie can control whether dynamically updated deep tag displays are supported. Specifically, some formal users may prefer to turn off this feature.
Dynamically updated movie selections enable movie system 10 to operate in a sort of narrowcast format. Different versions of dynamically updated movie selections are discussed. It will be appreciated that other versions of dynamically updated movie selections can be configured.
A first version of dynamically updated movie selections enables a formal user of movie system 10 to specify characteristics of movies that should be presented to the user, and then the user can just view in a passive mode, without selecting movies. Example characteristics are:
In a variation of this version of dynamically updated movie selections, instead of the whole movie being displayed, only its deep tags with certain characteristics are displayed, such as its three most popular deep tags, deep tags created by users with certain demographic characteristics, or deep tags created by specifically named users.
A viewing trail is a particular sequence of movie viewing for a particular user. The viewing trail includes not only which movies were viewed, but also the actions taken for the movie, such as play, pause, fast forward, rewind and so on.
In another version of dynamically updated movie selections, formal users can designate their viewing trails as public or private. Movie system 10 assumes a default of “private” for viewing trails. A formal user can specify the level of public access to their viewing trails, e.g., open to all, open to anyone who pays, open to designated users (whitelist), open to all except designated users (blacklist), open to users with certain demographic characteristics, open to all except users with certain demographic characteristics, and so on.
If a formal user has public viewing trails, then another formal user can instruct movie system 10 to show the other formal user the movies that were viewed by the first formal user, such as in the first formal user's most recent movie viewing, on a specific day or meeting other characteristics. This is useful when the first formal user is a celebrity, or carefully selects their viewing trails. It will be appreciated that a public viewing trail is a simple version of a mash-up movie. However, the creator of the mash-up usually recognizes revenue from ads inserted into the mash-up. The creator of a viewing trail can recognize revenue from something akin to a subscription model. In most cases, creation of a viewing trail is easier to accomplish than creation of a mash-up movie. In a variation, the first formal user can create a viewing trail, and then email it to friends.
In a variation of this version of dynamically updated movie selections, instead of the whole movie being displayed, only its deep tags created by the first formal user are displayed.
Although an illustrative embodiment of the invention, and various modifications thereof, have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to this precise embodiment and the described modifications, and that various changes and further modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims.
This application is a continuation of U.S. patent application Ser. No. 17/502,395, filed Oct. 15, 2021, now U.S. Pat. No. 11,589,087, entitled “MOVIE ADVERTISING PLAYBACK SYSTEMS AND METHODS,” which is a continuation of U.S. patent application Ser. No. 16/854,739, filed Apr. 21, 2020, now U.S. Pat. No. 11,153,614, entitled “MOVIE ADVERTISING PLAYBACK SYSTEMS AND METHODS,” which is a continuation of U.S. patent application Ser. No. 16/227,481, filed Dec. 20, 2018, now U.S. Pat. No. 10,650,863, entitled “MOVIE ADVERTISING PLAYBACK SYSTEMS AND METHODS,” which is a continuation of U.S. patent application Ser. No. 15/489,191, filed Apr. 17, 2017, now U.S. Pat. No. 10,192,587, entitled “MOVIE ADVERTISING PLAYBACK SYSTEMS AND METHODS,” which is a continuation of U.S. patent application Ser. No. 14/885,632, filed Oct. 16, 2015, now U.S. Pat. No. 9,653,120, entitled “MOVIE ADVERTISING PLAYBACK SYSTEMS AND METHODS,” which is a continuation of U.S. patent application Ser. No. 14/247,059, filed Apr. 7, 2014, now U.S. Pat. No. 9,330,723, entitled “MOVIE ADVERTISING PLAYBACK SYSTEMS AND METHODS,” which is a continuation of U.S. patent application Ser. No. 13/404,911, filed Feb. 24, 2012, now U.S. Pat. No. 8,739,205, entitled “MOVIE ADVERTISING PLAYBACK TECHNIQUES,” which is a continuation of U.S. patent application Ser. No. 11/713,116, filed Feb. 27, 2007, now U.S. Pat. No. 8,141,111, entitled “MOVIE ADVERTISING PLAYBACK TECHNIQUES,” which is a continuation-in-part of U.S. patent application Ser. No. 11/592,901, filed Nov. 3, 2006, now U.S. Pat. No. 8,145,528, entitled “MOVIE ADVERTISING PLACEMENT OPTIMIZATION BASED ON BEHAVIOR AND CONTENT ANALYSIS,” which is a continuation-in-part of U.S. patent application Ser. No. 11/439,600, filed May 23, 2006, now U.S. Pat. No. 8,755,673, entitled “METHOD, SYSTEM AND COMPUTER PROGRAM PRODUCT FOR EDITING MOVIES IN DISTRIBUTED SCALABLE MEDIA ENVIRONMENT,” Ser. No. 11/439,594, filed May 23, 2006, now U.S. Pat. No. 8,724,969, entitled “METHOD, SYSTEM AND COMPUTER PROGRAM PRODUCT FOR EDITING MOVIES IN DISTRIBUTED SCALABLE MEDIA ENVIRONMENT,” and Ser. No. 11/439,593, filed May 23, 2006, now U.S. Pat. No. 7,877,689, entitled “DISTRIBUTED SCALABLE MEDIA ENVIRONMENT FOR MOVIE ADVERTISING PLACEMENT IN USER-CREATED MOVIES,” which claims a benefit of priority from U.S. Provisional Application No. 60/683,662, filed May 23, 2005, entitled “DISTRIBUTED SCALABLE MEDIA ENVIRONMENT.” All applications listed in this paragraph are fully incorporated by reference herein in their entireties.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8141111 | Gilley | Mar 2012 | B2 |
| 8739205 | Gilley | May 2014 | B2 |
| 9330723 | Gilley | May 2016 | B2 |
| 9653120 | Gilley | May 2017 | B2 |
| 10192587 | Gilley | Jan 2019 | B2 |
| 10650863 | Gilley | May 2020 | B2 |
| 11153614 | Gilley | Oct 2021 | B2 |
| 11589087 | Gilley | Feb 2023 | B2 |
| 11626141 | Gilley et al. | Apr 2023 | B2 |
| 11706388 | Gilley et al. | Jul 2023 | B2 |
| 20020144263 | Eldering | Oct 2002 | A1 |
| 20020145622 | Kauffman et al. | Oct 2002 | A1 |
| 20040030599 | Sie | Feb 2004 | A1 |
| 20040103429 | Carlucci | May 2004 | A1 |
| 20040133909 | Ma | Jul 2004 | A1 |
| 20040268223 | Tojo | Dec 2004 | A1 |
| 20050058460 | Wang | Mar 2005 | A1 |
| 20060013554 | Poslinski | Jan 2006 | A1 |
| 20060224940 | Lee | Oct 2006 | A1 |
| 20080212937 | Son | Sep 2008 | A1 |
| 20080219638 | Haot | Sep 2008 | A1 |
| 20200251147 | Gilley | Aug 2020 | A1 |
| 20230197113 | Gilley et al. | Jun 2023 | A1 |
| 20230199234 | Gilley | Jun 2023 | A1 |
| 20230319235 | Gilley et al. | Oct 2023 | A1 |
| Entry |
|---|
| Final Office Action issued for U.S. Appl. No. 17/093,727 dated Mar. 31, 2023, 22 pages. |
| Final Office Action issued for U.S. Appl. No. 17/010,731 dated May 16, 2023, 21 pages. |
| Notice of Allowance issued for U.S. Appl. No. 17/093,727 dated Aug. 1, 2023, 9 pages. |
| Notice of Allowance issued for U.S. Appl. No. 17/196,087 dated Jul. 20, 2023, 14 pages. |
| Office Action issued for U.S. Appl. No. 18/170,764 dated Jul. 7, 2023, 14 pages. |
| Office Action issued for U.S. Appl. No. 17/010,731 dated Sep. 8, 2023, 9 pages. |
| Notice of Allowance issued for U.S. Appl. No. 17/093,727, dated Oct. 30, 2023, 9 pages. |
| Notice of Allowance issued for U.S. Appl. No. 18/309,557, dated Nov. 30, 2023, 9 pages. |
| Notice of Allowance issued for U.S. Appl. No. 17/010,731, dated Dec. 29, 2023, 9 pages. |
| Office Action issued for U.S. Appl. No. 18/170,764, dated Dec. 29, 2023, 12 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20230199234 A1 | Jun 2023 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60683662 | May 2005 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17502395 | Oct 2021 | US |
| Child | 18171287 | US | |
| Parent | 16854739 | Apr 2020 | US |
| Child | 17502395 | US | |
| Parent | 16227481 | Dec 2018 | US |
| Child | 16854739 | US | |
| Parent | 15489191 | Apr 2017 | US |
| Child | 16227481 | US | |
| Parent | 14885632 | Oct 2015 | US |
| Child | 15489191 | US | |
| Parent | 14247059 | Apr 2014 | US |
| Child | 14885632 | US | |
| Parent | 13404911 | Feb 2012 | US |
| Child | 14247059 | US | |
| Parent | 11713116 | Feb 2007 | US |
| Child | 13404911 | US | |
| Parent | 11439593 | May 2006 | US |
| Child | 11439594 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11592901 | Nov 2006 | US |
| Child | 11713116 | US | |
| Parent | 11439600 | May 2006 | US |
| Child | 11592901 | US | |
| Parent | 11439594 | May 2006 | US |
| Child | 17502395 | US |
