Processing content streaming

Abstract

A system for providing streaming services. The system includes a plurality of users each for generating a stream of an event on a connection of a public network. The system also includes a server configured to receive a plurality of the generated streams, determine content for at least one output stream, and output the at least one output stream on a connection of a public network, with at least one user receiving the at least one output stream. The content is determined based on one or more of content received from input streams, content requested by a viewer, and the user profiles of the contributors.

Description

BACKGROUND TO THE INVENTION

Field of the Invention

The present invention relates to an architecture in which data can be streamed and streamed data can be accessed.

Description of the Related Art

It is known in the art for data streamed from various sources, and various other sources to access streamed data. Thus capture devices may capture events for example using a video camera functionality of a mobile phone, and stream associated data. Viewing devices may receive the streamed data and vie the associated video.

It is an aim of the invention to provide an improved content streaming environment.

SUMMARY OF THE INVENTION

There is provided a system for providing streaming services, the system comprising: a plurality of users each for generating a stream of an event on a connection of a public network; and a server configured to: receive a plurality of the generated streams on connections of the public network; determine content for at least one output stream in dependence on one or more of: the content received on the input streams, the content requested by a viewer, and the user profiles of the contributors; and output the at least one output stream on a connection of a public network, at least one user for receiving the at least one output stream on a connection of the public network.

A key feature is the provision of control—the determination feature. This aspect is about multiple streams going into one output stream, but under some form of control—a decision is made. This aspect puts intelligence in the middle: the output stream uses part of the collection of input streams.

The server may include a first interface connected to the public network for receiving the plurality of the generated streams, and a second interface connected to the public network for outputting the at least one output stream. The server may be associated with a private network connected to the first and second interfaces.

The at least one of the plurality of users may provide a live stream on a connection of the public network, and the at least one output stream provided on a public network includes that live stream.

The at least one user may be one or more of: a content originator, a content consumer, a content requestor, a viewer, a contributor, a director, or a content responder. A director may be a client configured to control the output stream of the server. In general, all devices are users of the system, whether they are identified as a consumer or a client in parts of this description and/or parts of the drawings.

The at least one user may be one or more of a content requestor, an editor, an analyst, a syndicator, a curator, a moderator, an advertiser, or a content responder. The syndicator may identify input streams associated with certain events. The curator may choose specific inputs streams. The input streams may be associated with events, and the curator may choose specific events. The moderator may apply predetermined rules to the input streams.

Each user may generate a stream, each stream being allocated a timing reference. The timing reference may be relative to a master clock, an audio trigger or a visual trigger. The timing reference may be relative to a master clock, the timing reference being applied to each stream received by the server. Each stream may have one or more of a start time, a finish time and a continuity time or offset defined relative to the master clock. The timing reference may allow rights management to be applied. The timing reference may provide a time stamp on each stream generated by a user device. At least one stream may be allocated a geo-reference. The geo-reference may vary over time. The geo-reference may indicate or include a pointing direction of a camera associated with the device generating the stream. The geo-reference allocated to a stream may be updated in dependence on the device moving by a distance which is greater than a threshold. The threshold value may be set in dependence on an event. The geo-reference may be included as metadata associated with or embedded in a stream.

The server may group a plurality of input streams into an event. An input stream may be grouped into one or more events. A stream may be grouped according to its metadata.

Each consumer may be associated with an appliance. An appliance may be one of: a website, an application running on a mobile device, an application running on a television, a server application.

A server may comprise an API (application programmable interface) and a media gateway. The API may provide an interface to metadata services, and the media gateway has an interface to the streams. The media gateway may be configured to receive the input streams and generate one or more output streams. The media gateway may be configured to store the input streams. Each input stream may be allocated to an event. If it is identified that the input stream should be associated with an existing event, it is allocated to that event. If it is identified that the input stream is not associated with any existing event, then a new event is created and the input stream associated with that new event. Each input stream is allocated to an event in accordance with its metadata. The media gateway may be configured to receive data, metadata, and data with embedded metadata. The media gateway may be configured to process streamed time dependent data or metadata. The API may be configured to process data which is not time dependent.

The data may be control related. The control data is derived from metadata.

A request from a user is received by the API, and the request defines the rules which are applied to the media gateway and data streams that come into it. The rights holders may create rules that instruct the server to store content. The rights holders may assign rights to the streams.

An input stream of data and an input stream of metadata may be received by the media gateway. The media gateway may output an output stream. The media gateway may communicate with the API. The API receives control (e.g. a request), and user rules.

This provides, in an aspect, a server: determination of association with each event done by API, but then media gateway acts on it.

Static metadata includes, e.g., time. Preferably generated by originating device. Dynamic metadata includes, e.g. point of view. Preferably generated by the originating device. A password is static metadata. The media gateway is preferably only interested in data that changes with time. Static metadata goes into API.

The API may be configured to authenticate and/or authorize each input stream. The API may be configured to provide functionality associated with streaming services. The functionality may be applied to the input stream received at the media gateway in dependence on the metadata associated with the input stream.

There is provided a system for streaming data comprising: a plurality of capture devices for capturing events and generating captured data streams; a server for receiving the captured data streams and for generating at least one viewing data stream in dependence thereon, the server comprising: a media gateway for receiving the at captured data streams and for generating the at least one viewing data stream; and an application programmable interface for receiving control data associated with the captured data streams, wherein the at least one viewing stream is output by the media gateway under the control of the application programmable interface in dependence on the control data, the system further comprising: at least one viewing device for receiving the viewing data stream.

A user is enabled, on receipt of an input stream, to preferably join the input stream to an event. Such a user may also be referred to as a client.

If it is identified that the input stream should be associated with an existing event, it is joined to that event. There may be more than one streamed event.

If it is identified that the input stream is not associated with an existing event, a new event is created and it is joined to that event.

A user is enabled, on determination of provision of an invitation key or password with the input stream, to join the input stream to an associated event. Such a user may also be referred to as a client.

BRIEF DESCRIPTION OF THE FIGURES

The invention is now described by way of reference to the following figures, in which:

FIG. 1 illustrates an example architecture in which steaming services may be provided;

FIG. 2 illustrates further an exemplary provisions of appliances in a content streaming service;

FIG. 3 illustrates an exemplary server implementation;

FIG. 4 illustrates an exemplary implementation of an API;

FIG. 5 illustrates an exemplary implementation of a media gateway;

FIG. 6 illustrates an exemplary event;

FIG. 7 illustrates an exemplary process for creating a new stream;

FIG. 8 illustrates an exemplary process for joining an existing event; and

FIG. 9 illustrates an exemplary process for creating an event.

DESCRIPTION OF PREFERRED EMBODIMENTS

With reference to FIG. 1 there is illustrated a system architecture within which embodiments may be implemented.

With reference to FIG. 1 there is illustrated: a plurality of devices, labelled capture devices, denoted by reference numerals 12a, 12b, 12c; a plurality of devices, labelled viewing devices, denoted by reference numerals 16a, 16b; a device, labelled editing device, denoted by reference numeral 20a; a network denoted by reference numeral 4; and a server denoted by reference numeral 2.

Each of the devices 12a, 12b, 12c is referred to as a capture device as in the described embodiments of the invention the devices capture content. However the devices are not limited to capturing content, and may have other functionality and purposes. In examples each capture device 12a, 12b, 12c may be a mobile device such as a mobile phone.

Each of the capture devices 12a, 12b, 12c may capture an image utilising a preferably integrated image capture device (such as a video camera), and may thus generate a video stream on a respective communication line 14a, 14b, 14c. The respective communication lines 14a, 14b, 14c provide inputs to the network 4, which is preferably a public network such as the Internet. The communication lines 14a, 14b, 14c are illustrated as bi-directional, to show that the capture devices 12a, 12b, 12c may receive signals as well as generate signals.

The server 2 is configured to receive inputs from the capture devices 12a, 12b, 12c as denoted by the bi-directional communication lines 6, connected between the server 2 and the network 4. In embodiments, the server 2 receives a plurality of video streams from the capture devices, as the signals on lines 14a, 14b, 14c are video streams.

The server 2 may process the video streams received from the capture devices as will be discussed further hereinbelow.

The server 2 may generate further video streams on bi-directional communication line 6 to the network 4, to the bi-directional communication lines 18a, 18b, associated with the devices 16a, 16b respectively.

Each of the devices 16a, 16b is referred to as a viewing device as in the described embodiments of the invention the devices allow content to be viewed. However the devices are not limited to providing viewing of content, and may have other functionality and purposes. In examples each viewing device 16a, 16b may be a mobile device such as a mobile phone.

The viewing devices 16a and 16b may be associated with a display (preferably an integrated display) for viewing the video streams provided on the respective communication lines 18a, 18b.

A single device may be both a capture device and a viewing device. Thus, for example, a mobile phone device may be enabled in order to operate as both a capture device and a viewing device.

A device operating as a capture device may generate multiple video streams, such that a capture device such as capture device 12a may be connected to the network 4 via multiple video streams, with multiple video streams being provided on communication line 14a.

A viewing device may be arranged in order to receive multiple video streams. Thus a viewing device such as viewing device 16a may be arranged to receive multiple video streams on communication line 18a.

A single device may be a capture device providing multiple video streams and may be a viewing device receiving multiple video streams.

Each capture device and viewing device is connected to the network 4 with a bi-directional communication link, and thus one or all of the viewing devices 16A, 16B may provide a signal to the network 6 in order to provide a feedback or control signal to the server 2. The server 2 may provide control signals to the network 4 in order to provide control signals to one or more of the capture devices 12a, 12b, 12c.

The capture devices 12a, 12b, 12c are preferably independent of each other, and are independent of the server 2. Similarly the viewing devices 16a, 16b are preferably independent of each other, and are independent of the server 2.

The capture devices 12a, 12b, 12b are shown in FIG. 1 as communicating with the server 2 via a single network 4. In practice the capture devices 12a, 12b, 12c may be connected to the server 2 via multiple networks, and there may not be a common network path for the multiple capture devices to the server 2. Similarly the viewing devices 16a, 16b may be connected to the server 2 via multiple networks, and there may not be a single common network path from the server 2 to the viewing devices 16a, 16b.

The system architecture of FIG. 1 may be used to stream live content from capture devices to the server, and then for viewing devices to access the live content in streams from the server.

FIG. 2 sets out examples of the ‘appliances’ which may be provided in a content streaming and delivery service as described. This illustrates the broad scope of appliances. There are multiple appliances in which devices have slightly different personae/roles with different rights/task/feature abilities etc. FIG. 2 is not a front-end or back-end illustration: instead it lists the various roles or personae that devices within the system have.

Each role requires various services to function and, in turn, provides new or improved services for other roles to use.

As shown in FIG. 2, services 200 may interface with users (sometimes referred to as consumers) including a viewer 204, a contributor 206, and a director 208; further users (sometimes referred to as clients) including an editor 214, an analyst 216, a syndicator 218, a curator 220, a moderator 222, and an advertiser 224; a content requester 202 which is a user which may be a consumer and/or a client; and a content responder which is a user which may be a consumer and/or a client. The different roles may be allocated to single users. For example, a single content consumers other roles may be allocated to larger clients of the presented system that are enabled with a larger set of features to control the streams.

FIG. 3 illustrates an example top-level architecture that outlines the general structure of a platform. The core is that each feature of the system can be seen as an individual appliance based on this platform. Those different appliances could be applications on the web, smart phones, etc., but also products with different levels of feature sets. A product could be provided to a customer to manage live streams or the product that is publicity available to private users.

All of those appliances take the benefit of an aggregated API as well as a media gateway.

With further reference to FIG. 3, there is illustrated an API Aggregation architecture. FIG. 3 shows a simplified view of the platform's technical organisation. There may be a set of appliances 300 such as a consumer iOS app; a moderator's workstation app; an editor app; a consumer website etc.

Each of these applications 300 operates by interfacing with two platform end points: an API 302 and a media gateway 304.

The API 302 is an outward facing aggregation of each of the services wrapped with authentication, authorisation, load-balancing etc. This API manages the metadata of the platform: the users, the events—past, present and future—and the streams within those events; feedback; ratings; analysis of content for grouping and so on. The API 302 is shown as interfacing with a collection of services contributing to the aggregated API, as denoted by reference numeral 306.

The media gateway 304 manages the flow of the media itself from capture to playback and recording. Imagine the application as being a water utility company: the media gateway is the pipes through which the water flows; while the API 302 is the management, planning, customer and oversight function.

The media gateway 304 is shown as communicating with adapters 308 and a WebRTC 310, with the adapters 308 also being in communication with the API 302.

FIGS. 4 and 5 illustrate the underlying structure of the API 302 and the media gateway 304.

With reference to FIG. 4, there is shown various services collected together under the API header in the ‘API Aggregation’.

All access to this API is conducted through an authentication and authorisation layer 402 which sits on top.

Whereas the media gateway 304 backs onto a media store (as shown in FIG. 5), the API 302 is a series of services that derive their function from the metadata store 426. All services that wish to store persistent data do so by passing to the metadata store 426.

As shown in FIG. 4, the API 302 services include: a command and control service 404, a device registry 406, a user profile service 408, a marketplace service 410, an event service 412, a content matcher 416, a moderation service 414, a grouping service 418, an interactivity edit service 424, an interactivity generator 420, and content analysis services 422. The user profile service 408, the event service 412 and the grouping service 418 connect to the metadata store 426.

With reference to FIG. 5, the media gateway part of the platform is shown in more detail. The media gateway 304 includes a media gateway interface 402, a live event coordinator 404, a moderator 406, a syndicator 408, an event coordinator 410, a WebRTC 412, a VOD capture service 414, a VOD relay service 416, an authentication adapter 418, a metadata adapter 420, and a media store 422. The authentication adapter 418 and the metadata adapter 420 interface with the wider platform.

The WebRTC 412 provides only the means by which media is captured and distributed.

The event coordinator 410 organises this content into logical bundles by drawing on the metadata available to it through the metadata adapter 420. It is further assisted by the moderator 406 and live edit coordinator 404 which deliver the enabling features for the director, curator and moderator personae (see FIG. 2).

With reference to FIG. 6, there is illustrated that an event 602 is a collection of streams such as the four streams 604a to 604d. Streams are generated by capture devices. Certain events will have one or more output streams, and in FIG. 6 the event 602 is shown to have an output stream 606, if a director or editor is controlling the event. In general an event is a collection of one or more streams. Every stream is allocated to an event—if a new stream starts and there is no existing event for it to join, then a new event is created and that one stream becomes the first participant.

The “unit” the platform deals with is always the event. A viewer watches an event and, within that, watches one (or more) of the contributing streams. The viewer may choose which stream from an event to watch, or a director/editor might have chosen, or might have created an output stream by editing other streams together.

Streams and events have separate metadata. An event might be ‘Manchester United vs. Chelsea’ but the streams might be ‘Goal Camera 1, Home’, ‘Goal Camera 2, Away’, etc.

There are separate flows for editing event/stream metadata; merging events (e.g. two people create separate events for ‘Thomas’ Birthday Party’ but who then want to merge them into one); and marketplace content requests.

FIGS. 7 to 9 illustrate exemplary process with different appliances for different event service interactions. These illustrate examples of what occurs as a result of an appliance using the API to interact with the event service 412.

FIG. 7 illustrates an exemplary flowchart for creating a new stream, comprising steps 702 to 738.

FIG. 8 illustrates joining an existing event, comprising steps 802 to 818.

FIG. 9 illustrates creating an event, comprising steps 902 to 952.

Each step or box on each flowchart of FIGS. 7 to 9 is a simple question or a basic task. How a question is evaluated by the system or how a process is implemented is not detailed.

All the examples and embodiments described herein may be implemented as processed in software. When implemented as processes in software, the processes (or methods) may be provided as executable code which, when run on a device having computer capability, implements a process or method as described. The execute code may be stored on a computer device, or may be stored on a memory and may be connected to or downloaded to a computer device.

The invention has been described above with reference to particular examples. The invention is not limited to any particular example set out. In particular the invention may be implemented by using parts of any example described. The invention also may be implemented by combining different parts of any described example. The scope of the invention is set forth in the appended claims.

Claims

1. A system for providing a streaming services platform, the system comprising: a plurality of capture devices each comprising one or more applications and/or one or more products, each application and/or product for generating a media stream on a connection of a public network; anda server for receiving the generated media streams on the connection of the public network and for generating at least one output stream on a connection of the public network,the server configured such that each individual generated media stream is allocated to an event in accordance with its metadata, and the server configured to receive a request for a new event from an event owner, wherein in response to the request from the event owner: metadata for the new event is submitted to a grouping service;any existing streams overlapping with the new event are identified based on the metadata for the new event, wherein if any overlapping streams are identified, then the server is further configured to: retrieve a list of the overlapping streams;send the retrieved list of overlapping streams to the event owner of the new event;wait for the event owner of the new event to select the identified overlapping streams to invite; andsend an invitation to owners of the selected overlapping streams to join the new event, andany existing events overlapping with the new event are identified based on the metadata for the new event, wherein if any existing overlapping events are identified the server is configured to: retrieve a list of overlapping events;send the list of overlapping events to the event owner of the new event;wait for the event owner of the new event to select the identified overlapping streams to invite to merge; andsend an invitation to owners of the selected overlapping events to merge with the new event.

2. The system of claim 1 wherein the server includes: an aggregated API comprising an aggregation of services for each application and/or product of each of the plurality of capture devices, configured to: manage metadata of the platform and interface with a collection of services contributing to the aggregated API;to authenticate or authorize, or both authenticate and authorize, each input media stream;to provide functionality associated with streaming services; andto receive using requests for one or more streams such that the aggregated API is responsible for the management and control of services, features, and functionalities; anda media gateway configured to manage the flow of media from receipt of the input media streams, and to generate the at least one output stream, such that the media gateway is responsible for the flow of various data between the collection of services.

3. The system of claim 2 wherein the media gateway is configured to receive data, metadata, and data with embedded metadata.

4. The system of claim 3 further comprising an aggregated API, and wherein the media gateway is further configured to process one of streamed time dependent data and metadata and the API is configured to process control related data, derived from the metadata, which is not time dependent.

5. The system of claim 3 further comprising an aggregated API, and wherein a request from a user is received by the API, and the request defines rules which are applied to the media gateway and the individual generated media streams that come into it.

6. The system of claim 5 wherein rights holders can create rules that one or both of instruct the server to store content and assign rights to the individual generated media streams.

7. The system of claim 2, wherein the server is configured, on determination of provision of one of an invitation key and a password with the individual generated media stream, to join the individual generated media stream to an associated event.

8. The system of claim 1 wherein: if it is determined that an individual generated media stream is associated with an existing event, the individual generated media stream is allocated to the existing event, and the metadata for the existing event is updated, andif it is determined that the individual generated media stream is not associated with an existing event, then metadata for a new event is created and the individual generated media stream is associated with the new event such that the created new event comprises one generated media stream.

9. The system of claim 8 wherein allocating the individual generated media stream to an existing event is dependent upon initially identifying a plurality of potential events.

10. The system of claim 9 wherein the server is further configured to subsequently receive a selection from a user associated with the individual generated media stream allocated to one of the events, and join the allocated media stream to the event.

11. The system of claim 10 wherein if the user does not select an event, the server is further configured to subsequently create a new event.

12. The system of claim 8 wherein allocating the individual generated media stream to an event is dependent upon determining if the generated media stream is allowed to join the event.

13. The system of claim 12 wherein if the individual generated media stream is not allowed to join an event, the server is further configured to create a new event.

14. The system of claim 8 wherein if the individual generated media stream is added to an event, existing event members providing input streams to the event are notified.

15. The system of claim 1 wherein the server is further configured to determine of any of the overlapping events are rights holder events, and if so to filter those rights holder events and notify the owner associated with an overlapping stream that the event may be subject to rights restrictions, and add the new event to moderation queues for each possible rights holder event.

16. The system of claim 1 wherein the server is further configured to determine if the event owner is a rights holder, and if so: creating a moderation queue for the event; submitting the event to a metadata grouping service;and identifying any potentially infringing streams or events, wherein if potentially infringing streams or events are identified, then: notifying the owner associated with the potentially infringing streams or events that the content may be subject to rights restrictions; and adding the potentially infringing streams or events to the moderation queue.

17. The system of claim 1 wherein when the individual generated media stream includes a request to join an identified existing event, the server is configured to: retrieve details of the identified, existing event; anddetermine if the individual generated media stream is allowed to join the identified, existing event;responsive to the user not being allowed to join the identified, existing event, notify the use of rejection of the request to join the existing event;responsive to the user being allowed to join the identified, existing event:update the metadata of the identified, existing event by adding the individual generated media stream;notify the media gateway of the change to the identified, existing event;notify any members of the identified, existing event of the addition of the individual generated media stream; anddeliver the identified, existing event to members based on media streams allocated to that event, including the newly added individual generated media stream.

18. A method for providing a streaming services platform, the method comprising: generating a media stream of an event from each of a plurality of capture devices, each comprising one or more applications and/or one or more products, to a connection of a public network;receiving a plurality of the generated media streams at a server on connections of the public network and generating at least one output stream on a connection of the public network;receiving the at least one output stream on a connection of the public network;determining content for at least one output stream; andallocating each of the generated media streams to an event in accordance with metadata of the generated media streams, by: receiving a request for a new event from an event owner, wherein in response to the request from the event owner:submitting metadata for the new event to a grouping service;identifying any existing streams overlapping with the new event based on the metadata for the new event, wherein if any overlapping streams are identified, then the server is further configured to:retrieving a list of the overlapping streams;sending the retrieved list of overlapping streams to the event owner of the new event;waiting for the event owner of the new event to select the identified overlapping streams to invite; andsending an invitation to owners of the selected overlapping streams to join the new event, andidentifying any existing events overlapping with the new event based on the metadata for the new event, wherein if any existing overlapping events are identified the server is configured to: retrieve a list of overlapping events;send the list of overlapping events to the event owner of the new event;wait for the event owner of the new event to select the identified overlapping streams to invite to merge; andsend an invitation to owners of the selected overlapping events to merge with the new event.

19. The method of claim 18 wherein the each individual generated media stream is associated with a request to join an event, the method further comprising: providing metadata of the individual generated media stream to a grouping service;identifying any events to join;if any events to join are identified, then: retrieving a list of potential events to join;dispatching the list to a user associated with the individual generated media stream; andwaiting for the user to make a selection;wherein if the user makes a selection then running a process to join the individual generated media stream to the selected event and wherein if the user does not make a selection then creating a new event.

20. The method of claim 18 wherein the individual generated media stream is associated with a user request to join an identified event, the method comprising: retrieving details of the identified event; anddetermining if the user generating the individual generated media stream is allowed to join a group of users associated with the identified event, wherein if the user is not allowed to join the group of users associated with the identified event: notifying the user of rejection; creating a new event; updating the new event metadata by adding the individual generated media stream; and notifying the media gateway of the new event change.