Patent Application
20170192855
This application claims the benefit, under 35 U.S.C. §365 of European Application 15307176.6, filed on Dec. 30, 2015, herein incorporated by reference.
The present disclosure relates to the field of automatic selection of contents thanks to an electronic device, and notably to the field of automatic copy operations related to audiovisual content.
A method for selecting a content comprising audiovisual data and corresponding electronic device, system, computer readable program product and computer readable storage medium are described.
Electronic devices comprising audio and/or video capabilities are widely used today. Those electronic devices permit a user to acquire new audiovisual contents (for instance pictures, video, music), thanks either to their input/output interfaces and/or to their communication interfaces. However, those audiovisual contents stored in the electronic device are often numerous and very greedy in terms of memory. Furthermore, they can be lost in case of a failure, a loss or a stealing of the electronic device. Such a loss can be felt as very prejudicial by a user. These contents can also require large communication resource and bandwidth when the user of the electronic device decides to acquire them from a remote electronic device or to share them with a remote electronic device (like a remote backup server or an electronic device of another user).
Such, it is at the same time very important for a user to backup or share his audiovisual content and also to limit the backup or sharing operation (in terms of size or time notably).
Some solutions of the prior art have proposed to perform selective backup operation by using exclusion or inclusion rules related to particular type of contents or to location of contents in order to optimize the backup operation.
However, the known solutions do not fully meet user's expectations.
The present principles overcome at least one of the above mentioned disadvantages by proposing a method, adapted to be implemented in an electronic device, for selecting at least one content, during a backup operation, amongst a plurality of contents coding audiovisual data and being accessible by said electronic device.
According to at least one embodiment of the present disclosure, said method comprises:
Terms “performing a Backup operation”, or “backing up a content”, are to be understood in the present disclosure as a processing including duplicating, or copying, at least partially, the content (for saving or sharing notably) and/or eventually some associated items (like associated metadata or descriptors). The backup can be performed locally, (the content and its duplicated content being both located on the same device, for instance the electronic device), by at least one upload operation (to at least one destination device, being the electronic device and/or at least one different device) and/or by at least one download operation from at least one source device, being the electronic device and/or at least one different device. According to embodiments, a same content can be duplicated on several destination devices or on a unique destination device or the duplication can be split amongst different devices.
According to at least one embodiment of the present disclosure, said at least one backup rule comprises at least one criterion belonging to a group comprising:
According to at least one embodiment, said group further comprises:
According to at least one embodiment of the present disclosure, said backup rule is acquired from a user interface of said electronic device.
According to at least one embodiment of the present disclosure, said at least one audiovisual descriptor relates to at least one element belonging to a group comprising:
According to at least one embodiment of the present disclosure, said method further comprises obtaining at least one contextual descriptor of at least one electronic device involved in said backup operation and said backup rule takes into account said contextual descriptor of said involved electronic device.
According to at least one embodiment of the present disclosure, obtaining said audiovisual descriptor of said accessible content comprises assessing similarities between said accessible content and at least another accessible audiovisual content.
According to at least one embodiment of the present disclosure, said method comprises copying said selected content from at least one source device from which said selected content is at least partially accessible.
According to at least one embodiment of the present disclosure, said method comprises backing up at least partially said selected content on at least one destination device.
According to another aspect, the present disclosure relates to an electronic device comprising at least one memory and at least one processor adapted for selecting at least one content, during a backup operation, amongst a plurality of contents coding audiovisual data and being accessible from said electronic device.
According to at least one embodiment of the present disclosure, at least one of said processor is adapted for:
The electronic device of the present disclosure can be configured to perform the method for selecting of the present disclosure in any of its embodiments.
According to at least one embodiment of the present disclosure, at least one of said processor is further adapted for obtaining at least one contextual descriptor of at least one electronic device involved in said backup operation and said backup rule takes into account said contextual descriptor of said involved electronic device.
According to at least one embodiment of the present disclosure, said involved electronic device is said electronic device itself, a source device from which said content is accessible to said electronic device, and/or a destination device where said content is to be backed up during said backup operation.
According to at least one embodiment of the present disclosure, said electronic device is distinct from said source and/or said destination device. According to at least one embodiment of the present disclosure, said at least one backup rule comprises at least one criterion belonging to a group comprising:
According to at least one embodiment of the present disclosure, said backup rule is acquired from a user interface of said electronic device.
According to at least one embodiment of the present disclosure, said at least one audiovisual descriptor relates to at least one element belonging to a group comprising:
According to at least one embodiment of the present disclosure, said at least one processor is adapted for copying said selected content from at least one source device from which said selected content is accessible.
According to at least one embodiment of the present disclosure, said at least one processor is adapted for backing up said selected content on at least one destination device.
According to another aspect, the present disclosure relates to a system comprising:
According to at least one embodiment of the present disclosure, at least one of said processor of said electronic device is adapted for:
According to at least one embodiment of the present disclosure, said system further comprises at least one destination device where at least one of said selected content is to be at least partially backup during said backup operation.
The system of the present disclosure can be configured to perform the selecting method of the present disclosure in any of its embodiments.
The present embodiments related to the method or to the corresponding electronic device and/or system can be employed in any combination or sub-combination. For example, some embodiments of the selecting method can involve obtaining three audiovisual descriptors, describing respectively a presence of at least one known face in said accessible content, a presence of at least one known place in said accessible content and a similarity between said accessible content and another of said accessible contents, obtaining a contextual descriptor of at least the electronic (describing for instance a current location of the electronic device) and said backup rule takes into account all those descriptors.
According to another aspect, the present disclosure relates to a non-transitory program storage device, readable by a computer.
According to an embodiment of the present disclosure, said non-transitory computer readable storage device tangibly embodies a program of instructions executable by a computer to perform, in an electronic device, the selecting method of the present disclosure in any of its embodiments.
According to an embodiment of the present disclosure, said program of instructions is adapted to perform a selecting of at least one content, during a backup operation, amongst a plurality of contents coding audiovisual data and accessible by said electronic device.
According to at least one embodiment of the present disclosure, the program of instructions is adapted to perform:
According to another aspect, the present disclosure relates to a computer readable storage medium carrying a software program comprising program code instructions for performing, in an electronic device, the selecting method of the present disclosure, in any of its embodiments, when said software program is executed by a computer.
According to an embodiment of the present disclosure, said computer readable storage medium carries a software program comprising program code instructions for performing a selecting of at least one content, during a backup operation, amongst a plurality of contents coding audiovisual data and accessible by said electronic device.
According to at least one embodiment of the present disclosure, said program code instructions is adapted for performing:
The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description, the description making reference to the annexed drawings wherein:
It is to be noted that the drawings have only an illustration purpose and that the embodiments of the present disclosure are not limited to the illustrated embodiments.
As illustrated by
The audiovisual content can be stored in a memory of the electronic device 100, for instance a non-volatile memory like a flash memory or a Non Volatile Random Access Memory (NVRAM) database, and/or in a hard disk and/or in a removable storage unit (like a USB key) of the electronic device. The audiovisual content can also be stored on another communication device (111, 112, 114, 120, 122) accessible from a communication interface of the electronic device 100, like a local or remote server, notably a cloud server.
The electronic device 100 can be for instance a mobile terminal (notably a wireless device), like a smartphone, a tablet, a personal computer, a connected device. It can also be a wired device, like a personal computer, or a private or shared server.
According to at least some embodiments of the present disclosure, the contents to be backed up during a backup operation are selected according to at least one backup rule. This backup rule is defined by at least one criterion related notably to the specificities of the audiovisual data coded by the content (like in particular to audiovisual feature extracted from the audiovisual data).
Indeed, contrary to known backup, versioning or sharing solutions, which only consider a document as an undefined object to be synchronized or backed up, at least one embodiment of the present disclosure takes into account the nature and data contained in the document to be backed up.
According to
In the embodiment illustrated by
The electronic device 100 can also selectively recover contents from those devices.
Some use cases can relate for instance to:
According to the present disclosure, the content backed up during a backup operation can be selected according to at least one backup rule comprising at least one criterion related to some audiovisual descriptors extracted from feature descriptors related to audiovisual data coded by the audiovisual content. A backup rule can further take into account a context of the electronic device, or of another device implied in the backup operation like a source or destination device.
For instance, as illustrated in
In the particular embodiment of
In the illustrated embodiment, the acquiring can be performed during an initialization phase, for instance at a launching (notably the first launching) of an application that performs the selecting method of the present disclosure. In some embodiments, the acquiring can also be performed at the moment of the backup operation. For instance, according to
This acquiring is optional. Indeed, in some embodiments, at least one rule can have been obtained previously. For instance, a backup rule can have been acquired during a former launch of the application implementing the method of the present disclosure. In other embodiments, no acquiring of a backup rule is needed. For instance, a backup rule can be contained in the software code of the application implementing the method of the present disclosure. In some embodiments, at least one backup rule can be obtained from a configuration file present in the electronic device (for instance a downloaded configuration file).
In some embodiment, a backup rule can be acquired from a user interface of the electronic device. Such an embodiment can permit to get a personalized matching rule, that can be more fitted to the needs or willingness of a user of the electronic device (or to a third person acting as a supervisor of the electronic device—like a parent when the user of the electronic device is a child).
In some embodiment, a backup rule can be acquired thanks to a communication interface of the electronic device from a remote server. The acquiring can be performed for instance at the moment of a download of a software application to be launched on the electronic device (notably an application implementing the selecting method of the present disclosure), or during a downloading of a new version of the operating system of the electronic device, or at the moment of the backup. Such embodiment can permit to acquire a default backup rule when no backup rule exists on the electronic device. It can also permit, for a plurality of electronic devices (owned by a same company for instance) to get a same backup rule (for instance in order to comply with a security policy within a company).
The backup rule can be either an exclusion rule (defined by at least one condition, or a combination of conditions, to be realized for a content to be excluded from a backup operation) or an inclusion rule (defined by at least one condition, or a combination of conditions, to be realized for a content to be included in a backup operation).
Such a condition related to a content and used for defining a backup rule can be for instance a maximum size of a content or a type of extension of a file name of the content. It can also relate to an audiovisual descriptor derived from at least one characteristic of the data coded by the content. Examples of coded data characteristics can include an estimation of a nature of the data, or of a perceived quality of the content (for a user point of view) like, for a visual content, a quality indicator related to blur and/or to a visual contrast, or to a level of noise for an audio content, or the presence in a content of a required or forbidden element (like faces and/or naked character in a visual content or voice or sound of a particular music instrument in an audio content). Detecting a naked character can notably be based on an analyzing of color histograms of visual contents and/or on a comparing with a reference annotated data set (obtained during a preliminary training phase for instance).
A backup rule can further be defined by some other conditions. Some conditions can relate to the context of the electronic device (like a minimum or maximum level of available storage, a location of the electronic device, a connection to a particular network, like a private, trusted network).
Some other conditions can relate to a particular repository or to a specific source device where stored contents are to be excluded from, or included in the backup. They can also relate to a time schedule, a backup periodicity . . . .
The backup rule can further be defined by complementary requirements, specifying for instance a destination repository or a destination device (like a NAS server, a home server or a cloud server, like a server of a social network) where the backup contents are to be sent and/or saved.
Some conditions can also relate to a minimum bandwidth available. In embodiments where an audiovisual content can be available in several formats, or versions, some conditions can also describe a particular format or version to be selected, or a maximum number of versions of a same content to be backup. For instance, a backup rule can limit backup of a video content to its key frames when the available bandwidth is under a given threshold.
According to another embodiment, where the computing 240 of at least one audiovisual descriptor comprises computing a difference between successive frames (also called successive frames distance) of an audio or video content, a backup rule can limit a backup of an audiovisual content to parts of content where the distance between consecutive frames is higher than a given threshold, or (in case of a video content) to still image extracted from the video content when the distance between successive frames is lower than a given threshold and almost constant.
Some conditions of a backup rule can also be related to available resources (like a minimum batteries level, or a minimum availability of processing resources), of at least one of the source devices where some contents are stored, of the electronic device itself, and/or of at least one of the destination devices where some backed up contents will be stored.
Some conditions can notably permit to take into account privacy of contents. For instance, an exclusion rule can be defined by a condition about content comprising face or naked character (for content comprising visual data, like still images or a video contents).
In some embodiment, where the electronic device comprises a face recognition module, a condition can be expressed for a visual content as a result of a comparison between the visual content and a reference content.
Such an embodiment can permit a systematic backup of images comprising faces of known characters, registered on the electronic device (like members of the family of the user) or a systematic exclusion of images comprising faces of known characters (for a destination server being a social network server for instance).
Similarly, visual contents comprising background scenes identified as known places (either landscape or place registered in the electronic device, or famous place identified thanks to a web server) can be either excluded or included by a backup rule.
Depending upon embodiments, one or several backup rules can be acquired.
For instance, a user of the electronic device can enter several matching rules, depending on the destination device, like a default profile, a Facebook profile, a Home (or NAS) profile, a Dropbox profile, and/or a Mobility profile.
For embodiments where several backup rules are defined, the method can comprise checking the conditions of appliance of each matching rule, in order to detect inconsistencies between several rules. Such an inconsistency can exist for instance when an inclusion rule and an exclusion rule can both apply to a particular content. This checking can for instance be performed at the moment of the acquiring of a backup rule, at the moment of the backup operation (for instance during the determining of a backup rule to be applied), or periodically.
In some embodiments, such inconsistencies can be avoided by a mandatory presence in each backup rule of a condition relating to a particular variable and a limitation to a unique backup rule for each particular value of the particular variable. For instance, in some embodiments, where a condition must relate to a value of a location descriptor of the electronic device (that can take one of the following values “Not connected to a communication network”, “Connected to a WAN”, “Connected to a guest network” or “Connected to my home network”), only one backup rule can be defined for each value of the location descriptor. Thus, only one backup rule will be applied when the electronic device is located in the home network of the user for instance. Of course, in such an embodiment, backup rules can be more or less restrictive depending on the required value of the location descriptor.
According to
According to
The extracted features can be diverse. They can relate to particular fragments of an image or to global characteristics (such as compression type, pixel resolution, memory size . . . ).
The method also comprises computing 240 audiovisual descriptors from the extracted audiovisual features. In case of image or video data, the audiovisual descriptors can be for instance global descriptors such as distributions of intensities, colors or local patterns. They can also be related to a distance (or difference) between frames of a content comprising audio or video data (like a geometric, photometric and/or colorimetric distance between successive images). They can also describe properties relating to interestingness (like valence or arousal attributes), detected faces or perceptual characteristics of the coded data.
The audiovisual descriptors can be computed by different ways depending upon embodiments, notably by algorithms known by the one skilled in the art in the field of image processing, like a Scale-Invariant feature transform (SIFT) for image data, a Mel-Frequency Cepstral Coefficients (MFCC) transform for an audio content, spatio-temporal features for video content, and so on.
In some embodiments, the extracting 230 and/or the computing 240 of an audiovisual content can be performed on the fly, just after the obtaining 220 of the audiovisual content for instance. In other embodiments, the extracting 230 and/or the computing 240 of an audiovisual content can be performed after a user request. In still other embodiments, the extracting 230 and/or the computing 240 can be performed on batch of newly obtained contents, for instance periodically or when a certain amount of contents is to be analyzed. They can also be performed during moments where the electronic device is not in used, or by a software task with a very low priority. Such an embodiment can permit for the device to spread the use of necessary resources (in terms of time, bandwidth, and/or processing capabilities) and thus to avoid a detectable charge (or a lack of availability) of the electronic device, for a user point of view.
For instance, the extracting 230 and/or computing 240 can be performed for each audiovisual content received, created and/or modified since the last backup performed, and/or for each audiovisual content stored in a given repository.
The method can further comprise storing the extracted features and/or the computed descriptors, in association with the obtained contents.
In still other embodiments, the extracting 230 and/or computing 240 can be performed at the moment of the backup operation. Notably, in an embodiment where the extracting and/or the computing are performed after the determining of at least one backup rule to be applied for the selection of contents to be backed up, the extracting and/or the computing can be performed according to the determined backup rule to be applied.
According to
The backup order can also be generated automatically by the electronic device itself. For instance, a backup order can be generated periodically, or when a certain amount of contents has been obtained. A backup order can also be generated during moments where the electronic device is not in use, or by a software task with a very low priority.
In some embodiments, the backup order can also be generated automatically according or the location of the electronic device, or the location of at least one other device implied in the backup operation.
For instance, a backup order can be generated automatically when the electronic device detects its connection to a trusted network, and/or to the home network of the user of the electronic device. Such a detection can be performed thanks to a positioning sensor, like a Global Positioning System Module (GSM), of the electronic device and/or thanks to an analyzing of some network characteristic obtained from a communication interface of the electronic device. In a variant, the backup order can be generated automatically at a given time schedule if the electronic device is still connected to the trusted network, and/or to the home network.
According to
According to
According to
In the illustrated embodiment, the selecting 260 also comprises determining 266 at least one backup rule to be applied. For instance, the determining 266 can comprise accessing a configuration file where the rules to be applied are defined (notably previously acquired rules). In some embodiment, where several rules are defined, this determining can comprise receiving a confirmation information from a user interface and/or a communication interface of said electronic device. In other embodiment, this receiving of a confirmation information can be optional.
The selecting 260 also comprise the applying 268 of at least one determined backup rule.
It is to be noted that in some embodiments, the determining and the applying can be merged (each determined rule being applied immediately for instance, before the determining of another rule).
The obtaining 262 of contextual descriptors of devices, the computing 240 of audiovisual descriptors of contents and the identifying 264 of near duplicates have been illustrated by
As already explained, in some embodiments, the method can also comprise checking and resolving inconsistencies between determined/applied rules. Notably, the method can comprise generating an alert and/or requesting (not illustrated) a confirmation, on a user interface and/or a communication interface of the electronic device, in order to put in evidence and/or solve inconsistencies.
According to
Depending upon embodiments, the selected content can be stored and/or sent in one or several versions, notably in a compressed version, and/or in one or several formats.
In some embodiment, the backing up can comprise testing the presence of a version of a selected content in the destination device. In such an embodiment, the actual saving and/or sharing of the content can only be performed conditionally, according to the test result and/or to at least one characteristic of the version present on the destination server. For instance, the actual saving and/or sharing can be performed only when the version present on the destination server is more recent, has a different size, different audiovisual descriptors and/or different metadata than the version present on the source device.
In some embodiment, a content can be backed up entirely. In other embodiments, a content can be backed up partially. For instance, only video key frames or audio part of an audiovisual content can be backed up. In other embodiments, a content can be backed up either entirely or partially, depending upon the achievement of at least one criterion (related for instance to the content size, a time interval or a number of changes made on the content since the last backup operation involving a backup of this content, . . . ).
In some embodiment, the backup can comprise a backing up of at least some metadata related to the content, and/or the obtained audiovisual descriptors of the content. In other embodiment, the backup of metadata and/or audiovisual descriptor can be omitted or metadata and/or audiovisual descriptor can be backed up without the corresponding contents.
Depending upon embodiments, a content can be backup entirely in a unique server, on two or several servers, or backed up partially on different servers. At the latter case, the part being backed up on the different servers can form a partition of the content.
In an embodiment, a backing up of a content in a destination server that also implements the selecting method of the present disclosure, in a similar or different embodiment, can further imply a backing up of the saved or shared content on other destination device (being either a server or a client device) according to the method of the present disclosure. Such an embodiment can permit for instance to a set of users to selectively share audiovisual contents via a common server, each user applying its own backup rule.
In an exemplary use case, the electronic device can be a smart phone, used to take pictures on a beach during family holydays. Pictures have been taken in different formats (raw, jpeg), with different compression rates and in different exposures. Some pictures are nice and other ones are blurred. Some of them represent some family members, and include face of a child, or the user's wife wearing a bikini.
The user of the smart phone wants to share some of the pictures taken at the beach with other members of the family or with his contacts on a social network account. The bandwidth is limited.
The user can define, or have previously defined, some backup rules about its holiday pictures. Notably some backup rules can be adapted to limit the backup to selected pictures having the best quality (not blurred, right exposure), a format and/or file size adapted to the limited bandwidth available, and/or respecting some privacy constraints. Some selection criterion of the backup rule can also relate to the automatic selection of a subset of non-duplicated, or near duplicated, contents.
For instance, a first rule (named “sharing with relatives”) can be defined, with the purpose of a selective sharing of contents with other members of the family (for instance via a private social network, or a private group of a public social network). The first rule can limit the sharing to the pictures representing the family members and being of good quality. The criteria of the first rule can for instance:
A second backup rule (named “sharing with social contacts”) can also be defined for the sharing of pictures with contacts of a given public social network. Only small size pictures can be included in order to permit a quick upload. The criteria of the second rule can for instance:
A third and fourth backup rule can be defined for a backup on a private space of a cloud storage.
The criteria of the third rule (named “backing up my picture in mobility”) can be for instance
The criteria of the fourth rule (named “backing up my picture at home”) can be for instance:
If a backup order is obtained from the user interface of the smart phone, when the family is on the beach, determining a backup rule can comprise requesting a confirmation of the user about the applicability of either the first, second, third and/or fourth rule and the applying of the determined rule for selecting contents to be backed up. The content selected will differ upon the applied rule. Of course if the user selects the fourth rule, no content will be selected and an alert can be generated on the user interface of the electronic device. If a backup order is generated automatically when the smartphone detects that its GPS location is the one of the family home, the determined and applied rule can be the fourth rule. In a variant, one of the backup rule can further comprise a condition relating to an indicator limiting the backing up to the audiovisual descriptors of contents. Such an embodiment can be well adapted for indexing or searching of contents, on a destination device, similar to given audiovisual contents of a source device.
In the particular embodiment of
In some embodiments, the electronic device 30 can also include, or be connected to, a display module 33, for instance a screen, directly connected to the graphics card 32 by a dedicated bus 330.
In the illustrated embodiment, the electronic device 30 also comprises a GPS sensor, adapted to determine the geographic location of the electronic device.
Each of the mentioned memories can include at least one register, that is to say a memory zone of low capacity (a few binary data) or high capacity (with a capability of storage of an entire audio and/or video file notably).
When the electronic device 30 is powered on, the microprocessor 31 loads the program instructions 360 in a register of the RAM 36, notably the program instruction needed for performing at least one embodiment of the selecting method described herein, and executes the program instructions.
According to a variant, the electronic device 30 includes several microprocessors. According to another variant, the power supply 39 is external to the electronic device 30.
In the particular embodiment illustrated in
According to at least one embodiment, the at least one processor is adapted for:
As will be appreciated by one skilled in the art, aspects of the present principles can be embodied as a system, method, or computer readable medium. Accordingly, aspects of the present disclosure can take the form of an hardware embodiment, a software embodiment (including firmware, resident software, micro-code, and so forth), or an embodiment combining software and hardware aspects that can all generally be referred to herein as a “circuit”, module” or “system”. Furthermore, aspects of the present principles can take the form of a computer readable storage medium. Any combination of one or more computer readable storage medium(s) may be utilized.
A computer readable storage medium can take the form of a computer readable program product embodied in one or more computer readable medium(s) and having computer readable program code embodied thereon that is executable by a computer. A computer readable storage medium as used herein is considered a non-transitory storage medium given the inherent capability to store the information therein as well as the inherent capability to provide retrieval of the information therefrom. A computer readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
It is to be appreciated that the following, while providing more specific examples of computer readable storage mediums to which the present principles can be applied, is merely an illustrative and not exhaustive listing as is readily appreciated by one of ordinary skilled in the art: a portable computer diskette, a hard disk, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative system components and/or circuitry of some embodiments of the present principles. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable storage media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
| Number | Date | Country | Kind |
|---|---|---|---|
| 15307176.6 | Dec 2015 | EP | regional |
