Patent Application
20140337290
This application relates generally to cloud-based file storage.
More particularly, this application relates to the secure synchronization of files stored in a cloud-based file storage.
With the dramatic increase in use of mobile devices in recent years, it has become more important now than ever before that a user's files be synchronized between multiple devices. A single user may operate on a desktop computer, laptop computer, tablet computer, and mobile phone, editing the same document at different times on different devices. This issue is only going to become even more important as additional mobile devices, such as wearable computers and vehicle-based computers become popular mechanisms for editing files.
Problems, however, may be encountered as the synchronization of files across devices become more complex. For example, there is an increased likelihood of file-related events occurring concurrently, such as a user editing a document on both a mobile phone and a laptop computer while his desktop computer is offline.
The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
The description that follows includes illustrative systems, methods, techniques, instruction sequences, and machine-readable media (e.g., computing machine program products) that embody illustrative embodiments. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. It will be evident, however, to those skilled in the art that embodiments of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques have not been shown in detail.
In an example embodiment, various techniques are utilized in order to allow for efficient, accurate, and secure synchronization of files across multiple devices even in cases where a large number of events are occurring concurrently. This may include using an events-based technique to improve reliability during periods of file modifications being performed concurrently. In another example embodiment, name conflicts between different files may be resolved automatically.
Client device 104C may be a personal computer running a Windows™ operating system from Microsoft Corp. of Redmond, Wash. This may include a file system known as Windows Explorer 114, which may include a specialized plug-in 116 to allow Windows Explorer 114 to be altered in order to better operate with a proprietary application 118. Additionally, applications such as Microsoft Office™ 120 and Microsoft Outlook™ 122 may also include their own specialized plug-ins 124, 126, respectively, which allow them to better operate with the proprietary application 118. Client device 104D may be a web-based device that may operate a web browser 128 instead of a traditional operating system.
Each of the client devices 104A-104D may communicate with a back-end 130 hosted by the one or more servers 102. This communication may either take place directly between the back-end 130 and the proprietary applications 106, 112, 118, or indirectly through a web application 132. A provisioning service, such as HostPilot™ 134 may also be present, and may coordinate with a directory service such as Active Directories 136.
A central server 408 performs internal tasks such as cleanup, provisioning, and other backend services. A key management server 410 provides access for keys created for each user. When a user is created on the central server 408, a key may be issued by the key server 410. These keys are also used by an encryption server 412 to encrypt and decrypt files and folders. This is accomplished through the use of the front end server 400, which understands which organization the keys belong and submits file or folder content to the encryption server 412 for encryption. Active Directories 414 is used for integration with Exchange™
In an example embodiment, when a user saves some content by /some/path/to/file, the system may assume that it is a new version of file /some/path/to/file. All versions may be stored separately. Metadata stored for the content may allow the system to retrieve file content of past versions, as well as show various attributes of a file version, such as its author, when it was created, its size, etc. In this manner, the file system can act as a symbiosis of a file system, a source versioning system and a sharing service.
In an example embodiment, a specialized technique for determining a file name (as saved to disk) is utilized. This technique may be used anytime a file needs to be saved or retrieved. This technique involves calculating a hashsum of the file content, and then building a path using the first two characters of the hashsum as a parent folder and the next two characters of the hashsum as a subfolder. The entire hashsum may then be used as the file name within the subfolder. For example, a path may be built as: <root>/<1,2 chars of the hashsum>/<3,4 chars of the hashsum>/<hashsum> and save content there.
As an example, a particular file may have a hashsum of “1234c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7 852b855”. The file will be saved to <root>/12/34/1234c44298fc1c149afbf4c8996fb92427ae41e4649b93 4ca495991b7852b855
The hashsum created may also be referred to as a hashsum string. The values of particular characters of the hashsum string may be referred to as being in specific hashsum string character places (e.g., the first character of the string is in the first hashsum string character place).
This technique allows the system to scale a large number of files folded into a set of directories, and implement de-duplication on file level content.
In an example embodiment, every event that has happened to an object (file/folder/permission/etc.) can be stored. For example if a file was modified, or shared with a new user, or a folder deleted, these events can all be tracked. An API may then be provided for clients to retrieve events starting from a particular number. This allows for reliable and effective retrieval of event by clients. The user is able to easily see a list of all events for a particular object, as well as details about each event. In this way, an administrator, for example, may see everyone who accessed a particular file and made edits.
A file space may be defined as a set of files and folders owned by one user, along with links to folders shared by other users to this user. A revision may be defined as a natural number which describes current state of a file space. It is a counter of events happened in a file space.
The file space may be stored on a server, replicated to desktop applications, and can be accessed from mobile and web apps. File spaces can be—globally and uniquely identified by path that contains user id.
Each file space may have a number assigned to it, called a revision. In an example embodiment, the revision may be set to 1 when file space is first created, and is incremented on server every time file space changes (that is file, folder or permission is created, deleted or updated).
Once installed, a client can retrieve from the server the list of all events for all file spaces configured starting from 0.
In an example embodiment, when the client is started the first time, it does not retrieve a list of all events starting from revision 0, because there may be a huge number of events generated since the time the file space was created. In this case, the client can retrieve a list of all file space objects with their metadata. The server also returns the current file space revision to that request. After that initial request, the client periodically queries the server for new events as described above.
On the next attempt to update information the client would ask server to give all events starting from revision Rw, where Rw is the highest revision number (R) received from server on last update for a particular file space (w).
The item manager may store folder and file metadata from the local indexer 504 and the server indexer 512, calculate the state of file/folders, and asynchronously update a metadata database 516. The metadata database persistently stores local and server item metadata so that it would be available after the application is closed and reopened. Synchronization rules 518 are used to select files/folders that should be synchronized from the item manager 510 and creates tasks for 520 such synchronization. The tasks actually perform the file and folder synchronization, including, for example, uploading, downloading, creating, deleting, etc., resulting in synchronization between the local file system 508 and the storage service 514.
The revision is used by a client application to get file space metadata updates from the server. If client and server revisions are equal, then the client has the same metadata as the server for this file space. If the server revision is greater, then client requests a list of file space changes, starting from the revision that it knows. These changes are represented by events that contain updated metadata for files, folders and permissions.
The client application may store file space metadata in a local database, so that it would be available after application is restarted.
Using revisions in conjunction with storing file space metadata locally improves synchronization performance and reduces network traffic, because, after initial synchronization, the client application queries the server only for changes in file space, and not for the whole file space metadata.
Items (files and folders), can have several states. When an item is in the synced state, it means that this item is exactly the same on the server and on the client, and no further actions are needed for this item.
The client application contains rules that define what actions are needed to bring every item in the file space into synced state, based on its current conditions. In an example embodiment, these rules may include the following:
A file version conflict may occur in the following example situations (these examples being non-exhaustive):
In these cases, in an example embodiment, the local file is copied to the conflict file and then the local file is overwritten with the server version.
The conflict file is created in the same folder as the original file and has the name <original file name>(Conflicted copy <YYYY-MM-DD>—for <user name>).<original file extension>(e.g. “document(Conflicted copy 2013-04-04—for Nikita Uraltsev).docx”)
As described earlier, name conflicts can occur in various cases, including, for example:
Name conflicts can be divided into two categories—server-side name conflicts and client-side name conflicts.
A server-side name conflict occurs in the following cases:
Case 1: The user has a file or folder named F that is located in the user's root folder on the server. Another user shares a folder named F with the first user.
Case 2: The user has a temporarily deleted file or folder named F that is located in the user's root folder on the server. Another user shares a folder named F with the first user.
Case 3: The user has a link named F that points to a folder shared by another user. Another user shares a different folder named F with the first user.
A client-side conflict occurs in the following cases:
Case 4: A local file has the same name as a server folder or vice versa.
Case 5: A local file or folder has the same name as a link to a shared folder on the server.
It should be noted that in some embodiments the desktop application ensures that the local folder and shared folder are not merged. This prevents confidential data from being shared with the wrong people, even accidentally.
At operation 1406, an event may be generated for the first modification. The event may contain various pieces of metadata. In one example embodiment, the event contains enough information so that a system could use to the event to update an older version of the first file in the same way the first modification does. For example, the event may contain the terms, phrases, sentences, formatting, etc. that has changed due to the first modification, allowing another system to replicate the same changes on their version of the first file. At operation 1408, the event may be sent to a server. Operations 1402-1408 may be repeated each time a modification occurs. Notably, while not pictured, the server, upon receipt of the event, may increment a current revision number for the file space. This may be done each time an event is received from any of the client devices.
At operation 1410, a current revision number of the file space may be obtained from the server. At operation 1412, the current revision number is compared to a past revision number corresponding to a last time files were synchronized from the server to the first client device. At operation 1414, in response to a determination that the current revision number is higher than the past revision number, a number of events that have occurred to the file space may be requested from the sever, the number of events corresponding to a difference between the current revision number and the past revision number. At operation 1416, the events may be received. At operation 1418, the first instance of the file space may be updated based on the events.
Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules can constitute either software modules (e.g., code embodied (1) on a non-transitory machine-readable medium or (2) in a transmission signal) or hardware-implemented modules. A hardware-implemented module is tangible unit capable of performing certain operations and can be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., a standalone, client or server computer system) or one or more processors can be configured by software (e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations as described herein.
In various embodiments, a hardware-implemented module can be implemented mechanically or electronically. For example, a hardware-implemented module can comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module can also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware-implemented module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) can be driven by cost and time considerations.
Accordingly, the term “hardware-implemented module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily or transitorily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. Considering embodiments in which hardware-implemented modules are temporarily configured (e.g., programmed), each of the hardware-implemented modules need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules comprise a general-purpose processor configured using software, the general-purpose processor can be configured as respective different hardware-implemented modules at different times. Software can accordingly configure a processor, for example, to constitute a particular hardware-implemented module at one instance of time and to constitute a different hardware-implemented module at a different instance of time.
Hardware-implemented modules can provide information to, and receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules can be regarded as being communicatively coupled. Where multiple such hardware-implemented modules exist contemporaneously, communications can be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware-implemented modules. In embodiments in which multiple hardware-implemented modules are configured or instantiated at different times, communications between such hardware-implemented modules can be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware-implemented modules have access. For example, one hardware-implemented module can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware-implemented module can then, at a later time, access the memory device to retrieve and process the stored output. Hardware-implemented modules can also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
The various operations of example methods described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein can, in some example embodiments, comprise processor-implemented modules.
Similarly, the methods described herein can be at least partially processor-implemented. For example, at least some of the operations of a method can be performed by one of processors or processor-implemented modules. The performance of certain of the operations can be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processor or processors can be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other embodiments the processors can be distributed across a number of locations.
The one or more processors can also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application program interfaces (APIs).)
Example embodiments can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Example embodiments can be implemented using a computer program product, e.g., a computer program tangibly embodied in an information carrier, e.g., in a machine-readable medium for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers.
A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.
In example embodiments, operations can be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Method operations can also be performed by, and apparatus of example embodiments can be implemented as, special purpose logic circuitry, e.g., a FPGA or an ASIC.
The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In embodiments deploying a programmable computing system, it will be appreciated that both hardware and software architectures require consideration. Specifically, it will be appreciated that the choice of whether to implement certain functionality in permanently configured hardware (e.g., an ASIC), in temporarily configured hardware (e.g., a combination of software and a programmable processor), or a combination of permanently and temporarily configured hardware can be a design choice. Below are set out hardware (e.g., machine) and software architectures that can be deployed, in various example embodiments.
The example computer system 1600 includes a processor 1602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory 1604 and a static memory 1606, which communicate with each other via a bus 1608. The computer system 1600 can further include a video display unit 1610 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 1600 also includes an alpha-numeric input device 1612 (e.g., a keyboard or a touch-sensitive display screen), a user interface (UI) navigation device 1614 (e.g., a mouse), a disk drive unit 1616, a signal generation device 1618 (e.g., a speaker), and a network interface device 1620.
The disk drive unit 1616 includes a machine-readable medium 1622 on which is stored one or more sets of instructions and data structures (e.g., software) 1624 embodying or utilized by any one or more of the methodologies or functions described herein. The instructions 1624 can also reside, completely or at least partially, within the main memory 1604 and/or within the processor 1602 during execution thereof by the computer system 1600, with the main memory 1604 and the processor 1602 also constituting machine-readable media 1622.
While the machine-readable medium 1622 is shown in an example embodiment to be a single medium, the term “machine-readable medium” can include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more instructions or data structures 1624. The term “machine-readable medium” shall also be taken to include any tangible medium that is capable of storing, encoding or carrying instructions 1624 for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure, or that is capable of storing, encoding or carrying data structures utilized by or associated with such instructions 1624. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media. Specific examples of machine-readable media 1622 include non-volatile memory, including by way of example semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
The instructions 1624 can further be transmitted or received over a communications network 1626 using a transmission medium. The instructions 1624 can be transmitted using the network interface device 1620 and any one of a number of well-known transfer protocols (e.g., HTTP). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, mobile telephone networks, plain old telephone (POTS) networks, and wireless data networks (e.g., WiFi and WiMax networks). The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying instructions 1624 for execution by the machine, and includes digital or analog communications signals or other intangible media to facilitate communication of such software.
Although an embodiment has been described with reference to specific example embodiments, it will be evident that various modifications and changes can be made to these embodiments without departing from the broader spirit and scope of the disclosure. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. The accompanying drawings that form a part hereof, show by way of illustration, and not of limitation, specific embodiments in which the subject matter can be practiced. The embodiments illustrated are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments can be utilized and derived therefrom, such that structural and logical substitutions and changes can be made without departing from the scope of this disclosure. This Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.
Such embodiments of the inventive subject matter can be referred to herein, individually and/or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Thus, although specific embodiments have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose can be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description.
This application claims priority to U.S. Provisional Application No. 61/820,793 filed May 8, 2013, which is hereby incorporated herein by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61820793 | May 2013 | US |
