Patent Application
20200241994
None.
Not Applicable.
Not Applicable.
Not Applicable.
Not Applicable.
The disclosure relates to optimization of a computer system, specifically, reduction of latency in globally distributed file and processor resources.
As is known, globally distributed Information Technology resources are utilized in different ways as workflow follows the sun. Rather than hand offs among localized industrial-type day shifts, swing shifts, and night shifts, intellectual property teams which span meridians in longitude to continuously deploy a plurality of “fresh eyes” to optimize each critical path in project management.
As is known, complex workflows utilize workspaces which contain related files. Various means such as but not limited to version tracking enable recordation of meta data about file components. For the purpose of this application we will refer to file extents but our meaning is inclusive of but not limited to blocks, bytes, records, binary images, contents, segments, sectors, cylinders, compressed and uncompressed strings, encrypted and unencrypted digital values encoded onto computer readable media and suitable for data transmission, storage, and hashing as exemplary file extents. The subject matter applies to any file system or workspace which has relatively more invariant and relatively less invariant binary objects under its measurement and control such as version-controlled source code.
What is needed is a system to track, control, forecast, and anticipate the compute resources necessary in each zone of a global workflow to reduce latency, data transmission, and to optimize the performance of a networked computer system and its users.
The invention includes a system having a plurality of regional Peer Managers coupled to a Tracker which records the existence and location of Workspaces that contain related files having invariant components created by various means and further coupled to a Virtual Dataflow Aggregator (VDA). Each Peer Manager provisions compute and storage resources to users located within a region or timezone. Global dataflow supports a 24×7 workflow through a succession of Peer Managers. File operation requests and fulfillments are reported by each regional Peer Manager over a work day cycle.
Instead of staffing a day shift, a night shift, and a swing shift, the invention progressively resources the work product to follow the sun across a globally distributed team all operating on their optimum “day” cycle time zones. The system measures files and their constituent parts across a scale of Invariance Quality in a series of workflows.
The VDA converts file operation reports into a common format and determines patterns which are likely to repeat either preceding or succeeding data flows among the regional Peer Managers.
Conventional systems cannot record the Invariance Quality (IQ) of sub-elements of files. On a regular cycle, a hypothetical profile is constructed for the following day's workfile flow. The profile is scored for accuracy at each meridian's “end of day” and IQ's are incrementally refined.
Dataflow Meteorology Aggregation monitors data deployment across a global 24×7 workflow. File operation metrics are aggregated at regional server peers over a daily cycle of computation. The system forecasts demand for resources and stages prepositioned virtual machines in anticipation of demand to ameliorate latency. Patterns of demand for dynamic or invariant data among regional timezones over the course of a workday are profiled. A data centric workflow is optimized by tracking file operations during a 24-hour global computation cycle. Each file operation is monitored with location, file identifier, and location by date-time. Critical paths through the workflow are traced and bottlenecks identified. On a work day cycle, processor images and file contents are pre-positioned for anticipated demands to reduce searching for and transmission of portions of data sets. Peer delegated subspace servers are assigned resources and assigned to network node where they can perform with least latency.
Analogous to a weather map, hot spots of file extent mobility, variance, and invariance are tracked and forecasted in their progress from East to West. Resources are then pre-positioned. Heuristics that mis-position resources are “punished” by losing influence in future.
The foregoing and other objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:
The patentable subject matter of the application applies to apparatus and methods which optimize the performance of a computer system distributed globally with time-shifting of workfile flows to match local time of day and day of week project prosecution.
Another aspect of the invention is shown in
Another aspect of the invention is illustrated in
In
In an embodiment, the method also includes anticipating local file operations that precede or succeed regional data flows 540; packaging virtual machine images appropriate for each data flow arrival 560; and causing each peer manager to stage for each daily arrival or transfer of variants 580.
In an embodiment illustrated in
Aspects of the invention can be appreciated as methods, apparatuses, and systems combining such methods and apparatuses.
For example, an application may write out intermediate results and logs for use in problem identification when the application abnormally fails. An additional script may eliminate these breadcrumbs when the desired result is obtained. So these files are not useful for furthering the project except when testing changes to the methodology. These files are frequently opened and closed during a development cycle and seldom consulted and frequently deleted in production. Write once, read hardly ever.
The invention can easily be distinguished from conventional Most Recently Used (MRU) or Least Recently Used (LRU) wives tales on system performance optimization. Systems that optimize to MRU or LRU goals are belief driven rather than data driven. Conventional systems fail to disclose process flow of file operation messages to optimize peer configuration and required resources. Peers represent trackers and regional peer managers. Peers send messages to a Virtual Dataflow Aggregator Apparatus (VDA). Message represents each file operation and contains timestamp, peer interactions IDs, number of bytes in operation (in case of read/write operations), etc. All messages are sent to VDA. In an embodiment, multiple VDAs are used to process messages from all peers. Based on file operation types, timestamps, peer IDs and other information the system dynamically discovers which peers are very busy depending on day time, users, running projects, etc. and consequently require configuration and resource distribution changes to reduce latency. VDA sends messages to machine learning (ML) computational block. ML block based on previous model training results and new data applies heuristics to optimize peer configuration and resources to resolve bottlenecks and improve performance. As a result, ML generate a new peer configuration and updates existing configuration and resources. Each peer configuration is scored for efficiency and iterates its own heuristics. VDA may filter out messages by file operations according to ML block requests. This filter may be updated dynamically. The spacetime relationship of files is at least one of the most useful quantities. The kinds of operations are another dimension and can go a long way to warp the spacetime to its most useful set. For example a file that is continually being created and destroyed is probably not worth transmitting. On the other hand, file types of various file extents may be observed to be repeatably opened at locations distant from their commit location. A heuristic can pre-position them and be rewarded when these file extents are opened where predicted or deprecated if not required within a time budget. It can be appreciated that dynamic creation, tracking, and movement of file extents, e.g. a file block of variable size, is inherently automated within file system and version control and not accessible for mental or paper-based data management.
As is known, circuits disclosed above may be embodied by programmable logic, field programmable gate arrays, mask programmable gate arrays, standard cells, and computing devices limited by methods stored as instructions in non-transitory media.
Generally a computing devices 600 can be any workstation, desktop computer, laptop or notebook computer, server, portable computer, mobile telephone or other portable telecommunication device, media playing device, a gaming system, mobile computing device, or any other type and/or form of computing, telecommunications or media device that is capable of communicating on any type and form of network and that has sufficient processor power and memory capacity to perform the operations described herein. A computing device may execute, operate or otherwise provide an application, which can be any type and/or form of software, program, or executable instructions, including, without limitation, any type and/or form of web browser, web-based client, client-server application, an ActiveX control, or a Java applet, or any other type and/or form of executable instructions capable of executing on a computing device.
The central processing unit 621 is any logic circuitry that responds to and processes instructions fetched from the main memory unit 622. In many embodiments, the central processing unit 621 is provided by a microprocessor unit, such as: those manufactured under license from ARM; those manufactured under license from Qualcomm; those manufactured by Intel Corporation of Santa Clara, Calif.; those manufactured by International Business Machines of Armonk, N.Y.; or those manufactured by Advanced Micro Devices of Sunnyvale, Calif. The computing device 600 may be based on any of these processors, or any other processor capable of operating as described herein.
Main memory unit 622 may be one or more memory chips capable of storing data and allowing any storage location to be directly accessed by the microprocessor 621. The main memory 622 may be based on any available memory chips capable of operating as described herein.
Furthermore, the computing device 600 may include a network interface 618 to interface to a network through a variety of connections including, but not limited to, standard telephone lines, LAN or WAN links (e.g., 802.11, T1, T3, 56 kb, X.25, SNA, DECNET), broadband connections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET), wireless connections, or some combination of any or all of the above. Connections can be established using a variety of communication protocols (e.g., TCP/IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, CDMA, GSM, WiMax and direct asynchronous connections). In one embodiment, the computing device 600 communicates with other computing devices 600 via any type and/or form of gateway or tunneling protocol such as Secure Socket Layer (SSL) or Transport Layer Security (TLS). The network interface 118 may comprise a built-in network adapter, network interface card, PCMCIA network card, card bus network adapter, wireless network adapter, USB network adapter, modem or any other device suitable for interfacing the computing device 600 to any type of network capable of communication and performing the operations described herein.
A computing device 600 of the sort depicted in
In some embodiments, the computing device 600 may have different processors, operating systems, and input devices consistent with the device. In other embodiments, the computing device 600 is a mobile device, such as a JAVA-enabled cellular telephone or personal digital assistant (PDA). The computing device 600 may be a mobile device such as those manufactured, by way of example and without limitation, Kyocera of Kyoto, Japan; Samsung Electronics Co., Ltd., of Seoul, Korea; or Alphabet of Mountain View Calif. In yet other embodiments, the computing device 600 is a smart phone, Pocket PC Phone, or other portable mobile device supporting Microsoft Windows Mobile Software.
In some embodiments, the computing device 600 comprises a combination of devices, such as a mobile phone combined with a digital audio player or portable media player. In another of these embodiments, the computing device 600 is device in the iPhone smartphone line of devices, manufactured by Apple Inc., of Cupertino, Calif. In still another of these embodiments, the computing device 600 is a device executing the Android open source mobile phone platform distributed by the Open Handset Alliance; for example, the device 600 may be a device such as those provided by Samsung Electronics of Seoul, Korea, or HTC Headquarters of Taiwan, R.O.C. In other embodiments, the computing device 600 is a tablet device such as, for example and without limitation, the iPad line of devices, manufactured by Apple Inc.; the Galaxy line of devices, manufactured by Samsung; and the Kindle manufactured by Amazon, Inc. of Seattle, Wash.
As is known, circuits include gate arrays, programmable logic, and processors executing instructions stored in non-transitory media provide means for scheduling, cancelling, transmitting, editing, entering text and data, displaying and receiving selections among displayed indicia, and transforming stored files into displayable images and receiving from keyboards, touchpads, touchscreens, pointing devices, and keyboards, indications of acceptance, rejection, or selection.
It should be understood that the systems described above may provide multiple ones of any or each of those components and these components may be provided on either a standalone machine or, in some embodiments, on multiple machines in a distributed system. The phrases in one embodiment’, in another embodiment’, and the like, generally mean the particular feature, structure, step, or characteristic following the phrase is included in at least one embodiment of the present disclosure and may be included in more than one embodiment of the present disclosure. However, such phrases do not necessarily refer to the same embodiment.
The systems and methods described above may be implemented as a method, apparatus or article of manufacture using programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The techniques described above may be implemented in one or more computer programs executing on a programmable computer including a processor, a storage medium readable by the processor (including, for example, volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Program code may be applied to input entered using the input device to perform the functions described and to generate output. The output may be provided to one or more output devices.
Each computer program within the scope of the claims below may be implemented in any programming language, such as assembly language, machine language, a high-level procedural programming language, or an object-oriented programming language. The programming language may, for example, be PHP, PROLOG, PERL, C, C++, C#, JAVA, or any compiled or interpreted programming language.
Each such computer program may be implemented in a computer program product tangibly embodied in a machine-readable storage device for execution by a computer processor. Method steps of the invention may be performed by a computer processor executing a program tangibly embodied on a computer-readable medium to perform functions of the invention by operating on input and generating output. Suitable processors include, by way of example, both general and special purpose microprocessors. Generally, the processor receives instructions and data from a read-only memory and/or a random access memory. Storage devices suitable for tangibly embodying computer program instructions include, for example, all forms of computer-readable devices, firmware, programmable logic, hardware (e.g., integrated circuit chip, electronic devices, a computer-readable non-volatile storage unit, non-volatile memory, such as semiconductor memory devices, including EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and nanostructured optical data stores. Any of the foregoing may be supplemented by, or incorporated in, specially-designed ASICs (application-specific integrated circuits) or FPGAs (Field-Programmable Gate Arrays). A computer can generally also receive programs and data from a storage medium such as an internal disk (not shown) or a removable disk. These elements will also be found in a conventional desktop or workstation computer as well as other computers suitable for executing computer programs implementing the methods described herein, which may be used in conjunction with any digital print engine or marking engine, display monitor, or other raster output device capable of producing color or gray scale pixels on paper, film, display screen, or other output medium. A computer may also receive programs and data from a second computer providing access to the programs via a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc.
Having described certain embodiments of methods and systems for video surveillance, it will now become apparent to one of skill in the art that other embodiments incorporating the concepts of the disclosure may be used. Therefore, the disclosure should not be limited to certain embodiments, but rather should be limited only by the spirit and scope of the following claims.
