The present invention relates to computer image processing technologies, and in particular, to methods and system for distributed image processing with huge data flow.
Nowadays, common image applications include face recognition in images in cameras, recognition of vehicle license plate of vehicles on high ways, access control system, fingerprint-based attendance machine, and so on. Conventional imaging processing systems are stand along devices. The amount of data to be processed is small, which allows data collecting and processing to be conducted on a single computer or a single processor chip.
However, it is difficult for a single-chip computer to handle those applications that produce huge data flows. There is therefore a need for technologies for rapid and effective image processing for a huge amount of image data.
To solve the above-mentioned problems, this invention provides distributed image processing methods and image processing system, which conducts information analysis in a distributed arrangement while processing the results of information analysis at a central location. The present invention can effectively and rapidly process a huge amount of image data, through increased network bandwidth and the number of image processing device. The present invention can handle a huge data flow and accomplish image processing in a short time.
In order to achieve the above described objectives, referring to
Furthermore, the file server and the image analysis server store image state information. Each server collects the image state information on a regular basis, and transfers the image state information to the upper level servers. The file server records and stores data state information and data resend information (e.g. double re-send error is regarded as transmission failure). The forwarding server also stores and record data state information, including the time of receiving image data, image ID, which time to send, which server to send, and check image data ID from stored sending state information and store the analysis results.
In the above mentioned step 2), if the data flow is very large, the image data can be transferred to the forwarding server in the next level, and forwarded to the image analysis server from the forwarding server. The forwarding server analyzes the state information from the image analysis servers, and selects the image analysis server having the shortest waiting queue to forward image processing task.
The distributed image processing system disclosed in the present invention works from high-level to the low-level servers: the file server, the forwarding server, to the image analysis server. The servers are connected by Ethernet and sockets based on TCP/IP protocols. The file server is used to input image data. The forwarding server is used to distribute image data to image analysis server and to return the analysis results from the image analysis server to the file server. The image analysis servers analyze image data and return feedback to the forwarding server.
The disclosed distributed image processing system includes one or more file server(s), one or more forwarding server(s) (including high level forwarding server and lower forwarding server when massive application) and one or more image analysis server(s). The distributed image processing system is divided into 3 levels in logical structure. The file server is the top level, which is responsible for image generation and display of analysis results. The second level is the forwarding server, which is mainly in charge of the feedback of data receiving, formatting, distributing, analysis information and operation condition etc. The image analysis server is in the third level and it receives image data and transfers analysis results to the forwarding server after processing and analyzing the data.
The servers can be connected by Ethernet, especially using socket interfaces based on TCP/IP protocols. Within TCP/IP, the communication can be based on network communication protocols or Window's default sharing protocol. The communication is kept always on and the communication state is monitored. It keeps connecting repeatedly if the network is disconnected.
The file server uses two methods for data connections:
When data flow is not heavy, the image processing system can be set a unified Ethernet environment. For a large amount of image data, the forwarding server can be equipped with dual-network card for data transmission in different segments.
Under the condition of huge data flows, the invention image processing system provides the forwarding servers to divide data processing tasks horizontally and vertically.
The horizon division uses multiple forwarding servers to process files for one or more file servers. Specifically, the file server can include different directories or IP address.
The vertical division divides forwarding servers into high and lower level forwarding servers. A high level forwarding server needs to meet high performance requirements so as to accomplish receiving, formatting, transmission and distribution of the data from the file servers. Less performance capabilities are required for a lower level forwarding server, because it only needs to transmit and distribute data from the file server, but does not need to receive and format such data. Image analysis servers and forwarding servers can be added to increase processing capability. An additional high level forwarding server is used for image data transmission and distribution. The high level forwarding server has higher network speed than the lower level forwarding server under the resources saturation condition. Accordingly, the high level forwarding server has higher data distribution capacity similar to router. The telecom grade servers are sufficient to meet the needs for the maximum load of image processing. In practice, both horizon and vertical divisions can be applied at the same time in the disclosed distributed image processing system.
If a large amount of image data is to be processed, the image data is stacked in high level forwarding servers. Referring to
As shown in
The image files and video files can be stored chronically on the file server 310. The forwarding server 320 receives the image files and video files to be analysis, format the image files, and distributes them to the corresponding image analysis server. As the image analysis server starts up, it automatically initiates image analysis programs and keeps working, waiting at all times for receiving image data from the forwarding server 320. After the image analysis server 330 obtains image data. For example, the image analysis server 330 can poll new files in a directory in the file server 310. If the image files are new, the image analysis server 330 transfers, analyzes and receives image data through image algorithms, identifies focused information (which varies according to applications, for example, pornography) and then returns the analysis report to the forwarding server 320. According to analysis report, the forwarding server 320 returns analysis results to the presentation module in the file server 310. The presentation module is defined by algorithm and application. The data transmission protocol in this invention is completely adapted to customized filed extension to accomplish the image processing.
The distributed image processing system 300 provides monitoring interface in the forwarding servers 320 and the image analysis servers 330 by installing image analysis server IP and port in the forwarding server 320. The system can monitor the computer and operation status of the forwarding server 320, including the activity levels of the processor, the memory, and the hard drive. Information is regularly transferred to the relevant forwarding servers 320 by lower-level computer. High level forwarding server 320 collects operation status from lower-level servers when there are many levels. Program how images, input by the file server 310, is transferred to the forwarding server 320 and how the processed information is returned to the forwarding server 320 by installing image analysis server IP, port and file directory, so as to provide analysis results for presentation module.
In the present invention, the file server 310 can include a file module and a presentation module. The file module is used to input image and store results after processing. The presentation module can present the outcome of processing images. The forwarding server 320 can include a formatting module and transmission & distribution module. The formatting module can format image files from the file server 310 to the ones which can be identified by the image analysis server and return the formatted analysis results to the forwarding server 320. Transmission & distribution module is for transferring and distributing image files to the image analysis server 330. The image analysis server 330 is for analyzing and processing image data.
The method in the formatting module of this invention is: read image files from the specified file server 310 and format file suffix and files according to network communication protocol and then transfer.
The image file is transferred to the image analysis server 330 that has the shortest task-waiting queue according to feedbacks from the image analysis server 330.
Furthermore, installed monitoring module for operation status is in each level forwarding servers 320. Each level forwarding server 320 provides users with unified status monitoring interface for real time information of the system operation.
The disclosed methods in the present application can include the following advantages:
The presently disclosed invention can be better understood by the figures and detailed implementation examples as described below.
As shown in
The procedure of passive image processing of the file server can include the following steps:
The file servers 1-N are connected with the high level forwarding server 420 which is communication with lower-level forwarding servers 430 and image analysis servers 440 by a Gigabit router 450. The lower-level forwarding server 430 is connected with image analysis servers 440 by a 100 M bit router 460. The data flow through the Gigabit router 450 should be more than that the dataflow through the 100 M bit router 460.
As an example, image data is processed at a rate of 1000 images per second; each image analysis server 440 can process 10 images per second; each low-level forwarding server 430 can support 10 image analysis servers 440. There should be 100 image analysis servers, 10 lower-level forwarding servers 430 on the left side of the distributed image processing system 400. The gigabit router 450 connects the high-level forwarding server 420 and the lower-level forwarding servers 430. The 100 M bit router 460 connects a lower-level forwarding server 430 and image analysis servers 440.
The high level forwarding server 420 transfers and distributes image data, and does not need to format image files, which therefore has higher network speed than the lower level forwarding server 430. Thus the high level forwarding server 420 has a data distribution capacity similar to a router (e.g. 450). The high level forwarding server 420 can be provided with telecom-grade servers.
For operation status, a monitoring module can be installed in the high level or lower level forwarding servers 420, 430 in the distributed image processing system 400. Each level forwarding server can provide unified status monitoring interface for real time information of the system operation, including operation time, operation status, the number of processed data, the amount of data being processed, the processor usage, the hard drive usage, the network usage and so on.
Adequate solution is also provided for processing a small amount of image data in different system structures, as shown on the right side of in
For lower amount of image processing tasks, the distributed image processing system 400 includes a forwarding and image analysis server 470 that integrates the forwarding and image analysis functions within a server. The forwarding and image analysis server 470 can handle small volume of image processing tasks.
At a higher level of integration, a server 490 integrates the functions of a file server, a forwarding server, and image analysis server within a server. The server 490 is also suitable for handling a small volume of image processing tasks. The servers 470, 490 each can include multiple cores that can handle multiple tasks in a multiple processes.
In one implementation, a low volume of image processing tasks can be performed by separate low power servers. The distributed image processing system 400 includes a forwarding server 480 which is connected to a 100 Mbit router 465, which in turn is connected to an image analysis server 447. The forwarding sever 480 is equipped with processing power suitable for data forwarding while the image analysis server 447 is dedicated to image processing.
Network communication protocol for file transmission can realize image processing with massive data flow. This invention provides a network protocol, which can complete distributed image processing more rapidly. In this protocol, there are 3 major data transmission types: image data, image analysis results, i.e. analysis report and system operation status, i.e. status report. Detailed data transmission protocol is shown in Table 1.
Next is detailed data network transmission protocol. Image data in transmission has unique ID. ID's prefix is generated from different server and IP. Different random numbers has different types of contents of transmission data.
0: Image data ExtensionString is image file type, the format is as below:
DataSource tag: Source server IP of image data
Ext tag: Extension of image data
It can be extended other tags
There is no nested relation among tags.
1: Analysis report ExtensionString image analysis result data, the format is transferring string to array, as below:
DataID tag: ID relevant image data ID in analysis report
EEL tag, FLG tag, OCR tag is defined study and presentation contents in presentation module by users.
2: Status report ExtensionString is system operation status data, as below:
This includes: IP tag: Local host computer IP
TaskProcessed tag: Processed task
TaskProcessing tag: Processing task
AppState tag: System operation status
HostState tag: Host computer status
NetState tag: Network communication status
Time tag: System operation time, calculated in second
CPU tag: Host computer CPU usage
RAM tag: Host computer RAM usage
Disk tag: Highest usage in Hard disk
NetFlux tag: Network flow, calculated in MB/S
IBuffer tag: Usage of input buffer
OBuffer tag: Usage of output buffer
It should be understood that the above described detailed implementation and figures are aimed at helping readers to understand the present invention and implement accordingly. However, as practitioners in this field can appreciate, the present invention can be implemented in different variations and with approaches without deviating from the spirit of the invention. It should be understood that the scope of the prevent invention is not limited to the specific implementations and examples described above.
Number | Date | Country | Kind |
---|---|---|---|
200910083710.4 | May 2009 | CN | national |
The present application is a national-entry application based on and claims priority to PCT Patent Application PCT/CN2009/001310, entitled “Distributed image processing method and image processing system” by the same inventor, filed Nov. 24, 2009, which claims priority to Chinese Patent Application No. CN200910083710.4, filed May 8, 2009. The content of these applications is incorporated herein by reference.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/CN2009/001310 | 11/24/2009 | WO | 00 | 9/19/2011 |