Patent Application
20240414217
The present disclosure relates to a communication management system, a communication system, a communication management device, an image processing method, and a non-transitory computer-readable medium.
A conference system to hold a conference between a plurality of terminal apparatuses via a communication network, such as the Internet, has become widespread. In such a conference system, stroke data for reproducing a stroke image drawn by a user with the user's hand, an electronic pen, or the like, using a terminal apparatus such as an electronic whiteboard is exchanged with a terminal apparatus to share the stroke image.
With regard to such a conference system, a technique is known that allows a user at each site to previously know an operation performed by a user at a different site by displaying on a display or the like information regarding the operation performed by the user at the different site.
According to one or more embodiments, a communication management system manages a session in which a plurality of terminal apparatuses shares a stroke image. The communication management system includes circuitry configured to: manage stroke information including a plurality of pieces of stroke data representing the stroke image; receive, from a first terminal apparatus, group operation information for designating one or more pieces of stroke data, which are operation targets, from among the plurality of pieces of stroke data; and restrict, based on the group operation information, an operation regarding the one or more pieces of stroke data by a second terminal apparatus, which is different from the first terminal apparatus.
A more complete appreciation of the disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings, wherein:
The accompanying drawings are intended to depict embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.
In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.
Referring to the drawings, embodiments of the present disclosure are described below in detail.
Referring to
The conference such as “video conference”, “teleconference”, or “remote conference” is an example of the session in which a plurality of terminal apparatuses shares a stroke image. The session in which a stroke image is shared may include a session such as remote teaching, remote medical examination, discussion, or simple conversation as well as a conference.
The communication system 100 includes, for example, the electronic whiteboard 1a, the electronic whiteboard 1b, a relay device 3, a communication management system 5, and an image storage device 7.
The electronic whiteboard 1a and the electronic whiteboard 1b are terminal apparatuses that perform interactive communications of content data such as video data and audio data for calling and image data and stroke data for sharing. Each of the electronic whiteboard 1a and the electronic whiteboard 1b is also referred to as an interactive whiteboard (IWB).
Although in the example of
The electronic whiteboard 1 is an example of a terminal apparatus according to the present embodiment. The terminal apparatus may be a different terminal apparatus having a communication function, a drawing function, a display function, etc. For example, the terminal apparatus may include an information terminal such as a personal computer (PC), a tablet terminal, or a smartphone, in which the application program corresponding to the communication system 100 is installed.
The stroke data includes the data for reproducing a stroke image. The stroke data includes coordinate data, line width data, line color data, vector data, etc. The electronic whiteboard 1a and the electronic whiteboard 1b exchange video data and audio data for calling with the other end of the communication to reproduce video and sound of the other end so as to enable video communications.
The electronic whiteboard 1a and the electronic whiteboard 1b exchange the image data on a shared document image so that the user of the communication system 100 may share the identical document image. The document image includes an image presented on a display of the electronic whiteboard 1. The document image may include, for example, a document for a conference, a background image displayed on the display, or an image of the capture screen that is obtained when the display screen is captured.
The electronic whiteboard 1a and the electronic whiteboard 1b exchange the stroke data on a shared stroke image so that a participant using the communication system 100 may share the identical stroke image. The stroke image includes the image representing, for example, a handwritten stroke line by a user with an electronic pen, or the like. The stroke image is displayed by using the stroke data representing the points specifying the coordinates on the display.
In the following description, the electronic whiteboard 1 that requests the start of a video conference is referred to as a “source terminal”, and the electronic whiteboard 1 that is at the destination (relay destination) of the request is referred to as a “destination terminal”. In
The relay device 3, which is implemented by one or more computers, performs a process of relaying content data for communications between the electronic whiteboard 1a and the electronic whiteboard 1b.
The communication management system 5 is implemented by one or more computers. The communication management system 5 centrally controls, for example, the authentication of login from the electronic whiteboard 1a and the electronic whiteboard 1b, the communication statuses of the electronic whiteboard 1a and the electronic whiteboard 1b, a destination list, or the communication status of the relay device 3. The communication management system 5 relays stroke data to be shared between the electronic whiteboard 1a and the electronic whiteboard 1b.
The image storage device 7, which is implemented by one or more computers, stores the image data on a shared document image, which is uploaded from the electronic whiteboard 1a, and downloads the image data to the electronic whiteboard 1b, and vice versa. That is, the image storage device 7 stores the image data uploaded from the electronic whiteboard 1b and downloads the image data to the electronic whiteboard 1a.
In one example, each of the relay device 3, the communication management system 5, and the image storage device 7 is configured as a single computer. In another example, each of the relay device 3, the communication management system 5, and the image storage device 7 is configured as a plurality of computers to which one or more units (functions, means, or storages) are arbitrarily allocated.
In the communication system 100, a management information session sei (hereinafter referred to as “session sei”) for exchanging various types of management information is established between the electronic whiteboard 1a and the electronic whiteboard 1b via the communication management system 5. Four sessions for exchanging four pieces of data, i.e., high-resolution image data, medium-resolution image data, low-resolution image data, and audio data, are established between the electronic whiteboard 1a and the electronic whiteboard 1b via the relay device 3. As illustrated in
In the communication system 100, stroke data may be exchanged between the electronic whiteboard 1a and the electronic whiteboard 1b by using the session sei.
The resolution of the video of video data according to the present embodiment is described below. Low-resolution video data includes a base image including, for example, 160 pixels horizontally and 120 pixels vertically. Medium-resolution video data includes, for example, 320 pixels horizontally and 240 pixels vertically. High-resolution video data includes, for example, 640 pixels horizontally and 480 pixels vertically. Low-image quality video data including simple low-resolution video data, which includes a base image, is relayed in the case of a narrowband path. Medium-image quality video data including low-resolution video data, which includes a base image, and medium-resolution video data is relayed in the case of a relatively wide band. High-image quality video data including low-resolution video data, which includes a base image, medium-resolution video data, and high-resolution video data is relayed in the case of a very wide band. Audio data is relayed even in a narrowband path as the audio data has a small amount of data as compared with video data.
Next, an example of the hardware configuration of each device according to the present embodiment is described.
The CPU 101 includes an arithmetic device that controls the overall operation of the electronic whiteboard 1. The ROM 102 includes a non-volatile memory that stores a program, such as initial program loader (IPL) to boot the CPU 101. The RAM 103 includes a volatile memory used as a work area for the CPU 101. The SSD 104 includes a large-capacity storage device that stores various types of data such as programs for the electronic whiteboard 1.
The network I/F 105 includes a communication interface to couple the electronic whiteboard 1 to a communication network for communications. The external-device connection I/F 106 includes an interface to couple various external devices. In this case, examples of the external devices include a universal serial bus (USB) memory 131 and externally coupled devices (a microphone 132, a speaker 133, and a camera 134), etc.
The electronic whiteboard 1 further includes, for example, a capture device 111, a graphics processing unit (GPU) 112, a display controller 113, a contact sensor 114, a sensor controller 115, an electronic pen controller 116, a short-range communication circuit 119, an antenna 119a of the short-range communication circuit 119, a power switch 117, and various selection switches 118.
The capture device 111 captures (acquires) the display screen, or the like, presented on the display of an externally coupled PC 150 as a still image or a moving image. The GPU 112 includes a semiconductor chip (processor) dedicated for graphics. The display controller 113 controls and manages the screen display in order to output an image output from the GPU 112 to the display 160, etc. The contact sensor 114 detects the contact on the display 160 with the electronic pen 140, the hand H of the user, etc. The sensor controller 115 controls the processing of the contact sensor 114.
The contact sensor 114 for example inputs and detects coordinates by using an infrared blocking method. More specifically, the display 160 is provided with two light receiving elements disposed on both upper side ends of the display 160, and a reflector frame surrounding the sides of the display 160. The light receiving elements emit a plurality of infrared rays in parallel to a surface of the display 160. The light receiving elements receive lights passing in the direction that is the same as an optical path of the emitted infrared rays, which are reflected by the reflector frame. The contact sensor 114 outputs the IDs of the infrared rays emitted by the two light emitting and receiving devices and blocked by the object to the sensor controller 115. The sensor controller 115 identifies the coordinates position that is the contact position of the object. The electronic pen controller 116 communicates with the electronic pen 140 to determine whether there is a touch on the display 160 with the pen tip or the opposite end from the pen tip. The short-range communication circuit 119 includes a communication circuit for a near field communication (NFC), Bluetooth (registered trademark), etc.
The power switch 117 includes a switch for turning on/off the power of the electronic whiteboard 1. The various selection switches 118 include, for example, a group of switches that adjust the brightness, the color, and the like, of the display 160.
The electronic whiteboard 1 further includes a bus line 120. The bus line 120 includes, for example, an address bus, a data bus, and various control signals to electrically connect each component such as the CPU 101 illustrated in
For example, the contact sensor 114 may use, instead of the infrared blocking method, a capacitive touch panel that detects a change in the capacitance to identify the contact position or a resistive film touch panel that identifies the contact position in accordance with a change in the voltage of two opposing resistive films. In another example, the contact sensor 114 may use an electromagnetic induction touch panel that identifies a contact position by detecting electromagnetic induction caused by contact of an object to a display. The electronic pen controller 116 may determine whether there is a touch with the portion of the electronic pen 140 grasped by the user or other portions of the electronic pen 140 as well as the pen tip of the electronic pen 140 and the opposite end from the pen tip.
The communication management system 5, the relay device 3, the image storage device 7, the PC 150, etc. include the hardware configuration of a computer 400 illustrated in
The CPU 401 includes an arithmetic device that controls the overall operation of the computer 400. The ROM 402 includes a non-volatile memory that stores a program, such as IPL to boot the CPU 401. The RAM 403 includes a volatile memory used as a work area for the CPU 401. The HD 404 includes a large-capacity storage device that stores programs for the operating system (OS), applications, and the like, and various types of data. The HDD controller 405 controls the reading or writing of various types of data from or to the HD 404 under the control of the CPU 401.
The display 406 presents various types of information such as a cursor, a menu, a window, characters, or images. The external-device connection I/F 407 includes an interface that couples various external devices. The network I/F 408 includes a communication interface that executes data communications using a communication network. The keyboard 409 is an example of an input device (input means) provided with a plurality of keys to input characters, numerical values, various instructions, etc. The pointing device 410 includes one type of input unit that, for example, selects or executes various instructions, selects the target to be processed, or moves the cursor.
The DVD-RW drive 412 controls the reading or writing (storing) of data from or to a DVD-RW 411. The DVD-RW drive 412 may control the reading or writing (storing) of data from or to a disk such as Blu-ray (registered trademark) disc rewritable (BD-RE) instead of the DVD-RW 411. The medium I/F 414 controls the reading or writing (storing) of data from or to a recording medium 413 such as a flash memory. The bus line 415 includes, for example, an address bus, a data bus, and various control signals to electrically connect each component such as the CPU 401 illustrated in
Next, an example of the general arrangement of the communication system 100 is described.
As illustrated in
The electronic whiteboard 1a, the electronic whiteboard 1b, the relay device 3, the communication management system 5, and the image storage device 7 can exchange data with each other via a communication network 500 such as the Internet or a local area network (LAN). The communication network 500 may include a wireless communication part. In
Next, an example of the functional configuration of the communication system 100 according to the present embodiment is described.
Each of a data exchange unit 11, an acceptance unit 12, a video/audio processing unit 13, a display controller 14, an image processing unit 15, an information transmission unit 16, a short-range communication unit 17, and a storing/reading processing unit 18 of each electronic whiteboard 1 is implemented by the CPU 101 illustrated in
The data exchange unit 11 exchanges various types of data (or information) with a different terminal, device, or system via the communication network 500. The data exchange unit 11 also serves as a starting unit that performs processing for login to the communication management system 5, or the like, and performs processing to start the communication with a different terminal apparatus.
The acceptance unit 12 accepts various inputs from the user with the electronic pen 140, etc. For example, the acceptance unit 12 accepts the designation operation to designate one or more pieces of stroke data that is the operation target, the change operation regarding stroke data, or the operation to draw a stroke image.
The video/audio processing unit 13 performs the primary processing of the video conference function. For example, the video/audio processing unit 13 performs digital processing such as encoding on video data and audio data based on a signal output from the microphone 132 and a signal output from the camera 134. The video/audio processing unit 13, for example, generates a video signal or an audio signal based on the video data and the audio data received by the data exchange unit 11. Furthermore, the video/audio processing unit 13 also performs, for example, the process to combine pieces of video data having different resolutions.
The display controller 14 performs the control to output a video signal (image signal), or the like, to the display 160 and display the display screen.
The image processing unit 15 performs the primary processing of the electronic whiteboard function. For example, the image processing unit 15 performs, for example, the process to generate a stroke image or stroke data based on the stroke of the electronic pen 140, or the like, received by the acceptance unit 12 or the process to generate a stroke image based on the stroke data received by the data exchange unit 11. The image processing unit 15 also performs the process to generate an image signal based on the image data on the document image received by the data exchange unit 11.
The information transmission unit 16 transmits the stroke data generated by the image processing unit 15 to the communication management system 5. In response to the designation operation for designating one or more pieces of stroke data, which is the operation target, accepted by the acceptance unit 12, the information transmission unit 16 transmits the group operation information for designating one or more pieces of stroke data to the communication management system 5.
The short-range communication unit 17 acquires or provides data from or to each terminal including a short-range communication unit via a short-range wireless communication.
The storing/reading processing unit 18 performs, for example, the process to store various types of data in the storage unit 1001 or a recording medium 1002 such as the USB memory 131 and the process to read various types of data stored in the storage unit 1001 or the recording medium 1002.
The storage unit 1001 overwrites video data and audio data each time the video data and the audio data are received during the communication with a different terminal. The display 160 presents the image based on the video data before overwriting, and the speaker 133 outputs the sound based on the audio data before overwriting.
The communication management system (an example of a communication management device) 5 uses for example the CPU 401 illustrated in
The terminal ID is the identification information for identifying the electronic whiteboard (terminal apparatus) 1. The terminal ID is an example of the identification information for the communication management system 5 to manage a terminal apparatus such as the electronic whiteboard 1. Instead of the terminal ID, the communication management system 5 may manage the electronic whiteboard 1 by using, for example, a user ID for identifying a user, a contract ID for identifying a contractor, or a communication ID for identifying a communication source.
The password is an example of the authentication information for authenticating a terminal apparatus such as the electronic whiteboard 1. The authentication information may be authentication information, such as an access token, other than the password.
For example, in the terminal management table 800 illustrated in
The destination list is an example of destination information. The destination information may have a row of information, such as terminal IDs, regarding the destination rather than in list form.
In the example of
The “stroke” corresponds to the stroke data according to the present embodiment and is managed for each “page” by the stroke-information management unit 56. For example, the first stroke includes the stroke ID (id=aaaaaa) for identifying the first stroke, the information (parent=hijkl) for identifying the first page, which is a parent, Gid “g001” for identifying first drawing data, and the first drawing data. For example, when an operation for movement is performed on the stroke image represented by the first stroke, the first drawing data and the Gid “g001” for identifying the first drawing data are updated although the stroke ID (id=aaaaaa) is continuously used.
The first drawing data includes the drawing data for reproducing the first stroke. The first drawing data includes, for example, coordinate data, line width data, line color data, or vector data.
The electronic whiteboard 1 enables page switching, and “page” corresponds to each page of the electronic whiteboard 1. For example, the first page includes information such as the page ID (id=hijkl) for identifying the first page or the information (parent=abcdefg) for identifying the parent “conference”.
The “conference” corresponds to the session managed by the communication system 100 and is managed by using, for example, the session ID (id=abcdefg) for identifying the session.
For example, the image processing unit 15 of the electronic whiteboard 1 may use the first drawing data to fifth drawing data included in the first stroke to the fifth stroke to generate the stroke image drawn on the first page.
The session ID (id=abcdefg) is an example of first identification information for identifying a session (conference). The stroke ID (id=aaaaaa) is an example of second identification information for identifying stroke data. Further, the Gid “g001”, or the like for identifying the first drawing data is an example of third identification information for identifying drawing data.
Referring back to
The data exchange unit 51 exchanges various types of data (or information) with a different terminal, device, or system via the communication network 500. For example, the data exchange unit 51 functions as a reception unit that receives, from the electronic whiteboard 1, the group operation information for designating one or more pieces of stroke data that is the operation target. The data exchange unit 51 is an example of the reception unit.
The authentication unit 52 conducts authentication of the login request received via the data exchange unit 51. For example, the authentication unit 52 permits the login of the electronic whiteboard 1 when the authentication management table 700 illustrated in
The terminal management unit 53 manages the terminal management DB 5002. For example, the terminal management unit 53 updates the information such as the operating status, the reception time and date, and the terminal IP address in the terminal management table 800 illustrated in
The terminal management unit 53 manages the destination-list management table 900. For example, the terminal management unit 53 provides the destination-list information including the terminal ID of the destination terminal in the destination-list management table 900 illustrated in
The session management unit 54 manages the session held by the communication system 100. For example, the session management unit 54 generates the session ID for identifying the session based on the start request information requesting the start of a communication from the electronic whiteboard 1. The session management unit 54 stores and manages various types of information regarding a session in relation to the session ID in, for example, the session management table 1000 illustrated in
The relay-device management unit 55 manages the relay-device management DB 5005. The relay-device management unit 55 uses, for example, the relay-device management table 1100 illustrated in
According to the present embodiment, the relay device 3 used to relay a session may be selected by using any method. The number of the relay devices 3 included in the communication system 100 may be one.
The stroke-information management unit 56 stores and manages the stroke data transmitted via, for example, the session sei from the electronic whiteboard 1, which is a participant of a session, in relation to the session ID in the storage unit 5000, etc. For example, the stroke-information management unit 56 stores the stroke data received from the electronic whiteboard 1 in the stroke information 5006 illustrated in
Based on the group operation information designating one or more pieces of stroke data, which is the operation target, and received by the data exchange unit 51 from the electronic whiteboard 1, the operation restriction unit 57 restricts the operation performed by the different electronic whiteboard 1 regarding the designated stroke data. The process performed by the operation restriction unit 57 and the group operation information are described later.
The storing/reading processing unit 58 performs, for example, the process to store various types of data in the storage unit 5000 and the process to read various types of data stored in the storage unit 5000.
The image storage device 7 uses, for example, the CPU 401 illustrated in
The data exchange unit 71 exchanges various types of data (or information) with a different terminal, device, or system via the communication network 500. The storing/reading processing unit 72 performs, for example, the process to store various types of data in the storage unit 7000 or the process to read various types of data stored in the storage unit 7000.
With the above-described configuration, for example, the image storage device 7 stores the image data uploaded from the electronic whiteboard 1 in a specified uniform resource locator (URL) and provides the image data stored in a specified URL in response to a request from the electronic whiteboard 1.
The relay device 3 uses, for example, the CPU 401 illustrated in
The data exchange unit 31 exchanges various types of data (or information) with a different terminal, device, or system via the communication network 500. The data exchange unit 31 also serves as a transfer unit that, for example, transfers video data and audio data received from the electronic whiteboard 1 to the different electronic whiteboard 1 that is a participant of the same session as that of the electronic whiteboard 1. The determination unit 32 makes various determinations such as determination on a data delay state.
The storing/reading processing unit 33 performs, for example, the process to store various types of data in the storage unit 3000 or the process to read various types of data stored in the storage unit 3000.
The functional configuration of the communication system 100 illustrated in
Next, the process flows of a communication management method and an image processing method according to the present embodiment are described.
First, when the power switch 117 of the electronic whiteboard 1a is turned on, the acceptance unit 12 receives the power-on (step S22).
Subsequently, the data exchange unit 11 transmits the login request information for requesting the login to the communication management system 5 via the communication network 500 (step S23). The login request information includes, for example, the terminal ID of the electronic whiteboard 1a, a password, etc.
Subsequently, the authentication unit 52 of the communication management system 5 searches the authentication management table 700 illustrated in
When the login of the electronic whiteboard 1a is permitted, the terminal management unit 53 updates the information corresponding to the terminal ID “01aa” of the electronic whiteboard 1a in the terminal management table 800 illustrated in
Subsequently, the session management unit 54 adds the new record in which the terminal ID “01aa” of the electronic whiteboard 1a received at the above-described step S23 is the “terminal ID of the source terminal” to the session management table 1000 illustrated in
When receiving the authentication result information indicating the permission of login, the data exchange unit 11 of the electronic whiteboard 1a transmits the destination-list request information indicating the request for a destination list to the communication management system 5 via the communication network 500 (step S28). Accordingly, the data exchange unit 51 of the communication management system 5 receives the destination-list request information.
Subsequently, the terminal management unit 53 searches the destination-list management table 900 illustrated in
Subsequently, the data exchange unit 51 of the communication management system 5 reads, for example, the data on the destination-list frame and the data on the icon indicating the operating status stored in the storage unit 5000 via the storing/reading processing unit 58 (step S30). The data exchange unit 51 of the communication management system 5 transmits the “destination-list information (the destination-list frame, the icon, the terminal ID, and the destination name)” including the read destination-list frame and icon and the terminal ID and the destination name extracted in step S29 to the electronic whiteboard 1a (step S31). Accordingly, in the electronic whiteboard 1a, the data exchange unit 11 receives the destination-list information, and the storing/reading processing unit 18 stores the received destination-list information in the storage unit 1001 (step S32).
As described above, according to the present embodiment, each of the electronic whiteboards 1 does not manage the destination-list information but the communication management system 5 centrally manages the pieces of destination-list information on all the terminals. Thus, it is possible to save the task performed by each of the electronic whiteboards 1 to change the destination-list information even when the new electronic whiteboard 1 is registered in the communication management system 5, when a terminal apparatus with a new model is registered instead of the already registered electronic whiteboard 1, or when the appearance of the destination-list frame, or the like, is changed.
The terminal management unit 53 of the communication management system 5 searches the terminal management table 800 illustrated in
Subsequently, the data exchange unit 51 of the communication management system 5 transmits the “terminal status information” including the terminal ID used as the search key in step S33 and the operating status of the corresponding destination terminal to the electronic whiteboard 1a via the communication network 500 (step S34).
Subsequently, the storing/reading processing unit 18 of the electronic whiteboard 1a sequentially stores the terminal status information received from the communication management system 5 in the storage unit 1001 (step S35). Thus, the electronic whiteboard 1a may receive the status information on each of the electronic whiteboards 1 to acquire the current operating statuses of the electronic whiteboard 1b, the electronic whiteboard1c, etc., which are destination candidates that may communicate with the electronic whiteboard 1a.
Subsequently, the display controller 14 of the electronic whiteboard 1a generates the destination list reflecting the status of the terminal that is a destination candidate based on the destination-list information stored in the storage unit 1001 and the terminal status information. The display controller 14 causes the display 160 of the electronic whiteboard 1a to present the destination-list screen using the generated destination list (step S36).
The terminal management unit 53 of the communication management system 5 searches the destination-list management table 900 illustrated in
Subsequently, the terminal management unit 53 of the communication management system 5 searches the terminal management table 800 illustrated in
Then, the data exchange unit 51 transmits the “terminal status information” including the terminal ID and the operating status of the electronic whiteboard 1a to the electronic whiteboard 1 with the terminal ID, of which the operating status is “online” in the terminal management table 800, among the terminal IDs extracted in step S37 (step S39). When the terminal status information is transmitted to each of the electronic whiteboards 1, the data exchange unit 51 refers to the IP address of the electronic whiteboards 1 managed in the terminal management table 800 based on the terminal ID. Thus, it is possible to notify each of the electronic whiteboards 1 that may communicate with the electronic whiteboard 1a as a destination candidate of the terminal ID “01aa” and the operating status “online” of the electronic whiteboard 1a. Accordingly, each of the electronic whiteboards 1, which is a destination candidate, may display the status of the electronic whiteboard 1a (step S40).
When the electronic whiteboard 1a, which is a source terminal, receives the destination selected by the user A in step S1401, the process in step S1402 and subsequent steps is executed. Here, the user A selects the electronic whiteboard 1b as a destination terminal.
After the electronic whiteboard 1a receives the destination selected by the user A, the data exchange unit 11 of the electronic whiteboard 1a transmits the start request information for requesting the start of a session to the communication management system 5 in step S1402. The start request information includes, for example, the terminal ID “01aa” of the electronic whiteboard 1a that is the source terminal, the terminal ID “01ba” of the electronic whiteboard 1b that is the destination terminal, and the IP address of the electronic whiteboard 1a that is the source terminal. As a result, the data exchange unit 51 of the communication management system 5 receives the IP address of the source terminal (the electronic whiteboard 1a), which is a transmission source, as well as the start request information.
In step S1403, the terminal management unit 53 of the communication management system 5 updates the terminal management DB 5002 based on the terminal ID of the source terminal and the terminal ID of the destination terminal included in the start request information. For example, the terminal management unit 53 updates the “operating status” corresponding to each of the terminal IDs “01aa” and “01ba” to “online (communication being executed)” in the terminal management table 800 based on the terminal ID “01aa” of the source terminal and the terminal ID “01ba” of the destination terminal included in the start request information. In this state, although the electronic whiteboard 1a, which is the source terminal, and the electronic whiteboard 1b, which is the destination terminal, are not participating in the session, the electronic whiteboard 1a and the electronic whiteboard 1b are managed as being in the state of executing the communication.
In step S1404, the session management unit 54 of the communication management system 5 generates the session ID for identifying the session held between the electronic whiteboard 1a and the electronic whiteboard 1b. Here, the session management unit 54 generates the session ID “se01”.
In step S1405, the session management unit 54 stores the generated session ID, the terminal ID “01aa” of the source terminal, the terminal ID “01ba” of the destination terminal, and the like, in relation with one another in the session management table 1000 illustrated in
In step S1406, the relay-device management unit 55 of the communication management system 5 selects the relay device 3 that relays the session between the source terminal (the electronic whiteboard 1a) and the destination terminal (the electronic whiteboard 1b). For example, the relay-device management unit 55 selects the relay device 3 having the IP address that is close to the IP address of the source terminal (the electronic whiteboard 1a) in the terminal management table 800 with regard to the device IDs of the relay devices 3 whose operating status is “online” in the relay-device management table 1100. Here, the relay device 3 having the device ID “111a” is selected.
In steps S1407a and S1407b, the session management unit 54 of the communication management system 5 transmits the relay-device selection information to the source terminal (the electronic whiteboard 1a) and the destination terminal (the electronic whiteboard 1b) via the data exchange unit 51. The relay-device selection information includes, for example, the IP address of the relay device 3 selected in step S1406, the session ID generated in step S1404, and the IP address of the communication management system 5.
After receiving the relay-device selection information, the data exchange units 11 of the source terminal (the electronic whiteboard 1a) and the destination terminal (the electronic whiteboard 1b) transmit, to the communication management system 5, the reception completion information indicating that the relay-device selection information has been received in steps S1408a and S1408b. The reception completion information includes, for example, the session ID “se01” included in the relay-device selection information.
In steps S1409a and S1409b, the data exchange units 11 of the source terminal (the electronic whiteboard 1a) and the destination terminal (the electronic whiteboard 1b) use the device ID of the relay device 3 and the session ID included in the relay-device selection information to establish a session with the relay device 3. This allows the electronic whiteboard 1a and the electronic whiteboard 1b to participate in the session with the same session ID “se01” and exchange, for example, video data or audio data of the conference video with each other via the relay device 3.
By the above-described process, the user A of the electronic whiteboard 1a and the user B of the electronic whiteboard 1b may exchange a conference video, and the like, to have a video conference.
The communication system 100 allows the other electronic whiteboards 1 to display, as a shared image, the stroke image drawn on the electronic whiteboard 1 with the electronic pen 140, etc. for the user A of the electronic whiteboard 1a and the users B and C of the electronic whiteboard 1b participating in the video conference. Here, an example of the process to share a stroke image between the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session is described.
The electronic whiteboard 1a and the electronic whiteboard 1b share a document image and display the document image on the display 160 via the image storage device 7 as appropriate.
In step S1501, when the user A makes a drawing (movement) with the electronic pen 140 or the hand H in contact with the display 160 of the electronic whiteboard 1a, the acceptance unit 12 receives an input of the drawn stroke (trajectory).
In step S1502, the image processing unit 15 of the electronic whiteboard 1a generates the stroke data for displaying the stroke image corresponding to the stroke received by the acceptance unit 12.
In step S1503, the display controller 14 of the electronic whiteboard 1a causes the display 160 to present the stroke image based on the stroke data generated by the image processing unit 15. This allows the stroke image drawn by the user A with the electronic pen 140, or the like, on the display 160 to be presented on the display 160.
In step S1504, the data exchange unit 11 of the electronic whiteboard 1a transmits the stroke data for reproducing the stroke image drawn by the user A to the communication management system 5 during, for example, the session sei. Accordingly, the data exchange unit 51 of the communication management system 5 receives the stroke data transmitted from the electronic whiteboard 1a.
In step S1505, the stroke-information management unit 56 of the communication management system 5 registers the stroke data received from the electronic whiteboard 1a in, for example, the stroke information 5006 illustrated in
In step S1506, the stroke-information management unit 56 of the communication management system 5 transfers the stroke data to the electronic whiteboard 1b participating in the same session as that of the electronic whiteboard 1a. The data exchange unit 11 of the electronic whiteboard 1b receive the stroke data.
In step S1507, the image processing unit 15 of the electronic whiteboard 1b generates a stroke image based on the stroke data received by the data exchange unit 11.
In step S1508, the display controller 14 of the electronic whiteboard 1b causes the display 160 to present the stroke image generated by the image processing unit 15, i.e., the stroke image drawn by the user A in step S1501.
When the user B or the user C makes a drawing on the electronic whiteboard 1b with the electronic pen 140, or the like, the electronic whiteboard 1a may display the stroke image in the same manner.
Thus, the electronic whiteboard 1a and the electronic whiteboard 1b participating in the same session may share the stroke image.
During the conference where the electronic whiteboard (terminal apparatus) 1a and the electronic whiteboard 1b share a stroke image, the electronic whiteboards 1a and 1b may perform an editing process such as a moving process on the shared stroke image. In this case, for example, when the electronic whiteboard 1a and the electronic whiteboard 1b perform a moving operation on the shared stroke image, an operation different from the operation intended by the user may be performed on the stroke image in some operation timing.
In order to solve the above-described disadvantage, the communication management system 5 has an operation restriction function to, for example, restrict the user B's editing operation on the stroke image, which is the operation target, when the user A is performing an editing operation on the stroke image.
Before the operation restriction function is described, an example of the process to move a stroke image without use of the operation restriction function is described.
Here, as an example for the description, the user A of the electronic whiteboard 1a performs the operation to sequentially move the stroke images ST1 to ST5 to left, and the user B of the electronic whiteboard 1b performs the operation to sequentially move the stroke images ST2 to ST4 downward.
In step S1601, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST1 (Gid=g001) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST1. The operation information includes, for example, the coordinate information after the movement of the stroke image ST1.
Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the first drawing data and the Gid of the first stroke corresponding to the stroke image ST1 (Gid=g001) in the stroke information 5006 illustrated in
In step S1602, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1a, the success notification indicating that the stroke image ST1 has been successfully moved. The success notification includes the updated Gid (e.g., Gid=g006) of the stroke image ST1.
In step S1603, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST2 (Gid=g002) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST2. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the second drawing data and the Gid of the second stroke corresponding to the stroke image ST2 (Gid=g002) in the stroke information 5006 illustrated in
In step S1604, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1a, the success notification indicating that the stroke image ST2 has been successfully moved. The success notification includes the updated Gid (e.g., Gid=g007) of the stroke image ST2.
In step S1605, the information transmission unit 16 of the electronic whiteboard 1b transmits, to the communication management system 5, the operation information for moving the stroke image ST2 (Gid=g002) downward by a predetermined distance in accordance with the user B's operation to move the stroke image ST2. However, as the Gid of the stroke image ST2 has been updated to “g007”, it is difficult for the stroke-information management unit 56 of the communication management system 5 to perform the operation to move the stroke image ST2 (Gid=g002).
In step S1606, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1b, the failure notification indicating that the movement of the stroke image ST2 has failed.
In step S1607, the information transmission unit 16 of the electronic whiteboard 1b transmits, to the communication management system 5, the operation information for moving the stroke image ST3 (Gid=g003) downward by a predetermined distance in accordance with the user B's operation to move the stroke image ST3. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the third drawing data and the Gid of the third stroke corresponding to the stroke image ST3 (Gid=g003) in the stroke information 5006 illustrated in
In step S1608, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1b, the success notification indicating that the stroke image ST3 has been successfully moved. The success notification includes the updated Gid (e.g., Gid=g008) of the stroke image ST3.
In step S1609, the information transmission unit 16 of the electronic whiteboard 1b transmits, to the communication management system 5, the operation information for moving the stroke image ST4 (Gid=g004) downward by a predetermined distance in accordance with the user B's operation to move the stroke image ST4. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the fourth drawing data and the Gid of the fourth stroke corresponding to the stroke image ST4 (Gid=g004) in the stroke information 5006 illustrated in
In step S1610, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1b, the success notification indicating that the stroke image ST4 has been successfully moved. The success notification includes the updated Gid (e.g., Gid=g009) of the stroke image ST4.
In step S1611, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST3 (Gid=g003) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST3. However, as the Gid of the stroke image ST3 has been updated to “g008”, it is difficult for the stroke-information management unit 56 of the communication management system 5 to perform the operation to move the stroke image ST3 (Gid=g003).
In step S1612, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1a, the failure notification indicating that the movement of the stroke image ST3 has failed.
In step S1613, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST4 (Gid=g004) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST4. However, as the Gid of the stroke image ST4 has been updated to “g009”, it is difficult for the stroke-information management unit 56 of the communication management system 5 to perform the operation to move the stroke image ST4 (Gid=g004).
In step S1614, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1a, the failure notification indicating that the movement of the stroke image ST4 has failed.
In step S1615, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST5 (Gid=g005) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST5. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the fifth drawing data and the Gid of the fifth stroke corresponding to the stroke image ST5 (Gid=g005) in the stroke information 5006 illustrated in
In step S1616, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1a, the success notification indicating that the stroke image ST5 has been successfully moved. The success notification includes the updated Gid (e.g., Gid=g010) of the stroke image ST5.
As described above, without use of the operation restriction function, the stroke images ST1, ST2, and ST5 have moved to left in accordance with the operations of the user A and the stroke images ST3 and ST4 have moved downward in accordance with the operations of the user B, as illustrated in the display screen 1760 in
In order to solve the above-described disadvantage, the communication management system 5 has the operation restriction function to, for example, restrict the user B's editing operation on the stroke image, which is the operation target, when the user A is performing an editing operation on the stroke image.
At the start of the process illustrated in
Here, as an example for the description, the user A of the electronic whiteboard 1a performs the operation to sequentially move the stroke images ST1 to ST5 to left.
In step S1801, the acceptance unit 12 of the electronic whiteboard 1a accepts the group selection operation from the user A.
According to the present embodiment, before performing an editing operation on a shared stroke image, the user may select the stroke image, which is the operation target, to notify the communication management system 5 of the list of stroke data to be operated.
In step S1802, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, a first group operation request for designating one or more pieces of stroke data, which is the operation target, in accordance with the user A's group selection operation. The first group operation request is an example of the group operation information for designating one or more pieces of stroke data that is the operation target.
The character string ““list”: [“aaaaa”,“bbbbb”,“ccccc”,“ddddd”, “eeeee” ]” is the information for designating the stroke ID of the stroke data that is the operation target. The example of
In step S1803 in
When the operation target designated by the first group operation request includes no operation-restricted operation target, the process in step S1804 and subsequent steps is executed. Conversely, when the operation target designated by the first group operation request includes an operation-restricted operation target, the operation restriction unit 57 of the communication management system 5 transmits a rejection notification to the electronic whiteboard 1, which has transmitted the request, as described later.
In step S1804, the operation restriction unit 57 of the communication management system 5 starts an operation restriction process on the stroke data designated by the first group operation request. Thus, the operation restriction unit 57 prohibits the other electronic whiteboard 1b from performing operations on the stroke image designated by the first group operation request transmitted from the electronic whiteboard 1a.
In step S1805, the operation restriction unit 57 of the communication management system 5 transmits the reception notification indicating that the first group operation request has been received to the electronic whiteboard 1a that has transmitted the request.
In step S1806, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST1 (Gid=g001) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST1. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the first drawing data of the first stroke corresponding to the stroke image ST1 (Gid=g001) and updates the Gid to “g006” in the stroke information 5006 illustrated in
In step S1807, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST2 (Gid=g002) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST2. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the second drawing data of the second stroke corresponding to the stroke image ST2 (Gid=g002) and updates the Gid to “g007” in the stroke information 5006 illustrated in
Here, for example, in step S1808, the acceptance unit 12 of the electronic whiteboard 1b accepts the group selection operation from the user B.
In step S1809, the information transmission unit 16 of the electronic whiteboard 1b transmits, to the communication management system 5, a second group operation request for designating one or more pieces of stroke data, which is the operation target, in accordance with the user B's group selection operation. The second group operation request is another example of the group operation information for designating one or more pieces of stroke data that is the operation target.
The character string ““list”: [“bbbbb”,“ccccc”,“ddddd” ]” indicates that the stroke data, which is the operation target, are the second stroke, the third stroke, and the fourth stroke.
In step S1810 of
In step S1804, the operation restriction process starts on the stroke data with the stroke IDs “aaaaaa”, “bbbbbb”, “cccccc”, “dddddd”, and “eeeeee” based on the group operation information 1910 illustrated in
As described above, when all or part of the designated operation target (stroke data) is included in the operation-restricted operation target (stroke data), the operation restriction unit 57 transmits a rejection notification (or a failure notification) to the electronic whiteboard 1b, which has transmitted the request, in step 1811. In this case, the users B and C of the electronic whiteboard 1b are not allowed to perform operations on the stroke images ST2, ST3, and ST4 that are the operation target.
In step S1812, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST3 (Gid=g003) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST3. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the third drawing data of the third stroke corresponding to the stroke image ST3 (Gid=g003) and updates the Gid to “g008” in the stroke information 5006 illustrated in
In step S1813, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST4 (Gid=g004) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST4. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the fourth drawing data of the fourth stroke corresponding to the stroke image ST4 (Gid=g004) and updates the Gid to “g009” in the stroke information 5006 illustrated in
In step S1814, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, the operation information for moving the stroke image ST5 (Gid=g005) to left by a predetermined distance in accordance with the user A's operation to move the stroke image ST5. Accordingly, for example, the stroke-information management unit 56 of the communication management system 5 updates the fifth drawing data of the fifth stroke corresponding to the stroke image ST5 (Gid=g005) and updates the Gid to “g010” in the stroke information 5006 illustrated in
In step S1815, the operation restriction unit 57 of the communication management system 5 determines that the group operation has been completed and terminates the operation restriction process when, for example, operations have been received for all the stroke data designated as the operation target.
As another example, the operation restriction unit 57 may determine that the group operation has been completed when the termination information for the group operation is received from the electronic whiteboard 1a.
The operation restriction unit 57 may terminate the operation restriction process when a predetermined time has elapsed after the operation restriction process is started. Thus, it is possible to prevent a situation where, for example, the editing for the operation target is disabled after the electronic whiteboard 1a transmits the group operation information to the communication management system 5 and then leaves the session due to a trouble, or the like, before an operation has been completed on all the stroke information that is the operation target.
In step S1816, the operation restriction unit 57 of the communication management system 5 transmits the completion information indicating the completion of the group operation to the electronic whiteboard 1a. The Gid of each piece of stroke data updated during the group operation may be notified to the electronic whiteboard 1a by using the completion information or may be notified to the electronic whiteboard 1a by using, for example, the success notification illustrated in step S1602 in
Due to the above-described process, for example, as illustrated in a display screen 1770 of
In step S2001, the acceptance unit 12 of the electronic whiteboard 1b accepts the operation performed by the user B to move the stroke image ST3 downward by a predetermined distance while the user A performs the group operation on the stroke images ST1 to ST5.
In step S2002, the information transmission unit 16 of the electronic whiteboard 1b transmits, to the communication management system 5, the operation information for moving the stroke image ST3 (Gid=g003) downward by a predetermined distance in accordance with the user B's operation to move the stroke image ST3. However, as the operation on the stroke image ST3 is restricted due to the operation restriction process by the operation restriction unit 57 of the communication management system 5, the stroke-information management unit 56 is not allowed to perform the operation to move the stroke image ST3 (Gid=g003).
In step S2003, the stroke-information management unit 56 of the communication management system 5 transmits, to the electronic whiteboard 1b, the failure notification indicating that the movement of the stroke image ST3 has failed.
As described above, the operation restriction unit 57 according to the present embodiment receives the group operation information from the electronic whiteboard 1 and then restricts the operation from the different electronic whiteboard 1 on the stroke image designated by the group operation information.
Thus, according to the present embodiment, an operation different from an operation intended by a user is prevented from being performed on a stroke image due to operations performed by a plurality of terminal apparatuses during the session in which the plurality of terminal apparatuses shares the stroke image.
A description is given hereinafter of an example in which the communication system 100 includes three electronic whiteboard 1a, electronic whiteboard 1b, and electronic whiteboard 1c, and these three electronic whiteboard 1a, electronic whiteboard 1b, and electronic whiteboard 1c participate in a videoconference in which the same stroke image is shared
In step S2101, the acceptance unit 12 of the electronic whiteboard 1a accepts a group selecting operation from the user A using the electronic whiteboard 1a. In this example, it is assumed that a “page 1” of the electronic whiteboard 1a is selected through the group selection operation. As this example, the operation of selecting a group for designating one or more pieces of stroke data as an operation target can be performed on a page-by-page basis.
In step S2102, in response to the operation of selecting the group by the user A, the information transmission unit 16 of the electronic whiteboard 1a transmits group operation request to the communication management system 5. This group operation request includes the “page 1”, which is information of the selected operation target, and the terminal ID “10aa” of the electronic whiteboard 1a.
In step S2103, in response to receiving the group operation request transmitted from the electronic whiteboard 1a, the operation restriction unit 57 of the communication management system 5 determines whether the “page 1” designated by the group operation request includes an operation-restricted operation target (e.g., stroke data or a page).
When the operation target designated by the group operation request includes no operation-restricted operation target, the operation proceeds to step S2104. By contrast, when the operation target designated by the group operation request includes an operation-restricted operation target, the operation restriction unit 57 of the communication management system 5 stops execution of the processing of step S2104 and subsequent steps, and transmits a rejection notification to the electronic whiteboard 1a, which has transmitted the request. In this example, the following description is given on the assumption that the operation target designated by the group operation request includes no operation-restricted operation target.
In step S2104, the operation restriction unit 57 of the communication management system 5 starts operation restriction process for page 1 designated by the group operation request.
In step S2105, the operation restriction unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request has been received to the electronic whiteboard 1a, which has transmitted the request.
In steps S2106 and S2107, the operation restriction unit 57 of the communication management system 5 according to the second embodiment transmits an operation restriction notification indicating that the operation restriction process has started, to each of the electronic whiteboard 1b and the electronic whiteboard 1c participating in the videoconference. The operation restriction notification includes, for example, information (e.g., “page 1”) on the operation target for which operation is to be restricted and information (e.g., the terminal ID “10aa”) on the electronic whiteboard 1a, which has requested the operation restriction on the operation target. Alternatively, examples of the information on the electronic whiteboard 1a includes a terminal name of the electronic whiteboard 1a and a destination name managed in the terminal management table 800.
In steps S2108 and S2109, in response to receiving the operation restriction notification from the communication management system 5, the display controller 14 of each of the electronic whiteboard 1b and electronic whiteboard 1c displays a display screen 2200, for example, as illustrated in
The message 2201 is an example of display information indicating that the operation is being restricted. Instead of such character string, other examples of the display information indicating that the operation is being restricted include pictograms and icons.
The operation described heretofore enables the user A of the electronic whiteboard 1a to preferentially operate the stroke image included in the operation target “page 1”.
For example, in step S2111, when the user A moves the stroke image ST1 of the page 1 on the electronic whiteboard 1a to the right, the operation proceeds to step S2112.
In step S2112, the information transmission unit 16 of the electronic whiteboard 1a transmits, to the communication management system 5, operation information for moving the stroke image ST1 (stroke 1) of page 1 rightward by a predetermined distance.
In steps S2113 and S2114, the stroke-information management unit 56 of the communication management system 5 transmits operation information for moving the stroke image ST1 (stroke 1) rightward by the predetermined distance to the electronic whiteboard 1b and electronic whiteboard 1c.
Thereby, in steps S2115 and S2116, the stroke image ST1 of page 1 on each of the electronic whiteboard 1b and electronic whiteboard 1c is moved rightward, and the same stroke image is displayed on the page 1 of each the electronic whiteboard 1a, electronic whiteboard 1b and electronic whiteboard 1c.
As described heretofore, according to the communication system 100 according to the second embodiment, an operation different from an operation intended by a user is prevented from being performed on a stroke image due to operations performed by plural electronic whiteboards, e.g., the electronic whiteboard 1a, electronic whiteboard 1b, and electronic whiteboard 1c.
Even after the operation restriction unit 57 of the communication management system 5 starts the operation restriction process in step S2104 of
For example, it is assumed that the user B using the electronic whiteboard 1b moves the stroke image ST1 of the page 1 to the left in step S2301 in
In step S2302, the information transmission unit 16 of the electronic whiteboard 1b transmits, to the communication management system 5, operation information for moving the stroke image ST1 (stroke 1) of the page 1 leftward by a predetermined distance.
However, since the operation on the page 1 has already been restricted by the operation restriction process by the operation restriction unit 57 of the communication management system 5, the stroke-information management unit 56 cannot perform the movement process of the stroke image ST1 of the page 1. In such case, in step S2303, the stroke-information management unit 56 of the communication management system 5 transmits a failure notification indicating that the movement of the stroke image ST1 of the page 1 has failed to the electronic whiteboard 1b.
In step S2304, in response to receiving the failure notification from the communication management system 5, the image processing unit 15 of the electronic whiteboard 1b returns the stroke image ST1 of the page 1 of which movement has failed to the previous position. Thereby, the stroke images of the page 1 displayed on the electronic whiteboard 1a, electronic whiteboard 1b, and electronic whiteboard 1c are maintained in the same state.
In addition, in the second embodiment, when the electronic whiteboard 1b receives a moving operation on the stroke image ST1 of the page 1 while the display screen 2200 as illustrated in
Next, a description is given of an operation performed after step S2305, in which the operation restriction unit 57 of the communication management system 5 transmits a reception notification indicating that the group operation request has been received to the electronic whiteboard 1a, which has transmitted the request.
In steps S2306 and S2307, the operation restriction unit 57 of the communication management system 5 transmits an operation restriction notification indicating that the operation restriction process has started, to each of the electronic whiteboard 1b and the electronic whiteboard 1c participating in the videoconference.
In steps S2308 and S2309, in response to receiving the operation restriction notification from the communication management system 5, the electronic whiteboard 1b and electronic whiteboard 1c prohibit the operation on the stroke image of the page 1, which is an operation restriction target. For example, the display controller 14 of each of the electronic whiteboard 1b and electronic whiteboard 1c displays a display screen 2400 as illustrated in
In the example of
The operation described heretofore prohibits an editing operation on the stroke image of the page 1 from the electronic whiteboard 1b and the electronic whiteboard 1c, thereby the user A of the electronic whiteboard 1a can preferentially operate the stroke image included in the page 1.
Although in the second and third embodiments, the description given above is of an example in which an operation is restricted on the stroke image of the page 1, alternatively, stroke data as a target of operation restriction may be one or more selected stroke images, in substantially the same manner as the first embodiment. The stroke data as a target of the operation restriction may be stroke images of a plurality of pages.
According to one or more embodiments of the present invention, an operation different from an operation intended by a user is prevented from being performed on a stroke image due to operations performed by a plurality of terminal apparatuses during a session in which the plurality of terminal apparatuses shares the stroke image.
In a conference in which plural terminal apparatuses share a stroke image, there is a drawback that, when the plural terminal apparatuses perform operations on the shared stroke image, an operation different from the operation intended by a user is performed on the stroke image in some operating timing.
Such drawback is common to various communication systems that provide a session in which a stroke image is shared, e.g., a remote lesson or a remote medical care as well as the conference system for holding a conference.
According to one or more embodiments of the present disclosure, an operation different from an operation intended by a user is prevented from being performed on a stroke image due to operations performed by a plurality of terminal apparatuses during a session in which the plurality of terminal apparatuses shares a stroke image.
Each of the functions of the described embodiments may be implemented by one or more processing circuits or circuitry. Processing circuitry includes a programmed processor, as a processor includes circuitry. A processing circuit also includes devices such as an application specific integrated circuit (ASIC), digital signal processor (DSP), field programmable gate array (FPGA), and conventional circuit components arranged to perform the recited functions.
In the description according to the above embodiments, the terminal apparatus is the electronic whiteboard 1; however, the embodiments are not limited thereto. For example, each embodiment of the present invention is also applicable to an interactive projector 2501 as illustrated in
Other examples of the terminal apparatus may include an information apparatus such as a PC, a smartphone, a smart watch, or a car navigation device, which executes the application corresponding to the communication system 100. The terminal apparatus includes a medical device. In the case of a medical device, the document image may include images of a patient, etc.
In the description according to each of the embodiments described above, a video conference is held by the communication system 100; however, the embodiments are not limited thereto. The communication system 100 may be used for a meeting, a general conversion in a family or between friends, etc., a remote medical care, or presentation of information in one direction.
The above-described embodiments are illustrative and do not limit the present disclosure. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of the present disclosure.
Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2019-046413 | Mar 2019 | JP | national |
| 2020-037611 | Mar 2020 | JP | national |
This patent application is a Continuation of U.S. application Ser. No. 17/984,271, filed Nov. 10, 2022, which is a Divisional of U.S. application Ser. No. 16/816,260, filed Mar. 12, 2020 (now U.S. Pat. No. 11,539,764), which is based on and claims priority pursuant to 35 U.S.C. § 119(a) to Japanese Patent Application Nos. 2019-046413, filed on Mar. 13, 2019 and 2020-037611, filed on Mar. 5, 2020, in the Japan Patent Office, the entire disclosures of each are hereby incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16816260 | Mar 2020 | US |
| Child | 17984271 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17984271 | Nov 2022 | US |
| Child | 18811801 | US |
