The invention generally relates to screen sharing and more specifically to focusing attention to a specific area of the shared screen.
One example of a screen sharing application is the GOTOMEETING service offered for use by Citrix Online Division of Santa Barbara, Calif. During a screen sharing session, it is often desirable to focus attention to a specific area of the shared screen. One way to focus attention is to verbally explain the location to the other attendees of the screen sharing session. Another way is to move a pointing device, such as a cursor, to the desired area and rapidly moving the pointing device within a relatively small and constrained area.
Each of these solutions may not achieve the desired focusing effect for a number of reasons. For example, it may be difficult to accurately explain the precise location of interest to all the participants. Additionally, it may be difficult for each of the participants to focus on a rapidly moving pointing device for one reason or another.
At least one screen sharing solution features a “laser pointer” that highlights an area of a shared screen. The laser pointer is captured along with the other elements of the shared screen and transmitted to the participants of the screen sharing session as a portion of an image. As the laser pointer is moved about the shared screen, its position updated on the screens of the other participants of the screen sharing session. The motion of the laser pointer may appear jerky (i.e., not smooth) because the motion of the laser pointer is treated as a change to the shared screen image.
In one aspect, the invention features a method of highlighting a portion of a display generated by a first device and sharing the display with a second device. The method includes associating a substantially transparent window with the location of a cursor to position the cursor within the substantially transparent window and changing the appearance of the cursor to highlight a portion of the display.
In one embodiment, the method includes moving the location of the substantially transparent window as the position of the cursor changes so the cursor remains associated with the substantially transparent window.
In other embodiments, the method includes intercepting a mouse event from an input device prior to passing the event to an application executing in the shared windows environment, changing the appearance of the cursor comprises changing the appearance of the cursor to a translucent disk, or determining the number of resolution bits used generate the display of the first device and selecting the appearance of the cursor in response to the number of resolution bits used.
In still other embodiments, the associating and cursor change occur at each of the first device and the second device or controlling the z-order of the substantially transparent window relative to other windows of the display generated by the first device.
In another aspect, the invention features a system for highlighting a portion of a display generated by a first device and shared with a second device. The system includes means for associating a substantially transparent window with the location of a cursor to position the cursor within the substantially transparent window and means for changing the appearance of the cursor to highlight a portion of the display.
In still another aspect, the invention features a system for highlighting a portion of a display generated by a first device and being shared with a second device. The system includes an associator module and a display generator. The associator module is in communication with an input device, receives input commands, and locates a substantially transparent window behind a cursor. The display generator is in communication with associator module and changes the appearance of the cursor to highlight a portion of the display.
In one embodiment, the system includes a hooking module in communication with the input device. The hooking module monitors the movements of the location of the substantially transparent window as the position of the cursor changes such that the cursor remains associated with the substantially transparent window. The hooking module can also intercept a mouse event from the input device prior to passing the event to an application executing in the shared windows environment.
In another embodiment, the system also includes a determination module in communication with the display generator. The determination module determines the number of resolution bits used to generate the display of the first device and selects the appearance of the cursor in response to the number of resolution bits used.
In still another embodiment, the associator module controls the z-order of the substantially transparent window relative to other windows of the display generated by the first device.
The above and further advantages of this invention may be better understood by referring to the following description in conjunction with the accompanying drawings, in which like numerals indicate like structural elements and features in various figures. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
As an overview, one aspect of the invention replaces the current cursor of a presenter computing system that is participating in the screen sharing session with an oversized translucent cursor (also referred to as a replacement cursor or custom cursor through the specification) and transmits the translucent cursor to each participating viewer system. The resolution of oversized translucent cursor is determined on system-by-system basis. On a system where the display settings are set to thirty-two bits per pixel (highest quality, millions of colors) one embodiment of the oversized translucent cursor is represented as a thirty-two bit forty-eight pixel by forty-eight pixel cursor. On sixteen bit and eight bit systems, where thirty-two bit cursors are not supported, the current cursor of the shared screen is replaced with a replacement cursor that is a disc of dots having a size similar to the translucent disk. The replacement cursor aids the participants of the screen sharing session in focusing their attention to the desired area of the shared screen.
The invention displays replacement cursor regardless of which window in the shared screen the replacement cursor hovers over. In one embodiment, the invention uses low-level mouse hooks to intercept mouse events before they are passed to other applications of the shared environment. The invention generates and positions a “cursor helper” window directly under the replacement cursor. As used herein, a “cursor helper window” refers to a transparent window. As the replacement cursor moves, the position of the cursor helper window is continually updated so that the cursor helper window remains directly under the replacement cursor. Additionally, the cursor helper window can be sized to substantially cover the shared display. Also, the correct Z-order ordering of the windows of the shared screen is maintained. The cursor helper window is associated with the replacement cursor such that the cursor helper window is on top (in front) of the other windows on the desktop of the shared screen, but below (behind) the windows of the shared screen user interface.
Referring now to
The network connections 160, 170, 180 between the consumer nodes 150, 150′, 150″ and the source node 100 can be local area networks (LAN), metropolitan area networks (MAN), or a wide area network (WAN) such as the Internet. The source node 100 and the consumer nodes 150, 150′, 150″ may connect to the networks 160, 170, 180 through a variety of connections including standard telephone lines, LAN or WAN links (e.g., T1, T3, 56 kb, X.25), broadband connections (ISDN, Frame Relay, ATM), and wireless connections. Connections between the source node 100 and the consumer nodes 150, 159′, 150″ may use a variety of data-link layer communication protocols (e.g., TCP/IP, IPX, SPX, NetBIOS, NetBEUI, SMB, Ethernet, ARCNET, Fiber Distributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 802.11a, IEE 802.11b, IEEE 802.11g and direct asynchronous connections). Although shown in
In many embodiments, the source node 100 and the consumer nodes 150, 150′, 150″ are provided as personal computer or computer servers, of the sort manufactured by the Hewlett-Packard Corporation of Palo Alto, Calif. or the Dell Corporation of Round Rock, Tex.
The central processing unit 202 is any logic circuitry that responds to and processes instructions fetched from the main memory unit 204. In many embodiments, the central processing unit is provided by a microprocessor unit, such as: the 8088, the 80286, the 80386, the 80486, the Pentium, Pentium Pro, the Pentium II, the Celeron, or the Xeon processor, all of which are manufactured by Intel Corporation of Mountain View, Calif.; the 68000, the 68010, the 68020, the 68030, the 68040, the PowerPC 601, the PowerPC604, the PowerPC604e, the MPC603e, the MPC603ei, the MPC603ev, the MPC603r, the MPC603p, the MPC740, the MPC745, the MPC750, the MPC755, the MPC7400, the MPC7410, the MPC7441, the MPC7445, the MPC7447, the MPC7450, the MPC7451, the MPC7455, the MPC7457 processor, all of which are manufactured by Motorola Corporation of Schaumburg, Ill.; the Crusoe TM3800, the Crusoe TM5600, the Crusoe TM5500, the Crusoe TM5400, the Efficeon TM8600, the Efficeon TM8300, or the Efficeon TM8620 processor, manufactured by Transmeta Corporation of Santa Clara, Calif.; the RS/6000 processor, the RS64, the RS 64 II, the P2SC, the POWER3, the RS64 III, the POWER3-II, the RS 64 IV, the POWER4, the POWER4+, the POWER5, or the POWER6 processor, all of which are manufactured by International Business Machines of White Plains, N.Y.; or the AMD Opteron, the AMD Athalon 64 FX, the AMD Athalon, or the AMD Duron processor, manufactured by Advanced Micro Devices of Sunnyvale, Calif.
Main memory unit 204 may be one or more memory chips capable of storing data and allowing any storage location to be directly accessed by the microprocessor 202, such as Static random access memory (SRAM), Burst SRAM or SynchBurst SRAM (BSRAM), Dynamic random access memory (DRAM), Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended Data Output RAM (EDO RAM), Extended Data Output DRAM (EDO DRAM), Burst Extended Data Output DRAM (BEDO DRAM), Enhanced DRAM (EDRAM), Synchronous DRAM (SDRAM), JEDEC SRAM, PC100 SDRAM, Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM), Direct Rambus DRAM (DRDRAM), or Ferroelectric RAM (FRAM).
In the embodiment shown in
In the embodiment shown in
A wide variety of I/O devices 230 may be present in the computer system 200. Input devices include keyboards, mice, trackpads, trackballs, microphones, and drawing tablets. Output devices include video displays, speakers, inkjet printers, laser printers, and dye-sublimation printers. An I/O device may also provide mass storage for the computer system 200 such as a hard disk drive, a floppy disk drive for receiving floppy disks such as 3.5-inch, 5.25-inch disks or ZIP disks, a CD-ROM drive, a CD-R/RW drive, a DVD-ROM drive, tape drives of various formats, and USB storage devices such as the USB Flash Drive line of devices manufactured by Twintech Industry, Inc. of Los Alamitos, Calif.
In further embodiments, an I/O device 230 may be a bridge between the system bus 220 and an external communication bus, such as a USB bus, an Apple Desktop Bus, an RS-232 serial connection, a SCSI bus, a FireWire bus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, a Gigabit Ethernet bus, an Asynchronous Transfer Mode bus, a HIPPI bus, a Super HIPPI bus, a SerialPlus bus, a SCI/LAMP bus, a FibreChannel bus, or a Serial Attached small computer system interface bus.
General-purpose desktop computers of the sort depicted in
In some embodiments the consumer node 150, 150′, 150″ is a mobile device, such as a JAVA-enabled cellular telephone or personal digital assistant (PDA), such as the i50sx, i55sr, i58sr, i85s, i88s, i90c, i95cl, or the im11000, all of which are manufactured by Motorola Corp. of Schaumburg, Ill., the 6035 or the 7135, manufactured by Kyocera of Kyoto, Japan, or the i300 or i330, manufactured by Samsung Electronics Co., Ltd., of Seoul, Korea. In other embodiments in which the client device 140 is mobile, it may be a personal digital assistant (PDA), such as the Tungsten W, the VII, the VIIX, the i705, or a combination PDA/telephone device such as the Treo 180, Treo 270 or Treo 600, all of which are manufactured by palmOne, Inc. of Milpitas, Calif.
In these embodiments, the consumer nodes 150, 150′, 150″ connect to the source node 100 using any one of a number of well-known protocols from the GSM or CDMA families, such as W-CDMA. These protocols support commercial wireless communication services and W-CDMA, in particular is the underlying protocol supporting i-Mode and mMode services, offered by NTT DoCoMo.
With reference to
The user of the source node 100 may choose to share the desktop environment 500 the user of the consumer node 150, which can include all or some of the desktop environment elements 504. In one embodiment, the user of the source node 100 chooses to share the display 310 to enable collaboration with the user of the consumer node 150. In another embodiment, the user of the source node 100 chooses to share the display 310 to enable training or troubleshooting. It should be understood that in some embodiments, the source node 100 may pass control to the viewer node 150. As such, the roles of source node 100 and viewer node 150 are reversed.
With reference to
With reference to
In certain operating system environments, the standard cursor behavior is for the shape of the cursor to be determined by the application window that has focus. As such, when the cursor is moved, it is possible that the cursor will change to another shape or size. Once activated, it is desirable to keep showing the replacement cursor 516 in order to focus attention to the desired area regardless of which window the replacement cursor 516 is hovering over. To achieve this, in one embodiment, a cursor helper window 524 is associated with the replacement cursor 516. In one embodiment, the cursor helper window 524 is sized such that the replacement cursor 516 is always within its boundaries. In another embodiment, the position of the cursor helper window is continually updated so that the cursor helper window remains directly under the replacement cursor 516. Also, the correct Z-order of the window of the shared display is maintained. In one embodiment, the cursor helper window remains above (in-front) all windows of the desktop environment 500, but below (behind) the windows of the screen sharing application user interface.
In one embodiment, the source node 100 includes an input device 710, a hooking module 720, a transceiver 730, one or more applications 740 (e.g., the screen sharing application), and a display generator 750. The hooking module 720 is in communication with the input module 710 and the application 740. The transceiver module 730 is in communication with the application 740 and the network 700. The application 740 is also in communication with the display generator 750. In one embodiment, the hooking module 720, the transceiver 730, and display generator 740 can be implemented as a software such as an application, a module, a service, a computer program, a software component, a web service, a web component, a web page, a library, a function, a script, an interpreted language, or any other type and/or form of executable instruction.
In one embodiment, the viewer node 150 includes an input device 710′, a hooking module 720′, a transceiver 730′, one or more applications 740′ (e.g., the screen sharing application), and a display generator 750′. The hooking module 720′ is in communication with the input module 710′ and the application 740′. The transceiver module 730′ is in communication with the application 740′ and the network 700′. The application 740′ is also in communication with the display generator 750′. In one embodiment, the hooking module 720′, the transceiver 730′, and display generator 740′ can be implemented as a software such as an application, a module, a service, a computer program, a software component, a web service, a web component, a web page, a library, a function, a script, an interpreted language, or any other type and/or form of executable instruction. The hooking module 720′ generally does not hook mouse commands from the viewer node 150 unless the viewer node 150 becomes the source node 100 during the screen sharing session.
With reference to
In one embodiment, the user selects a button 510 of the control panel 506 of the screen sharing application. In one embodiment, the selected button 510 provides the user with a plurality of tools capable of annotating the shared desktop environment 500. Upon selection of an annotation tool (e.g., the spotlight tool), the screen sharing application enters the annotation mode (step 810). In other embodiments, a keyboard shortcut, hotkey, voice command, or other input event can select the button 510 and activate the annotation mode.
After activation of the annotation mode, the screen sharing application begins hooking (step 820) mouse commands. In one embodiment, a WINDOWS programming command such as WH_MOUSE_LL is used to hook the mouse commands. The WH_MOUSE_LL hook command provides monitoring of mouse input events about to be posted in a thread input queue.
The hooking module 720 receives (step 830) input commands from the input device 710. In one embodiment, the input commands are click and hold events of the left mouse button along with other mouse event such as right clicks, left clicks, and mouse move events. In other embodiments, the input commands can include voice commands, pen clicks, and the like.
After receiving input, the screen sharing application positions (step 840) a substantially transparent (“cursor helper”) window 524 directly below the cursor. In one embodiment, a transparent window 524 is generated solely at the source node 100. In another embodiment, a transparent window is generated at the source node 100 and each viewer node 150. Said another way, in this embodiment the transparent window 524 is local to each system executing the screen sharing application. The substantially transparent window 524 is a layered window as defined in WINDOWS programming. For example, programming commands such as SetLayeredWindowAttributes and UpdatedLayeredWindowAttibute provide a means to control the transparency of the layered windows.
A determination module (not shown) of the application 740 determines (step 850) the display setting for the source node 100. In one embodiment, the display settings are read from a memory location or graphics card storing the requested information. In other embodiments, the information is inputted manually by the user of screen sharing application.
The application 740 replaces (step 860) the present cursor with a custom cursor 516 at the source node as previously described. In one embodiment, if the display settings of the source node 100 indicate that thirty-two bits per pixel are in use then the custom cursor 516 is a thirty-two bit red translucent disk that is forty-eight pixels by forty-eight pixels. If the display settings of the system indicate sixteen or eight bits per pixel are in use, then the custom cursor 516′ is a similarly sized disc of dots. In other embodiments, the user is able to select a desired custom cursor 516 and control the transparency of the custom cursor 516. In still another embodiment, the custom cursor is selected automatically in response to the information being shared. For example, if the source node 100 is sharing a presentation, the custom cursor 516 is selected to provide an appropriate level of highlighting.
The screen sharing application associates (step 870) the transparent window 524 and the custom cursor 516 with one another. In one embodiment, the custom cursor shape 516 is specified as the default cursor shape for the cursor helper window 524. If the size of the cursor helper window 524 is such that is covers the entire desktop environment, the custom cursor 516 will be continually displayed regardless of its location. In another embodiment, a smaller cursor helper window 524 is used and moves as the custom cursor 516 is moved.
As the user of the source node 100 manipulates the input device, the screen sharing application monitors (steps 880) changes to the position of the custom cursor 516. The x and y coordinates of the mouse are captured using the mouse hooking commands as previously described. In one embodiment, the hooking module 720 captures the position and changes in position of the input device 710.
In one embodiment, the transceiver 740 transmits (step 890) the custom cursor 516 and its position to the each of the viewer nodes 150 and through the network 704. In another embodiment, the actual x and y position is transmitted to viewer node 150 and each viewer node 150 receives the updated input position information and updates (step 900) the position of the local custom cursor 516 and the local cursor helper window 524.
The described systems and methods can be used to implement a system for sharing screen data that allows several viewer systems to display and highlight the screen data from a single presenter system. This system is useful in a number of broadcast or “multicast” contexts and, in particular, it is useful in a conferencing context to allow multiple individuals to view the same graphical data during the conference.
As shown in
At a second point in time, the presenter system's screen 1510 has changed. The presenter system 100 identifies the particular tiles that have changed states, and creates a coded packet for each tile that has changed, i.e., data packets 19, 20, 21, and 22. If the presenter system 100 did not possess a transmission token but now receives one, the presenter system 100 will transmit the updated twelve data packets to the communications server 350, i.e., data packets 13, 14, 3, 4, 15, 19, 20, 17, 21, 22, and 12. If the source server has already transmitted the data packets representing the state of the previous screen 1500, then the presenter system 100 need only transmit to the communications server 350 data packets 19, 20, 21, and 22. In this manner, transmission of screen data between the presenter system 100 and the communications server 350 is performed in a bandwidth-adaptive manner.
In some embodiments, the presenter system 100 encrypts the data packets transmitted to the communications server 350. In other embodiments, the presenter system 100 compresses the data packets sent to the communications server 350. In still other embodiments, the presenter system 100 both encrypts and compresses the data packets.
In one embodiment, in addition to sending screen data through the communication server 350, the present invention can establish a virtual channel 1600 transmit the custom cursor 516, cursor helper window commands, and input device position information to each of the viewer nodes 150. In one embodiment, the virtual channel 1600 establishes a direct connection with each viewer node 150. In other embodiments, the virtual channel 1600 is established with the communication server 350, which in turn, broadcasts the custom cursor 516, cursor helper window commands, an the input device position information to each of the viewer nodes 150.
In many embodiments, the communications server 350 maintains a copy of each tile that comprises the most recent state of the server node screen. In some embodiments, each tile is associated with a timestamp when transmitted to the communication server 350. In other embodiments, each tile is associated with a number that monotonically increases with each new tile transmitted to the communications server 350.
The communications server 350 composes an update for a viewer node 150 as often as the bandwidth of the network connecting the viewer node 150 to the communications server 350 allows. As shown in
In another embodiment, the communication server 350 responds to an update request from the viewer node 150 by transmitting to the viewer node 150 every data packet having a timestamp newer than the timestamp of the viewer's screen. In some of these embodiments, the communication server 350 does not fully receive and store a set of data packets comprising a screen update before sending the update to the viewer node 150. In these embodiments, the communications server 350 sets the timestamp for each packet identified by metadata information as comprising the screen update to the same value. Then, as data packets arrive the communications server 350 streams those packets to the viewer node 150.
In one particular embodiment, metadata information is formatted into packets and metadata packets are associated with monotonically increasing numbers. As described above, each metadata packet describes the set of tiles comprising the current screen display state. In this embodiment, the communications server 350 stores, for each consumer node 150, the number of the latest metadata packet that has been transmitted to that consumer node 150, as well as the set of all data packets that have been delivered to the consumer node. When the communications server 350 determines that it is time to send an update to a consumer node 150, or upon receiving a request from a consumer node 150 for a screen update, the communications service first determines if the latest metadata packet (that is, the metadata packet having the highest number associated with it) has been transmitted to the consumer node 150. If not, the communications server 350 transmits the most recent metadata packet to the consumer node 150. The communications server 350 also transmits the set of data packets identified by the metadata packet, unless a particular data packet has already been transmitted to the consumer node 150.
The previously described embodiments may be implemented as a method, apparatus or article of manufacture using programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The term “article of manufacture” as used herein is intended to encompass code or logic accessible from and embedded in one or more computer-readable devices, firmware, programmable logic, memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.), hardware (e.g., integrated circuit chip, Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), etc.), electronic devices, a computer readable non-volatile storage unit (e.g., CD-ROM, floppy disk, hard disk drive, etc.), a file server providing access to the programs via a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc. The article of manufacture includes hardware logic as well as software or programmable code embedded in a computer readable medium that is executed by a processor. Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the present invention.
Although the present invention has been described with reference to specific details, it is not intended that such details should be regarded as limitations upon the scope of the invention, except as and to the extent that they are included in the accompanying claims.