Patent Application
20150253948
1. Field of the Invention
The present invention relates to an application program, and more particularly to an operation instruction reception method and a method of registering operation instruction reception means in an application program that displays a large image and enlarges an image so as to display a partial image while scrolling the partial image.
2. Description of the Related Art
An operation instruction method for issuing operation instructions such as scroll and magnify efficiently has been disclosed in the related art as a method of displaying a large image or enlarging an image so as to display a partial image while scrolling the partial image.
Japanese Patent Application Laid-open No. 2001-265532, for example, discloses a method of performing a scroll operation efficiently using a right hand mouse and a left hand mouse.
Pointing devices such as mice are provided by various manufacturers, and typically require a driver that is compatible with an operating system (also referred to as OS hereafter). An application program is notified of an operation instruction from the pointing device via the driver using a general-purpose notification method (a message, for example) provided by the OS. Hence, the application program need only recognize the general-purpose notification mechanism provided by the OS, and therefore the application program can be developed independently of the requirements of device manufacturers. Further, when a user changes a used device, the application program can remain in use without being modified.
Recently, pointing devices are often connected to a PC using a universal serial bus (USB), and a plurality of pointing devices can be connected to a single PC. As described above, however, the pointing devices are not identified by the application program, and therefore general-purpose operation instructions such as cursor movement, click, and double click can be input using any of the pointing devices. In a typical pointing device use method, however, individual devices are not distinguished, and it is therefore difficult to use different functions of the application program depending on the types of pointing devices.
In Japanese Patent Application Laid-open No. 2001-265532, operation instruction content from each device is determined in a controller or a driver of the left hand mouse.
However, although it is possible with Japanese Patent Application Laid-open No. 2001-265532 to perform a scroll operation efficiently, this operation requires the use of a special pointing device or driver. Therefore, the user may be unable to use a preferred pointing device, and when the user changes the pointing device, it may be necessary to modify the application program itself.
Further, particularly when an attempt is made to allocate functions respectively to a plurality of pointing devices, the functions must be set after identifying the respective pointing devices. Devices connected to a PC can typically be identified from driver information. However, when the devices are connected using a USB, as described above, a plurality of identical devices can be connected, and therefore information indicating a USB port must be identified in addition to information indicating the device itself. Here, the information required for identification is not always described in the pointing device itself. Furthermore, the user cannot easily obtain information identifying the plurality of USB ports provided in the PC. It is therefore difficult to allocate functions respectively to a plurality of pointing devices in accordance with user preferences on an application.
Here, Japanese Patent Application Laid-open No. 2001-265532 discloses a method of allocating functions during driver setting. When this method is used, a driver capable of allocating functions must be provided by the device manufacturer, and it may therefore be impossible for the user to allocate a function to a preferred device.
The present invention has been designed in consideration of the problems described above, and an object thereof is to provide an application program with which operation instructions can be input efficiently using a general-purpose pointing device.
The present invention provides a non-transitory computer readable medium storing an application program causing a computer to execute a method comprising steps of:
inputting an operation instruction using a plurality of pointing devices;
detecting information indicating a used pointing device on the basis of the operation instruction;
determining operation content on the basis of the operation instruction from at least one of the plurality of pointing devices; and
storing information indicating the pointing device corresponding to the operation content,
wherein, in the operation content determining step, the operation content is determined on the basis of the operation instruction, the information indicating the pointing device detected in the device detecting step, and the information indicating the pointing device stored in the device storing step.
The present invention also provides a non-transitory computer readable medium storing an application program causing a computer to execute a method comprising steps of:
inputting a general-purpose operation instruction of an operating system as an operation instruction transmitted from a pointing device;
detecting information indicating the used pointing device on the basis of the operation instruction;
determining operation content on the basis of the operation instruction; and
storing information indicating the pointing device corresponding to the operation content,
wherein, in the operation content determining step, the operation content is determined on the basis of the operation instruction, the information indicating the pointing device detected in the device detecting step, and the information indicating the pointing device stored in the device storing step.
The present invention also provides a non-transitory computer readable medium storing an application program causing a computer to execute a method comprising steps of:
inputting an operation instruction using a plurality of pointing devices;
detecting information indicating a used pointing device on the basis of the operation instruction;
determining operation content on the basis of an operation instruction from at least one of the plurality of pointing devices;
storing information indicating the pointing device corresponding to the operation content; and
displaying a registration window for setting the pointing device stored in the device storing step,
wherein, in the registration window displaying step, the registration window is displayed for each operation content in relation to which the information indicating the pointing device is stored,
in the device detecting step, information indicating the pointing device that issues the operation instruction to the registration window is detected, and
in the device storing step, information indicating the pointing device detected in the device detecting step is stored.
According to the present invention, an application program with which operation instructions can be input efficiently using a general-purpose pointing device can be provided.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Preferred embodiments of the present invention will be described below with reference to the drawings. However, note that dimensions, materials, shapes, relative arrangements, and so on of constituent components described below are to be modified appropriately in accordance with the configuration of the apparatus to which the invention is applied and various conditions, and the scope of the invention is not limited to the following description.
An outline of the present invention will be described below using the drawings.
A reference numeral 102 denotes a mouse serving as one of pointing devices according to the present invention, with which a user inputs an operation instruction. A reference numeral 103 denotes a left side button attached to the mouse 102, which is used to select an object located in a position indicated by a mouse cursor, not shown in the drawing, displayed on the display 101, and so on. A reference numeral 104 denotes a rotatable wheel, which is effective when issuing an operation instruction corresponding to a rotation amount. The wheel 104 may be used in an image display application, for example, to scroll in a vertical direction, modify a display magnification, and so on. A reference numeral 105 denotes a right side button attached to the mouse 102, which is used to display a usable menu or the like in a position indicated by the aforesaid cursor.
A reference numeral 106 denotes a trackball serving as a pointing device with which the position of the mouse cursor can be moved by rotating a ball 107 similarly to the manner in which the position of the mouse cursor is moved by moving the mouse 102. A reference numeral 108 denotes a left side button attached to the trackball 106, which has similar functions to the left side button 103. A reference numeral 109 denotes a right side button having similar functions to the right side button 105. A reference numeral 110 denotes a keyboard used to input alphanumeric characters. Note that the left and right side buttons on the mouse and the trackball are set on the left and right sides when seen from the viewpoint of the user gripping the mouse.
Next, using
In the case of an image such as a pathological diagnosis image, in which it is necessary to observe both the entire subject preparation and an enlarged detailed portion, the image must be scrolled and the display magnification thereof must be modified during observation. When the display magnification is too low, the detailed portion cannot be observed, and therefore, to obtain a favorable diagnosis, the display magnification must be set at least at a predetermined value. When the image is larger than the application window at this predetermined display magnification, a partial image, i.e. a part of the overall image, is displayed in the window, and a display position of the image (a cropped partial image) is modified by performing a scroll operation.
Further, operations may be performed to add marks or comments to a location of interest. A reference numeral 302 denotes a cursor used to specify a position on the display 101. The cursor 302 is moved by issuing a movement instruction using a pointing device, for example by moving the mouse 102 or rotating the ball 107 of the trackball 106.
When an image that does not fit onto a single screen is displayed, as shown in
Here, the mouse 102 and the trackball 106 are sold in various shapes by various manufacturers, and preferred products differ among users. However, when a special device or driver is required, as in Japanese Patent Application Laid-open No. 2001-265532, it is difficult to allocate devices to respective functions, as described above, if the devices and drivers are not compatible therewith.
The present invention makes it possible to use an optimum operation instruction device for each function of an application program without using special devices and drivers.
Here, a standard operation instruction command issued in relation to the mouse cursor 302 using a pointing device is defined as an operation instruction command generalized by the OS 401. Accordingly, information transmitted to the application program 400 from the OS 401 is transmitted as a generalized operation instruction command 404 without distinguishing between the mouse 102 and the trackball 106 serving as operation input devices, or between the drivers 402, 403. A message may be cited as a representative example of the generalized operation instruction command 404, and examples of messages include a message indicating that the mouse 102 has been moved, a message indicating that the left side button 103 has been pressed, and so on.
Further, the application program 400 is normally created independently of the pointing devices so that the application program 400 does not recognize the device from which the operation instruction is input. However, in the application program according to the present invention, the device is specified via the OS 401. In the application program 400, therefore, an operation of the application program 400 can be determined in consideration of both the operation instruction command 404 and information indicating the device from which the operation instruction corresponding to the operation instruction command 404 is input. As a result, an optimum device can be allocated to each function of the application program 400 using a plurality of general-purpose pointing devices.
A reference 501 denotes a message receiver including a message queue in which received messages are accumulated, determination means for determining whether or not a message has arrived, and so on. A reference numeral 502 denotes a device detector that detects identification information indicating the device corresponding to the received message. A reference numeral 503 denotes a device storage that stores the identification information of the corresponding device in association with each function of the application program 400. Note that preset identification information is stored in the device storage 503. Further, the device storage 503 does not have to store identification information in association with all functions, and may store information relating only to a function to be associated with an operation performed by a specific device.
A reference numeral 504 denotes an operation content determiner that determines operation content of the application program 400 on the basis of the received message, the identification information detected by the device detector 502, and the identification information stored in the device storage 503. A reference numeral 505 denotes a scroll controller that executes a scroll operation when the operation content determiner 504 determines that an operation to scroll the displayed image has been issued. A reference numeral 506 denotes a display magnification controller that executes an operation to modify the display magnification when the operation content determiner 504 determines that an instruction to modify the display magnification has been issued.
A reference numeral 507 denotes an image reader that reads and displays a specified image when the operation content determiner 504 determines that an instruction to display a new image has been issued. A reference numeral 508 denotes a termination processor that terminates the application program 400 when the operation content determiner 504 determines that an instruction to terminate the application program 400 has been issued. A reference numeral 509 denotes a cursor display condition controller that controls display of the cursor in accordance with the content determined by the operation content determiner 504.
In step 604, the content of the message is determined, and when the content indicates cursor movement, the processing advances to step 605. In step 605, a determination is made as to whether or not the identification information of the used device matches the identification information of a device used for scrolling. When a match is determined in step 605, a scroll amount is determined in step 606, whereupon scrolling is executed in step 607. In step 608, the cursor is set in a non-displayed condition, whereupon the processing returns to step 601 to wait for a message to be received.
When it is determined in step 605 that the identification information does not match the identification information of the device used for scrolling, it is determined that a normal mouse cursor movement instruction has been issued. The cursor is then set in a displayed condition in step 609, whereupon the processing returns to step 601 in order to wait for a message to be received.
When, in step 604, the message content is not content indicating cursor movement, on the other hand, the processing advances to step 610, where a determination is made as to whether or not the message content is content indicating wheel rotation. When, in step 610, the message content is content indicating wheel rotation, the processing advances to step 611, where the display magnification is determined on the basis of a rotation amount and a rotation direction of the wheel. The display magnification is then modified in step 612, whereupon the processing returns to step 601 in order to wait for a message to be received.
When, in step 610, the message content is not content indicating wheel rotation, the processing advances to step 613, where a determination is made as to whether or not the message content is content for opening a file. When, in step 613, the message content is content for opening a file, the processing advances to step 614, where a specified image file is read, or in other words image input is performed. The read image is then displayed in step 615, whereupon the processing returns to step 601 in order to wait for a message to be received.
When, in step 613, the message content is not content for opening a file, the processing advances to step 616, where a determination is made as to whether or not the message content is content indicating termination of the application program. When, in step 616, the message content is content indicating termination of the application program, the application program is terminated. When the message content is not content indicating termination of the application program, on the other hand, the processing returns to step 601 to wait for a message to be received.
Note that in the flow shown in
Further, in the flow of
In
Note that when a mouse movement instruction from the device not used for scrolling is switched to a mouse movement instruction from the device used for scrolling, the cursor position is stored and the cursor is not displayed. Conversely, when a mouse movement instruction from the device used for scrolling is switched to a mouse movement instruction from the device not used for scrolling, the cursor is preferably moved to the stored cursor position and set in the display condition. In so doing, repeated meaningless movements of the cursor over the screen are not displayed, and therefore an image can be displayed such that viewing of the image is not impaired.
Note that when the operation instruction is switched to a mouse movement instruction from the device not used for scrolling, the cursor position 706 may be moved to the position 705 on the assumption that the scroll operation has stopped. Further, a dead region 708 centering on 705 is preferably provided in the scroll control region. By ensuring that the cursor is not moved when the cursor exists in the region 708, scrolling can be stopped easily.
The movement amount may be set to be proportionate to a relative distance between the reference position and the cursor position. However, the cursor may be moved unintentionally by small movements of the hand. In this case, when the movement amount is simply proportionate to the distance, unevenness may occur in a movement speed and the movement direction, and as a result, it may be difficult to observe the image.
Therefore, a predetermined, fixed movement amount may be set. For example, the movement speed may be set at a single fixed value, and the movement direction may be limited to vertical and horizontal directions or the like. Further, a plurality of fixed values may be set, and the applied fixed value may be switched when the relative distance between the reference position and the cursor position exceeds a predetermined threshold or the like. For example, when the relative cursor position is removed by at least a predetermined value, unevenness is more likely to occur in the scrolling movement due to high speed hand movements, and therefore the movement speed is set at a fixed speed and the movement direction is limited to eight vertical, horizontal, and diagonal directions. Conversely, when the relative cursor position is not removed by at least the predetermined value, the movement amount may be set to be proportionate to the relative distance between the reference position and the cursor position in order to perform fine operations in a sensory manner using low speed hand movements.
In the embodiment described above, a message indicating cursor movement is determined when performing a scroll operation, but the present invention is not limited thereto. For example, when the trackball is used, a rotation amount of the ball must be detected. Accordingly, the cursor position may be obtained at predetermined time intervals using a periodic timer, and a difference between the obtained cursor position and a position obtained during a previous period may be determined as the movement amount. The scroll direction and the scroll amount may then be determined on the basis of the movement amount.
In this case, as described in relation to
Hence, according to the present invention, the cursor position may be obtained (detected) and reflected in the scroll operation on the basis of a different trigger from which cursor movement can be detected instead of the message indicating cursor movement.
Note that the present invention is not limited to the embodiment described above, and the device may be determined likewise when the message content indicates an operation other than cursor movement. For example, in a case where the two pointing devices are both capable of performing typical wheel operations, the device may be determined when the message indicates wheel rotation. In a preferred example of this case, Z-stack images, which are constituted by a plurality of captured pathological diagnosis images of the type described above arranged in a depth direction, are displayed. In this case, the display magnification is preferably modified using one of the pointing devices, and a depth direction position is preferably switched using the other pointing device.
Furthermore, the present invention may be applied to a menu operation such as opening a file. For example, in the case of the pathological diagnosis image described above, a plurality of images may be displayed side by side and compared, or a single image may be displayed. In a preferred example of an operation performed to open a file while a single image is already displayed, when one pointing device is used, the displayed image is closed and a single, newly specified image is displayed, and when the other pointing device is used, the display region is divided and the newly specified image is displayed side by side with the displayed image.
The application subject of the present invention is not limited to a menu operation, and the present invention may be applied to various subjects that are unique to the application 400, such as buttons and list displays. For example, operation content corresponding to a button may be switched in accordance with the used pointing device, or a list of selectable items in a combo box may be switched in accordance with the used pointing device. With the OS 401 and the driver 402, it is difficult to switch operation instructions issued in relation to subjects that are unique to the application, such as menu operations and buttons, in accordance with the pointing device. When conversion software is interposed between the application 400 and the OS 401, it is similarly difficult to switch operation instructions issued in relation to subjects that are unique to the application in accordance with the pointing device. In other words, the present invention makes it possible to switch operation instructions issued in relation to subjects that are unique to the application in accordance with the pointing device.
Furthermore, special operations are not limited to a case in which the trackball 160 is used, and an operation such as adding marks and comments to the image may be performed only when the used device is the mouse 102, for example. In this case, by moving the mouse 102, the mouse cursor 302 is moved over the screen to a position in which a mark or a comment is to be inserted. An operation such as inserting a comment, a character string, or the like is then performed using the left side button 103, the right side button 105, or the keyboard 110. The devices may thus be allocated appropriately in accordance with the functions.
Furthermore, in the above description, the mouse and the trackball are cited as pointing devices, but the present invention is not limited thereto.
Moreover, a touch panel 802 may be installed in a display of the laptop PC 800 so that a position on a screen is specified directly by touching the screen with a finger or the like. A position can be specified with the touch panel 802 in a similar manner to the mouse and so on, and a specified position on the screen may be considered as the current cursor position. Hence, the touch panel 802 constitutes the pointing device serving as the application subject of the present invention.
Similarly to the above description, an example in which the trackball 106 is allocated to scrolling and the touchpad 801 or the touch panel 802 is allocated to another operation, for example, may be considered. Note that in
In the above example, described using
Furthermore, when the cursor is not required in
An example in which only general-purpose pointing devices and drivers are used was described above as a preferred example of the present invention. However, in a case where a combination of a special pointing device and driver and a general-purpose pointing device is used, the method described above may be applied to a specific general-purpose pointing device. A device such as a game controller, in which a large number of buttons is provided and corresponding operation instruction content is allocated by driver setting, may be cited as an example of a special device that is combined with a general-purpose device.
The processing for switching the used device is not limited to the content described above, and may be performed in response to input of various other operation instructions. The executed operation content is likewise not limited to the content described above. Further, the plurality of devices may be connected using a desired method other than a USB connection method or a method of installing the devices in the computer, as described above. Moreover, the devices may be connected either wirelessly or with a wire.
According to this embodiment, described above, an optimum device can be allocated to each function of an application program using a plurality of general-purpose pointing devices. For example, wide-range image display can be realized easily such that both wide-range scroll processing, which is required to move the cursor by a great distance, and accurate pointing processing are possible. Moreover, the present invention can be realized using application program functions and typical messages, and therefore limitations on the operating system, the drivers, middleware, and so on can be reduced, and any desired devices can be used.
In the first embodiment, a method of operating different functions of an application using a plurality of pointing devices was described. In this embodiment, a method of registering the functions in association with the plurality of pointing devices will be described.
A reference 900 denotes a registration operation controller that performs operations to determine a function in association with which a device is to be registered, and store the identification information of the device, input from the device detector 502, in the device storage 503 in association with the function. A reference numeral 901 denotes a screen display content determiner that determines screen display content on the basis of information indicating a function for which registration is to be performed on a current screen, input from the registration operation controller 900. A reference numeral 902 denotes a window display that displays a window on the basis of the screen display content determined by the screen display content determiner 901.
When the processing starts, first, in step 1000, initialization of the registration processing is performed. Next, in step 1001, the processing waits for an instruction to start device registration to be issued. When an instruction to start device registration is issued in step 1001, a start window informing the user that registration is about to begin is displayed in step 1002. In step 1003, the processing waits for a button, described below, to be pressed as an operation instruction for displaying the next screen on the start window, and when the button is pressed, the processing advances to step 1004.
In step 1004, the function to be registered is determined on the next screen, whereupon a registration screen is displayed in step 1005 on the basis of information indicating the determined function. In step 1006, the processing waits for an operation instruction to be input from a pointing device on the registration screen, and when the operation instruction is input, the processing advances to step 1007. In step 1007, the identification information of the pointing device used to input the operation instruction on the registration screen is obtained, and in step 1008, the identification information of the device is stored in association with the function. In step 1009, a determination is made as to whether or not registration is complete in relation to all functions to be registered, and when an unregistered function exists, the processing returns to step 1004 to perform the registration processing on the next function. When it is determined in step 1009 that all functions have been registered, on the other hand, the processing advances to step 1010.
In step 1010, a completion window informing the user that the registration processing is complete is displayed. In step 1011, the processing waits for a button used to close the completion window to be pressed, and when the button is pressed, the processing is terminated.
Next, using
When the button 1102 is pressed using the cursor 1101, the registration window is displayed, as shown in
When the corresponding device has been registered on the registration window shown in
When the corresponding device has been registered on the registration window shown in
Note that the registration start window 1100 and the completion window 1109 do not have to be displayed, and may be displayed as required, or displayed within the registration windows 1103, 1106, and so on.
The display content in the windows 1100, 1103, 1106, 1109 shown in
Here, an example in which device registration is performed in response to a cursor movement instruction on the registration windows shown in
Furthermore, on the registration windows shown in
When devices are registered in association with a function using the method described in this embodiment, depending on the registration content, a defect may occur during execution of an operation. For example, when a single device is registered in association with a plurality of functions, the single device may be allocated to both the scroll operation and the pointing operation. When this type of registration is permitted and the registered device is moved, the cursor may move while scrolling is underway, and as a result, it may be difficult to perform a cursor operation in a desired position. Therefore, a combination of functions for which a single device cannot be registered is preferably stored in advance as an exclusion function. When an attempt is made to register a single device in association with the exclusion function, processing such as issuing a warning display or not executing the registration processing in relation to the device is performed. As a result, prohibited registration can be suppressed.
Note that the device registration method according to this embodiment is described as being executed in the application program 400 according to the first embodiment. However, the present invention is not limited thereto, and the device registration method may be executed in a different application program for registering a file, a registry, or the like storing the devices used by the application program 400, for example. A case in which an executable file with a different executable form of the application is practically used in combination is included within the scope of application of the present invention.
According to this embodiment, described above, an optimum device can be allocated to each function of an application program using a plurality of general-purpose pointing devices.
Embodiment (s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2014-041765, filed on Mar. 4, 2014, and Japanese Patent Application No. 2014-041766, filed on Mar. 4, 2014, which are hereby incorporated by reference wherein in their entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2014-041765 | Mar 2014 | JP | national |
| 2014-041766 | Mar 2014 | JP | national |
