Patent Application
20120302206
Windows based personal computers (PCs) allow the user to simultaneously connect to and view a plurality of information providing services, such as e-mail servers (e.g., corporate email servers), social networking servers (e.g., Facebook™), blogging servers (e.g., Twitter™), webmail servers (e.g. YahooMail™, GMail™), search engines, news servers (e.g., RSS feeds), Instant Messaging (IM) servers (e.g., Jabber™), photo servers (e.g., Shutterfly™, Flikr™), and a plurality of informational web sites (e.g., Wikipedia™). The amount of information available to users is overwhelming. The human brain, although advanced as compared to other animals, can still only assimilate limited amounts of information when presented simultaneously. One recent paper suggests that more than seven pieces of data cannot be assimilated easily, such that degraded performance results when more than seven items are presented simultaneously.
Typically, computer users simultaneously connect to many information sources that each provide multiple pieces of information with real-time updates. For example, an email program may display all recently received emails and update the display, with an optional sound, each time a new email is received for the user. Similarly, an IM server may display messages received from multiple people and immediately display, optionally with a sound, new messages when received for the user.
Where a service obtains financing from advertising (e.g., free services such as email, photos, and IM), the generated display often contains one or more advertisements. Each advertisement typically includes features (e.g., flashing, color, movement, content, etc.) that attempt to get more of the user's attention than the other advertisements; thus, advertisements are particularly distracting for the user. The user's experience is often totally dictated by advertising.
Although, in certain circumstances, advertising may be welcomed by a user, there are other times when it is an unwanted distraction. For example, when informational sources are accessed in a working environment, work productivity is usually reduced because of unwanted information and distractions; often to the point where a company blocks such services entirely from their computer systems.
The disclosure herein involves software and hardware that cooperate to transform the amount of data received through a display such as a computer screen, tablet PC or smart phone. Specifically, people today receive too much information. There are articles written about individual information overload and how too much information limits the ability to make wise choices. See, e.g., Newsweek, I Can't Think, 27 Feb. 2011. The technology of this disclosure can function simply to reduce the amount of data a person is exposed to, for example by removing entirely unwanted or rarely used newsfeeds or advertisements, leaving more pristine, important data to review and enjoy.
The idea is to control what is actually shown on the screen. You interpret social networking information and filter it so only stuff you care about shows up. You delete all ads. It is like a screen dump—it captures the image but manipulates the image quickly so what you see is filtered in desirable ways. You remove unwanted data leaving only “quality” data and prioritize to show it in smaller pieces—that match the 7 item limit of human short term memory. The system still allows interaction with the information sources, and everything is experienced correctly. In an embodiment, the surface software “interprets” what you like and what you don't like in social networking information or any application and automatically ignores the stuff you don't spend time on. This is the ultimate control of user experience.
In another embodiment, Surface Software cooperates with a social network server such that a user's experience on multiple devices is shared and consistent. Specifically, the social network receives and shares information from surface software application running on many different devices, thereby building comprehensive and powerful profiles that may be used by other devices running surface software applications. In certain embodiments, the surface software application and associated functionality are built into communication devices such that the user's experience is automatically configured for that device. Further, the social network server may determine the current status and environment of the user, and thereby configure the surface software accordingly. For example, where the user arrives at a particular location, such as their place of word or a restaurant, each surface software application associated with the user may be appropriately configured for operation at that location at that particular time (i.e., based upon the user current status).
Based upon a user configuration 213, SS service 212 connects to a plurality of information sources 220 to receive information desired by the user. Information sources 220 are accessed via Internet 202, and may be cloud based services and corporate sources of information. For example, information source 220(1) may represent an instant messaging (IM) service, information sources 220(2) and 220(3) may represent social networking servers (e.g., Facebook™, Twitter™), information source 220(4) may represent a webmail service (e.g., Yahoo™ Mail, GMail™), information source 220(5) may represent a search engine (e.g., Google™), information source 220(6) may represent a news server (e.g., Comcast™ News Server), and information source 220(7) may represent a corporate email server. User configuration 213 may define one or more URLs for each information source 220, together with a set of rules that define filters and a priority of the information received from the information source.
SS service 212 filters, orders, and limits information from information sources 220 to generate a flattened and controlled information display 211 for display on the user's PC within browser window 210. PC display 204 may contain other data, and is illustratively shown with a working window 206 that contains application data 208. Working window 206 may represent a display from a word-processing application, for example. By flattening and controlling information presented to the user within controlled information display 211, SS service 212 prevents continuous distractions to the user that typically result from display of information from information sources 220, and also keeps the PC display 204 from becoming overly cluttered with unwanted and unnecessary information. Since SS service 212 also control the update rate of flattened information 230 presented to the user, user distraction is minimized, allowing the user to concentrate on application data 208 without undue distraction.
Each information channel 420 includes an information receiver 422, an information filter and sorter 424, a data comparator 426, an update controller 428 and a channel display generator 430. Information receiver 422 connects to one information source 220, identified by source identifier 408 of channel configuration 407, to receive raw data 440 therefrom. Information filter and sorter 424, based upon content prioritization 410 and data limit 412 of channel configuration 407, processes raw data 440 to produce selected data 442 that contains prioritized, sorted and limited information from information source 220. A user may define source identifier 408 and content prioritization 410, such that SS 402 present information of interest and relevance to the user. The user any have more than one configuration 406 that is selected based upon a status of the user. For example, is the user is at work, configuration 406 may be defined to restrict and prioritize information relevant to the user's work. When at home, the user may switch to a ‘home’ configuration such that SS 402 presents information of interest and relevant to the user's current home environment. In one example, the “work” configuration is implemented and imposed by the user's office policies, thereby restricting the user's access to irrelevant information.
A data comparator 426 compared selected data 442 with previous display data 448 to determine changed data 446. Update controller 428 determines whether changed data 446 contains new information and, based upon an update rate 414 of configuration 406, determines whether display data 448 may be updated with this new information. For example, update rate 414 may define a period such that update controller 428 allows updates to occur once per that defined period. Channel display generator 430 takes display data 448 and generates channel display 450 that may form at least part of information displayed to the user.
In step 464, process 460 filters the received information based upon the defined configuration. In step 466, process 460 sorts the filtered information based upon the configuration. In one example of steps 464 and 466, SS 402 reads content prioritization 410 and data limits 412 of configuration 406 and filters and sorts raw data 440 to produce selected data 442.
Step 468 and 470 are optional. If included, step 468 compares the sorted information with sorted information of the previous iteration. In one example of step 468, data comparator 426 compares selected data 442 with display data 448 and generates changed data 446. Changed data 446 includes information that is selected (filtered and sorted) from the information source and has recently changed. Step 470, if included, is a decision. In step 470, if the information has changed, process 460 continues with step 472; otherwise process 460 terminates.
In step 472, process 460 determines an update rate for the information channel Step 474 is a decision. If, in step 474, process 460 determines that updates are allowed, process 460 continues with step 476; otherwise, process 460 terminates.
In one example of steps 472 and 474, SS 402 reads update rate 414 from configuration 406 to determine whether the changed information should be displayed based upon a time of a previous one or more updates. In one embodiment, updates are restricted to defined periods, such as “break time” or “lunch time” at work. In another example, update may occur at a rate of one per 10 minute period, wherein SS 402 stores the changed information for output. In another embodiment, processing of each information channel 420 occurs only at a defined time.
In step 476, process 460 generates output selected channel information. In one example of step 476, channel display generator 430 of SS 402 generates channel display 450 from display data 448.
Thus, as shown in the embodiment of
SS application 514 runs within processor box 502 to detect when the user runs applications 520 that access information sources 220. SS application 514 automatically switches displayed windows of applications 520 to a second graphics port 513 such that information within these windows is captured by dongle 510 and sent to SS application 514. Dongle 510 connects to second graphic port 513 of processor box 502 and processes a graphical output signal from graphics port 513. In one embodiment, graphics ports 512 and 513 a controlled by a graphics adapter, not shown. In another embodiment, graphics ports 512 and 513 are controlled by separate graphics adapters, not shown.
SS application 514 receives the windows and content information from dongle 510, for example via a USB port 516 of processor box 502 and/or an optional Wi-Fi communication ability. For example, dongle 510 may communicate with SS application 514 via wireless communication port 518 of processor box 502. SS application 514 then determines at least part of information received by application 520 for display on display 504 to the user based upon configuration 515. User configuration 515 may be similar to configuration 406 of
In step 804, process 800 identifies a source of information within the detected window. In one example of step 804, a window title area is identified by dongle 510 and an OCR is used to determine a possible source of information displayed within the window. In another example of step 804, dongle 510 uses graphical image recognition to identify an icon or logo within the detected window.
In step 806, process 800 assigns a unique identifier to the detected window. In one example of step 806, dongle 510 utilizes a unique number, for example determined from a predefined sequence such as a counter, to identify the detected window.
In step 808, process 800 processes the detected window area to generate textual information representation of the information visible within the window. In one example of step 808, dongle 510 utilizes OCR to create a textual representation of information displayed within the window.
In step 810, process 800 detects graphical areas within the detected window. In one example of step 810, dongle 510 determines coordinates, relative to the detected window, of graphical areas within the window.
In step 812, process 800 sends the determined information to the SS application. In one example of step 812, dongle 510 sends the determined window information, the identified source of information for the window (if any), and the textual representation of the window, and any coordinates of graphical areas that appear within the detected window, to SS application 514 via USB port 516, and/or wireless communication port 518.
In step 904, process 900 compares the received information with previously received information. In one example of step 904, SS application 514
The technology of this disclosure employs the term “surface software” because it operates to re-render information shown on a display (e.g., a computer screen). In an embodiment, this re-rendered information is chosen in accordance with preference (e.g., personal preference, preference of the workplace, etc). For example, a user simply may never wish to see advertisements—and the surface software may be configured to stop advertisements. In another example, a user may wish never to see unwanted newsfeeds or daily postings by people through social networks; the surface software may thus operate to obscure or remove such data so that it does not clutter your display of quality information. In a specific example, a user can select a mode such as “workplace” and automatically filter data throughput to data associated with a workplace (of course, the workplace too can beneficially demand that data be associated with workplace too). In another embodiment, this re-rendered information is chosen automatically through interaction with underlying software (e.g., social network applications, news feed applications, etc.). For example, the surface software may be set to monitor and determine the user of a particular computer never views a particular newsfeed; the re-rendered information thus is automatically devoid of such information in an embodiment hereof.
A user's interaction with a display (e.g., with a mouse or keyboard) operates normally. When an interaction occurs with a re-rendered display (made possible by surface software), the interaction is translated to underlying windows (e.g., email, browser, social networks) so that a user can appropriately interact with the application of choice. In an embodiment, a user can select to view ‘native’ applications instead of the re-rendered screen at any time.
SS limits display of information from each background application: e.g., two items from twitter, two items from social networks, three email messages, etc. Conventionally, the applications update as they find new stuff to display, and it can all happen at once. SS prevents more than one area from changing at once. I.e., one new social networking message pops in, then one tweet message pops in a defined period later. The areas can be prioritized, such that social networking is more important, and gets the priority to update over tweets, but is still limited as to when it can update. This prevents overload of the user when too much information is changing on the screen. Obviously, foreground applications, such as word processor, spreadsheets, and so on, get the focus and update as needed.
In one example of operation, all applications that provide information are run on the PC in full screen mode, and the system switched between each application to retrieve information being displayed and to control them. The user doesn't see this switching, but instead sees the flat controlled output of the SS system.
In one embodiment, eye motion is detected to determine a user's desired priority of presented information. In another embodiment, brainwaves are used for determining a user's desired priority of presented information.
An incorporated eye movement detector allows the user to request updates to the displayed information wherein SS may display buffered information from the associated information source, or may request updated information from that information source. SS still filters and prioritizes the information, but may change the priority of the displayed information based upon the user's eye movement or brainwaves.
In one embodiment, brainwaves are monitored to determine if the user is ‘overloaded’ with information, wherein SS controls the update rate of information on the flat screen. I.e., the update rate and the amount of information presented to the user is based upon monitored eye movement and detected brainwaves.
At the Graphics Systems Level
All systems calls from the relevant applications to the graphics system may be intercepted by an SS application running within the computer. This is similar to the dongle approach but without the need for a piece of hardware or OCR. The SS application stops other application from cluttering up the display. SS application may be configured to operate in a wide range of contexts, such as smart phones and other intelligent devices with displays.
Data Stream
The SS application may be extended to intercept data passing between other applications and their servers, e.g. by intercepting the HTML stream for applications using a browser.
Application Mimic
In one embodiment, the SS application provides an alternative and composite user interface for all other applications running on the computer. Where appropriate, the SS application interrogates the relevant serve as if it was the real application. For example, the SS application may determine application setting from the application configuration files and database. This is for example useful with applications such as email.
Other Considerations
Most applications require user interaction before displaying a full range of data; the SS application operates to automate this data acquisition process before filtering and prioritizing the returned information.
When a dongle is used to acquire displayed information, it may also provide encryption functionality, thereby enhancing security of the computer.
In one example of operation, through use of the eye tracking device, the SS application may filter and/or remove information that the user does not spend time looking at. For example, emails from certain people may be substantially ignored, whereas other important messages may be read in detail. The SS application includes software that “learns” what types of information the user is interested in and what types of information the user ignores and filters it out. In another example, the SS application learns which authors/senders of email messages are of interest to the user, and which are not. The SS application then automatically filters out the email messages that do not interest the user.
The SS application may operate to reorder, and filter, data from all window displays and selected information sources. The SS Application (in one embodiment) flattens this information and displays information within a single window, such that it is easier for the user to follow and digest.
When the user is using other application (e.g., word processors, spreadsheets, etc.) the SS window (containing the filtered and reordered information) may be hidden or removed from the user's display. However, this information may be displayed upon command from the user (e.g., using one or more of keystroke, mouse, brainwave detector, and eye movement detector).
The SS application may select the amount of (important) data to display. Recent studies indicate, for example, that limiting the displayed information to “7” items (e.g., seven items per article, or seven pieces of information) is optimal for not overloading the human brain.
The SS application may also operate as a high level time manager. The SS application may determine which windows are used and/or viewed the most, which application the user has open, when the user is in the office, using the phone, and may track when the user is attentive to the computer (working) or not. The SS application integrates “integrates” this collected data to populate a time tracking database to provide “instant” bills (for lawyers and consultants, etc.) that reflect the time spent working much more accurately. The SS application may integrate with other applications such as email wherein it associates time management for a particular email with a particular client and thereby knows where to apply the time spent reading and responding to each email.
Where the SS application is running on a PC, it may optionally be implemented as a screen saver, wherein display of the user selected information from one or more information sources is displayed only when no activity on the PC is detected for a defined period.
In one embodiment, the SS application replaces ads and unwanted data that would normally be displayed by an application with serene scenes or more important data. The SS application may display detected logos from web sites for example, as a defined smaller symbol. Thus, where information is presented to the user, the SS application may include a symbol to indicate the source without displaying the full logo or associated graphic typically found on web sites.
Corporations may use the SS application to reconfigure screens, limit uses, and track real productivity of employees using computers in the office.
The SS application may also provide parental control over displayed information.
The SS application may provide the “last ads” for the display and take over all ad revenue of the other companies.
The displayed screen of the SS application may be ‘swiped’ away easily to get to underlying applications and data. For example, using the mouse, touch screen, eye movement, and/or brainwaves.
The SS application may be implemented with intelligence, such as a neural net, that learns what the user “sees” as important. Over time, operation of SS improves in filtering and ordering information based upon the user's desires. The SS application may provide a selectable display of information that has been filtered out, such as in a “high level” form, thereby allowing the user to “un-filter” items if desired.
The SS application may operate on one or more platforms such as a computer screen, an iPad, a smart phone, and may cooperate between multiple devices used by the same user, and be based upon the user's current status and/or location.
When the SS application is implemented in a dongle, a more restrictive control of displayed information may be provided. This restrictive approach may be more useful to provide parental control of a device used by minors. In another example, a computer used at work may use the SS application to provide a more restrictive control of information displayed by the computer. Since the dongle is a self-contained device, it is not easily tampered with, and may include recognition software to stop access to unauthorized sites.
In one embodiment, the SS application processes HTML received from an information source.
In one embodiment, the SS application interface is implemented in the cloud, and doesn't control the PC. The SS application is a service that is configured with access information of selected information sources (e.g., email, social networking, news sources) and operates to access that service to retrieve information that is presented in a flat, filtered, ordered composite window within a browser on the PC. The SS application may be implemented as at least two different concepts—one controlling the PC entirely to make sure the screen is controlled, and the other to provide a controlled window onto data that is displayed by the PC.
One of the problems with a user's Internet experience is that everyone is trying to attract your attention with advertisements and other information which distracts the user from work. Advertisement in particular are designed to be very distracting in order to attract the user's attention. Information sources attempt to make their information the most noticeable to the viewer. For example, web pages usually contain bright colors, movement, and other attention attracting features that potentially distract the user from their intended operation. An information source that has certain desirable information, may often have other distractions present on the same display (e.g., web page). The SS application operates to filter out the distracting elements while allowing the desired information to be displayed.
In an embodiment, a timing application integrates data from many sources—email application, cell phone database usage information, social networks, desktop phone (landline) data, application data (e.g., word processor, spreadsheet, presentation or other application usage) to automatically track time spent by the user to a particular project. Lawyers and consultants use all these sources all the time but yet still manually insert time to a database for billing purposes. This is inefficient and often inaccurate. Instead, the timing application integrates “use” and “time” data from these sources and collects time spent on the sources by making intelligent choices. For example, if a lawyer spends 25 minutes writing an Outlook email to a client, that time is automatically clocked to that client by noting the time with which the user was active with the Outlook window. If that same lawyer also calls that client by cell, that time too is aggregated and added to the client's time billing. Often databases of modern law firms include data files (e.g., word processing documents) linked to a particular client or matter; the timing application also track the use and time spent on each such data file and attribute the time spent by the user, automatically, to the particular client associated with that file. Later of course the lawyer can review all charges and decide to bill the time or not, but at least the time is collected efficiently.
In another embodiment, the SS application “flattens” a user's experience. Interaction with the display simultaneously activates and interacts with separate application windows (e.g., email, social network, word processor, browser). The flatness simplifies the user experience, making the user more productive.
SS may utilize eye tracking to determine which particular information the user is assimilating, and thereby track time spent by the user on that information. Where this information is associated with a project, the user's time may be booked to that project.
In one embodiment, corporate policy may dictate the information that may be displayed by the SS application to the user while the user is at work.
In another example of operation, each SS application/module may send information to the social networking server 1004 indicating the activity of the user. Where this information indicates an operation that is associated with a billable function, such as when having a conversation with a client on the office phone or smart phone for example, time data 1008 may be recorded within the social networking server 1004. In another example, when the user is working on the office PC to read an email from the client, SS module./application 1002 may send that information to the social networking server 1004. In another example, where the user is working within a word-processing application on a file that is associated with the client, SS module/application 1002 may send information of the file and time spent within the word-processing application to the social networking server 1004. The social networking server 1004 may process the time data 1008 to generate a detailed time breakdown of the user's activities, which may then be used to accurately determine the time spent working on particular clients. Since the SS module/application 1002 is also operating on other devices used by the user, when work is performed outside the office, this time may also be accurately tracked within the social networking server 1004.
In another embodiment, the time data is collected within the social networking server 1004, and then sent to the SS application 1002 for determining appropriate allocation of the time to billing matters.
In one embodiment, where the SS application 1002 includes input from an eye tracking device, the amount of time spent by the user on information displayed on the personal computer may be accurately tracked, such that time is allocated to billing matters associated with that information. For example, although the user has a word processing window open on the office display, his attention may be focused on a different part of the screen, which is associated with a different client, or with non-billable information.
In one embodiment, a time keeping application and/or module is inserted into one or more of a mobile phone (e.g., as an app), an email program, a word processor, a spreadsheet program, a database program, and other software applications and devices. The time keeping module operates to determine a client associated with the work being performed by the software and devices when used by the user. Optionally, the module collect data associated with the user activities, and the client is determined later. In one example of operation, when the user is reading an email from a particular person, the time keeping module may determine a client associated with that person, such that time spent by the user within the email program on that particular email is billed to the appropriate client. Each time tracking module communicates with a server (e.g., a social network server) where the tracked time information is stored. The stored time information is processed (e.g., within the server, and/or exported to a separate program for processing) to determine billing information for the user.
Changes may be made in the above methods and systems without departing from the scope hereof. It should thus be noted that the matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense. The following claims are intended to cover all generic and specific features described herein, as well as all statements of the scope of the present method and system, which, as a matter of language, might be said to fall therebetween.
This application claims priority to U.S. Patent Application Ser. No. 61/490,943, filed May 27, 2011, which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61490943 | May 2011 | US |
