Modern cellular phones, notebook computers, tablets, and other electronic devices enable users to consume a wide array of information available on the Internet through their respective electronic devices. For example, such devices may operate a variety of different applications including news applications, blog applications, social media applications, mixed applications, search engines, and other applications through which the user may consume content originating from different webpages or other sources.
This disclosure describes, in part, techniques for identifying webpage content for later recall and rendering. Example methods of the present disclosure may include, among other things, rendering webpage content on a display, and capturing an image, such as a screenshot, of at least a portion of the rendered content. Such methods may also include sending and/or otherwise providing the captured image to one or more remote devices. Such remote devices may include, for example, one or more cloud-based service providers, remotely-located (e.g., cloud-based) servers, and/or other devices operably connected to the electronic device via the Internet or other networks. At least partially in response to receiving the captured image, the remote device may process the received image using optical character recognition or other techniques to recognize text, symbols, characters, and the like included in the captured image.
In some examples, the remote device may also form a plurality of text groups based on the text included in the captured image. For instance, the remote device may merge, separate and/or otherwise group adjacent lines and/or other portions of the recognized text according to one or more predetermined text grouping rules. The remote device may also generate a plurality of search queries based on the recognized text. The searches may each yield respective search results that include a plurality of webpage links. The remote device may also identify at least one of the webpage links as being indicative of a webpage or other forms of electronic documents (e.g., PDF, slideshows, manuals, medical records, etc.) that include the original webpage content rendered on the display and consumed by the user. In some examples, the remote device may also generate a content item using content from the identified webpage and/or other identified electronic documents. Once such a content item has been generated, the remote device may send and/or otherwise provide the content item, and/or a link to the content item, to the electronic device in response to a request received via the electronic device.
This Summary is provided to introduce a selection of concepts in a simplified form that is further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same reference numbers in different figures indicates similar or identical items.
The present disclosure describes, among other things, techniques for recalling and rendering webpage content. For example, users of electronic devices may consume webpage content using a variety of different applications. Such applications may enable the user to consume webpage content from a wide array of disparate sources, and such sources may have differing formats, protocols, and/or other configurations. For example, various content sources may employ formats presenting webpage content to the user in the form of a blog, message board, newspaper, journal, or magazine articles, book format, eBook format, graphical format (e.g., a comic book, diagram, map, etc.), or other configurations. However, as time passes it may be difficult for a user to recall, for example, the source of particular webpage content that was of interest to the user. As a result, users may struggle to revisit such content once the content is no longer being rendered on the electronic device. Further, although applications exist that enable the user to save portions of articles or other webpage content, such applications are not universally supported among all application providers or in all countries
Example devices of the present disclosure may enable the user to capture a screenshot or other image of the webpage content of interest via, for example, an image capture or screenshot application operable on the device. In some examples, such image capture or screenshot applications are included as standard applications or operating systems on electronic devices configured to render webpage content. As a result, example methods or devices of the present disclosure may enable the user to store and/or share webpage content regardless of the source or format of the webpage content being rendered by the device. In further examples, devices of the present disclosure may enable a use to capture a photograph of a physical content item such as, for example, a magazine article, a journal article, a book, and the like. In such examples, the physical content item may be indexed and/or otherwise searchable via a search engine, and may thus be recoverable by example methods described herein.
In some examples, the user may save the image locally on the device and/or on a cloud-based or otherwise remote service provider. The device or the service provider may recognize text included in the captured image and may form one or more text groups using the recognized text. While various examples of text recognition are described herein, the present disclosure should not be interpreted as being limited to the use of recognized text. For instance, in some examples numbers, symbols, characters, images, and the like may be recognized in the captured image instead of or in addition to text. Thus, in such examples, recognized text may include any type of content recognized in the captured image, and the recognized text may include numbers and/or other characters. In some examples, the recognized text in various text groups may be used to generate one or more searches, such as internet searches, directed towards finding the source webpage on which the originally rendered webpage content resides. In such examples, the one or more text groups formed utilizing the recognized text may be tailored to increase the accuracy of the results yielded by the searches described herein.
The electronic device and/or the service provider may also identify at least one search result indicative of a webpage that includes the originally rendered webpage content. For example, such a search result may be identified by virtue of being included in a predetermined number (e.g., a majority) of the results of the various searches. Additionally, in some examples, such a search result may be identified by virtue of having a relatively high score or other metric indicative of a correlation between the search query used in the respective internet search and content included on the webpage corresponding to the identified search result. Additionally or alternatively, in some examples a search result may be identified by virtue of a determined similarity between a title, URL, snippet, or other content identified in the screenshot and a corresponding title, URL, snippet, or other content of the search result returned by the one or more searches.
In some examples, the electronic device and/or the service provider may generate a content item using content from the webpage corresponding to the identified search result. In some examples, the content item may comprise a version of the website in modified form. For example, such a content item may be optimized for rendering on the display of the electronic device. The content item may be rendered on the device in response to a request received from the user.
The techniques and systems described herein may be implemented in a number of ways. Example implementations are provided below with reference to the following figures.
The electronic device 104 may represent any machine or other device configured to execute and/or otherwise carry out a set of instructions. In some examples, such an electronic device 104 may comprise a stationary computing device or a mobile computing device. For example, a stationary computing device 104 may comprise, among other things, a desktop computer, a game console, a server, a plurality of linked servers, and the like. A mobile computing device 104 may comprise, among other things, a laptop computer, a smart phone, an electronic reader device, a mobile handset, a personal digital assistant (PDA), a portable navigation device, a portable gaming device, a tablet computer, a portable media player, a smart watch and/or other wearable computing device, and so on. The electronic device 104 may be equipped with one or more processors 104a, computer readable media (CRM) 104b, input/output interfaces 104c, input/output devices 104d, communication interfaces 104e, displays, sensors, and/or other components. Additionally, the CRM 104b of the electronic device 104 may include, among other things, a webpage content storage and review framework 104f Some of these example components are shown schematically in
As shown in
In some examples, the electronic device 104 and/or the service provider 106 may access digital content via the network 108. For example, the electronic device 104 may access various websites via the network 108, and may, thus, access associated webpage content 110 shown on the website. Such webpage content 110 may be, for example, content that is available on respective webpages of the website. Such webpage content 110 may include, among other things, text, graphics, figures, numbers (such as serial numbers), characters, titles, snippets, URLs, charts, streaming audio or video, hyperlinks, executable files, media files, or other content capable of being accessed via, for example, the internet or other networks 108. In some examples, the webpage content 110 may comprise eBooks, magazine articles, newspaper articles, journal articles, white papers, social media posts, blog posts, PDFs, slideshows, manuals, health metrics (e.g., medical records personal to the user, or other such information accessible in accordance with relevant privacy laws), or other forms of electronic documents or other content published online. Such webpage content 110 may be accessed by the electronic device 104 via one or more internet browsers, search engines, applications, and/or other hardware or software associated with the electronic device 104. Additionally, such webpage content 110 may be accessed by the service provider 106 via one or more internet browsers, search engines, applications, and/or other hardware or software associated with the electronic device 104. For example, such webpage content 110 may be accessed using one or more news applications, blog applications, social media applications, email applications, search engines, and/or applications configured to provide access to a mixture of news, blogs, social media, search engines, and the like. In some examples, the webpage content 110 may include publicly available content that is freely accessible via the internet or other networks. In additional examples, the webpage content 110 may include privately available content that is accessible only to particular individual users 102 (e.g., users 102 that are employees of an organization, members of a club, etc.). In further examples, the webpage content 110 may include content that is accessible by subscription only (e.g., magazine subscription, newspaper subscription, search service subscription, etc.). In examples in which the webpage content 110 includes privately available content or content that is accessible by subscription only, the service provider 106 may also have access to such webpage content 110, such as via a subscription, license, seat, membership, etc. that is shared between the user 102 and the service provider 106.
In some examples, the one or more processors 202 may include one or more processing units. For instance, the processors 202 may comprise at least one of a hardware processing unit or a software processing unit. Thus, in some examples the processors 202 may comprise at least one of a hardware processor or a software processor, and may include one or more cores and/or other hardware or software components. For example, the one or more processors 202 may include a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, and so on. Alternatively, or in addition, the processor 202 may include one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc. The processor 202 may be in communication with, operably connected to, and/or otherwise coupled to memory and/or other components of the electronic device 200 described herein. In some examples, the processor 202 may also include on-board memory configured to store information associated with various operations and/or functionality of the processor 202.
The I/O interfaces 204 may be configured to enable the electronic device 200 to communicate with other devices, and/or with the service provider 106 (
The I/O interfaces 204 may be in communication with, operably connected to, and/or otherwise coupled to one or more I/O devices 206 of the electronic device 200. The I/O devices 206 may include one or more displays 208, cameras 210, controllers 212, microphones 214, touch sensors 216, orientation sensors 218, motion sensors, proximity sensors, pressure sensors, and/or other sensors (not shown). The one or more displays 208 are configured to provide visual output to the user 102. For example, the displays 208 may be connected to the processors 202 and may be configured to render and/or otherwise display content thereon, including the webpage content described herein. In some examples, the display 208 may comprise a touch screen display configured to receive touch input from the user 102. In further examples, the display 208 may comprise a non-touch screen display.
The display 208, camera 210, microphone 214, touch sensor 216, and/or the orientation sensor 218 may be coupled to the controller 212. In some examples, the controller 212 may include one or more hardware and/or software components described above with respect to the processor 202, and in such examples, the controller 212 may comprise a microprocessor, or other device. In further examples, the controller 212 may comprise a component of the processor 202. The controller 212 may be configured to control and receive input from the display 208, camera 210, microphone 214, touch sensor 216, and/or the orientation sensor 218. In some examples, the controller 212 may determine the presence of an applied force, a magnitude of the applied force, and so forth. In some implementations the controller 212 may be in communication with, operably connected to, and/or otherwise coupled to the processor 202. In such examples, one or more of the display 208, camera 210, microphone 214, touch sensor 216, and/or the orientation sensor 218 may be coupled to the processor 202 via the controller 212.
The electronic device 200 may also include or be associated with one or more additional I/O devices not explicitly shown in
As shown in
In some examples, the CRM 220 may include one or a combination of memory or CRM operably connected to the processor 202. Such memory or CRM may include computer storage media and/or communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, phase change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer storage media does not include communication media.
The CRM 220 may include software functionality configured as one or more “modules.” The term “module” is intended to represent example divisions of the software for purposes of discussion, and is not intended to represent any type of requirement or required method, manner or organization. Accordingly, various such modules, their functionality and/or similar functionality could be arranged differently (e.g., combined into a fewer number of modules, broken into a larger number of modules, etc.). Further, while certain functions and modules may be implemented by software and/or firmware executable by the processor 202, in other examples, one or more such modules may be implemented in whole or in part by other hardware components of the electronic device 200 (e.g., as an ASIC, a specialized processing unit, etc.) to execute the described functions. In some instances, the functions and/or modules are implemented as part of an operating system. In other instances, the functions and/or modules are implemented as part of a device driver (e.g., a driver for a touch surface), firmware, and so on.
In some examples, the CRM 220 may include at least one operating system (OS) module 222. The OS module 222 may be configured to manage hardware resources such as the I/O interfaces 204 and provide various services to applications or modules executing on the processors 202. Also stored in the CRM 220 may be a controller management module 224, a user interface module 226, a webpage content storage and review framework 228, and other modules 230. The controller management module 224 may be configured to provide for control and adjustment of the controller 212. For example, the controller management module 224 may be used to set user-defined preferences in the controller 212.
The user interface module 226 may be configured to provide a user interface to the user 102. This user interface may be visual, audible, or a combination thereof. For example, the user interface module 226 may be configured to present an image or other content on the display 208 and process various touch inputs applied at different locations on the display 208. The user interface module 226 may also be configured to cause the processor 202 and/or the controller 212 to take particular actions, such as paging forward or backward in an e-book or rendered webpage content 110. The user interface module 226 may be configured to respond to one or more signals from the controller 212. These signals may be indicative of the magnitude of a force associated with a touch input, the duration of a touch input, or both. Such signals may also be indicative of any of the non-touch inputs described herein, such as inputs received via one or more physical buttons, keys, mice, or other I/O devices 206.
The webpage content storage and review framework 228 (also referred to herein as “framework 228”) may comprise one or more additional modules of the CRM 220. The framework 228 may include instructions that, when executable by the processor 202, cause the processor 202 to perform one or more operations associated with saving images of webpage content and recalling websites including text that is contained in the saved images. For example, the framework 228 may comprise a module configured to cause the processor 202 to capture an image (e.g., a screenshot of webpage content rendered on the display 208, to save the captured image, to recognize text included in the image, and to form one or more text groups using the recognized text. The framework 228 may also cause the processor 202 to generate one or more searches, such as internet searches, using the recognized text of the text groups as search queries. Additionally, the framework 228 may cause the processor to identify at least one search result as being indicative of a webpage that includes the desired webpage content and to generate a content item by extracting content from the webpage. Such operations will be described in greater detail below with respect to, for example,
The CRM 220 may also include a datastore 232 to store information. The datastore 232 may use a flat file, database, linked list, tree, or other data structure to store the information. In some implementations, the datastore 232 or a portion of the datastore 232 may be distributed across one or more other devices including servers, network attached storage devices, and so forth. The data store 230 may store information about one or more user preferences and so forth. Other data may be stored in the datastore 232 such as e-books, video content, audio content, graphical and/or image content, and/or other webpage content 110. The datastore 232 may also store images, screenshots, or other content captured by one or more hardware components, software components, applications, or other components of the device 204.
The electronic device 200 may also include one or more communication interfaces 234 configured to provide communications between the electronic device 200 and other devices, such as between the electronic device 200 and the service provider 106 via the network 108. Such communication interfaces 234 may be used to connect to one or more personal area networks (“PAN”), local area networks (“LAN”), wide area networks (“WAN”), and so forth. For example, the communications interfaces 234 may include radio modules for a WiFi LAN and a Bluetooth PAN. The electronic device 200 may also include one or more busses or other internal communications hardware or software that allow for the transfer of data between the various modules and components of the electronic device 200.
While
In some examples, the user 102 may initiate one or more of the methods described herein by activating one or more applications on the electronic device 104. Such an application may, for example, enable the user to access and/or view webpage content via the display 208. Such applications may comprise one or more search engines, browsers, content viewers, news applications, blog applications, social media applications, and/or other applications operable on the electronic device 104. Such applications may be activated by, for example, directing one or more touch inputs to the electronic device 104 via the display 208. In other examples, such applications may be activated by directing one or more non-touch inputs to the electronic device 104, such as via one or more physical buttons or keys of the electronic device 104, a mouse connected to the electronic device 104, or other I/O devices 206. As shown in
The secondary content 406 described herein, the other hand, may comprise banner advertisements, background images, pop-up advertisements, headers, footers, sidebars, toolbars, UI controls, and/or other content that is rendered along with the primary content 402, but that is ancillary to, and in some cases unrelated to, the primary content 404. For example, the secondary content 406 illustrated in
The UI controls 408 may comprise, for example, one or more buttons, icons, or other UI configured to provide functionality to the user 102 associated with the primary content 404 rendered on the display 208. For example, such UI controls 408(1) may enable a user 102 to view, scroll, pan, and/or otherwise interact with a webpage corresponding to and/or that is the source of the webpage content 402 currently being rendered by the display 208. In such examples, the webpage content 402 may be accessed by the electronic device 104 via one or more applications that enable the user 102 to view other webpages therethrough. Alternatively, in other applications, webpage content may reside on a remote and/or cloud-based database. Example applications may include FLIPBOARD™, ZITE™, TUMBLR™, FACEBOOK™, TWITTER™, FACEBOOK PAPER™, KLOUT™, and/or other applications or websites. Such UI controls 408(2) may also enable the user 102 to share, via one or more social media applications, instant messaging applications, email applications, message board applications, and/or other applications, at least a portion of the webpage content 402 being rendered on the display 208. Still further UI controls 408(3) may enable the user 102 to capture an image of at least a portion of the webpage content 402. In some examples, such an image may comprise, among other things, a screenshot of at least a portion of the webpage content 402. In some examples, such UI controls 408(3) may activate and/or utilize one or more copy and/or save functions of the electronic device 104. Activation of such UI controls 408(3) may copy an image of at least a portion of the primary content 404 and/or the secondary content world 406 being rendered on the display 208, and may save the copied image in, for example, the CRM 220 of the electronic device 104. Additionally, the copied image may be emailed and/or otherwise provided to the service provider 106, via the network 108, in response to activation of the UI control 408(3), and the copied image may be saved in a memory of the service provider 106.
For example, as shown in operation 304 of
At 306, the processor 202 and/or applications or modules operable via the processor 202, such as the framework 228, may save the captured image (i.e., the screenshot) in the CRM 220 of the electronic device 104. Additionally, at 306 the processor 202 and/or applications or modules operable via the processor 202 cause the captured image to be sent to the service provider 106, via the network 108. In such examples, the service provider may save the captured image in a memory of the service provider 106 upon receipt, and such memory may be remote from the electronic device 104. In some examples, both the CRM 220 and the memory of the service provider 106 may be in communication with, coupled to, operably connected to, and/or otherwise associated with the electronic device 104.
In some examples, at least one of capturing the image at 304 or saving the image at 306 may cause, for example, the processor 202 and/or other hardware or software components of the electronic device 104 to send the captured image to the service provider 106. For example, a software application executed by the processor 202 may generate an email, including the captured image as an attachment thereto, in response to the captured image being detected in a designated folder, such as a “photos” folder or an “images” folder, of the CRM 220. In such examples, the software application may cause the processor 202 to send the email from the electronic device 104 to the service provider 106. In still further examples, any other methods or protocols may be utilized instead of and/or in combination with email in order to transfer the captured image from the electronic device 104 to the service provider 106, and such example protocols may include, among other things, file transfer protocol (FTP).
At 308, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106, such as the framework 228, may recognize, using optical character recognition (OCR), text that is included in the captured image. For example, such OCR may be performed by various programs, application, and/or other software saved in either the CRM 220 and/or in a memory of the service provider 106. In some examples, and OCR process performed by such software may convert portions of the captured image into machine-encoded/computer-readable text. In this way, at least a portion of the captured image may be electronically edited, searched, stored, displayed, and/or otherwise utilized by components of the device 14 and/or the service provider 106 for one or more of the operations described with respect to
In some examples, at least one of the top or left values of the array 502 may be utilized to determine, for example, a position of a corresponding line of text, a relationship between the corresponding line of text and at least one other line of text, and/or other characteristics associated with the webpage content 402 and/or the recognized text 504. Additionally, each array 502 may include respective values indicative of an overall width of the text corresponding to the recognized text 504 (i.e., the webpage content 402), and of an overall height of the text corresponding to the recognized text 504 (i.e., the webpage content 402). Such “width” and “height” values are illustrated as the third and fourth numerals of each array 502 shown in
At 310, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106, such as the framework 228, may form a plurality of text groups based at least in part on the text included in the captured image. For example, such text groups may be formed based at least in part on the text recognized at 308, and a plurality of example text groups 506(1), 506(2), 506(3), 506(4), 506(5), 506(6), 506(7), 506(8) (collectively, “text groups 506”) are illustrated in
For example, forming the plurality of text groups 506 may include grouping adjacent lines of recognized text 504 having respective widths that are approximately equal when the corresponding webpage content 402 is rendered on the display 208. For example, as can be seen in
In some examples, forming the plurality of text groups 506 may also include grouping adjacent lines of recognized text 504 having approximately equal vertical spacing between the respective text lines when the corresponding webpage content 402 is rendered on the display 208. For example, as can be seen in
In still other examples, forming the plurality of text groups 506 may include grouping adjacent lines of recognized text 504 having respective margins that are approximately equal when the corresponding webpage content 402 is rendered on the display 208. For example, as can be seen in
In additional examples, forming the plurality of text groups 506 may include grouping words or lines of recognized text 504 based on one or more of the respective margins, font sizes, font types, alignments, and/or other characteristics of the recognized text 504 when the corresponding webpage content 402 is rendered on the display 208. For example, when webpage content 402 is rendered on the display 208, two or more adjacent lines of text may have respective font sizes. The processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may determine the respective font sizes of the adjacent lines at 310. The adjacent lines of text may also have respective “left” values or other values indicative of the location and/or alignment of the respective lines of text. For example, the two or more adjacent lines of text may have a “left” value (as described above with respect to
For example, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may group two or more adjacent lines of text if a difference between the respective font sizes of the adjacent lines is below a font size difference threshold and if respective left, center, bottom, top, or other values of adjacent lines of text are substantially equal. In addition to determining a difference between the respective font sizes of the adjacent lines, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may determine a difference between the respective left, center, bottom, top, or other values of the adjacent lines of text. If the determined difference between the respective font sizes is below the font size difference threshold, and if the difference between one or more of the respective left, center, bottom, top, or other values of the adjacent lines of text is below a corresponding threshold, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may form a text group 506 with the adjacent lines of text at 310.
In still further examples, forming the plurality of text groups 506 at 310 may include grouping words or lines of recognized text 504 according to one or more grammar, syntax, definition, semantic, heuristic, and/or other rules (referred to collectively herein as “context-based grouping rules”). As can be seen in the example OCR results 500a shown in
At 310, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may analyze the recognized text 504 with reference to one or more context-based grouping rules and may, in response, determine that at least a portion of the recognized text 504 shares a common semantic meaning or other such contextual relationship and, thus, may be associated with a common label (e.g., a title, a body text, etc.). Such rules may include, for example, definition, grammar and/or syntax rules associated with the particular language (e.g., English, Spanish, Italian, Russian, Chinese, Japanese, German, Latin, etc.) of the recognized text 504, and some such rules may be language-specific. In response to making such a determination, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may form a single text group (e.g., 506(1)a) with such text even if the formation of such a text group 506(1)a may conflict with other text group formation rules described herein.
For example, although the text group 506(1)a may include a number of words greater than a predetermined threshold used to limit text groups, in some embodiments, such a threshold may be ignored if, for example, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 determines that at least a portion of the recognized text 504 shares a common semantic meaning. Such context-based rules may result in the formation of text groups 506 that are more linguistically and/or semantically accurate than some of the text groups 506 described above with respect to, for example,
In additional examples, such context-based rules may also be used to divide text groups into two or more individual text groups. For example, the text group 506(2) of
In further examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may associate at least one of a label 508(1), 508(2) . . . 508(n) (collectively, “labels 508”) or a weight 510(1), 510(2) . . . 510(n) (collectively, “weights 510”) with one or more of the text groups 506. In some examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may generate one or more such labels 508 based on, among other things, characteristics of the recognized text 504, context information, grammar, syntax, and/or other semantic information associated with the recognized text 504. For example, the OCR process employed at 308 may include, among other things, a syntax evaluation of the recognized text 504. Such a syntax evaluation may provide information regarding the type of recognized text 504 included in the OCR results 500. In particular, such an evaluation may provide information indicative of whether the recognized text 504 includes one of a title, author, date, body text (e.g., a paragraph), or source of the webpage content 402. Accordingly, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may associate one of a “title,” “author,” “date,” “body text,” or “source” label with at least one of the text groups 506 based on such information. In some examples, the label 508 associated with the respective text groups 506 may be used to determine, for example, whether or not to utilize the recognized text 504 included in the corresponding text group 506 when performing one or more searches, such as internet searches. In further examples, one or more additional labels 508 may also be associated with respective text groups 506. Additionally, the one or more labels 508 may, in some examples, identify a common contextual relationship shared by adjacent lines of text forming the respective text group 506 with which the label 508 is associated.
In some examples, the syntax evaluation described above may employ one or more characterization rules in associating a label 508 with the respective text groups 506. For example, in most webpage content a title of an article may be characterized by being positioned proximate or at the top of the webpage. Additionally the title of an article may typically be rendered with a larger font size than the remainder of the article and/or may be rendered with bold font. Thus the syntax evaluation performed during the OCR process employed at 308 may take such common title characteristics into account when associating a “title” label 508(1) with a respective text group 506(1). Similarly, in the English language the first letter of an author's first, last name, and middle initial may be capitalized, and in most instances, the author's name may be preceded by the word “by.” Additionally, in some instances an author's first name may be relatively common and, thus, may be included in one or more lookup tables stored in memory. As a result, the syntax evaluation performed during the OCR process employed at 308 may take such common author name characteristics into account when associating a “name” or “author” label 508 with a respective text group 506.
In additional examples, a date of publication and/or posting may sometimes be represented in the webpage content 402 in a fixed format. For example, it is customary to list a date using a month, day, year format in the English language. Additionally, in other countries it may be common to utilize a day, month, year format. Further, since the names of the 12 months are known, such months can be easily referenced in one or more lookup tables stored in memory. Accordingly, the syntax evaluation performed during the OCR process employed at 308 may take such common date characteristics into account when associating a “date” label 508(4) with a respective text group 506(4). In still further examples, the source of the webpage content 402 may often be represented using at least one of a “www” or a “http://” identifier. Thus, the syntax evaluation performed during the OCR process employed at 308 may recognize such common source identifiers when associating a “source” label 508(2) with a respective text group 506(2).
Further, the various weights 510 assigned to and/or otherwise associated with the various text groups 506 may have respective values indicative of, for example, the importance of recognized text of the type characterized by the corresponding label 508. For example, when performing an internet search in order to recover the webpage content 402, utilizing some types of text as a search query may result in more accurate search results than utilizing other different types of text as a search query. In particular, when performing an internet search to recover the webpage content 402 illustrated in
In still further examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may omit one or more of the text groups 506 when performing various searches based at least in part on a variety of additional factors. For example, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may determine that at least one text group 506 of the plurality of text groups 506 has a number of words less than a minimum word threshold. In some examples, searches performed using search queries that include less than a minimum word threshold (e.g., four words) may yield search results that are less accurate than, for example, additional searches that are performed using search queries that include greater than such a minimum word threshold. For example, a first internet search performed using the recognized text 504 of the text group 506(3) (i.e., that includes one word “books”) may yield search results that are relatively inaccurate when compared to, for example, a second internet search performed using the recognized text 504 of the text group 506(1). As a result, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may omit one or more text groups 506 from the plurality of searches to be generated based at least in part on determining that the at least one text group 506 has a number of words less than the predetermined minimum word threshold.
As shown in
As noted above, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may be selective when choosing the one or more text groups 506 from which recognized text 504 may be utilized as a search query for the searches generated at 312. For example, as noted above, a minimum word threshold may be employed to determine the one or more text groups 506 from which recognized text 504 may be utilized. As noted above, an example minimum word threshold may be approximately four words, and in such examples only text groups 506 including recognized text 504 of greater than or equal to four words may be utilized to generate searches, such as internet searches, at 312. The above minimum word thresholds are merely examples, and in further examples a minimum word threshold greater than or less than four (such as 2, 3, 5, 6, etc.), may be employed.
Further, as shown in the example 600 of
In some examples, various additional grouping or truncation rules may be used to form the search queries 602 described herein. For instance, in some examples respective search queries 602 may be formed by selecting a desired number of adjacent words in a text group 502. In such examples, a text group 502 may be segmented into a plurality of separate search queries 602, each separate search query including the desired number of adjacent words from the text group 502, and in the event that there is a reminder of words in the text group 502 less than the desired number, the remainder of words may be used as an additional separate search query 602. In such examples, there may be no overlap between search queries 602 formed from a particular text group 502 (e.g., none of the adjacent words in the text group 502 may be included in more than one search query 602).
Additionally, in some examples one or more modifiers may be used when forming search queries 602 of the present disclosure. For example, quotes (“ ”) may be employed to direct the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 to affect the search results resulting from the query. Using quotes, for example, may require that the search results contain the exact string of ordered words disposed between the quotes. Additionally, a plus sign (+) may be employed to combine two or more separate search queries. Further, the use of multiple modifiers (e.g., quotes and a plus sign) may be used in one or more internet searches in order to increase the accuracy of search results. For example, a combined search query in which the exact string of ordered words appearing in search queries 602(G1-1) and 602(G2-1) is desired may be as follows: “The Science of Humor and the Humor of Science: A”+“via www.brainprongs.org.”
As shown in
With continued reference to
In some examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may determine that each of the webpage links is included in the search results 702 only once. In such examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may associate a relatively low level of confidence with each of the search results. In such examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may not output and/or otherwise any of the search results or URLs at 314.
In further examples, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may identify the particular webpage link 706 at 314 based at least in part on the title 508 and/or the weight 510 associated with the text groups 506 from which the respective search query 602 has been generated. For example, as noted above the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may associate a weight 510 with one or more of the text groups 506 formed at 310. In some examples, such a weight 510 may be based at least in part on a corresponding label 508 associated with the respective text groups 506.
In addition, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may assign a respective score 704 to each webpage link included in the respective search results 702(1), 702(2), 702(5) yielded using corresponding search queries 602(1), 602(2), and 602(5) (i.e., at least a portion of the corresponding recognized text 504). In some examples, each respective score 704 may be indicative of, for example, the degree to which content included on the webpage corresponding to the respective webpage link is similar to and/or matches the respective search query 602 utilized to generate the corresponding internet search. Any scale may be used when assigning such scores 704. Although the scores 704 shown in
At 316, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106, such as the framework 228, may generate a content item by extracting various webpage content from a webpage corresponding to the particular webpage link 706. As shown in the example 800 of
In some examples, at 316 the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may generate a content item by extracting at least a portion of the primary content 804 from the webpage 802 and by omitting at least a portion of the secondary content 806 of the webpage 802. In performing such operations at 316, the processor 202 and/or other hardware or software components of either the electronic device 104 or the service provider 106 may employ one or more text recognition algorithms, syntax analysis algorithms, and/or other hardware or software components to distinguish the primary content 804 from the secondary content 806 such that, in some examples, only the primary content 804 may be utilized to generate the content item. For example, such text recognition algorithms, syntax analysis algorithms, and/or other hardware or software components may include, among other things, Microsoft® extractor software (Microsoft Corporation®, Redmond, Wash.) as included in Microsoft Windows® 8.11E11 and Microsoft Windows Phone® 8.1 IE11. In further examples in which alternate operating systems (e.g., OSX™ or LINUX™) are employed, alternative compatible extractor applications may be employed. In some examples, the text recognition algorithms, syntax analysis algorithms, and/or other hardware software components utilized at 316 to generate the content item may be configured to extract such primary content 804 from various websites 802 in order to generate, for example, a content item configured for viewing in alternate formats such as via a wireless phone, tablet, PDA, or other electronic device 104.
In some examples, the electronic device 104 may receive a request for the primary content 404 of the webpage content 402 shown in
In some examples, the content item 902 may be generated, at 316, by either the processor 202 of the electronic device 104 or by the service provider 106. In examples in which the content item 902 is generated by the processor 202 of the electronic device 104, such a content item 902 may be, for example, saved in the CRM 220 at 316. Thus, the electronic device 104 may, in response to receiving the request described above, retrieve the content item 902 from the CRM 220 and render the content item 902 on the display 208. In examples in which the content item 902 is generated by one or more processors and/or other components of the service provider 106 at 316, such a content item 902 may be, for example, saved in a memory of the service provider 106 at 316. In such examples, the electronic device 104 may, in response to receiving the request from the user 102, send a signal, message, and/or request to the service provider 106, via the network 108. In such examples, a signal sent by the electronic device 104 to the service provider 106 may include information requesting, among other things, a digital copy of the content item 902 generated by the service provider 106. In response to receiving such a signal from the electronic device 104, the service provider 106 may provide a copy of the content item 902 to the electronic device 104 via the network 108. In some examples, the electronic device 104 may render the content item 902 on the display 208 in response to receiving the content item 902 from the service provider 106.
Examples of the present disclosure may be utilized by various users 102 wishing to retrieve content viewed by the user from a plurality of different webpages or other sources. For example, it is common for users 102 to consume content on electronic devices 104 from a variety of different webpages, and using a variety of different and unrelated applications to do so. For example, such content may be viewed using different news applications, blog applications, social media applications, and/or other applications having a variety of different formats. Examples of the present disclosure enable the user 102 to save images (i.e., screenshots) from each of these different applications, regardless of application type. Thus, examples of the present disclosure comprise a universal framework configured to enable users 102 to save content having various different formats and originating from various different sources (i.e., regardless of the type, format, and/or source of the content). Such examples also enable the user 102 to recall the underlying content included in such saved images for consumption later in time. Additionally, since the underlying content is to be consumed via the electronic device 104, examples of the present disclosure may provide the underlying content to the user 102 in a modified format that is more easily and effectively rendered on the display 208 for consumption by the user 102.
Examples of the present disclosure may provide multiple technical benefits to the electronic device 104, the service provider 106, and/or the network 108. For instance, traffic on the network 108 may be reduced in examples of the present disclosure since users 102 will not need to submit multiple searches in an effort to find the content they had previously viewed. Additionally, since the electronic device 104 and/or the service provider 106 may save screenshots of content having various different formats and originating from various different sources, multiple different applications need not be employed by the electronic device 104 and/or the service provider 106 to recover webpages including the desired content. Since multiple applications are not needed, storage space in the CRM as well as processor resources may be maximized. As a result, examples of the present disclosure may improve the overall user experience.
Clause 1: In some examples of the present disclosure, a method includes receiving a captured image with a device, wherein the image is received by the device via a network and the captured image includes webpage content. The method also includes recognizing, using optical character recognition, text included in the image, forming a plurality of text groups based on the text included in the image, and generating a plurality of searches. In such a method, each search of the plurality of searches uses text from a respective text group as a search query, and yields a respective search result including at least one webpage link. Such a method also includes identifying at least one of the webpage links as being indicative of a webpage that includes the webpage content, generating a content item using the webpage content from the webpage, and providing access to the content item via the network.
Clause 2: The method of clause 1, wherein forming the plurality of text groups includes grouping adjacent lines of text sharing a common contextual relationship, and associating a label with at least one text group of the plurality of text groups, wherein the label identifies the common contextual relationship associated with the at least one text group.
Clause 3: The method of clause 1 or 2, wherein the image includes a screenshot captured while rendering the webpage content, the method further including saving the screenshot in memory associated with the device.
Clause 4: The method of clause 1, 2, or 3, further comprising receiving a request via the network, and sending the content item, via the network, in response to the request.
Clause 5: The method of clause 1, 2, 3, or 4, wherein at least one search seed includes text from a first text group and text from a second text group different from the first text group.
Clause 6: The method of clause 1, 2, 3, 4, or 5, wherein forming the plurality of text groups includes grouping adjacent text lines having respective widths that are approximately equal.
Clause 7: The method of clause 1, 2, 3, 4, 5, or 6, wherein forming the plurality of text groups includes grouping adjacent text lines having approximately equal vertical spacing between the text lines.
Clause 8: The method of clause 1, 2, 3, 4, 5, 6, or 7, wherein forming the plurality of text groups includes grouping adjacent text lines having respective margins that are approximately equal.
Clause 9: The method of clause 1, 2, 3, 4, 5, 6, 7, or 8, further including determining that at least one text group of the plurality of text groups has a number of words less than a minimum word threshold, and omitting the at least one text group from the plurality of searches based at least in part on determining that at least one text group of the plurality of text groups has the number of words less than the minimum word threshold.
Clause 10: The method of clause 1, 2, 3, 4, 5, 6, 7, 8, or 9, wherein identifying the at least one of the webpage links includes determining that the at least one of the webpage links is included in a greater number of the respective search results than a remainder of the webpage links.
Clause 11: The method of clause 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10, further including associating a label with at least one text group of the plurality of text groups, the label including one of title, author, date, text, or source.
Clause 12: The method of clause 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or 11, further including omitting the at least one text group from the plurality of searches based at least in part on the label associated with the at least one text group.
Clause 13: The method of clause 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, or 12, further including: associating a weight with the at least one text group of the plurality of text groups based at least in part on the label associated with the at least one text group; assigning a score to each webpage link included in the respective search result yielded using text from the at least one text group; and identifying the at least one of the webpage links based at least in part on the scores.
Clause 14: A method includes receiving a screenshot of webpage content; saving the screenshot in memory associated with a processor; recognizing, using optical character recognition, text included in the saved screenshot; generating a plurality of search queries using the text recognized using optical character recognition; and causing at least one search to be performed using the plurality of search queries. Such a method also includes receiving a search result corresponding to the at least one search, the search result including at least one webpage link; identifying the at least one webpage link as being indicative of a webpage that includes the webpage content; and generating a content item by extracting the webpage content from the webpage.
Clause 15: The method of clause 14, further including receiving a request for the webpage content, and providing the content item, via a network associated with the device, in response to the request, wherein the content item is configured to be rendered on an electronic device.
Clause 16: The method of clause 14 or 15, further including forming a plurality of text groups with the text recognized using optical character recognition, wherein each group of the plurality of text groups is formed based on at least one shared characteristic of adjacent text lines in the screenshot of webpage content.
Clause 17: The method of clause 16, further including: identifying a first set of groups of the plurality of text groups having a number of words greater than or equal to a minimum word threshold; identifying a second set of groups of the plurality of text groups having a number of words less than the minimum word threshold; and generating the plurality of search queries using text from the first set of groups and omitting text from the second set of groups.
Clause 18: The method of clause 16, further including: assigning a weight to each group of the plurality of text groups; assigning a score to the at least one webpage link, wherein the score is based at least in part on a corresponding weight; and identifying the at least one webpage link based at least in part on the score.
Clause 19: A device includes a processor, wherein the device is configured to receive a screenshot of webpage content from an electronic device remote from the device, the device configured to: recognize, using optical character recognition, text included in the screenshot; generate a plurality of search queries using the text recognized using optical character recognition; cause at least one search to be performed; receive a search result corresponding to the at least one search, the search result including at least one webpage link; identify the at least one link as being indicative of a webpage that includes the webpage content; and generate a content item by extracting content from the webpage, wherein the content item comprises a modified version of the webpage content and is configured to be rendered on a display associated with the electronic device.
Clause 20: The device of clause 19, further comprising memory disposed remote from the electronic device, the memory configured to store the screenshot and the content item.
Clause 21: The device of clause 19 or 20, wherein the device is further configured to cause a plurality of searches to be performed, wherein each search of the plurality of searches is performed by a different respective search engine.
The architectures and individual components described herein may include many other logical, programmatic, and physical components, of which those shown in the accompanying figures are merely examples that are related to the discussion herein.
Other architectures may be used to implement the described functionality, and are intended to be within the scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, the various functions and responsibilities might be distributed and divided in different ways, depending on circumstances.
Although the various examples have been described in language specific to structural features and/or methodological acts, the disclosure is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed subject matter.