Patent Application
20110153857
The present disclosure relates to portable electronic devices, and more particularly to a method of partial loading and viewing a document attachment on a portable electronic device using a consistent interface across different applications.
It is known in the wireless communication arts to send photographs, scanned documents, slide shows, PDF documents and other types of attachments in email messages or to download such documents from web sites. Each attachment is provided with a filename and is linked to an email message or web site in a known manner.
For example, it is known in the art to send a data request from a portable electronic device, such as a Personal Digital Assistant (PDA) or smart phone, to a server responsive to receipt of an email message at the portable electronic device that includes an attachment, and sending the data to the portable electronic device for display via an attachment viewer.
There is generally a delay between a request to view an attachment by a user and display of the attachment on the display screen of the device. The primary cause of the delay is server processing time and downloading the attachment to the portable device. Network speed is also a contributing factor when downloading large attachments.
The embodiments will be better understood with reference to the following Figures in which like numerals denote like parts and in which:
In one aspect there is provided a method for downloading an attachment for viewing on an attachment viewer of a portable electronic device, the method comprising: encapsulating, in a self-contained form, a DOM for a requested attachment; and communicating, in a self-contained form viewable on the attachment viewer, at least a portion of the requested attachment along with meta-data for subsequently locating said attachment
In another aspect there is provided a method of saving partial attachment data in a portable electronic device, said method comprising: serializing and persisting an initial portion of an attachment and meta-data for subsequently locating said attachment on said portable electronic device; displaying said initial portion on said portable electronic device; and responsive to a subsequent request for said attachment de-serializing said initial portion and said meta-data; displaying said initial portion and sending a further data request for additional portions, wherein said further request includes said meta-data.
Referring to
It will be appreciated that the mobile communication device 12 is movable within the coverage area and can be moved to coverage areas defined by other base stations. Further, as will be understood by one of ordinary skill in the art, wireless networks include GSM/GPRS, CDPD, TDMA, iDEN, Mobitex, DataTAC networks, EDGE, EVDO or UMTS and broadband networks such as Bluetooth and variants of 802.11.
A proxy server 16 handles client requests from the mobile communication device 12 for documents stored within an attachment server 18. The attachment server 18 communicates with the proxy server 16 to transmit attachments such as documents, spreadsheets, images, multimedia files, etc. for viewing via an attachment viewer of the mobile communication device 12 to allow a user to view attachments that are received in email messages. While only one server 18 is shown for illustration purposes, a person skilled in the art will understand that the attachment server 18 may alternatively be a network of attachment servers. Sources for the attachments stored within server 18 may include a web server 15, mail server 19, IM server 17, etc. Preferably the document data is downloaded to mobile communication device 12 in chunks of binary data in an attachment viewer readable format, for example Universal Content Stream (UCS) format.
Referring now to
The portable electronic device 12 includes a processor 20 connected to a read-only-memory (ROM) 21 that contains a plurality of applications executable by the processor 20 that enables the portable electronic device 12 to perform certain functions including, for example, PIN message functions, SMS message functions and cellular telephone functions, and an attachment viewer application for viewing attachments (e.g. document attachments to emails or documents from other sources, such as web servers). The processor 20 is also connected to a random access memory unit (RAM) 22 and a persistent storage device 23, which are responsible for various storage functions of the portable electronic device 12. The processor 20 receives input from input devices such as a keypad 24 and trackball 25. The processor 20 outputs to various output devices, such as an LCD display 26. A microphone 27 and phone speaker 28 are connected to the processor 20 for cellular telephone functions. The processor 20 is also connected to a modem and radio device 29. The modem and radio device 29 is used to connect to wireless networks and transmit and receive voice and data communications through an antenna 30. A content store 31, which is generally a file storage system for the portable electronic device 12, is also provided.
Request/view functionality for an attachment is provided by the client/server combination of attachment viewer within the portable electronic device 12 and the attachment server 18.
If the attachment has not been previously requested, attachment server 18 builds a DOM that represents the attachment by parsing the attachment document (step 36). In this manner, a graph structure is built within attachment server 18 representing a map of the original attachment file. The DOM contains textual content, font, style and formatting attributes as well as layout attributes, such as page/slide size, positioning information (i.e. x, y and z coordinates on the page), embedded graphics and tables, for example. DOM structure is well known and is disclosed in United States Patent Application No. 2006/0055693, which is herein incorporated by reference.
Once the DOM of the attachment is built, the attachment server 18 trascerpts and encapsulates the DOM in UCS data, as indicated at step 37. The UCS data is then sent to mobile electronic communications device 12 in chunks, as indicated at step 38. Each chunk is a self-contained data representation of a portion of the attachment. That is, there is sufficient data contained in a chunk to enable the attachment viewer to display the content of the chunk. Depending on the size of the chunks and the size of the attachment, the entire attachment can be transmitted in one chunk or in multiple chunks. Depending on the nature of the request from the attachment viewer, attachment server 18 can transmit the chunks in sequence or out of sequence. For example, if the attachment viewer requests the fifth page of an attachment, attachment server 18 can transmit the chunks corresponding to the fifth page, even if chunks for pages 1 to 4 have not been transmitted to the device 12.
As shown in the UML diagram of
As discussed in greater detail below, the attachment viewer includes a user interface, which accepts user actions (42) for enhanced image data request, retrieval of an embedded object or the textual data for rendered slides/pages, etc.
Turning to the class diagram of
The attachment viewer display engine 43 is invoked using a plug-in to the calling application. Application specific implementations of the persistence and transport interfaces 46, 47 and 50, are passed to the display engine 43 using rendering options of the plug-in architecture. Once displayed, display engine 43 renders the existing data in a consistent manner, makes data requests based on user input and is notified when data from server has arrived to update the display 30.
While the MoreTransport interface implementations 46 or 47 are optional, the PersistOperation must be valid when the display field is constructed, otherwise no data can be displayed. It is not required that the document data exist after a reset of the device 12. For example, the persistent store 23 can be session based and constructed in memory when document data chunks for a linked document are displayed (e.g. in the browser).
The PersistOperation provided by persistence interface 50 ensures that document data received from attachment server 18 via the available transport 46 or 47 is cached and retrieved in a similar manner and that the display engine 43 retrieves the data in a unified way. There are three generic keys used in accessing an atomic attachment in the persistent store 23 (int or long—message id for email, int or long—more part id for email, archive indicator—string indicating the archive index for a document in an archive). These values are used to initialize the display engine 43 and can have a different meaning for each calling application (e.g. a hash code for the downloaded url) or can be generic (except for the archive indicator which has a predefined meaning). These values are also used to route a received data response to the active display engine 43 displaying the particular document.
MoreTransport provides a mechanism for sending a request for data to the attachment server 18. The transport media can be email MORE, HTTP request etc. Common constraints for the MoreTransport implementation include that it accept and transport to the attachment server 18 an XML string descriptor and that it is able to receive an XML descriptor from the attachment server together with the binary data chunk.
DocViewUCSConverter 52 is a class used by the attachment viewer 45 in an embodiment, to register when the device 12 boots (i.e. starts or resets) so that it understands the MIME type “UCS”. Thus, every binary data with this MIME type (a file with extension ucs or binary data in memory coming from a data connection with UCS MIME type) is routed through the attachment viewer architecture.
In addition to configuring the attachment viewer for uniform presentation of documents, as discussed above with reference to
Turning now to
This functionality is implemented in the attachment viewer 45 by allowing it to open a browser channel (55) for document data communication and thereby make data requests (57).
If the requested document is of a type that is supported by attachment server 18, the resulting XML response and binary document data are sent back (67) to the device browser 45 via HTTP. The metadata information contains the initial XML attachment server response, as well. The XML response string is part of the HTTP headers for the particular request and it is accessible using a predefined property. The binary document data is sent to the device using a predefined content type (e.g. Universal Content Stream (UCS)).
The attachment viewer 43 detects this content type and is able to display (69) the initial data by using the input XML string and the binary data. The attachment viewer 45 sends such requests based on user input by adding the XML string to the HTTP headers property and issuing a GET HTTP request using the document download url. Proxy server 16 searches for the specific property in the HTTP request header and, if encountered, determines that the request is an attachment server request. Upon receiving the response from attachment server 18, proxy server 16 packages the XML descriptor in the http headers and sends it back to the device 12 together with the response binary document data. It should be noted that the XML string descriptor is not always the same; it is manipulated by attachment viewer 45 and server 16 by adhering to a predefined protocol. A correspondence between the initial proxy request and the attachment server ip address is saved in order to access the attachment server cache for subsequent requests from the device 12, and thereby eliminate the need for multiple downloads of the linked document from the web server.
In response to execution of an action (71) using the interface (46), such as to download an embedded object, view slides text, enhance image, etc., the attachment viewer creates (73) an XML string and adds the document ID to it (e.g. full url, attachment server IP address, etc.) as well as fields for identifying the request for the attachment server 18, sends (75) the request to the proxy server 16, and awaits response from the proxy server 16. The proxy server 16 adjusts the attachment server response to create a response XML string.
The following is an example XML string sent to the attachment viewer 45 upon the initial download of the linked document and display in the browser (note that the XML tags set forth below may not, in practice, be exactly as indicated (for example, extra tags are set forth and the <SIP>and <URL>tags are processed by proxy server 16 but never reach the attachment viewer 45):
The following is an example XML request added by the attachment viewer 45 to the HTTP request for the case of retrieving an embedded object:
The following is an example response XML string retrieved from the HTTP response:
The following is an example XML request sent by the attachment viewer 45 for enhancing a slide/page:
The following is an example response XML string:
The following is an example XML request for text of a ppt/pdf document that has already been rendered:
The following is an example response XML string:
Additional implementation aspects are as follows:
The web server 15 preferably populates HTTP responses with attachment viewer meta-data (i.e. ip used, url, etc) obtained from the attachment server 18, and accepts attachment viewer meta-data if present in the HTTP request header. (i.e. server ip to use, part, dom id, etc) and passes this information to the attachment server 18.
An IM server 17 may also be configured to support responding with attachment server meta-data by accepting attachment server meta-data and using it during transcoding (i.e. server ip, part, dom id, etc).
Other device application modifications may be implemented, as follows:
A FileExplorer application on the device 12 may be configured to provide a “View” verb in the display menu if the file extension matches one of a plurality of supported file extensions (doc, xis, pdf etc.) When the “View” menu item is selected, the File Explorer application (or a plugin) contacts the proxy server 16 MDS in a similar way as the browser plugin in order to create a transmission channel, whereupon the attachment server 18 converts the file to a viewer-readable format (e.g. UCS) for downloading to the device 12 via proxy server 16. The attachment viewer 45 is preferably invoked immediately after the document data starts to be downloaded. It will be understood that the attachment viewer application 45 registers itself as a File Explorer plugin for the viewer-readable format (e.g. UCS).
As discussed above, the persistence model for the attachment viewer plug-in to the mobile device browser is configured so as not to rely on the initial data cached by the browser and to generate only the minimum number of wireless requests required for viewing a previously downloaded document. Also, by aligning the functionality of the attachment viewer 45 for both email and browser document viewing, additional advantages may be realized such as the document data chunks being part of low memory manager implementation on the device 12 and the ability to backup and restore document data.
As discussed above, the PersistOperation results in partially retrieved documents and meta-data being persisted to persistent store 23, such that partially persisted data and metadata can be re-loaded along with generation of data requests, regardless of the application or transport type used. If the native document is available at the original location, then more data can be retrieved using the original transport. More particularly, data chunks are serialized to the binary file, together with information about the original location and transport used (e.g. email, browser, IM, etc.). In one embodiment, the meta-data includes source document (i.e. attachment) location address within the server 18 and the transport mechanism (e.g. email, browser, IM, etc.) used to retrieve the attachment.
Additional details of the persistence of partial document data according to an embodiment are set forth in
Thus, the first persisted data chunk contains all relevant information (i.e. meta-data) for later de-serializing of the data. When opened, it is de-serialized to recreate the memory data structure as it existed before saving (i.e. the same process as a backup/restore except that the serializing/de-serializing is performed on the partial document data and not on a hard disk).
The attachment viewer saving format preferably starts with a standard binary data header to indicate that the file is attachment viewer specific, followed by the type of document (e.g. attachment in an email, linked file, etc.), meta-data (e.g. remote link path on the web or on a remote machine) and then the binary document data. The binary data can contain multiple document data streams for the main document and embedded objects for documents from email or a single stream for linked documents.
As discussed above, three generic keys are used in accessing an atomic attachment in the persistent store 23: “message id” for email; “more part id” for email, and an “archive indicator” string indicating the archive index for a document in an archive). These values are used to initialize the docview display engine 43 and can have a different meaning for each calling application (like a hash code for the downloaded url) or can be generic, except for the archive indicator which has a predefined meaning. These values are also used to route a received data response to the active docview display engines showing the particular document.
Upon later requesting the document (83), the invoking application (e.g. email client, browser, IM, etc.) identifies the persistent store 23 and if the initial document data is not found in the store, the attachment viewer public classes (
If the initial document data has been saved to the store 23, device 12 de-serializes the data, parses the metadata (e.g. the original file url) and constructs a new persistent store instance therefrom (85). The resulting document, including embedded objects, is then displayed (87), following which additional document data requests may be issued (89) incorporating the de-serialized meta-data,
It will be noted that saving and serializing (79, 81) is performed by the Attachment Viewer plug-in 45 to the invoking application, so that the document data can be saved and viewed from a variety of different invoking applications. The invoking applications are containers that implement defined interfaces and user interactions that follow common rules, but operate on the downloaded data chinks rather than native document data. Therefore, the attachment viewer 45 can use different transport types to issue data requests.
The method set forth above is not limited to downloading attachment data from an Attachment Server 18. Native attachment downloads, which send attachment binary data from an Enterprise Server rather than data chinks from the Attachment Server, may also be performed. Native attachment download is useful for portable electronic devices having Microsoft Office™-type programs available. Such programs are capable of displaying .doc and .ppt files, for example, using the appropriate Office-type program on the portable electronic device. Other types of data may also be downloaded using the method disclosed herein.
A specific embodiment has been shown and described herein. However, modifications and variations may occur to those skilled in the art. All such modifications and variations are believed to be within the sphere and scope of the present embodiment.
