Embodiments relate to memory management, and more particularly to managing video memory and rendering video content in full-screen or windowed modes.
Content on a computer is frequently displayed on a display screen in one of two modes. One of the modes is called Full-Screen Mode, while the other mode is frequently referred to as Windowed Mode. Full-Screen Mode is used for applications that are the exclusive content provider for a display screen. In other words, when a single application displays content that occupies the full screen and there are no other applications that are displaying content on top of that screen, then that application might run in Full-Screen Mode. Windowed Mode is used when multiple applications, or processes, occupy or share the display screen concurrently. For example, a web browser application might be displayed such that it covers an entire display screen while a pop-up window having volume controls, or some other kind of accessory, might be displayed on top of that browser window. Another example might be a calendar reminder (e.g., issued by a calendaring program) that pops up in front of a word processing application that was previously occupying the full screen. In these situations, when multiple processes or applications are sharing the display screen, Windowed Mode is used to composite the content onto the display screen.
Both the Windowed Mode and Full-Screen Mode have various advantages and disadvantages. The advantage of Windowed Mode, obviously, is that content from multiple applications, or processes, can be composited onto a display screen at the same time. Full-Screen Mode, on the other hand, can have the advantage, for example, of reducing the need for buffer space in video memory and can allow certain types of content (e.g., video content) to display more efficiently and with less jitter, delay and/or other errors.
In a very generic way, the ability to manually switch between Windowed Mode and Full-Screen Mode is known in the art. However, there are challenges involved in automatically switching between the Windowed Mode and the Full-Screen Mode. Traditional systems may not automatically detect conditions that allow the system to switch between a Full-Screen Mode and Windowed Mode. Further complexity is added when a system for displaying video content is “double buffered.” Double buffering involves the use of multiple buffers to prepare video content, or other display content from an application, for rendering to a display screen.
A system for switching between Windowed Mode and Full-Screen Mode in a display is described herein. A window surface associated with the first application is detected automatically as being an exclusive window surface for a display screen. In response to detecting the exclusive window surface, the system automatically transitions to a Full-Screen Mode in which a graphics processor flushes content to a display screen. Included in this Full-Screen Mode is the ability to flip between a front-surface buffer and a back-surface buffer associated with the application. The system also automatically detects when a window surface associated with an application is not the exclusive window surface for the display. When the window surface is detected as being non-exclusive, the system automatically transitions to a Windowed Mode, in which the graphics processor flushes content to the display by flushing the system-frame buffer. The transition from a Full-Screen Mode to Windowed Mode includes a minimum number of buffer content copy operations between the front-surface buffer, the back-surface buffer of the application, and the system-frame buffer.
The following description includes discussion of figures having illustrations given by way of example of implementations of embodiments of the invention. The drawings should be understood by way of example, not by way of limitation. As used herein, references to one or more “embodiments” are to be understood as describing a particular feature, structure, or characteristic included in at least one implementation of the invention. Thus, phrases such as “in one embodiment” or “in an alternate embodiment” appearing herein describe various embodiments and implementations of the invention, and do not necessarily all refer to the same embodiment. However, they are also not necessarily mutually exclusive.
Embodiments described herein facilitate switching between Full-Screen Mode and Windowed Mode in a display system. Not only is the switching performed automatically, but also the switching is accomplished by using a minimum number of buffer content copy operations. This is significant, given that various embodiments described herein relate to application and processes that employ double buffering in video memory when rendering display content to a display screen.
Application 112 sends display content to kernel driver 116 for rendering to display 140. Application 112 may send content in the form of drawing commands, or the content may be sent as a completed window surface. As used herein, a “window surface” refers to the data necessary to render content from an application or process for display on the display screen 140. In various embodiments, a window surface may represent, for example, a single frame (e.g., a video frame) or a sequence of frames (e.g., a video clip, segment, movie, etc.) to be displayed on a displayed screen.
While window server 114 serves to render content to display 140, kernel driver 116 is notified by window server 114 of a window surface's visible rectangle changes. In other words, kernel driver 116 can detect various conditions and parameters associated with window server 114 to facilitate rendering content in Full-Screen Mode or Windowed Mode. In various embodiments, window surfaces are logically organized as rectangles, though it will be understand that other logical organizations of window surfaces could be employed. If a particular window surface is composed of a single visible rectangle, then it is possible that that single visible rectangle corresponds to a full screen rectangle on display 140. However, it is possible that a single visible rectangle only covers, for example, half of the display screen. To qualify for Full-Screen Mode, a single visible rectangle must encompass the entire screen, meaning that the size of the visible rectangle must be equal to the full size of the display screen. System-frame buffer 132 holds display content before it is flushed to display 140. Thus, if a window surface fills the entire system-frame buffer 132, it can be ascertained that the window surface covers the full screen of display 140. In other embodiments, different buffers (e.g., front buffer 134, back buffer 136, etc.) can be used to determine whether a window surface covers the full screen of a display. Though it is likely in various embodiments that the buffers described herein are of equal size, they are not necessarily equal in size.
In various embodiments, kernel driver 116 extracts visible rectangle information and window size information from window server 114 based on notifications from window server 114 and/or memory 130. Rectangle module 118 detects a number of visible rectangles in a window surface and determines whether the number of visible rectangles in the window surface is greater than one. Window server 114 logically organizes the window surface into visible rectangles, which information is made available to rectangle module 118. Widow-size module 120 detects the size of the window surface (e.g., based on buffer usage) and determines whether the size of the window surface is less than, or equal to, the size of the system-frame buffer. If, for a given window surface, the surface is composed of only one visible rectangle and the rectangle fills the entire system-frame buffer, the kernel driver 116 determines that the window surface is the exclusive window surface for the display 140. In other words, the kernel driver 116 knows that no other application is currently attempting to render content to display 140. If a window surface is the exclusive window surface being displayed by display screen 140, then Full-Screen Mode may be used as long as the bit depth of the window surface is equal to the bit depth of the display. For example, display 140 might be a 32-bit display, while the application currently rendering content to the display 140 might be a 16-bit application. In that scenario, the bit depths of the application and the display are not equal, meaning that some modification needs to be made to the window surface before rendering it to display 140. Such a window surface modification must be handled by window server 114 in Windowed Mode before rendering to display 140.
For this reason, bit-depth module 122 determines whether the bit depth of the application is equal to the bit depth of the display. Thus, if a window surface is the exclusive window surface for display and the bit depth of the window surface is equal to the bit depth of the display, then the window surface and the content associated with application 112 can be rendered in Full-Screen Mode.
If window server 114 detects that part of a window surface is clipped out, or, in other words, there are multiple visible rectangles defining the window surface, then window server 114 makes this information available to kernel driver 116 via rectangle module 118. Again, multiple visible rectangles signify that the current window surface is not the exclusive content provider for display 140. Window server 114 may also generate info that the window size for the window surface is less than the full size of system-frame buffer 132. In either case, kernel driver 116 determines that Full-Screen Mode can no longer be maintained and that switching to Windowed Mode is necessary.
In various embodiments, when application 112 sends drawing commands to create a window surface, that window surface is created in one of two buffers for the application. As shown in
In Windowed Mode, buffer content for an application (either in the front buffer, or the back buffer) must be moved to system-frame buffer 132 before being flushed to display 140. This buffer-content copy operation has some cost associated with it. In contrast, in Full-Screen Mode, buffer content for an application is flushed directly to display 140 (e.g., from either the front or back buffer) without having to be copied to system-frame buffer 132. In switching from Windowed Mode to Full-Screen Mode, display controller 124 can simply move a display content source pointer from system-frame buffer 132 to one of the two application buffers, 134 or 136. However, when switching from Full-Screen Mode back to Windowed Mode, the buffer contents must be reorganized so that data, and/or frames, are not lost.
Table 1, below, illustrates the various Windowed Mode to Full-Screen Mode memory configuration transitions:
One example of a transition from Windowed Mode to Full-Screen Mode is as follows. As shown in
Transitioning from one of the Full-Screen Mode states, as shown in
One example of a transition from Full-Screen Mode to Windowed mode is explained as follows. As shown in
Window surfaces associated with one or more graphics-related applications are monitored 410. In various embodiments, the monitoring is performed by a kernel driver in a graphics processor. However, other elements could be implemented to monitor window surfaces. As part of the monitoring process, it is determined 412 whether the window surface comprises more than one visible rectangle. As discussed above, a kernel driver may have visibility in various embodiments into the window server or may receive notifications from a window server. The window server makes various calculations for a window surface to composite the window surface, if necessary, with other window surfaces to be displayed concurrently on the display. The kernel driver, based on information from the window server, may have access to such calculations (e.g., number of visible rectangles in the window surface, size of window surface, etc.).
If it is determined there is more than one visible rectangle, then the current window surface is not the exclusive content provider for the display screen. In other words, multiple surfaces must be composited. Thus, Windowed Mode is used 414. Accordingly, if more than one visible rectangle is detected and the system is already in Windowed Mode, that mode will be maintained. In contrast, if more than one visible rectangle is detected and the system is currently in Full-Screen Mode, then the system transitions from Full-Screen Mode to Windowed Mode. In various embodiments, the transition from Windowed Mode to Full-Screen is performed based on the transitions described in Table 2 above. As described, the transitions of Table 2 minimize the number of buffer content copy operations to improve system efficiency.
If, during the monitoring, it is determined that there is only one visible rectangle for the window surface, it is then determined 416 whether the size of the window surface is equal to the size of the system-frame buffer. In some embodiments, the size of the window surface could be compared against the size of a different buffer or it could be compared against a threshold value. If the size is not equal to the system-frame buffer (or does not equal a threshold value, etc.), then the window surface is not an exclusive content provider for the display screen. In such a case Windowed Mode is, again, used 414 (either by maintaining Windowed Mode or transitioning to Windowed Mode).
If, however, the window size is equal to the system-frame buffer size (or equal to a threshold value, etc.), then it is determined whether the bit depth (e.g., 16 bit vs. 32 bit) of the window surface is equal to the bit depth of the display 418. (While bit depth is discussed herein as a factor in determining whether Full-Screen Mode can be used, other known compatibility factors can be considered instead of or in addition to bit-depth.) If the bit depth of the application buffer content is not equal to the bit depth of the display, then it is necessary to use Windowed Mode 414. But, if the bit depth of the buffer content is equal to the bit depth of the display, then Full-Screen Mode may be used 420.
Again, if the system is currently in Windowed Mode at the time it is determined to use Full-Screen Mode, then a transition to Full-Screen Mode is made. If the system is already in Full-Screen Mode, then Full-Screen Mode is maintained. In either case, the system continues to monitor the window surfaces being presented for display to determine whether to use Windowed Mode, or Full-Screen Mode.
According to various embodiments,
Processor 502 represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like. More particularly, the processor 502 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processor 502 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. Processor 502 is configured to execute the processing logic 522 for performing the operations and steps discussed herein.
The computer system 500 may further include a network interface device 516. The computer system 500 also may include a video display unit 510 (e.g., a liquid crystal display (LCD), light emitting diode (LED) display, a cathode ray tube (CRT)), and an input device 512 (e.g., a keyboard and/or mouse, etc.).
The secondary memory 518 may include a machine-readable storage medium (or more specifically a computer-readable storage medium) 524 on which is stored one or more sets of instructions (e.g., software 522) embodying any one or more of the methodologies or functions described herein. The software 522 may also reside, completely or at least partially, within the main memory 504 and/or within the processing device 502 during execution thereof by the computer system 500, the main memory 504 and the processing device 502 also constituting machine-readable storage media. The software 522 may further be transmitted or received over a network 520 via the network interface device 516.
In various embodiments, display controller 514 controls frame buffers 526 to provide display content (e.g., window surfaces) to video display 510.
While the machine-readable storage medium 524 is shown in an exemplary embodiment to be a single medium, the terms “machine-readable storage medium” or “computer-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The terms “machine-readable storage medium” or “computer-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine/computer and that cause the machine/computer to perform any one or more of the methodologies of the present invention. The terms “machine readable storage medium” or “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media.
Elements of embodiments may also be provided as a machine-readable or computer-readable medium for storing the machine-executable instructions. The machine or computer-readable medium may include, but is not limited to, flash memory, optical disks, CD-ROMs, DVD ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, or other type of machine-readable media suitable for storing electronic instructions. For example, embodiments of the invention may be downloaded as a computer program which may be transferred from a memory on a remote computer (e.g., a server) to a memory on a requesting computer (e.g., a client).
Various components described herein may be a means for performing the functions described herein. Each component described herein includes software, hardware, or a combination of these. The operations and functions described herein can be implemented as software modules, hardware modules, special-purpose hardware (e.g., application specific hardware, application specific integrated circuits (ASICs), digital signal processors (DSPs), etc.), embedded controllers, hardwired circuitry, etc.
Aside from what is described herein, various modifications may be made to the disclosed embodiments and implementations of the invention without departing from their scope. Therefore, the illustrations and examples herein should be construed in an illustrative, and not a restrictive sense.
Number | Name | Date | Kind |
---|---|---|---|
4853795 | Morton et al. | Aug 1989 | A |
4855936 | Casey et al. | Aug 1989 | A |
5343557 | Kiel et al. | Aug 1994 | A |
5357267 | Inoue | Oct 1994 | A |
5502808 | Goddard et al. | Mar 1996 | A |
5606707 | Tomassi et al. | Feb 1997 | A |
5692140 | Schmitt et al. | Nov 1997 | A |
5808629 | Nally et al. | Sep 1998 | A |
5945965 | Inoguchi et al. | Aug 1999 | A |
6078942 | Eisler et al. | Jun 2000 | A |
6128026 | Brothers, III et al. | Oct 2000 | A |
6348919 | Murphy | Feb 2002 | B1 |
6411302 | Chiraz | Jun 2002 | B1 |
6731756 | Pizano et al. | May 2004 | B1 |
7489318 | Wilt | Feb 2009 | B1 |
20020135585 | Dye et al. | Sep 2002 | A1 |
20020165922 | Wei | Nov 2002 | A1 |
20030140179 | Wilt et al. | Jul 2003 | A1 |
20030179208 | Lavelle | Sep 2003 | A1 |
20040130568 | Nagano et al. | Jul 2004 | A1 |
20050062760 | Twede | Mar 2005 | A1 |
20050168471 | Paquette | Aug 2005 | A1 |
20070024645 | Purcell et al. | Feb 2007 | A1 |
20070070074 | Jiang | Mar 2007 | A1 |
20070296718 | Tzruya et al. | Dec 2007 | A1 |
20080066006 | Kim | Mar 2008 | A1 |
20080122852 | Noyle et al. | May 2008 | A1 |
20080229232 | Schulz et al. | Sep 2008 | A1 |
20090310933 | Lee | Dec 2009 | A1 |
Entry |
---|
“Reading swap chain back buffer ”, (forum post) http://forums.xna.com/forums/p/15639/83023.aspx, (Aug. 11, 2008), 2 pages. |
Number | Date | Country | |
---|---|---|---|
20100289806 A1 | Nov 2010 | US |