This application relates to the communication field, and in particular, to a wireless projection method and apparatus.
Wireless projection is a technology in which screen information of a terminal such as a mobile phone or a tablet is coded and compressed, then wirelessly delivered to a large-screen device such as a television or a virtual reality (virtual reality, VR) device by using a Wi-Fi technology (a wireless local area network technology created by the Wi-Fi Alliance in the Institute of Electrical and Electronic Engineers (institute of electrical and electronic engineers, IEEE) 802.11 standard), and then decoded, displayed, and output.
An existing wireless projection compression and transmission technology is mostly performed in video frames, which has disadvantages such as a high end-to-end delay and a weak anti-interference capability, and greatly limits application of the wireless projection compression and transmission technology in scenarios such as office, games, and VR. A low-delay wireless projection solution combining source coding and a wireless channel transmission technology can resolve the foregoing problems to some extent. In this solution, a terminal continuously divides each frame of video data into a plurality of data slices (slices) through source coding, and uses a scalable video coding technology on each slice to obtain a plurality of scalable bitstreams (layers, for example, one basic layer and a plurality of enhancement layers) with scaled-quality (or resolution). The terminal performs Wi-Fi transmission on the scalable bitstream of the slice in a configured video transmission period (video service period, VSP). In this case, the terminal needs to transmit one slice in a fixed time period in the VSP. If the transmission is not completed, the terminal needs to discard the slice, so that normal transmission of subsequent slices is not affected. Because scalable video coding is performed on each slice, for each slice, when Wi-Fi transmission of the basic layer succeeds but transmission of the enhancement layer fails, a bitstream having poor quality can still be decoded, to achieve capabilities of low-delay transmission and adapting to a wireless channel transmission fluctuation.
However, wireless channel interference is usually burst interference. The foregoing solution resists a channel capacity fluctuation only through scalable coding, resulting in a limited capability. Especially, when interference is strong in Wi-Fi environments, the basic layer often fails to be correctly transmitted within a time period. In this case, image tearing and freezing and image quality deterioration occur in wireless projection. This affects user experience.
This application provides a wireless projection method and apparatus, to reduce image tearing and freezing and image quality deterioration occurring in wireless projection, and improve user experience.
According to a first aspect, a wireless projection method is provided. The wireless projection method is applied to a Wi-Fi communication system, and an electronic device is connected to a display device through wireless fidelity Wi-Fi. The wireless projection method includes: The electronic device transmits a scalable video coding slice of a first video frame to the display device in a first video transmission period; the electronic device obtains channel state information that is fed back by the display device and that is for transmitting any scalable video coding slice of the first video frame; and the electronic device selectively enables a retry mechanism based on the channel state information. The retry mechanism is that the electronic device transmits a first scalable video coding slice of a second video frame to the display device in a first time period in a second video transmission period, and transmits, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period. One another scalable video coding slice of the second video frame is transmitted in the at least one another time period. In this way, the electronic device can first transmit a scalable video coding slice of a video frame in a video transmission period, for example, transmit one scalable video coding slice in a time period in a video transmission period in an initial state. Certainly, if transmission of the scalable video coding slice is not completed in a corresponding time period, the scalable video coding slice is directly discarded. Then, the electronic device obtains the channel state information that is fed back by the display device and that is for transmitting any scalable video coding slice of the first video frame. For example, after each time period, the channel state information for transmitting the scalable video coding slice in the time period may be obtained. It may be understood that the channel state information may reflect a portion that is of the scalable video coding slice and that is discarded in the corresponding time period. For example, a channel state may be a total quantity of data packets of the scalable video coding slice transmitted in one time period and a quantity of data packets that are successfully transmitted. In this way, the electronic device may selectively enable the retry mechanism based on the channel state information. For example, if the electronic device determines, based on the channel state information, that channel quality is poor, the electronic device enables the retry mechanism; or when channel quality is good, the electronic device does not enable the retry mechanism; or when the retry mechanism is already enabled, the electronic device disables the retry mechanism when determining that channel quality is good. The retry mechanism is that the electronic device transmits one scalable video coding slice in a plurality of time periods in a subsequent video transmission period, for example, transmits a first scalable video coding slice of another video frame to a display device in a first time period in a subsequent video transmission period, and transmits, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period. One another scalable video coding slice of the second video frame is transmitted in the at least one another time period. In this way, when one scalable video coding slice has a discarded portion in a corresponding time period, the discarded portion may be transmitted in another time period. For example, when transmission of one or more of a basic layer and at least one enhancement layer of a scalable video coding slice of a slice is not completed in a corresponding time period, the basic layer and the at least one enhancement layer have an opportunity of being transmitted in a next time period. In this way, the display device may perform decoding with reference to the first scalable video coding slices transmitted in the two time periods, to ensure successful transmission of the scalable video coding slice as much as possible on the premise that a portion of a delay is sacrificed, reduce image tearing and freezing and image quality deterioration occurring in wireless projection, and improve user experience.
In a possible implementation, the scalable video coding slice includes one basic layer and at least one enhancement layer; and that the electronic device transmits a first scalable coding slice of a second video frame in a first time period in a second video transmission period includes that the electronic device transmits a basic layer and at least one enhancement layer of the first scalable video coding slice of the second video frame to the display device in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame. Generally, scalable video coding is performed on each slice of the first video frame to obtain the scalable video coding slice. Therefore, for each scalable video coding slice, based on the transmission priorities, when the basic layer is successfully transmitted but the enhancement layer fails to be transmitted, a bitstream having poor quality can still be decoded, to achieve capabilities of low-delay transmission and adapting to a wireless channel transmission fluctuation.
In a possible implementation, the scalable video coding slice includes the one basic layer and the at least one enhancement layer, the at least one another time period includes a second time period in the second video transmission period, and the another scalable video coding slice includes a second scalable video coding slice of the second video frame.
That the electronic device transmits, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period includes that the electronic device transmits, to the display device in the second time period based on the transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice and transmission priorities of a basic layer and at least one enhancement layer of the second scalable video coding slice, the portion that is of the first scalable video coding slice and that is discarded and the second scalable video coding slice. The transmission priority of the basic layer of the first scalable video coding slice is higher than the transmission priority of the basic layer of the second scalable video coding slice, the transmission priority of the basic layer of the second scalable video coding slice is higher than the transmission priority of the enhancement layer of the first scalable video coding slice, and the transmission priority of the enhancement layer of the first scalable video coding slice is higher than the transmission priority of the enhancement layer of the second scalable video coding slice. In this way, data transmitted in the second time period may be transmitted in a manner of preferentially transmitting the basic layer of the first scalable video coding slice, then transmitting the basic layer of the second scalable coding slice, then transmitting the enhancement layer of the first scalable coding slice, and finally transmitting the enhancement layer of the second scalable coding slice. This preferentially ensures that the basic layer of each scalable video coding slice is transmitted successfully in sequence in the corresponding time period, and then ensures that the enhancement layer of each scalable video coding slice is transmitted successfully in sequence in the time period.
In a possible implementation, the discarded portion of the first scalable video coding slice includes the basic layer and the at least one enhancement layer of the first scalable video coding slice; that the electronic device transmits, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period includes that the electronic device transmits the basic layer and the at least one enhancement layer of the first scalable video coding slice to the display device in the at least one another time period after the first time period. In this solution, if both the basic layer and the at least one enhancement layer of the first scalable video coding slice are discarded in the second video transmission period, that is, (transmission is not completed), the basic layer and the at least one enhancement layer of the first scalable video coding slice may be retransmitted in the at least one another time period after the first time period.
In a possible implementation, a discarded portion of the first scalable video coding slice includes the at least one enhancement layer of the first scalable video coding slice; that the electronic device transmits, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period includes that the electronic device transmits the at least one enhancement layer of the first scalable video coding slice to the display device in the at least one another time period after the first time period. In this solution, if only the at least one enhancement layer of the first scalable video coding slice is discarded (transmission is not completed) in the first time period, only the at least one enhancement layer of the first scalable video coding slice may be retransmitted in the at least one another time period after the first time period.
In a possible implementation, the method further includes: after the at least one another time period, reconstructing a reference frame based on the successfully transmitted portion of the first scalable video coding slice; and generating first scalable video coding of a third video frame based on the reference frame. In this way, when the electronic device transmits the third video frame, for the first scalable video coding slice of the second video frame, reference may be made to portions that are successfully transmitted in the first time period and the at least one another time period after the first time period. In this way, formed inter-predictive frame (predictive frame, P frame for short) coding can reduce an amount of data to be coded, and help ensure that the third video frame of the electronic device can be decoded based on an actually successfully transmitted portion after being transmitted to the display device.
According to a second aspect, a wireless projection methodis provided. The wireless projection applied to a Wi-Fi communication system, and an electronic device is connected to a display device through wireless fidelity Wi-Fi. The wireless projection method includes: The display device receives a scalable video coding slice of a first video frame that is transmitted by the electronic device to the display device in a first video transmission period; and the display device feeds back, to the electronic device, channel state information for transmitting the first video frame. The electronic device selectively enables a retry mechanism based on the channel state information. The display device decodes the scalable video coding slice transmitted by the electronic device in the first video transmission period; the display device receives a first scalable video coding slice of a second video frame transmitted by the electronic device in a first time period in a second video transmission period in the retry mechanism; and the display device receives a portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted by the electronic device in at least one another time period after the first time period. One another scalable video coding slice of the second video frame is transmitted in the at least one another time period; and the display device decodes the first scalable video coding slice of the second video frame transmitted by the electronic device in the first time period and a portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted in the at least one another time period after the first time period.
In a possible implementation, the scalable video coding slice includes one basic layer and at least one enhancement layer; and that the display device receives a first scalable video coding slice of a second video frame transmitted by the electronic device in a first time period in a second video transmission period includes: The display device receives the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame that are transmitted by the electronic device in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice.
In a possible implementation, the scalable video coding slice includes the one basic layer and the at least one enhancement layer; and that the display device receives a portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted by the electronic device in at least one another time period after the first time period includes: The display device receives the basic layer and the at least one enhancement layer that are of the first scalable video coding slice and that are transmitted by the electronic device in the at least one another time period after the first time period.
In a possible implementation, the scalable video coding slice includes the one basic layer and the at least one enhancement layer; and that the display device receives a portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted by the electronic device in at least one another time period after the first time period includes: The display device receives the at least one enhancement layer of the first scalable video coding slice transmitted by the electronic device in the at least one another time period after the first time period.
According to a third aspect, a wireless projection apparatus is provided, used in a Wi-Fi communication system. The wireless projection apparatus may be an electronic device, the wireless projection apparatus may be a module or a chip in the electronic device, or the electronic device may be a chip or a system on chip, and the wireless projection apparatus includes: A transmitter is configured to transmit, a scalable video coding slice of a first video frame to a display device in a first video transmission period. A receiver is configured to obtain channel state information that is fed back by the display device and that is for transmitting the first video frame. A processor is configured to selectively enable a retry mechanism based on the channel state information obtained by the receiver. The retry mechanism is that the transmitter is further configured to: transmit a first scalable video coding slice of a second video frame to the display device in a first time period in a second video transmission period, and transmit, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period. One another scalable video coding slice of the second video frame is transmitted in the at least one another time period.
In a possible implementation, the scalable video coding slice includes one basic layer and at least one enhancement layer; and the transmitter is specifically configured to transmit the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame to the display device in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame.
In a possible implementation, the scalable video coding slice includes the one basic layer and the at least one enhancement layer, the at least one another time period includes a second time period in the second video transmission period, and the another scalable video coding slice includes a second scalable video coding slice of the second video frame. The transmitter is specifically configured to transmit, to the display device in the second time period based on the transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice and transmission priorities of a basic layer and at least one enhancement layer of the second scalable video coding slice, the portion that is of the first scalable video coding slice and that is discarded and the second scalable video coding slice. The transmission priority of the basic layer of the first scalable video coding slice is higher than the transmission priority of the basic layer of the second scalable video coding slice, the transmission priority of the basic layer of the second scalable video coding slice is higher than the transmission priority of the enhancement layer of the first scalable video coding slice, and the transmission priority of the enhancement layer of the first scalable video coding slice is higher than the transmission priority of the enhancement layer of the second scalable video coding slice.
In a possible implementation, the discarded portion of the first scalable video coding slice includes the basic layer and the at least one enhancement layer of the first scalable video coding slice; and the transmitter is specifically configured to transmit the basic layer and the at least one enhancement layer of the first scalable video coding slice to the display device in the at least one another time period after the first time period.
In a possible implementation, the discarded portion of the first scalable video coding slice includes the at least one enhancement layer of the first scalable video coding slice; and the transmitter is specifically configured to transmit the at least one enhancement layer of the first scalable video coding slice to the display device in the at least one another time period after the first time period.
In a possible implementation, the processor is further configured to: after the at least one another time period, reconstruct a reference frame based on a successfully transmitted portion of the first scalable video coding slice; and generate first scalable video coding of a third video frame based on the reference frame.
According to a fourth aspect, a wireless projection apparatus is provided, used in a Wi-Fi communication system. The wireless projection apparatus may be a display device, the wireless projection apparatus may be a module or a chip in the display device, and the display device may be a chip or a system on chip, and the wireless projection apparatus includes: A receiver is configured to receive a scalable video coding slice of a first video frame that is transmitted by an electronic device to the display device in a first video transmission period. A transmitter is configured to feed back, to the electronic device, channel state information for transmitting the first video frame. The electronic device selectively enables a retry mechanism based on the channel state information. A processor is configured to decode the scalable video coding slice transmitted by the electronic device in the first video transmission period. A receiver is further configured to receive a first scalable video coding slice of a second video frame transmitted by the electronic device in a first time period in a second video transmission period in the retry mechanism; and receive a portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted by the electronic device in at least one another time period after the first time period. One another scalable video coding slice of the second video frame is transmitted in the at least one another time period. The processor is further configured to decode the first scalable video coding slice of the second video frame transmitted by the electronic device in the first time period and the portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted in the at least one another time period after the first time period.
In a possible implementation, the scalable video coding slice includes one basic layer and at least one enhancement layer; and the receiver is specifically configured to receive the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame that are transmitted by the electronic device in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice.
In a possible implementation, the scalable video coding slice includes the one basic layer and the at least one enhancement layer; and the receiver is specifically configured to receive the basic layer and the at least one enhancement layer of the first scalable video coding slice that are transmitted by the electronic device in the at least one another time period after the first time period.
In a possible implementation, the scalable video coding slice includes the one basic layer and the at least one enhancement layer; and the receiver is specifically configured to receive the at least one enhancement layer of the first scalable video coding slice transmitted by the electronic device in the at least one another time period after the first time period.
According to a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When the computer program is run on a computer, the computer is enabled to perform the method according to any one of the foregoing aspects.
According to a sixth aspect, a computer program product including instructions is provided. The computer program product includes computer program code. When the computer program code is run on a computer, the computer is enabled to perform the method according to any one of the foregoing aspects.
According to a seventh aspect, a communication system is provided. The communication system includes the wireless projection apparatus in the third aspect and the wireless projection apparatus in the fourth aspect. In an example, the wireless projection apparatus according to the third aspect may be an electronic device, for example, a mobile phone; and the wireless projection apparatus according to the fourth aspect may be a display device, for example, a radio television set.
For technical effects brought by any design manner of the second to the seventh aspects, refer to the technical effects brought by different design manners of the first aspect. Details are not described herein again.
The terms “first” and “second” mentioned below are merely intended for a purpose of description, and shall not be understood as an indication or implication of relative importance or implicit indication of the number of indicated technical features. Therefore, a feature limited by “first” or “second” may explicitly or implicitly include one or more features. In the description of embodiments of this application, unless otherwise specified, “a plurality of” means two or more than two.
Currently, a terminal can perform coding and compression on screen information of the terminal by using a wireless projection technology, then wirelessly deliver the screen information to a large-screen device such as a television or a VR device by using a Wi-Fi technology, and then decode, display, and output the screen information. A low-delay wireless projection solution combining source coding and a wireless channel transmission technology can reduce disadvantages such as a high end-to-end delay and a weak anti-interference capability to some extent. A main principle of the low-delay wireless projection solution combining the source coding and the wireless channel transmission technology is as follows: The terminal continuously divides each frame of video data into a plurality of slices based on the source coding, and uses a scalable video coding technology on each slice to obtain a plurality of layers with quality (or resolution) scaled. The terminal performs Wi-Fi transmission on a scalable bitstream of the slice in a configured VSP. In this case, the terminal needs to transmit one slice in a time period in the VSP. If the transmission is not completed, the terminal needs to discard the slice, so that normal transmission of subsequent slices is not affected. Because scalable video coding is performed on each slice, for each slice, when Wi-Fi transmission of a basic layer succeeds but transmission of an enhancement layer fails, a bitstream having poor quality can still be decoded, to achieve capabilities of low-delay transmission and adapting to a wireless channel transmission fluctuation. However, wireless channel interference is usually burst interference. The foregoing solution resists a channel capacity fluctuation only by using scalable coding, resulting in a limited capability. Especially, when interference is strong in Wi-Fi environments, the basic layer often fails to be correctly transmitted within a time period. In this case, image tearing and freezing and image quality deterioration occur in wireless projection. This affects user experience.
Embodiments of this application provide a wireless projection method. The wireless projection method may be applied to a Wi-Fi communication system. In the Wi-Fi communication system, an electronic device is connected to a display device through wireless fidelity Wi-Fi. The electronic device wirelessly delivers screen information of the electronic device to the display device, and then decodes, displays, and outputs the screen information. By using the wireless projection method provided in embodiments, the electronic device transmits a scalable video coding slice of a first video frame to the display device in a first video transmission period. One scalable video coding slice of the first video frame is correspondingly transmitted in one time period in the first video transmission period. For example, the electronic device performs scalable video coding on a plurality of data slices of the first video frame in sequence to generate a plurality of scalable video coding slices. For example, the electronic device codes a plurality of data slices of the first video frame into a plurality of the scalable video coding slices. The one scalable video coding slice may include a basic layer and at least one enhancement layer; and then the electronic device transmits the scalable video coding slice of the video frame on each time period in the video transmission period. The electronic device transmits one scalable video coding slice in a time period in a video transmission period in an initial state. Certainly, if transmission of the scalable video coding slice is not completed in a corresponding time period, the scalable video coding slice is directly discarded. Then, the electronic device obtains channel state information that is fed back by the display device and that is for transmitting any scalable video coding slice of the first video frame. For example, after each time period, the channel state information for transmitting the scalable video coding slice in the time period may be obtained. It may be understood that the channel state information may reflect a portion that is of the scalable video coding slice and that is discarded in the corresponding time period. For example, the channel state information may be a total quantity of data packets of the scalable video coding slice transmitted in one time period and a quantity of data packets that are successfully transmitted. In this way, the electronic device may selectively enable the retry mechanism based on the channel state information. For example, if the electronic device determines, based on the channel state information, that channel quality is poor, the electronic device enables the retry mechanism; or when channel quality is good, the electronic device does not enable the retry mechanism; or when the retry mechanism is already enabled, the electronic device disables the retry mechanism when determining that channel quality is good. The retry mechanism is that the electronic device transmits one scalable video coding slice in a plurality of time periods in a subsequent video transmission period, for example, transmits a first scalable video coding slice of another video frame to a display device in a first time period in a subsequent video transmission period, and transmits, to the display device in the at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period. One another scalable video coding slice of a second video frame is transmitted in the at least one another time period. In this way, when one scalable video coding slice has a discarded portion in a corresponding time period, the discarded portion may be transmitted in another time period. For example, when one or more of a basic layer and at least one enhancement layer of the scalable video coding slice of a slice is discarded (transmission is not completed) in a corresponding time period, the basic layer and the at least one enhancement layer have an opportunity of being transmitted in a next time period. In this way, the display device may perform decoding with reference to the first scalable video coding slices transmitted in the two time periods, to ensure successful transmission of the scalable video coding slice as much as possible on the premise that a portion of a delay is sacrificed, reduce image tearing and freezing and image quality deterioration occurring in wireless projection, and improve user experience. In addition, after the at least one another time period, a reference frame is reconstructed based on the successfully transmitted portion of the first scalable video coding slice, and the first scalable video coding of a third video frame is generated based on the reference frame. In this way, when the electronic device transmits the third video frame, for the first scalable video coding slice of the second video frame, reference may be made to portions that are successfully transmitted in the first time period and the at least one another time period after the first time period. In this way, formed inter-predictive frame (predictive frame, P frame for short) coding can reduce an amount of data to be coded, and help ensure that the third video frame of the electronic device can be decoded based on an actually successfully transmitted portion after being transmitted to the display device.
The following describes in detail a wireless projection method and an apparatus provided in embodiments of this application with reference to the accompanying drawings.
In some embodiments, the electronic device 100 sends content displayed on a screen to the display device 200 through the Wi-Fi network, and the display device 200 displays the content. In other words, in a wireless projection process, display content of the electronic device 100 is the same as that of the display device 200. For details, refer to a schematic diagram of a principle of wireless projection in
For example, the electronic device 100 includes a terminal device that has an image display function, such as a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), an in-vehicle device, a user terminal (user terminal, UT), a terminal device (user device, UD), user equipment (user equipment, UE), or an artificial intelligence (artificial intelligence, AI) device. A specific type of the electronic device 100 is not limited in this embodiment of this application.
The display device 200 includes, for example, a terminal device that can implement a large-screen display function, such as a laptop computer, a large-screen display device (such as a smart screen), a projection device, an AI device, or a tablet computer. A specific type of the display device 200 is not limited in this embodiment of this application.
Optionally, the electronic device 100 and the display device 200 in this embodiment of this application may be implemented by different devices. For example, the electronic device 100 and the display device 200 in this embodiment of this application may be implemented by using a wireless projection apparatus in
The processor 201 may be a central processing unit (central processing unit, CPU), or may be another general-purpose processor, a digital signal processor (digital signal processor, DSP), an application-specific integrated circuit (application-specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA), or another programmable logic device, a discrete gate or a transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.
The communication line 202 may be a circuit connecting the foregoing components to each other and transmitting information between the foregoing components.
The communication interface 204 is configured to communicate with another device. In embodiments of this application, the communication interface 204 may be a module, a circuit, a bus, an interface, a transceiver, or another apparatus that can implement a communication function, and is configured to communicate with another device. Optionally, when the communication interface 204 is the transceiver, the transceiver may be an independently disposed transmitter, and the transmitter may be configured to send information to another device. Alternatively, the transceiver may be an independently disposed receiver, and is configured to receive information from another device. Alternatively, the transceiver may be a component integrating functions of sending and receiving information. A specific implementation of the transceiver is not limited in embodiments of this application.
The memory 203 may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (read-only memory, ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (random access memory, RAM), used as an external cache. By way of example but not limitation, many forms of RAM are available, for example, static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchlink dynamic random access memory (synchlink DRAM, SLDRAM) and direct rambus random access memory (direct rambus RAM, DR RAM), or other magnetic storage devices, or any other medium that can be for carrying or storing expected program code in a form of instructions or a data structure and that can be accessed by a computer, but is not limited thereto. The memory 203 may exist independently, and be connected to the processor 201 by using the communication line 202. The memory 203 may alternatively be integrated with the processor 201.
The memory 203 is configured to store computer-executable instructions for implementing the solutions of this application, and the processor 201 controls the execution. The processor 201 is configured to execute the computer-executable instructions stored in the memory 203, to implement a determining parameter determining method provided in the following embodiments of this application.
It should be noted that the memory described in this specification aims to include but is not limited to these memories and any memory of another proper type.
Optionally, the computer-executable instructions in embodiments of this application may also be referred to as application program code, instructions, a computer program, or another name. This is not specifically limited in embodiments of this application.
During specific implementation, in an embodiment, the processor 201 may include one or more CPUs, for example, a CPU 0 and a CPU 1 in
During specific implementation, in an embodiment, the wireless projection apparatus 200 may include a plurality of processors, for example, the processor 201 and a processor 207 in
During specific implementation, in an embodiment, the wireless projection apparatus 200 may further include an output device 205 and an input device 206. The output device 205 communicates with the processor 201, and may display information in a plurality of manners. For example, the output device 205 may be a liquid crystal display (liquid crystal display, LCD), a light emitting diode (light emitting diode, LED) display device, a cathode ray tube (cathode ray tube, CRT) display device, or a projector (projector). The input device 206 communicates with the processor 201, and may receive an input from a user in a plurality of manners. For example, the input device 206 may be a mouse, a keyboard, a touchscreen device, or a sensor device.
It should be noted that the wireless projection apparatus 200 may be a general-purpose device or a dedicated device. A type of the apparatus is not limited in this embodiment of this application. A structure of the wireless projection apparatus 200 in
The following describes in detail, with reference to the accompanying drawings, the wireless projection method provided in embodiments of this application by using an example in which an electronic device is a mobile phone and a display device is a radio television set.
S501: A mobile phone establishes a Wi-Fi connection to a radio television set.
The mobile phone and the radio television set may establish the Wi-Fi connection by using a Wi-Fi P2P protocol. To establish the Wi-Fi connection between terminals, each terminal needs to have a transmission capability. In addition, the terminals need to know connection information about each other. The connection information may be a device identifier of the terminal, for example, an internet protocol (internet protocol, IP) address, a port number, or an account logged in on the terminal. The account logged in on the terminal may be an account provided by an operator for a user. The account logged in on the terminal may alternatively be an application account or the like. The transmission capability that the terminal has may be a near field communication capability, or may be a long-distance communication capability. To be specific, a wireless communication protocol used for establishing a connection between terminals, for example, a mobile phone and a radio television set, may be a near field communication protocol such as a Wi-Fi protocol, a Bluetooth protocol, or an NFC protocol, or may be a cellular network protocol. For example, the user may use the mobile phone to touch an NFC tag of the radio television set, and the mobile phone reads connection information stored in the NFC tag. For example, the connection information includes an IP address of the radio television set. Then, the mobile phone may establish, based on the IP address of the radio television set, the connection to the radio television set by using another protocol, for example, a Wi-Fi protocol For another example, Bluetooth functions and Wi-Fi functions are enabled on both the mobile phone and the radio television set. The mobile phone may broadcast a Bluetooth signal, to discover a surrounding terminal. For example, the mobile phone may display a discovered device list. The discovered device list may include an identifier of a device discovered by the mobile phone, for example, include an identifier of the radio television set. In a process of discovering a device, the mobile phone may also exchange the connection information, for example, the IP address, with a discovered device. Then, after the mobile phone receives an operation of selecting the identifier of the radio television set from the device list displayed by the user, the mobile phone may establish, based on the IP address of the radio television set, the connection to the radio television set by using the Wi-Fi protocol.
S502: The mobile phone negotiates a projection parameter with the radio television set.
For example, the mobile phone may negotiate the projection parameter by sending a Wi-Fi unicast frame to the radio television set. Specifically, the Wi-Fi unicast frame may carry a VSP, a VSP retry, and delay (delay) time of projection starting. A time period in the VSP is mainly for first transmission of a scalable video coding slice of a data slice in a video frame, and a time period in the VSP retry is mainly for re-transmission of a portion that is of the scalable video coding slice of the data slice in the video frame and that is discarded in the time period in the VSP. The time period in the VSP retry has a same length as the time period in the VSP. The delay time of projection starting is for projection start time agreed by both ends.
S503: The mobile phone transmits a scalable video coding slice of a first video frame to the radio television set in a first video transmission period.
One scalable video coding slice of the first video frame may be correspondingly transmitted in one time period in the first video transmission period. First, the mobile phone needs to perform scalable video coding on a plurality of data slices of the video frame in sequence to generate a plurality of scalable video coding slices. Refer to
S504: The mobile phone obtains channel state information that is fed back by the radio television set and that is for transmitting the first video frame.
The mobile phone transmits the coding 1, the coding 2, ..., and the like in sequence in the time periods VSP1, VSP2, ..., and the like in
S505: The radio television set decodes the scalable video coding slice transmitted by the mobile phone in the first video transmission period.
Specifically, after the coding 1 is transmitted to the radio television set in VSP1, the radio television set starts to decode, after VSP1 (for example, in VSP2), the scalable video coding slice received in VSP1.
S506: The mobile phone transmits the first scalable video coding slice of a second video frame to the radio television set in a first time period in a second video transmission period, and transmits, to the radio television set in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period. One another scalable video coding slice of the second video frame is transmitted in at least one another time period.
As shown in
In addition, after the at least one another time period, a reference frame is reconstructed based on a successfully transmitted portion of the first scalable video coding slice, and first scalable video coding of a third video frame is generated based on the reference frame. Refer to
S507: The radio television set decodes the first scalable video coding slice of the second video frame transmitted by the mobile phone in the first time period and a portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted in the at least one another time period after the first time period.
Correspondingly, before step S507, the radio television set receives the first scalable video coding slice of the second video frame transmitted by the mobile phone in the first time period in the second video transmission period, and receives the portion that is of the first scalable video coding slice and that is discarded in the first time period and that is transmitted by the mobile phone in the at least one another time period after the first time period. One another scalable video coding slice of the second video frame is transmitted in at least one another time period. In addition, it should be noted that the scalable video coding slice includes one basic layer and at least one enhancement layer. Specifically, the radio television set receives a basic layer and at least one enhancement layer of the first scalable video coding slice of the second video frame that are transmitted by the mobile phone in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice. In addition, when transmission of the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame is not completed in the first time period, the mobile phone receives the basic layer and the at least one enhancement layer of the first scalable video coding slice transmitted by the radio television set in the at least one another time period after the first time period. When transmission of the at least one enhancement layer of the first scalable video coding slice of the second video is not completed in the first time period, the mobile phone receives the at least one enhancement layer of the first scalable video coding slice transmitted by the radio television set in the at least one another time period after the first time period. Refer to
In this way, in the foregoing solution, the mobile phone can first transmit a scalable video coding slice of a video frame in a video transmission period, for example, transmit one scalable video coding slice in a time period in a video transmission period in an initial state. Certainly, if transmission of the scalable video coding slice is not completed in a corresponding time period, the scalable video coding slice is directly discarded. Then, the mobile phone obtains the channel state information that is fed back by the radio television set and that is for transmitting any scalable video coding slice of the first video frame. For example, after each time period, the channel state information for transmitting the scalable video coding slice in the time period may be obtained. It may be understood that the channel state information may reflect a portion that is of the scalable video coding slice and that is discarded in the corresponding time period. For example, the channel state may be a total quantity of data packets of the scalable video coding slice transmitted in one time period and a quantity of data packets that are successfully transmitted. In this way, the mobile phone may selectively enable the retry mechanism based on the channel state information. For example, if the mobile phone determines, based on the channel state information, that the channel quality is poor, the mobile phone enables the retry mechanism; or when the channel quality is good, the mobile phone does not enable the retry mechanism; or when the retry mechanism is already enabled, the mobile phone disables the retry mechanism when determining that the channel quality is good. The retry mechanism is that the mobile phone transmits one scalable video coding slice in a plurality of time periods in a subsequent video transmission period, for example, transmits a first scalable video coding slice of another video frame to the display device in a first time period in a subsequent video transmission period, and transmits, to the display device in at least one another time period after the first time period, a portion that is of the first scalable video coding slice and that is discarded in the first time period. One another scalable video coding slice of the second video frame is transmitted in the at least one another time period. In this way, when one scalable video coding slice has a discarded portion in a corresponding time period, the discarded portion may be transmitted in another time period. For example, when transmission of one or more of a basic layer and at least one enhancement layer of a scalable video coding slice of a slice is not completed in a corresponding time period, the basic layer and the at least one enhancement layer have an opportunity of being transmitted in a next time period. In this way, the radio television set may perform decoding with reference to the first scalable video coding slices transmitted in the two time periods, to ensure successful transmission of the scalable video coding slice as much as possible on the premise that a portion of a delay is sacrificed, reduce image tearing and freezing and image quality deterioration occurring in wireless projection, and improve user experience. In addition, after the at least one another time period, a reference frame is reconstructed based on the successfully transmitted portion of the first scalable video coding slice, and the first scalable video coding of the third video frame is generated based on the reference frame. In this way, when the electronic device transmits the third video frame, for the first scalable video coding slice of the second video frame, reference may be made to portions that are successfully transmitted in the first time period and the at least one another time period after the first time period. In this way, formed inter-predictive frame (predictive frame, P frame for short) coding can reduce an amount of data to be coded, and help ensure that the third video frame of the electronic device can be decoded based on an actually successfully transmitted portion after being transmitted to the display device.
It may be understood that, to implement the foregoing functions, the mobile phone and the radio television set include corresponding hardware structures and/or software modules for performing the functions. A person skilled in the art should be easily aware that, in combination with units and algorithm operations of the examples described in embodiments disclosed in this specification, this application can be implemented by hardware or a combination of hardware and computer software. Whether a function is performed by hardware or hardware driven by computer software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of this application.
In embodiments of this application, the mobile phone may be divided into functional modules based on the foregoing method examples. For example, each functional module may be divided based on each function, or two or more functions may be integrated into one processing module. The integrated module may be implemented in a form of hardware, or may be implemented in a form of a software functional module. It should be noted that, in embodiments of this application, module division is an example, and is merely a logical function division. In actual implementation, another division manner may be used.
For example, when each functional module is divided in an integrated manner,
In a possible implementation, the wireless projection apparatus 80 in
Optionally, the scalable video coding slice includes one basic layer and at least one enhancement layer; and the sending unit 801 is specifically configured to transmit the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame to the display device in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame.
Optionally, the scalable video coding slice includes the one basic layer and the at least one enhancement layer, the at least one another time period includes a second time period in the second video transmission period, and the another scalable video coding slice includes a second scalable video coding slice of the second video frame. The sending unit 801 is specifically configured to transmit, to the display device in the second time period based on the transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice and transmission priorities of a basic layer and at least one enhancement layer of the second scalable video coding slice, the portion that is of the first scalable video coding slice and that is discarded and the second scalable video coding slice. The transmission priority of the basic layer of the first scalable video coding slice is higher than the transmission priority of the basic layer of the second scalable video coding slice, the transmission priority of the basic layer of the second scalable video coding slice is higher than the transmission priority of the enhancement layer of the first scalable video coding slice, and the transmission priority of the enhancement layer of the first scalable video coding slice is higher than the transmission priority of the enhancement layer of the second scalable video coding slice.
Optionally, the discarded portion of the first scalable video coding slice includes the basic layer and the at least one enhancement layer of the first scalable video coding slice; and the sending unit 801 is specifically configured to transmit the basic layer and the at least one enhancement layer of the first scalable video coding slice to the display device in the at least one another time period after the first time period.
Optionally, the discarded portion of the first scalable video coding slice includes the at least one enhancement layer of the first scalable video coding slice; and the sending unit 801 is specifically configured to transmit the at least one enhancement layer of the first scalable video coding slice to the display device in the at least one another time period after the first time period.
Optionally, the processing unit 802 is further configured to: after the at least one another time period, reconstruct a reference frame based on a successfully transmitted portion of the first scalable video coding slice; and generate the first scalable video coding of the third video frame based on the reference frame.
All related content of the operations in the foregoing method embodiments may be cited in function descriptions of the corresponding functional modules. Details are not described herein again.
In this embodiment, the wireless projection apparatus 80 is presented in a form of functional modules divided in an integrated manner. The module herein may be an ASIC, a circuit, a processor that executes one or more software or firmware programs, a memory, an integrated logic circuit, and/or another component capable of providing the foregoing functions. In a simple embodiment, a person skilled in the art may figure out that the wireless projection apparatus 80 may be in the form in
For example, the processor 201 may invoke the computer-executable instructions stored in the memory 203 in
For example, functions/implementation processes of the sending unit 801, the processing unit 802 and the receiving unit 803 in
The wireless projection apparatus 80 provided in this embodiment may perform the wireless projection method. Therefore, for technical effects that can be achieved by the wireless projection apparatus 80, refer to the foregoing method embodiments. Details are not described herein again.
In embodiments of this application, the radio television set may be divided into functional modules based on the foregoing method examples. For example, each functional module may be divided based on each function, or two or more functions may be integrated into one processing module. The integrated module may be implemented in a form of hardware, or may be implemented in a form of a software functional module. It should be noted that, in embodiments of this application, module division is an example, and is merely a logical function division. In actual implementation, another division manner may be used.
For example, when each functional module is divided in an integrated manner,
In a possible implementation, the wireless projection apparatus 90 in
Alternatively, the scalable video coding slice includes one basic layer and at least one enhancement layer; and the receiving unit 901 is specifically configured to receive the basic layer and the at least one enhancement layer of the first scalable video coding slice of the second video frame that are transmitted by the electronic device in the first time period in the second video transmission period based on transmission priorities of the basic layer and the at least one enhancement layer of the first scalable video coding slice.
Optionally, the scalable video coding slice includes the one basic layer and the at least one enhancement layer; and the receiving unit 901 is specifically configured to receive the basic layer and the at least one enhancement layer of the first scalable video coding slice that are transmitted by the electronic device in the at least one another time period after the first time period.
Optionally, the scalable video coding slice includes the one basic layer and the at least one enhancement layer; and the receiving unit 901 is specifically configured to receive the at least one enhancement layer of the first scalable video coding slice transmitted by the electronic device in the at least one another time period after the first time period.
All related content of the operations in the foregoing method embodiments may be cited in function descriptions of the corresponding functional modules. Details are not described herein again.
In this embodiment, the wireless projection apparatus 90 is presented in a form of functional modules divided in the integrated manner. The module herein may be an ASIC, a circuit, a processor that executes one or more software or firmware programs, a memory, an integrated logic circuit, and/or another component capable of providing the foregoing functions. In a simple embodiment, a person skilled in the art may figure out that the wireless projection apparatus 90 may be in the form in
For example, the processor 201 may invoke the computer-executable instructions stored in the memory 203 in
For example, functions/implementation processes of the receiving unit 901, the processing unit 902 and the sending unit 903 in
The wireless projection apparatus 90 provided in this embodiment may perform the wireless projection method. Therefore, for technical effects that can be achieved by the wireless projection apparatus 90, refer to the foregoing method embodiments. Details are not described herein again.
Optionally, an embodiment of this application further provides a wireless projection apparatus (for example, the wireless projection apparatus may be a chip or a chip system). The wireless projection apparatus includes a processor and an interface, and the processor is configured to read instructions to perform the method in any one of the foregoing method embodiments. In a possible design, the wireless projection apparatus further includes a memory. The memory is configured to store necessary program instructions and necessary data. The memory may invoke program code stored in the memory, to instruct the wireless projection apparatus to perform the method in any one of the foregoing method embodiments. Certainly, the memory may alternatively not be in the wireless projection apparatus. When the wireless projection apparatus is a chip system, the wireless projection apparatus may include a chip, or may include a chip and another discrete component. This is not specifically limited in embodiments of this application.
Specifically, when the wireless projection apparatus 80 in
When the wireless projection apparatus in
All or a part of the foregoing embodiments may be implemented by software, hardware, firmware, or any combination thereof. When a software program is used to implement embodiments, embodiments may be implemented completely or partially in a form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the procedure or functions according to embodiments of this application are all or partially generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or another programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or may be transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired (for example, a coaxial cable, an optical fiber, or a digital subscriber line (digital subscriber line, DSL)) or wireless (for example, infrared, radio, or microwave) manner. The computer-readable storage medium may be any usable medium accessible by a computer, or a data storage device, such as a server or a data center, integrating one or more usable media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk drive, or a magnetic tape), an optical medium (for example, a DVD), a semiconductor medium (for example, a solid state drive (solid state drive, SSD)), or the like. In embodiments of this application, the computer may include the apparatus described above.
Although this application is described with reference to embodiments, in a process of implementing this application that claims protection, a person skilled in the art may understand and implement another variation of the disclosed embodiments by viewing the accompanying drawings, disclosed content, and appended claims. In the claims, “comprising” (comprising) does not exclude another component or another step, and “a” or “one” does not exclude a case of multiple. A single processor or another unit may implement several functions enumerated in the claims. Some measures are recorded in dependent claims that are different from each other, but this does not mean that these measures cannot be combined to produce a better effect.
Although this application is described with reference to specific features and embodiments thereof, it is clear that various modifications and combinations may be made to them without departing from the spirit and scope of this application. Correspondingly, the specification and accompanying drawings are merely example description of this application defined by the accompanying claims, and are considered as any of or all modifications, variations, combinations or equivalents that cover the scope of this application. It is clearly that a person skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. This application is intended to cover these modifications and variations of this application, provided that they fall within the scope of the claims and their equivalent technologies of this application.
This application is a continuation of International Application No. PCT/CN2020/141022, filed on Dec. 29, 2020, the disclosure of which is hereby incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2020/141022 | Dec 2020 | WO |
Child | 18343141 | US |