Patent Application
20140267727
The systems and methods described herein relate generally to determining the field of view of an image based on vehicle information, and, more specifically, the systems and methods described herein relate to changing the field of view of an image that is displayed in a vehicle, where the image is captured by an vehicle based image capturing device, and the field of view is determined by a vehicle computing device based on the velocity of the vehicle.
The systems and methods described herein can be used to determine a field of view for displaying an image captured from a vehicle mounted camera based on vehicle data.
In accordance with one embodiment, a system includes a receiver that is configured to receive an image having a first field of view and a processor that is communication with the receiver and configured to determine a second field of view based on vehicle data. The second filed of view is narrower than the first field of view. The processor is also configured to process the image to generate a processed image having the second field of view and output the processed image. The system also includes a transmitter that is in communication with the processor and configured to transmit the processed image.
In accordance with another embodiment, a method includes receiving by a processor vehicle data that is associated with a vehicle. The method also includes processing an image having a first field of view by the processor based at least in part on the vehicle data to generate a processed image having a second field of view narrower than the first field of view and outputting the processed image.
In accordance with another embodiment, a vehicle information system includes a means for capturing a forward-facing image from the vehicle, where the image has a first field of view, a means for processing the image to generate a processed image having a second field of view, the second field of view based at least in part on velocity data associated with the vehicle, and a means for displaying, in the vehicle, the processed image.
The systems, apparatuses, devices, and methods disclosed herein are described in detail by way of examples and with reference to the figures. It will be appreciated that modifications to disclosed and described examples, arrangements, configurations, components, elements, apparatuses, devices, systems, methods, etc. can be made and may be desired for a specific application. In this disclosure, any identification of specific techniques, arrangements, etc. are either related to a specific example presented or are merely a general description of such a technique, arrangement, etc. Identifications of specific details or examples are not intended to be, and should not be, construed as mandatory or limiting unless specifically designated as such.
The systems, apparatuses, devices, and methods disclosed herein describe systems, apparatuses, devices, and methods for selectively changing the field of view of a display based on vehicle information, with selected examples disclosed and described in detail with reference made to
References to components or modules generally refer to items that logically can be grouped together to perform a function or group of related functions. Like reference numerals are generally intended to refer to the same or similar components. Components and modules can be implemented in software, hardware, or a combination of software and hardware. The term “software” is used expansively to include not only executable code, but also data structures, data stores and computing instructions in any electronic format, firmware, and embedded software. The terms “information” and “data” are used expansively and includes a wide variety of electronic information, including but not limited to machine-executable or machine-interpretable instructions; content such as text, video data, and audio data, among others; and various codes or flags. The terms “information,” “data,” and “content” are sometimes used interchangeably when permitted by context. It should be noted that although for clarity and to aid in understanding some examples discussed herein might describe specific features or functions as part of a specific component or module, or as occurring at a specific layer of a computing device (for example, a hardware layer, operating system layer, or application layer), those features or functions may be implemented as part of a different component or module or operated at a different layer of a communication protocol stack.
The examples discussed below are examples only and are provided to assist in the explanation of the systems, apparatuses, devices, and methods described herein. None of the features or components shown in the drawings or discussed below should be taken as mandatory for any specific implementation of any of these the systems, apparatuses, devices, or methods unless specifically designated as mandatory. For ease of reading and clarity, certain components, modules, or methods may be described solely in connection with a specific figure. Any failure to specifically describe a combination or sub-combination of components should not be understood as an indication that any combination or sub-combination is not possible. Also, for any methods described, regardless of whether the method is described in conjunction with a flow diagram, it should be understood that unless otherwise specified or required by context, any explicit or implicit ordering of steps performed in the execution of a method does not imply that those steps must be performed in the order presented but instead may be performed in a different order or in parallel.
Referring now to
The vehicle computing device 106 can include computer executable instructions capable of executing on a computing platform such as a desktop, laptop, tablet, mobile computing device, an embedded processor, or other suitable hardware. The computer executable instructions can include software modules, processes, application programming interfaces or APIs, drivers, helper applications such as plug-ins, databases such as search and query databases, and other types of software modules or computer programming as would be understood in the art.
The vehicle 120 can include a cabin area 122 for occupants. The vehicle camera display system 100 can extend into the cabin area 122, can be completely within the cabin area 122, or can be viewable from the cabin area 122. The vehicle can also include vehicle electronics 124, and a vehicle network 126. The vehicle electronics 124 can provide vehicle data, including but not limited to vehicle velocity, speed, direction, acceleration, position, blinker activation, driving conditions, and other information. The vehicle network 126 can be a vehicle controller area network (CAN). The vehicle camera display system 100 can receive vehicle data. For example, the vehicle computing device 106 can be in communication with, and receive vehicle data from, the vehicle network 126. The vehicle computing device 106 can be physically connected via a wired connection such as an Ethernet connection, or other suitable data connection, to the vehicle network 126. The vehicle computing device 106 can use one or more wireless technologies to communicate through the vehicle network 126 with the vehicle electronics 124, including but not limited to WiFi™, Bluetooth™, ZigBee™, one of the IEEE 802.11x family of network protocols, or another suitable wireless network protocol.
The vehicle display 112 can display an image captured by the forward-facing vehicle-mounted camera 102. Referring now to
Referring to
Referring first to
Referring now to
The processor can use other suitable methods of determining a field of view for a processed image, including but not limited to using a lookup table to determine a field of view appropriate for the velocity of the vehicle, an algorithm for determining a field of view based on speeds or other vehicle data, a step algorithm, a curvilinear algorithm, a logarithmic algorithm, a proportional algorithm, or other suitable mapping or correlation of the field of view of the processed image to the vehicle data, such as speed, velocity or acceleration. The changes to the field of view, from a first processed image to subsequent processed images, can be smoothed, a hysteresis function can be applied, or other suitable methods of presenting changes to the field of view can be performed. As such, relatively rapid changes in field of view around speed thresholds can be prevented or reduced and sudden jump discontinuities in the field of view due to operational conditions can be mitigated.
A field of view of a processed image that is presented to an occupant of the vehicle can be configured to approximately correlate to the time of impact, based on vehicle velocity, with an object visualized in the field of view. By narrowing the field of view and resizing the image as speed increases, obstacles in the path of the vehicle can be made to appear larger in the displayed image, thereby bringing the obstacle to the driver's attention. For example, an animal, such as a deer, that is some distance away from the vehicle, may appear small, indistinct, or otherwise difficult to resolve either visually by the driver. Even if the vehicle is equipped with a forward-looking vehicle-mounted camera and associated display, if the image being displayed is an unmodified image, the animal may only occupy a relatively small portion of the display. At high speeds, a travelling vehicle may close the distance to the animal in a short time, providing only a limited amount of time for the driver to see the animal. By narrowing the field of view as the vehicle's speed increases, in accordance with the systems and methods described herein, the image presented to the driver can include an enlarged display of the animal, due to the resizing of the display caused by narrowing the field of view. As the vehicle approaches, the animal will continue to grow in size on the display, further alerting the driver or other occupants of the animal's presence in the roadway. This can provide a valuable, timely visual indicator to the driver that an animal, or any obstacle, is being approached. Similarly, by narrowing the field of view, the driver will be alerted to the presence of stalled or slower cars in the roadway ahead.
Referring now to
The computing device 500 also includes one or more memories 506, for example read only memory (ROM), random access memory (RAM), cache memory associated with the processor 502, or other memories such as dynamic RAM (DRAM), static ram (SRAM), flash memory, a removable memory card or disk, a solid state drive, and so forth. The computing device 500 also includes storage media such as a storage device that can be configured to have multiple modules, such as magnetic disk drives, floppy drives, tape drives, hard drives, optical drives and media, magneto-optical drives and media, compact disk drives, Compact Disk Read Only Memory (CD-ROM), Compact Disk Recordable (CD-R), Compact Disk Rewriteable (CD-RW), a suitable type of Digital Versatile Disk (DVD) or BluRay disk, and so forth. Storage media such as flash drives, solid state hard drives, redundant array of individual disks (RAID), virtual drives, networked drives and other memory means including storage media on the processor 502, or memories 506 are also contemplated as storage devices.
The network and communication interfaces 512 allow the computing device 500 to communicate with other devices across a network 514. The network and communication interfaces 512 can be an Ethernet interface, a radio interface, a Universal Serial Bus (USB) interface, or any other suitable communications interface and can include receivers, transmitter, and transceivers. For purposes of clarity, a transceiver can be referred to as a receiver or a transmitter when referring to only the input or only the output functionality of the transceiver. Example communication interfaces 512 can include wired data transmission links such as Ethernet and TCP/IP. The communication interfaces 512 can include wireless protocols for interfacing with private or public networks 514. For example, the network and communication interfaces 512 and protocols can include interfaces for communicating with private wireless networks such as a WiFi network, one of the IEEE 802.11x family of networks, or another suitable wireless network. The network and communication interfaces 512 can include interfaces and protocols for communicating with public wireless networks 512, using for example wireless protocols used by cellular network providers, including Code Division Multiple Access (CDMA) and Global System for Mobile Communications (GSM). A computing device 500 can use network and communication interfaces 512 to communicate with hardware modules such as a database or data store, or one or more servers or other networked computing resources. Data can be encrypted or protected from unauthorized access.
In various configurations, the computing device 500 can include a system bus 513 for interconnecting the various components of the computing device 500, or the computing device 500 can be integrated into one or more chips such as programmable logic device or application specific integrated circuit (ASIC). The system bus 513 can include a memory controller, a local bus, or a peripheral bus for supporting input and output devices 504, inertial devices 508, GPS and inertial devices 510, and communication interfaces 512. Example input and output devices 504 include keyboards, keypads, gesture or graphical input devices, motion input devices, touchscreen interfaces, one or more displays, audio units, voice recognition units, vibratory devices, computer mice, and any other suitable user interface. In a configuration, the input and output devices 504 can include one or more receivers 516 for receiving video signals from imaging devices, and one or more transmitters 518 for transmitting video signals to displays. The input and output devices 504 can also include video encoders and decoders, and other suitable devices for sampling or creating video signals and other associated circuitry. In a configuration, a transmitter includes the associated circuitry. In a configuration, a receiver includes the associated circuitry. For example, the receiver 516 can receive an NTSC video signal from a video camera, associated circuitry can capture the individual frames of video at a desired resolution to produce a full frame image, the processor 502 or another processing device can perform image processing on the full frame image to generate a processed image, associated circuitry can encode the processed image in a format suitable for display on a display, such as a video graphics array (VGA) or high definition media interface (HDMI) format, and the transmitter 518 can output a video signal in the appropriate format for display. An example GPS device 510 can include a GPS receiver and associated circuitry. Inertial devices 508 can include accelerometers and associated circuitry. The associated circuitry can include additional processors 502 and memories 506 as appropriate.
The processor 502 and memory 506 can include nonvolatile memory for storing computer-readable instructions, data, data structures, program modules, code, microcode, and other software components for storing the computer-readable instructions in non-transitory computer-readable mediums in connection with the other hardware components for carrying out the methodologies described herein. Software components can include source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, or any other suitable type of code or computer instructions implemented using any suitable high-level, low-level, object-oriented, visual, compiled, or interpreted programming language.
Referring now to
In process block 604, vehicle data is received. Vehicle data can include vehicle velocity, speed, direction, acceleration, blinker activation, steering wheel movement, and other information. In certain configurations, the vehicle data can be received from a vehicle controller area network (CAN). The vehicle data can also be received from any suitable source, including but not limited to information received from a Global Positioning System (GPS) device, mobile devices such as smartphones, inertial devices, user input, image processing determinations, and information from vehicle accessories. The vehicle data received in process block 604 can be received before, after or concurrent with the image data captured in process block 602. Processing continues to process block 606.
In process block 606, a processor receives the image data from the image capturing device captured in process block 602. The vehicle data received in process block 604 can be correlated with the image data captured in process block 602. Processing continues to process block 608.
In process block 608, a processor determines the field of view to be used for the processed image. To achieve a desired field of view, the processor can crop, resize, or perform other suitable image transformations to present a suitable field of view, including using the full frame image data as the processed image. The processor can use suitable methods of changing the field of view, including but not limited to using a lookup table to determine a field of view that is appropriate for the velocity of the vehicle, an algorithm for determining a field of view based on speeds or other vehicle data, a step algorithm, a curvilinear algorithm, a logarithmic algorithm, a proportional algorithm, or other suitable mapping of the field of view of the processed image to the vehicle data such as speed or velocity. Processing continues to process block 610.
In process block 610, a processor performs image processing to the image data to create a processed image. The processor can crop, resize, translate, or perform other suitable image transformations to present a suitable angle field of view in the processed image. Optionally, the changes to the field of view, from a first processed image to subsequent processed images, can be smoothed, a hysteresis function can be applied, or other suitable methods of presenting changes to the field of view can be performed. Such image processing techniques may seek to avoid rapid changes in field of view around speed thresholds or to prevent sudden jump discontinuities in the field of view. Processing continues to process block 612.
In process block 612, the processed image having the field of view determined by process block 608 is transmitted to the display. Processing continues to process block 614.
In process block 614, the processed image is displayed on a display device associated with the vehicle. The display device can be a display integrated into the vehicle, for example a display physically integrated in the dashboard of a vehicle. The display device can be any suitable display configured to provide the processed image to a vehicle occupant, including but not limited to a display mounted on the dashboard or attached to a vehicle structure, a mobile device such as a smartphone, a projection such as a heads up display, a wearable device such as glasses configured to display an image, or any other suitable display device. Processing continues to decision block 616.
In decision block 616, if there are more images to be display, processing returns to process block 602 to capture an additional image. Because images can be captured rapidly, for example video can be captured at 30 frames, or images, per second or higher, the received vehicle data operations of process block 604 need not be performed for each iteration. For example, the vehicle data operations of process block can be performed once every second, or approximately one per thirty operations of capturing and displaying the process image. If there are no more images to be displayed, operation terminates at end block 618 labeled END.
The above descriptions of various components, devices, apparatuses, systems, modules, and methods are intended to illustrate specific examples and describe certain ways of making and using the components, devices, apparatuses, systems, and modules disclosed and described here. These descriptions are neither intended to be nor should be taken as an exhaustive list of the possible ways in which these components, devices, apparatuses, systems, and modules can be made and used. A number of modifications, including substitution between or among examples and variations among combinations can be made. Those modifications and variations should be apparent to those of ordinary skill in this area after having read this document.
