INFORMATION PROCESSING DEVICE AND INFORMATION PROCESSING METHOD

Abstract

[Problem] The estimation accuracy of a class of an object is to be effectively improved. [Means of Solution] Provided is an information processing device including an estimation unit that estimates, based on an input image, a class of an object that is present in a real environment corresponding to an imaging range of the input image, wherein the object includes an acoustically useful object having an acoustic feature useful for class estimation, and the estimation unit estimates a class of the acoustically useful object based on acoustic data collected from around the acoustically useful object.

Description

Claims

1. An information processing device comprising: an estimation unit that estimates, based on an input image, a class of an object that is present in a real environment corresponding to an imaging range of the input image, wherein the object includes an acoustically useful object having an acoustic feature useful for class estimation, andthe estimation unit estimates a class of the acoustically useful object based on acoustic data collected from around the acoustically useful object.

2. The information processing device according to claim 1, wherein the estimation unit estimates the class of the acoustically useful object based on acoustic data collected at at least one sound collection position that is determined according to a position of the acoustically useful object in the real environment, which is estimated based on the input image.

3. The information processing device according to claim 1, wherein the estimation unit estimates the class of the acoustically useful object by using an estimator generated by machine learning using acoustic data related to the acoustically useful object as learning data.

4. The information processing device according to claim 1, wherein the estimation unit creates a composite image that visually shows the acoustically useful object.

5. The information processing device according to claim 4, wherein the estimation unit creates the composite image by using an estimator generated by machine learning in which acoustic data related to the acoustically useful object is input and an image related to the acoustically useful object is output.

6. The information processing device according to claim 5, wherein the estimator is generated by machine learning to reduce a difference between an image output from a neural network and an image of an acoustically useful object captured at the same time as acoustic data to be input to the neural network.

7. The information processing device according to claim 4, wherein the estimation unit creates a superimposed image in which the created composite image is superimposed on the input image.

8. The information processing device according to claim 1, wherein the acoustically useful object includes an object whose pixel-based class is difficult to estimate in the input image.

9. The information processing device according to claim 1, wherein the acoustically useful object includes an object in the real environment that does not appear as a subject in the input image.

10. The information processing device according to claim 9, wherein the estimation unit estimates the class of the acoustically useful object based on acoustic data collected at at least one sound collection position that is determined according to a position where the acoustically useful object may be present in the real environment, which is estimated based on the input image.

11. The information processing device according to claim 1, further comprising an acoustic collection unit that collects the acoustic data from around the acoustically useful object.

12. The information processing device according to claim 11, wherein the acoustic collection unit collects the acoustic data by utilizing a change in an optical signal accompanied by vibration of particles in air.

13. The information processing device according to claim 11, wherein the acoustic collection unit outputs a predetermined acoustic signal for the acoustically useful object and collects acoustic data related to a reflected sound reflected by the acoustically useful object.

14. The information processing device according to claim 13, wherein the acoustic collection unit outputs the acoustic signal at at least one output position that is determined according to a position of the acoustically useful object in the real environment, which is estimated based on the input image.

15. The information processing device according to claim 1, further comprising an imaging unit that captures the input image.

16. The information processing device according to claim 1, further comprising a presentation control unit that controls presentation of information related to the class of the acoustically useful object estimated by the estimation unit.

17. The information processing device according to claim 1, mounted on a moving object.

18. The information processing device according to claim 17, wherein the estimation unit controls the moving object based on the estimated class of the acoustically useful object.

19. An information processing method comprising: estimating by a processor, based on an input image, a class of an object that is present in a real environment corresponding to an imaging range of the input image, wherein the object includes an acoustically useful object having an acoustic feature useful for class estimation, andthe estimating includes estimating a class of the acoustically useful object based on acoustic data collected from around the acoustically useful object.