HEEL POSITION ESTIMATION DEVICE, HEEL POSITION ESTIMATION METHOD, AND NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIUM

Abstract

In a heel position estimation device, a contour extractor extracts, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion. A determination unit determines the type of the contour shape. A heel estimation unit estimates a heel point position in the contour shape, based on the type of the contour shape determined.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Japanese Patent Application No. 2023-109130, filed Jul. 3, 2023, the contents of which are hereby incorporated by reference.

BACKGROUND

Technical Field

The present disclosure relates to a heel position estimation device, a heel position estimation method, and a non-transitory computer-readable storage medium.

Background Information

There is known technology related to measuring a foot size, such as a foot length, of a measurement subject from an image including at least one foot of the measurement subject (hereinafter, referred to as a “one-foot image”). For example, Japanese Unexamined Patent Application Publication No. 2019-45463 describes the technology of acquiring, from an image capturing unit, one-foot image obtained by capturing, when a foot is placed on a foot placement sheet at its sole and the heel end of the foot is set in line with the lower side of the foot placement sheet, an image of the entire foot placement sheet and foot. Also, in this technology, an actual length ratio is calculated based on the one-foot image by dividing the actual length of the long side of the foot placement sheet by the number of pixels on the long side of the foot placement sheet.

SUMMARY

It has been determined that, in the technology described in Japanese Unexamined Patent Application Publication No. 2019-45463, the measurement subject himself or herself needs to set the heel end of his or her foot in line with the lower side of the foot placement sheet, and hence, there is room for improvement in measurement accuracy. In order to accurately measure the foot length of the measurement subject, estimating the position of the heel point of the measurement subject in the one-foot image is required.

The present disclosure has been made in view of such a situation, and a general purpose thereof is to provide a technology for enabling appropriate estimation of a heel point position from a one-foot image of a measurement subject.

A heel position estimation device according to one embodiment of the present disclosure includes: a contour extractor that extracts, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion; a determination unit that determines the type of the contour shape; and a heel estimation unit that estimates a heel point position in the contour shape, based on the type of the contour shape determined.

Another embodiment of the present disclosure relates to a heel position estimation method. This method includes: extracting, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion; determining the type of the contour shape; and estimating a heel point position in the contour shape, based on the type of the contour shape determined.

Yet another embodiment of the present disclosure relates to a program. This program causes a computer to implement: extracting, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion; determining the type of the contour shape; and estimating a heel point position in the contour shape, based on the type of the contour shape determined.

Optional combinations of the aforementioned constituting elements, and implementation of the present disclosure, including the constituting elements and expressions, in the form of methods, apparatuses, programs, transitory or non-transitory storage medium storing programs, or systems can also be practiced as additional modes of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, with reference to the accompanying drawings which are meant to be exemplary, not limiting, and wherein like elements are numbered alike in several figures, in which:

FIG. 1 illustrates a schematic diagram that shows an overview of a heel position estimation system;

FIG. 2 illustrates a functional block diagram that shows a schematic configuration of the heel position estimation system shown in FIG. 1;

FIG. 3 illustrates a functional block diagram that shows a schematic configuration of a computing unit of a heel position estimation server shown in FIG. 1;

FIGS. 4A and 4B illustrate schematic diagrams that show an example of a contour shape of a foot part extracted from a one-foot image by a contour extractor shown in FIG. 3;

FIG. 5 illustrates a flowchart that shows an example of processing performed by the heel position estimation server shown in FIG. 1;

FIG. 6 illustrates a flowchart that specifically shows processing performed by a determination unit at the step S14 and processing performed by a heel estimation unit at the step S16 shown in FIG. 5, in an example of the operations performed the heel position estimation server shown in FIG. 1;

FIG. 7A illustrates an example of a contour shape determined to be a first type; FIG. 7B illustrates an example of a contour shape determined to be a second type;

FIG. 8A illustrates another example of a contour shape determined to be the second type; FIG. 8B illustrates an example of a contour shape determined to be a third type; and

FIG. 9 illustrates a schematic diagram that shows foot length correction performed by a foot length correction unit shown in FIG. 3.

DETAILED DESCRIPTION

The disclosure will now be described by reference to the preferred embodiments. This does not intend to limit the scope of the present disclosure, but to exemplify the disclosure.

In the following, the present disclosure will be described based on a preferred embodiment with reference to each drawing. In the embodiment and modifications, like reference characters denote like or corresponding constituting elements, and repetitive description will be omitted as appropriate.

FIG. 1 illustrates a schematic diagram that shows an overview of a heel position estimation system 100. The heel position estimation system 100 includes a user terminal 10 and a heel position estimation server 20 connected to each other via wireless communication 16 and a network 18. A measurement subject shoots an image (a one-foot image) that includes his or her own one foot 12, using a camera function of the user terminal 10. The user terminal 10 transmits the captured one-foot image to the heel position estimation server 20 via the wireless communication 16 and the network 18. The heel position estimation server 20 estimates the position of a heel point in the received one-foot image. The heel position estimation server 20 can further estimate a toe position from the received one-foot image and can calculate the foot length based on the estimated toe position and the estimated heel point position. The heel position estimation server 20 can transmit foot size information including the calculated foot length to the user terminal 10 and display the foot size information on the screen of the user terminal 10.

Each of the user terminal 10 and the heel position estimation server 20 can be constituted by a mobile terminal or a computer that includes a central processing unit (CPU), a graphics processing unit (GPU), a random access memory (RAM), a read only memory (ROM), an auxiliary storage device, a display device, a communication device, and the like, and by a program stored in the mobile terminal or computer. For example, a program executed on the heel position estimation server 20 side can be used from the user terminal 10 side via the network 18. Also, the functions of the heel position estimation system can be implemented by a stand-alone device equipped with the functions of the user terminal 10 and the heel position estimation server 20, and a measurement subject can be able to directly operate the stand-alone device to execute a heel position estimation program. The stand-alone device can be provided in the form of, besides a personal computer, a mobile terminal, such as a smartphone, or a tablet terminal, along with a program stored in the computer or terminal. Alternatively, the device can be implemented in the form of a terminal installed in a shoe store and operated by an assistant such as a salesperson, along with a program stored in the terminal.

The “heel position estimation device” in the present specification can mean the entire heel position estimation system 100 or can mean the heel position estimation server 20. In the present embodiment, the heel position estimation server 20 substantially corresponds to the “heel position estimation device” because many of the characteristic functions included in the “heel position estimation device” are implemented such as to be included in the heel position estimation server 20. The characteristic functions of the “heel position estimation device” can also be distributed between the user terminal 10 and the heel position estimation server 20 or can also be implemented such that many of them are included in the user terminal 10.

FIG. 2 illustrates a functional block diagram that shows a schematic configuration of the heel position estimation system 100. Each of the functional blocks shown in the diagrams, including FIG. 2, can be implemented by an element such as a processor or memory of a computer or by a mechanism in terms of hardware, and by a computer program or the like in terms of software. FIG. 2 illustrates functional blocks implemented by coordination of those components. These functional blocks can be implemented in a variety of forms by hardware only, software only, or a combination thereof.

The user terminal 10 includes an image acquirer 30, a display unit 32, a communication unit 34, and an operation processing unit 36. The operation processing unit 36 accepts operation input from a measurement subject. The operation processing unit 36 accepts, from the measurement subject, an instruction regarding the start of a series of foot size measurement programs including the heel position estimation program. Based on the instruction accepted by the operation processing unit 36, the display unit 32 starts screen display of the contents of the foot size measurement programs and displays contents related to the measurement procedure. The operation processing unit 36 and the display unit 32 can be constituted by a touch panel, for example, in terms of hardware.

The image acquirer 30 captures an image of one foot of the measurement subject to acquire a one-foot image. The one-foot image is an image of a region including at least the heel point of the one foot 12. When the heel position estimation server 20 includes a toe estimation unit 58, which will be described later, the one-foot image is an image of a region further including at least the toe of the one foot 12. The one-foot image can be an image that can be captured by the measurement subject himself or herself by operating the user terminal 10. In this case, as shown in FIG. 1, for example, the measurement subject can use, as the one-foot image, an image of his or her own one foot 12 shot from diagonally above on the medial side. The one-foot image can be an image of the one foot 12 shot from diagonally above on the lateral side.

The one-foot image can be captured to include the one foot 12 together with a predetermined reference object, as described below. The “reference object” is an object to be captured together with one foot in the image, as a comparison object used to measure the foot length or the foot width of the one foot based on the one-foot image, and is a planar or three-dimensional object of which the size and shape are predetermined. The “reference object” can be a general-purpose product as long as the size and shape thereof are determined or can be an exclusive product of which the size and shape are determined exclusively for the measurement in the present embodiment. As a reference object in the present embodiment, a planar object of predetermined size and shape, such as A4 size (210 mm×297 mm) paper, can be used. The color of the paper can preferably be white but can be another color. The one-foot image in the present embodiment is an image obtained by shooting one foot along with the entire A4 white paper as the reference object on which the one foot is placed. It is premised that the A4 white paper is placed on a non-white floor during a shoot, so that the A4 white paper can be accurately extracted from the one-foot image by image recognition. The one-foot image can be an image that includes other objects besides the one foot 12 of a measurement subject and the reference object, such as part of the other foot of the measurement subject.

The image acquirer 30 can be constituted by a camera module, for example, in terms of hardware. The communication unit 34 transmits the one-foot image to the heel position estimation server 20 via the network 18 and receives the foot size information from the heel position estimation server 20 to transmit the information to the display unit 32. The communication unit 34 can be constituted by a wireless communication module for wireless LAN communication or cellular phone communication, for example, in terms of hardware. The display unit 32 displays, on the screen, the one-foot image and the foot size information of the measurement subject.

The heel position estimation server 20 includes a communication unit 40, a computing unit 42, a storage unit 44, and an output unit 46. The communication unit 40 receives a one-foot image from the user terminal 10 and transmits the computation results from the computing unit 42, such as the foot size information, to the user terminal 10. The communication unit 40 can be constituted by a communication module for a wired LAN, for example, in terms of hardware.

The computing unit 42 performs computing processing including the process of estimating the heel point position, based on a one-foot image received by the communication unit 40. The computing processing performed by the computing unit 42 will be detailed later.

The computing unit 42 stores the computation results in the storage unit 44. The computation results stored in the storage unit 44 are transmitted from the output unit 46 to the user terminal 10 via the communication unit 40.

FIG. 3 illustrates a functional block diagram that shows a schematic configuration of the computing unit 42 of the heel position estimation server 20. The computing unit 42 includes an image reader 50, a contour extractor 52, a determination unit 54, and a heel estimation unit 56.

The image reader 50 reads a one-foot image received by the communication unit 40 from the user terminal 10. From a one-foot image of a measurement subject read by the image reader 50, the contour extractor 52 extracts a contour shape of a foot part that includes at least part of a heel portion. FIGS. 4A and 4B illustrate schematic diagrams that show an example of a contour shape 74 of a foot part extracted by the contour extractor 52 from a one-foot image 70.

FIG. 4A illustrates a schematic diagram that shows the position of a foot part of which the contour shape 74 is extracted in the one-foot image 70. In the example shown in FIG. 4A, the one-foot image 70 is a two-dimensional image captured from diagonally above on the medial side such that a one foot placed on white paper 80 as the reference object is entirely captured. As will be described later, the heel estimation unit 56 estimates one point on the contour shape 74 extracted by the contour extractor 52 from the one-foot image 70, to be the heel point position. Therefore, the one-foot image 70 is an image captured from the medial side or lateral side of one foot such that the heel point is present on the contour of the captured one foot.

In the one-foot image 70, a region corresponding to a foot part of which the contour shape 74 is extracted is indicated as a cut-out region 72. The cut-out region 72 includes at least part of the heel portion but cannot include the entire heel portion. The cut-out region 72 can also include a portion of the one foot other than the heel portion. However, for the estimation of the heel point position by the heel estimation unit 56, the cut-out region 72 needs to include the heel point. As will be described later, the heel estimation unit 56 estimates the heel point position based on the type of the contour shape 74 determined by the determination unit 54. Accordingly, at the time when the contour extractor 52 extracts the contour shape 74, the estimated heel point position is unknown. Therefore, the contour extractor 52 needs to set, as the cut-out region 72, a region that includes all possible heel point positions to be estimated no matter which type the contour shape 74 is determined to be. The cut-out region 72 can be set based on a predetermined algorithm, such as pattern recognition. The shape of the cut-out region 72 is not limited to a rectangle as shown in FIG. 4A and can be an arbitrary shape.

FIG. 4B illustrates a schematic diagram that shows an example of the contour shape 74 extracted by the contour extractor 52. The contour extractor 52 extracts the contour shape 74 by detecting the contour of one foot shown in the one-foot image 70 and identifying a shape along the contour thus detected. For example, by binarizing the cut-out region 72, the contour extractor 52 can detect the contour between the region occupied by the one foot and the region occupied by the white paper 80 in the cut-out region 72. Along the contour thus detected, the contour extractor 52 extracts the contour shape 74. The data of the contour shape 74 can be in the raster format in which the contour shape 74 is represented by a set of points in a two-dimensional coordinate system, or in the vector format in which the contour shape 74 is represented by a set of lines in a two-dimensional coordinate system. When the data of the contour shape 74 is in the raster format as shown in FIG. 4B, the interval between points constituting the contour shape 74 can be arbitrary, and the number of points can vary depending on the shape of the contour shape 74 included in the cut-out region 72. When extracting the contour shape 74, the contour extractor 52 can perform an arbitrary smoothing process to remove noise from the one-foot image 70, for example.

As described above, the contour shape 74 thus extracted is two-dimensional information extracted from the one-foot image 70. Even if the same one foot of the same measurement subject is captured, the one-foot image 70 can differ depending on the capturing direction or capturing angle. In addition, there are individual differences in the three-dimensional shape of one foot of the measurement subject. Thus, the contour shape 74 can vary depending on the capturing direction or capturing angle of the one-foot image 70 and the individual differences in the three-dimensional shape of one foot of the measurement subject.

Referring back to FIG. 3, the determination unit 54 determines the type of the contour shape extracted by the contour extractor 52. The determination processing performed by the determination unit 54 will be detailed later.

Based on the type of the contour shape determined by the determination unit 54, the heel estimation unit 56 estimates the heel point position in the contour shape. The estimation processing performed by the heel estimation unit 56 will be detailed later.

In this way, the heel position estimation server 20 of the present embodiment determines the type of the contour shape 74 using the determination unit 54 and estimates the heel point position based on the type of the contour shape 74 using the heel estimation unit 56. Therefore, the heel position estimation server 20 can reduce the influence of the capturing conditions of the one-foot image 70 or the individual differences in the three-dimensional shape of one foot and can improve the accuracy of estimating the heel point position.

As shown in FIG. 3, the computing unit 42 of the heel position estimation server 20 can further include the toe estimation unit 58, a foot length calculation unit 60, and a foot length correction unit 62.

Based on a one-foot image of a measurement subject read by the image reader 50, the toe estimation unit 58 estimates the toe position. For example, the toe estimation unit 58 can binarize an arbitrary region on the toe side of the one-foot image 70 and detect the contour between the region occupied by the one foot and the region occupied by the white paper 80. In the contour thus detected, a point located on the most toe side can be estimated to be the toe position. In this case, the one-foot image 70 is an image captured from the medial side or lateral side of one foot such that the toe is present on the contour of the captured one foot.

The foot length calculation unit 60 calculates the foot length of the measurement subject based on the toe position estimated by the toe estimation unit 58 and the heel point position estimated by the heel estimation unit 56. In specific, when the foot length directions in the one-foot image 70, i.e., the left and right directions in FIG. 4A, are defined as X directions, using Xt as the X coordinate of the estimated toe position and Xh as the X coordinate of the estimated heel point position, the foot length calculation unit 60 can calculate, as the foot length, the absolute value of the difference between Xt and Xh. At the time, the foot length can also be calculated based on comparison with the reference object shown in the one-foot image 70.

The foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60. For example, the foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of and an angle at a predetermined position of the reference object in the one-foot image, a parameter based on information on the actual size of the reference object and information on the actual angle at the predetermined position, and a coefficient based on the type of the contour shape. The correction processing performed by the foot length correction unit 62 will be detailed later.

FIG. 5 illustrates a flowchart that shows an example of processing performed by the heel position estimation server 20. The image reader 50 reads a one-foot image received by the communication unit 40 from the user terminal 10 (S10).

From the one-foot image of a measurement subject read by the image reader 50, the contour extractor 52 extracts a contour shape of a foot part that includes at least part of a heel portion (S12). The determination unit 54 determines the type of the contour shape extracted by the contour extractor 52 (S14). Based on the type of the contour shape determined by the determination unit 54, the heel estimation unit 56 estimates the heel point position in the contour shape (S16). After the step S10, in parallel with the steps S12-S16, the toe estimation unit 58 estimates the toe position based on the one-foot image of the measurement subject read by the image reader 50 (S18). The foot length calculation unit 60 calculates the foot length of the measurement subject based on the toe position estimated by the toe estimation unit 58 at the step S18 and the heel point position estimated by the heel estimation unit 56 at the step S16 (S20). The foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60 (S22), and the processing is terminated.

FIG. 6 illustrates a flowchart that specifically shows the processing performed by the determination unit 54 at the step S14 and the processing performed by the heel estimation unit 56 at the step S16 shown in FIG. 5, in the example of the operations performed by the heel position estimation server 20. The determination unit 54 determines whether or not the contour shape is a first type having a maximum point when the heel side in a foot length direction of the one-foot image is defined as the positive direction (S30). When the determination unit 54 determines the contour shape to be the first type (Y at S30), the heel estimation unit 56 estimates the maximum point in the contour shape to be the heel point position (S32).

When the determination unit 54 determines the contour shape to be not the first type (N at S30), the determination unit 54 determines whether or not the contour shape is a second type in which the magnitude of an inclination of the contour shape with respect to a foot length direction or the degree of a change in the inclination satisfies a predetermined condition (S34). When the determination unit 54 determines the contour shape to be the second type (Y at S34), a position that satisfies the predetermined condition is estimated to be the heel point position (S36).

When the determination unit 54 determines the contour shape to be not the second type (N at S34), the determination unit 54 determines that the contour shape is a third type (S38). When the determination unit 54 determines the contour shape to be the third type (S38), the heel estimation unit 56 estimates a position in the contour shape located on the most heel side in a foot length direction to be the heel point position (S40).

FIGS. 7A, 7B, 8A and 8B illustrate examples of each type of the contour shape 74 determined by the determination unit 54. Although FIGS. 7A, 7B, 8A and 8B illustrate the contour shape 74 as a set of points, this does not mean that the data format of the contour shape 74 is limited to the raster format. Even when the data of the contour shape 74 is in the vector format, the contour shape 74 can be seen as a set of points calculated at predetermined intervals in a two-dimensional coordinate system based on the vector format data. In FIGS. 7A, 7B, and 8A, the points constituting the contour shape 74 are indicated as contour points 82a-82e. When the contour points 82a-82e are not distinguished from each other, they can be collectively referred to as contour points 82. Also, the contour points 82a-82e in each drawing are provided to distinguish the contour points 82 in the drawing and do not indicate the same contour points 82 in different drawings even though they have the same reference characters.

FIG. 7A illustrates an example of the contour shape 74 determined to be the first type. FIG. 7A schematically illustrates a magnified view of part of the entire contour shape 74 located on the most heel side in a foot length direction of the one-foot image 70. The determination unit 54 selects the contour points 82 constituting the contour shape 74 two by two from the plantar side toward the ankle side. At that time, the determination unit 54 selects the contour points 82 such that the contour point 82 on the ankle side of the two contour points 82 selected earlier coincides with the contour point 82 on the plantar side of the two contour points 82 selected immediately thereafter. The determination unit 54 sequentially calculates an angle θ between the vector from the contour point 82 on the plantar side to the contour point 82 on the ankle side thus selected and a straight line parallel to the direction from the plantar side to the ankle side (Y-axis direction). For example, the determination unit 54 calculates the angle θ between the vector from the contour point 82a to the contour point 82b and a straight line parallel to the Y-axis and then calculates the angle θ between the vector from the contour point 82b to the contour point 82c and a straight line parallel to the Y-axis. The sign of the angle θ is positive when the vector is directed toward the toe side with respect to a straight line parallel to the Y-axis, and the sign is negative when the vector is directed toward the heel side with respect to a straight line parallel to the Y-axis.

When the sign of the angle θ becomes positive for the first time, i.e., when the vector from the contour point 82 on the plantar side to the contour point 82 on the ankle side selected is directed toward the toe side with respect to a straight line parallel to the Y-axis, the determination unit 54 determines the contour shape 74 to be the first type. The heel estimation unit 56 then estimates the contour point 82 on the plantar side of the two contour points 82 selected at the time, to be the position of a heel point 76. In the example shown in FIG. 7A, the vector from the contour point 82c to the contour point 82d is directed toward the toe side with respect to a straight line parallel to the Y-axis, so that the contour point 82c is estimated to be the position of the heel point 76. In other words, the determination unit 54 determines the contour shape 74 to be the first type when the contour shape 74 has a maximum point when the heel side in a foot length direction of the one-foot image 70 is defined as the positive direction. The heel estimation unit 56 then estimates the maximum point to be the heel point position. When there are multiple contour points 82 that satisfy the aforementioned condition, the heel estimation unit 56 can estimate the contour point 82 located on the most plantar side to be the position of the heel point 76.

When the sign of the angle θ becomes positive for the first time and when the angle θ exceeds a predetermined upper limit, the determination unit 54 may not be capable of determining the contour shape to be the first type and can proceed to the selection of the next contour point 82. This process can reduce the cases where the heel point position is estimated incorrectly because of noise or the like in the one-foot image 70, for example. The upper limit of the angle θ can be 45 degrees, for example.

FIG. 7B illustrates an example of the contour shape 74 determined to be the second type. FIG. 7B schematically illustrates a magnified view of part of the entire contour shape 74 located continuously from the position on the most plantar side toward the heel side of the one-foot image 70. After judging the contour shape 74 to be not the first type, the determination unit 54 selects the contour points 82 constituting the contour shape 74 two by two from the plantar side toward the ankle side, as previously described. The determination unit 54 sequentially calculates the angle θ between the vector from the contour point 82 on the plantar side to the contour point 82 on the ankle side thus selected and a straight line parallel to the direction from the plantar side to the ankle side (Y-axis direction).

When the sign of the angle θ remains negative and when the angle θ is a predetermined angle or larger, i.e., the absolute value of the angle θ is less than or equal to the absolute value of the predetermined angle, the determination unit 54 can determine the contour shape 74 to be the second type. The heel estimation unit 56 then estimates the contour point 82 on the plantar side of the two contour points 82 selected at the time, to be the position of the heel point 76. In the example shown in FIG. 7B, the angle θ between the vector from the contour point 82c to the contour point 82d and a straight line parallel to the Y-axis is larger than or equal to the predetermined angle, so that the contour point 82c is estimated to be the position of the heel point 76. The predetermined angle in this case can be −5 degrees, for example. In other words, when the contour shape 74 includes a position at which the magnitude of the inclination with respect to a foot length direction (an X-axis direction) is a predetermined threshold or greater, the determination unit 54 determines the contour shape 74 to be the second type. The predetermined threshold in this case is a value obtained by subtracting the abovementioned predetermined angle from 90 degrees and can be 85 degrees, for example. When there are multiple contour points 82 that satisfy the aforementioned condition, the heel estimation unit 56 can estimate the contour point 82 located on the most plantar side to be the position of the heel point 76.

FIG. 8A illustrates another example of the contour shape 74 determined to be the second type. FIG. 8A schematically illustrates a magnified view of part of the entire contour shape 74 located continuously from the position on the most plantar side toward the heel side of the one-foot image 70. After judging the contour shape 74 to be not the first type, the determination unit 54 selects the contour points 82 constituting the contour shape 74 two by two from the plantar side toward the ankle side, as previously described. The determination unit 54 sequentially calculates an angle θ2, not illustrated, between the vector from the contour point 82 on the plantar side to the contour point 82 on the ankle side thus selected and a straight line parallel to a foot length direction (an X-axis direction).

When the angle θ2 for the selected two contour points 82 falls within a first inclination range α and when the angle θ2 for the two contour points 82 subsequently selected falls within a second inclination range β, the determination unit 54 determines the contour shape to be the second type. The heel estimation unit 56 then estimates the contour point 82 on the plantar side of the two contour points 82 selected later, i.e., the contour point 82 on the ankle side of the two contour points 82 selected earlier, to be the position of the heel point 76. In the example shown in FIG. 8A, the angle θ2 between the vector from the contour point 82b to the contour point 82c and a straight line parallel to the X-axis falls within the first inclination range α, and the angle θ2 between the vector from the contour point 82c to the contour point 82d and a straight line parallel to the X-axis falls within the second inclination range β, so that the contour point 82c is estimated to be the position of the heel point 76. The second inclination range β is greater than the first inclination range α by a predetermined amount of inclination. For example, the first inclination range α can be greater than 45 degrees and less than or equal to 85 degrees, and the second inclination range β can be greater than 85 degrees and less than or equal to 90 degrees. When there are multiple contour points 82 that satisfy the aforementioned condition, the heel estimation unit 56 can estimate the contour point 82 located on the most plantar side to be the position of the heel point 76.

The determination unit 54 can perform the determination for the second type described with reference to FIG. 7B and the determination for the second type described with reference to FIG. 8A, selectively or in stages. That is, the determination unit 54 can perform only one of the determination for the second type described with reference to FIG. 7B and the determination for the second type described with reference to FIG. 8A or can perform, when the contour shape is not determined to be the second type in one of the determinations, the other determination.

FIG. 8B illustrates an example of the contour shape 74 determined to be the third type. When the determination unit 54 determines the contour shape 74 to be not the first type or the second type, the determination unit 54 determines the contour shape 74 to be the third type. In this case, as shown in FIG. 8B, the heel estimation unit 56 estimates the contour point 82 at a position in the contour shape 74 located on the most heel side in a foot length direction (an X direction) to be the position of the heel point 76.

FIG. 9 illustrates a schematic diagram that shows foot length correction performed by the foot length correction unit 62. The foot length calculated by the foot length calculation unit 60 is likely to include an error due to the capturing angle of the one-foot image 70 or the like. In particular, in the present embodiment, the heel point position estimated based on the type of the contour shape 74 that has been determined is different in a direction perpendicular to a foot length direction and is therefore easily affected by the capturing angle of the one-foot image 70. Therefore, the foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60, based on a multiple regression model using a length along a foot length direction of the white paper 80, which is the reference object in the one-foot image 70, an angle of a corner of the white paper 80, a parameter based on information on the actual length along a foot length direction of the white paper 80 and information on the actual angle of the corner of the white paper 80, and a coefficient based on the type of the contour shape 74.

The foot length (length_raw) of the one foot in the one-foot image 70 is calculated based on the coordinates of the toe 78 and the coordinates of the heel point 76. The foot length (length_raw) is standardized based on the length along a foot length direction of the white paper 80 in the one-foot image 70 and information on the actual length along a foot length direction of the white paper 80, so that a standardized foot length (Xstd_length_raw) is calculated. Also, the angle (deg_corner) of a corner of the white paper 80 in the one-foot image 70 is calculated by contour extraction or the like. The corner of the white paper 80 can be an arbitrary corner of the four corners of the white paper 80; in the example shown in FIG. 9, the lower right corner is selected. The angle (deg_corner) of a corner of the white paper 80 is standardized using the actual angle of the corner of the white paper 80, such as the angle in plan view of the white paper 80, i.e., 90 degrees, so that a standardized angle (Xstd_deg_corner) is calculated.

A multiple regression equation in which the standardized foot length (Xstd_length_raw) and the standardized angle (Xstd_deg_corner) are set as explanatory variables and a foot length after correction (Ylength) is set as the objective variable is expressed by the following equation (1).

$\begin{array}{cc}\mathrm{Ylength}=a*\mathrm{Xstd\_length}\mathrm{\_raw}+b*\mathrm{Xstd\_deg}\mathrm{\_corner}+c& \left(1\right)\end{array}$

where each of a, b, and c is a coefficient and can be made different for each type of the contour shape 74

As described above, according to the present embodiment, the contour extractor 52 extracts the contour shape 74 of a foot part that includes at least part of a heel portion, from the one-foot image 70 of a measurement subject. The determination unit 54 determines the type of the contour shape 74. Based on the type of the contour shape thus determined, the heel estimation unit 56 estimates the heel point position in the contour shape. Therefore, the heel position estimation server 20 can appropriately estimate the heel point position, with reduced influence of the capturing conditions of the one-foot image 70 or the individual differences in the three-dimensional shape of one foot.

Also, according to the present embodiment, the determination unit 54 determines whether or not the contour shape 74 falls into the first type having a maximum point when the heel side in a foot length direction of the one-foot image 70 is defined as the positive direction. When the contour shape 74 is determined to be the first type, the heel estimation unit 56 estimates the maximum point to be the position of the heel point 76. Accordingly, in the case of a standard heel shape such that the position of the heel point 76 corresponds to a maximum point on the heel side, the heel position estimation server 20 can appropriately estimate the position of the heel point 76.

Also, according to the present embodiment, when the determination unit 54 has determined the contour shape to be not the first type, the determination unit 54 determines whether or not the contour shape 74 falls into the second type in which the magnitude of an inclination of the contour shape 74 with respect to a foot length direction or the degree of a change in the inclination satisfies a predetermined condition. When the contour shape 74 is determined to be the second type, the heel estimation unit 56 estimates a position in the contour shape 74 that satisfies the predetermined condition to be the position of the heel point 76. Accordingly, even when the contour shape 74 has no maximum point on the heel side, the heel position estimation server 20 can appropriately estimate the position of the heel point 76. Also, according to the present embodiment, when the inclination of the contour shape 74 with respect to a foot length direction changes, along the contour shape 74 from the plantar side toward the heel side in a foot length direction, from the first inclination range α to the second inclination range β, which is greater than the first inclination range α by a predetermined amount of inclination, the determination unit 54 determines the contour shape 74 to be the second type. When the contour shape 74 is determined to be the second type, the heel estimation unit 56 estimates a position in the contour shape 74 at which the inclination changes from the first inclination range α to the second inclination range β, to be the position of the heel point 76. Accordingly, based on the change in inclination of the contour shape 74, the heel position estimation server 20 can appropriately estimate the position of the heel point 76.

Also, according to the present embodiment, when the contour shape 74 includes a position at which the magnitude of the inclination with respect to a foot length direction is a predetermined threshold or greater, the determination unit 54 determines the contour shape 74 to be the second type. When the contour shape 74 is determined to be the second type, the heel estimation unit 56 estimates a position in the contour shape 74 at which the magnitude of the inclination is greater than or equal to the threshold and that is located on the most plantar side, to be the position of the heel point 76. Accordingly, based on the magnitude of the inclination of the contour shape 74, the heel position estimation server 20 can appropriately estimate the position of the heel point 76.

Also, according to the present embodiment, when the determination unit 54 has determined the contour shape 74 to be not the first type or the second type, the determination unit 54 determines the contour shape 74 to be the third type. When the contour shape 74 is determined to be the third type, the heel estimation unit 56 estimates a position in the contour shape 74 located on the most heel side in a foot length direction to be the position of the heel point 76. Accordingly, even when the contour shape 74 cannot be classified in the first type or the second type, the heel position estimation server 20 can appropriately estimate the position of the heel point 76 within the range where the contour shape 74 has been extracted.

The embodiment described above can be a program for causing a computer to perform functions to realize the method described above or can be a storage medium storing the program. The storage medium storing such a program can be a non-transitory and tangible computer readable storage medium and can be a non-volatile memory, a magnetic storage medium such as a magnetic tape or a magnetic disk, or an optical storage medium such as an optical disk.

The present disclosure has been described based on an embodiment. The embodiment is intended to be illustrative only, and it will be obvious to those skilled in the art that various modifications to a combination of constituting elements or processes could be developed and that such modifications also fall within the scope of the present disclosure.

Also, when the embodiment set forth above is generalized, the following aspects are obtained.

(1) A heel position estimation device according to one aspect of the present disclosure includes:

• • a contour extractor that extracts, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion;
  • a determination unit that determines the type of the contour shape; and
  • a heel estimation unit that estimates a heel point position in the contour shape, based on the type of the contour shape determined.

(2) In the heel position estimation device described in (1),

• • the determination unit can determine whether or not the contour shape is a first type having a maximum point when a heel side in a foot length direction of the one-foot image is defined as the positive direction, and
  • when the contour shape is determined to be the first type, the heel estimation unit can estimate the maximum point to be the heel point position.

(3) In the heel position estimation device described in (2),

• • when the determination unit has determined the contour shape to be not the first type, the determination unit can determine whether or not the contour shape is a second type in which the magnitude of an inclination of the contour shape with respect to the foot length direction or the degree of a change in the inclination satisfies a predetermined condition, and
  • when the contour shape is determined to be the second type, the heel estimation unit can estimate a position in the contour shape that satisfies the predetermined condition to be the heel point position.

(4) In the heel position estimation device described in (3),

• • when the inclination of the contour shape with respect to the foot length direction changes, along the contour shape from a plantar side toward the heel side in the foot length direction, from a first inclination range to a second inclination range, which is greater than the first inclination range by a predetermined amount of inclination, the determination unit can determine the contour shape to be the second type, and
  • when the contour shape is determined to be the second type, the heel estimation unit can estimate a position in the contour shape at which the inclination changes from the first inclination range to the second inclination range, to be the heel point position.

(5) In the heel position estimation device described in (3),

• • when the contour shape includes a position at which the magnitude of the inclination with respect to the foot length direction is a predetermined threshold or greater, the determination unit can determine the contour shape to be the second type, and
  • when the contour shape is determined to be the second type, the heel estimation unit can estimate a position in the contour shape at which the magnitude of the inclination is greater than or equal to the threshold and that is located on the most plantar side, to be the heel point position.

(6) In the heel position estimation device described in any one of (3) through (5),

• • when the determination unit has determined the contour shape to be not the first type or the second type, the determination unit can determine the contour shape to be a third type, and
  • when the contour shape is determined to be the third type, the heel estimation unit can estimate a position in the contour shape located on the most heel side in the foot length direction to be the heel point position.

(7) The heel position estimation device described in any one of (1) through (6) can further include:

• • a toe estimation unit that estimates a toe position from the one-foot image; and
  • a foot length calculation unit that calculates a foot length based on the toe position estimated and the heel point position estimated.

(8) The heel position estimation device described in (7) can further include a foot length correction unit that corrects a foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of a reference object in the one-foot image and an angle of a predetermined portion of the reference object, a parameter based on information on the actual size of the reference object and information on the actual angle of the predetermined portion of the reference object, and a coefficient based on the type of the contour shape.

(9) A heel position estimation method according to one aspect of the present disclosure includes:

• • extracting, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion;
  • determining the type of the contour shape; and
  • estimating a heel point position in the contour shape, based on the type of the contour shape determined.

(10) A program according to one aspect of the present disclosure causes a computer to implement:

• • extracting, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion;
  • determining the type of the contour shape; and
  • estimating a heel point position in the contour shape, based on the type of the contour shape determined.

Claims

1. A heel position estimation device, comprising: a contour extractor configured to extract, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion;a determination unit configured to determine a type of the contour shape; anda heel estimation unit configured to estimate a heel point position in the contour shape, based on the type of the contour shape determined.

2. The heel position estimation device according to claim 1, wherein the determination unit is configured to determine whether the contour shape is a first type having a maximum point when a heel side in a foot length direction of the one-foot image is defined as a positive direction, andwhen the contour shape is determined to be the first type, the heel estimation unit is configured to estimate a maximum point to be the heel point position.

3. The heel position estimation device according to claim 2, wherein, when the determination unit has determined the contour shape to be not the first type, the determination unit is configured to determine whether the contour shape is a second type in which a magnitude of an inclination of the contour shape with respect to the foot length direction or a degree of a change in the inclination satisfies a predetermined condition, andwhen the contour shape is determined to be the second type, the heel estimation unit is configured to estimate a position in the contour shape that satisfies the predetermined condition to be the heel point position.

4. The heel position estimation device according to claim 3, wherein, when the inclination of the contour shape with respect to the foot length direction changes, along the contour shape from a plantar side toward the heel side in the foot length direction, from a first inclination range to a second inclination range, which is greater than the first inclination range by a predetermined amount of inclination, the determination unit is configured to determine the contour shape to be the second type, andwhen the contour shape is determined to be the second type, the heel estimation unit is configured to estimate a position in the contour shape at which the inclination changes from the first inclination range to the second inclination range, to be the heel point position.

5. The heel position estimation device according to claim 3, wherein, when the contour shape includes a position at which the magnitude of the inclination with respect to the foot length direction is a predetermined threshold or greater, the determination unit is configured to determine the contour shape to be the second type, andwhen the contour shape is determined to be the second type, the heel estimation unit is configured to estimate a position in the contour shape at which the magnitude of the inclination is greater than or equal to the threshold and that is located on a most plantar side, to be the heel point position.

6. The heel position estimation device according to claim 3, wherein, when the determination unit has determined the contour shape to be not the first type or the second type, the determination unit is configured to determine the contour shape to be a third type, andwhen the contour shape is determined to be the third type, the heel estimation unit is configured to estimate a position in the contour shape located on a most heel side in the foot length direction to be the heel point position.

7. The heel position estimation device according to claim 4, wherein, when the determination unit has determined the contour shape to be not the first type or the second type, the determination unit is configured to determine the contour shape to be a third type, andwhen the contour shape is determined to be the third type, the heel estimation unit is configured to estimate a position in the contour shape located on a most heel side in the foot length direction to be the heel point position.

8. The heel position estimation device according to claim 5, wherein, when the determination unit has determined the contour shape to be not the first type or the second type, the determination unit is configured to determine the contour shape to be a third type, andwhen the contour shape is determined to be the third type, the heel estimation unit is configured to estimate a position in the contour shape located on a most heel side in the foot length direction to be the heel point position.

9. The heel position estimation device according to claim 1, further comprising: a toe estimation unit configured to estimate a toe position from the one-foot image; anda foot length calculation unit configured to calculate a foot length based on the toe position estimated and the heel point position estimated.

10. The heel position estimation device according to claim 2, further comprising: a toe estimation unit configured to estimate a toe position from the one-foot image; anda foot length calculation unit configured to calculate a foot length based on the toe position estimated and the heel point position estimated.

11. The heel position estimation device according to claim 3, further comprising: a toe estimation unit configured to estimate a toe position from the one-foot image; anda foot length calculation unit configured to calculate a foot length based on the toe position estimated and the heel point position estimated.

12. The heel position estimation device according to claim 4, further comprising: a toe estimation unit configured to estimate a toe position from the one-foot image; anda foot length calculation unit configured to calculate a foot length based on the toe position estimated and the heel point position estimated.

13. The heel position estimation device according to claim 5, further comprising: a toe estimation unit to estimate a toe position from the one-foot image; anda foot length calculation unit configured to calculate a foot length based on the toe position estimated and the heel point position estimated.

14. The heel position estimation device according to claim 9, further comprising a foot length correction unit configured to correct the foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of a reference object in the one-foot image and an angle of a predetermined portion of the reference object, a parameter based on information on an actual size of the reference object and information on an actual angle of the predetermined portion of the reference object, and a coefficient based on the type of the contour shape.

15. The heel position estimation device according to claim 10, further comprising a foot length correction unit configured to correct the foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of a reference object in the one-foot image and an angle of a predetermined portion of the reference object, a parameter based on information on an actual size of the reference object and information on an actual angle of the predetermined portion of the reference object, and a coefficient based on the type of the contour shape.

16. The heel position estimation device according to claim 11, further comprising a foot length correction unit configured to correct the foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of a reference object in the one-foot image and an angle of a predetermined portion of the reference object, a parameter based on information on an actual size of the reference object and information on an actual angle of the predetermined portion of the reference object, and a coefficient based on the type of the contour shape.

17. The heel position estimation device according to claim 12, further comprising a foot length correction unit configured to correct the foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of a reference object in the one-foot image and an angle of a predetermined portion of the reference object, a parameter based on information on an actual size of the reference object and information on an actual angle of the predetermined portion of the reference object, and a coefficient based on the type of the contour shape.

18. The heel position estimation device according to claim 13, further comprising a foot length correction unit configured to correct the foot length calculated by the foot length calculation unit, based on a multiple regression model using a size of a reference object in the one-foot image and an angle of a predetermined portion of the reference object, a parameter based on information on an actual size of the reference object and information on an actual angle of the predetermined portion of the reference object, and a coefficient based on the type of the contour shape.

19. A heel position estimation method, comprising: extracting, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion;determining the type of the contour shape; andestimating a heel point position in the contour shape, based on the type of the contour shape determined.

20. A non-transitory computer-readable storage medium storing a program causing a computer to implement: extracting, from a one-foot image of a measurement subject, a contour shape of a foot part that includes at least part of a heel portion;determining the type of the contour shape; andestimating a heel point position in the contour shape, based on the type of the contour shape determined.