Patent Application
20170135608
The invention relates to a method for determining the alignment or also for adjusting a system comprising measuring and display device, in which, using a measuring device, which has a measuring platform, horizontal and vertical forces as well as free moments, if applicable, are detected, which are exerted by a person located on the measuring platform, in which the person located on the measuring platform is photographed using at least one camera device, image data from the at least one camera device and measurement data from the measuring device are transmitted to the display device, and a force application line in the display device is superimposed on the image data and is displayed on the image of the person located on the measuring platform. The invention also relates to a display system for representing ground reaction forces, which are detected by a measuring platform on which a person stands, wherein the measuring platform has two measuring plates, which are decoupled from one another and which have sensors for detecting horizontal and/or vertical forces and/or free moments, comprising at least one camera device, which at least partially detects the person located on the measuring platform, and a display device, on which a force application line of the ground reaction forces are displayed superimposed on the person.
Display systems for representing ground reaction forces of a human being in response to the adjustment of a prosthetic or orthotic alignment is known from DE 10 2006 021 788 A1, for example. The vertical and horizontal components of a ground reaction force are detected via the sensor device and the force vector of the ground reaction force is projected onto the human body. This projection can be filmed.
U.S. Pat. No. 4,598,717 describes a display system for representing horizontal and vertical forces, which were captured via sensor units. The representation occurs on a monitor, on which the orientation, the location as well as the amount of the ground vector force are displayed. The image of the object to be examined is taken via a plurality of camera devices, the determined force vectors are superimposed on the camera image and are displayed. The display system is intended to display postural deformities as well as loads on different joint systems during a movement.
The representation of force application lines on the body itself is difficult, the filming of the person is cumbersome, and the storing and processing as well as the analysis of the data, in particular the quantification of the measurement data, is difficult or impossible to represent. A level of inaccuracy is furthermore present in the representation and an exact alignment and fixed geometric allocation of the projection device to the person is necessary.
It is the object of the invention at hand to provide a method and a display system, by means of which in particular the setup of orthopedic devices, such as orthoses or prostheses can be measured, displayed and oriented, so that the forces introduced into the measuring platform can be represented correctly on an image as force application lines.
According to the invention, the object is solved by means of a method comprising the features of the main claim as well as a device comprising the features of the independent claim. Advantageous embodiments of the invention are disclosed in the subclaims, the description as well as the figures.
The method according to the invention for determining the alignment of a system comprising measuring and display device in which, using the measuring device, which has a measuring platform, horizontal and vertical forces and/or free moments are detected, which are exerted by a person located on the measuring platform, provides for the person located on the measuring platform to be at least partially photographed using at least one camera device, image data from the at least one camera device and measurement data from the measuring device to be transmitted to the display device, and the force application line in the display device to be superimposed on the image data and to be displayed on the image of the person located on the measuring platform, wherein marker points with a fixed geometric relation to the measuring platform are detected on the measuring platform by the at least one camera device, wherein the geometric relation of the marker points to one another, in particular the distance thereof to one another, is known, and the position and/or orientation of the measuring platform to the at least one camera device is determined via an image analysis of the detected marker points. It is thus possible to adjust the measuring and display system in such a manner that the force application line can be represented in a positionally accurate manner on the image of the person of the display device, whereby an improved adjustment, e.g. of a prosthetic alignment, can take place. By determining the alignment and adjustment, if applicable, it is no longer necessary to carry out a rigid geometric allocation of camera device and measuring platform; it is possible, in fact, to provide for a flexible setup and a simple transport of the system via a camera, which can be set up freely. Due to the fact that a projector does not need to be allocated to the measuring platform any longer, the system requires significantly lower production costs. The image data as well as the measurement data of the measuring platform can be supplied to an integrated documentation, which can be analyzed and used simultaneously during the measurement as well as subsequently, for example in connection with a follow-up meeting.
A mobile embodiment of the display device and an overall wireless transfer of all data makes it possible to involve the patient in the measurement, so that the patient can recognize immediately, which the impacts of a change to a setup or an alignment of orthoses or prostheses components to one another are on the force path, based on the representation of the force application line. A subsequent analysis can be carried out by means of the storage possibility of the image and measurement data, which are present in digital form, so that repeat measurements can be avoided. Perception errors on the patient are furthermore avoided and an independence of the representation from the prevailing light conditions is reached. By detecting the marker points, which are arranged on the measuring plate or which are at least in a fixed and known geometric allocation to the measuring plate, by means of the camera device, it is possible to determine the orientation, in particular the distance of the measuring plate to the camera device via the image analysis in the case of a certain and known distance of the marker points to one another, so that it is possible to calculate the exact distance as well as the exact orientation of the camera device to the force measuring plate and thus also the exact allocation of the measured force application lines to the respective position of the force measuring plate and thus the correct position of the force application line in the image. It is thus possible to display the respective force application lines at the location of the force introduction into the force measuring plate, independent from light conditions, alignment directions or the like, so that a positionally accurate superimposition of the representation of the force application line with the image or partial image of the person located on the measuring platform is ensured. Positionally accurate means that the force application line on the image is positioned at that location, at which the ground reaction force acts on the person.
Distortion characteristics of the camera device are considered to determine the orientation and/or position of the camera device relative to the measuring platform. Each camera device has certain optical or electronic distortion characteristics, the so-called intrinsic camera parameters, which are determined by means of a camera calibration, and which are invariant in terms of time. If these intrinsic camera parameters are known, they can also be considered in the calculation of the position of the respective force application lines relative to the image, whereby the accuracy of the representation of the force application line, the adjustment of the display and the determination of the alignment of the components of the system to one another is further increased.
The marker points can be embodied as glyphs, illuminants and/or reflector devices, and are either activated or illuminated to determine the alignment or adjustment. The activation or illumination can take place in the visible as well as in the invisible light spectrum.
Periodic light signals can be transmitted by the illuminant or can be transmitted to the reflector devices, so as to verify, whether the allocation of the camera device to the measuring platform is still at hand and has not changed. Certain light patterns can be requested in order to carry out a calibration and readjustment of the system. Advantageously, the measuring platform is oriented as horizontally as possible, which can be verified easily by means of a built-in level. In the case of an uneven ground, the horizontal orientated can be ensured via adjustable feet. The camera device is aligned as orthogonally as possible to the measuring platform, whereby, in the case of a plurality of camera devices, one is oriented frontally and one is oriented sagitally. On principle, it is also possible to detect via an image recognition, whether a frontal alignment or sagittal alignment is present. A plurality of camera devices can also be arranged on top of one another so as to be oriented in the same viewing direction, for example so as to be mounted to a support, whereby the accuracy of the adjustment is increased.
Markers are advantageously attached to the measuring platform so as to standardize the alignment of the patient to the measuring platform and thus also to the camera device. Advantageously, the longitudinal extension of the feet is thereby either vertically or parallel to the viewing direction of the camera device.
The determination of the alignment and the adjustment, if applicable, can take place periodically and/or after shifts or accelerations of the measuring platform or of the camera device have been determined. To reduce the calculating effort, it can be verified prior to a periodic determination of the alignment and/or adjustment, whether the measuring device was moved, for example, which can be determined by means of acceleration sensors in the measuring platform.
To further optimize the alignment of the system and the adjustment, if applicable, the measuring platform can be detected via an object recognition in a first image section and the image section of the camera device can be aligned automatically on a marking of the measuring platform or on a side edge of the measuring platform, so that the lower edge of the captured image section is flush with the front edge of the measuring platform facing the camera device, which is preferably embodied so as to be rectangular.
The image section can also be set automatically on the basis of an object recognition of the persons located on the measuring platform. On principle, the overall representation of the patient is of no interest, but only that of the lower extremities, so that a corresponding sectional setting can take place automatically, so as to concentrate the representation to what is significant.
A lighting device, from which an alignment beam is projected onto the measuring platform, can be allocated to the camera device, whereby the measuring platform can be aligned more easily to the camera device. The alignment can either be made manually by means of markings located on the measuring platform, in the alternative at least one light detector is arranged on the measuring platform to detect the alignment beam and to trigger a confirmation signal, when the camera device is aligned correctly and as optimally as possible to the measuring platform. Two or more light detectors can also be arranged downstream from one another, which are positioned in such a manner that a confirmation signal is output in response to a detection of the aligned beam by all light detectors. The light detectors can also serve as marking. When taking a photograph in the frontal plane, it is advantageous when the light detectors or sensors are arranged downstream from one another in the direction of the beam in the center of the measuring platform, thus along a center line or between two measuring plates, so that it is ensured that the measuring platform is oriented substantially parallel or vertically to the alignment light beam, and is thus also substantially aligned vertically or parallel to the viewing direction of the camera device.
The measurement data and the image data are preferably transmitted wirelessly to the display device. For this purpose, the components advantageously have transmitting devices and receiving devices, if applicable, so that the data can be transmitted and received via networks or the like. Typically, the display device is a portable display device, in particular a tablet, computer or smart phone, by means of which a highly accurate display on the one hand and a storing of the measurement data and of the image data on the other hand can be carried out. The tablet and the like can be used by the technician during the measuring or by the patient as mobile display unit. It is furthermore possible to show different setting aids in the live image, for example the distance of the force application lines to reference lines, or markers, which are attached to the prosthesis or the person.
The measurement data and the image data are preferably detected synchronously and with time identifier and are synchronized or output or stored with the same identifier, so that it is also possible on principle that the camera image is displayed superimposed directly with force application lines. Having the same identification means that the image data and measurement data with the same time identifier are superimposed on one another, so as to obtain a realistic reproduction of the force application line on the image or when playing the picture during measurement. On principle, it is also possible that a static picture and a subsequent analysis and superimposition are carried out. The image data and the measured values are stored with a corresponding time signal and are analyzed subsequently, this may be beneficial in the case of limited online calculating power. The display device can be placed on a tripod or a support either for the camera or a separate support. The tablet or another display device can further be connected to a projector or can be connected to another display, so that the content of the display device can also be made accessible to third parties.
If a plurality of camera devices are directed to the measuring platform from different viewing angles, a determination of the alignment and adjustment of the system and of the display of the force application line is made from the respective viewing angle, so that the representation can take place simultaneously on one or a plurality of display devices or representations of different views. If the image data and the adjustment or calibration data, respectively, of a plurality of camera devices are transmitted to the display device, a switch-over of the respective view can be made there, so that the technician or patient gains multidimensional insight into the force path.
To improve the adjustment, it is possible that glyphs, which are detected and are used for the image analysis, are arranged around the marker points. The marker points themselves can also be embodied as glyphs.
Optical markers or glyphs can be attached to the person and/or measuring platform, wherein the force application line and/or reference lines are aligned along the optical markers or glyphs or will be aligned. The markers as well as the glyphs can be active or passive, they can thus emit light either autonomously, even in the non-visible spectrum, or can reflect light beams emitted by a lighting element. On principle, it is also possible that an illuminant is attached to the tablet or another display device, so as to illuminate the reflectors with light accordingly.
In order to save calculating power for object recognition, an image can initially be captured with a large image section, wherein the measuring platform is detected via the object recognition and an image recognition is subsequently carried out in the areas of the marker points with a higher or larger resolution. Due to the more accurate examination of the marker points, which are in particular arranged on the corners of the measuring platform, the accuracy of the position detection of the markers and thus also of the orientation and positioning of the camera device relative to the measuring platform can take place in an improved manner.
Typically, the setup and the orientation of prosthetic components to one another are carried out at a uniform load of both lower extremities. Advantageously, the measuring platform is divided into two parts and is decoupled from one another at least in terms of force, so that the left leg and the right leg is positioned alternately on the respective force measuring plate. If a significant deviation of the weight distribution from the right plate to the left plate is determined, a warning signal can be output in order to avoid that an incorrect setup is set in response to an uneven loading.
All of the determined data of the measuring platform can be displayed on the tablet or the display device, thus the force application line, the force, the distribution of the forces, free moments as well as the change of the forces.
In addition to the displaying of images and measurement data with the same time identifier or of image and measurement data captured at the same time, a corresponding storing of all of the available perspectives and measurement data is provided.
Different output formats can likewise be provided, for example as image data, as measurement data, or as a PDF data.
The display system for representing ground reaction forces, which are detected by a measuring platform, on which a person stands, wherein the measuring platform has two measuring plates decoupled from one another, which have sensors for determining horizontal and/or vertical forces and/or free moments, using at least one camera device, which at least partially detects the person located on the measuring platform, and a representation device, on which the force application line of the ground reaction forces and a display of the person are displayed in a superimposed manner, provides for the measuring platform to have marker points with a fixed geometric relation to one another and a processing unit, in which the geometric relations, in particular distances from one other, are stored advantageously, wherein the processing unit is connected to the measuring platform, the camera device and the display device, or is integrated into one of the devices. As mentioned above, an automatic calculation of the allocation can take place via trigonometric calculations by including extrinsic parameters and intrinsic parameters of the camera via the marker points. When the camera is attached, for example, to a tripod at a certain height, the distance and the orientation of the camera device can be detected relative to the measuring platform via the angular orientation and the extrinsic, known data of the measuring platform and of the marker positions to one another.
The marker points can be embodied as illuminant, glyphs and/or reflectors, in particular the marker points can be embodied as LED, which can be switched on and off, in the visible spectral range or in the spectral range, which is not visible for the human being.
In addition to the horizontal and vertical force sensors, which are arranged in a sufficient number and orientation in or on the respective measuring plate so as to detect the necessary horizontal and vertical force components and free moments, the measuring platform can have an acceleration sensor, which detects impacts or movements of the measuring platform. If the measuring platform is moved by entering or by an impact, a readjustment and new calculation of the relative position of the respective force application lines in the image is performed automatically. Such an acceleration sensor can additionally or alternatively be allocated to the camera device.
A plurality of camera devices in different viewing angles can be directed to the measuring platform, so as to enable a simple switch-over and a more accurate representation of the force application lines on the captured image.
The camera device can be arranged on a support, from which a projector sends an alignment beam in the direction of the measuring platform. If at least one light detector is arranged on the measuring platform to detect the alignment beam, a confirmation signal can be output, if the orientation light beam hits the light detector or the light detectors. If no light detectors are provided, a desired orientation can be achieved easily via the alignment beam by aligning markers on the measuring platform.
An alignment device can be arranged, in particular so as to be extendable or compressible, in the measuring platform or on the measuring platform, for example in the form of a plastic strip, which has markings, onto which the support of the camera device is to be placed. This facilitates the desired alignment without requiring a rigid geometric allocation of camera device and measuring platform. In particular, the vertical alignment of the front edge of the measuring platform to the viewing direction of the camera and of the alignment beam, if necessary, is ensured in this manner.
An exemplary embodiment of the invention will be explained in more detail below by means of the enclosed figures.
The measuring platform 10 is arranged in viewing direction of the camera device 20 frontally thereto, so that the leading edge 100 of the substantially rectangular measuring platform 10 is oriented vertically to the viewing direction of the camera device 20. To facilitate the alignment from the camera device 20 to the measuring platform 10, an alignment device 60 is provided on or in the measuring platform 10 in the form of an extendable mat, on or in the end of which, which faces the camera device 20, recesses or markings are arranged, which are placed into or on the feet of the support 30, so that an approximately vertical orientation of the leading edge 100 to the camera device 20 is ensured on the one hand and so that the distance between the camera device 20 and the measuring platform 10 is defined in a first approximation on the other hand. The alignment device 60 can also have a scaling, via which either the distance of the camera device 20 to the leading edge 100 of the measuring platform 10 can be read and can be input into a processing unit, so as to achieve an improved adjustment of the image of the camera device. The scaling can also be detected by the camera device 20 by means of an image recognition and can be transmitted automatically to a processing unit, so as to at least prepare the adjustment on the basis of the detected distance. On principle, however, the camera device 20 comprising the support 30 and the lighting device 40 are structurally decoupled from the measuring platform 10, so that freely selectable distances between the camera device 20 and the measuring platform 10 can be provided; the respective image section of the camera device 20 and the recognizability, which may be limited through this, of measuring platform 10 and of the person 1 located on the measuring platform 10 or the resolution and detectability, respectively, of marker points 11, 12, 13, 14 located on the measuring platform 10 serves as limitation.
It is advantageous, when the measuring platform 10 and the support 30 of the camera device 20 are on an equal level and the measuring platform 10 is aligned substantially horizontally. The height of the camera device 20 above the ground can be varied by means of the adjustable support 30. On the one hand, this serves to compensate for height differences of the ground and, on the other hand, to set the image section, depending on the size of the person 1 and on the orthopedic device 2 to be examined, such as orthosis or prosthesis.
On the upper side, four marker points 11, 12, 13, 14, which can either be embodied as reflectors or as self-luminous devices, such as LED or the like, are arranged on the measuring platform 10, which has two measuring plates 15, 16, which are decoupled from one another with regard to force and moment. The positions of the marker points 11, 12, 13, 14 to one another, thus also the distances of the marker points 11, 12, 13, 14 to one another, are known and are time-invariable, because the marker points 11, 12, 13, 14 are firmly mounted to the measuring platform 10. To be able to correctly display or show the contact points of the ground reaction forces and the size as well as orientation of the ground reaction forces on an image in the display device 50, the position on the image file must be set or adjusted correctly. On principle, the force application line 70 is an image superimposed on the image of the person 1 or the prosthesis 2, as it is illustrated in
Due to the fact that a rigid geometric allocation of the camera device 20 to the measuring platform 10 is not present, a regular inspection of the position of the marker points 11, 12, 13, 14 is made, so as to determine a possible shift of the measuring platform 10 to the camera device 20. In addition to a purely optical analysis, it is possible that at least one acceleration sensor is arranged inside the measuring platform 10, via which a movement of the measuring platform 10 can be detected and a recalibration or readjustment can be triggered, if necessary. In the alternative or in addition, an acceleration sensor can also be allocated to the camera device 20.
The weight distribution on both force measuring plates 15, 16 is illustrated laterally next to the image section, the right as well as the left foot is in each case loaded equally with 40 kg in the illustrated exemplary embodiment. If an excessive difference is present between the two weight displays, a warning signal, which prompts to correct the posture of the person, so that a technician can set the non-illustrated prosthesis 50 correctly, can be output via the display device 50 or via a separate warning device.
Distances to reference points or reference lines 75 are also represented in
The measurement data of the measuring platform 10 as well as the images can be stored in a synchronized manner in the display device 50, and all existing perspectives and measurement data can be stored in a synchronized manner in the case of a plurality of camera devices 20, so that a subsequent analysis of the measuring results and of the image data can be carried out easily. All of the data transport paths are preferably wireless, so as to allow unrestricted mobility. The reference lines 75 or reference points can be shown and changed in the image. In addition to the representation of the measuring results and of the captured image data with the same time identifier, they can also be output and printed out, if applicable, as still images. The respective perspective of the camera device 20 can be carried out automatically by an object recognition of the captured image data.
In addition to the use of a plurality of camera devices from different viewing angles, for example frontally and sagitally, it is also possible to arrange two camera devices on top of one another, resulting in an improved accuracy in the image analysis and the allocation of the ground reaction forces to the image data.
Advantageously, the measuring platform 20 is oriented as vertically as possible to the viewing direction of the respective camera device 20, but deviations are possible. To facilitate the allocation, the feet of the person 1 are placed inside markings on the measuring platform or are aligned to them.
The force application line 70, here only the center of gravity line, is superimposed on the image of the person 1, the respective coordinates are allocated to the force application point of the force path 70 via the image recognition.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2014 006 690.8 | May 2014 | DE | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/DE2015/100186 | 5/8/2015 | WO | 00 |
