Patent Application
20180352211
The field of the invention relates to automated garment digitization apparatus, and to related methods and computer program products.
Metail is the leading provider of body shape and garment visualisation technology for online retailers. Our system, in use by some major retailers in the UK, Brazil and India, allows online shoppers to generate images of their own body and dress the model to provide a visualisation of a garment/outfit—giving a virtual fitting experience and addressing issues that hinder more online clothes shopping.
The current system, whilst valuable to shoppers, and reducing returns and increasing conversion for retailers has limitations in two areas: (1) there is a need for increased garment throughput in the photography stage to support the largest retailers, with lower capital costs per garment and simpler operations, and (2) a high level of manual activities are needed to produce a digital garment before visualisation.
To address these issues, Metail has previously productionised 3 technologies: (1) automatic separation of garment images from the photo background (see, e.g. U.S. Pat. No. 8,605,148B2), (2) automated creation of 3D models of a garment from the captured images, and (3) real-time 3D visualisation of the garment. These advances have reduced manual processes and improved realism, but require additional photography and thus reduce garment throughput.
This patent specification describes not only various ideas and functions, but also their creative expression. A portion of the disclosure of this patent document therefore contains material to which a claim for copyright is made and notice is hereby given:
© Metail Limited (e.g. pursuant to 17 U.S.C. 401). A claim to copyright protection is made to all protectable expression associated with the examples of the invention illustrated and described in this patent specification.
The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but reserves all other copyright rights whatsoever. No express or implied license under any copyright whatsoever is therefore granted.
U.S. Pat. No. 8,605,148B2 discloses a system and method for image processing and generating a body model. U.S. Pat. No. 8,605,148B2 is incorporated by reference.
WO2012110828 (A1) discloses that a virtual body model of a person is created with a small number of measurements and a single photograph and combined with one or more images of garments. The virtual body model represents a realistic representation of the user's body and is used for visualizing photo-realistic fit visualizations of garments, hairstyles, make-up, and/or other accessories. The virtual garments are created from layers based on photographs of real garments from multiple angles. Furthermore the virtual body model is used in multiple embodiments of manual and automatic garment, make-up, and, hairstyle recommendations, such as, from channels, friends, and fashion entities. The virtual body model is sharable for, as example, visualization and comments on looks. Furthermore it is also used for enabling users to buy garments that fit other users, suitable for gifts or similar. The implementation can also be used in peer-to-peer online sales where garments can be bought with the knowledge that the seller has a similar body shape and size as the user as well as being used in manufacturing the garment chosen. WO2012110828 (A1) is incorporated by reference. WO2012110828 (A1) discloses methods of digitising a garment in 3D.
There is a need for increased garment throughput in the imaging stage, to support the larger retailers.
According to a first aspect of the invention, there is provided a garment digitization apparatus, including a mannequin loading system, a mannequin rotation system, a computer system and a camera system, wherein the apparatus is configured to load a mannequin wearing a garment using the mannequin loading system, to rotate the mannequin wearing the garment through at least 360° using the mannequin rotation system, and to capture images of the garment using the camera system during the mannequin rotation, wherein the mannequin loading, the mannequin rotation and the image capturing occur under control of the computer system.
An advantage is that image capture is standardized by the apparatus, which improves the reliability of the captured images. A further advantage is that garment arrangement on the mannequin is not disturbed during the mannequin loading, because the automated mannequin loading provides negligible garment disturbance. A further advantage is that mannequin lifetime is extended because the improved handling of the mannequin leads to reduced manual handling damage to the mannequin.
The garment digitization apparatus may be one wherein the captured images of the garment are stored on the computer system.
The garment digitization apparatus may be one wherein the captured images are suitable for 3D garment digitization of the garment.
The garment digitization apparatus may be one wherein the apparatus is arranged to support a loaded mannequin from above. An advantage is that lower parts of garments may be imaged more readily, because there is no need to support the mannequin from below.
The garment digitization apparatus may be one wherein the loaded mannequin includes a neck spigot, and is supported from the neck spigot by the apparatus. An advantage is secure supporting of the mannequin.
The garment digitization apparatus may be one wherein the apparatus includes a motion controller which controls rotation of the mannequin, wherein a signal from the motion controller is used to synchronise mannequin rotation and image capture. An advantage is a precise relation between mannequin rotation and captured images is established.
The garment digitization apparatus may be one wherein the loading system includes a gantry system to engage and to load the mannequin.
The garment digitization apparatus may be one wherein the gantry system includes 3 linear axes and 1 rotational axis. An advantage is very flexible control over mannequin loading and mannequin movement after loading.
The garment digitization apparatus may be one wherein the loading system includes a gantry system to engage and to load the mannequin, and wherein the gantry system includes 3 linear axes and 1 rotational axis and wherein the 3 linear axes and 1 rotational axis are controlled by the motion controller.
The garment digitization apparatus may be one wherein the gantry system includes a collet fixture which is movable downwards to receive, to engage and to secure mannequin tooling on an upper part of the mannequin to the collet fixture.
The garment digitization apparatus may be one wherein the loading system includes linear actuators. An advantage is reduced disturbance to the mannequin.
The garment digitization apparatus may be one wherein the loading system includes electromagnets.
The garment digitization apparatus may be one wherein the apparatus is arranged to support a loaded mannequin from below.
The garment digitization apparatus may be one wherein the loaded mannequin is supported using two spigots of the loading system onto which each of the legs is slotted.
The garment digitization apparatus may be one wherein the camera system includes an industrial camera. An advantage is suitability for large scale garment digitization.
The garment digitization apparatus may be one wherein the camera system has a calibrated position.
The garment digitization apparatus may be one wherein the camera system includes a shutterless camera. An advantage is prolonged camera life in the apparatus.
The garment digitization apparatus may be one wherein the camera system includes a GigE Vision camera.
The garment digitization apparatus may be one wherein the camera system includes a monoscopic camera.
The garment digitization apparatus may be one wherein the camera system includes a stereo camera.
The garment digitization apparatus may be one wherein the apparatus includes a lighting system.
The garment digitization apparatus may be one wherein the lighting system is under control of the computer system.
The garment digitization apparatus may be one wherein the lighting system comprises high-color rendering index (CRI) lighting.
The garment digitization apparatus may be one wherein the lighting system includes a backpanel and a ringlight.
The garment digitization apparatus may be one wherein the lighting system includes light emitting diode (LED) panels.
The garment digitization apparatus may be one wherein the lighting system includes texture lighting, alpha lighting and albedo lighting.
The garment digitization apparatus may be one wherein the apparatus includes a rig, wherein the rig supports or includes the camera system, the mannequin loading system, the mannequin rotation system, and the lighting system.
The garment digitization apparatus may be one wherein the rig includes columns and a roof, wherein the columns support the roof.
The garment digitization apparatus may be one wherein the rig roof is a rigid roof.
The garment digitization apparatus may be one wherein the rig roof is opaque.
The garment digitization apparatus may be one wherein the rig includes a back-panel, a lighting column, and a camera column which provide stability while housing their respective components.
The garment digitization apparatus may be one wherein the rig includes a frame which includes an extruded aluminium profile system.
The garment digitization apparatus may be one wherein the frame has adjustable feet.
The garment digitization apparatus may be one wherein the rig is readily movable within a room without disassembly of the rig.
The garment digitization apparatus may be one wherein the rig ensures that the location and orientation of all apparatus components is constrained so that consistency between garment images is maintained.
The garment digitization apparatus may be one wherein the apparatus is configured to receive a trolley and to receive the mannequin from the trolley.
The garment digitization apparatus may be one wherein the apparatus ensures the trolley does not tip over when the mannequin is being loaded to the apparatus.
The garment digitization apparatus may be one wherein the computer system is mounted in the rig.
The garment digitization apparatus may be one wherein the computer system is mounted in a roof of the rig.
The garment digitization apparatus may be one wherein the computer system includes a server.
The garment digitization apparatus may be one wherein the computer system includes a graphics processing unit for processing the captured images.
The garment digitization apparatus may be one wherein the apparatus is configured to load the mannequin from one side of the apparatus, and to unload the mannequin to the same side of the apparatus.
The garment digitization apparatus may be one wherein the apparatus is configured to load the mannequin from a position on one side of the apparatus, and to unload the mannequin to the same position on same side of the apparatus.
The garment digitization apparatus may be one wherein the apparatus is configured to load the mannequin from a first position on one side of the apparatus, and to unload the mannequin to a second position on the same side of the apparatus, wherein the first position and the second positions are different positions.
The garment digitization apparatus may be one wherein the apparatus is configured to load and to unload the mannequin from a first position on one side of the apparatus, and to load and to unload the mannequin to a second position on the same side of the apparatus, wherein the first position and the second positions are different positions.
The garment digitization apparatus may be one wherein the apparatus is configured to load the mannequin from one side of the apparatus, and to unload the mannequin to an opposite side of the apparatus.
The garment digitization apparatus may be one wherein the apparatus is configured to load the mannequin from a first position, or from a second position, on one side of the apparatus, and to unload the mannequin to an opposite side of the apparatus.
The garment digitization apparatus may be one wherein the apparatus is configured to load and to unload the mannequin from one side of the apparatus, and to load and to unload the mannequin from an opposite side of the apparatus.
The garment digitization apparatus may be one wherein the apparatus includes Radio-frequency identification (RFID) tracking for mannequin RFID tracking.
The garment digitization apparatus may be one wherein the apparatus includes the mannequin.
The garment digitization apparatus may be one wherein the mannequin includes a retroreflective surface.
The garment digitization apparatus may be one wherein the retroreflective surface is a retroreflective suit.
The garment digitization apparatus may be one wherein the image capture includes automatic texture cutout using the mannequin which includes a retroreflective surface.
The garment digitization apparatus may be one wherein the apparatus provides automatic texture cutout and transparency capture for improved layering of non-opaque garments.
The garment digitization apparatus may be one wherein the apparatus includes a portable device which is connectable to the computer system, in which the portable device is programmed to receive user input to control mannequin loading and/or garment image capture.
The garment digitization apparatus may be one wherein the portable device is a tablet.
The garment digitization apparatus may be one wherein a touch-based, device independent user interface (UI) is provided on the portable device, served as a web application for easy updates.
The garment digitization apparatus may be one wherein the user interface routes the user appropriately to optimize the workflow.
The garment digitization apparatus may be one wherein the portable device provides high quality image previews, to reduce a likelihood of storing unsuitable images.
The garment digitization apparatus may be one wherein the image capture includes raw image capture.
The garment digitization apparatus may be one wherein the image capture includes colour calibration.
The garment digitization apparatus may be one wherein the image capture includes fixed focusing.
The garment digitization apparatus may be one wherein the image capture includes a lossless imaging pipeline and local image processing.
The garment digitization apparatus may be one wherein the apparatus provides for standardizing relative mannequin position/orientation & apparent size to reduce digitization errors and allow for simpler staff training.
The garment digitization apparatus may be one wherein the apparatus provides for automatic mannequin loading & unloading for one-time dressing & reduced mannequin wear.
The garment digitization apparatus may be one wherein the mannequin loading system is operable to unload and to release the mannequin.
The garment digitization apparatus may be one wherein the apparatus supports digitisation of high-translucency garments, complex netting garments, and dipped hem garments.
The garment digitization apparatus may be one wherein the apparatus includes a single power switch.
The garment digitization apparatus may be one wherein the apparatus includes hidden cabling and cladding.
The garment digitization apparatus may be one wherein the apparatus is operable to rotate the mannequin wearing the garment using the mannequin rotation system through a plurality of 360° rotations.
According to a second aspect of the invention, there is provided a computer-implemented method of digitising a garment in 3D, using a garment digitization apparatus, the apparatus including a mannequin loading system, a mannequin rotation system, a computer system and a camera system, the method including the steps of:
(i) loading a mannequin wearing a garment using the mannequin loading system;
(ii) rotating the mannequin wearing the garment through at least 360° using the mannequin rotation system, and
(iii) capturing at least three images of the garment using the camera system during the mannequin rotation.
The method may include the step of digitising the garment in 3D, using the at least three captured images of the garment.
The method may include the step of storing the captured images of the garment on the computer system.
The method may be one wherein the mannequin is loaded from above.
The method may be one wherein the mannequin rotation and the image capture are synchronised.
The method may be one wherein the apparatus includes a lighting system, wherein the lighting system is controlled by the computer.
The method may be one wherein, to obtain automatic texture cutout results, the mannequin, which is a retroreflective mannequin, is rotated 360° 3 times under 3 different lighting conditions: texture lighting, alpha lighting, and albedo lighting.
The method may be one including using an apparatus of any aspect of the first aspect of the invention.
According to a third aspect of the invention, there is provided a computer program product executable on a computer system to digitise a garment in 3D, the computer system controlling a garment digitization apparatus, the apparatus including a mannequin loading system, a mannequin rotation system, the computer system and a camera system, the computer program product executable to:
(i) load a mannequin wearing a garment using the mannequin loading system;
(ii) rotate the mannequin wearing the garment through at least 360° using the mannequin rotation system, and
(iii) capture at least three images of the garment using the camera system during the mannequin rotation.
The computer program product may be further executable to digitise the garment in 3D, using the at least three captured images of the garment.
The computer program product may be further executable to perform a method of any aspect of the second aspect of the invention.
According to a fourth aspect of the invention, there is provided a mannequin including a stretchable retroreflective material covering.
The mannequin may include an upwardly projecting spigot.
According to a fifth aspect of the invention, there is provided a method of manufacturing a mannequin including a stretchable retroreflective material covering, comprising the steps of removing a top layer of a mannequin and replacing the top layer with stretchable retroreflective material to form a stretchable retroreflective material covering.
The method may include the step of clamping the retroreflective material in place on the mannequin.
According to a sixth aspect of the invention, there is provided a trolley including a support arranged to receive a mannequin and to support the mannequin from above.
The trolley may be one in which the support is arranged to secure the mannequin to the trolley.
According to a seventh aspect of the invention, there is provided a trolley including a support arranged to receive a mannequin and to support the mannequin from below.
According to an eighth aspect of the invention, there is provided a system including an apparatus of any aspect of the first aspect of the invention, and a trolley of any aspect of the sixth or sevenths aspects of the invention.
According to a ninth aspect of the invention, there is provided a mannequin loading apparatus comprising a gantry system and a mannequin, the gantry system including a vertically displaceable collet fixture, the mannequin including an upwardly projecting spigot, wherein the collet fixture is lowerable to engage and to secure the spigot, and wherein the collet fixture is raiseable by the gantry system to lift the mannequin.
The mannequin loading apparatus may be one wherein the collet fixture includes a tapered socket.
The mannequin loading apparatus may be one wherein the tapered socket includes a plurality of ball bearings that are lockable into a groove on tooling fixed to the top of the mannequin.
The mannequin loading apparatus may be one wherein when the collet fixture is lowered onto the mannequin tooling, a flange on the tooling activates an engage position pin on the collet fixture that triggers an engaging mechanism.
The mannequin loading apparatus may be one wherein when a fixture pin is depressed, the fixture lever springs into an engage position and touches a docking station lever pin.
The mannequin loading apparatus may be one wherein when a lever on the collet fixture springs, the plurality of ball bearings are locked into the groove of the mannequin tooling to engage and secure the mannequin to the collet fixture.
The mannequin loading apparatus may be one wherein the collet fixture is releasable from the spigot so as to release the mannequin.
The mannequin loading apparatus may be one wherein the collet fixture is releasable from the spigot so as to release the mannequin to a docking station.
The mannequin loading apparatus may be one wherein the gantry system rotates the collet fixture, so that a lever is pushed against a lever pin mounted at the docking station.
The mannequin loading apparatus may be one wherein when the lever is effectively pushed to an end of its travel, the locating bearings are unlocked and therefore released from grooves on the mannequin tooling.
The mannequin loading apparatus may further comprise an electronic circuit to detect a position of a release pin with regard to a docking station pin, and to verify a configuration of the collet fixture pre-mannequin loading and post-mannequin unloading.
Aspects of the invention will now be described, by way of example(s), with reference to the following Figures, in which:
There is provided a unified and automated imaging solution suitable for 3D Garment Digitisation.
1.1 Features and Innovations
There is provided a new garment imaging solution to address the challenges described in the Background to the Invention section. The solution includes overall improvements to Metail's existing garment photography hardware and software. These improvements advance the quality of the final image, ease the image capturing operation and reduce costs in digitization. In summary, the new garment imaging system may include the following hardware and procedural improvements:
Several software improvements may be provided. Examples are as follows:
Furthermore, operational process analysis has ensured both hardware and software are designed with the most efficient and rewarding user experience in mind.
The following novel elements may allow us to realise the above improvements:
1.2 Benefits
An example of the new garment image capturing system may provide the following three major benefits:
Improving the Quality of Garment Photography
Improved overall garment appearance, image quality and garment range can all contribute to improvements in our key performance metrics of virtual fitting products. The new garment imaging system improves the overall quality of garment photography. This may be achieved by:
1) Improving overall garment appearance, with the features of calibrated camera position, automatic texture cutout (see, e.g. U.S. Pat. No. 8,605,148B2), translucency capture, and automated loading (no need for rushed redressing).
2) Improving image quality, by using RAW capture, lossless compression, colour calibration, high-colour rendering index (CRI) lighting, and improved optics.
3) Supporting a wider range of garments, e.g. supporting digitisation of high-translucency and complex netting captured, dipped hems, and other complex legwear.
Improving Studio Staffs Experience.
The ease of integration and operational process is a key factor in acquiring the customers to the digitisation solution. The new garment photography system may implement the following four example features that improve the user experience, as shown in Table 1.1:
Decrease the Digitisation Cost
The cost for photography can be broken broadly into two categories: operational expenditure and capital expenditure. The capital expenditure would increase with the new photography system because expenditure on each rig in hubs will increase. However, this is more than balanced by operational expenditure savings. The new garment imaging system decreases our operational expenditure by:
1) Increasing garment throughput per rig: Speed reduction caused by the automatic texture cutout solution (see, e.g. U.S. Pat. No. 8,605,148B2) is now compensated with loading & capture speed increases & system downtime strongly reduced with operational process improvements.
2) Reducing the overall digitization time: Hardware and software for automatic texture cutout & auto-alpha channel capture (see, e.g. U.S. Pat. No. 8,605,148B2) are implemented. The system also provides small-file size for fast transfer to the off-site operation teams for post-processing.
The new garment photography system also reduces the operation cost of clients. This is achieved by:
1) Increasing garment throughput per stylist, with user experience (UX) improvements, automated garment, stylist and mannequin ID, redressing eliminated, operational model changed, and system sped up to reduce downtime.
2) Reducing supervision overhead: with more intuitive & informative UI, improved monitoring & feedback, and strong reduction of maintenance & repair.
1.3 Components
The new imaging system example has been split into 7 key components:
1. Frame and structure components, i.e. the part of the rig by which the location and orientation of all components is constrained so that consistency between garment images is maintained.
2. Camera components, including the imaging device, lens, connectors and image processing software.
3. Motion Control components, including the motion controller, drivers, power supplies, motors, actuators and all other componentry involved in moving and loading the mannequins.
4. Lighting components, including texture lighting, alpha lighting and albedo lighting necessary to capture as much accurate detail as possible, for instant cutout and aesthetically appealing results.
5. Mannequin components, including all elements of mannequin modification, including the retroreflective suit, spigot, and the stand to support the mannequin.
6. Back-end software and computer hardware components, including all software and computer hardware involved in controlling components, collating images and uploading them to the cloud for further processing.
7. Front end software components, including all software and UX/UIs involved in providing the studio staff with a user interface by which to control the system.
The design details, requirements, and options of each component are addressed in the following Sections 2-8.
2. Frame and Structure Components
In an example, the photo-rig structure fixes components to ensure consistency between all garment images. This allows us to optimise our image pipeline and pinpoint areas for improvement more easily. This also reduces the possibility of tampering or accidental displacement of components, thereby eliminating user-maintenance, allowing users to focus on great styling. The frame implements cable routing and thermal management for all components, providing a tidy and safe workspace for users and a long life for components.
Development work on the frame/structure started by noting spatial constraints for components. These constraints allowed for two overarching design concepts: 1) supporting the mannequin from below or 2) supporting the mannequin from above. The former concept is more traditional: it allows for easier redressing in-situ and shooting accessories and shoes without significant design engineering thought. The latter concept is more unconventional, but provides far superior automatic texture cutout (for cutout, see, e.g. U.S. Pat. No. 8,605,148B2). Supporting the mannequin from above was chosen as a core concept. We supplied various conceptual designs to incorporate this. In a preferred example design, the design uses a large rigid roof to fix the separation of components, with the back-panel, lighting column, and camera column providing stability while housing their respective components (see e.g.
A preferred example structure is primarily built using an extruded aluminium profile system (e.g. a Paletti System, Paletti USA, 145 Keystone Drive, Montgomeryville, Pa. 18936, USA). This allows for easy alterations and additions, is lightweight, strong and simple to disassemble and reassemble. The camera and light columns are further supported with steel plates, the backdrop incorporates acrylic sheets for diffusion and protection of the LEDs and the entire frame has adjustable feet to ensure stability in any environment.
Much of the work in this component involved integration of other components (see Sections 3-8). The profile system allowed for simple and strong housing of the three LED texture-lighting panels as well as the ring light. Brackets were produced to aid in the installation of the actuators and cable carriers in the roof section of the frame. Furthermore, brackets for the industrial PC, industrial camera and motion-control electronics cabinet enabled seamless and secure integration. Finally, in a preferred example, the mannequin loading bays used an aluminium frame and nylon contact points for smooth interaction with the mannequins.
2.1 Requirement Definition
The initial high-level requirements for an example frame and structure, ranked in preferred order of importance and ease of implementation, are given in Table 2.1:
A preferred example platform is not designed to be routinely stood on. Its sturdiness & stability is to ensure it is safe to stand on accidentally or for maintenance.
Furthermore basic requirements for a preferred example are illustrated diagrammatically (see e.g.
2.2 Concept Options and Decisions
A primary concept option choice was between supporting the mannequin from above and supporting the mannequin from below. Two sketches were made to illustrate examples of these options (see
Example concept options noted for the frame and the structure are given in Table 2.2:
A most challenging choice was deciding between supporting the mannequin from above or below. The mechanics of installing motion control systems above or below were similar in complexity. A major factor was a desire for a seamless even floor section and the range of garments that could be shot, seeing as the price difference between the two configurations was considered to be negligible.
The decisions made for preferred example designs and their reasons are given in Table 2.3:
Preferred examples are illustrated in
3. Camera Component
The use of an appropriate imaging device on the photo-rig was seen as very important to providing a high-quality and reliable system. The implementation of automatic texture cutout technology (see, e.g. U.S. Pat. No. 8,605,148B2) dictated cycles of up to 16,000 daily shutter actuations. SLRs (Single Lens Reflexes) and other such studio cameras employ mechanical shutters rated with a lifespan of around 150,000 actuations, so with their implied life expectancy of less than two weeks, it was determined that only shutterless cameras would be suitable. Research was undertaken to compare video cameras and industrial cameras, as these are the two most common types of shutterless camera. In principle, either would provide the endurance required, so appraisal was made with regard to cost and ease of integration.
There was a wide range of prices for video cameras, and we focused on the cheapest 4K camera available. This had a sufficient resolution, but the task of extracting lossless image data from the device was deemed to be too challenging. The SDI (Serial Digital Interface) output requires frame grabbing using specialized hardware, and even after this, the output is not comparable to RAW image data. More expensive cameras may have allowed finer control, but the principle of industrial cameras was more appealing.
A camera raw image file contains minimally processed data from the image sensor of e.g. a digital camera, image scanner, or motion picture film scanner. Raw files are so-named because they are not yet processed and therefore are not ready to be printed or edited with a bitmap graphics editor. Normally, the image is processed by a raw converter in a wide-gamut internal colorspace where precise adjustments can be made before conversion to a “positive” file format such as TIFF or JPEG for storage, printing, or further manipulation. This often encodes the image in a device-dependent colorspace.
Cameras of various industrial camera manufacturers were investigated. Our aim was to find the camera with the largest sensor affordable so a 40 mm lens could be used at an acceptable distance from the mannequin. In an implementation, we adopted a GigE Vision camera, which has a 4/3″ (22.7 mm diagonal) sensor, providing a crop factor relative to 35 mm of just 1.9×, allowing a camera-mannequin distance of 3.75 m. The GigE Vision camera is controlled and sends images via Ethernet, allowing very precise control and synchronization, and can output RAW images for a perfectly lossless imaging pipeline.
The GPIO (General Purpose Input/Output) on the camera enables high-speed triggering from the motion controller, eliminating dependency on non-realtime software for synchronization. During mannequin rotation, the images captured are therefore at precisely the same angle as those captured 360° earlier. This alignment between images is essential to achieving accurate automatic texture cutout (see, e.g. U.S. Pat. No. 8,605,148B2) without delaying the photography process.
In an example, the camera may be controlled via a camera control software development kit (SDK) (e.g. GigE Vision compliant SDK in our implementation). This allows fine control of many parameters. The images are passed on to RawTherapee for demosaicing and other RAW-level processing. The images are then passed on to an image processing software (e.g. ImageMagick) for further processing. Using open-source free software enables us to minimize costs and keep as much of the pipeline in a Linux environment as possible.
3.1 Requirement Definition
An example of high-level requirements for the cameras is given in Table 3.1, ranking in preferred example order of importance and ease of implementation:
Output quality, speed, and durability were considered most important features of a camera. An undesirable example camera (Canon EOS 6D) had excellent quality (20.2 MP), poor speed (4.5 fps) and unacceptable durability (150,000 shutter actuations). Cost calculations over 3 years were recorded. A 4K video camera (e.g. BlackMagic) seemed to be the most economical choice, with good quality (8.4 MP), excellent speed (30 fps) and excellent durability (electronic shutter).
3.2 Concept Options and Decisions
The primary concept option choice was between digital single-lens reflex cameras (DSLR), video or industrial cameras. Details of each are available from manufacturers' websites.
Example concept options noted for the cameras are given in Table 3.2:
Most of the concept options noted above are influenced by the choice of camera type. Initial analysis showed product cost was lowest for adopting a video camera (e.g. BlackMagic). However, integration cost and unknowns quickly made video seem to be a risky option. The decisions made and their reasons are given in Table 3.3:
The relevant software component uses an SDK to communicate over ethernet. In an implementation, triggering is done directly from motion controller to camera via Hirose connector. In an example, image processing is done using our own application to convert stream to DNG (Digital Negative), using RawTherapee to convert to TIFF, and using ImageMagick to convert to PNG.
4. Motion Control Component
Three downsides of an earlier photo-rig were noted. Firstly, we relied upon a third-party turntable (e.g. LinearX LT360) for mannequin rotation which overly-constrained our software and hardware choices. Secondly, lifting and moving mannequins manually resulted in dropping and knocking them, thereby reducing their average lifespan to less than a year. Thirdly, lifting mannequins (usually at the waist or the crotch) to load them often disturbed the clothing fabric enough to require redressing in-situ, causing a bottleneck in the system.
To address the aforementioned problems, we redesigned the motion control module for the rig system. After testing various studio layouts and ways of working, we eventually settled upon a 4-axis system, allowing overhead loading and unloading on both sides of the photo-rig (see
We designed and built a gantry system of 3 linear axes (see
An example mannequin motion control is centred on a tool changing mechanism more normally used in a CNC (Computer Numerical Control) machine. We have developed a special design that, in an example, allows the gantry to safely load a mannequin from the docking station (see
From this point, in an example, the servo motors move the carriages along the actuators to position the mannequin appropriately. In an example, rotation begins when in position, including an acceleration phase, and the motion controller sends input/output (I/O) signals at specified angles. These I/O signals are routed to the camera and lights as appropriate, so the correct shots and lighting are triggered respectively. When a cycle (e.g. 4 rotations) is completed, the mannequin is returned to the loading/unloading bay, where it is released back onto the trolley. The location of each carriage (and therefore the position and orientation of each mannequin) is frequently detected by appropriately placed limit switches.
4.1 Requirement Definition
High-level example requirements for the motion control, are given in Table 4.1, ranking in order of preferred importance and ease of implementation:
Discussion is focused on constructing a mannequin-specific rotational device rather than a generic turntable. Electronic loading would be a small addition to the cost of motion control, and negligible in comparison to designing and manufacturing a manual solution.
4.2 Concept Options and Decisions
The design of the preferred example of the motion control relied primarily upon the decision regarding the frame and structure (Section 2) between supporting the mannequin (Section 5) from above or below.
The concept options noted for the preferred example motion control design and loading mechanisms are given in Table 4.2:
Examples of 2 vs 3 axis and 1-sided vs 2-sided systems are illustrated in
When mannequin support from above was decided, it was seen that off the shelf turntables would not be feasible. This also eliminated “one motor, many turntables” and “manual loading” options. Example decisions made and their reasons are given in Table 4.3:
From the proof-of-concept experiment, we find that a 3 axis system (e.g.
4.4 Detailed Example Designs for Mannequin Loading and Unloading
We have provided a special design of the collet fixture (i.e. the tool head, see e.g.
The design of the docking station must be compatible with the gantry to ensure that the positioning sequencing is the same with the same relative distances between the main components of the docking station. Failure to observe this may result in the collision of the gantry with the fixed elements of the docking station and therefore cause damage to the gantry and/or the fixtures. The docking station position may be defined as the X, Y, Z coordinates where the gantry collet fixture fully engages from above with the mannequin connector, causing the collet mechanism to lock in to the engage position, without asserting force to the docking station framework.
In an example, the collet fixture at the gantry includes a tapered socket. Inside the socket are e.g. three ball bearings that lock into a groove on the tooling fixed to the top of the mannequin. The collet fixture moves downwards to load the mannequin. When the collet fixture is lowered onto the mannequin tooling, a flange on the tooling activates the engage position pin on the collet fixture that triggers the “Engaging” mechanism. The lever on the collet fixture springs e.g. about 40 degrees anticlockwise, and the locating bearings are locked into the groove of the mannequin tooling to engage and secure the mannequin to the collet fixture (this is the collet fixture engaged position), as shown in
To release the mannequin, in an example, the gantry rotates the collet fixture anti-clockwise, so that the lever is pushed against the lever pin mounted at the docking station. This effectively turns the lever in the clockwise direction. When the lever is effectively pushed to the end of its clockwise travel, the locating bearings are unlocked and therefore released from grooves on the mannequin tooling. The lever should stay in position and therefore allow another part to be engaged into the collet fixture (the collet fixture release position).
4.5 Electronic Designs for Mannequin Loading and Unloading
In a prototyping phase, we realised that there was an issue with the collet fixture in which it would sometime not lock-open ready to engage with the next mannequin. We tracked the three main causes of this to be the following:
The system needed to detect when the latch had failed so that an exception could be raised. Unfortunately, the collet fixture blocks the pickup of the tool chuck when the latch is closed, resulting in an artificial increase in length of the assembly with the knock-on effect of the collet fixture attempting to push the dock and the mannequin down by that distance resulting in slippage in the axis, and strain on the frame.
The solution was to detect the position of the release pin with regards to the docking station pin, and to check the positions of pre-mannequin loading and post-mannequin unloading. To achieve this a simple continuity circuit may be used where when docking station pin and input sensor are held at +10V, using a voltage divider from a 24V supply, and the sensor is grounded through the rig when the release pin makes contact with the dock pin (see
To detect if the head is in the correct locked-open position, the loading/unloading operation may be arranged to perform the following tasks:
5. Lighting Components
An example photo-rig was inadequate in terms of lighting, as it relied upon xenon flash-strobes with similarly low endurance for retro-reflective-mannequin-based photography (see, e.g. U.S. Pat. No. 8,605,148B2) as for the SLR camera, described above. As the mechanical shutter was replaced with a shutterless sensor, the strobes were replaced with LED panels, in an example. We firstly considered fluorescent tubes, as they boast an excellent always-on lifetime and strong spectral fidelity CRI (Colour Rendering Index). However, fluorescent tubes do stress their filaments on starting-up, so we are concerned that repeatedly switching for the retro-reflective-mannequin-based system would yield a poor overall lifetime. We adopted LED panels in our design example (see e.g.
In an example design, the lighting components of an example system includes 6 LED-panels: one for the ringlight to activate the retro-reflectivity, three for normal texture lightings (2 side panels and 1 overhead lighting), and two for the backlight (see
To obtain the automatic texture cutout results, the mannequin may be rotated 3 times under 3 different lighting conditions: texture lighting, alpha lighting, and albedo lighting. Firstly, in an example, under normal texture lighting where the two side panels and the single overhead light illuminate the detail of the garment (see e.g.
5.1 Requirement Definition
The initial high-level requirements for example lighting components, are given in Table 5.1, ranking in preferred order of importance and ease of implementation:
The most important requirement is that of reliability. Tripling shots per garment and increasing daily throughput could demand >16,000 images per day. With strobes, this would quickly result in blown bulbs. Furthermore, the frequency of flashes would be too fast for recharging, and the room would be subject to constant flashing, which is undesirable for staff.
5.2 Concept Options and Decisions
Fluorescent tubes were seen as possibly ideal, as they are cheap and have a very good light spectrum. LEDs can also have a very high CRI but are these are much more expensive per lumen. The concept options noted for the example lighting specifications and designs are given in Table 5.2:
The most important option is lighting type, which imaging quality and durability are dependent on. Our desire to repeatedly turn lights on and off made this option of using fluorescent lighting less appealing. Even with a programmed start (preheating filaments) the lamp life for fluorescent tubes is approximately 100,000 on/off cycles, which is too short for our goal of using components with 5-year lifespans. The warm up must be <0.75 s for a 10 rpm mannequin rotation rate, further casting doubt on the suitability of fluorescents.
The preferred decisions made and their reasons are given in Table 5.3:
Example light locations are illustrated in
6. Mannequin Related Designs
The mannequin was in an example supported using two steel spigots onto which each of the legs were slotted. In the design of an example system, we supported two brands of mannequin using interchangeable spigots depending on brand used. This allowed for quick swapping of mannequins after some practice, but was more challenging for shorter users or those with less upper-body strength. Consequently, the mannequins were often dropped or knocked against other items in the studio, reducing their average lifetime to approximately one year. Furthermore, lifting the mannequins (usually at the waist or crotch) disturbed the dressed garment and required redressing once in-situ. The above drawbacks led us to develop a system which relied less upon user strength and coordination. Aspects of a relevant solution are detailed in Section 4.
The neck spigots of some mannequins (e.g. Alvanon Alvaform), and of available sturdy stands, led us to develop a system to take advantage of these features (see e.g.
To implement automatic texture cutout technology to the highest possible standard required a rethink about how to develop a retro-reflective mannequin (see, e.g. U.S. Pat. No. 8,605,148B2). We constructed a covering for the mannequin using a stretchy retro-reflective material with the fewest seams possible. We took a mannequin apart, removed the top layer and replaced it with the retro-reflective material. Clamping the material under metal sections at the neck, shoulders and ankles allowed for a smooth and even covering (see e.g.
The stand needed little adaptation to make it appropriate for use in the new photo-rig. We designed and constructed an adaptor for the spigot holder to allow support of the male tool changer component (see e.g.
6.1 Requirement Definition
An initial high-level example of requirements for mannequins is given in Table 6.1, ranking in order of importance and ease of implementation:
These design requirements can be split into two primary streams: 1) retro-reflective suit construction and 2) mannequin stand/spigot modification. An example of a retro-reflective suit design is given in
6.2 Concept Options and Decisions
Example concept options noted for the mannequin and related designs are given in the Table 6.2:
The preferred decisions made and their reasons are given in Table 6.3. Preliminary investigations showed that retro-reflective paint was not a good option, so we focused on implementing a retro-reflective suit as neatly as possible. To cover the retro-reflective suit over the mannequin, welding, sewing and gluing were used, in an example.
Further detailed options were considered for the attachment mechanism, e.g. the stand of the mannequin (see Table 6.4). This first involved an adaptation of a tool changer mechanism; an example is illustrated in
Example decisions made on the attachments and their reasons are given in Table 6.5. Using the existing stand of the mannequin makes sense, as they are well made and need little modification. The mannequin to stand interface could be mostly kept, also with little modification. An electrical interface made sense, as this would require only an extension of the motion control system. An automated interface is also desirable for this reason. A specific mechanism for the interface may still incorporate the mechanism illustrated in
7. Back-End Software and Computer Hardware Components
The photo-rig of an example system used our custom software written in the .NET framework, running on Windows on an Intel NUC PC. The only connected components were the SLR camera via USB and the turntable via serial RS-232. The camera would automatically trigger the flashes. This system required a mouse, keyboard and barcode scanner for input, so a desk and chair were supplied to studios. This setup constrained one user to the PC and other users to their assigned mannequin. After much operational analysis, we decided to pursue a system allowing more operational fluidity, exploiting the benefits of a tablet for each user and easier sharing of roles to overcome occasional holdups.
The components in the new example system are all connected to an industrial PC, which acts as a server to the tablets, so multiple users can interact with the system simultaneously. The camera is connected to the PC via Ethernet and sends images and receives settings this way. The motion controller is also connected to the PC via Ethernet and communicates via Modbus over TCP/IP protocol. The lights and camera are switched/triggered by the motion controller via a 24V control signal from the bi-directional I/O ports.
In our designed example, the mannequin motion control system is controlled by the Trio motion controller. The controller has 5 stepper outputs/encoder inputs so can control either stepper motors or servo motors. The controller has enough memory to store simple programs written in a Basic programming language (e.g. TrioBASIC). This allowed our developers to write code for the industrial PC to call the motion control code, thereby controlling the mannequin movement indirectly. The controller is programmed to respond appropriately to limit switches to ensure the system works as expected.
Example back-end software also includes programs to 1) process images, 2) upload images, 3) store images, and 4) serve the UI to the tablets. The image processing is done using RawTherapee and ImageMagick—both open source free software. The tablet UI is hosted on the industrial PC and accessed on the tablets using a browser for device compatibility.
7.1 Requirement Definition
An initial high-level requirements example for the back-end software, is given in Table 7.1, ranking in order of preferred importance and ease of implementation:
The computer hardware should meet general requirements such as durability. All components should be server-quality and all interconnects ethernet or other non-consumer level. Early on it was decided that running on Linux would be preferred, so, for example, writing .NET code that runs in mono would allow this and Windows compatibility. The software development structure is to develop each software component as a separate deliverable module. This may include an orchestrator, a web server and any other back-end modules necessary.
Nonetheless, an example component communication wireframe was drawn up to illustrate the sequence of events in a possible deployment. This is shown simply in
7.2 Concept Options and Decisions
Most concept options were influenced by general system design. The back-end is supportive of many concept decisions made in other work packages. In an example, it was decided to run some of the back-end on a server unit so access to the components could be regulated. It soon became apparent that the processing power of a large unit and the mains power used supported moving all but the UI to the server, in a preferred example.
Example high-level concept options noted for the back-end software are given in Table 7.2:
Example preferred decisions made and their reasons are given in Table 7.3:
8. Front-End Software Components
In a preferred garment imaging system example, the front-end software implements a touch-based interface to allow simpler and faster data input and image review. Data input may be completed primarily with a barcode scanner but an on-screen keyboard may also be available (see e.g.
In an example, four possible user flows were identified. Depending on the number of layers, waiting time and reshooting may be necessary, hence the interface routes the user appropriately to optimize the workflow (see e.g.
RFID (Radio Frequency IDentification) tags on the mannequins or stands may allow the system to identify which mannequin corresponds to which tablet, and therefore which data. The tablets may be docked on the mannequin stands during operation and may be charged on the rig when not in use.
8.1 Requirement Definition
An initial high-level example requirements for the front-end software, is given in Table 8.1, ranking in preferred order of importance and ease of implementation:
In terms of functionality, the front-end software may be implemented as a web app with touch functionality. Certain features of the back end such as multiple simultaneous input process (i.e. multitasking) may, however, be necessary to effect this. The above requirements have been designed conceptually, as illustrated in
8.2 Concept Options and Decisions
The example concept optiocept options for the front-end software noted are given in Table 8.2
A cluster process may be provided; an example is mapped out in
Note
It is to be understood that the above-referenced arrangements are only illustrative of the application for the principles of the present invention. Numerous modifications and alternative arrangements can be devised without departing from the spirit and scope of the present invention. While the present invention has been shown in the drawings and fully described above with particularity and detail in connection with what is presently deemed to be the most practical and preferred example(s) of the invention, it will be apparent to those of ordinary skill in the art that numerous modifications can be made without departing from the principles and concepts of the invention as set forth herein.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1516700.0 | Sep 2015 | GB | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/GB2016/052933 | 9/21/2016 | WO | 00 |
