BARCODE IMAGE RECOGNITION METHOD AND DEVICE USING THE SAME

Abstract

A barcode image recognition device includes a camera module, an input component, a processing module, a storage module, an output module, and a display. The input component is coupled to the camera module, and the processing module is coupled to the camera module. The processing module includes a first artificial neural network, a conversion module, a second artificial neural network, and a comparison module. The first artificial neural network identifies the first target image of each target object in the default images captured by the camera module. The second artificial neural network identifies the second target image in the captured image captured by the camera module. The storage module is coupled to the processing module. The output module is coupled to the comparison module and the storage module. The display is coupled to the output module.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This non-provisional application claims priority under 35 U.S.C. § 119 (a) to patent application No. 202311123571.X filed in China, P.R.C. on Sep. 1, 2023, the entire contents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

The present disclosure relates to a barcode recognition technology, and in particular relates to a method and device for recognizing barcode images.

Barcodes may carry multi-character information, such as related information about product names, specifications, origins, prices, batch numbers and the like. Therefore, the barcodes are often applied to mark commodities, track logistics, manage inventory, control production flows and the like. In these logistics and management and control, accurate and controlled recognition of barcodes or texts can improve the operating efficiency of a related worker and reduce manual operations and errors.

SUMMARY OF THE INVENTION

In view of this, the present disclosure provides a barcode image recognition method and a device using the same, which are suitable for accurately and controllably outputting a string of indicated barcodes or text when there is a plurality of barcodes or texts.

In an example, a barcode image recognition method includes: enabling a camera module; continuously capturing at least one target object in advance by the camera module after the camera module is enabled to obtain a plurality of default images with continuous capturing time; identifying, by a first artificial neural network, a first target image of each target object in each default image; capturing each first target image from each default image; converting each first target image to a string; storing each first target image and the string corresponding to each first target image; triggering the camera module according to a confirmation signal to take a single capture of a predetermined mark to obtain a captured image; identifying, by a second artificial neural network, a marked image of the predetermined mark in the captured image and a second target image of a selected target object of the at least one target object that overlaps with the predetermined mark; comparing the second target image with each first target image to obtain a final selected image, the final selected image being the first target image with the highest similarity to the second target image; and reading a string corresponding to the final selected image from the stored strings and outputting the same.

The present disclosure further provides a barcode image recognition device, including a camera module, an input component, a processing module, a storage module, an output module, and a display. The camera module is configured to continuously capture at least one target object in advance to obtain a plurality of default images with continuous capturing time. The input component is coupled to the camera module, and is configured to generate a confirmation signal to trigger the camera module to take a single capture of a predetermined mark to obtain a captured image where the predetermined mark overlaps with the at least one target object. The processing module is coupled to the camera module, and includes a first artificial neural network, a conversion module, a second artificial neural network, and a comparison module. The first artificial neural network identifies a first target image of each target object in each default image. The conversion module converts each first target image to a string. The second artificial neural network identifies a marked image of the predetermined mark in the captured image and a second target image of a selected target object of the at least one target object that overlaps with the predetermined mark. The comparison module is coupled to the first artificial neural network and the second artificial neural network, and compares the second target image with each first target image to obtain a final selected image, the final selected image being the first target image with the highest similarity to the second target image. The storage module is coupled to the processing module, and stores each first target image and the string corresponding to each first target image. The output module is coupled to the comparison module and the storage module, and reads the string corresponding to the final selected image from the storage module. The display is coupled to the output module, and displays the string read by the output module.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a functional block diagram of a barcode image recognition device according to some examples.

FIG. 1B is a schematic diagram of an application scenario of a barcode image recognition device according to the example of FIG. 1A.

FIG. 2 is a flowchart of a barcode image recognition method according to some examples.

FIG. 3 is a schematic diagram of a default image in step S110 of FIG. 2.

FIG. 4 is a functional block diagram of an example of a conversion module according to the example of FIG. 1A.

FIG. 5 is a flowchart of a barcode image recognition method according to some examples after step S120 in FIG. 2.

FIG. 6A is a schematic diagram of an application scenario of a barcode image recognition device according to another example.

FIG. 6B is a functional block diagram of a barcode image recognition device in FIG. 6A.

FIG. 7 is a schematic diagram of an example of a barcode image recognition device in FIG. 6B.

FIG. 8 is a schematic diagram of another example of a barcode image recognition device of FIG. 6B.

FIG. 9 is a flowchart of a barcode image recognition method according to some examples before step S150 in FIG. 2.

FIG. 10 is a schematic diagram of an example of a barcode image recognition device in FIG. 6B.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1A, a barcode image recognition device includes a camera module 20, an input component 30, a processing module 40, a storage module 60, an output module 70, and a display 80. The input component 30 and the processing module 40 are each coupled to the camera module 20. The storage module 60 is coupled to the processing module 40. The output module 70 is coupled to both the processing module 40 and the storage module 60. The display 80 is coupled to the output module 70.

The processing module 40 includes a first artificial neural network 42, a conversion module 44, a second artificial neural network 46, and a comparison module 48. The first artificial neural network 42 receives images captured by the camera module 20. The conversion module 44 is coupled to the first artificial neural network 42 and processes the images captured by the camera module 20. The second artificial neural network 46 also receives the images captured by the camera module 20, and the comparison module 48 is coupled to the first artificial neural network 42 and the second artificial neural network 46 and also processes the aforementioned images.

Referring to FIG. 1B and FIG. 2, when the barcode image recognition device is used, the camera module 20 (step S100) is enabled, and the camera module 20 will continuously capture target objects 10 in advance to obtain a plurality of default images 22 with continuous capturing time (step S110). In some examples, the target objects 10 are barcodes 122 and 124 or strings 146 and 148.

Referring to FIG. 1B, FIG. 2 and FIG. 3 together, the default image 22 may be similar to a picture a user previews and captures in real time through a screen when capturing with an electronic device. The default image 22 has a first target image 220 of each target object 10. Taking the target object 10 having the barcode 124 and the string 146 as an example, the default image 22 will include a barcode image 224 of the barcode 122 and a string image 226 of the string 146. The barcode image 224 and the string image 226 are obtained by the first artificial neural network 42 in the processing module 40 identifying the default image 22 (step S120).

The processing module 40, after capturing these identified first target images 220, converts each first target image 220 to strings 450 and 452 (see FIG. 7 or FIG. 8) with the conversion module 44 (step S130).

Referring to FIG. 4, in some examples, the conversion module 44 may be a decoder 440. When the first target image 220 is converted to the string 450, the decoder 440 decodes the barcodes 122 and 124 to obtain information carried by the barcodes 122 and 124, where the information may be a text or a website address.

In some other examples, the conversion module 44 may be an identifier 442. When the first target image 220 is converted to the string 452, the identifier 442 performs text identification on the strings 146 and 148 and identifies the string image 226 as the string 452.

Referring to FIG. 4 and FIG. 5, in some further examples, in a case where the conversion module 44 includes both the decoder 440 and the identifier 442, the first artificial neural network 42 can identify a type of the first target image 220 (the type is the above-mentioned barcode image 224 or string image 226).

In detail, after the first artificial neural network 42 identifies the first target image 220 in the default image 22 (step S120), the first artificial neural network 42 further identifies each first target image 220 as the barcode image 224 or string image 226 (step S122). If the first artificial neural network 42 identifies the first target image 220 as the barcode image 224 (step S125), the decoder 440 in the conversion module 44 decodes the barcode image 224 into the string 450 (step S132). If the first artificial neural network 42 identifies the first target image 220 as the string image 226 (step S127), the identifier 442 in the conversion module 44 performs text identification on the string image 226 to obtain the string 452 (step S134).

Referring back to FIG. 2, the storage module 60 stores each of the above-mentioned first target images 220 and the strings 450 and 452 corresponding to the first target images 220 converted by the conversion module 44 (step S140). In some examples, a database of barcodes 122 and 124 and strings 146 and 148 can be formed after the first target images 220 and the corresponding strings 450 and 452 in the plurality of default images 22 are stored, so that the strings 450 and 452 corresponding to the barcodes 122 and 124 and the strings 146 and 148 are extracted at any time when needed later.

Referring back to FIG. 1B, when the user desires to obtain information about a particular target object 10, the particular target object 10 may be directly indicated by an indicating device 50 or a finger, and then a predetermined mark 52 overlapping the target object 10 may be generated on the target object 10, and the indicated target object 10 is taken as a selected target object 16. In some examples, the indicating device 50 is an infrared pen or a laser pen, and the generated predetermined mark 52 is a cursor. In some examples, the indicating device 50 can be implemented with the finger, and the predetermined mark 52 is a part where the finger points to the target object 10.

Referring to FIG. 1B, FIG. 2, FIG. 7 and FIG. 8, after the user indicates the target object 10 with the indicating device 50 or finger, the camera module 20 takes a photo of the selected target object 16 according to a confirmation signal (step S150) to obtain a captured image 24 (step S160). Here, the second artificial neural network 46 identifies the captured image 24 to obtain a second target image 26 of the selected target object 16 (step S170). The second target image 26 is the first target image 220 overlapping with marked images 240 and 242 of the predetermined mark 52. Subsequently, the comparison module 48 compares the second target image 26 and each previously stored first target image 220 (step S180), and the first target image 220 with the highest similarity to the second target image 26 is used as a final selected image 480. The output module 70 extracts the strings 450 and 452 corresponding to the final selected images 480 (i.e., the first target image 220 as the final selected image 480) from the storage module 60, and the strings 450 and 452 are displayed by the display 80 (step S190).

In this way, the user can quickly and easily select the strings 450 and 452 corresponding to the barcodes 122 and 124 or the strings 146 and 148 the user desires to output from the plurality of barcodes 122 and 124 or the strings 146 and 148, and after selection, quickly outputs the strings 450 and 452 corresponding to the selected barcodes 122 and 124 or the strings 146 and 148 from strings 450 and 452 obtained by decoding or identifying a plurality of barcodes 122 and 124 or the string 148.

Taking the indicating device 50 as the infrared pen or the laser pen as an example, after the user presses a switch of the infrared pen or laser pen, the infrared pen or the laser pen generates a cursor, and the marked image 240 of the cursor (i.e., the predetermined mark 52) also begins to appear in the default image 22 captured by the camera module 20 that performs capturing continuously. Taking the finger as the indicating device 50 as an example, when the finger of the user enters the capturing range of the camera module 20, the marked image 240 of the finger also begins to appear in the default image 22 captured by the camera module 20.

Referring to FIG. 6B and FIG. 9, in some examples, the processing module 40 includes a mark confirmation module 49. The mark confirmation module 49 is coupled to the camera module 20. The mark confirmation module 49 identifies the default image 22 repeatedly captured by the camera module 20 to confirm whether there are the marked images 240 and 242 in the default image 22 (step S150a), and after confirming that there are the marked images 240 and 242 in the default image 22, the position of the marked image 240 in the default image 22 is identified (step S150b), thereby knowing whether the marked image 240 overlaps with the first target image 220 or not. When the marked image 240 overlaps with the first target image 220 and the overlapping time reaches a predetermined time (step S150c), the processing module 40 generates a confirmation signal and sends the same back to the camera module 20. After the camera module 20 detects the confirmation signal (step S150), a single capture is taken again to obtain the captured image 24 with the second target image 26 (step S160).

The intention of generating the confirmation signal after the marked image 240 and 242 overlaps with one of the first target images 220 for the predetermined time is that the predetermined mark 52 cannot accurately point to the particular target object 10 from the beginning. Taking the indicating device 50 as an infrared pen or a laser pen as an example, the user often cannot directly place the cursor on the particular target object 10 at the moment of pressing the switch, but needs to move the position of the cursor from the moment of pressing the switch onto the particular target object 10 (i.e., the selected target object 16). In an example where the finger is used as the indicating device 50, there is also a similar situation where the finger cannot point to the particular target object 10 at the moment it appears in the default image 22.

During the movement of the cursor or finger, it may pass by other target objects 10 other than the selected target object 16, and the overlapping time between the finger and the other target objects 10 during the movement will be shorter. Therefore, it can be judged whether the cursor or finger is indeed resting on the selected target object 16 at present or not by “whether overlapping lasts for a predetermined time or not” (step S150c).

In some examples, if the mark confirmation module 49 identifies that there are no marked images 240 and 242 in the default image 22 or that there are the marked images 240 and 242 in the default image 22, but the overlapping time between the marked images 240 and 242 and the first target image 220 does not exceed the predetermined time, it indicates that the predetermined mark 52 (finger or cursor) has not yet accurately indicated the selected target object 16, and there is no need to make the camera module 20 perform a single capture with the confirmation signal to obtain the captured image 24. At this time, the mark confirmation module 49 can repeatedly judge the default image 22, and generate the confirmation signal when the default image 22 has the marked images 240 and 242, and the marked images 240 and 242 overlap with the first target image 220 and overlap for a predetermined time.

Referring back to FIG. 1B, in some examples, the user generates the confirmation signal through the input component 30, and the input component 30 is a key 32. The key 32 can be implemented directly by the switch of the infrared pen or the laser pen. After the key 32 is pressed, the infrared pen or the laser pen generates a cursor, and the camera module 20 also starts capturing. When the mark confirmation module 49 identifies that the default image 22 captured by the camera module 20 has the marked image 240, and the marked image 240 overlaps with the first target image 220 and overlaps for a predetermined time, a confirmation signal is generated. The camera module 20 generates the captured image 24 with the second target image 26 after detecting the confirmation signal.

In some examples, the input component 30 implemented with the key 32 is the key 32 independent of the switch of the infrared pen or the laser pen, then when the finger does overlap with the selected target object 16, the user can press the key 32. The camera module 20 takes the single capture according to the confirmation signal generated when the user presses the key 32 to obtain the captured image 24 with the second target image 26.

Referring to FIG. 2 or FIG. 7 and FIG. 8, in some examples, after confirmation of the final selected image 480, the output module 70 extracts the strings 450 and 452 stored after decoding by the decoder 440 or text identification by the identifier 442, and the strings 450 and 452 are outputted and displayed on the display 80. Display fields 82 of the strings 450 and 452 can be set on the display 80, and the strings 450 and 452 are displayed in the display fields 82 to facilitate the user to visually obtain information represented by the target object 10.

In some examples, the design of the fields in the display 80 can be adjusted according to the user's needs. For example, on the display 80, the default image 22 and the display field 82, the captured image 24 and the display field 82 (as shown in FIG. 7 and FIG. 8), or the second target image 26, the final selected image 480 and the display field 82 can be displayed side by side in the picture of the display 80. In some examples, the display field 82 can be presented in a window. The window pops up after the processing module 40 judges the final selected image 480, and displays the strings 450 and 452 corresponding to the final selected image 480.

Referring to FIG. 6A, FIG. 7 and FIG. 8, and FIG. 10, in some other examples, when the input component 30 is implemented with the key 32, the key 32 and the camera module 20 are located together on a portable electronic device 90. In some examples, the key 32 may be any button on the portable electronic device 90 or a button on a user interface of the portable electronic device 90. The camera module 20 may be a front lens or rear lens mounted on the portable electronic device 90.

In some examples, the camera module 20, the processing module 40, the storage module 60, the output module 70, and the display 80 can also be implemented with components of the portable electronic device 90 itself. For example, the processing module 40 and the output module 70 may be implemented with the processor 92 of the portable electronic device 90 itself. An internal memory in the processor 92 or an external memory of the portable electronic device 90 may be used as the storage module 60. The display 80 is implemented with a display screen of the portable electronic device 90. The indicating device 50 can also be provided on the portable electronic device 90.

In some other examples, the camera module 20, the processing module 40, the storage module 60, the output module 70, and the display 80 can also be independent of each other. For example, the camera module 20 is an external camera that can be connected through a USB port, a Type C port or the like. The processing module 40 is the processor 92 in the electronic device. The storage module 60 is an external hard disk. The display 80 is an external screen.

In some further examples, some of the camera module 20, the processing module 40, the storage module 60, the output module 70, and the display 80 can be implemented by the components of the portable electronic device 90 itself, and some of them are independent devices independent of the portable electronic device 90.

In some examples, the portable electronic device 90 may be apparatus such as a smartphone, a laptop and a tablet computer.

In some examples, the processing module 40 can be implemented by one or more processors 92. Each processor may be, but is not limited to, a central processing unit, a system on chip (SOC), a general-purpose or special-purpose microprocessor, a digital signal processor (DSP), a programmable logic controller (PLC), an application specific integrated circuit (ASIC), a programmable logic device (PLD), other similar processing devices, or a combination thereof.

In some examples, in addition to being implemented with the screen without a touch function of the above-mentioned portable electronic device 90, the display 80 may be, but is not limited to, a touch screen or a general external display screen (i.e., it has no touch function). The plurality of default images 22 with the continuous capturing time captured by the camera module 20 in advance can be presented in real-time in the display 80. When the display 80 is a touch screen, the input component 30 may be the display 80, or may be an input device additionally provided such as a keyboard, one or more keys, a writing tablet, or any combination thereof.

In some examples, the identifier 442 identifies the strings 146 and 148 by optical character recognition (OCR). In addition, the identifier 442 may be implemented with data marking and supervised learning, a template matching method, and a method of support vector machine (SVM) or decision tree in a machine learning model.

In some examples, the comparison module 48 obtains the final selected image 480 by using a method for comparing image similarities between the second target image 26 and each first target image 220. The method for comparing the image similarities may be mean squared error (MSE), structural similarity index (SSIM), histogram comparison, and feature point comparison and the like. The other parts of the second target image 26 should be the same as one of the plurality of first target images 220 stored, except for the part of second target image 26 that overlaps with the predetermined mark 52 or finger.

In some examples, the barcode image recognition device is used for managing goods in warehouses or logistics. There are one or more barcodes 122 and 124 or strings 146 and 148 on the goods or shelves, where each of the barcodes 122 and 124 or strings 146 and 148 can carry related information about product names, specifications, origins, prices, batch numbers and the like. In some examples, these barcodes 122 and 124 or strings 146 and 148 can be provided (such as pasted or printed) on the goods or shelves. During processing, storage and transportation, the barcodes 122 and 124 are obtained by direct scanning to obtain the information carried or obtain the strings 146 and 148.

In some examples, the barcodes 122 and 124 may have a same barcode type or different barcode types. Specifically, the barcodes 122 and 124 may have a same encoding way (i.e., the same barcode type) or different encoding ways (i.e., different barcode types). In some examples, the barcodes 122 and 124 may be, but is not limited to, any one-dimensional barcode (such as Code 39 barcodes, interleaved 25 barcodes, EAN-13 barcodes, EAN-8 barcodes and Code 128 barcodes) or any two-dimensional barcode (such as QR Code barcodes, PDF 417 barcodes, composite barcodes and Data Matrix barcodes). The one-dimensional barcodes 122 and 124 may represent the string 450 of tens of characters, and the two-dimensional barcode may represent the string 450 of thousands of characters.

In some examples, the strings 146 and 148 are not limited to languages, and the strings 146 and 148 may represent the recognition code of the goods in order to identify a particular commodity. For example, global trade item number (GTIN, including UPC and EAN), international standard book number (ISBN, used for books), or other related recognition codes. The strings 146 and 148 can also represent a manufacturer or supplier code, a batch number, or a serial number, where the manufacturer or supplier code, the batch number, or the serial number can be used for tracking the production batch or serial number of the products.

In summary, in some examples, the barcode image recognition method and device can accurately and controllably output the information carried by the indicated barcodes 122 and 124 or the strings 146 and 148 that can be displayed in the display 80 after being converted when there are a plurality of barcodes 122 and 124 or strings 146 and 148.

Claims

1. A barcode image recognition method, comprising: enabling a camera module;continuously capturing, by the camera module, at least one target object in advance after the camera module is enabled to obtain a plurality of default images with continuous capturing time;identifying, by a first artificial neural network, a first target image of each target object in each default image;capturing each first target image from each default image;converting each first target image into a string;storing each first target image and the string corresponding to each first target image;triggering the camera module according to a confirmation signal to take a single capture of a predetermined mark to obtain a captured image;identifying, by a second artificial neural network, a marked image of the predetermined mark in the captured image and a second target image of a selected target object of the at least one target object that overlaps with the predetermined mark;comparing the second target image with each first target image to obtain a final selected image, the final selected image being the first target image with the highest similarity to the second target image; andreading the string corresponding to the final selected image from the stored string and outputting the string.

2. The barcode image recognition method according to claim 1, further comprising: identifying, by the first artificial neural network, a type of each first target image, wherein the type is one of a barcode image and a string image.

3. The barcode image recognition method according to claim 2, wherein the first target image is the barcode image, and the step of converting each first target image to the string comprises: decoding the first target image into the string, wherein the string is information carried by the first target image.

4. The barcode image recognition method according to claim 2, wherein the first target image is the string image, and the step of converting each first target image to the string comprises: performing text identification on the first target image to obtain the string.

5. The barcode image recognition method according to claim 1, further comprising: receiving the confirmation signal from an input component.

6. The barcode image recognition method according to claim 5, wherein the input component is a key.

7. The barcode image recognition method according to claim 6, wherein the key is located on an indicating device and the indicating device is independent of the camera module.

8. The barcode image recognition method according to claim 7, further comprising: generating, by the indicating device, the predetermined mark on one of the at least one target object.

9. The barcode image recognition method according to claim 8, wherein the indicating device is an infrared pen or a laser pen, and the predetermined mark is a cursor.

10. The barcode image recognition method according to claim 6, wherein the key and the camera module are located on a portable electronic device.

11. The barcode image recognition method according to claim 10, further comprising: generating, by the portable electronic device, the predetermined mark on one of the at least one target object, wherein the predetermined mark is a cursor.

12. The barcode image recognition method according to claim 1, further comprising: identifying the marked image of the predetermined mark in each default image to obtain a marked position of the marked image in each default image; andgenerating the confirmation signal according to the capturing time of the default images with the marked image in these default images and the marked position.

13. The barcode image recognition method according to claim 12, further comprising: pointing to, by an indicating device, the selected target object in the at least one target object to form the predetermined mark overlapping with the selected target object.

14. The barcode image recognition method according to claim 1, further comprising: providing that the predetermined mark overlaps with the selected target object in the at least one target object.

15. A barcode image recognition device, comprising: a camera module, configured to continuously capture at least one target object in advance to obtain a plurality of default images with continuous capturing time;an input component, coupled to the camera module, and configured to generate a confirmation signal to trigger the camera module to take a single capture of a predetermined mark to obtain a captured image where the predetermined mark overlaps with the at least one target object;a processing module, coupled to the camera module, comprising: a first artificial neural network, identifying a first target image of each target object in each default image;a conversion module, converting each first target image to a string;a second artificial neural network, identifying a marked image of the predetermined mark in the captured image and a second target image of a selected target object of the at least one target object that overlaps with the predetermined mark; anda comparison module, coupled to the first artificial neural network and the second artificial neural network, and comparing the second target image with each first target image to obtain a final selected image, the final selected image being the first target image with the highest similarity to the second target image;a storage module, coupled to the processing module, and storing each first target image and the string corresponding to each first target image;an output module, coupled to the comparison module and the storage module, and reading the string corresponding to the final selected image from the storage module; anda display, coupled to the output module, and displaying the string read by the output module.

16. The barcode image recognition device according to claim 15, wherein the first artificial neural network further identifies a type of each first target image, and the type is a barcode image or a string image.

17. The barcode image recognition device according to claim 16, wherein the conversion module comprises a decoder, and the decoder decodes the first target image into the string.

18. The barcode image recognition device according to claim 16, wherein the conversion module comprises an identifier, and the identifier performs text identification on the first target image to obtain the string.

19. The barcode image recognition device according to claim 15, wherein the input component is a key.

20. The barcode image recognition device according to claim 19, further comprising an indicating device, the key being located on the indicating device, and the indicating device being independent of the camera module, the indicating device generating the predetermined mark on one of the at least one target object.

21. The barcode image recognition device according to claim 20, wherein the indicating device is an infrared pen or a laser pen, and the predetermined mark is a cursor.

22. The barcode image recognition device according to claim 19, wherein the key and the camera module are located on a portable electronic device.

23. The barcode image recognition device according to claim 22, wherein the portable electronic device generates the predetermined mark on one of the at least one target object.

24. The barcode image recognition device according to claim 15, wherein the predetermined mark is a finger, a cursor generated by an infrared pen, or a cursor generated by a laser pen.

25. The barcode image recognition device according to claim 24, wherein the processing module further comprises a mark confirmation module, configured to identify the marked image of the predetermined mark in each default image to obtain a marked position of the marked image in each default image, and generate the confirmation signal according to the capturing time of the plurality of default images with the marked image in these default images and the marked position.