Embodiments of the present invention generally relate to a technique for assisting in communication using voice and text (for sharing of recognition, conveyance of intention and other purposes), and more particularly, to a communication evaluation technique.
Communication by voice is performed, for example, with transceivers. A transceiver is a wireless device having both a transmission function and a reception function for radio waves and allowing a user to talk with a plurality of users (to perform unidirectional or bidirectional information transmission). The transceivers can find applications, for example, in construction sites, event venues, and facilities such as hotels and inns. The transceiver can also be used in radio-dispatched taxis, as another example.
It is an object of the present invention to perform evaluation of entire group utterances of a communication group and evaluation of personal (individual) utterances in the group utterance to assist in improved quality of information transmission.
In a communication system according to embodiments, a plurality of users carry their respective mobile communication terminals, and a voice of utterance of one of the users input to his mobile communication terminal is broadcast to the mobile communication terminals of the other users. The communication system includes a communication control section including a first control section configured to broadcast utterance voice data received from one of the mobile communication terminals to the other mobile communication terminals and a second control section configured to control text delivery such that the result of utterance voice recognition from voice recognition processing on the received utterance voice data is displayed on the mobile communication terminals in synchronization; and an evaluation control section configured to use the result of utterance voice recognition to perform communication evaluation. The evaluation control section includes a first evaluation section configured to evaluate a dialogue between two or more of the users based on a group dialogue index to produce group communication evaluation information, a second evaluation section configured to evaluate utterances constituting the dialogue between the two or more of the users based on a personal utterance index to produce personal utterance evaluation information, and a third evaluation section configured to use the group communication evaluation information and the personal utterance evaluation information to produce entire communication group evaluation information.
As shown in
The user terminal 500 may be a multi-functional cellular phone such as a smartphone, or a portable terminal (mobile terminal) such as a Personal Digital Assistant (PDA) or a tablet terminal. The user terminal 500 has a communication function, a computing function, and an input function, and connects to the management apparatus 100 through wireless communication over the Internet Protocol (IP) network or Mobile Communication Network to perform data communication.
A communication group is set to define the range in which the utterance voice of one of the users can be broadcast to the user terminals 500 of the other users (or the range in which a communication history, later described, can be displayed in synchronization). Each of the user terminals 500 of the relevant users (field users) is registered in the communication group.
The communication system according to Embodiment 1 assists in information transmission for sharing of information, conveyance of intention and other purposes based on the premise that the plurality of users can perform hands-free interaction with each other. Specifically, the communication system according to Embodiment 1 evaluates utterances of users performed for sharing of information or conveyance of intention based on indices including group dialogue indices and personal utterance indices, and then uses the evaluation results to evaluate the entire communication group.
The efficiency of works depends on communication quality of each user or a group of users participating in a conversation, including ways of talking, ways of questioning, and contents of answers to questioning. For example, when a precise instruction is provided, a person who should respond thereto can perform associated works smoothly. When a precise response is made to the instruction, a person who provided the instruction can find that the intention of the instruction was conveyed to the target user. Consequently, the works can be performed property.
In contrast, when the response to the instruction is slow and not clear, the intention of the instruction may not have been conveyed to the target user. This may lead to lowered work efficiency since it is necessary to reissue the instruction or issue another instruction to a user other than the target user to whom the first instruction was issued. When the content of the instruction is ambiguous or the response is unclear, work mistakes may occur due to erroneous recognition or erroneous transmission.
As described above, the communication quality in the entire communication group is an important factor in evaluating the work efficiency. In light of the foregoing, the communication system according to Embodiment 1 objectively evaluates utterance logs of the communication group based on two types of indices including the group dialogue index and the personal utterance index.
Group communication evaluation information produced based on the group dialogue index is used to evaluate the quality of a “dialogue” serving as an index of smooth conversations. Personal utterance evaluation information produced based on the personal utterance index is used to evaluate the quality of an “utterance” serving as an index of smooth information transmission.
The communication system performs the evaluation based on the group dialogue index and the evaluation based on the personal utterance index to achieve evaluation of the entire communication group in an evaluation field having axes corresponding to these two indices. Such a configuration allows objective evaluation of the work efficiency based on the relative relationship between the “dialogue” and “utterance” in the entire communication group. The evaluation of the group and the evaluation of the personal user are fed back, and in view of specific good and bad points in “dialogue” and “utterance,” the improved efficiency of the overall work can be facilitated as intended by each communication group.
The management apparatus 100 includes a control apparatus 110, a storage apparatus 120, and a communication apparatus 130. The communication apparatus 130 manages communication connection and controls data communication with the user terminals 500. The communication apparatus 130 controls broadcast to distribute utterance voice data from one of the users and text information representing the content of the utterance (text information provided through voice recognition processing on the utterance voice data) to the user terminals 500 at the same time.
The control apparatus 110 includes a user management section 111, a communication control section 112, a voice recognition section 113, a voice synthesis section 114, and an evaluation control section 115. The storage apparatus 120 includes user information 121, group information 122, communication history (communication log) information 123, a voice recognition dictionary 124, a voice synthesis dictionary 125, and communication evaluation information 126.
The voice synthesis section 114 and the voice synthesis dictionary 125 provide a voice synthesis function of receiving a character information input of text form on the user terminal 500 or a character information input of text form on an information input apparatus other than the user terminal 500 (for example, a mobile terminal or a desktop PC operated by a manager, an operator, or a supervisor), and converting the character information into voice data. However, the voice synthesis function in the communication system according to Embodiment 1 is an optional function. In other words, the communication system according to Embodiment 1 may not have the voice synthesis function. When the voice synthesis function is included, the communication control section 112 of the management apparatus 100 receives text information input on the user terminal 500, and the voice synthesis section 114 synthesizes voice data corresponding to the received text characters with the voice synthesis dictionary 125 to produce synthesized voice data. The synthesized voice data can be produced from any appropriate materials of voice data. The synthesized voice data and the received text information can be broadcast to the other user terminals 500. It should be noted that the synthesized voice data is also accumulated in the communication history and thus can be treated as a log targeted for the evaluation function.
The user terminal 500 includes a communication/talk section 510, a communication application control section 520, a microphone 530, a speaker 540, a display input section 550 such as a touch panel, and a storage section 560. The speaker 540 is actually formed of earphones or headphones (wired or wireless). A vibration apparatus 570 is an apparatus for vibrating the user terminal 500.
Group information 122 is group identification information representing separated communication groups. The communication management apparatus 100 controls transmission/reception and broadcast of information for each of the communication groups having respective communication group IDs to prevent mixed information across different communication groups. Each of the users in the user information 121 can be associated with the communication group registered in the group information 122.
The user management section 111 according to Embodiment 1 controls registration of each of the users and provides a function of setting a communication group in which first control later described (broadcast of the utterance voice data) and second control (broadcast of the text resulting from the utterance voice recognition) are performed.
Depending on the particular facility in which the communication system according to Embodiment 1 is installed, the facility can be classified into a plurality of divisions for facility management. In an example of an accommodation facility, bellpersons (porters), concierges, and housekeepers (cleaners) can be classified into different groups, and the communication environment can be established such that hotel room management is performed within each of those groups. In another viewpoint, communications may not be required for some tasks . For example, serving staff members and bellpersons (porters) do not need to directly communicate with each other, so that they can be classified into different groups. In addition, communications may not be required from geographical viewpoint. For example, when a branch office A and a branch office B are remotely located and do not need to frequently communicate with each other, they can be classified into different groups.
The communication control section 112 of the management apparatus 100 serves as control sections including a first control section and a second control section. The first control section controls broadcast of utterance voice data received from one user terminal 500 to the other user terminals 500 (group calling control). The second control section chronologically accumulates the result of utterance voice recognition from voice recognition processing on the received utterance voice data in the user-to-user communication history 123 and controls text delivery such that the communication history 123 is displayed in synchronization on all the user terminals 500 including the user terminal 500 of the user who performed the utterance.
The function provided by the first control section is broadcast of utterance voice data. The utterance voice data mainly includes voice data representing user’s utterance. When the voice synthesis function is included as described above, the synthesized voice data produced artificially from the text information input on the user terminal 500 is also broadcast by the first control section.
The function provided by the second control section is broadcast of the text resulting from the voice recognition of user’s utterance. The voices input to the user terminals 500 and the voices reproduced on the user terminals 500 are all converted to text data which is then accumulated chronologically in the communication history 123 and displayed on the user terminals 500 in synchronization. The voice recognition section 113 performs voice recognition processing with the voice recognition dictionary 124 and outputs text data as the result of utterance voice recognition. The voice recognition processing can be performed by using any of known technologies.
The communication history information 123 is log information including contents of utterance of the users, together with time information, accumulated chronologically on a text basis. Voice data corresponding to each of the texts can be stored as a voice file in a predetermined storage region, and the position of the stored voice file is recorded in the communication history 123, for example. The communication history information 123 is created and accumulated for each communication group. The result of voice quality evaluation may be accumulated to be included in the communication history information 123, or may be associated with the evaluated utterance content and accumulated in a separate storage region.
As shown in the example of
The communication evaluation according to Embodiment 1 is now described in detail. The evaluation control section 115 performs the communication evaluation using the result of utterance voice recognition and includes evaluation functions of a first evaluation section 115A, a second evaluation section 115B, and a third evaluation section 115C.
The first evaluation section 115A evaluates dialogues between users based on the group dialogue index to produce group communication evaluation information.
The second evaluation section 115B evaluates utterances constituting the dialogue between users based on the personal utterance index to produce personal utterance evaluation information.
The third evaluation section 115C uses the group communication evaluation information and the personal utterance evaluation information to produce entire communication group evaluation information. As later described, the entire communication group evaluation information is evaluation information provided by plotting the relative relationship between “dialogue” and “utterance” in the evaluation field having the horizontal axis and the vertical axis corresponding to the group communication evaluation information and the personal utterance evaluation information, respectively.
For the personal utterance evaluation, utterance sentences constituting the group of utterances (dialogue) extracted in the group dialogue evaluation are extracted as a target for evaluation.
The group dialogue index according to Embodiment 1 includes indices of the response time, presence or absence of thanks, presence or absence of confirmation, and bottom-up rate. The response time is an index for evaluating a user-to-user utterance response time (in seconds) and corresponds to a time period between the utterance of a contacting user and the utterance of a contacted user. The presence or absence of thanks is an exemplary index for evaluating the presence of a specified keyword in utterances constituting a dialogue, and is used, for example, to evaluate the presence or absence of a phrase (keyword) such as “Thank you” or “I appreciate your efforts” used by the contacting user to appreciate the response from the contacted user.
The presence or absence of confirmation is used to evaluate the presence or absence of a confirmative response from the contacted user to a message from the contacting user. The confirmative response is a repetition of the message, for example. The bottom-up rate relates to an extracted spontaneous action of any user. For example, what is performed in response to a message, that is, in accordance with an instruction, is a passive action, and what is performed spontaneously without any instruction is a spontaneous action. An exemplary utterance sentence for evaluating the bottom-up rate is an utterance sentence in a report of work completion, and an utterance sentence including “I have done --” or “I have done -- first” can be extracted. The management apparatus 100 can check that a predetermined number of utterance sentences before the extracted utterance sentence include no utterance sentence corresponding to a message or instruction from the contacting user. This can extract the spontaneous action in distinction from any passive action in response to a message or instruction from the contacting user.
The personal utterance index include indices of the presence or absence of a proper noun, message redundancy (the length of instruction conversation, presence or absence of a filler), and the presence or absence of a demonstrative pronoun. The proper noun corresponds to the user’s first name or last name. The message redundancy is divided into indices of the length of an instruction conversation and the presence or absence of a filler. The length of an instruction conversation is used to evaluate a plurality of messages (instruction sentences) included in a single sentence or many characters included per utterance. The presence or absence of a filler is used to evaluate any filler included in utterances such as “say,” and “well.” The demonstrative pronoun is a word indicating an item, place, or direction, and is used to evaluate an ambiguous word included in utterances such as “this,” “there,” “that,” and “over there.”
Set values including threshold values in the respective indices such as the number of included demonstrative pronouns can be set as appropriate, and this applies to the group dialogue indices. While the evaluation values are calculated as the rates by way of example, the evaluation values may be calculated as scores. For example, a score can be added when any condition for the indices is satisfied and a score can be reduced when any condition is not satisfied, or a score can be added only when any condition is satisfied, or a score can be reduced only when any condition is not satisfied.
The “response time” of the group dialogue index has a set value of 30 seconds. In an example on the left side, cleaner B responds to an utterance of instruction from leader A in 6 seconds, so that a high (good) evaluation is provided. In an example on the right side, the cleaner B responds in 33 seconds, so that a low (bad) evaluation is provided. As the response time to the message is shorter, the communication efficiency is higher and the work efficiency is more improved.
The “presence or absence of thanks” of the group dialogue index has an evaluation condition whether any phrase or keyword expressing thanks is included. In an example on the left side, the leader A utters the phrase “Thank you” to the response (utterance for report) from the cleaner B, so that a high evaluation is provided. In an example on the right side, the leader A utters no phrase or keyword expressing thanks to the response from the cleaner B, so that a low evaluation is provided. Expressing thanks to the user action conveys the feeling of appreciation to motivate the user to work.
The “presence or absence of confirmation” of the group dialogue index is used to evaluate whether the utterance content of the cleaner B in response to the instruction from the leader A includes a repetition of the instruction word (phrase or keyword relating to the instruction content) from the leader A. In an example on the left side, in response to the instruction utterance “please clean room 201” of the leader A, the cleaner B says “room 201, I understand” by repeating part of the instruction utterance of the leader A, “room 201,” so that a high evaluation is provided. In an example on the right side, the cleaner B only says “I understand,” and any instruction word from the leader A is not included, so that a low evaluation is provided. For smooth communication, mutual understanding of transmitted information is essential. When the contacting user can confirm that the intended content is accurately conveyed to the contacted user, this can save the contacting user the effort of making the same contact for double check. In addition, the contacted user can understand the instruction content more clearly by repeating the instruction word. In this manner, the accuracy of mutual understanding of transmitted information can be improved.
The “bottom-up rate” of the group dialogue index is used to evaluate a spontaneous action of any user. In an example on the left side, the cleaner B reports (performs utterance expressing) his spontaneous action, so that a high evaluation is provided. In an example on the right side, the cleaner B takes a passive action in response to the instruction utterance of the leader A, so that a low evaluation is provided. The “bottom-up rate” in Embodiment 1 can include not only utterances relating to the spontaneous action of the user described above, that is, the action on user’s own judgement, but also utterances relating to a spontaneous proposal or suggestion of any user, or an action at the initiative of any user, such as the utterance “it’s about time to get busy, and I’ll do inspection and replenishment works.” or “I’m free now and I’ll go help with cleaning work. Is that all right?”
The evaluation based on the bottom-up rate is an important factor in terms of work efficiency. The users can think and act on their own to improve the work efficiency.
The “presence or absence of a requested user” of the personal utterance index is used to evaluate whether a user performs utterance specifying a contacted user. In an example on the left side, the utterance includes the name of the cleaner B to be requested, so that a high evaluation is provided. In an example on the right side, the utterance includes no name of a requested user, so that a low evaluation is provided. In some cases, the contacted user may be specified, or the contacting user may wish to make contact with the entire communication group without specifying any contacted user. In the latter case, the same message is transmitted to the users of the group, and confusion may arise among the users as to which one of them should respond and take actions. To address this, the utterance specifying the contacted user is evaluated to improve the work efficiency.
The “length of an instruction conversation” of the personal utterance index is used to evaluate the redundancy of utterance content. In an example on the left side, the leader A produces simple and short utterances (messages) for respective instructions with a few characters per utterance and a few instruction contents per utterance, so that a high evaluation is provided. In an example on the right side, the leader A produces a lengthy utterance with long sentences including a plurality of instructions, so that a low evaluation is provided. When the lengthy utterance including a plurality of instructions is produced, the instruction contents are not clearly separated from each other and the accuracy of information transmission is reduced. Thus, simpler and shorter messages from the contacting user can provide higher accuracy of information transmission to improve the work efficiency.
From the same viewpoint, the “presence or absence of a filler” is used to evaluate redundancy based on any filler such as “say,” or “well,” included in utterances. As shown in
The “demonstrative pronoun” of the personal utterance index is used to evaluate any ambiguous word such as “this,” “there,” “that,” and “over there” included in utterances. In an example on the left side, the utterance clearly specifies the name of a user to be requested, the place (the elevator hall on the second floor), and the purpose (take the flower-patterned vase to the warehouse), so that a high evaluation is provided. In an example on the right side, the place and the purpose are indicated by demonstrative pronouns, so that a low evaluation is provided. More demonstrative pronouns may be used in messages depending on the degree of communication skill, although the accuracy of information transmission is reduced when the messages do not clearly express who should perform what work, where. Thus, the work efficiency is improved by utterances including few demonstrative pronouns and clearly specifying the user to be instructed, and the place and purpose of the work requested of the user.
In Embodiment 1, the entire communication group evaluation information is provided by using the result of evaluation based on the group dialogue indices (group communication evaluation information) and the result of evaluation based on the personal utterance indices (personal utterance evaluation information).
The communication system can individually provide the result of evaluation based on the group dialogue indices and the result of evaluation based on the personal utterance indices. Only the result of evaluation based on the group dialogue indices cannot evaluate each user of the group, and only the result of evaluation based on the personal utterance indices cannot reveal the actual state of the entire group. To address this, in Embodiment 1, the evaluation field represented on the vertical axis and horizontal axis is produced, and the group communication evaluation information and the personal utterance evaluation information are associated with the two axes. The result of evaluation based on the group dialogue indices and the result of evaluation based on the personal utterance indices are mapped as parameters in the evaluation field to produce the entire communication group evaluation information. The size of circles corresponds to the amount of utterance (number of utterance sentences to be evaluated), with a bigger circle indicating a larger amount of utterance.
With reference to the entire communication group evaluation in
In the example of
In an example of
As described above, the evaluation control section 115 can have the first weight value setting function of setting the weight values (first weight values) to the group dialogue indices for producing the group communication evaluation information and the second weight value setting function of setting the weight values (second weight values) to the personal utterance indices for producing the personal utterance evaluation information.
The storage apparatus 120 can hold the weight value setting information for each communication group. The first evaluation section 115A can produce the group communication evaluation information with the weight values applying thereto, and the second evaluation section 115B can produce the personal utterance evaluation information with the weight values applying thereto. For example, the sections 115A and 115B can apply the set weight values (coefficients) to the evaluation values of the indices shown in
The time of text delivery of the result of voice recognition and the time of text delivery of the evaluation comment associated with the result of evaluation based on each index can be set as appropriate in Embodiment 1. For example, the evaluation comment can be delivered together with the text delivery of the result of voice recognition (processing in the second control section), or the evaluation comment can be delivered at a time after the text delivery of the result of voice recognition, or the evaluation comment can be received at any time during works or after the completion of works in response to a request for displaying the evaluation comment from the user terminal 500.
As described above, the communication system according to Embodiment 1 produces the evaluation information based on the group dialogue indices and the personal utterance indices and provides the produced evaluation information as the evaluation result for each communication group. For the processing of feedback to the user described above, a Weak Point may also be fed back.
Specifically, the evaluation comment shown in
Similarly, the evaluation control section 115 can use the result of a comparison between the evaluation information (personal utterance evaluation information) based on the personal utterance indices in
When a comparison with the predetermined threshold shows that the result is greater than the threshold value, the evaluation control section 115 can produce and provide the group characteristic information in the form of an evaluation comment including a strong point, “You generally tend to have good communication in a short time to response. Keep up the speedy response.”
Each of the users starts the communication application control section 520 on his user terminal 500, and the communication application control section 520 performs processing for connection to the management apparatus 100. Each user enters his user ID and password on a predetermined log-in screen to log in to the management apparatus 100. The log-in authentication processing is performed by the user management section 111. At the second and subsequent log-ins, the input operation of the user ID and password can be omitted since the started communication application control section 520 can automatically perform log-in processing with the user ID and password input by the user at the first log-in.
After the log-in, the management apparatus 100 automatically performs processing of establishing a communication channel in a group calling mode with each of the user terminals 500 to open a group calling channel centered around the management apparatus 100.
After the log-in, each user terminal 500 performs processing of acquiring information from the management apparatus 100 at any time or at predetermined intervals.
When a user A performs utterance, the communication application control section 520 collects the voice of that utterance and transmits the utterance voice data to the management apparatus 100 (S501a). The voice recognition section 113 of the management apparatus 100 performs voice recognition processing on the received utterance voice data (S101) and outputs the result of voice recognition of the utterance content. The communication control section 112 stores the result of voice recognition in the communication history 123 and stores the utterance voice data in the storage section 120 (S102).
The communication control section 112 broadcasts the utterance voice data of the user A, who performed the utterance, to the user terminals 500 of the users other than the user A. The communication control section 112 also transmits the utterance content (in text form) of the user A stored in the communication history 123 to the user terminals 500 of the users within the communication group including the user A for display synchronization (S103).
The communication application control sections 520 of the user terminals 500 of the users other than the user A perform automatic reproduction processing on the received utterance voice data to output the reproduced utterance voice (S502b, S502c). The user terminals 500 of all the users including the user A display the utterance content of text form corresponding to the output reproduced utterance voice in the display field D (S502a, S503b, S503c).
The management apparatus 100 performs communication evaluation processing (S104). As described above, the evaluation processing is performed at any time. The evaluation control section 115 refers to the communication history information 123 to extract groups of utterances of each communication group performed in predetermined time periods such as in days or months. The evaluation control section 115 produces group communication evaluation information from the extracted groups of utterances based on the group utterance indices (S105). The evaluation control section 115 also procures personal utterance evaluation information from the individual utterances of the same groups of utterances based on the personal utterance indices (S106). The evaluation control section 115 uses the produced group communication evaluation information and personal utterance evaluation information to produce entire communication group evaluation information illustrated in
To use the weight values described above, they are applied in the processing of steps S105 and S106. The evaluation comment or the evaluation comment including the weak point in the example of
The user makes a request for evaluation information on the user terminal 500 (S503a), and the management apparatus 100 provides the evaluation information (S108) in separate processing from the delivery of the utterance voice and the result of voice recognition in the group calling (such that the evaluation information is not included in the delivered text of the result of voice recognition).
In an example of
Specifically, as shown in
The evaluation control section 115 performs communication evaluation processing on the result of voice recognition of the received utterance voice data (S104), produces group communication evaluation information based on the group dialogue indices (S105), and produces personal utterance evaluation information based on the personal utterance indices (S106). The evaluation control section 115 produces an evaluation comment based on the produced evaluation information (S1071).
Step S1031 shows processing including the broadcast of the utterance voice data and the text delivery of the result of voice recognition. As described above, the text delivery is performed such that the result of voice recognition is transmitted with the real-time evaluation comment produced at step S1071 added thereto. In addition, the delivery of the evaluation comment can be notification processing, for example, in cooperation with the vibration apparatus 570 of the user terminal 500 of the user who performed the utterance.
In the example of
For example, vibration control values can be preset in association with evaluation comments such that different vibration patterns (vibrating patterns) can correspond to different contents of evaluation comments as appropriate. This allows the notification using the different vibration patterns depending on the evaluation content, thereby achieving the environment in which real-time feedback is provided for the user who performed the utterance.
Embodiment 1 of the present invention has been described. The functions of the communication management apparatus 100 and the user apparatus 500 can be implemented by a program. A computer program previously provided for implementing the functions can be stored on an auxiliary storage apparatus, the program stored on the auxiliary storage apparatus can be read by a control section such as a CPU to a main storage apparatus, and the program read to the main storage apparatus can be executed by the control section to perform the functions.
The program may be recorded on a computer readable recording medium and provided for the computer. Examples of the computer readable recording medium include optical disks such as a CD-ROM, phase-change optical disks such as a DVD-ROM, magneto-optical disks such as a Magnet-Optical (MO) disk and Mini Disk (MD), magnetic disks such as a floppy disk® and removable hard disk, and memory cards such as a compact flash®, smart media, SD memory card, and memory stick. Hardware apparatuses such as an integrated circuit (such as an IC chip) designed and configured specifically for the purpose of the present invention are included in the recording medium.
While an exemplary embodiment of the present invention has been described above, the embodiment is only illustrative and is not intended to limit the scope of the present invention. The novel embodiment can be implemented in other forms, and various omissions, substitutions, and modifications can be made thereto without departing from the spirit or scope of the present invention. These embodiment and variations are encompassed within the spirit or scope of the present invention and within the invention set forth in the claims and the equivalents thereof.
100
110
111
112
113
114
115
115A
115B
115C
120
121
122
123
124
125
126
130
500
510
520
530
540
550
560
570
Number | Date | Country | Kind |
---|---|---|---|
2020-126410 | Jul 2020 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2021/026571 | 7/15/2021 | WO |