Patent Application
20220188110
Machine learning (ML) systems are generally computationally intensive and generally perform large amounts of floating point (FP) operations. FP arithmetic operators for the FP operations are generally compliant with IEEE-754 standard. FP hardware exceptions are generated when input to and/or output from a FP arithmetic operator is one of positive infinity, negative infinity, a signaling not-a-number (SNAN), etc. Large amounts of resources are typically needed for handling large numbers of FP hardware exceptions being generated from large numbers of FP operations in the ML systems. Moreover, additional resources are needed to handle circumstances where the input to or output from the FP arithmetic operators is a denormal number or when the input is a quiet not-a-number (QNAN), SNAN, infinity, etc. Denormal number refers to a non-zero number in floating point arithmetic where its magnitude is smaller than the smallest normal number
Currently, additional data paths are often needed for each FP arithmetic operator to handle the values of inputs or outputs with denormal numbers, QNANs, SNANs, infinities, etc. These additional data paths result in a larger footprint, higher power consumption, and increase in complexity of the ML systems.
The foregoing examples of the related art and limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent upon a reading of the specification and a study of the drawings.
Aspects of the present disclosure are best understood from the following detailed description when read with the accompanying figures. It is noted that, in accordance with the standard practice in the industry, various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.
The following disclosure provides many different embodiments, or examples, for implementing different features of the subject matter. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. In addition, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
Before various embodiments are described in greater detail, it should be understood that the embodiments are not limiting, as elements in such embodiments may vary. It should likewise be understood that a particular embodiment described and/or illustrated herein has elements which may be readily separated from the particular embodiment and optionally combined with any of several other embodiments or substituted for elements in any of several other embodiments described herein. It should also be understood that the terminology used herein is for the purpose of describing the certain concepts, and the terminology is not intended to be limiting. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood in the art to which the embodiments pertain.
According to some embodiments, input data to a FP arithmetic operator of a ML system is modified in order to avoid generating one or more FP hardware exceptions. For a non-limiting example, value of the input data (i.e. operand) for the FP arithmetic operator is replaced with a maximum supported number or a minimum supported number of the system when the input data is a positive infinity or negative infinity respectively. Moreover, when an input data for a FP arithmetic operator is an SNAN, the input data may be replaced with a zero. As such, the input data as modified would not generate a FP hardware exception resulting from its original value, e.g., positive infinity, negative infinity, an SNAN, etc. Furthermore, the input data may be manipulated to handle other circumstances such as denormal numbers, QNAN inputs, etc., which may not cause a FP hardware exception but nonetheless may require additional data path for each FP arithmetic operator to handle the circumstances. In some non-limiting examples, an input data being a denormal number or being a QNAN input may be replaced with zero. Accordingly, the need for additional data paths for each FP arithmetic operator to handle the FP hardware exception or to handle denormal numbers or a QNAN input is eliminated.
In some embodiments, the output of the FP arithmetic operator, e.g., addition, subtraction, add-reduce, multiplication, negation, maximum, minimum, max-reduce, min-reduce, division, FPx to FPy (where x>y), FPx to FPy (where x<y), FP to integer (Int), etc., is similarly monitored and replaced in order to avoid additional data paths for each FP arithmetic operator. For a non-limiting example, even if the input data into the FP arithmetic operator may not cause a FP hardware exception or a special circumstance handling, the output may nonetheless require special handling. As an illustrative example, two input operands may each be within the supported numerical range but when added together may be greater than the maximum supported number, thereby generating infinity and requiring a special handling or generating a FP hardware exception. Accordingly, the output of a FP operator may be replaced with a maximum supported number if the FP operator results in positive infinity. Similarly, the output of a FP operator may be replaced with a minimum supported number if the FP operator results in negative infinity.
It is appreciated that in some nonlimiting examples the number of FP hardware exceptions being generated is reduced. In some nonlimiting examples, the output of a FP operator may be replaced with zero if the output is a denormal number. It is appreciated that the discussion of the operations with respect to addition is merely for illustration purposes and should not be construed as limiting the scope of the embodiments. For example, similar process may take place for other operations, such as subtraction, add-reduce, multiplication, negation, maximum, minimum, max-reduce, min-reduce, division, FPx to FPy (where x>y), FPx to FPy (where x<y), FP to Int, etc.
Accordingly, the need for additional data paths for each FP arithmetic operator to handle the FP hardware exceptions or to handle denormal, infinity, SNAN or QNAN input is eliminated. Thus, hardware footprint, power consumption, and complexity are reduced.
In some embodiments, the logic engine 120 may transmit the results of its determination 122 to the convertor engine 130. For a non-limiting example, the logic engine 120 may transmit whether the input data 112 would generate a FP hardware exception or whether the input data 112 would require a special handling once it is operated on by the FP arithmetic operator. It is appreciated that in some nonlimiting embodiments, the determination 122 may further include the input data 112. However, it is appreciated that the determination 122 including the input data 112 is for illustrative purposes and the convertor engine 130 may independently receive the input data 112, e.g., from the memory 110. The convertor engine 130 in response to the determination by the logic engine 120 may change the value or content of the input data 112. In some non-limiting example, the input data 112 is changed to a maximum supported number or a minimum supported number of the system when the input data 112 is a positive infinity or negative infinity respectively. Moreover, when an input data 112 is an SNAN the input data may be replaced with a zero. As such, the input data 112 as modified by the convertor engine 130 would not generate a FP hardware exception resulting from its original value, e.g., positive infinity, negative infinity, an SNAN, etc. Furthermore, the input data 112 may be manipulated to handle other circumstances such as denormal numbers, QNAN inputs, etc., that may not generate a FP hardware exception but nonetheless may require additional data paths for each FP arithmetic operator to handle the circumstances. In some nonlimiting examples, the convertor engine 130 replaces the input data 112 with zero if the input data 112 is a denormal number or is a QNAN. It is appreciated that the convertor engine 130 may leave the input data 112 unaltered if the input data 112 is neither a QNAN nor if it is an SNAN, positive infinity, negative infinity, or a denormal number.
In some embodiments, the convertor engine 130 outputs data 132 to the ALU 140. It is appreciated that the data 132 may be the same as the input data 112 if it is unaltered or it may be an altered version of the input data 112, as altered by the convertor engine 130. In some embodiments, the ALU 140 is configured to perform a FP arithmetic operation on the received data 132. It is appreciated that no FP hardware exception is generated resulting from the input data 112 being positive infinity, negative infinity, and an SNAN and that no special handling is needed for input data 112 being either a QNAN or a denormal number because the input data 112 is changed to avoid generation of the FP hardware exception or the need for special handling. However, even though the input to the ALU 140 may be a valid number, the output may nonetheless generate an exception or need special handling. For a non-limiting example, two valid numbers may generate a denormal number when added to one another or result in positive or negative infinity when added to one another. Thus, the output of the ALU 140 is monitored for denormal numbers or positive or negative infinity. The output 142 of the ALU 140 is input to the logic engine 120 that is configured to determine whether the output 142 is a denormal number or whether it is positive or negative infinity. If the logic engine 120 determines that the output 142 is neither a denormal number nor it is positive or negative infinity, then the logic engine 120 outputs the content 124 without a need to change its value. In other words, the content 124 has the same value as the output 142 from the ALU 140. On the other hand, if the logic engine 120 determines that the content 142 is either a denormal number or positive infinity or negative infinity then the content 142 is transmitted as content 126 to the convertor engine 130 in order for the content to be modified. For example, if the content 126 is a denormal number then the convertor engine 130 changes the value to zero and outputs the changed value as output 134. In contrast, if the content 126 is positive or negative infinity then the convertor engine 130 changes the value to the maximum supported number or minimum supported number by the system and outputs it as content 134. It is appreciated that the process is repeated for each input data (i.e. operand) for a FP arithmetic operator and its output. It is further appreciated that the content 126 is passed from the logic engine 120 to the convertor engine 130 for illustrative purposes and the embodiments should not be construed as limiting the scope. For example, in some embodiments the convertor engine 130 may receive the content 126 directly from the logic engine 120. In one nonlimiting example, the convertor engine 130 may receive the data directly from the ALU 140.
For example, if the logic engine 120 determines that the content 142 is either a denormal number or positive infinity or negative infinity, then the ALU 140 may be signaled to send the content 142 from the ALU 140 to the convertor engine 130 in order for the content to be modified. It is appreciated that the process is repeated for each input data (i.e. operand) for a FP arithmetic operator and its output. It is appreciated that in some nonlimiting embodiments, the logic engine 120 and the convertor engine 130 may be integrated within a same processing block. It is further appreciated that the communication between the integrated processing block and the ALU 140 may be a bidirectional communication. Moreover, it is appreciated that in some nonlimiting embodiments, the logic engine 120, the convertor engine 130 and the ALU 140 may be integrated within a same processing block, thereby eliminating the need for data communication between different engine blocks.
It is appreciated that since the input data and output data is changed to avoid FP hardware exception generation or requiring special handling, the amount of required resources, power consumption, and the complexity are reduced.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., addition, subtraction, or add-reduce, along with changes thereof to the input, and its output, as described above is shown below.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., negation, as described above is shown below.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., multiplication, as described above is shown below.
It is appreciated that the multiplication operation may have a 32-bit input/output but the operation may be performed as a 16-bit floating point. In other words, a 32-bit floating point data is converted into a 16-bit floating point number and its value may be clipped to a maximum, minimum or zero. The conversion may cause an out-of-bound exception that is handled according to the embodiments, as described above.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., maximum, minimum, max-reduce, min-reduce, as described above is shown below.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., division, as described above is shown below.
It is appreciated that the division operation may have a 32-bit input/output similar to multiplication, as described above. In other words, the operation may be performed as a 16-bit floating point. In other words, a 32-bit floating point data is converted into a 16-bit floating point number and its value may be clipped to a maximum, minimum or zero. The conversion may cause an out-of-bound exception or a divide by zero exception that is handled according to the embodiments, as described above.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., FPx to FPy (where x>y), as described above is shown below.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., FPx to FPy (where x<y), as described above is shown below.
For illustrative purposes that should not be construed as limiting the scope of the embodiments, various input data for a FP arithmetic operator, e.g., FPx to Int, as described above is shown below.
It is appreciated that the floating point to integer operation may have a 16-bit input and as such it may not need a 32-bit to 16-bit conversion. In other words, the operation may be performed as a 16-bit floating point and it may be converted to integer, e.g., int9 (as described and as incorporated by reference in its entirety in patent application number <TBD>, filed on <TBD>, entitled “System and Method for INT9 Quantization). The value may be clipped to integer maximum or minimum and it may trigger out-of-bounds exception that is handled according to the embodiments, as described above.
At step 450, it is determined whether the output result of the FP arithmetic operation generates a FP hardware exception, before a FP hardware exception is generated. For example, if the output result of the FP arithmetic operator is a positive infinity, a negative infinity, etc., then it is determined that the output result would generate a FP hardware exception, as described in
It is appreciated that at step 470, a flag is optionally generated when the input data is determined to generate a FP hardware exception (prior to generating a FP hardware exception), or when the input data would require a special handling (i.e. QNAN, denormal number, etc.), or when the output result of the FP arithmetic operator would generate a FP hardware exception (prior to the FP hardware exception being generated) or if the output result would require a special handing (i.e. output result is a denormal number), etc. The generated flag may be an out-of-bounds flag if the value is positive infinity, negative infinity, a denormal number, etc. The generated flag may be an un-initialized flag when the data is a QNAN or an SNAN. The generated flag may be a divide-by-zero flag when the dividend of a division operation is non-zero and divisor is zero.
It is appreciated that at step 570, a flag is optionally generated when the first or the second input data is determined to generate a FP hardware exception (prior to generating a FP hardware exception), or when the first or the second input data would require a special handling (i.e. QNAN, denormal number, etc.), or when the output result of the FP arithmetic operator would generate a FP hardware exception (prior to the FP hardware exception being generated) or if the output result would require a special handing (i.e. output result is a denormal number), etc. The generated flag may be an out-of-bounds flag if the value is positive infinity, negative infinity, a denormal number, etc. The generated flag may be an un-initialized flag when the input data is QNAN or SNAN. The generated flag may be a divide-by-zero flag when the dividend of a division operation is non-zero and divisor is zero.
Referring now to
According to some examples, computer system 1100 performs specific operations in which processor 1104 executes one or more sequences of one or more instructions stored in system memory 1106. Such instructions can be read into system memory 1106 from another computer readable medium, such as storage device 1108 or disk drive 1110. In some examples, hard-wired circuitry can be used in place of or in combination with software instructions for implementation. In the example shown, system memory 1106 includes modules of executable instructions for implementing an operation system (“O/S”) 1132, an application 1136 (e.g., a host, server, web services-based, distributed (i.e., enterprise) application programming interface (“API”), program, procedure or others). Further, application 1136 includes a logic engine 1138 that determines whether the input data would generate a FP hardware exception if operated on by the FP arithmetic operator or if the input data requires a special handling (i.e. denormal number, QNAN, etc.), as described above in
The term “computer readable medium” refers, at least in one embodiment, to any medium that participates in providing instructions to processor 1104 for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as disk drive 1110. Volatile media includes dynamic memory, such as system memory 1106. Transmission media includes coaxial cables, copper wire, and fiber optics, including wires that comprise bus 1102. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
Common forms of computer readable media include, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, electromagnetic waveforms, or any other medium from which a computer can read.
In some examples, execution of the sequences of instructions can be performed by a single computer system 1100. According to some examples, two or more computer systems 1100 coupled by communication link 1120 (e.g., LAN, PSTN, or wireless network) can perform the sequence of instructions in coordination with one another. Computer system 1100 can transmit and receive messages, data, and instructions, including program code (i.e., application code) through communication link 1120 and communication interface 1112. Received program code can be executed by processor 1104 as it is received, and/or stored in disk drive 1110, or other non-volatile storage for later execution. In one embodiment, system 1100 is implemented as a hand-held device. But in other embodiments, system 1100 can be implemented as a personal computer (i.e., a desktop computer) or any other computing device. In at least one embodiment, any of the above-described delivery systems can be implemented as a single system 1100 or can implemented in a distributed architecture including multiple systems 1100.
In other examples, the systems, as described above, can be implemented from a personal computer, a computing device, a mobile device, a mobile telephone, a facsimile device, a personal digital assistant (“PDA”) or other electronic device.
In at least some of the embodiments, the structures and/or functions of any of the above-described interfaces and panels can be implemented in software, hardware, firmware, circuitry, or a combination thereof. Note that the structures and constituent elements shown throughout, as well as their functionality, can be aggregated with one or more other structures or elements.
Alternatively, the elements and their functionality can be subdivided into constituent sub-elements, if any. As software, the above-described techniques can be implemented using various types of programming or formatting languages, frameworks, syntax, applications, protocols, objects, or techniques, including C, Objective C, C++, C#, Flex™, Fireworks®, Java™, Javascript™, AJAX, COBOL, Fortran, ADA, XML, HTML, DHTML, XHTML, HTTP, XMPP, and others. These can be varied and are not limited to the examples or descriptions provided.
The foregoing description of various embodiments of the claimed subject matter has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the claimed subject matter to the precise forms disclosed. Many modifications and variations will be apparent to the practitioner skilled in the art. Embodiments were chosen and described in order to best describe the principles of the invention and its practical application, thereby enabling others skilled in the relevant art to understand the claimed subject matter, the various embodiments and the various modifications that are suited to the particular use contemplated.
The present application is a Continuation Application that claims the benefit and priority to the Nonprovisional U.S. application Ser. No. 16/864,069 that was filed on Apr. 30, 2020, which claims the benefit and priority to the Provisional U.S. Application No. 62/950,626 that was filed on Dec. 19, 2019, which are incorporated herein by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62950626 | Dec 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16864069 | Apr 2020 | US |
| Child | 17686689 | US |
