The present invention concerns a method for determining an optimal distribution of temperature sensors on an apparatus in order to determine the spatial and temporal thermal distribution of the temperature on a chip. The present invention concerns further a method for determining the spatial and temporal thermal distribution of the temperature on an apparatus on the basis of temperature sensors distributed on an apparatus. The present invention concerns also an apparatus with temperature sensors and a work management in dependence of the measured temperature distribution on the apparatus.
The continuous evolution of process technology enables the inclusion of multiple cores, memories and complex interconnection fabrics on a single die. Although many-core architectures potentially provide increased performance, they also suffer from increased IC power densities and thermal issues have become serious concerns in latest designs with deep submicron process technologies. In particular, it is key to design many-core designs that prevent hot spots and large on-chip temperature gradients, as both conditions severely affect system's characteristics, i.e., increasing the overall failure rate of the system, reducing performance due to an increased operating temperature, and significantly increasing leakage power consumption (due to its exponential dependence on temperature) and cooling costs.
Designers organize the floorplan to limit these thermal phenomena, for example, by placing the highest power density components closer to the heat sink. However, the workload execution patterns are fundamental to determine the transient on-chip temperature distribution in multicore designs and, unfortunately, these patterns are not fully known at design time. Furthermore, these issues are amplified in many—core designs, where thermal hot-spots are generated without a clear spatio-temporal pattern due to the dynamic task set execution nature, based on external service requests, as well as the dynamic assignment to cores by the many-core operating systems (OS).
Therefore, latest many-core designs include dynamic thermal management approaches that incorporate thermal information into the workload allocation strategy to obtain the best performance while avoiding peaks or large gradients of temperature.
The temperature map of a processor can be estimated by the solution of the direct problem, given the heat sources and the physical model of the temperature diffusion (e.g. a nonlinear diffusion equation). This approach is limited by its requirements: the knowledge of the heat sources can be ascribed to the knowledge of the detailed power consumption of the different components. This information is not usually known at runtime. Even if we can estimate this power distribution, the computation of a solution would require an excessive computational power.
Alternatively, the temperature distribution, mostly an instantaneous temperature map, of a processor can be estimated by the solution of the inverse problem, given the value of the temperature in some locations and some a-priori information about the temperature map. It is impossible to solve the inverse problem from few, spatially localized, imprecise measurements without some a-priori constraints on the temperature map, such as e.g. limited bandwidth. The performance is significantly impacted by the small number of available sensors and the structure we consider for the thermal map, i.e. the a-priori information. Nowadays, a few sensors are already deployed on chips to obtain this thermal information. However, their number is limited by area/power constraints and the optimal placement of sensors to detect all the worst-case temperature scenarios is a very complex problem that has received significant attention in recent years.
Unfortunately, the reconstruction of the entire thermal map from a limited number of thermal sensors poses many—and still unresolved—questions. In particular, for each specific many-core architecture, the two fundamental questions to answer are the possible trade-offs regarding the number of sensors to place and the reachable degree of temporal and spatial thermal precision, as well as the sensor locations to maximize the thermal map reconstruction performance.
In “Thermal monitoring of real processors: techniques for sensor allocation and full characterization” published by Nowroz, A. N., Cochran, R., And Reda, S. in DAC (2010), the optimal location of k sensors for measuring the temperatures on the many-core architecture are determined on the basis of a K-means algorithm representing the K centers of energy on the chip. The thermal map is estimated on the basis of the measurements of the sensors on the chip and using the fact that the frequency representation of the temperature map is a sparse matrix, since only low frequencies are different from zero. However, the errors of the estimated temperature map compared to the real temperature map are large and the thermal hot spots and high gradients of the temperature map cannot be determined with sufficient exactness. In addition, it is not that easy to consider the constraints of the allocation of the sensor on the chip with this allocation determining algorithm.
Therefore, it is an object of the invention to find a method and apparatus for estimating the temperature distribution of a chip or apparatus.
It is another object of the invention to find a method for allocating the temperature sensors on the chip or apparatus and a chip/apparatus with such an optimal sensor allocation.
According to the invention, these aims are achieved by a method according to claim 1 for determining the allocation of M temperature sensors on an apparatus for estimating the temperature distribution on the apparatus comprising the following step. An N-dimensional temperature vector with N temperature variables describing temperatures at N locations on the apparatus is provided. The vector space of the temperature vector is approximated by K basis vectors, whereby the allocation of the M temperature sensors is based on the K basis vectors.
According to the invention, these aims are achieved by an apparatus according to claim 10 comprising the following features. M sensors for measuring the temperature on M locations of the apparatus. An estimator configured to estimate a temperature vector of the apparatus with N temperature variables, whereby the estimator is configured to approximate the vector space of the temperature vector by K basis vectors, whereby the M temperature sensors are allocated on the apparatus on the basis of the K basis vectors.
According to the invention, these aims are achieved by a method according to claim 18 for estimating a thermal distribution of an apparatus comprising the following steps. Providing an N-dimensional temperature vector with N temperature variables describing temperatures at N locations on the apparatus. The vector space of the temperature vector is approximated by K basis vectors of a vector transformation of the standard basis. The temperature at M locations on the processor is measured. The K coefficients corresponding to the K basis vectors are estimated on the basis of the M measurements of the temperature. The temperature vector is estimated on the basis of the K estimated coefficients, whereby the basis vectors are predetermined on the basis of a plurality of realizations of the temperature vector.
The invention suggests to choose a basis system which represents the temperature map with a low number K of basis vectors. This already yields very good results for estimating the temperature map. In order to further optimize the estimation result, the points of measurement of the temperature on the apparatus is predetermined on the basis of the chosen K basis vectors. Therefore, the allocation of the measurement points on the apparatus is adapted to the method of estimating the temperature vector and the estimation result is dramatically improved.
The dependent claims refer to further advantageous embodiments of the invention.
In one embodiment, the allocation of the M temperature sensors is based on the K basis vectors which are the same as used in the apparatus to estimate the temperature distribution on the apparatus.
In one embodiment, a K×N dimensional first transformation matrix is provided whose columns are proportional to the K basis vectors, and the M locations of the M temperature sensors are selected on the basis of the condition number of a second transformation matrix resulting from removing M-N rows from the first transformation matrix, wherein the locations corresponding to the M remaining rows of the first transformation matrix correspond to the M locations of the M temperature sensors.
In one embodiment, the allocation of the M temperature sensors is based on the correlation between the K basis vectors.
In one embodiment, a correlation matrix of the K basis vectors are determined and the M-N rows with the highest non-diagonal elements are removed and the M temperature sensors are located on the apparatus on the M locations corresponding to the M remaining rows of the correlation matrix.
In one embodiment, the number M is chosen such that the correlation matrix resulting from removing the N-M rows with the highest non-diagonal element from the first transformation matrix has rank K and a minimal number of rows.
In one embodiment, the K basis vectors are determined on the basis of a plurality of realizations of the temperature vector.
In one embodiment, the K basis vectors are eigenvectors of the covariance matrix of the temperature vector.
In one embodiment, K is smaller than N and K is equal to or smaller than M.
In one embodiment, the temperature vector {circumflex over (x)} is estimated by {circumflex over (x)}=ΦK({tilde over (Φ)}*K{tilde over (Φ)}K)−1{tilde over (Φ)}*KxS, wherein ΦK is the K×N Matrix comprising the K basis vectors as columns, {tilde over (Φ)}K is the K×M Matrix comprising the K basis vectors as columns with only the M rows corresponding to the M locations on the apparatus of the measured temperature and xS is the M dimensional vector of measured temperatures.
In one embodiment, the apparatus is a chip, preferably a processor.
The invention will be better understood with the aid of the description of an embodiment given by way of example and illustrated by the figures, in which:
Before describing the methods and apparatuses according the embodiments of the invention, the model for estimating the temperature distribution of the chip 1 is presented.
In order to describe the temperature distribution of the chip 1, a discretized temperature map t is defined as shown in
In other words, the columns of the discrete thermal map are stacked to transform the matrix t into a vector x. Preferably, the natural numbers H and W are chosen such that the geometry of the surface of the chip 1 is covered by equidistant coordinates and that the existence of temperature variations between two neighbouring coordinates is excluded. However, it is understood that any coordinate system can be chosen. For example the regions prone to higher thermic stress, e.g. regions with higher temperature and/or regions with more complex and irregular temperature spreading patterns and/or regions with higher temperature gradients, could include a more dense net of coordinates than the remaining regions on the chip.
Then, the N-dimensional temperature vector is approximated by a projection onto the low-dimensional linear subspace that minimizes the mean square error. This allows to describe the N-dimensional thermal map t or the equivalent temperature vector x with only K, coefficients, where K is much smaller than N. Any vector x can be represented using a basis Φ as,
where α[n] are the coefficients of the expansion over the vector basis Φ with the N-dimensional basis vectors Φ[i]. Note that, once we define a vector basis for the data, knowing the coefficients α is equivalent to knowing the temperature map x.
It is looked for the optimal approximation subspace using a basis. Considering that we want to keep only K coefficients α out of N, we suggest that the optimal subspace is the K-dimensional one introducing the smallest error. The approximated temperature vector {circumflex over (x)} is given by the following over-determined system of equations,
where the subscript K indicates the selection of the first K columns for a matrix or the first K elements for a vector. This approximation is equivalent to a projection onto the K-dimensional subspace spanned by the columns of ΦK. The following optimization problem is defined to find this basis. Note that, the first K columns of Φ will define the optimal subspace we are looking for. Problem 1: Find the set of basis vectors Φ such that the approximation {circumflex over (x)} with the first K<N components,
minimizes the following error,
This dimensionality reduction technique is well known in other fields under different names, such as Principal Component Analysis (PCA) and Karhunen-Loeve Transform (KTL). It has an analytic solution and it requires the covariance matrix Cx, that is defined for real zero-mean random variables as
Cx[i,j]=E[x[i],x[j]]. (6)
In order to estimate this matrix, a plurality of temperature vectors x for several work load scenarios is determined. Such temperature vectors x can be retrieved either by measuring the temperature maps during use or by simulating the temperature maps on the basis of the electrical inputs in the components of the chip. The latter has the advantage that the basis can already be determined, when the chip is still in the design stage. Using the set of temperature vectors simulated or measured, the covariance matrix Cx can be estimated. The quality of the available dataset impacts the quality of the estimate Cx. This estimation is a well-studied topic and will not be discussed here. The solution to Problem 1 is given in the following proposition for optimal approximation: Consider a set of temperature vectors {x} with zero mean and covariance matrix Cx. The orthonormal basis ΦK that defines the approximation {circumflex over (x)} with the minimum error e, is formed by the first K eigenvectors of Cx ordered in decreasing values of its eigenvalues λn. Moreover, the approximation error is monotonically decreasing when increasing K as
The connection between Cx and the optimal basis has an intuitive explanation. In fact, if the temperatures at different spatial points are statistically correlated, then Cx has some elements outside its diagonal different from zero. These elements can be used to infer the temperature at points without sensors. Moreover, if the correlation is strong, then the eigenvalues λn of Cx decay fast and the temperature x can then precisely be approximated with a lower K, see (7). Recall that K is the number of parameters we have to estimate from the sensor measurements; having the approximation with the minimum K while keeping a good precision is fundamental to have a truthful reconstruction with just few sensors. Since the Eigenvectors can even be represented as maps by inverting (1), the eigenvectors of Cx are also called Eigenmaps.
The temperature vector x is now defined by only its K coefficients αK in the basis ΦK. In the following, it will be explained how to estimate the coefficients αK from the sensors measurements. In principle, the coefficients αK can be found by inverting the over-determined linear system of equations given in (3). However, this would require the knowledge of the temperature x[i] at every spatial location i. Assuming that only M sensors at are placed at locations S={j1, j2, . . . , jM}. Considering (3), it is equivalent to remove all the rows of ΦK beside those indexed by S:
where {tilde over (Φ)}K is a matrix formed by the rows of ΦK corresponding to the sensor locations S, xS is a vector containing the sensor measurements and αK is the unknown vector. Before the solution of (8) is characterized, noise needs to be introduced into the picture. More precisely, there are two different noise sources affecting the measurements. First, there is the approximation error e=x−{circumflex over (x)} that is systematic and it is due to the approximation on the K dimensional subspace. Second, the measurements are corrupted by a significant amount of noise due to many factors, such as thermal noise, quantization and calibration inaccuracies. Therefore, the following modification of (8),
x
S
+w={tilde over (Φ)}
KαK, (9)
is considered, where w is the M-dimensional noise vector. There is no exact solution to (9). However, the coefficients {circumflex over (α)}Kcan be found such that the error with respect to the measured temperature xS is minimized. Namely, the following least square problem,
is solved. If S, i.e. the location of the M sensors, is chosen such that M≧K and rank({tilde over (Φ)}K)=K, then the reconstruction of the temperature vector {tilde over (x)} is unique. In addition, the reconstruction error is bounded by the condition number κ({tilde over (Φ)}K) of {tilde over (Φ)}K and the noise energy
Consequently, given M sensors and an optimal K-dimensional subspace ΦK, the optimal sensor location is the one that minimizes the condition number of {tilde over (Φ)}K. If this condition number is minimal, the reconstruction error is minimal for the given amount of noise w. Note that increasing K will in general increase κ({tilde over (Φ)}K) and consequently will increase the reconstruction error er. Therefore, an optimal K is such that the sum of e and er is minimal. Thus, the condition number is the perfect metric to evaluate different sensing patterns and find the optimal one. The solution of problem (10) is
{tilde over (x)}Φ
K({tilde over (Φ)}*K{tilde over (Φ)}K)−1{tilde over (Φ)}*KxS. (12)
This gives a linear estimator for the temperature vector and thus for the temperature map of the chip on the basis of the M temperature measurements.
In step S1, a set of temperature maps and consequently also a set of temperature vectors, which correspond to the temperature maps by equation (1), is determined. The set of temperature maps are determined in one embodiment by simulating the temperature distribution of the chip on the basis of the known parts of the chip and their electrical inputs. Consequently, the development of the temperature map of the chip over time for constant and varying electrical inputs could be retained already at design time. In another embodiment, the set of temperature maps is by measuring the temperature distribution of the hardware-chip e.g. by a sensitive infrared camera or other measuring sensors for measuring high-resolution and highly sensible temperature distributions. In both embodiments, the set of temperature maps should be temperature maps at discrete time points for a large number of work scenarios of the chip such that the set of temperature maps is a good statistical representation of the statistical temperature vector x.
In step S2, a vector basis is determined which represents the N-dimensional vector space of the N-dimensional temperature vector x and is different from the standard basis. A good vector basis for the statistical temperature vector x is found on the basis of the set of temperature vectors determined in step S1. In one embodiment, the vector basis for the statistical temperature vector x is determined on the basis of the covariance matrix Cx of the statistical temperature vector x. This covariance matrix Cx is estimated on the basis of the set of temperature vectors determined in step S1. In one preferred embodiment, the vector basis is chosen as the N eigenvectors of the covariance matrix Cx. However, the invention is not restricted to the use of the eigenvectors. Any other vector basis such as a discrete Fourier transform, discrete cosinus transform, etc. can be used.
In step S3, the vector space of the statistical temperature vector x is approximated by only K<N basis vectors of the basis vectors chosen in step S2. The K eigenvectors with the largest eigenvalues are the optimal solution to approximate the vector space of the statistical temperature vector x with K<N coefficients and providing the lowest approximation error e. However, the invention is not restricted this selection of eigenvectors, any other selection of eigenvectors or any other selection of K basis vectors of another vector basis is also within the scope of the invention. The selection of the dimension of the approximation-dimension K will be described later.
In step S4, the K×N dimensional transformation matrix ΦK defined in equation (3) is determined on the basis of the K N-dimensional basis vectors defined in step S3. In step S5, the number M and location of the temperature sensors on the chip are determined. Preferably, the allocation of the temperature sensors is determined according to the method for determining the allocation of temperature sensors as described further below. However, the method for estimating the temperature distribution is not restricted to such allocations of the temperature sensors. The method for estimating the temperature distribution is also applicable to chips with predetermined temperature sensor allocations or other methods for determining the allocation of the temperature sensors. In step S6, the K×M dimensional matrix {tilde over (Φ)}K is provided as determined in equation (8) by the locations S of the M temperature sensors. In step S6, the estimation matrix
M=Φ
K({tilde over (Φ)}*K{tilde over (Φ)}K)−1{tilde over (Φ)}*K (13)
is calculated and stored for the online estimation method described in the following.
{tilde over (x)}=MxS (14)
such that the estimator {tilde over (x)} for the temperature vector x at the one time instance is calculated. The temperature vector estimator {tilde over (x)} can be transformed in a temperature map estimator {tilde over (t)}. The temperature map estimator can then be used for example for controlling the temperature of a chip. The steps S11 and S12 are periodically repeated in order to estimate the evolution of the temperature map estimator over the time. This evolution can be used to control the power allocation of the single components of the chip in order to prevent hot spots and high temperature gradients on the chip. This is done most simply by reducing the usage of a component whose peak temperature is over a certain temperature threshold or whose temperature gradient is over a certain threshold. In the example of a multi-core chip, the temperature information like the temperature vector estimator or the temperature map estimator would be plugged in the workload manager that allocates different jobs to different cores. Knowing the evolution of the temperature up to the last instant, it could directly avoid thermal stress scenarios by opportunely allocate the future jobs on the basis of the temperature information.
In the following, an embodiment of the method for determining the allocation of temperature sensors will be described.
Since the reconstruction error er of the estimator (12) depends on the condition number κ({tilde over (Φ)}K) of {tilde over (Φ)}K, for a given number of M temperature sensors the optimal allocation the optimal sensor location is the one that minimizes the condition number κ({tilde over (Φ)}K) of {tilde over (Φ)}K. Therefore, in one embodiment, the allocation of the temperature sensors on the chip is based on the condition number of matrix {tilde over (Φ)}K. For example, the condition number could be calculated for all M out of N combinations of allocating the M temperature sensors and the allocation with the lowest condition number could be chosen. Since the temperature map has normally a very high resolution (e.g. N=64000), the calculation of the condition number of all M out of N combinations includes very long computation times.
Even if the invention is described in the context of a chip, the invention is not restricted to a chip, but is applicable to any kind of apparatus. Such an apparatus might be any chip, any integrated circuit, any computer, any server, any data center comprising a large number of computer, server, network devices and/or storage systems. The apparatus is prefereably anything which creates heat by its electrical work. However, this invention is also applicable to mechanical or other apparatuses which create heat by their function. The apparatus might also be a house or a room comprising further heat creating devices such as server rooms.
This application claims priority of U.S. provisional application 61/569,799, the contents whereof are hereby incorporated.
Number | Date | Country | |
---|---|---|---|
61569799 | Dec 2011 | US |