Patent Application
20170213148
Machine learning is increasingly being used to understand and characterize large datasets. Through various approaches, machine learning typically enables relationships to be understood by analyzing known training data and formulating models or equations consistent with the training data. These models can then be applied to analyze new data.
Despite recent increases in processing power, memory, and other computing capabilities, machine learning is typically resource intensive and slow. Various approaches have been attempted to compensate for these deficiencies, including using multiple computers processing in parallel to speed up training. Parallel processing, however, is difficult to perform accurately for inherently sequential machine learning approaches.
Examples described herein relate to machine learning systems configured to perform parallelized stochastic gradient descent (SGD). An initial data portion analyzer can be configured to perform, using a first processor, SGD analysis on an initial portion of a training dataset. The SGD analysis includes initializing values for a set of output model weights for the initial portion of the training dataset. A local model builder can be configured to perform, using a second processor, symbolic SGD analysis on an additional portion of the training dataset. The symbolic SGD analysis includes using a symbolic representation as an initial state of a set of output model weights for the additional portion of the training dataset. The initial data portion analyzer and the local model builder are configured to perform the SGD analysis and symbolic SGD analysis in parallel. A global model builder can be configured to integrate an output of the local model builder and an output of the initial data portion analyzer into a global model that reflects both the initial portion and the additional portion of the training dataset.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
The foregoing and other objects, features, and advantages of the claimed subject matter will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.
Using the systems, methods, and computer-readable media described herein, machine learning can be performed using a parallelized stochastic gradient descent (SGD) analysis. SGD is a machine learning approach with favorable convergence properties that also scales to large data. However, when training data is stored on disk, which is typically true for large datasets, training is limited by the speed with which data can be read. Because of the sequential nature of SGD analysis, parallel computing approaches that might be applicable to other machine learning approaches are typically not available to increase speed in SGD analysis.
The examples described herein use a symbolic approach to SGD analysis that maintains the accuracy of the inherently sequential SGD while allowing the data to be distributed to multiple machines (or multiple processors or processor cores) for parallel processing. Training data can be divided into multiple portions, and individual machines or processors can process a data portion in parallel. In a conventional SGD analysis, an initial state of output model weights is known for each data portion because the data is being processed sequentially, and the results for the previous data portion are thus available. In a parallel processing scenario, however, results for the previous portion are not available. The described examples allow parallel SGD analysis by using a symbolic representation of an initial state of the output model weights for the different data portions. The symbolic representation allows a local result for the individual data portions to be determined without knowing actual concrete initial states. The local results can subsequently be combined into a global result.
The parallel SGD analysis of the described examples reduces the time necessary to perform machine learning on a given training dataset, providing faster, accurate computation of results. Examples are described below with respect to
SGD analysis is a type of gradient descent (GD) analysis that can be used to solve “minimization problems.” Many problems in statistical estimation and regression can be cast as risk minimization problems. To identify trends and associations in data, a “best” fitting curve from a family of curves is sought to filter the noise from signals. As used herein, “best” refers to maximizing a likelihood. Consider, for example, ordinary least squares (OLS) regression. OLS regression amounts to finding the line, or more generally, the plane, with minimal total Euclidean distance to the data points. In other words, the plane that minimizes the distance between the data points and their orthogonal projections onto the plane is sought. Formally, a regression vector w is sought that minimizes the “loss” function
f({right arrow over (o)})=½∥{right arrow over (y)}−X{right arrow over (w)})∥2. (1)
where {right arrow over (y)} is a vector response variables and X is the data matrix (n examples in the k-feature space).
To solve this kind of minimization problem, one way is to use iterative gradient methods: at each step, the gradient of the loss function based on all date points is computed, and the parametric vector w is moved along the direction of the gradient. SGD typically has lower computational complexity and faster convergence speed than GD. Compared to GD, SGD uses one point to compute a “local gradient,” and the parametric vector is moved along that direction. SGD often converges within just one iteration over all training examples. Batch GD, in comparison, will have only completed one iteration, with more iterations to come. SGD does not require movement in the best direction every time, but instead can move any potentially good direction in a much faster manner. As discussed above, however, SGD is inherently sequential and hence difficult to scale with the growing number of the processors in modern computing systems. Compared to GD, it eliminates the averaging step in computing the gradient descent from all points.
A local model builder 112 is configured to perform, using second processor 108, symbolic SGD analysis on an additional portion of the training dataset. The symbolic SGD analysis comprises using a symbolic representation as an initial state of a set of output model weights for the additional portion of the training dataset. Initial data portion analyzer 110 and local model builder 112 are configured to perform the SGD analysis and symbolic SGD analysis in parallel (using first processor 106 and second processor 108, respectively).
Initial data portion analyzer 110 can be configured to initialize output model weights for the initial portion of the training dataset using concrete values because the initial data portion does not have a dependency on a previous data portion (it is the first data portion). The symbolic representation used as the initial state of output model weights for additional portion(s) of the training dataset allows corresponding additional processors to perform symbolic SGD analysis on respective additional data portions without waiting for completion of the analysis of the previous data portion, thus enabling parallelization. Processing in “parallel” does not imply that the processing performed by different processors is fully coextensive in time. That is, some processors may begin or end sooner or later than others.
A global model builder 114 is configured to integrate an output of local model builder 112 and an output of initial data portion analyzer 110 into a global model that reflects both the initial portion and the additional portion of the training dataset. For example, the output model weights for the different data portions can be combined into a single global model that can include a set of output model weights representing the entire training dataset. The output of local model builder 112 can include a local model representing the output model weights for the additional portion of the training dataset. In some examples, the local model is a vector representation. The output of the local model builder can also include a model combiner indicating how to combine the output from initial data portion analyzer 110 with the local model. In some examples, the model combiner is a matrix representation. Example local models and model combiners are discussed in detail with respect to
In some examples, the local model and/or global models are linear models. The global model can include values for a set of global output model weights. In some examples, the SGD analysis and symbolic SGD analysis are linear regression analyses, although nonlinear SGD analysis and nonlinear symbolic SGD analysis are also contemplated.
Computing device(s) 102 can be a single computing device or multiple computing devices. In some examples, first processor 106 and initial data portion analyzer 110 are part of a first computing device, and second processor 108 and local model builder 112 are part of a second computing device. Global model builder 114 can be part of the first computing device, the second computing device, or can be part of still another computing device. In some examples, first processor 106 is part of a first mobile device, second processor 108 is part of a second mobile device, and global model builder 114 is in the cloud (not shown). In other examples, first processor 106 is part of a first server computer in a data center and second processor 108 is part of a second server computer in the data center.
In some examples, initial data portion analyzer 110 can be configured to perform symbolic SGD analysis similar to local model builder 112, and rather than using concrete initial values to perform SGD analysis, concrete initial values can be combined with an output of symbolic SGD analysis for the initial data portion.
Initial data portion analyzer 210 is configured to perform SGD analysis on the initial portion of the training dataset and generate an output 216 for the initial data portion. The SGD analysis is performed by initial data portion analyzer 210 using first processor 218. As was described with respect to
Additional local model builder 214 is configured to perform symbolic SGD analysis, similar to local model builder 212, on a further additional portion of the training dataset and to generate a local model and model combiner 224. Additional local model builder 214 is configured to perform the symbolic SGD analysis, in parallel with the SGD analysis and the symbolic SGD analysis performed by local model builder 212, using third processor 226. global model builder 228 combines output 216 for the initial data portion, local model and model combiner 220, and local model and model combiner 224, into a global model representing the training dataset.
Computing device 304 includes an initial data portion analyzer 314 and a first processor 316; computing device 306 includes a local model builder 320 and a second processor 322; and computing device 308 includes an additional local model builder 326 and a third processor 328. Initial data portion analyzer 314, local model builder 320, and additional local model builder 326 can be similar to the corresponding components in
Segmentation of training data and parallelization of SGD is further illustrated in
The symbolic execution framework described herein avoids the loss of information from previous machines. While the first machine can start from the initial decision vector w, i.e. [0,0,0], all other machines would instead start with a symbolic vector w, i.e. [x, y, z]. In a symbolic execution, variables contain symbolic expressions and programs manipulate these expressions rather than concrete values. For example, the previous vector w after an update like w+=1 becomes [x+1, y+1, z+1]. Each subsequent machine can create a symbolic function in terms of the input symbolic variable w.
In
g(xi,w)=xiT(xi·w−y) (2)
for a data point xi and decision vector w. The update function shown in
Process block 510 illustrates an efficient representation of symbolic output 508 as a matrix-vector multiplication that includes a local model 512 (a local model shown for each data pair) in a vector representation and a model combiner 514 (a model combiner shown for each data pair) in a matrix representation. Local model 512 and model combiner 514 include only concrete values, but in other examples, local model 512 and model combiner 514 contain symbolic values as well. By deducing how local model 512 and model combiner 514 can be computed, symbols can be removed from the computation, and some matrix-vector libraries, such as BLAS (Basic Linear Algebra Subprograms) and LAPCK (Linear Algebra Package), for example, can be used to improve efficiency. Given a set of data points in the k-dimensional feature space, the matrix of interest to learn is be k+1 by k+1 in size, where the ith row demonstrate the linear combination of the k symbolic terms of w and one symbolic interception. Interception can be regarded as one special feature and, as a consequence, can be considered part of the decision variable w.
w*
j
=w*
j
−a×z
i
T(xi×w*j−yi) (3)
where a is the learning rate, and x and w are defined as with respect to
M
j
=M
j
−a×x
i
T(xi×Mj) (4)
as shown in
w
j
=M
j
w
j-1
+w*
j. (5)
The final, global model is then provided in process block 612.
The following proof indicates that the described approaches, including method 600, yield similar results to sequential SGD. Equations (3) and (4) are denoted below as (3) and (4). The proof is as follows:
Substitute w#=M·w+w*, which is the formula in process block 610:
w
#
=w
#−α·(xi·w#−) (9)
where α is the learning rate, a predefined parameter used to control the convergence rate and convergence error.
Updating of w# follows the same update rule as depicted in equation (5). M and w* can be initialized to ensure that for any winitial, a suitable w can be picked so that w#=M·w+w*=winitial, where winitial is the initial decision vector supplied to the sequential algorithm, then the same final wfinal is obtained.
Process block 606 indicates one possible way to initialize M and w*: setting M to identity matrix and w* to all zeros. And for any winitial, w=winitial, could be chosen, which satisfies w#=M·w+w*=winitial. Another approach is to initialize w* to some vector wguess, which is potentially close to winitial (M is still set to identity, for example). Similarly for any winitial, w=winitial−wguess could be used, which satisfies w#=M·w+w*=winitial. Many additional initialization methods are also possible. In various examples, M is initialized to full rank.
There is another way to interpret what equations (3), (4), and (5) are calculating. In fact, equation (3) follows the same iterative rule as that shown in the update formula in process block 502 of
Another interesting thing is that with such matrix representation, the reason why different initial conditions converge to the same final result can be clearly understood. Suppose there are p machines, and on each local machine, M and w* are set to the identity matrix and an all-Os vector respectively for all local machines. The following equations result:
When the data set is properly scaled, then largest eigenvalue of M would be less than 1. And the first term in the formula above would be very small. In other words, different winitial would not affect the final wfinal that would be learned.
In process block 714, process blocks 716, 718, and 720 are performed on the respective computers of the plurality of computers and in parallel with the SGD analysis performed on the first computer. Initial values for a set of output model weights to a symbolic representation are set in process block 716. The output model weights correspond to the data portion provided to the computer. In process block 718, symbolic SGD analysis is performed on the data portion provided to the computer. In process block 720, a local model is determined based on the symbolic SGD analysis, the local model representing at least some of the output model weights for the data portion. A model combiner can also be determined. In process block 722, based on the output for the data portion provided to the first computer and the local models for the data portions provided to the respective computers of the plurality of computers, a global model reflecting the training dataset is determined. The global model can also be determined based on the model combiners.
In process block 806, a third stage output is received. The third stage output is generated by a third processor used to perform symbolic SGD analysis on a third portion of a training dataset. The symbolic SGD analysis comprises using a symbolic representation as an initial state of a set of output model weights for the third portion. The third stage output comprises a third stage local model for the third portion and a third stage model combiner for the third portion. The third stage local model and the third stage model combiner represent the set of output model weights for the third portion. The first, second, and third stage outputs were generated in parallel by the first, second, and third processors, respectively. In process block 808, a global model reflecting the training dataset is generated based on the first stage output, the second stage local model, the second stage model combiner, the third stage local model, and the third stage model combiner.
If the data for learning is dense, then M is dense and has a size of k×k, where k is the number of features. When k gets larger, the space overhead for storing M and the computation overhead of calculating M are both potentially problematic. In general, however, when the number of features becomes larger, the data set also becomes sparse. Suppose k is the number of existing features for all data examples, and d is the averaged nonzero features for each example. Then it is common that d is much less than k when k gets large.
There are many reasons why real-world data is sparse. In machine learning instances with large numbers of features, not all input points contain all features. Further, many features often have the default values or categories.
Two properties of the sparsity of datasets can be used to “optimize” M (i.e., reduce storage space and computation time overhead). First, when all the datasets in a machine do not have a certain feature, then the matrix M projected on these missing features is an identity matrix. Specifically, the following theorem holds.
When a feature i does not appear on local machine p, then following proof can be shown:
w
out
p
=M
p
*w
in
p
+w
p (14)
M
p
[i][j]=0 (15)
M
p
[i][i]=1wp[i] (16)
w
out
[i]=w
out
[i] (17)
This implies that M should only contain entries for features in the machine.
Second, as a corollary of this, if a particular feature is only present in one machine, then parts of M belonging to this feature need not be shared with other machines. Specifically, the following theorem holds. For one feature, i, that only appears on local machine p:
w
final
[i]=w
p
[i] (18)
is a corollary of the proof illustrated in equations (14)-(17).
Moreover, since a machine does not care about parts of M corresponding to features not in its dataset, a machine needs to only share parts of M that correspond to shared features. The shared features creates a “dependence” graph among machines. This further reduces the communication cost of sharing M across machines.
With reference to
A computing system may have additional features. For example, the computing system 900 includes storage 940, one or more input devices 950, one or more output devices 860, and one or more communication connections 970. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system 900. Typically, operating system software (not shown) provides an operating environment for other software executing in the computing system 900, and coordinates activities of the components of the computing system 900.
The tangible storage 940 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information and which can be accessed within the computing system 900. The storage 940 stores instructions for the software 980 implementing one or more innovations described herein. For example, storage 940 can store: initial data portion analyzer 110, local model builder 112, and/or global model builder 114 of
The input device(s) 950 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing system 900. For video encoding, the input device(s) 950 may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or a CD-ROM or CD-RW that reads video samples into the computing system 900. The output device(s) 960 may be a display, printer, speaker, CD-writer, or another device that provides output from the computing system 900.
The communication connection(s) 970 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can use an electrical, optical, RF, or other carrier.
The innovations can be described in the general context of computer-executable instructions, such as those included in program modules, being executed in a computing system on a target real or virtual processor. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Computer-executable instructions for program modules may be executed within a local or distributed computing system.
The terms “system” and “device” are used interchangeably herein. Unless the context clearly indicates otherwise, neither term implies any limitation on a type of computing system or computing device. In general, a computing system or computing device can be local or distributed, and can include any combination of special-purpose hardware and/or general-purpose hardware with software implementing the functionality described herein.
For the sake of presentation, the detailed description uses terms like “determine” and “use” to describe computer operations in a computing system. These terms are high-level abstractions for operations performed by a computer, and should not be confused with acts performed by a human being. The actual computer operations corresponding to these terms vary depending on implementation.
The illustrated mobile device 1000 can include a controller or processor 1010 (e.g., signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, input/output processing, power control, and/or other functions. An operating system 1012 can control the allocation and usage of the components 1002 and support for one or more application programs 1014. The application programs can include common mobile computing applications (e.g., email applications, calendars, contact managers, web browsers, messaging applications), or any other computing application. The application programs 1014 can also include machine learning technology using parallelized SGD analysis. Functionality 1013 for accessing an application store can also be used for acquiring and updating application programs 1014.
The illustrated mobile device 1000 can include memory 1020. Memory 1020 can include non-removable memory 1022 and/or removable memory 1024. The non-removable memory 1022 can include RAM, ROM, flash memory, a hard disk, or other well-known memory storage technologies. The removable memory 1024 can include flash memory or a Subscriber Identity Module (SIM) card, which is well known in GSM communication systems, or other well-known memory storage technologies, such as “smart cards.” The memory 1020 can be used for storing data and/or code for running the operating system 1012 and the applications 1014. Example data can include web pages, text, images, sound files, video data, or other data sets to be sent to and/or received from one or more network servers or other devices via one or more wired or wireless networks. The memory 1020 can be used to store a subscriber identifier, such as an International Mobile Subscriber Identity (IMSI), and an equipment identifier, such as an International Mobile Equipment Identifier (IMEI). Such identifiers can be transmitted to a network server to identify users and equipment.
The mobile device 1000 can support one or more input devices 1030, such as a touchscreen 1032, microphone 1034, camera 1036, physical keyboard 1038 and/or trackball 940 and one or more output devices 1050, such as a speaker 1052 and a display 1054. Other possible output devices (not shown) can include piezoelectric or other haptic output devices. Some devices can serve more than one input/output function. For example, touchscreen 1032 and display 1054 can be combined in a single input/output device.
The input devices 1030 can include a Natural User Interface (NUI). An NUI is any interface technology that enables a user to interact with a device in a “natural” manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls, and the like. Examples of NUI methods include those relying on speech recognition, touch and stylus recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence. Other examples of a NUI include motion gesture detection using accelerometers/gyroscopes, facial recognition, 3D displays, head, eye, and gaze tracking, immersive augmented reality and virtual reality systems, all of which provide a more natural interface, as well as technologies for sensing brain activity using electric field sensing electrodes (EEG and related methods). Thus, in one specific example, the operating system 1012 or applications 1014 can comprise speech-recognition software as part of a voice user interface that allows a user to operate the device 1000 via voice commands. Further, the device 1000 can comprise input devices and software that allows for user interaction via a user's spatial gestures, such as detecting and interpreting gestures to provide input to a gaming application.
A wireless modem 1060 can be coupled to an antenna (not shown) and can support two-way communications between the processor 1010 and external devices, as is well understood in the art. The modem 1060 is shown generically and can include a cellular modem for communicating with the mobile communication network 1004 and/or other radio-based modems (e.g., Bluetooth 1064 or Wi-Fi 1062). The wireless modem 1060 is typically configured for communication with one or more cellular networks, such as a GSM network for data and voice communications within a single cellular network, between cellular networks, or between the mobile device and a public switched telephone network (PSTN).
The mobile device can further include at least one input/output port 1080, a power supply 1082, a satellite navigation system receiver 1084, such as a Global Positioning System (GPS) receiver, an accelerometer 1086, and/or a physical connector 1090, which can be a USB port, IEEE 1394 (FireWire) port, and/or RS-232 port. The illustrated components 1002 are not required or all-inclusive, as any components can be deleted and other components can be added.
In example environment 1100, the cloud 1110 provides services for connected devices 1130, 1140, 1150 with a variety of screen capabilities. Connected device 1130 represents a device with a computer screen 1135 (e.g., a mid-size screen). For example, connected device 1130 can be a personal computer such as desktop computer, laptop, notebook, netbook, or the like. Connected device 1140 represents a device with a mobile device screen 1145 (e.g., a small size screen). For example, connected device 1140 can be a mobile phone, smart phone, personal digital assistant, tablet computer, and the like. Connected device 1150 represents a device with a large screen 1155. For example, connected device 1150 can be a television screen (e.g., a smart television) or another device connected to a television (e.g., a set-top box or gaming console) or the like. One or more of the connected devices 1130, 1140, 1150 can include touchscreen capabilities. Touchscreens can accept input in different ways. For example, capacitive touchscreens detect touch input when an object (e.g., a fingertip or stylus) distorts or interrupts an electrical current running across the surface. As another example, touchscreens can use optical sensors to detect touch input when beams from the optical sensors are interrupted. Physical contact with the surface of the screen is not necessary for input to be detected by some touchscreens. Devices without screen capabilities also can be used in example environment 1100. For example, the cloud 1110 can provide services for one or more computers (e.g., server computers) without displays.
Services can be provided by the cloud 1110 through service providers 1120, or through other providers of online services (not depicted). For example, cloud services can be customized to the screen size, display capability, and/or touchscreen capability of a particular connected device (e.g., connected devices 1130, 1140, 1150).
In example environment 1100, the cloud 1110 provides the technologies and solutions described herein to the various connected devices 1130, 1140, 1150 using, at least in part, the service providers 1120. For example, the service providers 1120 can provide a centralized solution for various cloud-based services. The service providers 1120 can manage service subscriptions for users and/or devices (e.g., for the connected devices 1130, 1140, 1150 and/or their respective users). The cloud 1110 can store local models and model combiners 1160 and a global model builder 1162.
Although the operations of some of the disclosed methods are described in a particular, sequential order for convenient presentation, it should be understood that this manner of description encompasses rearrangement, unless a particular ordering is required by specific language set forth below. For example, operations described sequentially may in some cases be rearranged or performed concurrently. Moreover, for the sake of simplicity, the attached figures may not show the various ways in which the disclosed methods can be used in conjunction with other methods.
Any of the disclosed methods can be implemented as computer-executable instructions or a computer program product stored on one or more computer-readable storage media and executed on a computing device (e.g., any available computing device, including smart phones or other mobile devices that include computing hardware). Computer-readable storage media are any available tangible media that can be accessed within a computing environment (e.g., one or more optical media discs such as DVD or CD, volatile memory components (such as DRAM or SRAM), or nonvolatile memory components (such as flash memory or hard drives)). By way of example and with reference to
Any of the computer-executable instructions for implementing the disclosed techniques as well as any data created and used during implementation of the disclosed embodiments can be stored on one or more computer-readable storage media. The computer-executable instructions can be part of, for example, a dedicated software application or a software application that is accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software can be executed, for example, on a single local computer (e.g., any suitable commercially available computer) or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a client-server network (such as a cloud computing network), or other such network) using one or more network computers.
For clarity, only certain selected aspects of the software-based implementations are described. Other details that are well known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any specific computer language or program. For instance, the disclosed technology can be implemented by software written in C++, Java, Perl, JavaScript, Adobe Flash, or any other suitable programming language. Likewise, the disclosed technology is not limited to any particular computer or type of hardware. Certain details of suitable computers and hardware are well known and need not be set forth in detail in this disclosure.
Furthermore, any of the software-based embodiments (comprising, for example, computer-executable instructions for causing a computer to perform any of the disclosed methods) can be uploaded, downloaded, or remotely accessed through a suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.
The disclosed methods, apparatus, and systems should not be construed as limiting in any way. Instead, the present disclosure is directed toward all novel and nonobvious features and aspects of the various disclosed embodiments, alone and in various combinations and sub combinations with one another. The disclosed methods, apparatus, and systems are not limited to any specific aspect or feature or combination thereof, nor do the disclosed embodiments require that any one or more specific advantages be present or problems be solved.
The technologies from any example can be combined with the technologies described in any one or more of the other examples. In view of the many possible embodiments to which the principles of the disclosed technology may be applied, it should be recognized that the illustrated embodiments are examples of the disclosed technology and should not be taken as a limitation on the scope of the disclosed technology.
