Patent Application
20250232187
A portion of the disclosure of this patent document contains material which is subject to copyright protection. This patent document may show and/or describe matter which is or may become tradedress of the owner. The copyright and tradedress owner has no objection to the facsimile reproduction by anyone of the patent disclosure as it appears in the U.S. Patent and Trademark Office files or records, but otherwise reserves all copyright and tradedress rights whatsoever.
Embodiments of the present invention are in the field of problem-solving systems based on Large Language Models (LLMs).
The statements in this section may serve as a background to help understand the invention and its application and uses, but may not constitute prior art.
Large language models (LLMs) include a type of artificial intelligence model designed to understand and generate human-like text. They can be trained on a relatively large amount of text data and can generate coherent and contextually relevant sentences by predicting the likelihood of a word given the previous words used in the text. LLMs can be based on a type of machine learning model called a transformer, which uses self-attention mechanisms to weigh the influence of different words in the input when generating the output. The “large” in Large Language Models refers to the number of parameters that the model has, often in the range of billions or even trillions. These parameters are learned during the training process, where the model is exposed to a large corpus of text data. LLMs have a wide range of applications, including but not limited to, text generation, translation, summarization, question answering, and more. They can be used to create chatbots, write articles, generate creative content, and assist in drafting legal documents. They are also used in research to advance our understanding of natural language processing. Despite their impressive capabilities, LLMs also have limitations. They can sometimes generate incorrect or nonsensical responses, and they can be sensitive to slight changes in input phrasing. They also require a large amount of computational resources for training and can inadvertently learn and reproduce biases present in the training data.
Self-attention-based auto-regressive LLMs such as GPT-4 have recently been developed. These LLMs can perform tasks that were previously thought as extremely difficult or even impossible. For example, LLMs can handle various logical and mathematical reasoning tasks, particularly those that entail “short-range reasonings” necessitating only a few steps to arrive at conclusions. Such capabilities may include an initial version of artificial general intelligence (AGI). However, LLMs can exhibit limitations in certain domains, especially for “long-range” reasoning tasks, where long-term planning and solution exploration are necessary. When presenting LLMs such as GPT-4 with a challenging problem-solving task, such as multi-step reasoning problems, the model does not always succeed. Although the generated answer may be indicative of the correct direction, the derivation process may frequently include logical errors.
Therefore, in view of the aforementioned difficulties, there is an unsolved need to provide an LLM-based architecture enabling long range problem-solving.
It is against this background that various embodiments of the present invention were developed.
Methods and systems are provided for an improved LLM-based problem-solving system. More specifically, in one aspect, one embodiment of the present invention is a system including computational modules comprising a prompter agent, a checker module, a memory module, and a controller. The system is configured to receive a problem, engage the computational modules in a problem-solving process comprising a multi-round conversation with a large language model, record the multi-round conversation and state history associated with the problem-solving process on the memory module to determine a first outcome, and potentially backtrack to a previous step of the problem-solving process and determine a second outcome.
In one illustrative embodiment, a non-transitory computer-readable storage medium is provided, having instructions stored therein, which when executed by a processor, cause a tree-of-thought (ToT) controller of a problem-solving system to query a state of a memory module in the problem-solving system. The state comprises a problem description, a current node, a validity of the current node, and a node visit history. The current node and nodes in the node visit history are nodes in a search tree, where the nodes in the search tree represent partial solutions. The validity of the current node is determined by a checker module in the problem-solving system. The instructions also cause the ToT controller to determine, based on the state of the memory module in the problem-solving system, a next search step, where the next search step is selected from the group consisting of the current node, a parent node of the current node, and an ancestor node of the current node in the search tree. The instructions also cause the TOT controller to activate, by sending a control signal, a prompter agent in the problem-solving system to prompt a large language model (LLM) to generate an intermediate solution for the next search step.
In some embodiments, the determining of the next search step by the ToT controller is based on the output of a controller policy network, the controller policy network is trained with a policy gradient reinforcement learning algorithm.
In some embodiments, the determining of the next search step by the ToT controller is based on a rule-based backtracking algorithm.
In some embodiments, the prompter agent is implemented by a prompter policy network, and the prompter policy network takes as input a prompt template, a node visit history, and a set of in-context learning examples, and outputs a prompt for the TOT controller. In some embodiments, the determining of the next search step by the ToT controller is based on the output of a controller policy network, and the controller policy network and the prompter policy network are simultaneously trained with a multi-agent reinforcement learning algorithm.
In some embodiments, the checker module is a neural network classifier.
In some embodiments, the LLM is implemented on one or more edge nodes in a decentralized blockchain-based network.
In some embodiments, the ToT controller is implemented on one or more edge nodes in a decentralized blockchain-based network.
In some embodiments, the instructions further cause the ToT controller to submit the intermediate solution generated by the LLM to a reward smart contract deployed on a blockchain in the decentralized blockchain-based network, wherein the intermediate solution is marked as a valid final solution by the checker module in the problem-solving system. The ToT controller then receives a reward from the reward smart contract for submitting the intermediate solution. In some embodiments, the ToT controller communicates with the memory module and prompter agent through a peer-to-peer connection on the decentralized blockchain-based network.
In some embodiments, the prompter module is run on one or more edge nodes in a decentralized blockchain-based network.
In some embodiments, the problem description is an instance of a puzzle (e.g., Sudoku problem), the intermediate solution is a partial puzzle solution, the checker module is a rule-based checker of a partial puzzle solution, the memory module stores a conversation history between the LLM and the prompter agent, and the ToT controller uses a rule-based backtracking algorithm.
In some embodiments, the problem description is an instance of a multi-step problem-solving task.
In some embodiments, the ToT controller is a module that directs a search process of the problem-solving system by sending the control signal to the prompter agent based on the state of the memory module; the memory module is a module that stores a conversation history between the LLM and the prompter agent; the checker module is a module that determines the validity of the current node; and the prompter agent is a module that generates a prompt for the LLM based on the control signal from the ToT controller.
In other illustrative embodiments, a ToT problem-solving system is provided. The system may comprise access to a large language model (LLM), access to a processor, and a non-transitory physical medium for storing program code executable by the processor, the program code when executed by the processor causing the processor to implement: a prompter agent adapted to generate a prompt for the LLM based on a control signal; a memory module adapted to store a conversation history between the LLM and the prompter agent; a checker module adapted to determine a validity of a current node in a search tree of a search process; and a ToT controller adapted to direct the search process of the TOT problem-solving system by sending the control signal to the prompter agent based on a state of the memory module. The non-transitory physical medium further stores program code that when executed by the processor causes the ToT controller to query the state of the memory module, where the state comprises a problem description, the current node, the validity of the current node, and a node visit history, where the current node and nodes in the node visit history are nodes in the search tree of the search process, where the nodes in the search tree represent partial solutions, and where the validity of the current node is determined by the checker module. The ToT controller may also determine, based on the state of the memory module, a next search step, where the next search step is selected from the group consisting of the current node, a parent node of the current node, and an ancestor node of the current node in the search tree. The TOT controller may also activate, by sending the control signal, the prompter agent to prompt the LLM to generate an intermediate solution for the next search step.
Yet other aspects of the present invention include methods, processes, and algorithms comprising the steps described herein, and also include the processes and modes of operation of the systems and servers described herein.
Other aspects and embodiments of the present invention will become apparent from the detailed description of the invention when read in conjunction with the attached drawings.
Embodiments of the present invention described herein are exemplary, and not restrictive. Embodiments will now be described, by way of examples, with reference to the accompanying drawings, in which:
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these specific details. In other instances, structures, devices, activities, and methods are shown using schematics, use cases, and/or flow diagrams in order to avoid obscuring the invention. Although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to suggested details are within the scope of the present invention. Similarly, although many of the features of the present invention are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the invention is set forth without any loss of generality to, and without imposing limitations upon the invention.
THETA is a trademark name carrying embodiments of the present invention, and hence, the aforementioned trademark names may be interchangeably used in the specification and drawings to refer to the products/services offered by embodiments of the present invention. As the context may require, the term THETA may be used in this specification to describe the overall decentralized computing network or platform, the public ledger system for rewarding computation resource sharing, as well as the company providing said network, platform, system, or services. With reference to the figures, embodiments of the present invention are now described in detail.
Self-attention-based auto-regressive LLMs such as GPT-4 have recently been developed. These LLMs can perform tasks that were previously thought as extremely difficult or even impossible. For example, LLMs can handle various logical and mathematical reasoning tasks, particularly those that entail “short-range reasonings” necessitating only a few steps to arrive at conclusions. Such capabilities may include an initial form of artificial general intelligence (AGI). However, LLMs can exhibit limitations in certain domains, especially for “long-range” reasoning tasks, where long-term planning and solution exploration are necessary. When presenting LLMs such as GPT-4 with a challenging problem-solving task, such as System-2 reasoning problems, the model does not always succeed. Although the generated answer may be indicative of the correct direction, the derivation process may frequently include logical errors.
There may be at least two main contributing factors which limit the problem-solving ability of LLMs: (1) Lack of correctness checking: To ensure correctness, a human solver may carry out verification procedures at every step of the problem-solving process, thereby ensuring the credibility of the final solution. In comparison, an auto-regressive language model may not explicitly perform logical correctness checks as the model generates a new token based on the previous tokens. This can limit the model's capacity to rectify its own mistakes. A minor error can be amplified as the model generates more tokens, thereby leading to rapid solution quality deterioration, making it difficult to recover from mistakes. (2) Solution generated linearly: As mentioned above, LLMs may generate a token based on the preceding sequence of tokens without backward editing. On the contrary, human solvers may attempt to solve a problem by backtracking to previous steps if a derivation step is incorrect, or if they become stuck and unable to make further progress towards arriving at the final answer. The problem-solving process may thus include a tree-like thinking process, rather than a linear chain-of-thought process. The limitation of linear response generation can also be apparent from a computational complexity perspective. The number of computational steps an auto-regressive LLM can perform can be polynomial in terms of its input length. Unless P=NP holds, there are problems in NP that are not solvable by auto-regressive LLMs.
Based at least on the shortcomings of auto-regressive LLMs described above, the disclosure proposes systems and methods which augment an LLM with several additional modules including an automatic prompter agent and a Tree-of-Thought (ToT) controller. The system implements a solution search strategy (i.e., the Tree-of-Thought, or “ToT” in short). This system can solve a problem through a multi-round conversation between the LLM and the prompter agent that is guided by the ToT controller. In this disclosure, the terms “solution search” and “problem-solving” are used interchangeably.
In some aspects, the system can include a checker module to perform correctness checks. In
As an illustration of this process, an example instance of solving a Sudoku puzzle in the ToT framework may begin at root node 102. A first search step 104 guided by the LLM is taken. The partial solution 106 is marked invalid by the checker module. The TOT controller issues a signal to take a backtracking step 108 back to root node 102. The LLM then takes another search step 110 and provides a new partial solution 112 for the problem in the root node 102. The partial solution 112 is considered valid by the checker module, and the ToT controller issues a signal which causes the LLM to take the next search step 114 from the state with partial solution 112, to yield a partial solution 116. The partial solution 116 is marked invalid by the checker module, and the ToT controller commands a backtracking step 118 to node 112. From node 112, the LLM then provides another search step 120 to partial solution 122, which is also marked invalid. In order to explore alternate routes, the ToT controller issues a signal to backtrack two steps, step 124 and step 126, back to the root node 102. At this point, the LLM provides a series of search steps 128, 132, 136, and 140 which lead to valid partial solutions 130, 134, 138, and finally the correct solution 142, respectively. The search process returns the correct solution 142 and terminates.
Note that while the above discussion utilized Sudoku solving as a tangible example to illustrate certain aspects of the disclosed systems and methods, the ToT framework can be applied to more general mathematical and logical reasoning, problem-solving, solution searching, and decision making tasks. For example, in the context of mathematical theorem proving, a full solution corresponds to the complete proof, encompassing a total of n derivation steps. On the other hand, a partial solution refers to a subset of these steps, specifically the initial k steps, where k is less than n. The checker verifies the logical correctness of a given partial proof. In parallel, the prompter agent and the ToT controller can offer hints and suggestions to the LLM, encouraging the ToT controller to analyze the subsequent proving step, or explore different directions for theorem proving when necessary.
To evaluate the effectiveness of the TOT framework, a ToT-based Sudoku puzzle solver can be implemented and evaluated on a suite of Sudoku puzzle benchmarks. As follows, the disclosure provides the details of the TOT system architecture, further shows an example implementation of a ToT-based Sudoku puzzle solver, presents the experimental results, and describes the implementation of the TOT framework in a decentralized blockchain-supported edge computing network.
In some respects, the problem-solving process can include a step where the user inputs the problem description 202. The prompter agent 220 then relays the problem to the LLM 230 in a prompt 222, which may include additional prompt text such as in-context examples. Based on the prompt 222, the LLM 230 outputs a response 232 that contains an intermediate solution (e.g., instead of trying to reach the full solution in a single attempt). After receiving the response 232 from the LLM 230, the checker module 240 is invoked to check the validity of the intermediate solution in the response 232. If it passes the correctness check, the intermediate solution 242 can be parsed and stored, along with the conversation and node visit history, in the memory module 250. If the correctness check is not passed, the conversation history and node visit history are still stored in the memory module 250. Based on the conversation and node history, the ToT controller 210 issues control signals 212 (e.g., backtrack or continue to search for the solution) to the prompter to either continue to search for the solution from the current node, or to backtrack to a previous node. If the intermediate solution 242 was correct, then the prompter agent 220 generates a new prompt 222 to encourage the LLM 230 to generate the next step, where the new prompt 222 is generated in part based on the content of the memory module 250, including the conversation and node history as shown in one prong of the information flow line 252. Conversely, if the LLM 230 generates an invalid intermediate solution, the ToT controller 210 will send control signals 212 to activate the prompter agent 220 to offer hints to the LLM and request it to consider again in the new prompt 222. In some respects, a valid intermediate solution may not always lead to the correct final solution. In order to prevent the system from getting stuck, the ToT controller 210 can monitor the search process by accessing the content of the memory module, including the conversation and node visit history, as shown in the other prong of the information flow line 252, and determine whether to continue trying from the current node or backtrack to a parent or an ancestor node and explore alternative directions.
The ToT strategy comprises a tree-search algorithm using an LLM as a heuristic for generating the search steps. In this setting, the LLM can be used for the “short-range reasoning” tasks, i.e., deriving the next intermediate solution, which is a type of tasks that have been shown to have a high success rate for LLMs. On the other hand, by introducing the checker module, the system can have a higher likelihood to discover the mistakes it makes as it generates the solutions. Moreover, by allowing the system to backtrack from a valid but somewhat “hopeless” intermediate solution, the system can explore a larger solution space, which enhances the “long-range reasoning” capability of the system as a whole. The TOT framework thus combines the best of both worlds. Furthermore, this multi-round conversation technique increases the number of computational steps the system can perform. Thus, based on the time-hierarchy theorem in computational complexity theory, the TOT framework can expand the range of problems that can potentially be solved compared to relying solely on a single round of conversation with an LLM.
In various respects, the checker module can be rule-based or can be implemented as a deep neural network. For problems that have an explicit polynomial time algorithm for correctness checking (i.e., problems in NP), rule-based checkers can be implemented. Numerous important mathematical and logical problems are in this category, for example, equation solving, polynomial factoring, 3SAT, and puzzles like Sudoku. With a rule-based checker, the ToT system can be viewed as a hybrid system which allows explicitly encoding prior knowledge (e.g., the Sudoku rules) into a neural network powered system. An alternative is to train and use a neural network-based classifier as the checker. This can be useful for problems where a rule-based checker is difficult to implement, e.g., checking whether a mathematical proof is correct.
One exemplary embodiment of the checker module comprises an example implementation shown in Table 1, which is listed at the end of the present disclosure.
The memory module may store an entire conversation history between the LLM and the prompter agent, as well as other supplemental data useful for problem-solving. The data stored can serve as the information source for the prompter agent to generate helpful hints for the LLM.
One exemplary embodiment of the memory module comprises an example implementation shown in Table 2, which is listed at the end of the present disclosure.
The TOT controller may oversee the entire ToT search. It can be implemented in a number of ways. In some embodiments, the ToT controller may be encoded with three simple rules: 1) if the checker module thinks the current partial solution is invalid, backtrack to the parent node, 2) if the current partial solution is valid, but the TOT search tree has explored C children of the current partial solution and yet failed to find the final solution, then backtrack to the parent node, and 3) if the current partial solution is valid and the TOT search tree has not explored C of its children, continue finding solutions from the current partial solution. Here, C is a pre-configured integer.
A more advanced version of the ToT controller can employ a policy network to determine the backtracking policy. The network's inputs include the recent search history comprising the sequence of the last k+1 nodes visited in the search tree si-k, . . . , si-1, si, where k is a hyper-parameter and si is the ith visited node in the search history. The network also takes in ci a Boolean variable which indicates whether the checker module considers the current node si is valid. In some aspects, the system can sample from the policy to determine the next action di:
In the above formula, “∥” is the string concatenation operator. The disclosure further discusses the training algorithm for the ToT controller policy network in the section “Controller and Prompter Training.”
In some aspects, a prompter agent gives hints to the LLM for it to generate the next search step. One basic hint can be a generic prompt using the following template: generic_tmpl=“For the given problem: [problem description], we have come up with a partial solution: [partial solution summary]. Please derive the next step on top of this partial solution, and return the next step in the following JSON format {next step: next step}”. Note that the template requires the LLM to respond with a structured string such as a structured JSON string. This can facilitate for the checker to extract the next solution step from the LLM response. To create an actual prompt from this template, the prompter needs the [problem description] and the [partial solution summary], both of which can be queried from the memory module.
Similar to the ToT controller, the prompter agent can also be implemented as a policy network, which can generate prompts based on the current partial solution and the conversation history. First, the prompt template can be defined as follows: prompt_tmpl=generic_tmpl∥“Here are a few examples: [in-context learning examples].”, where ∥ is the string concatenation operator. The variable [in context learning examples] represents in-context learning examples for the problem being solved, which can be picked by the prompter policy network from a set of candidate examples. The rationale is that given the current and recently attempted intermediate solution, some in-context examples might work better than others as hints for the next step. Given the recently visited node sequence si-k, . . . , si-1, si, our goal is to select/examples ei={ei1, ei2 . . . , eil|eij ∈Ecand}, where Ecand is a pool of in-context learning example candidates. The examples can be selected according on a policy:
The prompter policy network can be trained together with the ToT controller using multi-agent reinforcement learning methods. The training algorithm of the prompter's policy network is discussed further below.
In the previous sections, the disclosure described a multi-agent ToT framework. This section describes methods of training the agents, in particular, the policy networks of the TOT controller and the prompter agent. In addition to a variety of multi-agent reinforcement learning algorithms (MARL), a relatively simple approach, which uses a modified version of the REINFORCE algorithm, can be used to train the policy networks of the TOT controller and the prompter agent directly.
A run of the TOT system can be defined as the process where a user inputs the problem description, and the ToT system attempts to solve the problem until it thinks the problem is solved, or a pre-specified maximum round of conversations is reached. Next, the reward r of a run may be set according to the rules: if the problem is correctly solved, then r=+1; otherwise, if the system outputs an incorrect solution, or the maximum round of conversations is reached, then r=−1.
A policy gradient based training algorithm for the ToT system is provided in Algorithm 1 as shown in
After the TOT system is trained, it can be used for inference, i.e. problem-solving. As shown in
Next, in step 406 (Line 4) up to K rounds of conversations with the LLM are conducted for problem-solving. In each round, in step 408 (Line 5), the LLM first produces a response for the given prompt. Then, in step 410 (Line 6) the checker analyzes the response, and returns a result. The result contains the partial solution extracted from the LLM response, as well as information comprising whether the checker considers the solution as a valid final solution, a valid intermediate solution, an invalid partial solution, or other classifications. In step 412 (Lines 7-9), if the solution is a valid final solution, the algorithm simply returns it. Otherwise, the result is stored in the memory module in step 414 (Line 10). Based on the content of the memory module, the ToT controller issues control signals, e.g., backtracking by l levels, to the prompter in step 416 (Line 11). Finally, based on the control signal, the prompter looks up the relevant information from the memory module, and produces the next prompt for the LLM in step 418 (Line 12). If no valid final solution is found within K rounds of conversations, the algorithm returns nil indicating it fails to solve the problem in step 420 (Line 14).
An evaluation methodology and experimental results for the TOT system including the ToT framework are described and results are shown in reference to
The ToT-based Sudoku solver follows the generic framework described in Section 3 with some specific tweaks for the Sudoku problem. It allows a user to input a Sudoku puzzle using natural languages, for example: “Please solve this 4×4 Sudoku puzzle [[3,*,*,2],[1,*,3,*],[*,1,*,3],[4,*,*,1]] where * represents a cell to be filled.”
In an example embodiment, the ToT-based Sudoku solver is implemented in Python. In some embodiments, a rule-based approach for the checker module is adopted since the Sudoku rules are precise and easy to check. The memory module stores the conversation history between the prompter and the LLM, as well as a search tree which maintains all the partially filled Sudoku boards the LLM has generated so far. This way, when backtracking happens, the previous board configuration can be retrieved. In this example embodiment, the ToT controller is also rule-based. It returns to the parent node in the search tree if either the current node is considered invalid by the checker, or the search algorithm has explored more than 5 children of the current node. Finally, the prompter agent uses a variation of the generic template mentioned above, with the/problem description/being the initial configuration of the Sudoku board input by the user, and/partial solution summary/being the partially filled board represented by the current node in the search tree. The LLM utilized in this example embodiment is the “gpt-3.5-turbo” model, which is accessible through the OPENAI API suite, and the temperature parameter is set to 1.
In an illustrative example, four LLM-based Sudoku puzzle solvers are implemented and their performance is compared, as shown in the plot in
In comparison, the exemplary ToT-based solver demonstrates superior performance when compared to the other solvers. For the 3×3 benchmark set, it was able to solve all the puzzles. The success rate improves by 11% compared to the second best for the two benchmark sets. For the 4×4 benchmark set, the exemplary ToT-based solver failed to find the solution for 1 out of the 10 puzzles before reaching the maximum round of conversations (which is set to 100 in our experiments). This may be due to the limited capability of the rule-based ToT controller. In particular, the rule-based controller may not have information about whether the current partially-filled board can be completed without violating the Sudoku rules, which decreases the efficiency of the solution search. In some embodiments, a neural network based ToT controller may perform better. Despite this, the success rate of the exemplary ToT based solver is still 80% higher compared to that of the one-shot (os) and few-shot (fs) based solvers. Finally, for the 5×5 puzzles, the exemplary ToT-based solver failed with 2 puzzles before reaching the maximum round of conversations. Nonetheless, the success rate is 60% higher compared to that of the one-shot and few-shot based solvers.
In this disclosure, the Tree-of-Thought framework is described, which enhances LLMs with additional control and prompter agents, and checker and memory modules, resulting in improved performance for mathematical problem-solving tasks and other problem-solving tasks. To evaluate the performance of this technique, an exemplary Sudoku puzzle solver based on the ToT framework was disclosed. The system can utilize a rule-based checker that contains custom logic, which may, in some situations, make the system less easily adaptable to other problems. For more generic problems, for example, general mathematical and logical reasoning problems, where rule-based solution checking is difficult to implement, the checkers can be based on neural networks or other probabilistic models. In other aspects, a neural network based ToT controller may enhance the system performance. Additionally, the policy-gradient based training algorithm described may be relatively simple and may be susceptible to training stability issues. To further optimize the ToT system, more advanced multi-agent reinforcement learning algorithms, particularly those designed for cooperative agents, can be included.
In other aspects, the framework can comprise a “self-play” technique to enable the ToT system to develop novel problem-solving strategies that are not found in the LLM's training text corpus. The self-play training method can include a reinforcement learning technique which was popularized by the development of competitive game-playing agents where an AI agent learns to improve its own strategy by playing against itself. Some LLMs are typically trained using self-supervised learning techniques. They may have limitations when it comes to problem-solving, as they may not be able to generate samples (i.e. novel problem-solving strategies) that fall outside the distribution of the training data. In other words, they may not be able to “think outside the box”, which can be a crucial human trait that facilitates the discovery of new knowledge. Compared to self-supervised learning, self-play-based reinforcement learning enables the system to access a much broader solution space beyond the provided training examples, allowing for greater improvement. AlphaGo and similar systems have demonstrated the ability to devise strategies that surpass those of human experts. For ToT system training, instead of relying on the training data set Ptrain, the system can use a “quizzer” module which can come up with problem descriptions on its own to train the ToT controller and the prompter agent. An enabler for training some systems can be that the environment reward can be precisely determined, as it is straightforward to determine whether the gameplay results in a win or a loss. In various aspects, the system including the ToT framework can incorporate a checker that can assess the correctness of the solution, functioning similarly to the environment, particularly for problems that have well-defined solution validation rules. Thus, the reinforcement learning training methods can be readily applied. The self-driven learning approach, similar to the self-play method, can be an effective means of improving the ToT framework's problem-solving capabilities beyond the solution examples provided in the training text corpus for the LLMs.
Some embodiments of a system comprising a hardware processor, a non-transitory storage medium, and computational modules comprising a prompter agent, a checker module, a memory module, and a ToT controller are described. The non-transitory storage medium comprises program code to receive a problem, engage the computational modules in a problem-solving process comprising a multi-round conversation with an LLM, record the multi-round conversation and state history associated with the problem-solving process on the memory module to determine a first outcome, and then, responsive to the first outcome, potentially backtrack to a previous step of the problem-solving process to determine a second outcome.
In some embodiments, the problem-solving system comprises a ToT controller, wherein the ToT controller uses one or more large language models (LLMs) to derive one or more partial solutions to a problem, and wherein the TOT controller is capable of backtracking to a previously derived partial solution.
In some embodiments, the problem-solving system further comprises a blockchain comprising one or more blockchain nodes, wherein a blockchain node runs one instance of the one or more LLMs. In other embodiments, the problem-solving system further comprises a blockchain comprising one or more blockchain nodes, wherein a blockchain node runs a ToT controller task.
In some embodiments, the memory module and the checker module may be integrated with the ToT controller. For example, the memory module and the checker module can provide methods that can be called by the ToT controller to update the current state, and validate the partial solution.
In various aspects, the disclosed systems and architectures can be used to integrate built-in artificial intelligence and/or large language model (LLM) functions into the blockchain, such as the THETA edge network supported by the THETA blockchain. Some machine learning (ML) tasks like classification, regression, text completion, and the like can be completed on the THETA Blockchain. In some aspects, users can call these AI functions through smart contracts. In other respects, the disclosed systems can allow users to contribute their device's computational resources to train and power GPT-like applications, and can earn tokens accordingly which in turn can be used to make their own queries, and get answers.
Accordingly, distributed embodiments of the architecture of
In some embodiments, the present invention relates to methods and systems for an incentivized and decentralized peer-to-peer network for bandwidth, data, and/or computational resource sharing. More specifically, embodiments of the present invention are directed to an incentivized and decentralized computing network or platform (hereafter the “THETA edge computing platform”, “THETA edge computing network”, “THETA edge network”, or “THETA network”) in which peer-to-peer (P2P) computational resource sharing is facilitated through smart contracts on a blockchain (hereafter the “THETA blockchain”) maintained by a blockchain network (hereafter the “THETA blockchain network”). In various embodiments, a smart contract-based infrastructure implements functionalities including task assignment, data distribution, solution verification, reward pooling, and incentive micropayments. That is, peer edge nodes in the THETA network invoke, via blockchain nodes, decentralized applications (DApps) in the form of smart contracts on the THETA blockchain to share redundant or unused processing power, to function as “edge computing nodes” or “edge workers”, to support other peer nodes' computational demands by exploiting otherwise wasted idle CPU cycles, and to automatically receive rewards for the work done for other peer nodes.
A peer-to-peer mesh network can allow interconnected peer nodes to share resources with each other without the use of a centralized managing server or stable host. The THETA network, as described in issued U.S. Pat. No. 10,771,524 (Methods and Systems for a Decentralized Data Streaming and Delivery Network, incorporated by reference in its entirety herein), enables the sharing of bandwidth by edge cacher nodes, to cache and relay video and other data, in a network infrastructure that is scalable to millions of concurrent users through native THETA blockchain protocol-level support for cryptocurrency micropayments. As bandwidth sharing utilizes minimal Central Processing Unit (CPU) and Graphical Processing Unit (GPU) resources, the present disclosure provides an additional framework to enable edge nodes to distributively perform computationally intensive tasks such as video transcoding and drug design and discovery-related computations.
In one aspect, embodiments of the present invention establish a self-organized and dynamic universal edge computing platform including task initiator nodes and edge computing nodes that interact directly with each other and with smart contracts hosted by the THETA blockchain. The THETA blockchain includes a decentralized public ledger system that builds upon a multi-level Byzantine Fault Tolerant (BFT), Proof-of-Stake consensus mechanism carried out by a small number of validator nodes and a large number of guardian nodes in the THETA blockchain network. The THETA blockchain can provide native protocol level support for resource-oriented micropayment pools and smart contracts for computation task management and solution verification. A smart contract can include a self-executing contract or self-enforcing agreement in the form of computer programming codes or instructions managed by and stored on a blockchain. A smart contract represents a decentralized application that is executed based on predetermined conditions or based on an agreement between participating parties. In various embodiments of the present invention, one or more smart contracts may be called or triggered to register computation tasks from task initiators, to verify solutions received from edge computing nodes, and to distribute token awards from a reward pool to edge computing nodes once solutions are verified. Depending on its size and/or format, the disclosed systems can verify a solution on-chain through a verifier smart contract, or off-chain through the original task initiator.
The THETA decentralized computing platform's decentralized nature and low barrier to deployment allow peers to use the system for computation tasks of both large and small scales, without the need for specialized project servers for task allocation and management. For example, individual users who do not own professional software on powerful enough personal computers can put up simple image rendering tasks for individual edge workers, developers can tap into the system to delegate computation intensive tasks such as machine learning model training and evaluation to hundreds of edge workers, gaming and video streaming platforms that already have millions of concurrently active users can distribute rendering and transcoding jobs among its own users seamlessly, and large-scale scientific projects such as SETI@home and Folding@home are inherently compatible with the THETA platform.
Furthermore, as edge computing nodes can be automatically awarded and incentivized for their work, participants in the THETA edge computing network may be more likely to provide significant contributions in a reliable manner when compared to existing crowd-sourcing donation platforms that run on volunteer computers, with many more edge computing nodes willing to participate. Additional protection against malicious task initiators can further be provided by the smart contracts via collateral locking and reputation ranking.
In short, the THETA decentralized edge computing platform combines the advantages of P2P distributed computing techniques and blockchain-based smart contracts to achieve high scalability and high resource availability, and attains the following characteristics:
In what follows, a THETA decentralized edge computing platform and infrastructure are disclosed, software architecture of individual nodes within the THETA network are presented, and designs for the THETA blockchain ledger system and smart contracts are also disclosed.
In some decentralized computing projects, individual nodes or users are connected to a central server directly to obtain allocation and assignment of computation tasks. In some cases, individual users in a local area network may function as a worker farm to solve tasks from a central server. By comparison, nodes within a peer-to-peer (P2P) distributed computing network communicate directly with each other, without the need of a central server. One advantage of P2P resource sharing is availability and robustness. Peers can come and go at any time, and tasks can be batched or subdivided depending on resource availability.
In the present disclosure, “edge nodes” and “edge computing nodes” refer to individual peer nodes in the THETA network. As previously discussed, the THETA network may implement edgecast functionalities for data and bandwidth resource sharing, as well as edge compute functionalities for computation resource sharing. Thus, an edge node may be implemented to function as one or more of an edge cacher, a viewer, a task initiator, or an edge worker. An edge cacher node shares its storage and bandwidth resources with other peers; a viewer node downloads data from edge cacher nodes; a “task initiator” or “task server” node uses the THETA network to solve its computational tasks; while an “edge worker” or “edge computing node” is a generic computational platform or generic end user client that shares and contributes its processing power and CPU time to solve computational tasks registered by task initiators in exchange for rewards. In what follows, unless otherwise specified, the terms “edge node”, “edge computing node”, and “edge worker” may be used interchangeably, with the understanding that an edge node can implement one or more protocols to share with others, or receive from others, one or more types of resources. Similarly, unless otherwise specified, the terms “THETA network”, “THETA decentralized network”, “THETA edge computing platform”, “THETA edge computing network”, “THETA edge network”, and “THETA EdgeCast network” are used interchangeably, with the understanding that the THETA network is designed for the universal sharing of different types of resources among its member peer nodes.
In some embodiments, an edge computing node such as 730 in
Once a task is solved by edge computing node 730, the edge computing node may upload the solution directly to task initiator 710, or indirectly through another intermediate node or network, and in turn be rewarded once the solution has been verified. In some embodiments, each edge computing node may have one or more associated availability scores indicating its up time, response time, its “skill level” or hardware specification (e.g., processor number, speed, memory size, bandwidth, memory storage, combinations thereof, and/or the like), and other characteristics such as computation history. Such availability scores may be used for task/job matching, or task allocation and assignment by task initiators.
Task initiators are users or user computers that utilize the THETA edge computing platform to solve their computational tasks. In the exemplary setup shown in
In what follows, both task initiators and edge computing nodes are discussed in exemplary embodiments, for illustrative purpose only, without limiting the scope of the methods, systems, and devices as disclosed herein, which are capable of distributing or performing computation tasks with various latency, accuracy, or precision requirements. “Computation”, “computation task”, “computational task”, or simply “task” refers to any computer-executed processes that use multiple CPU cycles to execute a given set of computation instructions, to convert a set of input data into a set of output data, and/or to compute a “solution” to a proposed or registered problem. Exemplary computation tasks include, but are not limited to, any one or combination of data cleaning, search, comparison, sampling, interpolation, extrapolation, transformation, transcoding, equation solving, statistical analysis, numerical modeling, and machine model training and validation. A solution to a task may take on different alphanumerical formats and sizes depending on the nature of the task, and a solution verification process may check whether the solution meets one or more given conditions or requirements. For example, an integer factorization task may result in a unique deterministic result in the form of numerical tuples that can be multiplied together to verify its correctness. On the other hand, a video transcoding task may have multiple video output files as the solution, each with a different resolution. Each video file may be sampled to verify its resolution meets an initial requirement of the task, while being a non-unique output file at that particular resolution, depending on the transcoding technique used. Some tasks may be batched together, while some may be subdivided. For example, an edge computing node such as 730 in
Moreover, a task initiator or an edge computing node may receive, store, and transmit chunks, fragments, slices, or batches of input data, output data, data hashes, problem stems, and/or task metadata, and may support “range requests” to avoid the need to download or upload a relatively large chunk of data when only a relatively small fraction is needed.
As discussed, although individual nodes are labeled as either a task initiator or an edge computing node in
To facilitate task allocation, assignment, solution generation, verification and rewarding, a THETA blockchain network may provide native protocol level support for reward pools and smart contracts.
While some payment systems may use a central authority to verify and clear transactions to maintain trust, a blockchain ledger can achieve global, decentralized consensus without such a central authority. The THETA blockchain uses a Proof-of-Stake (POS) distributed consensus approach, where a blockchain node may mine or validate a block according to various combinations of random selection, wealth and/or age (i.e., the “stake”). For example, a stake may be a fixed amount of cryptocurrency funds (e.g., THETA tokens) that is committed to the blockchain by a miner in order to participate in block creation and validation. The more stake a miner commits, the more mining power it may have. In various embodiments, other types of block consensus mechanisms such as Proof-of-Work, Proof-of-Engagement may be used instead. Furthermore, smart contracts are immutable computer programs executed and run deterministically on blockchain nodes. Once deployed, a smart contract can be executed but cannot be changed. Each edge node in the THETA decentralized network may access smart contracts deployed on blockchain 810 to participate in distributed computing as disclosed herein.
In various embodiments, each component or node within THETA network 900 may be implemented as different types of computing clients or modules, such as stand-alone edge computing clients, WebApps, SDKs, and the like. For example, edge node 930 may be implemented as a dedicated software module that runs on any suitable device including, but not limited to, mobile computing devices such as tablets or smartphones 932, personal computers or desktops 934, game consoles, and server machines 936. Other examples of suitable computing entities are provided with reference to
As discussed previously, task initiator nodes and edge computing nodes may utilize any peer discovery methods to self-organize into semi-randomly connected networks based on node specifications, bandwidth availability and cost, network distance/geo-distance, and/or other factors. For example, each edge computing node such as 930 and 960 in
In some embodiments, edge computing nodes with specs above a certain threshold may be made available for particular task initiators known to host difficult tasks that require more than a threshold number of parallel processors, a threshold number of CPU cycles, with specific needs on local storage or computation delay, with additional requirements on solution precision, or the like. In some embodiments, some task initiators and/or edge computing nodes may be partitioned into clusters, for example, based on task type, and to share rewards from the same reward pool. In some embodiments, edge computing nodes are not partitioned into clusters, to improve robustness of the network at least because the network may continue to function if any task initiator or edge computing node leaves the network. As more edge nodes are added to the network, the network may become more efficient and may achieve higher computation offload. In some embodiments such as shown in
To facilitate task registration, solution, verification, and rewarding, each of task initiator 910 and edge computing nodes 930 and 960 may have direct access to a THETA blockchain 990 that hosts one or more smart contracts such as 992. A blockchain can include a list of public transaction records, which are data structures that encode the transfer of value from one user to another. Transactions can be structured and bundled into blocks (e.g., through a Merkle tree), and blocks are chained through cryptography, and in turn managed by a blockchain peer-to-peer network with protocols for inter-node communication and block validations, to achieve global, decentralized consensus without a central authority. A smart contract is a decentralized application stored and run on a blockchain. When a transaction has a smart contract address as a destination, the smart contract is executed and a function as specified by the transaction is called. In embodiments of the present invention, one or more smart contracts deployed on the THETA blockchain may be invoked, called, or triggered to register computation tasks from task initiators, to verify solutions received from edge computing nodes, and to distribute token awards from a reward pool to edge computing nodes once solutions are verified. Depending on its size and/or format, a solution may be verified on-chain through a verifier smart contract, or off-chain by the original task initiator. More details on the THETA blockchain are provided in the next section.
In summary, the THETA decentralized edge computing platform includes three main types of participants:
First, a THETA protocol layer 1030 as designed and developed by THETA labs provides the framework for enabling blockchain operations as well as data delivery and resource sharing capabilities. In some embodiments, THETA protocol layer 1030 may include a ledger protocol 1036, a decentralized computation protocol 1032, and a decentralized data delivery protocol 1034. These protocols may work together to implement one or more transaction verification, block assembly and consensus, cryptocurrency token economics, and decentralized applications that may assign, allocate, or share different types of resources via smart contracts enabled by the THETA blockchain. Exemplary embodiments of the decentralized data delivery protocol 1034 are explained in issued U.S. Pat. No. 10,771,524 (Methods and Systems for a Decentralized Data Streaming and Delivery Network, incorporated by reference in its entirety herein).
A blockchain is a decentralized, distributed, public ledger, in the form of a list of public transaction records or transactions that are written into blocks of data and linked through cryptography. A blockchain is typically managed through a peer-to-peer network, where each peer node maintains a full or partial copy of the blockchain. A blockchain relies on decentralized, cryptographic consensus among peers to achieve immutability, where a transaction record cannot be modified once written into a block and the block is verified and accepted by peers. Block mining and validation refer to specific steps in the distributed consensus process that are required whenever a new transaction is added to the blockchain.
More specifically, a transaction is a data structure or signed message that is transmitted through a blockchain network of peer nodes, and recorded on the blockchain. A transaction may encode or record the transfer of value between different parties when used for payment, with one or more transaction inputs and one or more transaction outputs viewed as debits and credits against different entities involved in the transaction. The output of one transaction may be used as the input of another transaction. A user's balance, as recorded on a blockchain, is an aggregation of all transaction outputs controlled by the user through a cryptographic key, and that have not been spent as the input to other transactions yet. Furthermore, a transaction may contain a data payload for smart contract invocation or other similar purposes.
While some payment systems require a central authority to verify and clear transactions to maintain trust, a THETA blockchain ledger system such as implemented using ledger protocol 1036 can achieve global, decentralized consensus without such a central authority. That is, even though fiat currencies are often stored and transmitted digitally, fraud, double-spending, and other issues and disputes are prevented by clearing and settling electronic transfers or payments through centralized authorities such as individual banking institutions or clearing houses. By comparison, a blockchain-based cryptocurrency payment system relies on cryptography and distributed consensus among all peers for trusting the legitimacy of a transaction without dependence on third-parties. A blockchain is immutable, where modifications to transaction data may be nearly impossible, a property making it suitable for use by cryptocurrencies as a payment method in the above-mentioned reward system for decentralized computation.
In some embodiments, the THETA decentralized public ledger system may be built upon the following novel designs with Proof-of-Stake (POS) mining. First, a multi-level Byzantine Fault Tolerant (BFT) consensus mechanism allows thousands of blockchain nodes to participate in a consensus process while still supporting very high transaction throughput, for example, in the range of 1,000+ transactions per second. To minimize transaction confirmation delays, the THETA protocol may use a small set of blockchain nodes to form a validator committee, producing a chain of blocks as fast as possible using a practical BFT (PBFT)-like process. With a sufficient number of validators such as 10 to 20 nodes, the validator committee may produce blocks at a fast speed, while still retaining a high degree of difficulty to prevent an adversary from compromising the integrity of the blockchain. A transaction is “committed” once it is included in a new block. To be eligible to join the validator committee, a node may lock up a certain amount of stake for a period of time. The locked stake could be slashed or removed if malicious behavior is detected. For example, slashed cryptocurrency tokens may be burned or redistributed to other validators proportionally. The blocks that the committee reaches consensus on are called settled blocks, and the process by which they produce a chain of blocks is called the block settlement process.
Next, consensus participants called guardian nodes may validate and finalize the chain generated by the validator committee at checkpoint blocks. The guardian network is a super set of the validator committee, where a validator is also a guardian. With a certain amount of token lockup for a period of time, any node in the network may become a guardian. The guardians may download and examine the chain of blocks generated by the validator committee and try to reach consensus on the checkpoints. “Finalization” refers to convincing each honest or non-malicious guardian that more than a certain portion (e.g., ⅔) of all the other guardians see the same chain of blocks. Blocks that the guardian nodes have reached consensus on are called finalized blocks, and the process by which they finalize the chain of blocks is called the block finalization process. Checkpoint blocks are a selected subset of blocks that satisfy a given set of conditions, for example, whose height is a multiple of some integer. The block height of a particular block refers to the number of confirmed blocks preceding it in the blockchain, starting from the genesis block (i.e., block zero). The “leapfrogging” finalization strategy leverages the immutability characteristic of the blockchain data structure, where as long as at least two guardian nodes agree on the hash of a block, with overwhelming probability, they would have the same copy of the entire blockchain up to that block. The validator/guardian division provides multiple levels of security guarantee. The validator committee provides a first level of consensus, and the guardian pool forms a second line of defense. With thousands of nodes, it is substantially more difficult to compromise the integrity of the network, and thus provides a higher level of security. This consensus mechanism achieves good trade-offs among transaction throughput, consistency, and level of decentralization.
In some embodiments, the THETA blockchain network uses an aggregated signature gossip scheme to significantly reduce messaging complexity. Each guardian node keeps combining partially aggregated signatures from all its neighbors, and then gossips out the aggregated signature. This way the signature share of each node can reach other nodes at an exponential rate. In addition, signature aggregation keeps the size of the node-to-node messages relatively small, and thus further reduces communication overhead.
Furthermore, in some embodiments, edge nodes in the THETA network may be configured into “elite edge nodes” with stakes to avoid certain types of attacks such as Sybil attacks, and the aggregated signature gossip routine may be enhanced to prove the uptime of such elite edge nodes, with rewards to encourage edge node counts and resource capacity in the network. In particular, elite edge nodes may be connected to validator and guardian nodes to form a joint mesh network. Elite edge nodes may broadcast their verification signatures such as Boneh-Lynn-Shacham (BLS) signatures of the latest checkpoint block to guardian nodes to aggregate. Such BLS signature shares of the active elite nodes are aggregated by guardians into one signature, and written into the blockchain through new blocks proposed by validators. That is, the BLS signature share of an elite node proves that the node was up and running when the corresponding block was produced. In some embodiments, for each checkpoint block, a subset of elite edge nodes may be sampled, deterministically or randomly, to receive a “uptime mining” reward. To avoid missing such rewards, an elite edge node needs to stay online, download the latest block header, check if it is in the sampled set, and sign the checkpoint block hash and gossip out the signature if selected. This incentive mechanism maximizes the uptime of edge nodes, and thus improves the availability of the THETA network.
In some embodiments, the THETA ledger system offers off-chain Resource-Orientated Micropayment Pools. A micropayment pool enables small one-to-one, one-to-many and many-to-one payments using off-chain transactions. For decentralized computing, a task initiator can pay for computation by multiple edge nodes, and an edge node can be paid for computation completed for multiple tasks and multiple task initiators, all with only a limited number of on-chain transactions.
In some embodiments, the THETA ledger system provides native support of smart contracts which implements decentralized computer applications stored on a blockchain and can be called and executed when predetermined or preexisting terms and conditions are met, or when an agreement between participating parties is reached (e.g., as confirmed via electronic signatures). More specifically, a smart contract is a computer program written into a blockchain. Within the peer-to-peer blockchain network, each node may be viewed as part of an emulated global computer that runs the smart contracts. Once created and deployed, contract code cannot be modified, and the smart contract becomes immutable. Changes to a smart contract requires the deployment of new instances. A deployed smart contract may be invoked, called, and executed deterministically, for example, upon the submission of a transaction directed to a contract address. That is, when a transaction destination is a contract address, the commitment of the transaction to the blockchain may cause the contract to be run with the transaction's data payload as input. For example, transaction data may indicate which specific function of the smart contract to call with which specific input parameter values. Furthermore, a smart contract may react to a transaction by further calling or invoking another smart contract, or functions within the other smart contract. In various embodiments, smart contracts may be written in high-level programming languages such as Solidity, Serpent, Vyper, Bamboo, and the like.
In some embodiments, specialized incentive smart contracts may be used in the creation and management of micropayment pools or reward pools, especially when a large number of different parties are involved. Rather than requiring complex application logic, an incentive smart contract simplifies the process of collecting and distributing rewards based on certain pre-defined criteria and allocations across different parties, such as task initiators, edge workers, advertisers, content providers, viewers, elite edge nodes, and even miners in the blockchain network. In order for a recipient to receive a payment, a certain proof may be sent to the smart contract. Upon proof validation, the smart contract may initiate the reward, without involving original funders of the incentive smart contract.
In some embodiments, smart contracts may facilitate distributed computing and automatic reward processes. That is, the deployment of smart contracts on the THETA blockchain may allow computing tasks to be registered or “posted”, verified, and further paid for when verified. A smart contract may be triggered to verify a solution when the solution is received from an edge computing node, or be triggered to transfer a token award from a reward pool to an edge computing node if a notification is received from a task initiator indicating that a solution generated by the edge computing node has been successfully verified.
As further shown in
Lastly, an applications layer 1010 in the THETA ecosystem 1000 provides a platform for programming and integrating decentralized applications (DApps) by DApp partners and developers. A Decentralized Application may include one or more smart contracts on a blockchain, and a web frontend user interface. The THETA network enables user interfaces (UIs) and program codes implementing application-level logic consistent with user expectations of the decentralized applications. In addition, a THETA JavaScript mesh networking library may be used to build the decentralized data delivery and computing network, and a THETA SDK may be used for integration of the applications with existing software and/or hardware.
Decentralized Edge Computing with On-chain Solution Verification
In this section and the subsequent one, the design and system architecture of a blockchain and smart contract-powered decentralized edge computing platform are provided.
In some embodiments, the task solution does not need to be kept secret. For such cases, unencrypted or plain-text solutions may be submitted to a blockchain smart contract directly for on-chain verification.
In some embodiments, task initiator 910 may be configured to not reveal solutions to the public. For such cases, edge node 930 may submit encrypted solutions to the blockchain. For a smart contract to verify the correctness of an encrypted solution without decrypting it, zero-knowledge proof techniques like non-interactive zero-knowledge proof may be used. In cryptography, a zero-knowledge proof technique or protocol is a method in which a first party known as the “prover” can prove to a second party known as a “verifier” that the prover possesses knowledge of certain information without revealing the information itself, or any other information apart from the fact that the prover knows the information. A non-interactive zero-knowledge proof is a zero-knowledge proof that requires no interactions between the prover and verifier. That is, after an initial setup phase, a single message may be sent from the prover to the verifier, without additional back-and-forth communication between the two parties. An exemplary non-interactive zero-knowledge proof is zk-SNARK, short for zero-knowledge Succinct Non-Interactive Arguments of Knowledge. “Succinct” refers to zero-knowledge proofs where the message is small in size when compared to the length of the information or solution being verified. With zk-SNARK, the correctness of computation may be verified without having to execute the computations or having to know what was executed. In the exemplary embodiment shown in
An added benefit of zk-SNARK is to reduce the computational cost of solution validation or verification, when the cost of on-chain solution verification using smart contracts is proportional to the number of computational steps of the verification process. The zk-SNARK technique can turn any computation in the class nondeterministic polynomial (NP) time complexity into a verification process with a constant number of steps, which can be conducted on-chain.
Table 3, listed at the end of the present disclosure, provides code snippets for an exemplary smart contract RewardPoolWithOnChainVerification written in the Solidity programming language for on-chain solution verification. In some embodiments, this smart contact may be initially deployed and/or later invoked by task initiator 910, and may have multiple functions or APIs including the following two functions:
In the code example shown in Table 3, if multiple edge computing nodes solve the same task, only the first node that successfully submits the solution to the smart contract may obtain the reward. In some embodiments, a solution computed by this first node may be verified by the verifierContract contract before solutions computed by other nodes are verified. In some embodiments, multiple edge computing nodes may share the reward instead, with more than one solution verified by the verifierContract contract, and portions of an allocated reward transferred to individual nodes that have provided the solutions respectively. For example, reward splitting among multiple edge nodes may be implemented when the designated computation task involves certain randomness, and solutions provided by different edge computing nodes are all correct without being identical. In some embodiments, reward splitting among multiple edge computing nodes may be proportional to or scaled based on some characteristics, quality, or metadata of individual solutions submitted, such as a resolution, size, accuracy, sample variance, and the like.
In some embodiments, the various modules and agents in the TOT framework may be implemented by edge nodes in a blockchain-supported edge computing network, such as those described above.
In some embodiments, a task initiator node may post a task to be solved by the ToT system. The task may be posted to a reward smart contract on a blockchain associated with the edge network. Tasks may include but are not limited to general mathematical and logical reasoning, problem-solving, solution searching, and decision-making tasks, which may require multi-step reasoning to solve. One or more edge nodes in the edge network may host and run the ToT system, including the ToT controller, the prompter agent, the LLM, the checker module, and the memory module. Each of the agents and modules may be run on the same or different sets of one or more edge nodes. The ToT controller, prompter agent, LLM, checker module, and memory module may communicate with each other on the peer-to-peer communication network of the edge computing network. The edge nodes running the ToT system can be automatically awarded for their work when a correct solution is submitted by the ToT system. In some embodiments, the ToT controller submits the final solution to the reward smart contract after the checker module deems the result a valid final solution. In other embodiments, any edge node running any module or agent within the ToT system, may submit the final solution to the reward smart contract. A submitted solution may be deemed correct with the on-chain verification method previously described, or an off-line solution verification by the task initiator node, or other means of solution verification. The rewards may be distributed to the edge nodes that ran the ToT system according to the reward smart contract on which the task was posted, and may be in the form of cryptocurrency tokens.
Additionally, in a similar way, the training of the TOT system, as described in the section “Controller and Prompter Training” may be carried out by one or more edge nodes in the edge computing network, such as the THETA Edge network. The edge nodes may execute training tasks such as policy gradient-based training for the ToT controller and prompter agent policy, as well as other multi-agent reinforcement learning algorithms. The training tasks may be posted to a reward smart contract deployed on a blockchain, and edge nodes that carry out the training task may be incentivized and rewarded by the reward smart contract on the blockchain.
Machine learning (ML) algorithms are characterized by the ability to improve their performance at a task over time without being explicitly programmed with the rules to perform that task (i.e., learn). A machine learning (ML) model is the output generated when a ML algorithm is trained on data. As described herein, embodiments of the present invention use one or more artificial intelligence (AI) and machine learning (ML) algorithms for implementing one or more components of LLM-based TOT solvers, including but not limited to the LLM, the checker module, the memory module, the ToT controller, and the prompter agent, as well as combinations thereof. Various exemplary ML algorithms are within the scope of the present invention. The following description describes illustrative ML techniques for implementing various embodiments of the present invention.
A neural network is a computational model inspired by the structure of the human brain, consisting of interconnected units called neurons that work together to process information. It is a type of ML algorithm that is particularly effective for recognizing patterns and making predictions based on complex data. Neural networks are widely used in various applications such as image and speech recognition and natural language processing, due to their ability to learn from large amounts of data and improve their performance over time.
The neural network 1302 undergoing the training may comprise one or more components of the TOT solver, including but not limited to the LLM, the checker module, the memory module, the ToT controller, and the prompter agent, as well as combinations thereof. The training of a neural network involves repeatedly updating the weights and biases 1310 of the network to minimize the difference between the predicted output 1304 and the true or target output 1306, where the predicted output 1304 is the result produced by the network when a set of inputs from a dataset is passed through it. The true or target output 1306 is the true desired result. The difference between the predicted output and the true output is calculated using a loss function 1308, which quantifies the error made by the network in its predictions. In an exemplary neural network model deployed in the LLM-based ToT solver system, the predicted 1304 and true/target outputs 1306 of the neural network may be an embedding or context vector representing an analysis result, a control signal, an intermediate solution, a prompt, a memory or state of the system, or a decision boolean variable.
The loss function is a part of the cost function 1308, which is a measure of how well the network is performing over the whole dataset. The goal of training is to minimize the cost function 1308. This is achieved by iteratively adjusting the weights and biases 1310 of the network in the direction that leads to the steepest descent in the cost function. The size of these adjustments is determined by the learning rate 1308, a hyperparameter that controls how much the weights and biases change in each iteration. A smaller learning rate means smaller changes and a slower convergence towards the minimum of the cost function, while a larger learning rate means larger changes and a faster convergence, but with the risk of overshooting the minimum.
Neural network training combines the processes of forward propagation and backpropagation. Forward propagation is the process where the input data is passed through the network from the input layer to the output layer. During forward propagation, the weights and biases of the network are used to calculate the output for a given input. Backpropagation, on the other hand, is the process used to update the weights and biases 1310 of the network based on the error (e.g., cost function) 1308 of the output. After forward propagation through neural network 1302, the output 1304 of the network is compared with true output 1306, and the error 1308 is calculated. This error is then propagated back through the network, starting from the output layer and moving towards the input layer. The weights and biases 1310 are adjusted in a way that minimizes this error. This process is repeated for multiple iterations or epochs until the network is able to make accurate predictions.
The neural network training method described above, in which the network is trained on a labeled dataset (e.g., sample pairs of input user prompts and corresponding output recommendations), where the true outputs are known, is called supervised learning. In unsupervised learning, the network is trained on an unlabeled dataset, and the goal is to discover hidden patterns or structures in the data. The network is not provided with the true outputs, and the training is based on the intrinsic properties of the data. Furthermore, reinforcement learning is a type of learning where an agent learns to make decisions from the rewards or punishments it receives based on its actions. Although reinforcement learning does not typically rely on a pre-existing dataset, some forms of reinforcement learning can use a database of past actions, states, and rewards during the learning process. Any neural network training method that uses a labeled dataset is within the scope of the methods and systems described herein, as is clear from the overview below.
The transformer architecture, which is used by LLM models in the TOT system depicted in
The architecture (see
The layers of self-attention in the transformer model allow it to weigh the relevance of different parts of the input sequence when generating an output, thereby enabling it to capture long-range dependencies in the data. On the other hand, the fully connected layers are used for transforming the output of the self-attention layers, adding complexity and depth to the model's learning capability.
The transformer model is known for its ability to handle long sequences of data, making it particularly effective for tasks such as machine translation and text summarization. In the transformer architecture, positional encoding is used to give the model information about the relative positions of the words in the input sequence. Since the model itself does not have any inherent sense of order or sequence, positional encoding is a way to inject some order information into the otherwise order-agnostic attention mechanism.
In the context of neural networks, tokenization refers to the process of converting the input and output spaces, such as natural language text or programming code, into discrete units or “tokens.” This process allows the network to effectively process and understand the data, as it transforms complex structures into manageable, individual elements that the model can learn from and generate.
In the training of neural networks, embeddings serve as a form of distributed word representation that converts discrete categorical variables (i.e., tokens) into a continuous vector space (i.e., embedding vectors). This conversion process captures the semantic properties of tokens, enabling tokens with similar meanings to have similar embeddings. These embeddings provide a dense representation of tokens and their semantic relationships. Embeddings are typically represented as vectors, but may also be represented as matrices or tensors.
The input of a transformer typically requires conversion from an input space (e.g., the natural language token space) to an embedding space. This process, referred to as “encoding,” transforms discrete inputs (tokens) into continuous vector representations (embeddings). This conversion is a prerequisite for the transformer model to process the input data and understand the semantic relationships between tokens (e.g., words). Similarly, the output of a transformer typically requires conversion from the embeddings space to an output space (e.g., natural language tokens, programming code tokens, etc.), in a process referred to as “decoding.” Therefore, the training of a neural network and its evaluation (i.e., its use upon deployment) both occur within the embedding space.
In the remainder of this document, the processes of tokenization, encoding, decoding, and detokenization are assumed. In other words, the processes described below occur in the “embeddings space.” Hence, while the tokenization and encoding of training data and input prompts may not be represented or discussed explicitly, they are implied. Similarly, the decoding and detokenization of neural network outputs are also implied.
The training process begins at step 1410 with the acquisition, retrieval, assimilation, or generation of data related to the machine learning application. At step 1420, acquired data are pre-processed, or prepared. At step 1430, the ML model (e.g., an exemplary component of LLM-based ToT solver) is trained using training data 1425. At step 1440, the ML model is evaluated, validated, and tested, and further refinements to the ML model are fed back into step 1430 for additional training. Once its performance is acceptable, at step 1450, optimal model parameters are selected.
Training data 1425 is a documented data set containing multiple instances of system inputs and correct outcomes. It trains the ML model to optimize the performance for a specific target task. In
In some embodiments, an additional fine-tuning 1460 phase including iterative fine-tuning 1460 and evaluation, validation, and testing 1470 steps, is carried out using fine-tuning data 1455. Fine-tuning in machine learning is a process that involves taking a selected 1450 pre-trained model and further adjusting or “tuning” its parameters to better suit a specific task or fine-tuning dataset 1455. This technique is particularly useful when dealing with deep learning models that have been trained on large, general training datasets 1425 and are intended to be applied to more specialized tasks or smaller datasets. The objective is to leverage the knowledge the model has already acquired during its initial training (often referred to as transfer learning) and refine it so that the model performs better on a more specific task at hand.
The fine-tuning process typically starts with a model that has already been trained on a large benchmark training dataset 1425, such as ImageNet for image recognition tasks. The model's existing weights, which have been learned from the original training, serve as the starting point. During fine-tuning, the model is trained further on a new fine-tuning dataset 1455, which may contain different classes or types of data than the original training set. This additional training phase allows the model to adjust its weights to better capture the characteristics of the new fine-tuning dataset 1455, thereby improving its performance on the specific task it is being fine-tuned for.
In some embodiments, additional test and validation 1480 phases are carried out using test and validation data 1475. Testing and validation of a ML model both refer to the process of evaluating the model's performance on a separate dataset 1475 that was not used during training, to ensure that it generalizes well to new unseen data. Validation of a ML model helps to prevent overfitting by ensuring that the model's performance generalizes beyond the training data.
While the validation phase is considered part of ML model development and may lead to further rounds of fine-tuning, the testing phase is the final evaluation of the model's performance after the model has been trained and validated. The testing phase provides an unbiased assessment of the final model's performance that reflects how well the model is expected to perform on unseen data, and is usually carried out after the model has been finalized to ensure the evaluation is unbiased.
Once the model is trained 1430, selected 1450, and optionally fine-tuned 1460 and validated/tested 1480, it is deployed 1490. Deployed ML models (e.g., components of LLM-based TOT solvers) 1495 usually receive new data 1485 that was pre-processed 1420.
In machine learning, data pre-processing 1420 is tailored to the phase of model development. During model training 1430, pre-processing involves cleaning, normalizing, and transforming raw data into a format suitable for learning patterns. For fine-tuning 1460, pre-processing adapts the data to align with the distribution of the specific targeted task, ensuring the pre-trained model can effectively transfer its knowledge. Validation 1480 pre-processing mirrors that of training to accurately assess model generalization without leakage of information from the training set. Finally, in deployment (1490 and 1495), pre-processing ensures real-world data matches the trained model's expectations, often involving dynamic adjustments to maintain consistency with the training and validation stages.
In various embodiments of the present invention, the data pre-processing step 1420, model training step 1430, model evaluation and testing step 1440, model selection step 1450, model fine-tuning step 1460, model evaluation and testing step 1470, model testing and validation step 1480, and model deployment step 1490 may be completed on the blockchain-supported edge computing network. The steps may be completed as different tasks or as part of a larger multi-step task that is coordinated by a coordinator node such as a task initiator node. In some embodiments, the steps may be assigned to be run on one or more cloud-based networks, one or more edge networks such as the blockchain-supported and incentivized THETA Edge Network, or a combination of cloud and edge networks.
Various exemplary ML algorithms are within the scope of the present invention. Such machine learning algorithms include, but are not limited to, random forest, nearest neighbor, decision trees, support vector machines (SVM), Adaboost, gradient boosting, Bayesian networks, evolutionary algorithms, various neural networks (including deep learning networks (DLN), convolutional neural networks (CNN), and recurrent neural networks (RNN)), etc.
ML modules based on transformers and Large Language Models (LLMs) are particularly well suited for the tasks described herein. The online article “Understanding Large Language Models—A Transformative Reading List,” by S. Raschka (posted Feb. 7, 2023, available at sebastianraschka.com), describes various LLM architectures that are within the scope of the methods and systems described herein, and is hereby incorporated by reference in its entirety herein as if fully set forth herein.
The input to each of the listed ML modules is a feature vector comprising the input data described above for each ML module. The output of the ML module is a feature vector comprising the corresponding output data described above for each ML module.
Prior to deployment, each of the ML modules listed above may be trained on one or more respective sample input datasets and on one or more corresponding sample output datasets. The input and output training datasets may be generated from a database containing a history of input instances and output instances, or may be generated synthetically by subject matter experts.
Implementation using Computer Program Products, Methods, and Computing Entities
An exemplary embodiment of the present disclosure may include one or more end user computing entities 1500, blockchain nodes, or other management computing entities 1600, as shown in
The computing entities 1500 and 1600 can be used to implement the algorithms, methods, and processes described herein. Non-transitory storage media store program code which encodes program instructions that implement the algorithms, methods, and processes described herein. The program code when executed by one or more hardware processors implement said algorithms, methods, and processes.
In general, the terms device, system, computing entity, entity, and/or similar words used herein interchangeably may refer to, for example, one or more computers, computing entities, desktops, mobile phones, tablets, phablets, notebooks, laptops, distributed systems, gaming consoles (e.g., Xbox, PlayStation, Wii), watches, glasses, key fobs, radio frequency identification (RFID) tags, ear pieces, scanners, cameras, wristbands, kiosks, input terminals, servers or server networks, blades, gateways, switches, processing devices, processing entities, set-top boxes, relays, routers, network access points, base stations, the like, and/or any combination of devices or entities adapted to perform the functions, operations, and/or processes described herein. Such functions, operations, and/or processes may include, for example, transmitting, receiving, retrieving, operating on, processing, displaying, storing, determining, creating, generating, monitoring, evaluating, comparing, and/or similar terms used herein interchangeably. In various embodiments, these functions, operations, and/or processes can be performed on data, content, information, and/or similar terms used herein interchangeably. On the other hand, a task initiator/server, a tracker server, or payment server may be implemented according to the exemplary schematic diagram shown in
As shown in
Via these communication standards and protocols, user computing entity 1500 can communicate with various other computing entities using concepts such as Unstructured Supplementary Service Data (USSD), Short Message Service (SMS), Multimedia Messaging Service (MMS), Dual-Tone Multi-Frequency Signaling (DTMF), and/or Subscriber Identity Module Dialer (SIM dialer). User computing entity 1500 can also download changes, add-ons, and updates, for instance, to its firmware, software (e.g., including executable instructions, applications, program modules), and operating systems.
In some implementations, processing unit 1510 may be embodied in several different ways. For example, processing unit 1510 may be embodied as one or more complex programmable logic devices (CPLDs), microprocessors, multi-core processors, coprocessing entities, application-specific instruction-set processors (ASIPs), microcontrollers, and/or controllers. Further, the processing unit may be embodied as one or more other processing devices or circuitry. The term circuitry may refer to an entirely hardware embodiment or a combination of hardware and computer program products. Thus, processing unit 1510 may be embodied as integrated circuits, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), hardware accelerators, other circuitry, and/or the like. As will therefore be understood, processing unit 1510 may be configured for a particular use or configured to execute instructions stored in volatile or non-volatile media or otherwise accessible to the processing unit. As such, whether configured by hardware or computer program products, or by a combination thereof, processing unit 1510 may be capable of performing steps or operations according to embodiments of the present invention when configured accordingly.
In some embodiments, processing unit 1510 may comprise a control unit 1512 and a dedicated arithmetic logic unit 1514 (ALU) to perform arithmetic and logic operations. In some embodiments, user computing entity 1500 may comprise a graphics processing unit 1540 (GPU) for specialized image and video rendering tasks, and/or an artificial intelligence (AI) accelerator 1542, specialized for applications including artificial neural networks, machine vision, and machine learning. In some embodiments, processing unit 1510 may be coupled with GPU 1540 and/or AI accelerator 1542 to distribute and coordinate processing tasks.
In some embodiments, user computing entity 1500 may include a user interface, comprising an input interface 1550 and an output interface 1552, each coupled to processing unit 1510. User input interface 1550 may comprise any of a number of devices or interfaces allowing the user computing entity 1500 to receive data, such as a keypad (hard or soft), a touch display, a mic for voice/speech, and a camera for motion or posture interfaces. User output interface 1552 may comprise any of a number of devices or interfaces allowing user computing entity 1500 to provide content and information to a user, such as through a touch display, or a speaker for audio outputs. In some embodiments, output interface 1552 may connect user computing entity 1500 to an external loudspeaker or projector, for audio or visual output.
User computing entity 1500 may also include volatile and/or non-volatile storage or memory 1530, which can be embedded and/or may be removable. A non-volatile memory may be ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrack memory, and/or the like. The volatile memory may be RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, and/or the like. The volatile and non-volatile storage or memory may store an operating system 1515, application software 1516, data 1518, databases, database instances, database management systems, programs, program modules, SDKs, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like to implement the functions of user computing entity 1500. As indicated, this may include a user application that is resident on the entity or accessible through a browser or other user interface for communicating with a management computing entity and/or various other computing entities.
In some embodiments, user computing entity 1500 may include location determining aspects, devices, modules, functionalities, and/or similar words used herein interchangeably. For example, user computing entity 1500 may include outdoor positioning aspects, such as a location module adapted to acquire, for example, latitude, longitude, altitude, geocode, course, direction, heading, speed, universal time (UTC), date, and/or various other information/data. In one embodiment, the location module may acquire data, sometimes known as ephemeris data, by identifying the number of satellites in view and the relative positions of those satellites. Alternatively, the location information may be determined by triangulating the user computing entity's position in connection with a variety of other systems, including cellular towers, Wi-Fi access points, and/or the like. Similarly, user computing entity 1500 may include indoor positioning aspects, such as a location module adapted to acquire, for example, latitude, longitude, altitude, geocode, course, direction, heading, speed, time, date, and/or various other information/data. Some of the indoor systems may use various position or location technologies including RFID tags, indoor beacons or transmitters, Wi-Fi access points, cellular towers, nearby computing devices (e.g., smartphones, laptops) and/or the like. For instance, such technologies may include the iBeacons, Gimbal proximity beacons, Bluetooth Low Energy (BLE) transmitters, NFC transmitters, and/or the like. These indoor positioning aspects can be used in a variety of settings to determine the location of someone or something to within inches or centimeters. Location information thus obtained may be used in determining nearby peers for data distribution and retrieval.
In some embodiments, two or more users may establish a connection between their computing devices using any of the networking protocols listed previously, and any peer-to-peer protocols including BitTorrent, or that provided by the THETA network. In some embodiments, the user computing devices may use a network interface such as 1522 to communicate with various other computing entities, to exchange data content, information, and/or similar terms used herein interchangeably that can be transmitted, received, operated on, processed, displayed, stored, and/or the like.
In some embodiments, data (e.g., audio, video, etc.) may be downloaded by one or more user computing devices to a server such as shown in
As indicated, in one embodiment, management computing entity 1600 may include one or more network or communications interface 1620 for communicating with various computing entities, such as by communicating data, content, information, and/or similar terms used herein interchangeably that can be transmitted, received, operated on, processed, displayed, stored, and/or the like. For instance, management computing entity 1600 may communicate with user computing device 1500 and/or a variety of other computing entities. Network or communications interface 1620 may utilize a wired data transmission protocol, such as fiber distributed data interface (FDDI), digital subscriber line (DSL), Ethernet, asynchronous transfer mode (ATM), frame relay, data over cable service interface specification (DOCSIS), or any other wired transmission protocol. Similarly, management computing entity 1600 may be configured to communicate via wireless external communication networks using any of a variety of standards and protocols as discussed with reference to user computing device 1500.
As shown in
Although not shown explicitly, management computing entity 1600 may include or be in communication with one or more input elements, such as a keyboard, a mouse, a touch screen/display, a camera for motion and movement input, a mic for audio input, a joystick, and/or the like. Management computing entity 1600 may also include or be in communication with one or more output elements such as speaker, screen/display, and/or the like.
In various embodiments, one or more of the components of management computing entity 1600 may be located remotely from other management computing entity components, such as in a distributed system or in the cloud. Furthermore, one or more of the components may be combined and additional components performing functions described herein may be included in the management computing entity 1600. Thus, the management computing entity 1600 can be adapted to accommodate a variety of needs and circumstances. As will be recognized, these architectures and descriptions are provided for exemplary purposes only and are not limited to the various embodiments.
One of ordinary skill in the art knows that the use cases, structures, schematics, and flow diagrams may be performed in other orders or combinations, but the inventive concept of the present invention remains without departing from the broader scope of the invention. Every embodiment may be unique, and methods/steps may be either shortened or lengthened, overlapped with the other activities, postponed, delayed, and continued after a time gap, such that every end-user device is accommodated by the server to practice the methods of the present invention.
The present invention may be implemented in hardware and/or in software. Many components of the system, for example, signal processing modules or network interfaces etc., have not been shown, so as not to obscure the present invention. However, one of ordinary skills in the art would appreciate that the system necessarily includes these components. A computing device is a hardware that includes at least one processor coupled to a memory. The processor may represent one or more processors (e.g., microprocessors), and the memory may represent random access memory (RAM) devices comprising a main storage of the hardware, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or back-up memories (e.g., programmable or flash memories), read-only memories, etc. In addition, the memory may be considered to include memory storage physically located elsewhere in the hardware, e.g., any cache memory in the processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device.
The hardware of a computing device also typically receives a number of inputs and outputs for communicating information externally. For interface with a user, the hardware may include one or more user input devices (e.g., a keyboard, a mouse, a scanner, a microphone, a camera, etc.) and a display (e.g., a Liquid Crystal Display (LCD) panel). For additional storage, the hardware may also include one or more mass storage devices, e.g., a floppy or other removable disk drive, a hard disk drive, a Direct Access Storage Device (DASD), an optical drive (e.g., a Compact Disk (CD) drive, a Digital Versatile Disk (DVD) drive, etc.) and/or a tape drive, among others. Furthermore, the hardware may include an interface to one or more networks (e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and/or the Internet among others) to permit the communication of streaming content and information with other computers coupled to the networks. It should be appreciated that the hardware typically includes suitable analog and/or digital interfaces to communicate with each other.
In some embodiments of the present invention, the entire system can be implemented and offered to the end-users and operators over the Internet, in a so-called cloud implementation. No local installation of software or hardware would be needed, and the end-users and operators would be allowed access to the systems of the present invention directly over the Internet, using either a web browser or similar software on a client, which client could be a desktop, laptop, mobile device, and so on. This eliminates any need for custom software installation on the client side and increases the flexibility of delivery of the service (software-as-a-service), and increases user satisfaction and ease of use. Various business models, revenue models, and delivery mechanisms for the present invention are envisioned, and are all to be considered within the scope of the present invention.
The hardware operates under the control of an operating system, and executes various computer software applications, components, program code, libraries, objects, modules, etc. to perform the methods, processes, and techniques described above.
In general, the method executed to implement the embodiments of the invention may be implemented as part of an operating system or a specific application, component, program, object, module, or sequence of instructions referred to as “computer program(s)” or “program code(s).” The computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computing device or computer, and that, when read and executed by one or more processors in the computer, cause the computer to perform operations necessary to execute elements involving the various aspects of the invention. Moreover, while the invention has been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments of the invention are capable of being distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of machine or computer-readable media used to actually affect the distribution. Examples of computer-readable media include but are not limited to recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD-ROMS), Digital Versatile Disks, (DVDs), etc.), and digital and analog communication media.
Although specific embodiments of the disclosure have been described, one of ordinary skills in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and/or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skills in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure.
Blocks of the block diagrams and flow diagrams support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flow diagrams, and combinations of blocks in the block diagrams and flow diagrams, may be implemented by special-purpose, hardware-based computer systems that perform the specified functions, elements or steps, or combinations of special-purpose hardware and computer instructions.
A software component may be coded in any of a variety of programming languages. An illustrative programming language may be a lower-level programming language such as an assembly language associated with a particular hardware architecture and/or operating system platform. A software component comprising assembly language instructions may require conversion into executable machine code by an assembler prior to execution by the hardware architecture and/or platform.
A software component may be stored as a file or other data storage construct. Software components of a similar type or functionally related may be stored together such as, for example, in a particular directory, folder, or library. Software components may be static (for example, pre-established or fixed) or dynamic (for example, created or modified at the time of execution).
Software components may invoke or be invoked by other software components through any of a wide variety of mechanisms. Invoked or invoking software components may comprise other custom-developed application software, operating system functionality (for example, device drivers, data storage (for example, file management) routines, other common routines and services, etc.), or third-party software components (for example, middleware, encryption, or other security software, database management software, file transfer or other network communication software, mathematical or statistical software, image processing software, and format translation software).
Software components associated with a particular solution or system may reside and be executed on a single platform or may be distributed across multiple platforms. The multiple platforms may be associated with more than one hardware vendor, underlying chip technology, or operating system. Furthermore, software components associated with a particular solution or system may be initially written in one or more programming languages but may invoke software components written in another programming language.
Computer-executable program instructions may be loaded onto a special-purpose computer or other particular machine, a processor, or other programmable data processing apparatus to produce a particular machine, such that execution of the instructions on the computer, processor, or other programmable data processing apparatus causes one or more functions or operations specified in the flow diagrams to be performed. These computer program instructions may also be stored in a computer-readable storage medium (CRSM) that upon execution may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement one or more functions or operations specified in the flow diagrams. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process.
Although embodiments have been described in language specific to structural features and/or methodological acts, it is to be understood that the disclosure is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as illustrative forms of implementing the embodiments. Conditional language, such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments could include, while other embodiments do not include, certain features, elements, and/or steps. Thus, such conditional language is not generally intended to imply that features, elements, and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements, and/or steps are included or are to be performed in any particular embodiment.
Although the present invention has been described with reference to specific exemplary embodiments, it will be evident that the various modifications and changes can be made to these embodiments without departing from the broader scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than in a restrictive sense. It will also be apparent to the skilled artisan that the embodiments described above are specific examples of a single broader invention which may have greater scope than any of the singular descriptions taught. There may be many alterations made in the descriptions without departing from the scope of the present invention.
If an Application Data Sheet (ADS) has been filed on the filing date of this application, it is incorporated by reference herein. Any applications claimed on the ADS for priority under 35 U.S.C. §§ 119, 120, 121, or 365 (c), and any and all parent, grandparent, great-grandparent, etc. applications of such applications, are also incorporated by reference, including any priority claims made in those applications and any material incorporated by reference, to the extent such subject matter is not inconsistent herewith. This application is also related to U.S. Pat. No. 11,763,332, filed on Apr. 30, 2021, entitled “EDGE COMPUTING PLATFORM SUPPORTED BY SMART CONTRACT ENABLED BLOCKCHAIN NETWORK” (Docket No. THETA-1004), U.S. Pat. No. 11,611,615, filed on Nov. 16, 2022, entitled “DECENTRALIZED EDGE STORAGE NETWORK WITH FLEXIBLE FILE SHARDING” (Docket No. THETA-1007), and U.S. Pat. No. 11,659,015, filed on Oct. 24, 2021, entitled “TRACKER SERVER IN DECENTRALIZED DATA STREAMING AND DELIVERY NETWORK” (Docket No. THETA-1002E), the entire disclosures of which are hereby incorporated by reference in their entireties herein.
| Number | Date | Country | |
|---|---|---|---|
| 63621292 | Jan 2024 | US |
