Patent Application
20080301148
Advances in computer technology (e.g., microprocessor speed, memory capacity, data transfer bandwidth, software functionality . . . ) have generally contributed to increased computer application in various industries, such that organization of computer systems has changed dramatically. The concept of a large computer center with a single large computer to which all users bring their work is obsolete. Similarly, Data Base Management Systems (DBMS systems) have long since moved from a centralized mainframe environment to a de-centralized or distributed environment. For example, one or more PC “client” systems can be connected via a network to one or more server-based database systems (Structure Query Language—SQL database server). Known examples of computer networks include local-area networks (LANs) where the computers are geographically close together (e.g., in the same building), and wide-area networks (WANs) where the computers are farther apart and are connected by telephone lines or radio waves.
Often, networks are configured as “client/server” networks, such that computers on the network classify as a “client” or a “server.” Servers are powerful computers or processes dedicated to managing shared resources, such as storage (e.g. disk drives), printers, modems, or the like. Servers are often dedicated, meaning that they perform virtually no other tasks besides their server tasks. For instance, a database server manages database information such as processing database queries from various clients. The client portion of client-server architecture typically includes PCs or workstations, which rely on server(s) to perform operations. Typically, a client runs a “client application” that relies on a server to perform some operations, such as returning particular database information. Over such networks, various request-response protocols transport information according to predetermined set of rules.
One such request-response protocol is the Tabular Data Stream (TDS) protocol, which is a message oriented application level protocol employed for transfer of requests and responses between client and server systems. TDS can include native support for traditional SQL data types, such as character (char), variable-length character (vchar), binary (blob), date-time, time stamp, vendor-specific data types, and the like. In systems employing TDS, it is typical for a client (e.g., user or application program) to establish a long-lived connection with a server (e.g., database.) Upon a successful hand shake and establishment of the connection via a suitable supported transport/session level protocol, a complete message transfers from client to server after which a complete response travels from server to client.
The following presents a simplified summary of the specification in order to provide a basic understanding of some aspects of the specification. This summary is not an extensive overview of the specification. It is intended to neither identify key or critical elements of the specification nor delineate the scope of the specification. Its sole purpose is to present some concepts of the specification in a simplified form as a prelude to the more detailed description that is presented later.
The subject innovation enhances a Tabular Data Stream (TDS) protocol by enabling clients to send tabular formed data as a single parameter to servers (e.g., sending tables inside of a parameter to SQL servers). Such TDS protocol can employ a Table Valued Parameter (TVP) transporting component that allows client servers to transmit entire database tables as a single parameter when invoking a server side procedure(s), for example. As such, values can be passed for functions associated with a stored procedure, and a parameter styled Application Program Interface (API) can be implemented; wherein the server can also send output parameters or return values to the client by employing the TVP type that can encapsulate entire tables of data.
Accordingly, as opposed to merely sending individual values as parameters; a table shaped set of values can be sent as a single parameter from a client to a server. Hence, a client can send data in from of “N” number of columns and “M” number of rows (where M, N are integers) as a single parameter to a server. Likewise, such single parameter can be treated as a single entity on the server side. In a related aspect, the TVP transporting component employs metadata associated with the tabular form that is sent as a single parameter to the server. Such can include a syntax form that designates data types/strings, number of rows, number of columns, and the like
According to a methodology of the subject innovation, metadata associated with defining a table can initially be designated, in a syntax form, such as setting flags to indicate nullable types and the like. Subsequently, the tabular shaped data forms can be sent as a single parameter to a server from a client. Next, the server can treat such single parameter as a single entity. Associated queries can then be executed on the server side.
To the accomplishment of the foregoing and related ends, the innovation, then, comprises the features hereinafter fully described. The following description and the annexed drawings set forth in detail certain illustrative aspects of the invention. However, these aspects are indicative of but a few of the various ways in which the principles of the invention may be employed. Other aspects, advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the drawings.
The various aspects of the subject innovation are now described with reference to the annexed drawings, wherein like numerals refer to like or corresponding elements throughout. It should be understood, however, that the drawings and detailed description relating thereto are not intended to limit the claimed subject matter to the particular form disclosed. Rather, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the claimed subject matter.
The system 100 is associated with a data storage system 110, which can be a complex model based database structure, wherein an item, a sub-item, a property, and a relationship are defined to allow representation of information within a data storage system as instances of complex types. For example, the data storage system 110 can employ a set of basic building blocks for creating and managing rich, persisted objects and links between objects. An item can be defined as the smallest unit of consistency within the data storage system 110, which can be independently secured, serialized, synchronized, copied, backup/restored, and the like. Such item can include an instance of a type, wherein all items in the data storage system 110 can be stored in a single global extent of items. Furthermore, the data storage system 110 can be based upon at least one item and/or a container structure, and can act as a storage platform exposing rich metadata that is buried in files as items. The data storage system 110 can include the database (not shown), to support the above discussed functionality, wherein any suitable characteristics and/or attributes can be implemented. Furthermore, the data storage system 110 can employ a container hierarchical structure, wherein a container is an item that can contain at least one other item. Such containment concept can be implemented via a container ID property inside the associated class, wherein the store can also be a container in form of a physical organizational and manageability unit. In addition, the store represents a root container for a tree of containers within the hierarchical structure. As illustrated in
Typically, the stored procedures 204, 206, 208 (1 thru m, where m is an integer) are programs (or procedures) that are physically stored within the database. Such programs are usually written in a proprietary database language and in response to a user request, run directly by an engine of the database 211.
The stored procedures 204, 206, 208 can have direct access to the data that requires manipulation, and typically need only send results back to the user, thus mitigating the overhead of communicating large amounts of data back and forth. For example, typical uses for stored procedures 204, 206, 208 can include data validation, which is integrated into the database structure (stored procedures used for this purpose are often called triggers), or encapsulating some large or complex processing (such as manipulating a large dataset to produce a summarized result). Stored procedures 204, 206, 208 can also be employed when the database 211 is manipulated from many external programs. The TVP transporting component 220 allows a parameter styled Application Program Interface (API) to be implemented; wherein the server can also send output parameters or return values to the client using the TVP type, which can encapsulate entire tables of data.
The following discussion describes a particular aspect of the TVP transporting component 220 in accordance with an aspect of the subject innovation. For example, considering the following table definition;
For the related TVP format,
In such syntax, DbName, OwningSchema, and TypeName can include up to 128 characters (128 WCHARs is the maximum identifier length), for example. It is to be appreciated that DbName can also be zero-length, and OwningSchema and TypeName can be specified. Such DbName, OwningSchema, and TypeName are optional fields and can contain zero length strings. Caller should typically follow the following, namely; if the TVP is a parameter to a stored procedure or function where parameter metadata is available on the server side, then the client can send all zero length strings for TVP_TYPENAME; and/or; if the TVP is a parameter to an adhoc SQL statement, parameter metadata information is typically not available on a stored procedure or function on the server, and if so the client is responsible to send sufficient type information with the TVP to allow the server to resolve the TVP type from sys.types, for example.
For the exemplary syntax above, typically, one new flag fDefault can be added, and if the fDefault flag is set, the client driver can skip emitting the column's data value when sending the TVP_ROW. Moreover, ColName is typically zero length within a TVP, and a user UserType can be ignored, unless where UserType=x_utypTimeStamp (0×50) and TYPE_INFO TDS type is BIGBINARY. If so, the client is indicating that the column is a TIMESTAMP column, and for such particular TIMESTAMP case, the client can flag the column as default, to avoid rejection of the TVP. It is to be appreciated that TvpColumnMetaData for TVP does not include TableName for TEXT, NTEXT, and IMAGE columns as with normal COLMETADATA token. Moreover, TVP_NULL_TOKEN token can be employed if the entire TVP parameter is a default value (e.g., employing the TVP_NULL_TOKEN if the parameter is not default will typically be rejected by the server).
Flag Usage in TVP
Typically, for an input TVP, if the fDefault flag is set on a column, then the client is required to not emit the corresponding TvpColumnData data for the associated column when sending each TVP_ROW. Such improves operation efficiency (e.g., to mitigate sending dummy or null data values), wherein the server inserts the missing value(s) on the server side when building the TVP table.
Likewise, for an input TVP, the fCaseSen, usUpdateable, and fFixedLenCLRType flags can be ignored. Moreover, fCaseSen can be calculated from collation, and the server ignores such value internally, hence the client driver is not burdened with associated calculations. In addition. usUpdateable is ignored by server on input, and the fFixedLenCLRType flag is not employed by server.
For output TVPs the Flags mirror the values send by a typical ColMetaData token in a resultset.
Typically, the buffer 600 is the unit written or read at one time, and can be modeled as a “packet”, wherein a message can consist of one or more buffers. A buffer can include a buffer header 602, which can be followed by buffer data 604 that contains the message. Moreover, each new message can start in a new buffer.
In general, both the Client and Server attempt to read a buffer full of data, and can pick out the header to observe how much more (or less) data exists in the communication. When the underlying network interface insists on blocking until all bytes specified in the read have been read, the client can read the header and then determine how much more to read. For example, at login time, client can specify a requested “packet” size, which can identify a size employed to break large messages into smaller “packets”. Moreover, messages that pass between client and the server for passing whole tables can typically include one of two types; either a “token stream” or a “tokenless stream”. A token stream consists of one or more “tokens” each followed by some token specific data. A “token” represents a single byte identifier employed to describe data that follows it (e.g. contains token data type, token data length and the like.) Tokenless streams are typically used for simple messages, while messages that can require a more detailed description of the data within it are sent as a token stream.
Exemplary tokens associated with TVP can include the following;
TVP_COLUMN_ORDERING
TVP_COLUMN_ORDERING is typically employed to re-order the columns in a TVP. For example if a TVP is defined as:
The TDS client can send the f2 field last inside the TVP as an optimization (streaming the large value last). Accordingly, the client can send TVP_COLUMN_ORDERING with order 1,3,2 to indicate that inside the TVP_ROW section the column f1 is sent first, f3 is sent second, and f2 is send third.
Hence, the TVP_COLUMN_ORDERING token on the wire for the example above can include:
Duplicate ColNum values can be considered an error condition. The ordinal values of the columns in the actual TVP type are ordered starting with 1 for the first column and adding one for each column from left to right. The client is required to send one ColNum for each column described in the TVP_COLMETADATA (hence Count should typically match number of columns in TVP_COLMETADATA).
The following discussion provides examples associated with a particular TVP in accordance with an aspect of the subject innovation. Example 1 relates to a sample TVP with 3 columns (int, varchar, datetime) and 4 rows. Ordering is by column ordinal 1 ASC and column ordinal 2 DESC., and Unique index on f1. Accordingly, input TVP data can closely resemble the resultset returned by the following sq1 script:
In this example, a default TVP is represented, wherein when the client desires to indicate to the server that the entire TVP parameter value is default.
This example relates to when the client designates an entire column to be sent to server as default. Server can have default defined on column of TVP, and can further populate value on the server side with predefined default. In general, client will not send any dummy null value (as is send with normal single parameter that is sent as default). For example, such feature can be useful for TVP parameter overloading or for populating a TVP column with a server calculated value, similar to getdate( ) or newid( ).
Accordingly, the following is created and inserted with default values:
The AI component 730 can employ any of a variety of suitable AI-based schemes as described supra in connection with facilitating various aspects of the herein described invention. For example, a process for learning explicitly or implicitly how parameters are to be created for sending whole tables can be facilitated via an automatic classification system and process. Classification can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to prognose or infer an action that a user desires to be automatically performed. For example, a support vector machine (SVM) classifier can be employed. Other classification approaches include Bayesian networks, decision trees, and probabilistic classification models providing different patterns of independence can be employed. Classification as used herein also is inclusive of statistical regression that is utilized to develop models of priority.
As will be readily appreciated from the subject specification, the subject innovation can employ classifiers that are explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via observing user behavior, receiving extrinsic information) so that the classifier is used to automatically determine according to a predetermined criteria which answer to return to a question. For example, with respect to SVM's that are well understood, SVM's are configured via a learning or training phase within a classifier constructor and feature selection module. A classifier is a function that maps an input attribute vector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class—that is, f(x)=confidence(class).
Responses to client commands that are returned can be self-describing, and record oriented; (e.g. the data streams can describe names, types and optional descriptions of rows being returned.) On the client side 820 the data can be a login record, or a Structured Query Language (SQL) command being in a language that the server side 850 can accept, a SQL command followed by its associated binary data (e.g. the data for a bulk copy command), or an attention signal. When a connection is desired, the client 820 can send a login data stream to the server. Even though the client 820 can have more than one connection to the server 450, each connection path can be established separately and in the same manner.
Once the server 850 has received the login record from the client 820 it will notify the client that it has either accepted or rejected the connection request. Like wise to send SQL command or batch of SQL commands; then the SQL command (e.g. represented by a Unicode format) can be copied into the data section of a buffer and then sent to the SQL Server side 820. A SQL batch may span more than one buffer. In addition, various Open Data Base Connectivity (ODBC) routines can cause SQL command to be placed into a client message buffer, or can cause the message buffer to be sent to the server.
In addition, for an SQL command with binary data, the insert bulk operation can represent a case of a SQL command (e.g. in a Unicode format) followed by binary data. Initially, an insert bulk command can be sent to the server 850 in the normal way, and once an acknowledgment is received from the server 850, the client 820 can then send formatted binary data to the server 850. Such functionality can be provided by routines included in the ODBC, in accordance with one exemplary aspect of the subject innovation. Moreover, the client 820 can initially send an insert bulk SQL statement, followed by a COLMETADATA token, which describes the raw data, followed by Multiple rows of binary data, to the server 850. For example, the data is not formatted in storage engine row format, but rather the format described by the COLMETADATA token. The stream is the same as if the data was being selected from the server 850 rather than being sent to the server 850.
Moreover, a TVP token stream can be named as TVP_ROW, which can be defined by the TVP_COLMETADATA token from client to server or from server to client. Such can include a token value of 0×01/1, having the following Token Stream Specific Rules;
The word “exemplary” is used herein to mean serving as an example, instance or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Similarly, examples are provided herein solely for purposes of clarity and understanding and are not meant to limit the subject innovation or portion thereof in any manner. It is to be appreciated that a myriad of additional or alternate examples could have been presented, but have been omitted for purposes of brevity.
As used in this application, the terms “component”, “system”, “engine” are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers.
Furthermore, all or portions of the subject innovation can be implemented as a system, method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware or any combination thereof to control a computer to implement the disclosed innovation. For example, computer readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical disks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ), smart cards, and flash memory devices (e.g., card, stick, key drive . . . ). Additionally it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN). Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.
In order to provide a context for the various aspects of the disclosed subject matter,
With reference to
The system bus 918 can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, 11-bit bus, Industrial Standard Architecture (ISA), Micro-Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), and Small Computer Systems Interface (SCSI).
The system memory 916 includes volatile memory 920 and nonvolatile memory 922. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer 912, such as during start-up, is stored in nonvolatile memory 922. By way of illustration, and not limitation, nonvolatile memory 922 can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory 920 includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).
Computer 912 also includes removable/non-removable, volatile/non-volatile computer storage media.
It is to be appreciated that
A user enters commands or information into the computer 912 through input device(s) 936. Input devices 936 include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processing unit 914 through the system bus 918 via interface port(s) 938. Interface port(s) 938 include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB). Output device(s) 940 use some of the same type of ports as input device(s) 936. Thus, for example, a USB port may be used to provide input to computer 912, and to output information from computer 912 to an output device 940. Output adapter 942 is provided to illustrate that there are some output devices 940 like monitors, speakers, and printers, among other output devices 940 that require special adapters. The output adapters 942 include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device 940 and the system bus 918. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 944.
Computer 912 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 944. The remote computer(s) 944 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to computer 912. For purposes of brevity, only a memory storage device 946 is illustrated with remote computer(s) 944. Remote computer(s) 944 is logically connected to computer 912 through a network interface 948 and then physically connected via communication connection 950. Network interface 948 encompasses communication networks such as local-area networks (LAN) and wide-area networks (WAN). LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 802.3, Token Ring/IEEE 802.5 and the like. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).
Communication connection(s) 950 refers to the hardware/software employed to connect the network interface 948 to the bus 918. While communication connection 950 is shown for illustrative clarity inside computer 912, it can also be external to computer 912. The hardware/software necessary for connection to the network interface 948 includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.
What has been described above includes various exemplary aspects. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing these aspects, but one of ordinary skill in the art may recognize that many further combinations and permutations are possible. Accordingly, the aspects described herein are intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims.
Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
This application claims the benefit of U.S. Provisional Application Ser. No. 60/941,570 entitled “METHODS AND APPARATUS RELATING TO SERVER/CLIENT SQL ENVIRONMENTS”, filed on Jun. 1, 2007, the entirety of which is herein incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60941570 | Jun 2007 | US |
