Requests from client to server

• cmd_load: load object from server
• cmd_forget: client remove reference to loaded object
• cmd_throw: client throw away instance of object from cache
• cmd_getclass: get information about class from server
• cmd_putclass: register new class at server
• cmd_modclass: modify existed class at server
• cmd_lock: request to server from client to lock object
• cmd_unlock: request to server from client to unlock object
• cmd_transaction: client sends transaction to coordinator
• cmd_subtransact: client sends local part of transaction to server
• cmd_login: client login at server
• cmd_logout: client finish the session
• cmd_alloc: client request server to allocate object
• cmd_free: client request to free object

Replies from server to client

• cmd_object: server send object to client
• cmd_invalidate: notification about object modification
• cmd_classdesc: server send class descriptor requested by cpid
• cmd_classid: class identifier returned to client by server
• cmd_lockresult: result of applying lock request at server
• cmd_transresult: coordinator returns transaction status to client
• cmd_bye: notification about server shutdown
• cmd_ok: successful authorization
• cmd_refused: authorization procedure is failed at server
• cmd_location: address of allocated object

Common issues

All data is passed in network format (big endian) as unaligned stream of bytes (as in Java). Object reference occupies 6 bytes, first 4 of them specify object identifier, which is unique for the storage, and last two bytes specify storage identifier. Class identifier is two bytes long and is also unique for the storage.

Objects are passed as unaligned sequence of fields. All reference fields should be placed before all non-reference fields (such order increase performance of many server algorithms, such as garbage collector, ...). Only object fields described in the object class descriptor should be passed to the server.

Most of servers messages are synchronous and are sent as replies to client requests. There are only two notification messages: cmd_invalidate and cmd_bye, which can be sent asynchronously by the server, so client should be ready to receive them in any moment (but this notification message can't interrupt another message, so all message data is arrived sequentially). Client should not sent other messages until it receives reply to previous message (if it requires reply). So multithreaded client application should use some synchronization discipline to provide consistent access to the storage server by different threads.

Requests from client to server

cmd_load: load object from server

Request to load object or cluster of objects from the server. Server will reply with cmd_object message containing instance of requested object (and my be some other object from the same object cluster).

load objects flags

flag	meaning
1	Client request another copy of object. By setting of this flag client can load most recent version of object while still keeping old version. If this flag is not set, server consider that client replace old version of the object with new one.
2	Allow sending cluster of objects together with requested object. Strategy of including objects in the cluster depends on server. This flag is normally set, unless object is reloaded.

cmd_forget: client remove reference to loaded object

Client informs server that there s no more reference to specified object from client application. Server will remove association between this object instance and client process. Client can transfer more than one notification requests in one send operation. If field 5 of the request is not zero, server will read specified extra number of requests. This request should be only of cmd_forget or cmd_trow type. Value of filed 5 of all this extra requests is ignored.

cmd_throw: client throw away instance of object from cache

Client informs server that object was thrown away from client cache as a result of LRU cache management discipline or because it was modified by some other process. Server can remove old version of object from the storage only if all clients throw deteriorated instance of this object from the cache. Client can transfer more than one notification requests in one send operation. If field 5 of the request is not zero, server will read specified extra number of requests. This request should be only of cmd_forget or cmd_trow type. Value of filed 5 of all this extra requests is ignored.

cmd_getclass: get information about class from server

Load class descriptor from the storage by specified class identifier. Server will reply with cmd_classdesc message, containing class descriptor with specified identifier.

cmd_putclass: register new class at server

Request to look for descriptor of the class in the storage. If there is no class with such descriptor in the storage, then new class identifier is assigned to this class and it's descriptor is stored in the storage. Server will reply with cmd_classid message, containing identifier assigned to this class. Body of class descriptor is sent immediately after this request header. Format of class descriptor header is the following:

Class descriptor header is followed by specified number of field descriptors. Format of class descriptor is described below:

byte 0	byte 1	byte 2	byte 3
Size of fixed part of the class
Size of varying part of the class
Number of references in fixed part of the class
Number of references in varying part of the class
Number of all fields in the class including subclasses
Size of class name dictionary - total size of all class fields names

byte 0	byte 1	byte 2	byte 3
type of the field		offset to field name
Size of the field
Number of components
Index of the next field in the class

Field "number of components" should contain:

• size of array, if component is array with fixed size;
• 1, if component is not an array;
• 0, If component if varying array.

Field types

value	meaning
0	structure
1	object reference
2	signed integer
3	unsigned integer
4	real number

cmd_modclass: modify existed class at server

Modify descriptor of existing class in the storage. This request can be used by administration utilities to rename class fields. It should not be used by normal application, because GOODS automatically handles scheme evaluations using fields names to establish mapping between different version of classes. This request header is followed by class descriptor.

cmd_lock: request to server from client to lock object

Lock object in specified mode. Possible values of lock type is 1 for shared lock (prevent object from been modified or exclusively locked by other processes) or 2 for exclusive lock (prevent object from been modified or locked by other processes). The only supported lock attribute is "no wait" attribute with value 1. If "no wait" attribute is set and granting requested lock is not possible, negative reply will be returned immediately, otherwise request will be blocked until lock will be set. Server will return client about the result of request by means of cmd_lockresult message.

cmd_unlock: request to server from client to unlock object

Unlock previously locked object. In this request lock type specify mode to which lock should be downgrade. To release lock completely pass 0 as lock type.

cmd_transaction: client sends transaction to coordinator

Send local transaction to the server or part of global transaction to coordinator. Protocol of local and global transaction fixing protocol are described in the section Transaction protocol, Transaction is passed to the server as a sequence of object headers. If object was modified by transaction then object header is followed by modified object instance. Modified objects should be placed before non-modified objects. It is necessary to pass headers of non-modified objects to the server to validate these object state and unlock locked objects. Format of object header is specified in the following table:

byte 0	byte 1	byte 2	byte 3
transaction flags
object identifier
object class identifier		object storage identifier
object size

Transaction flags in object header specify action performed by server for this object while committing transaction. This field is bit combination of the following flags:

Value	Meaning
1	update the object (object header should be followed by modified object instance)
2	validate the object (check if client has the most recent version of the object)
4	unlock the object (remove all locks granted by the client)
8	downgrade exclusive lock of the object to shared lock

cmd_subtransact: client sends local part of transaction to server

Send part of global transaction to the server, which is not coordinator of this transaction. Details of global transaction fixing protocol are described in the section Transaction protocol, Format of transaction body, passed after this request header, is the same as in cmd_transaction request.

cmd_login: client login at server

Open client session at the server. Login name of the client is passed immediately after this request header. Server will perform authorization and reply with cmd_ok or cmd_refused messages.

cmd_logout: client finish the session

Request from the client to terminate client session at the server. Server will reply with cmd_bye response message.

cmd_alloc: client request server to allocate object

Allocate object of specified class and size at the server. Server will return identifier of created object by means of cmd_location message.

cmd_free: client request to free object

Request to explicitly deallocate object. As far as GOODS is using garbage collector to free unreferenced objects, this request should be used with case only when client is sure that there are no references to deleted object in the storage (and in other clients). In current GOODS implementation this request can be send during abort of transaction for objects created by this transaction.

Replies from server to client

cmd_object: server send object to client

Server send this message as a reply to cmd_load request. This message header is followed by one or more objects (if sending cluster of objects was allowed in the request). Field 4 specifies total size of all objects in this message. Each object is represented by objects header followed by the object body. Object header has the same format as described in cmd_transaction request:

byte 0	byte 1	byte 2	byte 3
-
object identifier
object class identifier		object storage identifier
object size

cmd_invalidate: notification about object modification

This message is send asynchronously to the clients by the server. When some object is changed by some transaction, server send notification about object modification to all clients having instances of this object (except client initiating this transaction). Several notifications can be included in the same message. In this case field 5 specifies number of extra notifications, following the message header. These notifications have the same format as message header, except that filed 5 of should be ignored.

cmd_classdesc: server send class descriptor requested by cpid

This message is a reply to cmd_getclass client request. Class descriptor is passed in the same format as described in cmd_put_class reply.

cmd_classid: class identifier returned to client by server

This message is a reply to cmd_putclass client request. Server returns to the client identifier assigned to the class, passed in cmd_putclass request.

cmd_lockresult: result of applying lock request at server

This message is response to cmd_lock request. Field 1 is set to 1 (TRUE) if lock is successfully granted, otherwise it is set to 0 (FALSE).

cmd_transresult: coordinator returns transaction status to client

This message is send by coordinator of transaction or by server of local transaction to the client with status of transaction fixing. For global transaction, coordinator first assign global identifier to the transaction and send cmd_transresult message with this identifier to the client. Client then sends local parts of global transaction to all other servers, involved in transaction by means of cmd_subtransact message. Then servers perform global commit of transaction using two-phase protocol and than coordinator inform client about status of transaction fixing. If transaction is successfully committed, status field of cmd_transresult message is set to 1, if transaction is aborted - it is set to 0. In case of local transaction, only one cmd_transresult message is sent to the client, containing status of fixing transaction at the server.

cmd_bye: notification about server shutdown

This message can be send to the client either asynchronously, when server shutdown is initiated by database administrator, or synchronously - as the reply to cmd_logout request.

cmd_ok: successful authorization

Positive reply to cmd_login client request. It is send when client successfully pass authorization check. Only in this case database session is really created for the client.

cmd_refused: authorization procedure is failed at server

Negative reply to cmd_login client request. It is send when authorization procedure is failed for this client. No database session is opened for the client in this case.

cmd_location: address of allocated object

This message is reply to cmd_alloc request. Identifier of object created in this storage is returned back to the client.

Transaction protocol

In case of global transaction, first of all coordinator of this transaction should be chosen. Current agreement in GOODS is to use server with the smallest identifier among all servers involved in transaction as coordinator of this transaction. Such agreement is significant for recovery procedure which should restore servers in special order. Then client sends part of global transaction to coordinator (objects located at coordinator storage should be included in this part). Format of passing transaction is specified in cmd_transaction. Here I only want to notice, that modified object should be placed before headers of non-modified objects (such order allows server not to reorder records in transaction buffer). Coordinator will reply with cmd_transresult message, containing global identifier of transaction assigned to the transaction by coordinator. Also status field should be checked in this message. By some reasons transaction can be aborted by the coordinator at this stage. If status is TRUE, client should send local parts of transaction to all other servers, including in cmd_subtransact message global identifier of transaction assigned by coordinator. After it, servers will perform two-phase transaction fixing protocol and result will be returned by coordinator to the client by cmd_transresult message.

In case of local transaction, only one cmd_transaction request should be sent to the server. Server will reply with cmd_transresult message, informing client whether transaction is successfully committed or aborted.

Two ensure transaction serializability, locks set within transaction, should be kept until the end of transaction. It is possible to automatically unlock objects by passing special flag in header of object in transaction body. If transaction is aborted, locks are not released and client application should unlock objects itself, by cmd_unlock requests.