This is a demonstration of programming to the Server Smalltalk (SST) Transport Communications API. The subject for the demonstration is a workload balancer (WLB) for SST servers. The subject use case and example implementation are described broadly, example code is shown to illustrate the use of some available API, and demo code is provided to illustrate the behavior of the example implementation.
Some familiarity with implementing distributed applications using SST is assumed.
The WLB sits in the communications path between an SST client and an SST server. It presents a facade to the SST client, masquerading as an instance of the target server. The WLB is configured to know about a collection of clones to which it forwards requests from clients.
This example implements a kind of least-used load balancing policy with server affinity. If a client request comes from a client which has previously issued a request, then the request is forwarded to the same target clone that served the previous request. If we call this kind of association a "session", then we can say that the target server clone for a client's first request is selected as the clone with the least number of associated sessions.
The server affinity property is desireable for applications which maintain some form of client session state on the server. Alternate dispatching policies can be plugged in by specifying a #dispatchPolicyClass.
In the example realization, a single server process waits for client requests. The server process receives an assembled SstHeaderedByteMessage from its server endpoint, hands this off to an SstWorkerManager, and returns to receive the next inbound client request message.
The WLB server process code
|shouldTerminate| shouldTerminate := false. [shouldTerminate] whileFalse: [|msg| (msg := serverEndpoint receive) isSstError ifTrue: [shouldTerminate := self handleReceiveError: msg] ifFalse: [self workerManager dispatch: [self forward: msg]]]]
The SST API used here is SstLocalEndpoint>>receive. This Smalltalk message send returns the next inbound client request message. Since there is no marshaler in play, request messages are not marshaled into instances of SstRequest; they are simply forwarded on as "opaque" data to the selected target server.
The WLB worker manager manages a pool of worker processes. The forwarding of a specific request message is performed by a selected worker process. The worker process sends the message, receives a reply, and forwards the reply to the origin client.
The worker process forwarding code
|selectedDelegate connection result| selectedDelegate := self dispatchPolicy selectDelegateFor: msg sender. (connection := self clientEndpoint transport connectionManager connectionFor: selectedDelegate) isSstError ifTrue: [^self handleDelegateConnectError: connection]. (result := connection send: msg) isSstError ifTrue: [^self handleDelegateSendError: result]. (result := connection receive) isSstError ifTrue: [^self handleDelegateReceiveError: result]. self clientEndpoint transport connectionManager finishedUsing: connection. (result := self serverEndpoint reply: result to: msg sender) isSstError ifTrue: [^self handleReplyError: result].
The selectedDelegate is an SstRemoteEndpoint. The transport has a connection manager which maintains a pool of TCP socket connections. We ask the connection manager for a connection on which we can send messages to the selected remote endpoint. If an established connection to this endpoint is available in the pool, we will be given this connection. Otherwise, the connection manager will establish and return a new connection to this remote endpoint.
The connection is an instance of SstSocketConnection, which the connection manager has configured to know about its transport. The connection delegates to the transport's assembler for the actual reading and writing of bytes to the underlying socket. The transport's assembler is the only object that knows the on-the-wire message format.
Note that with the design we have in place above, the assembly of client requests is serialized on a single Smalltalk process. This is not going to be a problem, provided our requests are not large and our clients are well-behaved. As a rule, however, these are not good assumptions for a server such as our WLB. In particular, if a single client sends a partial message, we do not want all other client requests to block until the server times out.
So, our simple #receive loop seems inadequate to this use case. Let's consider what would be more appropriate.
Our server design needs to recognize two types of client activity: new message arrival, and new connection request. A new connection request corresponds to activity on our server's listening socket. In this event, we want to accept the connection and possibly perform some application-specific handshaking. In the event of a new message arrival, we want to do message assembly.
What we want, in particular, is that 1) message assembly for one client can occur concurrently with message assembly for another, and that 2) listening for new client connections can occur concurrently with message assembly.
It turns out that in our use of SstLocalEndpoint>>#receive we are indirectly making use of an SstTcpSocketSieve to detect socket activity, and it is the socket sieve that actually makes the SCI select call. We get to this select call through the send of SstTcpSocketSieve>>#next.
SstTcpSocketSieve maintains a mutex to guard against concurrent attempts to invoke this SCI select. The point of this discussion is that SstLocalEndpoint>>#receive is thread-safe. Further, it is perfectly legitimate to perform concurrent receives in multiple Smalltalk processes, and in doing so we get precisely the desired concurrency in request handling that is outlined above.
The way we will apply this feature to our WLB problem is by providing our WLB server with a configurable number of server processes. This number should probably be some multiple of the number of clones running in the delegate server group. I would probably begin with one-for-one (or three-for-three, in the example here).
The standard SST transport classes for TCP are SstTcpTransport and SstTcpLightTransport. The former is generally used for servers, and the latter is useful for simple clients. The WLB presented here makes use of SstTcpTransport for its server endpoint.
SST applications typically make use of the InvocationHandler framework for remote message sends and for message dispatch. Using this framework, an SST server application would generally make use of its server endpoint for sending requests to other servers as well as for receiving requests from its own clients. In this scheme, marshaled messages are distinguishable as requests and replies, and replies are associated with requests by internal keys.
Since our WLB server is (deliberately) working only with unmarshaled transport messages, it has no ready means of associating a reply with its source request. We could have the server process wait for a reply from the selected delegate server, but we opted to have the server process hand off the request delegation to another Smalltalk process in order to avoid the potential for a server process getting hung due to an unresponsive delegate server.
As a result, it's is not appropriate to make use of a single transport instance for both our server processes and our forwarding dispatchers, since their #receive calls would be in conflict. So we need an additional endpoint for messaging with our delegates.
We are strictly a client in our use of this second endpoint, so we don't want all the server baggage that comes with SstTcpTransport. The SstTcpLightTransport, however, provides for just a single connection to a single remote endpoint. What we really want for this use case is a kind of client transport for TCP that provides a pool of connections for each of several remote endpoints. The WLB implementation here provides a customized SST transport extension - SstTcpMultiClientTransport - which gives us just this kind of behavior.
This custom transport is registered with SST in the usual fashion for registering transports.
| mcConfig reachables | mcConfig := SstTcpCommunications lightTransportConfiguration transportIdentifier: 'mctcp'; transportClass: SstTcpMultiClientTransport; connectionManagerClass: SstPooledConnectionManager; yourself. reachables := Set with: mcConfig transportIdentifier with: SstTcpCommunications lightTransportConfiguration transportIdentifier with: SstTcpCommunications serverTransportConfiguration transportIdentifier. SstTransport register: mcConfig mutuallyReachableBy: reachables
With this transport configuration in place, the client endpoint is created as follows.
(clientEndpoint := SstLocalEndpoint fromUrl: 'sst:[sst:mctcp]//:0') isSstError ifTrue: [clientEndpoint raise]. (startUpResult := clientEndpoint startUp) isSstError ifTrue: [startUpResult raise].
In this example the assembler class is the system default for the 'tcp' transport - SstBasicAssembler. This assembler knows the message format used by SST for transmission of serialized SstRequests.
The WLB knows very little about its clients and servers. Since its knowledge of the TCP message format is encapsulated in the configured assembler, it could easily be modified for alternate use cases.
This approach could be used, for example, to implement a WLB facade for Smalltalk server clones with C clients using a custom message format directly over TCP. And of course the API used by the forwarder could be applied in the absence of WLB - say in the integration of a Smalltalk application in the role of a client with a legacy TCP server.
The following code illustrates how to set the assembler class for the transport configuration registered as the 'tcp' transport configuration. (Details on custom transport configurations are left for another discussion.)
(SstTransport configurationRegistry at: 'tcp') assemblerClass: AcmeXMLMessageAssembler
The WLB demo consists of a sample client, the WLB, and a set of server clones. In this example realization the client, WLB, and servers are each run in distinct images on different machines.
The sample application scenario is standard SST Smalltalk-Smalltalk by-value over TCP. Messages are sent from a client by direct use of SstInvocationHandler API. The demo server makes its class compiler available to clients as a "do-it" server. Client requests, then, consist of snippets of code for the server to evaluate.
Here is the demo code used to configure and start a WLB.
|config| config := SstWlbConfiguration new numberOfServerProcesses: 2; delegateMachines: #('viper:8771' 'cobra:8771'); serverUrl: 'tcp://:8773'; dispatchPolicyClass: SstWlbServerAffinityPolicy; yourself. ^(SstWorkloadBalancer using: config) startUp
Here is some code to start an SST invocation handler to represent the "do-it" server.
| url localEndpoint invocationHandler | url := 'pingPongValue:/tcp/:8771'. localEndpoint := SstLocalEndpoint fromUrl: url sstAsUrl. invocationHandler := SstInvocationHandler on: localEndpoint. invocationHandler startUp. invocationHandler space export: self compiler as: #DoItServer. ^invocationHandler
The "do-it" client handler is started in just the same way, with the exception that the #export:as: message send is unnecessary.
Note that in the local endpoint URL we specify 'pingPongValue' as the invocation scheme to be used. These invocation handlers are not part of the WLB implementation. They are used only for demonstrating and testing the WLB behavior. The configuration registered to support the SST Ping Pong By-value example application can be considered a canonical by-value invocation configuration, and is a convenient invocation scheme to use for this demonstration.
With client and server invocation handlers started in this manner, and with the WLB running on the host "shagbark", messages can be sent from a client to a server as shown below.
|url| url := 'pingPongValue:/tcp/shagbark:8773' sstAsUrl. ^invocationHandler invoke: (DirectedMessage receiver: #DoItServer selector: #evaluate: arguments: #('DateAndTime now')) at: url sstAsRemoteEndpoint].
The sample WLB provides for a limited degree of execution logging. A sample of output to the Transcript is shown below.
174726378|(3/12/2002 4:57:02 PM)|INFO|Server listening at (tcp://shagbark.oti.raleigh.ibm.com:8773) 174744204|Delegate Worker 22611|TRACE|Begin Forward from: 9.25.62.136:8771 to cobra:8771 174744224|Delegate Worker 22611|TRACE|Forward Complete from: 9.25.62.136:8771 to cobra:8771 174856107|(3/12/2002 4:57:02 PM)|INFO|Server stopped - SstLocalEndpoint(invalid,tcp://:8773) 174856107|Server Process #1|ERROR|SstTransportShutDownError in Server Process #1
These few sample messages note the following.
This article has described how SST transport byte message receipt and assembly can be used independently of marshaling and method dispatch, and has demonstrated this capability in the implementation of a workload balancer for SST servers.