Infrastructural Programs with Progsbase

Computational Programs, A Recap

Computational programs with progsbase are easy to develop and test. They run on a single CPU or thread and use only memory, no other devices like the screen or files etc. To test a computational program, simply call a function. Given sufficient memory and enough time, the functional will always give the same output for the same input.

Take, for example, the function IsValidJSON from the JSON library version 0.5.7. It takes a string as input and returns a boolean and a string as output. The boolean is set to true if the JSON string is valid, and false otherwise. If invalid, then the message is filled with the reason the JSON string is invalid JSON.

StringReference message = ...;
char [] json = ...;
boolean valid = IsValidJSON(json, message);

Infrastructural Programs

An infrastructural program is a program that interacts with other devices than the single CPU and RAM of computational programs. For example, it interacts with a disk or another CPU. Such programs are notoriously difficult to test reliably: They require the devices, such as the disk or an external service, to respond reliably each time a test is run. This makes the tests brittle, and they tend to gradually fail over time, as the services become unavailable.

In progsbase, infrastructural programs are always developed as computational programs. By default, the devices are mocked. This means that the infrastructural programs can be developed and tested in a completely stable environment. If a test has passed once, it will always pass, no changing external factor will ever impact it. And since progsbase itself is a timeless programming language, the test will basically never again fail.

Lets look at an example: a cache. The following is a cache that will cache the responses of a web service. The function ResponseCacheIteration from the ResponseCache library version 0.1.6 implements this functionality.

Here is the signature of the function. A cache sits between two devices: A client, doing the requst, and a server, serving the response. These two devices are passed as inputs to the function. In addition, the cache has a state.

void ResponseCacheIteration(ProcessingUnitServerStructure client, ProcessingUnitServerStructure adminPort, ProcessingUnitStructure server, ResponseCacheState state, NumberReference idleRef)

The cache works as follows: It gets a request from the client. If this exact request has not been seen before, i.e. is not in the response cache state, then it forwards the request to the server, gets the response back, stores the request-response pair in its state and returns that response to the client.

The second time the same request comes, the cache sees that it has the response stored. Then, it simply returns the response without calling the server.

We can set up a test for this using our mocks. The client has two requests that are the same. We then run the function ResponseCacheIteration two times. Then we check that the server only had to respond once.

ProcessingUnitServerStructure client = ...
ProcessingUnitStructure server = ...

ResponseCacheInitialize(client, adminPort, server, 10, 200, state);

ResponseCacheIteration(client, adminPort, server, state, idleRef);
ResponseCacheIteration(client, adminPort, server, state, idleRef);

AssertEquals(client.responseCount, 2, failures);

AssertEquals(server.requestCount, 1, failures);

Completing Infrastructural Programs

In order to make a complete response cache, we need to replace the mocks. Let us use client and server devices interacting with Unix Domain Sockets: a unix socket server, and a unix socket client.

We simply delete the folder with the ProcessingUnit mock, and copy in the Unix Socket Driver. We then set up the infrastructural program as shown below. It always follows the same structure: First the devices are set up. Here they set up unix domain sockets. Then the state is initialized. Finally there is an infinite loop with the iteration step. After each iteration step, there iteration can ask for an idling time. The idle is aborted if a message arrives on one of the server ports.

public class ResponseCacheMain {
  public static void main(String[] args) {
    ProcessingUnitServerStructure client = CreateSocketServer("CachedWebSever");
    ProcessingUnitStructure server = CreateSocketClient("WebSever");
    ProcessingUnitStructure adminPort = CreateSocketServer("CachedWebSeverAP");
    NumberReference idleRef = CreateNumberReference(0);

    ResponseCacheState state = new ResponseCacheState();
        
    ResponseCacheInitialize(client, adminPort, server, 1000, 1000000, state);

    for(;;){
      ResponseCacheIteration(client, server, state, idleRef);

      if(idleRef.numberValue > 0){
        ProcessingUnitServer
          .GetPUMessageReady()
          .tryAcquire(
            Math.round(idleRef.numberValue * 1000),
            TimeUnit.MILLISECONDS
          );
      }
    }
  }
}

The initialization step initializes a cache consisting of 1000 responses of maximum 10 mega bytes.

The webservice that was previously exposed on the socket named "WebSever" is now exposed as a cached web serivce on the socket "CachedWebSever".

We can inspect the inner workings of the cache by sending a request to the adminitration port called CachedWebSeverAP. Here is an example of a response:

{
  "idles": 1553468,
  "Client Reqs": 8400,
  "Cached Reqs": 5594,
  "Duplicate Hash Req": 880,
  "Over Cache Max Req": 0
}

It says that the cache has idled 1,553,468 times, each being a second. It has gotten 8,400 client request out of which 5,594 were served from the in-memory cache. 880 requests were for a resource with a duplicate has value, which means that the value was not cached after all. There were zero requests for data over the maximum size.